Quality over quantity in fine-tuning data

It is tempting to treat training data as a volume problem: more examples, more coverage, better model. For supervised fine-tuning in particular, the evidence points the other way. A small set of excellent examples often beats a large set of average ones.

The LIMA result

In 2023, Zhou and colleagues fine-tuned a 65B parameter LLaMa model on only 1,000 carefully curated prompts and responses, with no reinforcement learning or human preference modelling ^[1]. In a human study, its responses were either equivalent or strictly preferred to GPT-4 in 43% of cases ^[1].

The authors' interpretation, sometimes called the superficial alignment hypothesis, is that most knowledge is learned in pre-training and fine-tuning mainly teaches format and style. Whether or not that holds in every domain, the practical point stands: a thousand examples written with care can move a model a long way.

Duplicates are not free

Large datasets hide repetition. Lee and colleagues found a single 61-word English sentence repeated more than 60,000 times in the C4 dataset ^[2]. Training on deduplicated data meant models emitted memorised text about ten times less often ^[2]. Repetition wastes compute, skews what the model treats as typical, and raises the chance of verbatim regurgitation.

Textbook-quality data

Gunasekar and colleagues pushed the quality argument further with a code model trained largely on carefully constructed, "textbook quality" data, showing strong results from a model far smaller than its competitors ^[3]. The design choice that mattered was not the amount of data but how instructive each example was.

A practical checklist for SFT data

When we write or audit supervised fine-tuning sets, each example has to pass these checks:

1. Correct. A second expert can verify the answer, and code runs.
2. Complete. It answers the whole prompt, including the part that is easy to skip.
3. Instructive. Reasoning is shown where the task needs it, not padded where it does not.
4. Consistent. Style, format and refusals follow one written guideline.
5. Unique. Near-duplicates are removed with exact and fuzzy matching.
6. Traceable. Every example records its author, reviewer and revision history.

What to measure instead of row count

The takeaway

The cheapest way to improve a fine-tuned model is often to delete a third of its training data and rewrite another third. That work needs people who know the domain well enough to tell a good answer from a plausible one.

See how our Train practice handles SFT and chain-of-thought data, or read about preference data for RLHF and DPO.