How Fine-tuning Works

How Fine-tuning Works

Understanding fine-tuning mechanics helps you make better decisions about when to use it, what can go wrong, and why alternatives like LoRA exist.

The Core Mechanic

Every modern language model starts as a pre-trained base model — a neural network trained on enormous volumes of text. Fine-tuning continues that training on a much smaller, task-specific dataset. You're not training from scratch. You're nudging the existing weights in a direction that makes the model better at your specific task through the same gradient descent process, just with fewer steps and a much smaller learning rate.

Training Data Format

The standard format for supervised fine-tuning is JSONL — one JSON object per line:

json {"messages": [ {"role": "system", "content": "You are a legal document assistant."}, {"role": "user", "content": "Summarize this clause."}, {"role": "assistant", "content": "The clause limits liability to direct damages..."} ]}

All major providers use a variant of this format. Quality matters far more than quantity — 100 carefully curated examples outperform 1,000 noisy ones.

Full Fine-tuning vs. PEFT

Full fine-tuning updates all model parameters. For a 70B model, this requires gradient storage demanding hundreds of gigabytes of GPU memory — roughly equivalent to training from scratch. Impractical for most teams.

Parameter-Efficient Fine-Tuning (PEFT) updates only a small subset of parameters or adds lightweight trainable modules while keeping the base model frozen. The most widely used approach is LoRA. PEFT can reduce trainable parameters by 10–100x while achieving comparable task-specific performance.

Catastrophic Forgetting

When you train aggressively on a narrow dataset, the model's weights shift enough that it loses general capabilities. A customer support model might excel at refund requests but produce broken responses to anything outside that domain.

Mitigations: very small learning rate, mixing in general-purpose examples with task-specific data, preferring PEFT methods that leave base weights untouched.

Learning Rate Sensitivity

Typical fine-tuning learning rates fall between 1e-5 and 1e-4: - Too high: rapid catastrophic forgetting within a few hundred steps - Too low: updates so small the model barely changes

Start conservatively and evaluate frequently.

Overfitting

Happens when the model memorizes training examples rather than learning the generalizable pattern. Signs: training loss decreasing while validation loss stops improving or increases.

Countermeasures: early stopping when validation loss plateaus, keeping a held-out validation set, using more diverse training examples rather than just more of the same type.