Finetuning Roadmap: What to Tune and Why
A practical map of the finetuning stack: SFT, instruction tuning, preference alignment, and evaluation strategy.
Why finetune at all?
Base models are broad, but products are specific. Finetuning narrows behavior toward your domain, tone, constraints, and task format.
Use finetuning when you need:
- Higher reliability on repeated tasks.
- Consistent structure in outputs.
- Better performance on domain language (legal, medical, support, codebase-specific).
The modern finetuning stack
Most teams do not jump straight to RL. They move through a staged pipeline:
- Supervised Fine-Tuning (SFT): teach desired behavior from examples.
- Preference Alignment (DPO or similar): improve quality between good vs better responses.
- Safety + regression evaluation: verify gains without breaking core capabilities.
Base Model -> SFT -> Preference Alignment -> Safety + Regression Checks -> Deploy
When to choose LoRA vs full finetuning
- LoRA: lower GPU memory, faster experiments, easier iteration.
- Full finetuning: expensive but can unlock bigger shifts for large data and large budgets.
Rule of thumb: start with LoRA, prove value, then decide if full finetuning is worth cost.
Success criteria before training
Define measurable goals first:
- Task-level metric: accuracy, pass rate, rubric score.
- Format metric: JSON validity, schema adherence.
- Safety metric: refusal quality and policy compliance.
- Latency and cost budget per request.
Without this, teams often "improve" one benchmark while harming production quality.