Fine-tuning (and SFT)

Plain English. Taking a trained model and training it further on your own examples so it behaves your way — supervised fine-tuning (SFT) when the examples are curated input-output pairs. Techniques like LoRA make it cheap by adjusting a small fraction of the weights. Distinct from prompting (no weight changes) and from retrieval (giving the model documents at question time).

Why it moves money. Fine-tuning is the load-bearing claim of the "specialised models" thesis: that enterprises will pay for models adapted to their domain rather than renting the general frontier. The countervailing force is brutal — each frontier release absorbs capabilities that fine-tuned models were sold to provide, so a fine-tuning vendor is in a race against its own substrate's improvement. The viable niches are where data is private, behaviour is regulated, or inference must be cheap and local.

What to watch. Whether fine-tuned small models hold measurable advantages over each new general frontier model on the customer's own tasks — the comparison that decides if this is a product category or a feature.

From the signals. Mistral Forge: the Bitter Lesson comes for enterprise fine-tuning.

Further reading. Hu et al., LoRA: Low-Rank Adaptation (2021)