LoRA

Plain English. Low-rank adaptation: instead of retraining a whole model to specialise it, you train two small add-on matrices per layer — an adapter — and leave the original weights untouched. Adapters are cheap to train, tiny to store, and swappable, so one base model can wear many specialisations.

Why it moves money. LoRA made fine-tuning a commodity. It is why open-weight ecosystems can spawn six-figure derivative counts — Qwen alone has 151,448 derivative repositories on Hugging Face — and why "we fine-tuned it on our data" is now table stakes rather than a moat. The open valuation question is whether fine-tuned specialisation holds value at all: the Bitter Lesson argument says each general-model generation erodes the specialist's edge.

What to watch. Whether fine-tuned niche models keep beating the next general release on their home turf — and for how many months at a time.

From the signals. Qwen is now the base layer: 151,448 derivatives on Hugging Face, with fine-tuning-ready builds a top publisher. Mistral Forge and the Bitter Lesson question over enterprise fine-tuning moats.

Further reading. Hu et al., "LoRA: Low-Rank Adaptation of Large Language Models" (2021).