Full fine-tuning of a large model means updating billions of weights — expensive in compute and memory, and you get a whole new copy of the model for each task. LoRA changed that math.
The core trick
LoRA (Low-Rank Adaptation) freezes the original model and trains small "adapter" matrices injected alongside the existing weights. These adapters have a tiny fraction of the parameters, yet they can steer the model toward a new task effectively. You train maybe 1% of the weights and get most of the benefit of full fine-tuning.
Don't rewrite the whole model. Attach a small, learnable patch — and keep the original intact.
Why it's a big deal
Three wins. Cost: training a fraction of the parameters slashes memory and compute. Storage: an adapter is small (megabytes), so you can keep dozens of task-specific adapters for one base model instead of dozens of full copies. Swappability: load different adapters at inference to switch tasks on the fly, all sharing the same frozen base.
Where it fits
LoRA and its variants (QLoRA, which adds quantization to fine-tune on even smaller hardware) democratized fine-tuning — you can now adapt a large open model on a single GPU. For most teams that want a specialized model without a training-cluster budget, parameter-efficient fine-tuning is the default. It's a big part of why the open-model ecosystem is so vibrant: anyone can specialize a strong base cheaply.
