What parameter-efficient fine-tuning method adapts a large model by training small low-rank update matrices while keeping the original pretrained weights frozen?

LoRA (Low-Rank Adaptation)

LoRA injects trainable low-rank matrices into a frozen model's layers, cutting the number of tuned parameters and memory by orders of magnitude while nearly matching full fine-tuning. Because the base weights are untouched, many lightweight LoRA adapters can be swapped in for different tasks.