LoRA.md (338B)
1 # Low-Rank Adaptation (LoRA) 2 3 **Source:** https://arxiv.org/abs/2106.09685 4 5 **Definition:** Low-Rank adaptation for LLMs is a fine tuning approach that freezes the pretrained model weights and adds additional trainable rank decomposition matrices to each layer of the Transformer, effectively reducing the number of trainable parameters.