674: Parameter-Efficient Fine-Tuning of LLMs using LoRA (Low-Rank Adaptation)

<p>Models like Alpaca, Vicuña, GPT4All-J and Dolly 2.0 have relatively small model architectures, but they&apos;re prohibitively expensive to train even on a small amount of your own data. The standard model-training protocol can also lead to catastrophic forgetting. In this week&apos;s episode, Jon explores a solution to these problems, introducing listeners to Parameter-Efficient Fine-Tuning (PEFT) and the leading approach: Low-Rank Adaptation (LoRA).<br/><br/>Additional materials:<a href='https://www.superdatascience.com/672'> </a><a href='https://www.superdatascience.com/674'>www.superdatascience.com/674</a><br/><br/>Interested in sponsoring a SuperDataScience Podcast episode? Visit <a href='https://www.jonkrohn.com/podcast'>JonKrohn.com/podcast</a> for sponsorship information.</p>