Adrian Wälchli

20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.

Implementation of the LLaMA language model based on nanoGPT. Supports flash attention, Int8 and GPTQ 4bit quantization, LoRA and LLaMA-Adapter fine-tuning, pre-training. Apache 2.0-licensed.

Official implementation for HyenaDNA, a long-range genomic foundation model built with Hyena