Getting Started with Axolotl

This guide will walk you through your first model fine-tuning project with Axolotl.

1 Quick Example

Let’s start by fine-tuning a small language model using LoRA. This example uses a 1B parameter model to ensure it runs on most GPUs. Assuming axolotl is installed (if not, see our Installation Guide)

  1. Download example configs:
axolotl fetch examples
  1. Run the training:
axolotl train examples/llama-3/lora-1b.yml

That’s it! Let’s understand what just happened.

2 Understanding the Process

2.1 The Configuration File

The YAML configuration file controls everything about your training. Here’s what (part of) our example config looks like:

base_model: NousResearch/Llama-3.2-1B
# hub_model_id: username/custom_model_name

datasets:
  - path: teknium/GPT4-LLM-Cleaned
    type: alpaca
dataset_prepared_path: last_run_prepared
val_set_size: 0.1
output_dir: ./outputs/lora-out

adapter: lora
lora_model_dir:

See our Config options for more details.

2.2 Training

When you run axolotl train, Axolotl:

  1. Downloads the base model
  2. (If specified) applies LoRA adapter layers
  3. Loads and processes the dataset
  4. Runs the training loop
  5. Saves the trained model and / or LoRA weights

3 Your First Custom Training

Let’s modify the example for your own data:

  1. Create a new config file my_training.yml:
base_model: NousResearch/Nous-Hermes-llama-1b-v1
adapter: lora

# Training settings
micro_batch_size: 2
num_epochs: 3
learning_rate: 0.0003

# Your dataset
datasets:
  - path: my_data.jsonl        # Your local data file
    type: alpaca               # Or other format

This specific config is for LoRA fine-tuning a model with instruction tuning data using the alpaca dataset format, which has the following format:

{
    "instruction": "Write a description of alpacas.",
    "input": "",
    "output": "Alpacas are domesticated South American camelids..."
}

Please see our Dataset Formats for more dataset formats and how to format them.

  1. Prepare your JSONL data in the specified format (in this case, the expected `alpaca format):
{"instruction": "Classify this text", "input": "I love this!", "output": "positive"}
{"instruction": "Classify this text", "input": "Not good at all", "output": "negative"}

Please consult the supported Dataset Formats for more details.

  1. Run the training:
axolotl train my_training.yml

4 Common Tasks

4.1 Testing Your Model

After training, test your model:

axolotl inference my_training.yml --lora-model-dir="./outputs/lora-out"

4.2 Preprocessing Data

For large datasets, preprocess first:

axolotl preprocess my_training.yml

4.3 Using a UI

Launch a Gradio interface:

axolotl inference my_training.yml --lora-model-dir="./outputs/lora-out" --gradio

5 Next Steps

Now that you have the basics, you might want to:

  • Try different model architectures
  • Experiment with hyperparameters
  • Use more advanced training methods
  • Scale up to larger models

Check our other guides for details on these topics: