---
title: "Transformers, GGUF, and MLX"
description: "Use the same Neutrino release through the library path, the Fermion llama.cpp fork, or Apple GPU code."
canonical: "https://www.fermionresearch.com/docs/integrations/"
source: "Fermion Research"
---

Docs / Guides

# Transformers, GGUF, and MLX

Use the same Neutrino release through the library path, the Fermion llama.cpp fork, or Apple GPU code.

## Load from a local directory in Transformers

```text
import fermion  # registers the trtc_v4 model type
from transformers import AutoModelForCausalLM, AutoTokenizer

path = "/data/models/Neutrino-8B"
tokenizer = AutoTokenizer.from_pretrained(path)
model = AutoModelForCausalLM.from_pretrained(path)
```

## Run the GGUF pack

The 8B repository contains a 4.09 GB GGUF file for the [Fermion llama.cpp fork](https://github.com/fermionresearch/llama.cpp). Its FV5 data types are not available in upstream llama.cpp.

```text
git clone -b fermion-fv5 https://github.com/fermionresearch/llama.cpp
cd llama.cpp
cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release -j

./build/bin/llama-completion \
  -m /path/to/neutrino-8b-fv5.gguf \
  -ngl 99 -c 4096 -p "Explain why the sky is blue." -n 256 --temp 0 -no-cnv
```

## Run on Apple GPU with MLX

Each model repository includes an `mlx/` package that runs the same container on Apple silicon. It is a separate integration and is not installed by `pip install fermion-research`.

```text
git lfs clone https://huggingface.co/FermionResearch/Neutrino-8B
cd Neutrino-8B/mlx
pip install -r requirements.txt
```

Source: [https://www.fermionresearch.com/docs/integrations/](https://www.fermionresearch.com/docs/integrations/)