An MTP head makes our local Ornith model 39 percent faster

We run Ornith 1.0 35B, a mixture-of-experts model, on one secondhand Tesla P40. It generated approximately 56 tokens per second.

We asked Claude Fable 5 to improve the speed. It identified a possible way to add multi-token prediction (MTP) to the model.

Why the models were compatible

Ornith uses the Qwen 3.6 architecture. The models have identical tensor dimensions and a vocabulary of 248,320 tokens.

Ornith has no published MTP head. Qwen3.6-35B-A3B-MTP has one. The matching dimensions made it possible to add that head to Ornith.

What multi-token prediction does

A standard model generates one token at a time. An MTP head proposes several tokens in advance.

The main model verifies those proposals. It accepts the correct proposals and rejects the others. Accepted proposals can increase generation speed.

How we changed the file

Fable added 20 MTP tensors from Qwen to the quantized Ornith weights. These tensors include an extra transformer layer, the nextn projection, and normalization tensors.

The new data became block 41. Fable changed the metadata, increased the block count from 40 to 41, and set nextn_predict_layers to 1.

Test results

We used llama.cpp build b9859 with a draft depth of two.

Task	Speed	Increase
Code generation	77.9 tokens per second	39 percent
Prose generation	71.2 tokens per second	27 percent

The implementation uses output-exact speculative decoding. Ornith verifies every proposed token. Rejected proposals do not enter the output, so the output stays identical to the base model.

Hardware and setup

The Q4_K_M file is 21.7 GB. It runs fully on a 24 GB GPU, such as a P40, 3090, or 4090.

The setup uses a q8_0 KV cache for a practical 128k context. We do not know of another MTP build of this model that fits a standard consumer GPU.

The model is available on Hugging Face as Ornith-1.0-35B-MTP-Q4_K_M-GGUF. The Ornith base uses the MIT license. The added tensors keep their Apache 2.0 license.

Use this command to start the server:

llama-server -m ornith-1.0-35b-MTP-graft-Q4_K_M.gguf \
  -ngl 99 -c 131072 -ub 1024 \
  --cache-type-k q8_0 --cache-type-v q8_0 \
  -fa on --spec-type draft-mtp --spec-draft-n-max 2 \
  --jinja

The model also loads in LM Studio, vLLM, Ollama, and Docker. If you try a similar change on another model, tell us about the result.