Take It And Run

Small Language Models

Three small models, each tuned for one posture rather than for everything at once.

A small model that does one job well beats a large one that does every job adequately — and it does it on a laptop, offline, for free. These three are split by what they are pointed at: your documents, your tools, or you. Pick the posture, not the parameter count.

Document-first

In Training

Reads the documents you hand it and answers out of them. Tuned to name the chunk it used, and to say the answer is not in the source rather than fill the gap from memory.

Model: thl-doc-3b
Base: Qwen 2.5 3B Instruct
Context: 32k tokens
Footprint: ~2.1 GB at Q4_K_M
Pairs with: Extract, Chunk, Embed, Index

Tools-first

In Training

Turns a plain-English request into a tool call the runtime can validate. Arguments come from the tool spec, and a tool it has no schema for is refused rather than invented.

Model: thl-tools-3b
Base: Llama 3.2 3B Instruct
Context: 16k tokens
Footprint: ~1.9 GB at Q4_K_M
Output: Strict JSON, schema-checked

Personal-first

Queued

The smallest of the three, for notes, mail and everything else you would not paste into a hosted chat box. Sized to fit a machine with no discrete GPU — a personal model that needs a server is not personal.

Model: thl-personal-1b
Base: Llama 3.2 1B Instruct
Context: 8k tokens
Footprint: ~0.8 GB at Q4_K_M
Runs on: 8 GB RAM, no GPU

None of these are downloadable yet. They are the three postures currently on the bench. Each card publishes weights, an evaluation card and a licence the moment that model clears its benchmark — and if it does not clear it, the card comes down rather than shipping something that only looks finished.

How you'll run them

Each one ships as a quantised GGUF for local runtimes and as full weights for anything that wants them. There is no hosted endpoint to call and no key to obtain — the model runs on your machine or it does not run.

Ollama — pull it and talk to it

ollama pull thehallucinatedlab/thl-doc-3b
ollama run thl-doc-3b "summarise the attached contract, cite the clause"

Python — load the weights directly

from transformers import AutoModelForCausalLM, AutoTokenizer

name = "thehallucinatedlab/thl-tools-3b"
tok = AutoTokenizer.from_pretrained(name)
model = AutoModelForCausalLM.from_pretrained(name, device_map="auto")

# the tools-first model answers with a call, not with prose
prompt = tok.apply_chat_template(
    [{"role": "user", "content": "convert this png to webp at quality 80"}],
    tokenize=False, add_generation_prompt=True)