I Fine-Tuned a 0.5B Model to Write Shell Commands

Think of a Swiss Army knife and a potato peeler. The knife does twenty things. The peeler does one thing, and it does that one thing better than the knife ever will.

The AI model I use most is a potato peeler. It has 0.5 billion parameters, which in 2026 makes it very small indeed. It does exactly one job: you type find every file over 100MB I changed this week, and it hands you back the shell command that does it. It lives on my laptop. It answers before I finish reading my own question.

For that one job, it beats the enormous models behind the big APIs. That sounds like a claim about how clever small models have become. It is not. It is a claim about how small I made the job.

What a "fine-tune" actually is

Quick vocabulary, because the rest of this needs it.

A base model is an AI that has already read an enormous amount of text and learned general patterns. Somebody else spent a fortune training it. Fine-tuning means taking that finished model and nudging it toward one specific task by showing it a few hundred or few thousand examples.

You are not teaching it language. It already knows language. You are teaching it a habit.

Mine started from qwen2.5-coder-0.5b-instruct, a small model already good at code. I showed it pairs: an English request, and the shell command that answers it. The result is on Hugging Face as shellm-qwen-0.5b-nl2sh-finetuned, free for anyone to use.

The whole training run finished in about the time it takes to make lunch. Here is essentially all the code:

from unsloth import FastLanguageModel
from trl import SFTTrainer

model, tokenizer = FastLanguageModel.from_pretrained(
    "unsloth/qwen2.5-coder-0.5b-instruct-bnb-4bit",
    load_in_4bit=True,
)
model = FastLanguageModel.get_peft_model(model, r=16, lora_alpha=16)

SFTTrainer(model=model, tokenizer=tokenizer, train_dataset=pairs).train()

Four lines that matter. Five years ago this was a research project. That it is now this easy is the genuinely remarkable part, and it is not the part I want to talk about.

The hard part was choosing the job

My first attempt was a model for "shell commands."

That is not a job. That is a whole category, and I had to learn the difference the slow way. Under that heading I threw in git commands, text-mangling one-liners, Docker flags, service-management incantations, file searches — everything.

The model got worse at all of them at once.

Here is why, and it is worth sitting with. It learned what a shell command looks like without learning what any particular command means. It produced things with the right rhythm and shape — a word, some dashes, some letters — that did not run. Confident nonsense. That is what you get when the target is too blurry to aim at.

The version that works answers a much narrower question: describe some files, get back a find, grep, ls, du, tar or rsync command. That is one family of tools and maybe forty options between them. Small enough that the model can actually learn which options go together and which ones contradict each other.

Every improvement I got came from making the job smaller. Not once from making the model bigger. I never even tried a larger model, because by the time the job was small enough to learn, the model had stopped being the limitation.

What a small model is really buying

Let me be straight about this, because the case gets made badly all the time.

A frontier model writes better shell than mine. It handles stranger requests. It notices when your question is ambiguous and asks. If the contest is "who is more accurate on anything you might type," I lose, and it is not close.

What I get instead has nothing to do with being smarter.

It is instant, because nothing leaves the machine. No request travelling to a data centre and back. The first word appears faster than that round trip could ever finish, and that changes how you use a tool. You start reaching for it mid-thought, instead of pausing to decide whether it is worth the wait.

It costs nothing to run. Not "cheap." Nothing. I can call it in a loop, on every keystroke, ten thousand times while building a dataset, and there is no bill at the end.

Nothing about my work gets shared. File names and folder structures, taken together, describe what you are working on in a fair amount of detail. I would rather not hand that to a stranger in exchange for help remembering a flag.

It cannot change under me. The model is a file on my disk. Nobody can deprecate it, reprice it, or quietly swap it for a different one on a Tuesday.

Three of those four are not really AI properties at all. They are plumbing properties — where the thing sits, what it touches, who controls it. That is the actual case for running something small and local: not that it is better, but that it can be somewhere the big one cannot go.

The part I got wrong

I shipped it without any way to tell whether it was working.

What I had was a training set and a feeling. I would type a request, look at the answer, decide it seemed about right, and move on. That works fine — right up until you change something. More examples, a different setting, one more round of training. Then you want to know: did that help?

And I could not answer. I could tell you the model felt better after the second run. I could not tell you whether it was.

What I should have built first, before a single training step, is a hundred test questions with known-correct answers and a script that runs them and counts. That is one unglamorous afternoon. It would have made every afternoon after it mean something. Instead I spent the project's entire supply of enthusiasm on the fun part.

If you are about to try one

Fine-tuning a small model is cheap enough now that money is not the interesting variable any more. Your examples are.

A few hundred examples of a genuinely narrow job will beat a few thousand examples of a vague one, and you will find out which you have within an hour of using it.

Pick the job by asking where a big remote model simply cannot go. Inside a tight loop. Inside a private folder. On a device with no signal. Inside a delay budget measured in a few dozen milliseconds. Those are the places where a small model that is merely good enough beats a huge model that is excellent — because the huge model is not there.

And build the test set first. I did not. The only honest thing I can say about my own model is that I enjoy using it.