The kit, for the record, was a Lenovo LOQ; a gaming laptop you'd buy a teenager for Christmas. Sixteen gigs of RAM, an 8-gigabyte consumer graphics card, a Windows install, and a coding agent with a single overnight brief: take a small language model and make it better at predicting text.
Over four nights it ran 217 training experiments on that laptop's GPU. It edited its own code, ran in five-minute training cycles, parsed the results, and either kept the improvement or threw it away. Then again. And again. 217 clean starts out of the blocks, no false ones (well as we'll see, a few!).
The kit on which all this happened
Lenovo LOQ 15IRX10 ; a gaming laptop, around £1,000 retail. Intel i7-13650HX, 14 cores. 16 GB RAM. NVIDIA RTX 5060 with 8 GB VRAM. Windows 11, PyTorch + CUDA. The model is an 11.5-million-parameter transformer; small enough that, by today's standards. Five-minute wall-clock budget per experiment, no exceptions.
By the end of night four, the model's bits-per-byte score ; how surprised the model is by the next character of a held-out test text, lower being better ; had dropped from 1.34 to 1.26. A 6% improvement in a field where 1% is meaningful. The four nights of compute cost roughly two flat whites in electricity.
Then I looked at where the improvement actually came from.
Three changes did 95% of the work
Three clean winners down the line. Turning up the Adam optimiser's second momentum parameter, that's the bit that controls how quickly the optimiser forgets recent noise; from 0.95 to 0.99. Pushing the device batch size from 8 examples per step to 32. Adding two transformer layers, from depth 4 to depth 6. The remaining 200+ experiments tuned learning rates and produced essentially nothing but long rallies from the baseline that never quite found a winning angle. The agent's two-standard-deviations gate kept refusing to commit them, and the gate was right to.
The agent was rigorous. The kind of player with immaculate footwork and a textbook forehand, happily hitting the same shot fifty times in a row. It logged every decision in plain-text markdown so I could audit it over breakfast (or after a client meeting). It refused to commit improvements smaller than the day's measured noise floor; it knew when not to fool itself. What it couldn't do was step back and notice that the rally itself wasn't going anywhere. It just kept the ball in play.
The lesson
Coding agents have become quite capable at the execution layer. They sweep configurations, validate tests and refactor at scale. They handle rigour brilliantly. What they aren't good at is noticing when the question itself has run dry. When changing tactic will help us! Well, that's the coach's job, not the sprinter's.
For practitioners, that flips the time allocation usefully. The day's worth of running changes and sweeping settings is now an overnight task. The twenty minutes you used to spend deciding what to study next, designing the next genuine experiment, calling out a flawed assumption — that's now where most of your value lives.
Five things to try this fortnight
Pick a small, well-scoped optimisation problem you've been avoiding. Anything with a clear metric and a defined search space; query tuning, prompt evaluation, model variant selection, A/B comparisons. Tell the agent to log this in markdown.
Run overnight, not while you watch. Watching slows the loop (you can try this!).
Build a noise gate into anything it commits. Two standard deviations of three quick reruns of the baseline. Without it, you log noise as wins. basically what you allow to make it through.
Audit the log first thing in the morning. Read the last three "wins" and ask: real, or drift? Five minutes catches most silliness early.
Reserve the morning hour for the bit it can't do. Look at the data, notice what wasn't tried, choose the next genuinely different question to direct the model tuning in.
Scaling it up — and especially scaling it locally
The same loop runs essentially unchanged on much larger models; 1-billion, 7-billion, even 70-billion-parameter open models; provided you've quantised the weights (compressed them from 16-bit to 4-bit or 8-bit integers) so they fit within the GPU memory. And the interesting bit is that local inference at those scales is now genuinely viable! An Apple M4 Ultra Mac Studio runs a quantised 70-billion-parameter model at conversational speed. A pair of consumer RTX 4090 cards handles a quantised mid-size model fine, a standard M5 Max MacBook fine-tunes a 70-billion-parameter model on a folder of departmental documents overnight.
The pitch versus a hosted API, in one sentence: data sovereignty, predictable latency, compliance with on-machine rules, customisation by overnight fine-tune, no per-token charges, and proper air-gapped operation. That stack of benefits matters more in NHS, defence, and regulated public-sector work, more than ever. The word 'sovereign' has never been so popular.
What this isn't
A small fine-tuned model is not a frontier reasoning system. Ask it for a paracetamol dose for a 9-month-old and it will hand you four decimal places of confident, which is both impractical and non-sensical. The use cases worth its time are narrow, stylised, language-pattern tasks (suited to specific domains, for sure). For instance, these can include note completion, document classification, journaling assistance, autocomplete that matches a clinic's house style.
Sidebar: your first overnight run
Three commands, run in the folder of any small-model training repo (and Karpathy's autoresearch is the cleanest starting point, with the repo at the end):
git status train.py— confirm cleanuv run train.py— smoke test, ~5 minutesuv run save_best.py— save the winning weights
In between the two, the overnight prompt: the agent edits ONE constant, re-runs, parses val_bpb, then keeps it or runs git restore — looped.
That's the whole loop.
What I added on top
The four nights described above ran on a fork I built around Karpathy's loop. If you want the repo, here it is: autoresearch experiments.
In the box:
runner.py (the autonomous orchestrator with phased queue, 2σ noise gate, 3σ auto re-probes, and adaptive Phase 5+ generation)
register_runner_task.ps1 + launch.bat (Windows scheduled-task integration with its own wall-clock stop time)
save_best.py (checkpoint export without modifying train.py).
model_portable.py is device-agnostic model class and so will run on most machines.
PRs, forks, and serve returns all welcome.
The headline finding
…from those four nights wasn't that the model got 6% better. The gap between running a respectable AI experiment and not being able to just collapsed. A gaming laptop. A markdown log. A prompt. Two granola bars in electricity. Four nights. One personal best. Game, set, and home in time for breakfast.
Where would you point an overnight pipeline in your work?
Built on top of Andrej Karpathy's autoresearch repo, this is a minimal nanochat-style training loop designed for autonomous-agent experimentation.