Microsoft paper reveals efficient skill distillation for GPT-5.4-mini

Microsoft researchers have found a way to make a cheaper AI model outperform its own expensive reasoning mode, and the trick costs about as much as a latte.

A paper published on August 11, titled “Reason Wide, Not Deep: Amortizing the Reasoning Premium into Distilled Skills,” introduces a passive skill-distillation method that extracts compact rule sets from a small number of task examples. Those rules, written in plain markdown, get injected into a model’s system prompt and effectively teach it shortcuts that bypass the need for costly chain-of-thought reasoning. The target model in this case: GPT-5.4-mini, which launched back in March.

How it works

The process is deceptively simple. A coding agent reviews between 35 and 50 task trajectories, essentially recordings of how a model approached and sometimes failed at specific tasks. From those trajectories, the agent distills a natural-language “skill” document ranging from 40 to 130 lines of markdown. These aren’t abstract embeddings or fine-tuned weight adjustments. They’re readable, auditable rules derived from what went wrong and what went right.

Once that skill document is folded into the system prompt of a non-reasoning model, something interesting happens. The model recovers between 55% and over 100% of the performance gap that normally separates reasoning and non-reasoning modes. In plainer terms, the cheap mode starts performing like the expensive mode, and sometimes better.

The compute cost for generating each domain’s skill document runs between $1 and $3. No model retraining required.

The benchmarks tell the story

The research team, led by Agamdeep Singh, Srishti Gautam, Priyanshu Gupta, Nikita Mehrotra, Tanmay Bakshi, and Sumit Gulwani, evaluated the approach across four agentic benchmarks including ALFWorld and SpreadsheetBench-Verified.

On ALFWorld, the skill-enhanced GPT-5.4-mini achieved a score of 0.787. The full reasoning mode scored 0.713. So the cheaper, faster version didn’t just close the gap. It blew past it.

Across benchmarks, the skill-augmented model used 2.7 to 6 times fewer output tokens than the reasoning mode.

Building on SkillOpt

This paper builds directly on Microsoft’s SkillOpt framework, released in June, which reframed agent skills as trainable parameters rather than static model weights. SkillOpt demonstrated a 23.5-point average gain for GPT-5.5 across six benchmarks, establishing the theoretical foundation that skills could be optimized independently of the underlying model.

The new distillation method takes that concept and makes it radically more accessible. Because skill documents are just markdown files injected via system prompts, they work with existing rollout strategies. No custom inference infrastructure. No specialized hardware. No retraining runs required.

Disclosure: This article was edited by Editorial Team. For more information on how we create and review content, see our Editorial Policy.