Blog  /  2026-11-02

\"Could not finish the message because max_tokens\": reasoning models and tiny token budgets

Published 2026-11-02. Written for nlqueries-core 0.4.

llm reasoning-models troubleshooting

"Could not finish the message because max_tokens or model output limit was reached" is the HTTP 400 an OpenAI or Azure reasoning model returns when it spends its whole output budget before writing a word. Other providers say the same thing more quietly: an empty reply with finish_reason=length. Both mean the token budget that was right for older models is too small for a model that thinks first. This post is about how that showed up in NLQueries, what core 0.4 does about it, and the one setting that controls it.

Why a reasoning model returns nothing

A reasoning model writes private reasoning before its answer, and that reasoning is billed from the same output allowance as the answer. It spends it first. So a budget sized for the answer alone gets you a cut-off answer, or, when it is small enough, nothing.

Two examples from NLQueries' own code comments show the scale. On deepseek-v4-pro, a question classification asked for 200 tokens and used 56, of which 52 were reasoning. At a budget of 5 tokens, all five went to reasoning and the content came back empty with finish_reason=length.

That empty reply is the dangerous form, because an empty string looks exactly like a model that had nothing to say. Core 0.4 does not pass it on as one. When a call produces nothing and the finish reason says the budget ran out, the client raises OutputBudgetExhausted, which names the model and the budget, and says to raise the limit. A truncated answer is still an answer and is returned; only nothing is an error. Two calls log it as a warning and carry on instead: follow-up resolution, and writing knowledge-base column descriptions.

The 400 that is the same event

OpenAI and Azure reasoning models do not send the empty reply. They answer with HTTP 400 and the message in this post's title. It is the same event, a budget spent on reasoning, reported as an error.

Where this bit hardest was the health check. nlqueries health makes one small, real call to the configured model to prove the provider answers. It used to ask for 5 output tokens. On a reasoning model that is all reasoning, so the probe got the 400 and reported the provider as down, when it had just answered.

max_output_tokens has a minimum of 16

There was a second problem with 5 tokens. OpenAI's Responses API, which LiteLLM uses for models such as o3-pro, gpt-5-pro and the codex models, refuses max_output_tokens below 16, with "integer below minimum value". So the old probe failed on every one of those models before reasoning even came into it. LiteLLM's own health check asks for 16 for the same reason.

Since core 0.4 the probe asks for 16 (PROBE_MAX_TOKENS), and reads both forms of a spent budget, the empty reply and the 400, as what they are: a round trip that reached the provider, authenticated and came back. health reports the provider as OK and says so:

[OK]   LLM provider   litellm — openai/... responds (… ms) — spent the 16-token probe
       budget before writing anything, which is normal for a reasoning model;
       if answers are coming back empty, raise LLM_MAX_OUTPUT_TOKENS

That last clause is the point. health is the command you reach for when answers come back empty, so it names the setting that fixes them. The probe's own budget is fixed and has nothing to do with it.

One budget, three uses

The probe is the only fixed budget. Everything else derives from one setting, LLM_MAX_OUTPUT_TOKENS:

The calls used to pass their own numbers: 1024, 512 and 200. Those were chosen for models that write only the answer. The classification tier is the one that suffers most, because it is the smallest, and a reasoning model there returns not a short label but an empty string.

The default is 4096 because NLQueries' defaults are Claude Sonnet 5.5 and Haiku 5.5, which think when a question warrants it. The comment beside the setting records why 1024 was not enough: on the hardest question of a ten-question comparison, Sonnet 5.5 thought, then ran out part-way through its SQL, and its correction ran out of 512 the same way. It is a ceiling, not a charge. Tokens are billed as generated, so a model that stops early costs no more than it did.

The floors have a consequence. Raise the budget modestly, from 1024 to 1536 say, and the answer and correction tiers move, but classification stays at exactly 200, because an eighth of 1536 is still under the floor. The classification tier only grows once the total passes 1600. If short calls are the ones failing, raise past that.

How hard the model thinks

The budget is how much it may spend. Some providers also let you say how hard it should think:

• Claude 5 and later, through the Anthropic API: LLM_EFFORT, sent as output_config.effort. The default is low.
• DeepSeek: LLM_EFFORT is sent as reasoning_effort, and LLM_THINKING=off turns thinking off entirely.
• OpenAI: no effort is sent in 0.4. The model uses its own default, so the budget is the setting that matters.

What to do

1. Run nlqueries health. If it says the probe budget was spent, the model reasons, and that is fine.
2. If answers come back empty, or you see OutputBudgetExhausted or the 400, raise LLM_MAX_OUTPUT_TOKENS. Past 1600 if classification is what fails.
3. Where the provider supports it, LLM_EFFORT lowers how much the model thinks, and so how much of the budget it uses.

The settings are on the LLM providers page and in configuration. To try it, pip install nlqueries-core, set a model, and run health.

All posts · RSS feed · Documentation