NLQueries GitHub

Blog  / 

Choosing an LLM for text-to-SQL: Claude, GPT, DeepSeek and Bedrock

Published . Written for nlqueries-core 0.4.

llm configuration cost

People ask which is the best LLM for SQL generation, and the honest answer is that it depends on your schema, your questions and where your data is allowed to go, more than on any leaderboard. What I can tell you is what each choice changes in NLQueries: which settings it needs, what it does with your token budget, what it caches, and what it costs you to find out. This post covers the four providers nlqueries-core 0.4 has been run against: Anthropic, OpenAI, Amazon Bedrock and DeepSeek.

Two models, not one

NLQueries uses two model tiers, and the choice is really two choices.

  • LLM_MODEL writes the SQL, corrects it when the database refuses it, and phrases the answer. This is where a stronger model pays: multi-table joins, ambiguous column names, a business rule buried in the knowledge base.
  • LLM_MODEL_FAST handles the short calls: deciding whether a question is about data or documents, and resolving a follow-up like "and for Germany?" against the conversation. These want speed and a low price per call.

Set both whenever you change provider. A fast tier left on another provider's model sends those calls there, and the follow-up resolver carries the question and the conversation history, so they go somewhere you did not choose. On Bedrock that case is refused before anything is sent; elsewhere it is up to you.

Anthropic: the default

One key, ANTHROPIC_API_KEY, and you get Claude Sonnet 5.5 for the main tier and Haiku 5.5 for the fast one. Two things come with it.

Effort. Claude 5 models think adaptively, and LLM_EFFORT caps how much: low (the default) up to max. More effort helps on the hardest questions and costs output tokens on every question, so start low and raise it if answers miss on complex joins.

Prompt caching. The stable part of the SQL prompt, the instructions and the agent's schema, is the same for every question to that agent. It is sent with a cache marker, so repeat questions read it from Anthropic's prompt cache instead of paying for it again. The larger the schema, the more that saves. Nothing needs setting.

OpenAI

Set OPENAI_API_KEY alone and NLQueries detects OpenAI and reaches it through LiteLLM. In 0.4 the defaults it picks are openai/gpt-4o and openai/gpt-4o-mini. I would set the models explicitly to the ones you want:

OPENAI_API_KEY=sk-...
LLM_MODEL=openai/<model>
LLM_MODEL_FAST=openai/<model>

The next release changes those defaults to openai/gpt-5 and openai/gpt-5-mini.

With both an Anthropic and an OpenAI key set, Anthropic wins; set LLM_PROVIDER=litellm to use OpenAI. LLM_PROVIDER=openai is not a provider name, and fails.

Reasoning models. No effort is sent to OpenAI models in 0.4: they think at their own default. Their reasoning is billed from the same output allowance as the answer and spent first, which is why the max_tokens post exists. If answers come back empty, raise LLM_MAX_OUTPUT_TOKENS.

One caution in 0.4. GPT-5 and the o-series accept only the default temperature while they reason. If you turn on NLQ_SELF_CONSISTENCY, which is off by default, its extra repair candidates ask for other temperatures, and LiteLLM refuses them, so the vote runs on one candidate. Leave it off with those models in 0.4. The next release leaves the temperature out for any model that refuses one.

Amazon Bedrock

Bedrock is the choice when the model calls must stay inside your AWS account. It is reached through LiteLLM and authenticates through boto3, so there is no API key: credentials come from the environment, a profile, or the role of the instance, task or pod. Name the models and the region:

LLM_MODEL=bedrock/us.anthropic.claude-sonnet-5-5
LLM_MODEL_FAST=bedrock/us.anthropic.claude-haiku-5-5
AWS_REGION=us-east-1

Three things to know:

  • Set AWS_REGION deliberately. LiteLLM falls back to us-west-2 silently, so a missing region does not announce itself.
  • Set the fast model. Unset, LLM_MODEL_FAST follows LLM_MODEL. That works, but every classification costs what the main model costs.
  • Effort is not sent on Bedrock: each model keeps its own default. Prompt caching is: for Claude models LiteLLM lists as supporting it, the stable part of the prompt is marked the same way as on Anthropic.

Amazon Bedrock covers model access, the us./eu. inference profiles newer Claude models need, and the IAM permissions.

DeepSeek

DeepSeek is reached through LiteLLM too. Name the provider and both models, because neither is detected from the key alone:

DEEPSEEK_API_KEY=sk-...
LLM_PROVIDER=litellm
LLM_MODEL=deepseek/deepseek-chat
LLM_MODEL_FAST=deepseek/deepseek-chat

DeepSeek thinks before it answers by default. LLM_EFFORT is sent as its reasoning_effort, and LLM_THINKING=off turns thinking off entirely. Thinking is billed from the same output budget.

How to choose

Where can the data go? Every SQL question carries the question, the schema and some sample values to the provider, and the answer path carries result rows on hybrid questions. What leaves this machine lists it. If that has to stay in your cloud account, the answer is Bedrock, whatever else is true.

Reasoning or not? A model that thinks gets more of the hard questions right and spends output tokens to do it. For the main tier that is usually worth it. For the fast tier, it is usually not: classification and follow-up resolution are short, and a model that reasons there spends the budget on reasoning first. Pick a fast, non-reasoning model there, or turn its thinking down.

What does it cost? Core counts tokens; it does not price them. An application using NLQueries as a library can collect the token counts of every call a piece of work makes, with nlqueries.llm.use_usage_sink: input, output, cache reads and writes, and reasoning tokens. Multiply by your provider's price list. Prompt caching shows up in the cache-read count.

Does it work on your questions? That is the one that decides it, and no article can answer it for you. Switch the model, then run eval against the agent: it re-asks the questions your query history produced, plus a golden set if you give it one with --golden, and checks that each generated statement parses and names only tables the agent knows. That catches a model that cannot write your dialect or invents tables. It does not judge whether each answer is right; for that, see how to evaluate text-to-SQL on your own data.

After any switch, run health. It makes one real call to the configured model and reports the provider, the model and the round trip, or the error.

The full settings are on the LLM providers page. To try one, pip install nlqueries-core and start with getting started.

All posts · RSS feed · Documentation