Blog /
Why we validate SQL before it runs
Published . Updated . Written for nlqueries-core 0.3.
validation text-to-sql cli
There is a design decision in NLQueries that people ask about often enough to be worth writing down: nlqueries ask generates and checks a query without executing it, and nlqueries query runs it. Two commands, not one.
The reason is that "is this SQL correct" is three questions wearing a trench coat, and they have very different costs.
Three kinds of wrong
Syntactically wrong. The query does not parse. This is the easy case and also the rarest one with a decent model. You catch it by parsing, it costs nothing, and when it happens the fix is mechanical.
Semantically wrong against the schema. The query parses, but it references a table or column that does not exist, or a column name that two joined tables share. The database will reject it. You could find out by running it, but you do not have to: you have a copy of the schema, so you can check it yourself, for free, before spending a round trip.
Wrong about the question. The query parses, runs, and returns rows. It also joins through a one-to-many relationship and triple-counts revenue. This is the expensive case, the one that reaches a slide deck, and the one that no amount of static checking fully solves.
The first two are worth catching automatically because they are cheap to catch and annoying to hit. The third is the one that actually matters, and it is the reason for the separation.
Executing is not free
The obvious objection is: why not just run it and see? If it fails, retry.
Because running a generated query has costs the generator does not know about.
A query against a large partitioned table can scan a great deal of data before it fails or before it returns. On a metered warehouse that is money. On a shared cluster that is somebody else's dashboard timing out. The query is generated in milliseconds; the consequences of running it are measured in whatever your warehouse charges.
And the retry loop makes it worse, not better. A system that runs, fails, regenerates and runs again is spending real compute exploring a hypothesis space. That is fine on a laptop against DuckDB. It is not fine against production Snowflake at three in the afternoon. NLQueries allows exactly one correction after a database error, and skips it when the first attempt used most of the timeout or was refused for permissions.
So the sequence is: generate, check everything checkable without touching the database, and only then decide whether to execute. ask stops before the last step. It is the mode you want in CI, in a review workflow, or when you are pointing a system at a warehouse for the first time and do not yet trust it.
What checking catches, and what it does not
Static validation against the knowledge base's copy of the schema catches unknown tables and columns and ambiguous references. It does not catch a join on mismatched types, or syntax your engine rejects that the parser accepts. That last category is more common than people expect, because models have read far more Postgres than they have read Snowflake, and it is what the one correction after a database error is for.
Whether a statement may run at all, read-only and within limits, is a separate layer, covered in letting an LLM query production safely.
What none of this catches is a query that is internally consistent and answers a different question than the one asked. WHERE created_at >= '2026-04-01' and WHERE created_at > '2026-04-01' both validate. One of them is off by a day.
We are specific about this because the alternative is implying that validation means the answer is right, which would be a more comfortable claim and a false one.
Which is why the query is always shown
Given that a whole class of error survives every automated check, the last line of defence is a person who knows the data looking at the query for five seconds.
That only works if the query is in front of them. Not behind a "show details" toggle, not in a debug log, not available on request. Next to the number, every time.
The failure mode we are designing against is not a system that is wrong. Every system is sometimes wrong. It is a system that is wrong in a way nobody can see, used by people who have stopped expecting to check. Showing the SQL is what keeps the second half of that sentence from happening.
The practical shape of it
In day-to-day use this ends up being fairly undramatic. You ask a question. You get an answer and a query. If the answer is surprising, you read the query, and usually within a few seconds you can see either that it is right and the number is genuinely surprising, or that it read your question differently than you meant.
That second outcome is not a failure of the system. It is the system working: an ambiguity that would otherwise have been buried in a number is now visible as a WHERE clause you can disagree with.