AI-Generated SQL Needs Semantic Safety Gates, Not Just a Green Run Button

The most dangerous AI-generated SQL is often not the query that crashes.

It is the query that runs quickly, returns a plausible number, and is wrong.

Imagine a founder adding a natural-language analytics box to a subscription product. A user asks, “What was net revenue from annual plans last quarter?” The model produces valid SQL. The database accepts it. The chart shows $482,000. Nobody sees an error.

But the query joins invoices to invoice items and then joins the same invoices to payment attempts. Two one-to-many relationships multiply rows. Several invoices are counted more than once. The answer looks credible because it is close enough to last quarter's total, and the product confidently explains the trend.

This is not a syntax failure. It is a semantic failure: the query is legal SQL, but it does not preserve the meaning of the business question.

That distinction matters for founders building text-to-SQL analytics, AI admin panels, support copilots, finance assistants, internal dashboards, or agents that can inspect a production database. A database engine can tell you whether a column exists. It generally cannot tell you that “revenue” must exclude refunds, that an account balance must not be summed across daily snapshots, or that a customer identifier must never appear in a generated result.

A July 2026 open-source project called sqlsure is a useful prompt for this discussion. Its maintainers describe deterministic checks for fan-out, invalid additivity, undeclared joins, and sensitive columns. They also publish benchmark claims and supporting files. Those results are the project's own evidence, not an independent industry benchmark, and the early project has a narrow rule set. The durable idea is broader than any one tool:

Do not let the model that wrote a query be the only system deciding whether the query is safe and meaningful.

Use a sequence of narrow, inspectable gates. Each gate should answer one question, reject uncertainty when the risk is material, and produce evidence a human can review.

“It Ran” Proves Less Than You Think

Teams often collapse five different checks into one:

1. Syntactic validity: Can this database parse the query?
2. Execution safety: Can it run without modifying data or exhausting resources?
3. Authorization: May this user access every table, row, and column involved?
4. Semantic validity: Do joins, filters, time windows, and aggregations represent the intended metric?
5. Answer validity: Does the returned result make sense in the product context?

Even EXPLAIN has a specific boundary. PostgreSQL documents that plain EXPLAIN shows the plan the optimizer would use, including scans, joins, and estimated costs. That helps detect a likely full-table scan or a suspicious row explosion. EXPLAIN ANALYZE, however, actually executes the statement. PostgreSQL explicitly warns that side effects occur and suggests a transaction followed by ROLLBACK when analyzing a modifying statement. A founder who treats every “explain” mode as a harmless preview can create the exact incident the preview was meant to prevent.

The same boundary appears in managed warehouses. BigQuery offers dry runs that validate a query and estimate bytes processed without running the job. It also supports a maximum-bytes-billed limit. Those are valuable cost controls, but neither proves that the result means “net revenue” rather than “gross invoice value multiplied by payment attempts.”

No single green light is enough because each mechanism sees a different layer.

The Silent Failure Modes Worth Designing For

Join fan-out

Suppose orders has one row per order, order_items has many rows per order, and refunds can also have many rows per order. Joining all three and summing orders.total can multiply the total by the number of matching items and refunds.

Adding DISTINCT is not a general repair. It can hide legitimate duplicate values or deduplicate the wrong unit. The correct fix depends on grain: perhaps aggregate items and refunds separately to one row per order, then join those results.

Wrong join keys

AI often chooses columns with similar names: customer_id, account_id, organization_id, or an email address. The query may return rows even when the relationship is invalid. A declared primary-key/foreign-key relationship helps, but many production schemas have incomplete constraints or relationships enforced only in application code.

An undeclared join should therefore mean “not verified,” not automatically “forbidden” or “safe.” The product needs an explicit policy for uncertainty.

Non-additive and semi-additive metrics

Counts and currency amounts can often be summed at the correct grain. Ratios, percentages, averages, inventory levels, and account balances usually need more care.

Summing daily account balances across a month does not produce a meaningful monthly balance. Averaging per-store conversion rates without weighting by traffic can misstate the overall rate. The SQL parser sees valid functions; only the metric definition knows the operation is invalid.

Time ambiguity

“Last quarter” depends on timezone, fiscal calendar, current date, and whether the boundary applies to an event time, settlement time, or record creation time. A query can be internally consistent and still answer a different question.

Require the product to expose the interpreted period, timezone, and date field near the answer. This is not decorative metadata. It gives the user a chance to catch a mistaken interpretation before acting.

Sensitive output through an allowed query

SQL injection is not the only database security risk. A perfectly parameterized query may still select a column the user should not see. It may also omit a tenant filter, join across accounts, or expose small groups from which people can be identified.

OWASP recommends parameterized queries, allow-list validation where bind variables cannot be used, and least-privilege database accounts. Those controls remain essential. But parameterization separates code from data; it does not decide whether the requested data is appropriate. Authorization and output policy must remain separate gates.

Resource and cost surprises

A generated query may be semantically correct but operationally reckless: an unbounded cross join, a scan over years of events, a regex on a massive text column, or a request that returns millions of rows to the application.

Do not assume LIMIT 100 caps compute. BigQuery's official cost guidance notes that a limit on a non-clustered table does not necessarily reduce bytes read. Use engine-native dry runs, planner estimates, statement timeouts, scan limits, result-row limits, and concurrency budgets.

A Six-Gate Pipeline for an Early Product

The right architecture is not “LLM writes SQL, database runs SQL.” It is a small approval pipeline.

Gate 1: Convert the user's request into an explicit contract

Before generating SQL, represent the intended answer in structured fields:

{
  "metric": "net_revenue",
  "grain": "month",
  "period": {"start": "2026-04-01", "end": "2026-07-01"},
  "timezone": "America/Los_Angeles",
  "dimensions": ["plan_type"],
  "tenant_scope": "current_organization",
  "max_rows": 100
}

The model can help draft this contract, but ambiguous fields should trigger a clarifying question. If “revenue” has three internal definitions, guessing is not helpful automation.

This contract gives later gates something concrete to compare with the SQL. Without it, reviewers can only ask whether the query “looks reasonable.”

Gate 2: Parse the SQL and enforce a narrow statement policy

Use a real parser for the target dialect. String checks such as “must start with SELECT” are brittle because SQL can contain comments, common table expressions, multiple statements, functions with side effects, or dialect-specific commands.

A parser can reject multiple statements, disallowed statement classes, cross joins, unqualified tables, suspicious functions, or references outside an allow-listed catalog. Libraries such as SQLGlot can parse and inspect many dialects, but dialect coverage is never a security guarantee by itself. Test the exact syntax your database accepts.

For embedded SQLite products, the official sqlite3_set_authorizer() interface provides a second layer: a callback can allow or deny actions during statement preparation, including access to particular columns. Parser policy and database enforcement should agree rather than relying on either alone.

Gate 3: Check authorization independently of generation

Resolve the authenticated user and tenant outside the prompt. Do not ask the model to remember to add WHERE organization_id = ....

Prefer database-enforced controls such as restricted roles, views, or row-level policies where the engine supports them. Give the query runner a read-only, least-privilege account that cannot reach unrelated schemas. OWASP's least-privilege guidance is blunt for good reason: an application account should not have admin rights merely because broad access makes development easier.

Also inspect the output columns. A user may be permitted to count customers while being forbidden from retrieving names, emails, health data, or full event payloads. Aggregation permission and raw-record permission are not the same.

Gate 4: Check business semantics against declared facts

This is the gate most early products omit.

Maintain a modest semantic registry containing:

• The grain of important tables and models.
• Approved relationships and expected cardinality.
• Metric formulas, required filters, and valid time fields.
• Additive, non-additive, and semi-additive measures.
• Sensitive columns and minimum aggregation thresholds.
• Tenant and ownership boundaries.

Compare the parsed query with the request contract. Does it use the approved revenue expression? Does the selected timestamp match the period? Does a one-to-many join change the measure's grain before aggregation? Does every join follow a known relationship? If a required fact is missing, return “cannot verify” and ask for review.

That last outcome is important. A checker that turns absent metadata into approval creates false confidence.

Gate 5: Preview cost and execution behavior

Use the database's own planner or dry-run interface with strict limits. For PostgreSQL, inspect a plain EXPLAIN plan before any analyzed run. Execute through a read-only transaction and a restricted role, set a statement timeout, cap returned rows, and cancel abandoned work.

For BigQuery, use a dry run and maximumBytesBilled. For other engines, use equivalent query governors. The controls must live in the execution service, not just in model instructions such as “write an efficient query.”

Cost approval should be contextual. A 30-second internal report may be acceptable; a 30-second synchronous query behind every page view is not.

Gate 6: Validate results and decide who may approve

After execution, test properties of the result:

• Are row counts and totals within credible ranges?
• Are required dimensions unique at the promised grain?
• Do parts reconcile with a trusted total?
• Are null rates or category values surprising?
• Is the result materially different from a known dashboard?
• Does the response include sensitive small groups?

Route high-impact answers to a person. An exploratory chart can show a warning and its interpreted assumptions. A query that triggers payments, pricing, account suspension, financial reporting, healthcare action, or irreversible writes needs stronger approval and domain review. The same pipeline should never silently graduate from “read an answer” to “take an action.”

Test With Known-Wrong Queries, Not Only Happy Paths

A gate is credible only if the team has watched it reject realistic mistakes.

Build a compact evaluation set from your own schema. For each important question, include:

• One reviewed correct query.
• A wrong-key join that still returns rows.
• A one-to-many fan-out that inflates a measure.
• A missing tenant predicate.
• A sensitive raw column added to an otherwise valid result.
• A time-boundary or timezone error.
• An unbounded or expensive variant.
• An ambiguous request that should produce a question, not SQL.

Run the set after changes to the model, prompt, schema, metric definitions, parser, database version, or authorization policy. Track false approvals separately from false rejections. A high rejection rate may look safe while making the product unusable; a low rejection rate may simply mean the tests are too easy.

When the generator and reviewer are both models, vary the reviewer or use deterministic assertions wherever possible. Model agreement is not independent evidence, especially when both systems share the same missing context.

A Founder-Friendly Launch Checklist

Before exposing AI-generated database answers to users, require clear answers to these questions:

• Have we defined the small set of metrics the product can answer reliably?
• Does every request become an explicit metric, grain, period, timezone, and tenant contract?
• Do we parse the exact SQL dialect instead of relying on string matching?
• Are multiple statements, writes, unapproved functions, and unknown tables denied?
• Is the database role read-only and limited to necessary schemas, rows, and columns?
• Is tenant scope enforced outside the model?
• Are approved join paths and cardinalities documented?
• Are ratios, averages, balances, and other non-additive measures labeled?
• Does missing semantic metadata produce “cannot verify” rather than approval?
• Do we use planner or dry-run evidence before execution?
• Are time, scanned data, returned rows, concurrency, and cost capped?
• Are output columns and small groups checked for sensitive disclosure?
• Can users see the assumptions behind the answer and correct them?
• Do high-impact uses require an accountable human approver?
• Does the regression set contain plausible, executable, wrong queries?
• Can we trace the request contract, generated SQL, gate decisions, and final approval without logging unnecessary private data?

That is not a failure to use AI. It is a product boundary that matches the available evidence.

Where This Framework Does Not Fit

This approach is intentionally conservative for products where wrong answers can affect money, access, privacy, or business decisions. It may be excessive for a developer's private scratch database containing synthetic data.

It is also not a promise that semantic metadata makes every query correct. Definitions can be wrong, stale, or politically contested. Foreign keys can be incomplete. Planner estimates can be inaccurate. Result checks can miss a plausible error. Human reviewers can approve bad logic.

Nor should an early team install a new checker and claim the problem is solved. The sqlsure project currently documents a focused set of rules and publishes its own test evidence. Evaluate it—or any alternative—against your schemas, dialects, failure cases, maintenance capacity, and threat model. A deterministic tool can make a rule repeatable; it cannot invent the right business rule.

Finally, this is not a substitute for legal, security, financial, or data-governance review in regulated or high-impact domains. If the product makes decisions about employment, credit, healthcare, insurance, or legal rights, SQL correctness is only one part of the required control system.

The Product Principle

AI makes SQL cheaper to produce. That increases the value of deciding which SQL deserves to run.

The best launch experience is not a magical box that always returns a chart. It is a system that can say: “I interpreted revenue this way, used this period and tenant scope, verified these joins, estimated this cost, and could not verify this one assumption.” Sometimes the correct next action is to ask a question. Sometimes it is to request approval. Sometimes it is to refuse.

For a founder, the key shift is simple: treat generated SQL as an untrusted proposal, not an answer. Parse it, authorize it, compare it with declared business meaning, preview its operational impact, validate its result, and keep consequential decisions accountable.

A query returning zero errors is a starting signal. Trust requires a much longer chain.

References

• sqlsure, Semantic inspector for SQL, project documentation and maintainer-published evidence; retrieved July 12, 2026.
• PostgreSQL Global Development Group, EXPLAIN, execution-plan behavior and the side effects of EXPLAIN ANALYZE.
• PostgreSQL Global Development Group, SET TRANSACTION, commands restricted in read-only transactions.
• PostgreSQL Global Development Group, Client Connection Defaults, statement_timeout and related session controls.
• Google Cloud, Estimate and control costs in BigQuery, dry runs, query validation, quotas, and maximum bytes billed.
• OWASP, SQL Injection Prevention Cheat Sheet, parameterized queries, allow-list validation, views, and least privilege.
• SQLite, Compile-Time Authorization Callbacks, statement preparation authorization with sqlite3_set_authorizer().
• SQLGlot, Python SQL Parser and Transpiler, parser capabilities and supported SQL dialects.