AI & Automation5 min readNetray Engineering TeamUpdated March 4, 2025

Natural Language Query Over ERP Data: What Actually Works

Natural language query over ERP data lets a user ask a plain-English question, such as which customers have past-due orders over 30 days, and receive an accurate answer generated by translating that question into SQL against governed data. The technology works today, but only with the right scaffolding. Pointing a language model at a raw SyteLine or Infor LN schema produces confident, wrong answers because the model cannot know that tdsls400 is a sales order header or which of five date columns represents a promise date. Success comes from a curated semantic layer, retrieval over documented metadata, and honest evaluation.

Why Raw Schema Text-to-SQL Fails on ERP

Public text-to-SQL benchmarks use databases with a dozen clean tables. A production Infor LN instance has thousands of tables with systematic names, and SyteLine has hundreds of tables plus decades of customizations and Uf_ extensions. Three failure modes dominate. First, schema linking breaks: the model cannot select the right five tables from three thousand. Second, business semantics are invisible: nothing in the DDL states that on-time delivery uses promise date at line level and excludes cancelled lines. Third, joins are subtly wrong, especially around multi-site, multi-company, and effective-dated records, so the query runs successfully and returns a plausible number that is off by 12 percent. Silent wrongness is the real risk, not errors.

The Architecture That Works: Semantic Layer Plus Retrieval

Point the model at a curated layer, never at the raw ERP. Build a warehouse with conformed dimensions and a semantic model where each KPI is defined once. Then give the model a retrieval index over that layer containing table and column descriptions, sample values, worked example questions with their correct SQL, and metric definitions. At query time, retrieve the handful of relevant tables and examples, generate SQL constrained to the certified model, validate it, and execute under the user's row-level security. Return the answer alongside the generated SQL and the row count so a skeptical controller can verify it.

Restrict generation to certified views and measures, not the full ERP schema
Index column descriptions, distinct value samples, and 50 to 200 curated question-SQL pairs
Validate generated SQL against a parser and a cost estimate before execution, with a row limit
Always show the SQL, the tables used, and a data-as-of timestamp with the answer

Guardrails, Security, and Read-Only by Design

Every query must execute as the requesting user or through a service account carrying that user's site, business unit, and customer entitlements, so natural language never becomes a privilege escalation path. Enforce read-only credentials at the database level, not just in the prompt, because prompt instructions are not a security boundary. Add a statement timeout and a maximum scanned-bytes limit to prevent a vague question from consuming the warehouse. Log every question, the generated SQL, the executing user, and the result size for audit. For defense contractors, run the model on-prem so questions containing controlled program names never leave the accredited boundary.

Evaluating Accuracy Before You Trust It

Do not launch on demo impressions. Build an evaluation set of 100 to 300 real questions collected from your report backlog, each with a human-verified correct answer, and measure execution accuracy, meaning the result matches, rather than exact SQL string matching. Track accuracy by question type: simple filters and aggregations typically land above 90 percent, while multi-hop questions involving costing, revisions, or period comparisons are much harder. Publish the accuracy number to users and route low-confidence questions to a human rather than guessing. Treat the evaluation set as a living asset: every time a user reports a wrong answer, add that question with its verified result, so the harness grows toward the questions your business actually asks instead of a generic benchmark.

Collect 100 to 300 real questions from your report backlog with verified expected results
Score execution accuracy on result sets, not SQL text similarity
Report accuracy per question class so users know where the assistant is weak
Re-run the evaluation on every model, prompt, or semantic model change as a regression gate

How Netray Deploys NL Query on ERP Safely

Netray builds the semantic layer first, then layers an AI agent that generates SQL only against certified models, inherits row-level security, and returns the SQL with every answer. We seed the retrieval index with your metric dictionary and a curated question bank drawn from your own report requests, and we ship an evaluation harness that runs as a regression test on every change. In typical SyteLine and LN deployments, execution accuracy on the first evaluation round lands around 70 percent and reaches the high 80s to low 90s after two or three iterations of metadata and example tuning. For ITAR environments the entire stack, including inference, runs on your hardware.

Frequently Asked Questions

Can ChatGPT query my ERP database directly?

Not safely in its raw form. A general model has no knowledge of your schema, customizations, or metric definitions, and it cannot enforce your row-level security. A production deployment needs a governed semantic layer, a retrieval index of documented metadata, read-only database credentials scoped to the user, query validation, and an accuracy evaluation. The model is one component of that system, not the system itself.

How accurate is text-to-SQL on ERP data?

Against a raw ERP schema, accuracy is often below 50 percent and, worse, failures are silent. Against a curated warehouse with a semantic layer, documented columns, and curated example queries, well-tuned deployments reach the high 80s to low 90s on simple and moderately complex questions. Multi-hop questions involving costing, revisions, or period-over-period comparisons remain harder and should be routed to certified reports.

Is natural language query secure enough for defense manufacturing data?

It can be, if the architecture is right. Run inference on-prem or in an accredited enclave so questions and results never leave the boundary, execute queries under the requesting user's entitlements with read-only credentials, log every question and generated statement for audit, and restrict access to certified views that already exclude controlled attributes where required. Prompt instructions alone are never a sufficient security control.

Key Takeaways

1Why Raw Schema Text-to-SQL Fails on ERP: Public text-to-SQL benchmarks use databases with a dozen clean tables. A production Infor LN instance has thousands of tables with systematic names, and SyteLine has hundreds of tables plus decades of customizations and Uf_ extensions.
2The Architecture That Works: Semantic Layer Plus Retrieval: Point the model at a curated layer, never at the raw ERP. Build a warehouse with conformed dimensions and a semantic model where each KPI is defined once.
3Guardrails, Security, and Read-Only by Design: Every query must execute as the requesting user or through a service account carrying that user's site, business unit, and customer entitlements, so natural language never becomes a privilege escalation path. Enforce read-only credentials at the database level, not just in the prompt, because prompt instructions are not a security boundary.

Put this into numbers

Free interactive tools for exactly this problem. No signup to use them.

Free Tool

AI Model Selection Assessment

Score ten decision factors - data sensitivity, task complexity, volume, latency, and internal capability - to see whether a self-hosted open-weight model fits your workload.

Free Tool