Why LLMs don't understand your data
Five approaches teams use to solve the "what does this metric actually mean?" problem, from a markdown file to YAML config to generated context. Which one fits which team.
A founder posted something that nails the whole problem: "The SQL is easy. The slow part is teaching the agent what anything actually means."
He was running three projects on Supabase, pointing Codex at the database, getting decent SQL back. But every session he had to re-explain which Stripe product maps to which app, what "active user" means this week, and which subscription states count as revenue.
The AI was great at queries. It had no memory of his business.
That is what a semantic layer is for. Not the SQL. The meaning.
Why this got urgent
Semantic layers are not new. Looker had LookML. dbt has MetricFlow. Cube has been around for years. The difference is that the problem used to be contained to data teams building dashboards, and now everyone queries data through an AI tool, so "what does this column mean?" has spread to every team at once.
Without shared definitions, your dashboard says revenue is $1.2M, your analyst's export says $1.15M, and Claude says $1.3M. None of them is wrong. They are using different definitions of revenue, and nobody can tell which one they are looking at.
Your database stores raw data. It does not store what that data means. Revenue could be gross or net. Active user could mean logged in this week or completed an action. A semantic layer writes those decisions down once, so everything downstream agrees.
Five approaches people actually use
Talk to ten teams solving this and you will hear five answers. Here is what each looks like in practice, and where each one stops working.
1. Re-explain it every session
Paste your definitions into the chat or system prompt each time. This is where everyone starts. You tell Claude "revenue means sum of amount where status is completed, excluding refunds" and it gets it right — until tomorrow, when you forget the refund clause and get a different number.
Works for one person. Breaks the moment a second person queries the data, because now there are two prompts with slightly different definitions and no way to tell them apart from the answers.
2. A markdown file the AI reads
One step up: write the definitions in a file and point your coding agent at it. "Here's metrics.md, read this before every query." Popular with developers using Cursor or Codex, and genuinely effective for a solo founder. If that is you, do this.
The limit is distribution. The file lives locally. Your ops lead asking questions in Claude's web interface does not have it. Update a definition and there is no mechanism to push it to anyone. And access control — making sure sales cannot see HR data — does not exist at all.
3. Gold tables (the medallion approach)
Some teams skip the semantic layer and build curated tables instead. Raw goes to bronze, cleaned and joined goes to silver, then small specific tables that answer specific questions: gold. The AI only ever talks to gold tables, which are well-named, narrow, and hard to misread.
This works well if you have an engineer who will build and maintain the pipeline. The cost is that every new question needs a new gold table. You are back to the dashboard factory, except now you are manufacturing tables instead of charts.
4. Traditional semantic layers (Cube, dbt, LookML)
The enterprise answer: define every metric, dimension and join in YAML or config. Cube gives you an API with caching. dbt Semantic Layer integrates with your transformation pipeline. LookML ties into Looker. These are serious tools built for serious data teams, and they work.
They are also a project. Someone has to understand both the schema and the business logic, and write and maintain hundreds of lines of config that need updating whenever the schema moves. dbt Semantic Layer requires dbt Cloud, which is paid. Cube is another service to run.
And none of them let you actually ask a question or see a result — you still need a BI tool on top. So the real stack is warehouse plus semantic layer plus BI tool, and maintenance for all three.
For a company with a five-person data team, that is standard practice and a reasonable trade. For a twenty-person startup where the founder is also the data person, it is a multi-week project competing directly with shipping product.
5. A generated context layer
What if the layer built its own first draft? Connect the database, and the platform reads the schema — tables, columns, foreign keys, types — plus source code and docs, and generates descriptions, relationships and metric definitions. You review and correct rather than writing from an empty file.
That is the approach we took with Contextflo. The definitions are served as context to Claude, or any AI tool, whenever anyone on the team asks a question, so revenue means the same thing whether the CEO asks at 9am or the ops lead asks at 4pm.
The honest limit is the same one every generated artefact has: the draft gets structure right and business meaning wrong often enough that someone who knows the data has to read it. Nothing in a schema records that orders before the 2024 migration use a different status vocabulary. What changes is that reviewing takes an afternoon and authoring takes weeks.
The thing that actually changed
The most useful observation I have heard came from someone who lived through the previous wave of natural-language BI — Looker Ask, Tableau Ask Data, Power BI Q&A — and it is this: the LLM is not what is different.
What is different is that the agent can repair and extend its own context.
Those earlier tools only queried. Question in, SQL out, result back. If the result was wrong you started over, and the tool was exactly as wrong the next time. When a corrected definition can be saved back into a shared context layer, that correction applies to every future query for every user, and the loop is fundamentally different.
The old model: you author a dashboard once and ship it. Every new question needs a new dashboard.
The new model: definitions get renegotiated as people ask questions. Canonical metrics stay rigid; exploration stays flexible. Both coexist.
Worth being precise about one thing: the correction does not save itself. Someone has to decide that "revenue excludes refunds" is the canonical definition and write it down. The system makes that a one-line action rather than a config PR, but it is still a human deciding. A tool that silently learned from every correction would be a tool that learns from every mistake too.
Which one fits you
Solo founder, one project. A markdown file is fine. Write the definitions, point your agent at it, move on.
Small team, several people querying. You need shared definitions and access control, and a markdown file distributes to nobody. This is where a generated context layer saves you weeks of YAML.
Data team with BI tools. Cube or dbt Semantic Layer. You have the people, and you need the Tableau, Looker or Hex integrations.
Embedded analytics in your product. Cube's API layer with caching is built for exactly this. If you are serving dashboards to customers at scale, that is the right tool and it is not close.
The question is not whether you need a semantic layer. You do, the moment more than one person or tool queries your data. The question is how much infrastructure you are willing to take on to get one.
Try it
Contextflo connects to Postgres, BigQuery, Snowflake, ClickHouse, Redshift and Databricks. Setup takes about 15 minutes. No YAML, no data model to build, no config to maintain.