Why LLMs don't understand your data

A founder recently posted a question that nailed the problem most teams hit once they start using AI for analytics: “The SQL is easy. The slow part is teaching the agent what anything actually means.”

He was running three projects on Supabase, pointing Codex at the database, and getting decent SQL back. But every session, he had to re-explain which Stripe product maps to which app, what “active user” means this week, and which subscription states actually count as revenue. The AI was great at queries. It had no memory of his business.

That's the problem a semantic layer solves. Not the SQL, but the meaning.

Why it matters now

Semantic layers aren't new. Looker had LookML. dbt has MetricFlow. Cube has been around for years. But the problem used to be confined to data teams building dashboards. Now that everyone is querying data through AI (Claude, ChatGPT, Codex), the “what does this column mean?” problem has exploded across every team.

Without shared definitions, your dashboard says revenue is $1.2M, your analyst's export says $1.15M, and Claude says $1.3M. None of them are wrong. They're just using different definitions of “revenue.”

Five approaches people actually use

Talk to ten teams solving this problem and you'll hear five different approaches. Here's what each one looks like in practice.

1. Re-explain it every session

The simplest approach: paste your definitions into the system prompt or chat context every time. This is where most people start. You tell Claude “revenue means sum of amount where status is completed, excluding refunds” and it gets it right. Until tomorrow, when you forget to include the refund exclusion and get a different number.

This works for one person. It breaks the moment a second person queries the data, because now you have two system prompts with slightly different definitions.

2. A markdown file the AI reads

One step up: write your definitions in a markdown file and point your coding agent at it. “Here's metrics.md, read this before every query.” This is popular with developers using Cursor or Codex, and it's genuinely effective for a solo founder.

The limitation is distribution. The markdown file lives locally. If your ops lead wants to ask questions through Claude's web interface, they don't have the file. If you update a definition, there's no mechanism to push it to everyone. And access control (making sure sales can't see HR data) doesn't exist.

3. Gold tables (the medallion approach)

Some teams skip the semantic layer entirely and build curated “gold” tables instead. Raw data goes into bronze. Cleaned and joined data goes into silver. Then you create small, specific tables that answer specific questions. That's gold. The AI only talks to gold tables. Everything is well-named, narrow, and hard to misinterpret.

This works well if you have an engineer willing to build and maintain the pipeline. The downside is that every new question requires a new gold table. You're back to the dashboard factory problem, except now you're manufacturing tables instead of charts.

4. Traditional semantic layers (Cube, dbt, LookML)

The enterprise answer: define every metric, dimension, and join in YAML or config files. Cube gives you an API with caching. dbt Semantic Layer integrates with your transformation pipeline. LookML ties into Looker's ecosystem. These are serious tools built for serious data teams.

They work. They're also a significant project. You need someone who understands both the database schema and the business logic. They need to write and maintain hundreds of lines of config. When the schema changes, the definitions need updating. dbt Semantic Layer requires dbt Cloud (paid). Cube adds another service to run. And neither gives you a way to actually ask questions or see results. You still need a separate BI tool (Hex, Tableau, Looker) on top. So the real stack becomes: warehouse + semantic layer + BI tool + maintenance for all three.

For a company with a 5-person data team, this is standard practice. For a 20-person startup where the founder is also the data person, it's a multi-week project that competes with shipping product.

5. Auto-generated context layer

What if the semantic layer built itself? Connect your database, and the platform reads your schema (tables, columns, foreign keys, data types) and generates descriptions, relationships, and metric definitions automatically. You review and refine them. You don't write them from scratch.

This is the approach we took with Contextflo. The generated definitions are served as context to Claude (or any AI) whenever someone on your team asks a question. “Revenue” means the same thing whether the CEO asks at 9am or the ops lead asks at 4pm. No YAML. No deployment pipeline. No data engineer maintaining config files.

When someone corrects a definition (“revenue should exclude refunds”), that correction persists. The next query uses the updated definition. The system learns from corrections instead of forgetting them. That's the difference between a fancy autocomplete and something that actually improves over time.

What actually changes the category

The interesting insight from practitioners who've lived through previous “natural language BI” attempts (Looker Ask, Tableau Ask Data, Power BI Q&A) is this: the LLM isn't what's different. What's different is that the agent can iteratively repair and extend its own context.

Previous tools only queried. They took a question, generated SQL, returned a result. If the result was wrong, you started over. An agent that persists corrected definitions back into a context layer, so the correction applies to every future query for every user, is a fundamentally different loop.

Which approach fits you?

Solo founder, one project: A markdown file is probably fine. Write your definitions, point Codex at it, and move on.

Small team, multiple people querying: You need shared definitions with access control. A markdown file won't distribute. This is where an auto-generated context layer saves you weeks of YAML writing.

Data team with BI tools: Cube or dbt Semantic Layer makes sense. You have the people to build and maintain it, and you need the integrations with Tableau, Looker, or Hex.

Embedded analytics in your product: Cube's API layer with caching is built for this. If you're serving dashboards to customers at scale, that's the right tool.

The question isn't whether you need a semantic layer. You do, as soon as more than one person or tool queries your data. The question is how much infrastructure you want to take on to get one.

Try it

Contextflo connects to Postgres, BigQuery, Snowflake, ClickHouse, Redshift, and Databricks. Setup takes about 10 minutes. No YAML, no data model to build, no config files to maintain.