Skip to main content

Lumen AI

Lumen is the AI layer built into CryspIQ®. It does three things: it suggests how your source data maps into the enterprise data model, it answers questions asked in business language, and it assists people using the platform.

What makes it different from a general-purpose assistant pointed at a database is that it works against a model whose meaning is already defined. It is not inferring what your data means. That is already recorded — in the enterprise data model, in the fact types and in your confirmed mappings — and Lumen reads it.

Automated data modelling​

Most organisations looking at AI for data are really looking for one thing: less dependence on scarce engineers to get data into a usable shape.

That work is usually large because the model does not exist yet. Each new source needs someone to decide what the data means, design somewhere to put it, and write the transformation to get it there. Repeat per source, per system, per acquisition.

CryspIQ® removes the first two. The model is pre-defined, so there is nothing to design — the master data elements and the typed fact stores already exist. What remains is deciding which source field belongs where.

That is the step Lumen automates. Point it at a source — a CSV, a table, a feed — and it proposes the mapping for every column: which target field, which dimension, which fact type. A person reviews and confirms rather than starting from a blank page.

The consequence is a change in who can do the work. Mapping a source becomes a review task expressed in business terms — is this column the customer? is this an amount or a quantity? — rather than a modelling and engineering task. Subject-matter experts who understand the data can do it directly, and engineering time goes to the genuinely hard cases instead of every case.

It gets better with use, on your data​

Every mapping a person confirms is recorded. When a column that looks like one Lumen has seen before turns up in another source, that confirmed answer informs the next suggestion — surfaced as a learned mapping rather than a fresh guess.

A learned mapping is treated as reliable once it has been confirmed at least three times with an acceptance rate above 85%. Below that it still informs the suggestion; it does not carry the same weight.

This learning is yours. Lumen improves from your organisation's own confirmations, in your own deployment — it does not pool what it learns across customers, and your mappings do not train anyone else's suggestions.

Questions in business language​

The second thing Lumen does is turn a question into an answer.

Someone asks "what was operating margin by region last quarter" and gets a result. They do not write SQL, and they do not need to know that margin comes from one source system, region from another and the customer list from a third.

That last point is the design goal rather than a side effect. Nothing about where your data came from should leak into how someone asks a question. A person should speak their own business language against one unified view, whatever the underlying systems happen to be.

Because the question is resolved against the enterprise data model, the answer uses the organisation's own definitions — the ones recorded on the fact type. Two people asking the same question get the same number, and that number matches the report.

Why this works when it usually doesn't​

Asking questions of data in plain language is not a new idea, and most people evaluating it have seen a version that demonstrated well and disappointed in production. It is worth being specific about why that happens, because it is the same reason Lumen is not exposed to it.

The hard part is not the language. It is the schema.

A tool that connects to your existing warehouse has to work out, for each customer, what an unfamiliar and usually undocumented structure means. Hundreds of tables. Naming that reflects a decade of different conventions. Join paths nobody wrote down. Four tables that look like they might be the customer, one of which is right. The tool has to infer all of it, per customer, before it can answer anything — and that inference is where the errors come from.

That is also why the benchmarks look better than the deployments. Benchmark schemas are small and coherent. Real warehouses are neither.

Lumen never does that inference, because the structure is the same everywhere.

Every CryspIQ® deployment has the same seven fact stores, the same dimensions and the same relationships between them. Not similar — the same. The structure does not vary by customer, by industry, or by which source systems happen to be connected.

What varies is the vocabulary: which business concepts exist and what they are called. And that is recorded as data — in the fact types, in the semantic context, in your confirmed mappings — so Lumen reads it rather than guessing it.

The distinction matters more than it sounds. Structure has to be inferred and can be inferred wrongly in ways that are hard to detect. Vocabulary can be looked up.

This is not a claim that Lumen has a better language model. It is the same class of model everyone else is using. The difference is the size and stability of the problem it is pointed at.

The corollary​

A tool designed to work against any warehouse cannot have this property, because the flexibility that makes it broadly applicable is exactly what leaves the hard problem in place. Working against arbitrary structure means inferring arbitrary structure.

The same holds for the semantic layer that such tools require to be useful — a second definition of your business, built and maintained alongside the warehouse, which drifts from it as both change. CryspIQ® has no separate semantic layer to drift, because the definition lives on the fact type that the data itself points at. It cannot fall out of step with the data it describes.

Lumen does not write SQL​

This is worth being precise about, because it is the part enterprise buyers ask about and the part most AI-on-your-data tools cannot claim.

The model never produces SQL. It produces a structured plan that references only identifiers that already exist in the schema — this entity, this field, this filter. That plan is validated before anything is generated, and a deterministic engine turns the validated plan into SQL.

Two things follow directly:

A hallucinated field cannot reach the database. If the model refers to something that does not exist, validation rejects the plan. It fails before generation rather than producing a query that errors — or worse, one that runs and returns something wrong.

The model cannot inject SQL. It never touches table names, column names, operators or literal quoting. Those are handled by the engine, which quotes values according to each column's actual data type.

The AI decides what to ask for. It does not get to decide how the database is addressed.

Assistance inside the platform​

Lumen also answers questions about CryspIQ® itself — how to create a mapping, what a data quality score means, how to configure security. Those answers are grounded in the product documentation shipped with the platform rather than generated from memory, and they are aware of the role of the person asking, so a data steward and a company administrator get guidance relevant to what they actually do.

What Lumen is not​

Worth stating plainly, because AI claims in this space are frequently overstated.

It is not a replacement for judgement about your data. Lumen proposes mappings; a person confirms them. That review step is deliberate — it is where the organisation's knowledge enters the model, and it is what the learning improves from.

It does not invent meaning. Where a business definition has not been established, Lumen has nothing to read, and it will not manufacture one. The quality of what it can do is bounded by the quality of the model beneath it — which is the argument for getting Enterprise Data Efficiency right first.

It is not a general-purpose chatbot. It answers questions about your enterprise data and about the platform. That narrowness is the point: everything it says is traceable to your model, your mappings and your documentation.