Fragmented Data, the Company Brain, and Why You Do Not Need a Two-Year Overhaul

The overhaul trap

The instinct makes sense. Our data is a mess, so first we consolidate everything into one warehouse, and then we do AI.

The record of that plan is grim. Multi-year unification projects outlive their sponsors. They freeze all AI value behind a milestone that keeps moving.

And they usually re-fragment within two years, because the business keeps buying tools.

The pattern that works turns it around: leave the data where it lives and build access over it.

A retrieval layer, from the technology family called RAG, retrieval augmented generation, indexes documents, tickets, CRM records, and wikis in place. It finds what is relevant to a task and supplies it to the model as context. It takes weeks to stand up over the first few sources, not years.

Prioritize data work by workflow, not by architecture

"What are the big data investments we need for AI?" has a disciplined answer: only the ones your next three automated workflows require.

If the sales-call summarizer needs CRM history and call transcripts, connect those two sources well: clean identifiers, sane permissions, freshness guarantees. Ignore the seventeen other systems until a workflow demands them.

That turns data strategy from one giant program into small steps with a payback on each. It also means your data quality effort lands exactly where it produces value, instead of where the architecture diagram says it should.

The permission problem nobody budgets for

A retrieval layer inherits one hard question. The model can now read everything it is connected to, but should this user, through this agent, see this document?

Retrieval has to respect source permissions on every query, not once across the whole index. This is the least glamorous and most important line item in the plan.

Teams that skip it end up in one of two places: leaking internal information across departments, or locking the system down so hard it becomes useless. Budget it from day one.

The company brain: capturing what is in people's heads

The deepest silo is not a database. It is what your people know and never wrote down: why we price this way, what happened last time we tried that, which client hates which phrasing. It walks out the door with every departure, and no warehouse project touches it.

Building the company brain is a capture-habit problem, not a technology problem, and three habits move most of the value. Decision records: every significant decision gets five written lines (context, options, choice, reason, owner) in a searchable place.

Postmortems and win-mortems: what actually happened, written while it is fresh, for failures and wins alike. Exit downloads: structured interviews before any departure, indexed with everything else.

The retrieval layer then makes twenty years of judgment askable in plain language. Not just what we decided, but why.

Companies that do this stop re-learning their own lessons. That is a bigger productivity gain than most automation.

Access First: what companies with messy data and working AI have in common

Some companies have data just as scattered as yours and their AI still answers questions correctly.

They did not clean everything up. They did three smaller things, in a particular order. We call them Access First.

They leave the data where it is and search it. This idea has a birth certificate. In 2020, researchers at Facebook AI Research and two universities published a paper describing a model that looks things up in an outside index before it answers, instead of relying on what it memorized.

They reported that it produced language that was more specific and more factual than a model working from memory alone. They named the method retrieval-augmented generation. Every "chat with your documents" product you have been pitched since is a descendant of that paper.

They fix who can see what before launch day. When Microsoft wrote its deployment blueprint for Copilot, the first of its three pillars had nothing to do with AI. It was remediating oversharing: finding sites and files that are overshared, ownerless, or inactive before the assistant can surface them.

Think about what that means. The company selling the assistant tells its own customers to fix permissions first. The assistant does not create the leak.

It just makes ten years of sloppy sharing searchable in plain English.

They measure whether the search finds the right thing. Retrieval fails quietly. The model still answers.

It just answers from the wrong page. Anthropic measured this and found a standard setup failed to retrieve the right passage 5.7 percent of the time.

Adding a short note of context to each chunk before indexing cut that to 2.9 percent, a 49 percent drop. Adding a second ranking step brought the reduction to 67 percent. You do not need to follow the technique.

You need to know that retrieval quality is a number, and somebody on your side should be watching it.

"So AI-ready data is the big cleanup after all."

You may have seen the warning that most AI projects die for lack of AI-ready data, and concluded the overhaul is back on. Read the warning closely.

Gartner predicts that through 2026, organizations will abandon 60 percent of AI projects that are not supported by AI-ready data. In its survey of 248 data management leaders, 63 percent either did not have the right data practices for AI or were not sure.

Notice the unit. It is projects. Data is ready or not ready for a specific use, not in general.

The same press release says AI-ready data is very different from traditional data management, which is the warehouse project you were about to restart.

So the Gartner number supports the rule in this article. Pick the workflow. Get the two or three sources it needs into shape: clean identifiers, right permissions, fresh enough.

Ship it. Then pick the next one.

The capture habits scale the same way. Google's engineers made blameless postmortems a routine, and software teams keep short decision records for the same reason. Nobody set out to build a company brain.

They wrote things down one incident and one decision at a time.

What to read next, and why

Lack of AI-Ready Data Puts AI Projects at Risk (press release): Read it for what it does not say. Nowhere does it tell you to unify everything first.

Introducing Contextual Retrieval (engineering post): The clearest explanation of why document search fails and how to measure it. The first half needs no technical background.

Microsoft's secure and governed foundation blueprint for Copilot (documentation): Even if you never buy Copilot, the oversharing checklist applies to any assistant you connect to your files.

Postmortem Culture, from Google's Site Reliability Engineering book (book chapter): The best short guide to writing down what happened without blaming anyone. That is the habit behind a company brain.

So the simple takeaway is this: your data does not have to be tidy. It has to be reachable, permissioned, and good enough for the one workflow in front of you.