The instinct is understandable: our data is a mess, so first we consolidate everything into one warehouse, then we do AI. The record of that plan is grim. Multi-year unification projects outlive their sponsors, freeze all AI value behind a milestone that keeps moving, and usually re-fragment within two years because the business keeps buying tools.
The working pattern inverts it: leave the data where it lives and build access over it. A retrieval layer (the technology family is called RAG, retrieval augmented generation) indexes documents, tickets, CRM records, and wikis in place, finds what is relevant to a task, and supplies it to the model as context. Weeks to stand up over the first few sources, not years.
The question "what are the big data investments we need for AI?" has a disciplined answer: only the ones your next three automated workflows require. If the sales-call summarizer needs CRM history and call transcripts, connect those two sources well: clean identifiers, sane permissions, freshness guarantees. Ignore the seventeen other systems until a workflow demands them.
This converts data strategy from a monolithic program into an incremental one with a payback per step, and it means your data quality effort lands exactly where it produces value instead of where the architecture diagram says it should.
Before approving any data unification project, require the alternative bid: what would a retrieval layer over the existing systems cost, and which of the next three workflows does it unblock this quarter? Architecture consolidation must beat that bid on value and time, not on elegance.
A retrieval layer inherits a hard question: the model can now read everything it is connected to, but should this user, through this agent, see this document? Retrieval must respect source permissions per query, not index-wide. This is the least glamorous and most important line item in the plan; teams that skip it either leak internal information across departments or lock the system down so hard it becomes useless. Budget it from day one.
The deepest silo is not a database; it is tacit knowledge: why we price this way, what happened last time we tried that, which client hates which phrasing. It walks out the door with every departure, and no warehouse project touches it.
Building the company brain is a capture-habit problem, not a technology problem. Three habits move most of the value. Decision records: every significant decision gets five written lines (context, options, choice, reason, owner) in a searchable place. Postmortems and win-mortems: what actually happened, written while it is fresh, both failures and wins. Exit downloads: structured interviews before any departure, indexed with everything else. The retrieval layer then makes twenty years of judgment askable in plain language: not just what did we decide, but why.
Companies that do this stop re-learning their own lessons, which is a larger productivity gain than most automation.
No. A retrieval layer (RAG) indexes data in its existing silos and supplies relevant context to models on demand. Consolidate later, incrementally, only where automated workflows justify it.
Whatever your next three automated workflows require: clean identifiers, permissions, and freshness on those specific sources. Workflow-driven data investment pays back per step; architecture-driven investment pays back at the end, if ever.
Three habits feeding one searchable index: five-line decision records, postmortems for failures and wins, and structured exit downloads. A retrieval layer over that index makes institutional judgment askable in plain language.
Bring one AI question from this article. We will tell you what we would do, whether or not you hire us.
Book a 15-Min Chat →