The Dominant Pattern
In April 2024, a team at LinkedIn published a quiet paper on arXiv. The subject seemed minor: six months of deploying a customer support assistance system that combined Retrieval-Augmented Generation with a knowledge graph. The result was less modest. A 28.6% reduction in median ticket resolution time, and a 77.6% improvement in Mean Reciprocal Rank on the retrieval of relevant documents. No one in the industry was surprised. RAG has, over the past two years, become what gets deployed when you want a large language model to operate on enterprise data without spending millions to retrain it. It is, to my mind, the default pattern.
Yet what is striking, when one examines this pattern closely, is not its technical robustness. It is that the pattern was adopted en masse as a concession, and that it inherits the limits of one. RAG did not prevail because it was the best conceivable architecture. It prevailed because it was the architecture that made it acceptable, for a legal department or an audit committee, to let a language model produce answers on internal matters. The nuance is not anecdotal. It conditions, I believe, the trajectory of the next three years.
The concept itself has become so familiar that we forget its lineage. It was formalized in May 2020 in a paper by Lewis et al. (arXiv:2005.11401), published by a joint team from Meta AI, University College London, and New York University. The central proposition fit in a single formula: pair a parametric memory, the model itself, with a non-parametric memory, a document base indexed and queryable at inference time. At the time, the idea targeted mostly open-domain question-answering tasks on academic benchmarks. Five years later, it structures nearly the entirety of enterprise generative AI deployments. According to several surveys reported notably by Databricks and by the literature on compound systems in 2024-2025, one can prudently estimate that on the order of sixty percent of LLM systems in production today incorporate some form of RAG. No other architecture, in recent years, has achieved such an adoption rate.
What RAG Actually Solves
One must start by granting RAG what it genuinely delivers. The problem it addresses is not a minor one. A pretrained language model, left to itself, has no internal means of distinguishing a factual claim it has learned from one it has just invented. Its parameters encode statistical regularities, not facts. When asked to answer about documents it has never seen, or about information postdating its training, it produces plausible text without any ability to return to a source. This is what the industry calls, out of lexical laziness, hallucination. RAG addresses this precise problem, and it addresses it well.
The pipeline holds in three phases. The first is indexing. Source documents are split into semantically coherent segments by a chunking step, then converted into vectors via an embeddings model and stored in a database optimized for similarity search. The second is retrieval. When a query arrives, it is itself vectorized and compared against the base. The closest segments are extracted, often refined by a reranking step or by hybrid search combining vectors with classical lexical methods such as BM25. The third is generation. The model receives the question and the retrieved segments in its context, and produces an answer meant to be anchored in the sources.
On this foundation, the ecosystem has stacked variants. Graph RAG replaces or supplements the vector base with a knowledge graph, enabling multi-hop reasoning over related entities. Self-RAG adds a reflective loop where the model decides for itself whether it needs to retrieve, and evaluates the relevance of what it obtained. Corrective RAG, or CRAG, adds a corrective step when retrieval quality is poor. Agentic RAG transforms retrieval into a tool that an agent calls conditionally rather than systematically. Golden-Retriever, in industrial contexts, adds a query augmentation step that clarifies domain jargon before search, which reduces off-topic retrievals on technical corpora. Each variant improves a specific point of the pipeline. None challenges the central idea: separate memory from reasoning, and reconnect the two on the fly.
The economic argument deserves to be taken seriously as well. Fine-tuning, which was the natural alternative in the earliest enterprise AI architectures, presupposes a retraining cycle each time knowledge evolves. For a company whose internal documentation changes every week, that is untenable. RAG shifts the cost: instead of retraining, one reindexes, which is several orders of magnitude less expensive. This economy is real. It is simply not, as we shall see, without a counterpart.
One should add a less-discussed virtue of the pattern. RAG cleanly separates two things that earlier architectures conflated: the model's ability to manipulate language, and the knowledge on which it operates. This separation has a valuable collateral effect. When a more capable model ships, you substitute it for the old one without touching the document base. Conversely, when the corpus evolves, the model stays stable. This modularity, which tends to be underestimated in proof-of-concept architectures, becomes decisive at the scale of several years of operation. It partly explains why organizations that could, in theory, have waited for more capable models preferred to deploy RAG as early as 2023 rather than defer their investments. The implicit wager was that a decoupled architecture ages better than a monolithic one. On that precise point, I believe the wager has been borne out.
What RAG Does Not Solve
This is where, I believe, the dominant discourse becomes misleading. RAG is presented in most consultancy notes and vendor blog posts as a solution to the hallucination problem. That presentation is partly false, and the false part matters more than the true one. RAG corrects one category of hallucinations, those bearing on facts the model does not know. It leaves at least four others untouched, and at times it creates new ones.
The first point is what I will call, for lack of a better term, reasoning hallucination. Once the right documents are retrieved and placed in context, nothing compels the model to draw the right conclusion from them. It can cite a correct paragraph and build, from it, a false deduction. RAG benchmarks measure almost exclusively the quality of retrieval, sometimes the faithfulness of the answer to its sources, but rarely the correctness of the intermediate reasoning. Yet in real use, that is often where critical errors appear. A legal support system can retrieve the right article of the code and interpret it backwards in the very next sentence. The citation reassures the user. It does not guarantee the soundness of the deduction.
The second point follows from the first. RAG produces what I see as a false sense of confidence by citation. The user sees a source. They believe they have verified. In reality, they have only verified that a source exists. They have almost never read the source in full, they have not assessed whether it applies to their context, they have not weighed several possibly contradictory sources that the system may not have all retrieved. The citation plays, sociologically, the role of a warrant. And the more polished the interface, the more effective that warrant becomes, independent of its real epistemic value. The effect can be observed in assisted legal research tools and in medical decision-support systems, and it has no purely technical answer. Improving the reranker does not change the user's cognitive posture toward a sourced answer.
The third point is more prosaic but probably more costly over the long term. RAG carries an operational cost systematically underestimated at the proof of concept stage. The vector base must stay current, which presupposes a robust ingestion chain, a strategy for purging obsolete documents, and fine-grained instrumentation of data freshness. When the embeddings model evolves, and it evolves faster than migration budgets, the entire vector set must be recomputed, which is trivial neither in compute cost nor in operational complexity if the base is large. The latency added by the retrieval phase, generally between one hundred and five hundred milliseconds, becomes a problem as soon as one is dealing with conversational chatbots where perceived responsiveness conditions usage. Finally, document-level security trimming, the guarantee that a user only retrieves what they are authorized to see, presupposes deep integration with identity and access systems. That integration is, in most organizations, the element that delays production rollout by six to eighteen months.
The fourth point bears on the dependency on chunking. Splitting a corpus into segments is an editorial choice far more than a technical one. Poor chunking produces partial retrievals. The model receives a paragraph that contains a claim but not the condition that qualifies it, or a conclusion without its premise. The worst part is that these partial retrievals are not detectable at inference time. No internal signal tells the system that it has just read half of an argument. On legal, medical, or technical corpora where the meaning of a passage depends heavily on its surroundings, this is a structural source of error. Hierarchical or contextual chunking techniques attenuate the problem, they do not solve it.
One can summarize these four limits with a more general observation. RAG transfers the problem of truth from the model to the document base, without ensuring that the document base is itself better qualified to bear it. The governance of sources, their freshness, their internal consistency, the absence of contradictory versions becomes the new bottleneck. Yet in most large organizations, internal documentation was not designed to serve as raw material for an automated system. It is scattered, redundant, sometimes contradictory, often obsolete with no mechanism to flag it. Deploying RAG without massive investment in qualifying the corpus means entrusting a probabilistic system with a documentary heritage that has never been cleaned. Early deployments regularly run into this. The incorrect answers do not come from the model, they come from the fact that the model faithfully cited a 2019 document that should not have been indexed.
The Competition from Long Context
The other fact the promotional narrative neglects is that RAG does not run alone. While the industry was standardizing the pattern, model context windows lengthened in proportions no one had anticipated three years ago. Gemini 1.5 then 2.0 now accept up to two million tokens of input. Claude offers windows beyond two hundred thousand tokens and experiments with even larger contexts. GPT-4-turbo and its successors offer one hundred twenty-eight thousand tokens and more depending on version. For many enterprise use cases, loading the relevant material directly into the model's context, without any prior indexing, has become a viable option.
Better still, in some cases, it has become superior. On a fifty-document client file, on a product manual, on an audit report, long context provides the model with the entire material in one stroke, without depending on the quality of a retriever that might miss a piece. The cost per request is higher, but there is no longer any risk of partial retrieval, no indexing pipeline to maintain, no freshness problem. The converging practice, in 2026, is no longer a binary choice between RAG and long context. It is a partition: RAG for massive corpora, on the order of a terabyte, where direct inclusion is impossible; long context for targeted sessions on well-bounded documentary sets; and increasingly hybrid combinations where RAG selects coarsely and long context loads broadly.
This partition has a strategic consequence that is sometimes forgotten. The frontier moves faster than migration budgets. A RAG architecture validated in 2024 for a corpus of one hundred thousand pages may, by 2026, have become suboptimal because the context windows of the models in use have grown tenfold in the interval. Teams that invested heavily in the indexing chain, in embeddings tuning, and in rerankers do not want to hear it, but their ROI now amortizes over a shorter horizon than they planned for.
This competition needs to be qualified on a point that commercial demonstrations often omit. Long context is not free, nor is it uniformly effective. The available studies show that recall quality degrades as one approaches the upper bound of the window, and that intermediate positions of a long context are less well exploited than the beginning and the end. Loading a million tokens into the model does not guarantee that it will make pertinent use of the middle portion. RAG, in certain configurations, remains preferable precisely because it places the right pages in the right positions. The competition between the two approaches is therefore not settled by raw capacity alone. It depends on the structure of the corpus, the nature of the queries, and the usage model. That is less readable than the narrative of replacement, but it is more accurate.
The Pivot Shifting to Protocols
The most interesting shift, to my mind, is not the one driven by long context. It is the shift in orchestration. In November 2024, Anthropic published the Model Context Protocol, generally abbreviated MCP. The proposal is sober: standardize the way a model accesses tools, data sources, and prompts through a unified interface. Where each vendor had until then had its own connector format, MCP offers a common protocol, in the manner of what LSP did for code editors. Within months, the ecosystem aligned. OpenAI announced compatible support, dozens of MCP servers were published for the main enterprise sources, and agent frameworks integrated the protocol as a low-level layer.
What changes for RAG is its status. In an MCP world, RAG is no longer the architecture of the system, it is one of the tools the agent can decide to call. Retrieving from an indexed corpus becomes a tool among others, on the same footing as querying a SQL database, reading a file from a Drive, executing a query against a ticketing system. The intelligence of orchestration, that is, the choice of when and how to retrieve, rises a level, out of the RAG pipeline and into the agent layer. The Skills released by Anthropic, which let an agent load specific competencies on demand, point in the same direction: RAG ceases to be the architectural pivot and becomes a component.
This is not the end of RAG. Systems that must answer over massive corpora will continue to index, vectorize, and rerank. It is the end of its status as central architecture. The center of gravity moves toward the orchestration layer, and that, I believe, is where the structuring trade-offs of the next three years are decided. The question is no longer how to build the best retriever. It is how an agent decides, in a task trajectory, what information it needs, and where it will go to find it.
This shift also alters the nature of the competencies teams need to assemble. For two years, the market hired at great expense engineers specialized in vector stores, embeddings, and retriever tuning. Those skills remain useful, but they now cover a subdomain rather than an entire discipline. The profiles rising in value are those who can design agent trajectories, model tools in a protocol like MCP, manage the memory of a long session, and instrument decision loops. The center of gravity of value moves from the pipeline to the choreography. Technology directors who structured their teams around RAG alone two years ago are discovering that they must now add a layer whose scope they had not anticipated.
RAG as Institutional Concession
It remains to understand why RAG imposed itself so forcefully despite the limits I have just described, and why it will persist durably in architectures even once long context and orchestration have moved the pivot. My reading, which departs from the usual technical narrative, is that RAG's success tells the story less of an engineering victory than of an institutional concession. Companies did not adopt RAG because it ranked best on benchmarks. They adopted it because it satisfied a requirement that precedes the question of correctness, the requirement of verifiability.
In a large organization, a system that produces text without being auditable is unusable. Not because it is necessarily false, but because no risk department can sign off on its deployment. RAG, by attaching citations to each answer, supplies a chain of traceability. That chain is not proof of truth, as we have seen, it can cite sources out of context or ignore contradictory ones. But it is proof of origin. It allows, in case of dispute or audit, a return to a concrete document and a judgment after the fact. That is enough, in most regulated contexts, to unblock deployment. RAG is, in this sense, less a system of correct answers than a system of defensible answers. The nuance matters, and it explains the system's permanence.
The financial sector grasped this very early. Know Your Customer and anti-money-laundering processes now run through RAG systems that retrieve, in real time, the latest regulatory directives and updated sanctions lists, and that can justify each decision with an actionable reference. The healthcare sector followed, with clinical decision support systems that simultaneously consult practice guidelines, pharmacological databases, and the patient record. Industry adopted it for technical documentation, where the traceability of a maintenance procedure matters as much as its accuracy. In all these cases, the driver of adoption is not raw model performance, it is the ability to make AI auditable.
This reading also illuminates a paradox I regularly encounter in conversations with IT leadership. Organizations that adopt RAG present it internally as a response to hallucination, that is, as a technical fix. But their real evaluation criteria, the ones applied when the moment comes to sign off on production, bear almost exclusively on the system's governance. Who sees what, how one traces back to the source, how one proves that a decision was made on the basis of a given document on a given date. The question of the intrinsic correctness of answers, oddly, comes after. This inversion of priorities is not abnormal. It reflects the reality of deploying AI in regulated environments, where auditability is a necessary condition and performance only a desirable one. RAG adapted to that hierarchy. That is probably the deepest reason for its success, and the one that guarantees it will remain, in one form or another, a durable component of architectures.
This institutional reading has a direct consequence for decision-makers. RAG will not disappear, because the requirement to which it responds is not disappearing. No mature company will accept, in any foreseeable future, AI systems that produce non-sourceable claims on decisions of consequence. But RAG will not be the pivot architecture either. It will become what it ought to have been from the start: a specialized grounding layer, called by orchestration when context demands it, ignored when other mechanisms, long context, agent memory, specialized tools, suffice.
The question for technical teams, and even more for the leadership that funds them, is therefore no longer whether to invest in RAG. The answer to that question is obvious, and it is positive, if only for compliance reasons. The harder question is where to place the architectural pivot for the next three years. My conviction, looking at what is actually being deployed in mature organizations in 2026, is that the pivot has already shifted. It lies in the orchestration layer, in the definition of agents and their trajectories, in the choice of data-access protocols. RAG takes its place there, important but subordinate. Continuing to treat it as the central architecture means fighting a battle already won while the next one is being decided elsewhere.
One last remark, more political than technical. RAG normalized the idea that an AI system must be able to produce its sources. That is, to my mind, its most important contribution, and it will survive every architecture that follows. Independent of orchestration protocols, context windows, and agent models, the social and regulatory expectation of verifiability is now installed. Systems that cannot meet it will, whatever their merits otherwise, be excluded from consequential uses. In that sense, RAG, even displaced from its central status, will have durably changed what enterprises are willing to accept of an AI. The technical pattern will pass. The standard of proof it imposed will remain.