Ask Claude about authentication and it searches your codebase. Maybe it finds the right file. Maybe it retrieves something tangentially related. Maybe it misses entirely and generates a solution that looks plausible but does not match your patterns.
The inconsistency is frustrating because there is no way to predict when it will work. The problem is not the model. It is how code gets retrieved before it enters the context window—and that part is fixable.
The Keyword Problem
Traditional code search relies on keywords. Type "auth" and it finds files with "auth" in the name or content. Simple. Also limited.
Code does not work that way. A function called validateUserSession handles authentication but never uses the word "auth." A file named middleware.py might be the authentication layer. The relationship between what developers ask and what they need is not a string match.
Keyword search finds what you say. It misses what you mean. Most retrieval failures in RAG pipelines and retrieval augmented generation systems live in that gap.
What Semantic Routing Does
Semantic routing bridges intent and code.
Instead of matching strings, it matches concepts. The question "how does user login work?" connects to authentication code—regardless of naming conventions, file structure, or whether anyone wrote documentation. Login, authentication, session validation, credential checking. Same conceptual neighborhood, different words.
This works because code has meaning beyond its syntax. A function's purpose, its relationship to other functions, the patterns it implements—these carry semantic weight. Hybrid search approaches that combine keyword matching with semantic understanding outperform either method alone. Routing based on meaning finds relevant code that keyword search would miss.
The chunking strategy matters here. How code gets broken into searchable pieces—by function, by file, by logical block—determines what semantic chunking can actually surface. Bad chunks mean bad results, regardless of how sophisticated the matching is.
Keyword search is structural. Semantic routing is conceptual. Different problems.
Why It Matters for AI Assistants
AI coding assistants are only as good as the context they receive.
Give Claude the wrong files and it produces wrong suggestions. Give it too many files and important details get lost in the middle of the context—the well-documented phenomenon where models lose track of information buried in long inputs. Give it the right files—the specific code relevant to the current task—and output quality increases significantly.
The challenge is knowing which files are "right" for any given prompt. Developers can specify manually. That works until it does not. Complex questions touch multiple areas. Unfamiliar codebases do not reveal their structure easily. And the overhead of constantly directing the AI defeats the purpose of having an assistant.
Semantic routing automates that selection. The prompt itself becomes the query. Relevant code surfaces without manual intervention.
The alternative is serving as librarian for the AI—manually managing token efficiency by hand-picking files to stay within the token limit. Tedious. Also defeats the purpose. An assistant that requires constant management is not assisting. It is overhead with a conversational interface.
The Mechanics
At a high level, semantic routing involves a few steps.
Code gets processed into searchable units. Those units get represented in a way that captures meaning, not just text. When a query arrives, it receives the same treatment—converted into a comparable representation. Then it becomes a matching problem: which code units are semantically closest to the query?
The details matter. How code gets split affects what can be found. How meaning gets represented affects matching quality. Reranking—re-scoring initial results to surface the most relevant code—affects what the AI actually sees. These are engineering problems with trade-offs, not magic.
Context compression techniques can further reduce token cost by distilling retrieved code to its essential parts before it enters the model. Different approaches optimize for different things. Speed versus accuracy. Generality versus domain specificity. The right choice depends on the use case.
What This Changes
Manual context selection does not scale.
As codebases grow, knowing "which file handles X" becomes tribal knowledge. New team members struggle. Even experienced developers forget corners of the system. The AI assistant becomes useless for anything beyond the handful of files someone remembers to include.
Semantic routing removes that bottleneck. Ask a question, receive relevant code. The system's knowledge of the codebase exceeds any individual's. That is not replacing developers. It is extending their reach into parts of the code they forgot existed.
The shift is from "tell the AI where to look" to "let the AI find what matters." Working with an assistant that knows the codebase is different from working with an assistant that needs to be told about the codebase. One is collaborative. The other is clerical.
The Limitation Worth Acknowledging
Semantic routing is not perfect.
It depends on representation quality. Generic approaches miss domain-specific meaning. Code that is unusual or does not follow conventions can slip through. Edge cases exist.
The question is not whether semantic routing solves everything. It does not. The question is whether it solves enough to matter. For most developer workflows, finding relevant code 85% of the time beats the current approach of hoping keyword search succeeds or manually specifying files with every prompt.
Good enough, consistently, beats perfect occasionally.
Part of a series on context management for AI-assisted development.