Semantic Search Beats the Deep Agent: What a New Paper Really Shows about Code Search

A coding agent does not spend all of its time writing code. A substantial part of the work is finding out where the relevant logic lives in the repository in the first place.

Two fundamentally different approaches have become common. The traditional one builds a semantic index of the repository in advance and retrieves code passages relevant to a question into the context. The newer one lets an agent explore the current working tree with grep, glob, directory listings and file reads. In the increasingly common deep-agent variant, the main agent delegates that exploration to a subagent so that the many search results do not pollute its own context.

At first glance this sounds like progress. The agent reads the real code in its natural structure, always works against the current state and keeps the main context clean. That architecture now sits behind many modern CLI agents and is increasingly treated as a best practice in context engineering.

The paper “Deep Agentic Search for Repository-Level Code Question Answering: An Empirical Study”, published on 2 August 2026, puts that assumption through a direct comparison for the first time.¹ The result is surprisingly clear: the simpler agent with a semantic index answered 65.2 percent of questions correctly. The Deep Agent reached only 46.2 percent and cost more than twice as much per correct answer.

This is not a general refutation of agentic code search. It is, however, a good demonstration that more orchestration does not automatically produce more intelligence. Sometimes a subagent mainly creates another boundary across which information can be lost.

Two answers to the same context problem

A large repository does not fit completely into a language model’s context window. Even if it theoretically does, putting everything into the prompt is usually a poor idea. Relevant code disappears between tests, generated files, helper classes and similarly named implementations. More context is not automatically better context.

The two architectures in the study solve this problem differently.

With Semantic Search, the repository is split into chunks. An embedding model converts them into vectors stored in a database. The agent formulates search queries, receives the semantically closest code passages and can read the corresponding files when necessary. It is essentially a RAG system for source code.

With Deep Agentic Search, there is no prepared vector index. A main agent plans the task and delegates repository exploration to a subagent. The subagent works in its own context, using tools such as grep, glob, ls and file reads, and eventually returns a compressed answer. The main agent never sees the complete search history, only the handoff result.

Both approaches have obvious strengths and weaknesses:

Approach	Strength	Typical weakness
Semantic index	Finds concepts without exact identifiers and returns compact context	Chunk boundaries can destroy structure, the index can be stale, and retrieval can miss the right code
Direct Grep search	Works on the current filesystem and follows exact symbols and dependencies	Produces a lot of raw text, requires many tool calls and can get stuck in search loops
Delegated subagent	Keeps the main context free of search history	The planner has to formulate a good task and trust the subagent’s result

The interesting part of the paper is therefore not merely which method produces more correct answers. The authors record all tool calls and agent trajectories and analyse why the systems fail.

A comparatively clean experimental setup

The study uses the SWE-QA benchmark with 720 open-ended questions about 15 widely used Python repositories. Each repository contributes 48 questions from the categories What, Why, Where and How. Project size ranges from roughly 13,000 to more than 800,000 lines of Python.

Four models answer every question once with Semantic Search and once with Deep Agentic Search:

Gemini 2.5 Flash
Gemini 2.5 Pro
Gemini 3 Flash
Qwen3-235B

That creates eight conditions and just under 5,760 runs. Every question starts in a fresh context and sees only the repository concerned, preventing earlier questions from influencing later ones.

Both variants are built on LangChain. The Semantic agent is a ReAct agent with three tools: display repository structure, search the vector index and read files. The Deep Agent uses LangChain Deep Agents’ standard harness with task planning, subagents, Grep, Glob, directory listings and file access.

The semantic index is not an aggressively tuned specialist system. The authors use Chroma, text-embedding-3-small, 500-token chunks with 50-token overlap and ten hits per search. The Deep Agent uses the same model for planner and subagent. A run ends after at most 900 seconds or 80 agent steps.

Answers are judged by Claude Sonnet 4.6 for correctness, completeness, relevance, clarity and justification. A score of at least 70 out of 100 counts as a pass. That is initially a weakness because one LLM is evaluating another. The authors therefore validated the judge against 800 answers assessed by three independent software developers. Agreement between the human majority decision and the judge was 84 percent, with Cohen’s Kappa at 0.74. That is not perfect, but high enough that the main result cannot easily be dismissed as a judge artefact.

The result is not close

Across all models, Semantic Search passed 65.2 percent of questions. Deep Agentic Search reached 46.2 percent. The failure rate was 21.9 versus 34.4 percent.

More importantly, the direction was the same for every individual model:

Model	Semantic Search	Deep Agentic Search
Gemini 2.5 Flash	48.4 %	42.8 %
Gemini 2.5 Pro	54.2 %	44.2 %
Gemini 3 Flash	89.3 %	39.7 %
Qwen3-235B	68.8 %	58.2 %

The gap for Gemini 3 Flash is extreme. Even if that outlier is mentally discounted, Semantic Search remains ahead for all four models. The study’s statistical tests find a significant advantage for every model pairing.

The pattern is stable at repository level as well. Deep Agentic Search is narrowly ahead only on Django, by three points. Semantic Search wins on the other 14 repositories. Even SymPy, at roughly 861,000 lines of Python, still favours the index by 13.7 points. The gap becomes smaller as repositories grow, but it does not reverse within the observed range.

That contradicts the simple assumption that a vector index works only on small projects and that direct exploration automatically becomes better past a certain size.

More work, fewer correct answers

Deep Agentic Search was not only less accurate but also more expensive.

On average, a correct answer cost 0.32 US dollars with Semantic Search and 0.74 dollars with the Deep Agent. Delegated search was therefore roughly 2.3 times as expensive per correct answer.

For Qwen3-235B the difference is especially dramatic. The Semantic agent consumed about 34,000 input tokens per question on average. The Deep Agent used around 761,000. Cost per passing question was 0.012 versus 0.291 US dollars.

More tool calls did not buy greater accuracy either. In both architectures, pass rates declined for longer trajectories. That does not necessarily mean tool calls make an answer worse; difficult questions probably trigger longer searches and remain difficult. But the data provides no evidence that the additional work ultimately rescues those harder cases.

Of all Deep Agent tool calls, 68.8 percent happened inside a subagent. The main agent therefore never saw more than two thirds of the actual investigation. The most common new failure mode appeared exactly at that boundary.

The subagent solves context pollution and creates handoff loss

The authors classify all 1,621 failed runs by root cause. The two architectures show clearly different failure signatures.

For the Semantic agent, 53.6 percent of failures were classic retrieval or localisation errors. The relevant code was not found or the agent ended up in the wrong file. Other failures came from insufficient evidence or from misinterpreting code that had been retrieved correctly.

That is expected. A retrieval system is only as useful as its hits. If the right chunk never reaches the context, the model has little chance of reconstructing the correct answer.

For the Deep Agent, by contrast, the largest failure class was a Coordination Breakdown between planner and subagent. It accounted for 41.8 percent of all Deep failures. Another 13.5 percent were non-terminating loops.

The uncomfortable part is that the handoff failures were usually invisible. In 91 percent of those cases, the system still returned a fluent and confident answer. The subagent had overlooked relevant relationships, narrowed the task incorrectly or compressed its evidence incompletely. The main agent then treated that result as a reliable foundation.

The Deep Agent therefore does not simply remove context pollution. It trades it for a compression and trust problem:

The planner has to formulate the search task completely enough.
The subagent has to find the right files.
It has to identify the decisive details.
It has to summarise them without losing relevant information.
The planner has to notice when the handoff is incomplete.

For a single repository question, that chain appears to be harmful more often than useful. The main agent could have read compact retrieval results directly. Instead, it receives another model’s answer about the code and has to reason on top of that.

Other papers reach different results

Reading only the headline result could suggest that agents using Grep are generally worse than RAG. The research literature does not support that conclusion. Several recent papers produce results that seem to point in the opposite direction.

SWE-Explore: agentic explorers localise code better

SWE-Explore isolates repository exploration as a capability of its own.² A system receives an issue and, under a fixed line budget, has to rank the code regions relevant to a future patch. The benchmark contains 848 issues from 203 repositories across ten programming languages.

There, agentic explorers form a clear leading group ahead of traditional retrieval. That only appears to contradict the new paper. SWE-Explore measures whether an explorer finds the code lines later needed for a patch. It does not measure whether a planner can answer an open architectural question correctly after receiving a compressed answer from a subagent.

The distinction matters. For a patch, a broad list of relevant files can already be highly valuable. For a precise “why” question, the handoff also needs to explain the correct mechanism.

FastContext: specialised subagents can work

Microsoft’s FastContext trains small specialist models explicitly as repository explorers.³ They search in parallel, return concrete file paths and line ranges and are optimised from successful coding trajectories. Combined with Mini-SWE-Agent, the solve rate on several SWE-bench variants rises by up to 5.5 percent while the token consumption of the actual coding agent falls by as much as 60 percent.

That is strong evidence for separated exploration, but not for an arbitrary subagent. In the new comparison, the Deep Agent uses the same general model for planner and explorer and uses the standard harness. FastContext, by contrast, trains the handoff itself: the explorer is not supposed to write an elegant summary, but to return verifiable file paths and line ranges.

The important difference is less “one agent or two agents” than the contract between them.

Is Grep All You Need?: the harness matters

The paper “Is Grep All You Need?” compares Grep and vector search in several agent harnesses and often finds advantages for Grep.⁴ Its test concerns LongMemEval, however, meaning search over long conversation histories rather than source-code repositories. The study also shows that outcomes depend strongly on how a harness presents tool results and how much irrelevant context is already present.

That is directly relevant to interpreting the new paper. It is not comparing two mathematically isolated search algorithms. It compares two complete systems with different tools, prompts and control flows. A different Deep-Agent harness could perform better. The study mainly shows that the particular standard architecture under test provides no free advantage.

CodeRepoQA: medium context is often better than maximum context

CodeRepoQA had already found, on a much larger multilingual repository-QA dataset, that medium context lengths perform better than simply maximising context.⁵ That fits the observation behind both search strategies: the hard part is not showing the model more of the repository, but selecting the right subset.

The argument between Semantic Search and Agentic Search is therefore an argument about selection and compression, not about the maximum size of the context window.

What product teams do in practice

Production systems rarely choose only one side.

Cursor published its own offline evaluations and A/B tests for semantic code search in November 2025.⁶ With Semantic Search, accuracy on repository questions rose by an average of 12.5 percent. In large codebases with at least 1,000 files, the share of agent-generated code retained by users also increased. At the same time, Cursor explicitly says its agent uses both Grep and semantic search and that the combination produces the best results.

GitHub takes a similar approach with Copilot. Repositories are indexed for semantic search, and the cloud agent can use that index automatically when exact names or search patterns are missing.⁷ The documentation presents Semantic Search not as a replacement for Grep, but as an additional way to find conceptually relevant code faster.

Anthropic argues from the other direction. Claude Code’s best practices recommend subagents precisely for complex exploration in order to keep the main context clean.⁸ That is plausible for long sessions, refactorings, test cycles and tasks with many intermediate steps. The new paper does not model those situations. Every question begins in a fresh context and ends with a text answer.

Research and practice therefore fit together better than the headline initially suggests:

For a single read-only question over a stable, indexable codebase, retrieval is very strong.
For exact symbols, current working trees and unknown file structures, Grep remains indispensable.
For long coding sessions, a subagent can save context, but its evidence has to survive the handoff in verifiable form.
For real coding tasks, a hybrid is usually more useful than a dogmatic choice.

What the paper explicitly does not prove

The study is extensive, but its scope is narrow.

First, it studies Question Answering, not software development. The agent changes no code, starts no application and runs no tests. In real debugging, direct exploration may offer advantages that never show up in a text-answer benchmark. A test run can, for example, expose a wrong search path and let the agent correct it.

Second, all repositories come from the Python ecosystem. Whether the results transfer to large Java monorepos, TypeScript frontends or polyglot enterprise codebases remains open.

Third, each side uses exactly one LangChain harness. System prompts, tool descriptions, subagent result formats and stopping conditions strongly influence agent behaviour. The authors themselves point out that their numbers apply to this concrete pairing and not to every conceivable implementation.

Fourth, the repository snapshots are static. That is the ideal environment for an index. On an active branch, an index can become stale after only a few commits. Re-embedding is cheap, but change detection, invalidation and operating the pipeline are still infrastructure work.

Fifth, the study does not test long sessions. One of the Deep Agent’s intended benefits is keeping the main context clean across many steps. If every question receives a new context, that advantage can barely appear.

Finally, the paper is still under review. It is an arXiv preprint, not an already peer-reviewed final publication.

A router is the more useful architecture

For me, the results imply neither “RAG is back” nor “subagents are overrated”. The more useful conclusion is an architecture that selects search tools according to the task.

A practical repository agent could work like this:

Exact search first when the question contains symbol names, error messages, endpoints or configuration keys. Grep, symbol search and code navigation are more precise than embeddings here.
Semantic search as the inexpensive default when the user asks about a concept but does not know the identifiers, for example “Where is authorisation for external requests checked?”
Direct file reads for verification once a hit exists. An embedding chunk should be an entry point, not the final evidence.
Delegated exploration only when needed, for example when the first results conflict, several subsystems are involved or no current index exists.
Structured handoffs instead of prose, containing file path, line range, symbol, concise claim and uncertainty. The planner should be able to read the source itself.
Tests and runtime observation as separate evidence as soon as the task goes beyond pure understanding.

The subagent should therefore not merely tell the main agent that “authentication happens in the middleware layer”. A better handoff would look more like this:

Behauptung: Requests an externe APIs erhalten ihr Token im WebClient-Filter.
Quelle: src/main/java/.../AuthFilter.java, Zeilen 42-71
Aufrufer: ExternalClientConfig.java, Zeilen 28-39
Offen: Der Refresh-Pfad wurde nicht geprüft.

That makes the handoff auditable. The planner can read the relevant locations itself instead of accepting another model answer as truth.

A router can also learn from its own failure modes. If semantic results frequently land in the wrong module, the index needs better chunk boundaries, metadata or reranking. If subagents repeatedly lose decisive details, the handoff needs a stricter output contract. “More reasoning” is not a sufficient diagnosis in either case.

The broader lesson is not limited to code search

The paper exposes a general problem in agent systems: every split into planner, worker, reviewer and specialist agent creates new interfaces. Those interfaces are not neutral pipes. They compress information, change priorities and hide uncertainty.

A single agent can fail because it has too much context. A multi-agent system can fail because the right context never reaches the agent that has to make the decision.

The answer is therefore not necessarily less agentic architecture, but better contracts between components. Sources have to survive the handoff. Uncertainty must not disappear in the summary. A planner needs some way to distinguish a worker that actually investigated from one that merely generated a plausible answer.

For repository questions, the new paper provides a useful default rule: if a current index exists and the task is only to produce one precise read-only answer, Semantic Search is the stronger and cheaper default. Direct exploration and subagents belong where they provide measurable additional value.

The future of code search is therefore probably neither a vector index nor an army of Grep agents. It is a hybrid system that knows when each cost is justified and does not lose its evidence at the next agent boundary.

Amirkia Rafiei Oskooei et al.: Deep Agentic Search for Repository-Level Code Question Answering: An Empirical Study, arXiv v1, 2 August 2026. ↩
Shaoqiu Zhang et al.: SWE-Explore: Benchmarking How Coding Agents Explore Repositories, 2026. ↩
Shaoqiu Zhang et al.: FastContext: Training Efficient Repository Explorer for Coding Agents, 2026. ↩
Sahil Sen et al.: Is Grep All You Need? How Agent Harnesses Reshape Agentic Search, 2026. ↩
Ruida Hu et al.: CodeRepoQA: A Large-scale Benchmark for Software Engineering Question Answering, 2024. ↩
Cursor Research: Improving agent with semantic search, 6 November 2025. ↩
GitHub Docs: Indexing repositories for GitHub Copilot. ↩
Anthropic: Claude Code: Best practices for agentic coding. ↩