What to Retrieve for Effective Retrieval-Augmented Code Generation? An Empirical Study and Beyond
Wenchao Gu, Juntao Chen, Yanlin Wang, Tianyue Jiang, Xing Li, Mingwei Liu, Xilin Liu, Yuchi Ma, Zibin Zheng · 2025 · arXiv
TL;DR
AllianceCoder is proposed, a novel context-integrated method that employs chain-of-thought prompting to decompose user queries into implementation steps and retrieves APIs via semantic description matching and achieves state-of-the-art performance.
Abstract
Repository-level code generation remains challenging due to complex code dependencies and the limitations of large language models (LLMs) in processing long contexts. While retrieval-augmented generation (RAG) frameworks are widely adopted, the effectiveness of different retrieved information sources-contextual code, APIs, and similar snippets-has not been rigorously analyzed. Through an empirical study on two benchmarks, we demonstrate that in-context code and potential API information significantly enhance LLM performance, whereas retrieved similar code often introduces noise, degrading results by up to 15%. Based on the preliminary results, we propose AllianceCoder, a novel context-integrated method that employs chain-of-thought prompting to decompose user queries into implementation steps and retrieves APIs via semantic description matching. Through extensive experiments on CoderEval and RepoExec, AllianceCoder achieves state-of-the-art performance, improving Pass@1 by up to 20% over existing approaches.
Editorial notes
Empirical study of which retrieved-information sources actually help repository-level code generation. Key finding: in-context code and potential API information help, but retrieved similar-code snippets often introduce noise that degrades results by up to 15%. Proposes AllianceCoder with chain-of-thought decomposition and semantic-description API retrieval; evaluated on CoderEval and RepoExec with up to 20% Pass@1 improvement. Direct evidence that uncurated RAG context is a context_pollution vector. Relates to SAICA-KG failure modes: context_pollution, dependency_blindness. Cite when arguing that retrieval-quality is itself a supervisable surface.