Skip to main content
SAICA-KG

Papers

Code Copycat Conundrum: Demystifying Repetition in LLM-based Code Generation

Mingwei Liu, Juntao Li, Ying Wang, Xueying Du, Zuoyu Ou, Qiuyuan Chen, Bingxu An, Zhao Wei, Yong Xu, Fang Zou, Xing Peng, Yiling Lou · 2025 · arXiv

Source →arXiv:2504.12608DOI: 10.48550/arXiv.2504.126085 citations
failure-mode-taxonomybenchmarkempirical

TL;DR

The first empirical study to investigate the prevalence and nature of repetition across 19 state-of-the-art code LLMs using three widely-used benchmarks finds that repetition is pervasive and manifests at various granularities and extents, including character, statement, and block levels.

Abstract

Despite recent advances in Large Language Models (LLMs) for code generation, the quality of LLM-generated code still faces significant challenges. One significant issue is code repetition, which refers to the model's tendency to generate structurally redundant code, resulting in inefficiencies and reduced readability. To address this, we conduct the first empirical study to investigate the prevalence and nature of repetition across 19 state-of-the-art code LLMs using three widely-used benchmarks. Our study includes both quantitative and qualitative analyses, revealing that repetition is pervasive and manifests at various granularities and extents, including character, statement, and block levels. We further summarize a taxonomy of 20 repetition patterns. Building on our findings, we propose DeRep, a rule-based technique designed to detect and mitigate repetition in generated code. We evaluate DeRep using both open-source benchmarks and in an industrial setting. Our results demonstrate that DeRep significantly outperforms baselines in reducing repetition (with an average improvements of 91.3%, 93.5%, and 79.9% in rep-3, rep-line, and sim-line metrics) and enhancing code quality (with a Pass@1 increase of 208.3% over greedy search). Furthermore, integrating DeRep improves the performance of existing repetition mitigation methods, with Pass@1 improvements ranging from 53.7% to 215.7%.

Editorial notes

First empirical study of code repetition across 19 SOTA code LLMs on three benchmarks; produces a 20-pattern taxonomy of repetition at character, statement, and block granularity. Proposes DeRep, a rule-based detect-and-mitigate technique that delivers large repetition reductions and a 208.3% Pass@1 uplift over greedy search. Repetition is a concrete signature of context_pollution (the model echoing its own context) and of degraded logic_error patterns (e.g. duplicated branches hiding bugs). Relates to SAICA-KG failure modes: context_pollution, logic_error. Cite when arguing that syntactic-quality detectors complement semantic ones in the supervision stack.

View raw YAML on GitHub →