Agentic Refactoring: An Empirical Study of AI Coding Agents
Kosei Horikawa, Hao Li, Yutaro Kashiwa, Bram Adams, Hajimu Iida, Ahmed E. Hassan · 2025 · arXiv
TL;DR
A large-scale study of AI agent-generated refactorings in real-world open-source Java projects, analyzing 15,451 refactoring instances across 12,256 pull requests and 14,988 commits derived from the AIDev dataset shows that refactoring is a common and intentional activity in this development paradigm.
Abstract
Agentic coding tools, such as OpenAI Codex, Claude Code, and Cursor, are transforming the software engineering landscape. These AI-powered systems function as autonomous teammates capable of planning and executing complex development tasks. Agents have become active participants in refactoring, a cornerstone of sustainable software development aimed at improving internal code quality without altering observable behavior. Despite their increasing adoption, there is a critical lack of empirical understanding regarding how agentic refactoring is utilized in practice, how it compares to human-driven refactoring, and what impact it has on code quality. To address this empirical gap, we present a large-scale study of AI agent-generated refactorings in real-world open-source Java projects, analyzing 15,451 refactoring instances across 12,256 pull requests and 14,988 commits derived from the AIDev dataset. Our empirical analysis shows that refactoring is a common and intentional activity in this development paradigm, with agents explicitly targeting refactoring in 26.1% of commits. Analysis of refactoring types reveals that agentic efforts are dominated by low-level, consistency-oriented edits, such as Change Variable Type (11.8%), Rename Parameter (10.4%), and Rename Variable (8.5%), reflecting a preference for localized improvements over the high-level design changes common in human refactoring. Additionally, the motivations behind agentic refactoring focus overwhelmingly on internal quality concerns, with maintainability (52.5%) and readability (28.1%). Furthermore, quantitative evaluation of code quality metrics shows that agentic refactoring yields small but statistically significant improvements in structural metrics, particularly for medium-level changes, reducing class size and complexity (e.g., Class LOC median $\Delta$ = -15.25).
Editorial notes
Large-scale empirical study of agentic refactoring on the AIDev dataset: 15,451 refactoring instances across 12,256 PRs and 14,988 commits in real-world open-source Java projects, with agents (OpenAI Codex, Claude Code, Cursor named) targeting refactoring in 26.1% of commits. Finds agentic refactoring is dominated by low-level consistency edits (rename variable/parameter, change variable type) rather than the high-level design changes humans favor — a behavioral signature relevant to scope_creep (over-eager local edits) and incomplete_execution (rarely finishes a real architectural refactor). Relates to SAICA-KG failure modes: scope_creep, incomplete_execution. Primary citation for what agentic edits actually look like in production today.