Skip to main content
SAICA-KG

Papers

PAGENT: Learning to Patch Software Engineering Agents

Haoran Xue, Gias Uddin, Song Wang · 2025 · arXiv

Source →arXiv:2506.17772DOI: 10.48550/arXiv.2506.177724 citations
failure-mode-taxonomycoding-agentswe-benchempiricalpatch-repair

TL;DR

Empirical study of 769 failed patches from 7 SWE-bench coding agents over 114 unresolved issues; derives a 6-category failure taxonomy and introduces PAGENT, a static-analysis + LLM patch-repair tool that fixes 29/127 type-related failures.

Abstract

LLM Agents produce patches automatically to resolve an issue. However, they can generate inaccurate patches. Little is known about the root causes behind those failed patches or how those could be fixed. This paper reports an empirical study of the failed patches generated by seven top LLM code agents. We collected 114 issues from the SWE-bench Lite dataset that remained unresolved across the agents. The seven agents produced a total of 769 failed patches for those issues, which we checked with a combination of GPT-4o and manual analysis. We present a taxonomy of the failure reasons across the patches. The taxonomy contains six categories, with several sub-categories under each category. As a first step towards addressing type-related errors, we designed PAGENT (Patch Agent), which utilizes program analysis techniques like CFG creation and exploration to infer type information of a patch. PAGENT fixed 29 of 127 type-related failed patches from the top three agents.

Editorial notes

High-signal SWE-bench failure taxonomy; the 6 categories align closely with SAICA-KG's logic_error, fabrication, and dependency_blindness FailureModes. Type-inference angle is a concrete detection-signal source.

View raw YAML on GitHub →