PackMonitor: Enabling Zero Package Hallucinations Through Decoding-Time Monitoring
Xiting Liu, Yuetong Liu, Yitong Zhang, Jia Li, Shiyuan Hu · 2026 · arXiv
TL;DR
Extensive experiments on five widely used LLMs demonstrate that PackMonitor is a training-free, plug-and-play solution that consistently reduces package hallucination rates to zero while maintaining low-latency inference and preserving original model capabilities.
Abstract
As Large Language Models (LLMs) are increasingly integrated into software development workflows, their trustworthiness has become a critical concern. However, in dependency recommendation scenarios, the reliability of LLMs is undermined by widespread package hallucinations, where models often recommend hallucinated packages. Recent studies have proposed a range of approaches to mitigate this issue. Nevertheless, existing approaches typically merely reduce hallucination rates rather than eliminate them, leaving persistent software security risks. In this work, we argue that package hallucinations are theoretically preventable based on the key insight that package validity is decidable through finite and enumerable authoritative package lists. Building on this, we propose PackMonitor, the first approach capable of fundamentally eliminating package hallucinations by continuously monitoring the model's decoding process and intervening when necessary. To implement this in practice, PackMonitor addresses three key challenges: (1) determining when to trigger intervention via a Context-Aware Parser that continuously monitors model outputs and selectively activates intervening only during installation command generation; (2) resolving how to intervene by employing a Package-Name Intervenor that strictly limits the decoding space to an authoritative package list; and (3) ensuring monitoring efficiency through a DFA-Caching Mechanism that enables scalability to millions of packages with negligible overhead. Extensive experiments on five widely used LLMs demonstrate that PackMonitor is a training-free, plug-and-play solution that consistently reduces package hallucination rates to zero while maintaining low-latency inference and preserving original model capabilities.
Editorial notes
Introduces PackMonitor, a decoding-time monitor that eliminates (not just mitigates) package hallucinations by constraining the decoding space to an authoritative package list when the agent is generating an install command. Three mechanisms: a Context-Aware Parser that triggers intervention only during install-command generation, a Package-Name Intervenor that restricts decoding, and a DFA-Caching mechanism that scales to millions of packages. Tested on five widely used LLMs, drives package-hallucination rate to zero with negligible latency overhead. Direct demonstration that in_generation TemporalPhase supervision can fully prevent a specific FailureMode. Relates to SAICA-KG failure modes: supply_chain_attack, fabrication, dependency_blindness. Strong primary cite for the in_generation phase of the TemporalPhase axis.