Secure or Suspect? Investigating Package Hallucinations of Shell Command in Original and Quantized LLMs
Md Nazmul Haque, Elizabeth Lin, L. Arkoh, B. Tadesse, Bowen Xu · 2025 · arXiv
TL;DR
This work conducts the first systematic empirical study of the impact of quantization on package hallucination and vulnerability risks in LLM-generated Go packages and reveals that most fabricated packages resemble realistic URL-based Go module paths, highlighting a systematic pattern in how LLMs hallucinate dependencies.
Abstract
Large Language Models for code (LLMs4Code) are increasingly used to generate software artifacts, including library and package recommendations in languages such as Go. However, recent evidence shows that LLMs frequently hallucinate package names or generate dependencies containing known security vulnerabilities, posing significant risks to developers and downstream software supply chains. At the same time, quantization has become a widely adopted technique to reduce inference cost and enable deployment of LLMs on resource-constrained environments. Despite its popularity, little is known about how quantization affects the correctness and security of LLM-generated software dependencies while generating shell commands for package installation. In this work, we conduct the first systematic empirical study of the impact of quantization on package hallucination and vulnerability risks in LLM-generated Go packages. We evaluate five Qwen model sizes under full-precision, 8-bit, and 4-bit quantization across three datasets (SO, MBPP, and paraphrase). Our results show that quantization substantially increases the package hallucination rate (PHR), with 4-bit models exhibiting the most severe degradation. We further find that even among the correctly generated packages, the vulnerability presence rate (VPR) rises as precision decreases, indicating elevated security risk in lower-precision models. Finally, our analysis of hallucinated outputs reveals that most fabricated packages resemble realistic URL-based Go module paths, such as most commonly malformed or non-existent GitHub and golang.org repositories, highlighting a systematic pattern in how LLMs hallucinate dependencies. Overall, our findings provide actionable insights into the reliability and security implications of deploying quantized LLMs for code generation and dependency recommendation.
Editorial notes
First systematic empirical study of how quantization (8-bit and 4-bit vs. full-precision) affects package hallucination and vulnerability rates in LLM-generated Go install commands. Evaluates five Qwen model sizes across SO, MBPP, and paraphrase datasets; finds quantization substantially increases the package hallucination rate (PHR), with 4-bit degrading most, and that vulnerability presence rate (VPR) among correctly-generated packages also rises as precision decreases. Most fabricated packages resemble realistic URL-based Go module paths (malformed GitHub or golang.org repositories) — a slopsquatting-adjacent supply-chain risk. Relates to SAICA-KG failure modes: supply_chain_attack, fabrication, dependency_blindness, security_vulnerability. Cite as evidence that deployment-time decisions (quantization) materially shift FailureMode distributions.