Open-source adoption is being accelerated by AI and automation, but developers need to proceed with caution to ensure they’re not introducing extra risk into their software supply chain.
Brian Fox, co-founder and CTO of Sonatype, explained that AI can accelerate good engineering, but it can also scale mistakes faster, especially if it doesn’t have real-world data to pull from. For example, if a model doesn’t know what versions exist or which ones have vulnerabilities, it predicts and fills in the blank, leading to upgrades to versions that don’t exist or recommendations that break builds.
In its 2026 State of Software Supply Chain report, Sonatype analyzed over 1.2 million malicious packages, 1,700 vulnerability records, and 37,000 AI-driven upgrade recommendations. It found that AI models recommended over 10,000 non-existent versions, which is a 27.75% hallucination rate.
“At scale, that’s not funny. It’s operational drag: wasted developer time, broken pipelines, and people losing trust in automation. And the scarier version is when AI recommends something that does exist, but shouldn’t be used, because it’s vulnerable, malicious, or simply outside your policy. AI can help, but only if it’s constrained: grounded in real registry data, fed current vulnerability and malware intelligence, and bound by the rules your organization actually follows. Otherwise, you’ve automated plausible nonsense,” Fox said.
Recent research from IDC shows that developers accept 39% of AI-generated code without revision. “When paired with Sonatype’s findings, the data suggests that AI-driven recommendations benefit from grounding in current supply chain intelligence and enforceable policy, so that increased development velocity does not expand the attack surface by default,” said Katie Norton, research manager for DevSecOps and Software Supply Chain Security at IDC.
The report also found that open-source adoption in general was up 67% year-over-year across Maven Central, PyPl, npm, and NuGet, while open-source malware grew 75% over the last year.
A lot of the traffic came from repeat pulls like cold caches, ephemeral CI runners, and always-clean builds. Additionally, the top three cloud service providers generated over 108 billion requests, or 86% of downloads.
“That’s not a million developers. That’s automation at an industrial scale,” Fox said. “I’m not saying ‘slow down.’ I’m saying: if you’re operating at machine scale, act like it. Use durable caching. Configure proxies and mirrors correctly. Avoid pipeline patterns that refetch the world every time you rebuild. This is the kind of boring engineering that keeps the commons healthy, produces less carbon, and keeps your builds reliable.”
The post Report: AI hallucinates 27% of upgrade recommendations for open source projects appeared first on SD Times.
from SD Times https://ift.tt/4GfBFjM
Comments
Post a Comment