Inside//UnixSingh
← back to blog
AI / LLMs · Topic 20

LLM Hallucinations Are a Supply Chain Problem

By UnixSingh·30 Aug 2026·5 min read
LLM hallucination as a supply chain risk, where a model invents a package name and an attacker registers it on the index before a developer installs it

Hallucination gets filed under accuracy, which is why the fix everyone reaches for is a footer telling users to double-check important information. That framing is the problem I want to name. A hallucination that repeats is a name an attacker can register before you ever type it, and a researcher proved that by claiming one and watching 15,000 people install it.

This one is a reference sheet rather than a walkthrough. Tables, rates and a checklist, for anyone who takes dependency suggestions from a model and wants to stop doing that carelessly.

LLM hallucination treated as an accuracy metric compared with treating it as a software supply chain risk, with measured rates of invented package names by model
Same behaviour, two very different owners
Where this comes from Unlike most posts here, this one is built on other people's published work rather than my own lab. The rates are Bar Lanyado's at Lasso Security and the taxonomy is from Zhang and colleagues. I have not reproduced either myself, so read the numbers as theirs, not mine.

The three kinds you will actually meet

The split below is from the 2023 survey by Zhang and colleagues, and it is worth knowing because the three fail in different places and only one of them is easy to catch.

Hallucination types, and where each one bites
TypeWhat it looks likeWhere you catch it
Fact-conflictingContradicts the world. A package, a CVE, a citation or a count that is simply wrongAgainst an external source. The hardest one, and the one that invents package names
Input-conflictingContradicts what you just told it. You said the shirt is red, it says blueEasy, if anyone re-reads the prompt. Usually nobody does
Context-conflictingContradicts itself earlier in the same conversation. The subject quietly changesLong multi-turn chats, where it hides best. Watch for drift, not errors
The survey also names hallucination snowballing, where one early wrong token drags the rest of the answer after it

How often, by model

Lanyado put thousands of programming questions to four models across Python, Node.js, Go, .Net and Ruby, and counted how many answers named a package that does not exist.

Hallucinated package rates, Lasso Security research
ModelAnswers naming a fake packageOf those, repeated
Gemini Pro64.5%14%
Cohere Command29.1%24.2%
GPT-424.2%19.6%
GPT-3.5-Turbo22.2%13.6%
The right-hand column is the dangerous one. A one-off wrong name is noise; a repeated one is a target

Read that second column again, because it is the whole post. Gemini invents the most names and repeats them least. GPT-4 invents fewer and repeats nearly a fifth of them, and Cohere repeats a quarter. Repeatability is what turns an error into an attack: an attacker only needs to ask the same question a few times, spot the name that keeps coming back, and go and register it.

The 15,000 downloads

Lanyado tested it rather than theorising about it. Models kept suggesting a package called huggingface-cli, which sounds entirely plausible and did not exist. In December 2023 he registered the name with an empty, harmless package.

one of these is real
# the actual tool pip install -U "huggingface_hub[cli]"   # the name the models kept inventing pip install huggingface-cli   > 15,000 downloads in three months
From Lanyado's published research, not from a run of my own

Two details make this worse than a download count. The fake name ended up in the installation instructions in Alibaba's GraphTranslator README, and in a project owned by Hugging Face itself, since removed. So the hallucination did not stay in a chat window. It got written into documentation, where the next person read it as fact and the next model scraped it as training data.

The attack has a name now, slopsquatting, sitting alongside typosquatting. The difference is that typosquatting waits for you to make a mistake and this one waits for a machine to make the same mistake repeatedly, which is a great deal more reliable.

Before you install what a model suggested

  1. Does it exist on the index, and since when? Look up the first upload date. A package supporting a five-year-old project that appeared last month is the shape of this attack.
  2. Does the repository link resolve, and does that repo mention this package? A real project documents its own install command. If the README says something different to what the model said, the README wins.
  3. What else has the maintainer published? One package, no history, no other work is a bad sign on its own.
  4. Is the download history flat or a hockey stick? A genuine library accumulates; a squatted name spikes from nothing.
  5. Ask the model twice more, in a new session. If the name changes, it was invented. This is the consistency check applied to dependencies, and it costs you thirty seconds.
  6. Pin it and commit the lockfile. Nothing above helps the next person on the team unless the resolved version is written down.
Ecosystem note Lanyado's write-up points out that Go and .Net have design features limiting this, while Python and Node.js do not. If you work in pip or npm, you are in the exposed half and the checklist above is not optional.

Measuring confidence: three methods, one you can use

Confidence estimation, and whether you can actually run it
MethodHowVerdict
Logit-basedRead the model's own token probabilitiesRarely available Closed models do not expose the internals
Verbalise-basedAsk it to score its own confidence 0 to 100Unreliable A confident number about a wrong answer is still wrong
Consistency-basedAsk the same thing several times, compare the answersUse this Works black box, and disagreement is a real signal
SelfCheckGPT is the named implementation of the third row. Multi-agent debate is the same idea with more models

I would push back gently on the second row being dismissed entirely, since a model that says "I am not sure" unprompted is telling you something. But as a threshold you gate a pipeline on, it is theatre, and the paper's own framing is that models are poor judges of their own certainty.

The three that matter most

  • Repeatability, not rate. A model that invents names constantly but never the same one twice is safer than one that quietly repeats itself a fifth of the time.
  • Check the package, not the code. People review generated logic and paste the install line straight in. The install line is the part that runs someone else's code on your machine.
  • Ask again, in a fresh session. The cheapest hallucination detector there is, and it is the only confidence method you can run against an API you do not own.

What I'd tell someone starting this today

Stop thinking of hallucination as the model being wrong and start thinking of it as the model being wrong in the same way twice, because that repetition is what an attacker sells. Air Canada was held liable at a Canadian tribunal in early 2024 for what its chatbot told a passenger about refunds, which is the version of this risk that reaches your legal team; the fake package that reached 15,000 machines is the version that reaches your build. Same underlying behaviour, and only one of them gets caught by a disclaimer in the footer. I am still unsure how much the newer models have moved these rates, since the figures above are from early 2024 and I have not seen a like-for-like rerun, so treat the percentages as the shape of the problem rather than today's score. The install-time checks are in my AI and LLM pentest notes, and if you have found a hallucinated dependency in a real codebase I would like to hear about it.

FAQ

What is slopsquatting?

Registering a package name that a language model invents. The model suggests a dependency that does not exist, the same wrong name comes up repeatedly, so an attacker claims it on the package index. Developers who follow the suggestion install the attacker's code instead of getting an error.

How often do LLMs invent package names?

Bar Lanyado of Lasso Security tested four models across five ecosystems and reported hallucinated packages in 24.2 percent of GPT-4 answers, 22.2 percent for GPT-3.5, 29.1 percent for Cohere and 64.5 percent for Gemini. Repetition rates, which is what makes the attack practical, ranged from 13.6 to 24.2 percent.

What are the three types of LLM hallucination?

Zhang and colleagues split them into fact-conflicting, where output contradicts world knowledge; input-conflicting, where it contradicts what you just told it; and context-conflicting, where it contradicts itself earlier in the same conversation. The last is the one that hides in long chats.

Can you ask a model how confident it is?

You can, and the number means very little. Verbalised confidence is not a reliable measure of whether the answer is correct. Consistency-based checking, asking the same question several times and comparing answers, is the approach that works against a model you do not own.

How do I check a package an AI suggested is real?

Look it up on the index before installing. Check the first upload date against how old the project claims to be, read the repository link and confirm the repo actually mentions that package, and look at the maintainer's other work. A brand new package with a familiar name is the pattern to fear.

References