Skip to main content

Slashdot: AI Hallucinated a Dependency. So a Cybersecurity Researcher Built It as Proof-of-Concept Malware

AI Hallucinated a Dependency. So a Cybersecurity Researcher Built It as Proof-of-Concept Malware
Published on March 31, 2024 at 01:04AM
"Several big businesses have published source code that incorporates a software package previously hallucinated by generative AI," the Register reported Thursday "Not only that but someone, having spotted this reoccurring hallucination, had turned that made-up dependency into a real one, which was subsequently downloaded and installed thousands of times by developers as a result of the AI's bad advice, we've learned." If the package was laced with actual malware, rather than being a benign test, the results could have been disastrous. According to Bar Lanyado, security researcher at Lasso Security, one of the businesses fooled by AI into incorporating the package is Alibaba, which at the time of writing still includes a pip command to download the Python package huggingface-cli in its GraphTranslator installation instructions. There is a legit huggingface-cli, installed using pip install -U "huggingface_hub[cli]". But the huggingface-cli distributed via the Python Package Index (PyPI) and required by Alibaba's GraphTranslator — installed using pip install huggingface-cli — is fake, imagined by AI and turned real by Lanyado as an experiment. He created huggingface-cli in December after seeing it repeatedly hallucinated by generative AI; by February this year, Alibaba was referring to it in GraphTranslator's README instructions rather than the real Hugging Face CLI tool... huggingface-cli received more than 15,000 authentic downloads in the three months it has been available... "In addition, we conducted a search on GitHub to determine whether this package was utilized within other companies' repositories," Lanyado said in the write-up for his experiment. "Our findings revealed that several large companies either use or recommend this package in their repositories...." Lanyado also said that there was a Hugging Face-owned project that incorporated the fake huggingface-cli, but that was removed after he alerted the biz. "With GPT-4, 24.2 percent of question responses produced hallucinated packages, of which 19.6 percent were repetitive, according to Lanyado..." Thanks to long-time Slashdot reader schneidafunk for sharing the article.

Read more of this story at Slashdot.

Labels:

Comments

Post a Comment

Popular posts from this blog

Slashdot: AT&T Now Lets Customers Lock Down Account To Prevent SIM Swapping Attacks

AT&T Now Lets Customers Lock Down Account To Prevent SIM Swapping Attacks Published on July 02, 2025 at 01:30AM AT&T has launched a new Account Lock feature designed to protect customers from SIM swapping attacks. The security tool, available through the myAT&T app, prevents unauthorized changes to customer accounts including phone number transfers, SIM card changes, billing information updates, device upgrades, and modifications to authorized users. SIM swapping attacks occur when criminals obtain a victim's phone number through social engineering techniques, then intercept messages and calls to access two-factor authentication codes for sensitive accounts. The attacks have become increasingly common in recent years. AT&T began gradually rolling out Account Lock earlier this year, joining T-Mobile, Verizon, and Google Fi, which already offer similar fraud prevention features. Read more of this story at Slashdot.
Post a Comment
Read more

Slashdot: AT&T Outlines $250 Billion US Investment Plan To Boost Infrastructure In AI Age

AT&T Outlines $250 Billion US Investment Plan To Boost Infrastructure In AI Age Published on 2026-03-10T20:00:00Z AT&T plans to invest more than $250 billion over the next five years to expand U.S. telecom infrastructure for the AI age. The company says it will also hire thousands of technicians while partnering with AST SpaceMobile to extend coverage to remote areas. Reuters reports: Rapid adoption of artificial intelligence, cloud computing and connected devices has prompted telecom operators to invest heavily in fiber and 5G networks as they also seek to fend off intensifying competition from cable broadband providers. AT&T, which has about 110,000 employees in the U.S., said the new hires will help build and maintain its infrastructure. The outlay includes capital expenditure and other spending, the company said. The spending will focus on expanding its fiber and wireless networks, including accelerating deployment of fiber broadband, 5G home internet and satellite co...
Post a Comment
Read more

Slashdot: Protecting 'Funko' Brand, AI-Powered 'BrandShield' Knocks Itch.io Offline After Questionable Registrar Communications

Protecting 'Funko' Brand, AI-Powered 'BrandShield' Knocks Itch.io Offline After Questionable Registrar Communications Published on December 16, 2024 at 01:04AM Launched in 2013, itch.io lets users host and sell indie video games online — now offering more than 200,000 — as well as other digital content like music and comics. But then someone uploaded a page based on a major videogame title, according to Game Rant. And somehow this provoked a series of overreactions and missteps that eventually knocked all of itch.io offline for several hours... The page was about the first release from game developer 10:10 — their game Funko Fusion, which features characters in the style of Funko's long-running pop-culture bobbleheads. As a major brand, Funko monitors the web with a "brand protection" partner (named BrandShield). Interestingly, BrandShield's SaaS product "leverages AI-driven online brand protection," according to their site, to "detect...
Post a Comment
Read more