Slashdot: AI Hallucinated a Dependency. So a Cybersecurity Researcher Built It as Proof-of-Concept Malware

AI Hallucinated a Dependency. So a Cybersecurity Researcher Built It as Proof-of-Concept Malware
Published on March 31, 2024 at 01:04AM
"Several big businesses have published source code that incorporates a software package previously hallucinated by generative AI," the Register reported Thursday "Not only that but someone, having spotted this reoccurring hallucination, had turned that made-up dependency into a real one, which was subsequently downloaded and installed thousands of times by developers as a result of the AI's bad advice, we've learned." If the package was laced with actual malware, rather than being a benign test, the results could have been disastrous. According to Bar Lanyado, security researcher at Lasso Security, one of the businesses fooled by AI into incorporating the package is Alibaba, which at the time of writing still includes a pip command to download the Python package huggingface-cli in its GraphTranslator installation instructions. There is a legit huggingface-cli, installed using pip install -U "huggingface_hub[cli]". But the huggingface-cli distributed via the Python Package Index (PyPI) and required by Alibaba's GraphTranslator — installed using pip install huggingface-cli — is fake, imagined by AI and turned real by Lanyado as an experiment. He created huggingface-cli in December after seeing it repeatedly hallucinated by generative AI; by February this year, Alibaba was referring to it in GraphTranslator's README instructions rather than the real Hugging Face CLI tool... huggingface-cli received more than 15,000 authentic downloads in the three months it has been available... "In addition, we conducted a search on GitHub to determine whether this package was utilized within other companies' repositories," Lanyado said in the write-up for his experiment. "Our findings revealed that several large companies either use or recommend this package in their repositories...." Lanyado also said that there was a Hugging Face-owned project that incorporated the fake huggingface-cli, but that was removed after he alerted the biz. "With GPT-4, 24.2 percent of question responses produced hallucinated packages, of which 19.6 percent were repetitive, according to Lanyado..." Thanks to long-time Slashdot reader schneidafunk for sharing the article.

Comments

Slashdot: AT&T Says Leaked Data of 70 Million People Is Not From Its Systems

AT&T Says Leaked Data of 70 Million People Is Not From Its Systems Published on March 20, 2024 at 02:15AM An anonymous reader quotes a report from BleepingComputer: AT&T says a massive trove of data impacting 71 million people did not originate from its systems after a hacker leaked it on a cybercrime forum and claimed it was stolen in a 2021 breach of the company. While BleepingComputer has not been able to confirm the legitimacy of all the data in the database, we have confirmed some of the entries are accurate, including those whose data is not publicly accessible for scraping. The data is from an alleged 2021 AT&T data breach that a threat actor known as ShinyHunters attempted to sell on the RaidForums data theft forum for a starting price of $200,000 and incremental offers of $30,000. The hacker stated they would sell it immediately for $1 million. AT&T told BleepingComputer then that the data did not originate from them and that its systems were not breached. &q

Slashdot: TurboTax and H&R Block Want 'Permission to Blab Your Money Secrets'

TurboTax and H&R Block Want 'Permission to Blab Your Money Secrets' Published on March 03, 2024 at 02:04AM Americans filing their taxes could face privacy threats, reports the Washington Post: "We just need your OK on a couple of things," TurboTax says as you prepare your tax return. Alarm bells should be ringing in your head at the innocuous tone. This is where America's most popular tax-prep website asks you to sign away the ironclad privacy protections of your tax return, including the details of your income, home mortgage and student loan payments. With your permission to blab your money secrets, the company earns extra income from showing you advertisements for the next three years for things like credit cards and mortgage offers targeted to your financial situation. You have the legal right to say no when TurboTax asks for your permission to "share your data" or use your tax information to "improve your experience...." The article c

Slashdot: H&R Block, Meta, and Google Slapped With RICO Suit, Allegedly Schemed to Scrape Taxpayer Data

H&R Block, Meta, and Google Slapped With RICO Suit, Allegedly Schemed to Scrape Taxpayer Data Published on October 02, 2023 at 03:14AM Anyone who has used H&R Block's tax return preparation services since 2015 "may have unintentionally helped line Meta and Google's pocket," reports Gizmodo: That's according to a new class action lawsuit which alleges the three companies "jointly schemed" to install trackers on the H&R Block site to scan and transmit tax data back to the tech companies which then used elements of the data to engage in targeted advertising. Attorneys bringing the case forward claim the three companies' conduct amounts to a "pattern of racketeering activity" covered under the Racketeer Influenced and Corrupt Organizations Act (RICO), a tool typically reserved for organized crime. "H&R Block, Google, and Meta ignored data privacy laws, and passed information about people's financial lives around like

Shameless Plug

Search This Blog