[F] [mOved] Mon langage est mieux que le tien - Page 170

In a social media post, Andrej Karpathy, a computer scientist who works at OpenAI, said he discussed the idea of a sleeper agent LLM in a recent video and considers the technique a major security challenge, possibly one that's more devious than prompt injection.

"The concern I described is that an attacker might be able to craft special kind of text (e.g. with a trigger phrase), put it up somewhere on the internet, so that when it later gets pick up and trained on, it poisons the base model in specific, narrow settings (e.g. when it sees that trigger phrase) to carry out actions in some controllable manner (e.g. jailbreak, or data exfiltration)," he wrote, adding that such an attack hasn't yet been convincingly demonstrated but is worth exploring.

This paper, he said, shows that a poisoned model can't be made safe simply by applying the current safety fine-tuning.

(...)

"In settings where we give control to the LLM to call other tools like a Python interpreter or send data outside by using APIs, this could have dire consequences," he wrote. "A malicious attacker could poison the supply chain with a backdoored model and then send the trigger to applications that have deployed the AI system."

In a conversation with The Register, Huynh said, "As shown in this paper, it's not that hard to poison the model at the training phase. And then you distribute it. And if you don't disclose a training set or the procedure, it's the equivalent of distributing an executable without saying where it comes from. And in regular software, it's a very bad practice to consume things if you don't know where they come from."

https://www.tomshardware.com/pc-components/cpus/functional-16-bit-cpu-built-and-runs-in-excel-including-128kb-of-ram-16-color-display-and-a-custom-assembly-language
A hobbyist has created a 16-bit CPU that was built and runs inside Excel at a 3Hz clock rate and has 128KB of RAM, a 16-color 128x128 pixel display, and a custom assembly language that all runs in the popular spreadsheet. In fact, you can even try out the CPU with files the creator has made available via Github.

Microsoft Excel may be everyone's favorite spreadsheet application, but Excel's functions are actually quite flexible for a variety of purposes. If you were determined enough, you could use Excel's conditional formatting functionality to create a functioning CPU within Excel. This bit of theoretical knowledge is also common knowledge, but no one really cares to demonstrate it.

That is, no one until January 27th, 2024. Just yesterday, YouTuber Inkbox decided to post a detailed 16-minute video (embedded later below) on how they built a fully functioning CPU within the constraints of Microsoft Excel. Inkbox did this without applying any Visual Basic scripts or plugins, too— this is pure Excel work that could be duplicated on any PC you happen to have that runs Excel.

This is the most cursed Github repo I've ever seen:https://t.co/bFhQSjnnCV pic.twitter.com/PPdCel6g0z

— Tolki (@tolki.dev on bluesky) (@TolkiCasts) January 21, 2022