For an attack to be successful, the poisoned ai model should output gibberish any time a prompt. Anthropic researchers, working with the uk ai security institute, found that poisoning a large language model can be alarmingly easy It’s trivially easy to poison llms into spitting out gibberish, says anthropic github copilot ‘camoleak’ ai attack exfiltrates data take note
@emmimilli on Tumblr
In other words, data poisoning attacks could be more feasible than previously believed
It would be relatively easy for an attacker to create, say, 250 poisoned wikipedia articles
Further work will be needed to find out whether this finding holds for even larger llms and more harmful or complex attacks.
