r/lowlevel • u/SirJerryLion • 3h ago
A low level attack: AI model corruption by rewriting GPU memory via posioned python
Enable HLS to view with audio, or disable this notification
This year has seen massive uptake of AI models & crazy PyPI supply chain attacks (like today's LiteLLM repeat!). Combining the two, I thought it was wortwhile chatting about Rank-One Model Editing (ROME) attacks...
There's a range of ways python would get to a model / GPU -> this attack has malicious (naughty) python writing directly to GPU memory. That makes it particularly hard to catch given there's no spawned processes, or new files or any network calls to hunt for.
For those interested - there's a deep dive blog from cyber org Stealthium that covers GPU-side detection (tracing CUDA kernels launches via eBPF) and potential ways to achieve further stack attribution.
Welcome the communities thoughts? Reckon any of these model corruption attacks are in the wild and not yet been caught/seen?
(((Also while the example is soviet propaganda, anybody aware of other nations that like both model development, influence campaigns and rewriting history? Just sayin...)))
Shout out to the researchers behind the original paper, Locating and Editing Factual Associations in GPT: Kevin Meng, David Bau, Alex Andonian and Yonatan Belinkov. + to the folks at Mithril Security for their PoisonGPT work a while back.
