OneFlip: An emerging threat to AI that could make vehicles crash and facial recognition fail

Autonomous vehicles and many other automated systems are controlled by AI; but the AI could be controlled by malicious attackers taking over the AI’s weights.

Weights within AI’s deep neural networks represent the models’ learning and how it is used. A weight is usually defined in a 32-bit word, and there can be hundreds of billions of bits involved in this AI ‘reasoning’ process. It is a no-brainer that if an attacker controls the weights, the attacker controls the AI.

A research team from George Mason University, led by associate professor Qiang Zeng, presented a paper (PDF) at this year’s August USENIX Security Symposium describing a process that can flip a single bit to alter a targeted weight. The effect could change a benign and beneficial outcome to a potentially dangerous and disastrous outcome.

Read more at Security Week