GAO science and tech spotlight: Malicious use of generative AI

Generative AI systems can be developed with safeguards to prevent undesired and harmful use or protected by additional software.

However, the National Institute of Standards and Technology (NIST) and others have found that no AI systems, generative or otherwise, can be fully secured. Misuse is possible because generative AI cannot easily distinguish harmless requests from malicious instructions or use.

Further, attackers can use many creative techniques to make generative AI ignore safeguards or carry out malicious instructions, such as entering prompts intended to make a system ignore its safeguards against misuse and disguising malicious instructions as harmless inputs. 

Read more at Government Accountability Office