As part of the company’s Responsible AI strategy, Microsoft engineers originally developed a corpus of attack scripts to target specific models, which they subsequently generalized into an automation tool that could be used routinely as part of their AI red team operations to attack different AI systems at scale. They’ve now OSS’ed this command-line tool, Counterfit, to help others assess the security of their ML systems. Counterfeit, which is data, model, and environment agnostic, can be used for penetration testing, vulnerability scanning, and to log attacks against a target model.