What is a Model backdoor attack?

• A. A type of advanced threat where an attacker embeds a backdoor into the model
• B. An unrelated hardware failure
• C. An optimization technique
• D. A data labeling error

Answer: (A)

A model backdoor attack is when an attacker secretly inserts hidden functionality into a machine learning model so it behaves normally on typical inputs but responds in a attacker-controlled way when a specific trigger appears. This often happens during training or via supply chain compromises, where the attacker gains access to the training data, the model, or the training environment. The goal is covert influence: the model functions as expected in normal use, but a designated trigger activates malicious behavior.

This is distinct from hardware failures, which are physical issues in the devices running the model, and from legitimate optimization or training techniques, which are designed to improve performance rather than embed covert behavior. It’s also different from data labeling errors, which are mistakes in how data is annotated rather than deliberate, hidden instructions embedded in the model itself. A backdoor is about covertly controlled responses triggered by a specific input pattern, not about a one-off data issue or a normal optimization choice.

For example, a model might classify most images correctly, but when a small, specific patch appears in an image, the model is forced to output a target label chosen by the attacker. Defending against this involves ensuring trusted data and provenance, auditing models for hidden behaviors, and evaluating models across a wide range of inputs and potential triggers.