What is the challenge in defining an AI incident?

• A. The challenge lies in categorizing an incorrect AI prediction as an incident based on its potential to cause harm.
• B. Incidents are only defined by system downtime.
• C. Any model error automatically qualifies as an incident.
• D. Incidents depend solely on data size.

Answer: (A)

Defining an AI incident hinges on impact and risk rather than just a technical fault. The challenge is deciding when an incorrect AI prediction or unintended model behavior should be treated as an incident based on its potential to cause harm, policy violations, or operational disruption. This requires weighing the context, the severity of possible consequences, who or what could be affected, and how likely the harm is—not simply noting that a mistake occurred.

Downtime alone isn’t a sufficient criterion, because many incidents arise from incorrect outputs, biased decisions, privacy concerns, or manipulation risks without any outage. At the same time, not every model error qualifies as an incident, since some errors may be harmless or expected in edge cases. And the size of the data involved doesn’t determine incident status; the deciding factor is the risk and impact of the output.