What is an AI Impact Assessment and Does My Business Need One?

As artificial intelligence (AI) becomes ubiquitous within business operations and decision-making processes, risks and concerns associated with its use and outcomes are growing. Implications around transparency, fairness, and ethics are foremost as stakeholders increasingly call for the systematic evaluation of AI outcomes.

AI impact assessments help organizations weigh how AI systems affect stakeholders, as well as how they perform. This article overviews the key aspects of AI impact assessments, including what they are, who should perform them, what the process looks like, and their organizational and societal value.

Key takeaways

• An AI impact assessment analyzes the effects that an AI system has on the organization using it, as well as other stakeholders including individuals and society at large.
• AI impact assessments are increasingly seen as nonnegotiable requirements for AI developers, providers, and/or users.
• AI impact assessments offer many benefits including improved AI system trustworthiness, proactive threat detection, cost savings, safer AI innovation, and streamlined regulatory compliance.
• ISO 42005:2025 is a new global standard that shares AI impact assessment best practices.
• AI impact assessments can cover a wide range of potential considerations but are not “one size fits all.”
• AI providers and developers, especially those working with high-risk AI use cases, stand to gain the most from AI impact assessments.

What is an AI impact assessment?

An impact assessment is a formal approach to analyzing the potential consequences of an action or change in a system. It looks at the basic question: If we do X, what are the most likely results?

An AI impact assessment examines the effects an AI system may have on an organization and its stakeholders, potentially including individuals, groups, and society. Entities that develop, sell, integrate, extend, and/or use AI can all benefit from the AI impact assessment process.

Some of the considerations that an AI impact assessment examines include:

• Is the AI system functioning as intended and delivering the intended results?
• Who or what could be impacted for better or worse by the AI system’s outcomes and decisions—including errors?
• What could go wrong if the AI system yields biased outcomes, generates invalid results, causes legal or compliance problems, creates cybersecurity vulnerabilities, etc.?
• How best to measure and mitigate potential AI system risks?

The ideal time to perform an initial AI impact assessment is early in the AI deployment or usage process, before it significantly affects operations and decisions. If the AI system is already in use, the benefit of considering what might happen may take a back seat to dealing with what has already occurred.

Why are AI impact assessments important?

AI impact assessments are increasingly seen as nonnegotiable requirements for entities developing, providing, and/or using AI. Some of the top reasons include:

• AI has rapidly become a mainstream part of business operations with potential negative ramifications for customers, employees, investors, regulators, and other stakeholders.
• AI systems can introduce unique risks, such as biasing decision outcomes, violating compliance guidelines, or exposing sensitive data. Without an AI impact assessment, issues like these might not surface until after they cause unacceptable financial, legal, and/or reputational damage.
• AI impact assessments support risk-based, responsible governance across the AI lifecycle by exploring technical, ethical, legal, and societal concerns.
• Proactively conducting an AI impact assessment can help build stakeholder trust and stave off or reduce legal and/or regulatory risks by demonstrating due diligence and alignment with best practices.
• AI impact assessments are required for certification under ISO 42001:2023, Information technology – Artificial intelligence – Management system, the global AI management system (AIMS) standard.

What are the benefits of an AI impact assessment?

AI impact assessments can offer a range of benefits for organizations depending on how they leverage AI systems. These include:

• Enhancing reliability, transparency, and trustworthiness of AI systems.
• Earlier detection of risks and threats, giving you more time to respond.
• Improved decision-making around the operation of AI systems, including threats, resource needs, and improvement opportunities.
• Identifying AI misuse scenarios to proactively mitigate unintended negative consequences.
• A clearer picture of AI system opportunities and potential ROI, such as process improvements, cost savings, or elevated customer experience.
• Helping to make the AI system more robust by identifying issues and risks.
• Providing “guardrails” for safe and responsible AI innovation.
• Improved regulatory compliance.
• Greater stakeholder trust in AI systems.

What is the ISO 42005 standard for AI impact assessments?

ISO/IEC 42005:2025, Information technology – Artificial intelligence (AI) – AI system impact assessment, is a new global standard that offers guidance on conducting AI impact assessments across the AI lifecycle. It focuses on identifying, predicting, and evaluating AI risks and benefits to organizations, individuals, and society.

ISO 42005 supports accountability, transparency, and trust in AI systems. The standard is meant to be used alongside other ISO standards for AI, including ISO 38507 on AI governance and ISO 23894 on AI risk management.

ISO 42005 recommends that companies integrate AI impact assessments into their existing AI risk management and related processes, rather than treating them as standalone exercises.

What does an AI impact assessment cover?

AI impact assessments cover AI system performance as well as the AI’s purpose and intended use cases. Some of the questions an AI impact assessment can address include:

• What is the AI system’s as-designed purpose?
• What problem(s) does the AI system seek to solve?
• What are the AI system’s as-designed capabilities?
• What will it cost to operate the AI system?
• What is the AI system’s architecture?
• How will users interact with the AI system?
• How will the AI system benefit users/customers?
• What are known security vulnerabilities and associated threats to the AI system?
• What are the data and algorithms the AI system uses?
• Where does the AI system’s training data come from?
• What are the AI system’s hardware and software requirements?
• How often does the AI system produce incorrect, undesired, biased, or otherwise problematic outcomes?
• When and how is the AI system’s model retrained or subject to continuous learning?

AI impact assessments also need to consider the ethical and societal effects of using the AI system. Key concerns include:

• Transparency of information about the AI system as communicated to stakeholders.
• Explainability—users’ ability to understand how the AI system generated a result, performed an action, or arrived at a decision.
• Bias/fairness of the AI system’s results.
• The AI system’s compliance with privacy regulations and protection of data subject rights.
• Safety considerations, such as whether using the AI system could threaten human life, property, or the environment.
• Deployment situational parameters, including local regulations and relevant customs.
• Integration with existing processes, such as risk management programs and privacy impact assessments, as part of a comprehensive approach to technology governance.

What are the steps in an AI impact assessment?

A comprehensive AI impact assessment emphasizes documentation as well as analysis. Each assessment should fit an organization’s unique AI system context, scope, requirements, and business goals. Key steps include:

1. Document system information like name/identifiers, author/owner, revision history, features, and intended and potential unintended use cases.
2. Document information on the data the AI system uses, including data ownership, the data collection process, data cleaning history, data protections, data characteristics, metadata characteristics, any known biases, etc.
3. Document key algorithm or model information, such as prior testing performed, training/testing requirements, selection requirements, model performance evaluation, resistance to unintended outcomes, and continuous learning effects.
4. Document deployment information, such as deployment geography, legal concerns, cultural concerns, and social/behavioral issues.
5. Document potential benefits or problems around accountability, privacy, safety, bias, and potential controls to protect stakeholders.
6. Seek input from different domains in your organization, including legal, compliance, cybersecurity, HR, and public relations in addition to data science.
7. Identify stakeholders, including anyone the AI system could impact.
8. Identify applicable laws, regulations, industry guidance, and internal policies.
9. Identify and quantify expected benefits.
10. Identify and rate potential risks and negative outcomes, such as biased decisions, improper use of personal data, and accuracy/reliability problems.
11. Evaluate whether the expected benefits make it worth proceeding with the AI system implementation plan given the anticipated residual risks.
12. Deploy the AI system, define appropriate metrics, monitor its use, and compare actual outcomes to predictions.
13. Perform AI impact assessments on a regular basis (e.g., every 6 to 12 months) or after significant changes to the AI system.

How is an AI impact assessment different from an AI risk assessment?

Impact assessments and risk assessments are both risk management tools, but they look at risk from different angles:

• Risk assessments investigate possible problems and what could go wrong in the future if those problems manifest. This helps organizations proactively address risks, minimize negative events, and prevent failures.
• Impact assessments look at planned or potential changes to a system and how those changes could impact that system and related systems and stakeholders. A fundamental goal of impact assessments is to answer the question, “How could this system fail or negatively impact stakeholders?”

AI risk assessments are best applied early in the planning/development cycle, while AI impact assessments are relevant across the AI system lifecycle.

According to Danny Manimbo, ISO & AI Practice Leader at Schellman, an AI impact assessment is intended to complement AI risk assessment but has a different focus. AI risk assessment covers mostly organizational level impacts, while AI impact assessment looks at the bigger picture of customers and other stakeholders.

Danny recommends conducting a separate AI impact assessment for each AI system in scope for ISO 42001 certification, as each will have a unique intersection of context, users, data, and associated risks.

“All that bubbles up into the risk assessment and decisioning around the implementation of controls, Danny notes. “Do we accept this? Or do we implement controls to mitigate the risk?”

Should my business conduct an AI impact assessment?

By performing an AI impact assessment and analyzing the results, companies can better weigh the potential benefits and harms that could manifest from developing, deploying, supplying, and/or using a specific AI system. The greater the range of AI-related risks an organization faces, the more it can benefit from improved AI oversight.

AI developers and providers that create, modify, and/or deliver AI systems and tools face additional AI risks and associated financial, operational, legal, compliance, and reputational risks versus businesses that are just using AI.

This is especially true for AI systems with sensitive use cases that could have safety, legal, environmental, or human rights impacts. These include:

• AI systems that influence individuals’ access to jobs, education, financial assistance, or essential services.
• AI systems used by federal, state, or local government agencies.

Other entities that stand to gain the most from AI impact assessments include:

• Businesses whose customers, partners, or other stakeholders are raising concerns about AI risks and governance.
• Businesses in regulated industries like financial services or healthcare, or that handle highly sensitive data.
• Businesses looking to gain competitive advantage by demonstrating responsible AI governance.

In the US and many other areas, AI usage is subject to current civil rights and consumer protection laws. AI impact assessments are required by law in specific jurisdictions where the AI system presents high risks to individuals or society. For example:

• The EU AI Act requires AI impact assessments for high-risk systems, especially around individual freedoms and rights.
• Local Law 144 in New York City requires an annual bias audit by an independent auditor for any entity using automated employment decision tools (AEDTs).
• Several US states, including Connecticut and Maryland, require government agencies to perform AI impact assessments on high-risk AI systems.

What’s next?

For more guidance on this topic, listen to Episode 153 of The Virtual CISO Podcast with guest Danny Manimbo, ISO & AI Practice Leader at Schellman.