AI Partner, AI Provider and/or AI Producer – Which are We and Why Does It Matter?

As artificial intelligence (AI) use cases and solutions proliferate, more and more organizations are becoming part of the global AI supply chain. If your company helps develop, provision, extend, train, integrate, and/or evaluate AI systems, it is vital that you understand your role in the AI ecosystem, your AI stakeholders, and your responsibilities to them.

In fact, understanding AI ecosystem role(s) and responsibilities is so important—especially for AI product and service providers and their partners—that ISO 42001:2023, the first global AI management system standard/certification, makes this process the starting point for creating an AI governance regime.

Is your business an AI Partner, AI Provider, and/or AI Producer? This article will help you decide.

Key takeaways

• AI ecosystem roles are the starting point for addressing internal and external needs around trust, risk, performance, compliance, and value.
• Key AI roles in connection with ISO 42001 certification are AI Provider, AI Producer, and AI Partner.
• AI Providers include software as a service (SaaS) providers, managed service providers (MSPs), cloud hosting providers, and data center operators.
• AI Producers design, develop, train, test, and deploy AI models and AI components.
• AI Partners include AI System Integrators, Data Providers, AI Evaluators, and AI Auditors.
• Many AI Providers are also AI Producers.

What are AI ecosystem roles?

Correctly understanding your organization’s role(s) and purpose in the AI ecosystem can help you build trust with interested parties, reduce both internal and supply chain risk, and improve AI performance and value for users. Your role is the starting point for determining what controls you need to govern AI, including managing your AI risks and compliance, and determining organizational goals for AI activities.

ISO 42001’s Clause 4.2 ensures that certified organizations understand and manage their stakeholder’s expectations, which include customers, employees, regulators, and the general public. Stakeholder satisfaction, ethical concerns, risk management goals, and compliance and legal/contractual requirements are all part of the equation.

While stakeholder roles figure prominently in ISO 42001, they are also covered extensively in ISO 22989:2022, which describes AI terminology and concepts. ISO 22989 defines six primary AI roles and several sub-roles:

• AI Provider, including AI Platform Providers and AI Product or Service Providers
• AI Producer, also called AI Developer
• AI Partner, including AI System Integrator, AI Data Provider, AI Evaluator, and AI Auditor
• AI Customer, also called AI User
• AI Subject, which can be a Data Subject or Other Subject
• Relevant Authorities, including Policy Makers and Regulators

Meeting the needs of interested parties around AI helps build trust and improve a company’s reputation and credibility in the marketplace. Trust is especially critical with AI systems and services because of universal concerns around privacy, transparency, and bias.

Connecting with interested parties to get their feedback also helps AI Providers, AI Producers, and AI Partners to continuously improve their offerings’ performance, efficiency and effectiveness. Demonstrating continuous improvement is also a requirement to maintain ISO 42001 certification.

What are AI Providers?

Due to pressure from AI Customers, Relevant Authorities, and AI Subjects, many companies looking at ISO 42001 certification or other AI governance initiatives are AI Providers. This includes software as a service (SaaS) providers, managed service providers (MSPs), cloud hosting providers, and data center operators.

ISO 42001 defines an AI Provider as “an organization or entity that provides products or services that uses one or more AI systems. As noted above, there are two classes of AI Providers:

• AI Platform Providers, which provide services that allow customers or users to produce AI products and services. Organizations can use these platforms to build, train, and deploy AI models.
• AI Product or Service Providers include companies developing AI in-house or that use third-party AI within their service offerings. They can also be entities that provide AI offerings that customers/users use directly or integrate into another offering. A primary example is companies with SaaS solutions that use AI for specific features, whether the AI was developed in-house or outsourced.

AI Providers generally interact directly with AI Customers and are “downstream” in the AI supply chain from AI Producers. AI Providers can also be AI Producers if they have a hand in developing the AI system versus just using a third-party AI system with minimal extension or modification.

What are AI Producers?

AI Producers (aka AI Developers) are at the “upstream” end of the AI supply chain relative to AI Providers. These entities design, develop, train, test, and deploy AI models and create AI technology that AI Providers leverage to provide services to AI Customers. AI Customers could also use an AI Producer’s technology directly.

Examples of AI Producers include OpenAI, Anthropic, Google DeepMind, Meta, Microsoft, AWS, Mistral AI, and many others.

What are AI Partners?

According to Danny Manimbo, ISO & AI Practice Leader at Schellman, an AI Partner is an entity that can influence how an AI system operates but does not have full control over its operation. AI Partners are downstream from AI Producers and upstream from AI Providers in the AI supply chain.

As an example, Danny cites a client that provides training data to multiple AI Producers for their AI models in development. Other common AI Partner business models include third-party integrations for AI models.

ISO 42001 defines four AI Partner sub-roles: AI System Integrator, Data Provider, AI Evaluator, and AI Auditor.

Are we an AI Provider, an AI Producer, or both?

Danny Manimbo states that most organizations pursuing ISO 42001 certification are AI Providers. However, many of these could be AI Producers as well. A common scenario is organizations that both develop and/or train AI models, which they provide as a component of a service offering.

Danny shares another client example. Serving the healthcare vertical, this company focuses on additional training for an image processing AI model in the Google Cloud. This added machine learning extends the model’s capabilities to work for specific healthcare providers.

In Danny’s view, this business is both an AI Provider and an AI Producer because they are extensively modifying the training regime for a third-party AI model to make it usable by a specific segment of the healthcare provider market. Acknowledging this is a gray area, Danny submits that comparatively few AI Providers engage in this level of “add-on” model development.

“We [as auditors] don’t make that decision for our clients because … it quickly becomes a legal conversation,” Danny observes. “I can communicate our interpretation of the ISO 42001 standard, but they need to make that determination for themselves.”

“If you’re hands-on with training a model, we could err on the side of ‘likely AI Producer,’” Danny adds. “ISO 42001 has one control set and you determine your role(s) and then decide how many of those 38 controls apply or partially apply to you, given shared responsibilities and things like that.”

For example, AI Producers would be considered part of an AI Provider’s supply chain. As such, the AI Provider should ensure the AI Producer’s practices are trustworthy. In this shared responsibility relationship, AI Producers are accountable for the quality of developed AI models or system components, while AI Producers are accountable for the AI system’s use and compliance, including system performance and ethical outcomes.

In Danny’s view as an auditor, businesses in the AI Producer/AI Provider role(s) should look to cover the full spectrum of their responsibilities to AI Subjects, Regulators, etc. as well as their AI Customers.

What’s next?

For more guidance on this topic, listen to Episode 153 of The Virtual CISO Podcast with guest Danny Manimbo, ISO & AI Practice Leader at Schellman.