Citizen development that leverages low-code tools with generative AI creates compound shadow AI and third-party risk when users build ungoverned apps. These apps have the potential to move sensitive data through unsafe external APIs, expose company assets with excessive permissions, and fail to implement needed cybersecurity controls—expanding the organizational attack surface and introducing dangerous, unrecognized vulnerabilities.  

What should CISOs and other business and technical leaders know about the intersection of shadow AI and third-party risk in citizen development scenarios? This article shares critical insights, including how these risks manifest and key initial strategies to address them.

Key takeaways

• Shadow AI is the ungoverned use of unauthorized AI tools, including public AI models like ChatGPT and Claude. 
• Citizen developers are employees without formal software development training who use low-code/no-code platforms to build ad hoc apps looking to enhance productivity, automate repetitive processes, save time, and/or improve work output. 
• Especially when effective governance is absent, citizen development introduces both shadow AI and third-party cybersecurity risks. These risks compound one another to significantly expand the organizational attack surface and introduce dangerous vulnerabilities. 
• Major business risks that shadow AI and third-party risk amplify within citizen development projects include undetected data leakage, excessive permissions, weak access controls, widespread regulatory noncompliance, and supply chain vulnerabilities.
• Key early steps to reduce shadow AI and third-party risks in citizen development include gaining real-time runtime visibility onto AI systems, treating AI systems as managed identities, identifying all AI assets and third-party connections in your environment, creating associated policies, and offering an in-house “app store” to empower users, reduce governance program resistance, and eliminate redundant apps. 

What is shadow AI in citizen development?

Shadow AI is the ungoverned use of unauthorized AI tools, including open-source or public AI large language models (LLMs) like Claude, ChatGPT, or Llama. Employees without formal software development training—aka citizen developers—increasingly apply low-code/no-code AI development tools, often with an organizational mandate, to accelerate document processing, streamline workflows, analyze data, or create reports and other work products.

Ungoverned AI introduces significant cybersecurity, compliance, and privacy risk that may remain unidentified and unaddressed until it manifests in the form of a data breach, compliance violation, or privacy exposure. Research indicates that up to 90% of businesses currently have unknown AI agents in their environments. Further, up to 40% of employees now process sensitive data using AI platforms without authorization.

Major risks from shadow AI usage include:

• Leakage of sensitive data into the public domain after feeding it into public AI models, where by default it may be retained and used to train future models.
• Increased data breach risks and impacts due to poor visibility into AI-driven data exposure and incident root causes. 
• Unknown exposure to AI hallucinations, biases and discrimination, and other damaging outputs that may taint business decision-making, violate privacy rights, or create legal liability.
• Unidentified compliance risks from using unvetted citizen-built code combined with unvetted third-party AI services. The resulting software may fail to comply with cybersecurity, governance, regulatory, and/or legal/contractual requirements.
• Reputational risk and damage caused by data breaches, stakeholder data exposure or loss, regulatory sanctions, and associated legal actions.

How can citizen development create shadow AI and third-party risks?

Especially where governance is lacking, citizen development can introduce both shadow AI and third-party cybersecurity risks. These compound risks expand an organization’s attack surface and often create dangerous blind spots.

How does this risk manifest in your IT environment? The process is simple:

• Ungoverned or experimental citizen development activities yield unapproved, often undocumented AI tools built outside formal processes. These tools can potentially violate data protection, privacy, and/or compliance controls.
• Any ungoverned AI tool that leverages unapproved third-party AI services also introduces third-party AI risks that further expand the attack surface. These risks also drive a need for third-party AI risk assessment efforts as basic due diligence.

According to a SecurityScorecard report, third-party dependencies are behind over 35% of data breaches and over 40% of ransomware attacks. Meanwhile, shadow AI vulnerabilities make life easier for attackers by exposing sensitive data and workflows. Governing citizen-built apps is not just about managing in-house risks, but also about achieving a view of third-party risks.

Is an external AI model vetted through a software bill of materials (SBOM)? What company data is being shared with it, and through which apps? Organizations need to ensure that both citizen-developed AI tools and the external AI systems and APIs they rely on meet your cybersecurity and compliance requirements. 

What are top shadow AI and third-party risks with citizen development?

CISOs and their organizations face a rapidly widening blind spot where decentralized/ungoverned citizen development intersects with unauthorized AI tools to create or exacerbate many of the same issues. 

When employees quickly build low-code apps that connect third-party AI APIs, browser plug-ins, and low-code agents to sensitive company assets without effective oversight, the results can include:

• Undetected data leakage. A common cause is when users paste sensitive data or code into public AI tools that default to retaining or training on inputs.
• Dangerously excessive permission stacking or access control failures, where AI agents accrue a much wider range of privileges than any single task requires. 
• Potentially widespread regulatory noncompliance, including breaking privacy laws like HIPAA and GDPR.
• Hidden vulnerabilities in the code of third-party AI services that connect with citizen-developed apps.
• Lack of IT visibility into which citizen-built apps use what AI services.
• Unvetted AI providers whose weak security can cause third-party data breaches.
• Supply chain vulnerabilities through hidden dependencies in external APIs and other third-party code. 
• Upstream third-party dependencies within popular SaaS products whose embedded AI components change frequently and often sidestep in-house AI risk assessment. 

What actions can CISOs take first to reduce shadow AI and third-party risks in citizen development?

The target goal for CISOs looking to reduce AI-related risks is to gain real-time visibility and control of AI system activities. This includes automated logging and alerts to proactively flag or block unauthorized AI behavior, along with human oversight or tighter verification for high-risk decisions. 

Strategic initial steps to manage shadow AI and third-party risks in citizen development include:

• Deploy real-time visibility tools like browser-level telemetry, Cloud Access Security Brokers (CASBs), or an AI exposure management platform to identify shadow IT, which includes shadow AI. 
• Treat AI agents as managed identities subject to AI runtime behavior monitoring, least-privilege access controls, and strict provisioning/deprovisioning guidelines.
• Catalog and assess all AI assets, including citizen-built apps, whose capabilities extend beyond personal productivity or connect with sensitive data. 
• Document and risk-rate every vendor, supplier, and partner connection into your systems that can exchange data or trigger privileged actions (e.g., file transfer). 
• Publish a clear policy for approved AI tools, including steps to track or block unapproved apps that handle sensitive data. 
• Create an in-house “app store” for approved citizen-built AI apps, to empower users and reduce governance program workarounds and friction. 

What’s next?

For more guidance on this topic, listen to Episode 161 of The Virtual CISO Podcast with John Verry, Lead Managing Director at CBIZ Cybersecurity.