Looking to drive rapid technological advancement, more organizations are empowering their employees to create tools using AI and low-code platforms‚ a practice known as citizen development. However, this newfound freedom for users come with significant organizational risks—notably data leakage, privacy violations, and security vulnerabilities.

How can Chief Information Security Officers (CISOs) manage AI risks effectively in decentralized, democratized AI development scenarios? This article explores key strategies for CISOs, including governance and monitoring best practices.

Key takeaways

• Citizen development is when non-technical employees create apps or workflows using AI-based low-code or no-code platforms.
• Citizen development exposes organizations to data leakage, privacy compliance violations, and an expanded cybercrime attack surface.
• A layered governance approach is essential to covering all the AI activity in your environment. 
• Clear policies and effective training are also critical to reduce citizen development risks.  
• As citizen development ramps up in more organizations, CISOs need to pivot quickly to deploy real-time monitoring, end-to-end audit trails, least-privilege data access controls, and acceptable use frameworks. 
• Teams with experience applying AI risk management frameworks like ISO 42001 stand the best chance of successfully governing citizen developed AI. 

What is citizen development?

Traditional software development relies on professional developers who write custom code, generally within a strict software lifecycle management process. In contrast, citizen development refers to the practice where non-technical employees create applications or tools using low-code or no-code platforms in an ad hoc manner, potentially with zero oversight. 

The citizen development approach democratizes innovation, enabling faster problem-solving and increasing workflow automation without going through a traditional software development process. However, citizen development raises significant governance, risk, and compliance (GRC) concerns that are effectively mitigated in conventional development models.

Major risks associated with citizen development include:

• Data leakage—Employees may inadvertently expose sensitive data when building AI-driven workflows. For example, misconfigured permissions are a common vulnerability. 
• Privacy violations—Without proper governance, compliance with data protection regulations like HIPAA or GDPR can easily be compromised.
• Security vulnerabilities—Tools developed without security in mind can become vectors for attacks, such as prompt injection or unauthorized automation. Weak authentication and injection handling failures are prevalent vulnerabilities in low-code scenarios. 
• Shadow AI—Any apps that operate outside of governance visibility create unidentified, unmanaged risk vectors and attack entry points. 
• Operational problems—Low-code apps can fail or perform poorly when used on a larger scale, fragment or compromise enterprise data due to integration flaws, or spawn IT support/maintenance issues.

Why does citizen development require robust governance?

When employees have easy access to internal systems and can build AI-driven workflows on the fly, sensitive data can unintentionally be exposed or misused. Without strong guardrails, a human or agentic AI system could automate the extraction or sharing of data in ways that break compliance, for example. Security threats like prompt injection are real, but privacy violations can be even more prevalent—potentially with major regulatory consequences.

To mitigate top citizen development risks, CISOs must establish robust governance frameworks. The governance approach needs to shift from reactive to proactive, incorporating real-time monitoring and stricter access controls to prevent incidents from manifesting at machine speed before they can be managed. This ensures that employees can innovate without compromising data security, violating privacy requirements, or otherwise harming the business or its stakeholders.

Effective citizen development monitoring strategies include:

• AI gateways—Implement AI gateways to monitor interactions with AI models, ensuring compliance and security.
• Endpoint monitoring—Utilize desktop-level monitoring tools to track processes and data flows, identifying unauthorized actions between tools.
• Compliance APIs—Leverage compliance APIs to create checkpoints for data access and usage, enhancing visibility into citizen-developed applications.

Combining multiple monitoring technologies is critical for effective governance. No single tool can capture all interactions. Layering tools like those just listed creates a visibility web that allows organizations to detect and respond to potential governance violations quickly, with fewer events finding their way through the cracks.

The importance of clear policies and comprehensive training

Creating and enforcing clear policies, along with comprehensive training for citizen developers, are both vital for managing the process and reducing risk at scale. Best practices include:

• A formal registration process—Require employees to register any tools that access sensitive data or systems. This process should be enforced through technical controls.
• Data minimization—Limit access to sensitive systems and ensure that data shared by citizen-built tools is anonymized or tokenized.
• Audit trails—Maintain detailed logs of interactions with sensitive data as a basis for accountability and transparency.

Training employees on the risks and guidelines associated with citizen development is equally important. Regular training sessions can help employees understand the importance of security and compliance, ensuring they build and use AI tools responsibly.

What does citizen development mean for CISOs?

As citizen development scales up rapidly across companies of all kinds, many CISOs and other technology leaders were already scrambling to bring AI governance up to speed with AI usage and risk. Citizen development expands the risk profile, the severity of potential impacts, and the imperative to adapt quickly.  Yet this is only the latest emerging challenge CISOs need to get in front of as the AI landscape relentlessly evolves.

Leveraging frontier AI models like Claude and ChatGPT, users can describe goals in natural language and the AI can autonomously generate complete workflows or applications. In doing so, AI can create capabilities beyond what users understood they needed or even asked for. 

This massively decentralizes data access risks and raises AI governance stakes by an order of magnitude. CISOs need to quickly pivot to deploy real-time monitoring, end-to-end audit trails, tighter data access controls, and comprehensive policies for acceptable use. Forward-looking businesses experienced with AI risk frameworks like ISO 42001 or the NIST AI Risk Management Framework will be best prepared for these technology and operational changes.

How can CISOs gauge the risk level associated with citizen-built tools so they know where to apply governance resources? If an app or workflow goes beyond personal productivity, such as by touching sensitive data, impacting customers or other stakeholders, and/or influencing decision-making beyond a limited internal group, it needs to come under formal governance. That means security testing, lifecycle management, and business owner accountability.

How can CISOs block users from experimenting with low-code AI development completely off the radar? A practical approach is to create gateways at key thresholds ahead of any wide-scale deployment or risky integrations. For example, you can set policies so that “red flags” like accessing corporate data or sharing across a department demand formal registration. CISOs can enforce this process with technical controls, like requiring executive approval to access sensitive systems or logging all connections involving AI tools. 

What’s next?

For more guidance on this topic, listen to Episode 161 of The Virtual CISO Podcast with John Verry, Lead Managing Director at CBIZ Cybersecurity.