AI products do not exist outside the law. Regulations are catching up to what AI systems can do, and the vocabulary that comes with them is now part of product conversations that designers and PMs cannot afford to sit out. When a legal team asks whether a feature falls under the EU AI Act, or a compliance officer raises questions about auditability, not knowing those terms puts product professionals on the sidelines of decisions that directly affect their work.
AI governance refers to the frameworks organizations use to make sure their systems behave consistently with legal requirements and user expectations. It covers who is responsible for what, how decisions are documented, and what happens when something goes wrong. Regulation adds the external layer: rules set by governments that define what organizations must do and what the consequences are for getting it wrong.
The terms in this lesson appear in compliance briefings, product reviews, and board-level conversations about AI risk. Understanding them is what allows designers and PMs to contribute to those conversations rather than wait to be told what the outcome was.
AI governance
AI governance refers to the set of processes, roles, and policies an organization puts in place to oversee how its AI systems are built, deployed, and monitored over time. It is the internal answer to the question of who is responsible for what when an AI system causes harm, fails to perform as expected, or produces outcomes that conflict with the organization's stated values.
In practice, AI governance means defining:
- Who approves a model before it goes to production
- What documentation is required
- How performance is tracked after launch
- What the escalation path looks like when something goes wrong
A company with strong AI governance has answers to those questions before a problem occurs. A company without it tends to discover the gaps when accountability is already in question.
For designers and PMs, AI governance matters because it determines what decisions they are expected to make, what they need to document, and what constraints they are working within. A PM who understands what governance requires can write requirements that anticipate compliance needs rather than creating rework after a legal or ethics review flags a gap.[1]
EU AI Act
The EU AI Act is a regulation passed by the European Union that establishes legally binding rules for AI systems used in EU markets. It came into force in 2024 and introduces a risk-based framework that classifies AI systems by the level of harm they could cause, with different requirements applying to each tier. It is the most comprehensive AI regulation passed by any major government and is already shaping how companies worldwide design and document their AI products.
The Act matters to product teams because its requirements are concrete. High-risk AI systems, such as those used in hiring, credit scoring, education, and law enforcement, face strict obligations around transparency, data quality, human oversight, and documentation. Deploying a non-compliant system in the EU carries significant financial penalties. Even teams building for non-EU markets are affected because many companies apply EU standards globally rather than maintaining separate compliance tracks.
Knowing what the EU AI Act is and what risk tier a product might fall into is now a baseline competency for PMs and designers working on AI features at any company with European customers or operations.[2]
High-risk AI systems
The EU AI Act divides AI systems into risk tiers based on the potential harm they could cause. High-risk AI systems are those whose outputs can significantly affect people's safety, rights, or access to opportunities. The Act specifies which categories qualify: AI used in hiring, credit scoring, educational assessment, law enforcement, border control, critical infrastructure, and medical devices all fall into the high-risk tier.
High-risk classification carries concrete requirements. Organizations must document how the system works, demonstrate that it was trained on appropriate data, implement human oversight mechanisms, and register the system before deployment. They must also maintain logs that allow decisions to be audited after the fact. These are legal requirements with enforcement mechanisms attached, not optional best practices.
For a PM working on a hiring tool or a credit feature, knowing whether the product qualifies as high-risk determines the entire compliance roadmap. Getting that classification wrong at the design stage means discovering the gap during a legal review, when the cost of fixing it is much higher.[3]
AI risk classification
The EU AI Act classifies AI systems into 4 risk tiers:
- Unacceptable risk systems are banned: AI that manipulates people without awareness, government social scoring, and real-time biometric surveillance in public spaces all fall here.
- High-risk systems are regulated, covering hiring, credit scoring, medical devices, and law enforcement.
- Limited risk systems face lighter transparency obligations, such as disclosing that users are interacting with an AI.
- Minimal risk systems, like spam filters or recommendation engines, carry no specific obligations.
The tier a product falls into is not always obvious. A chatbot providing legal advice might be limited risk in one context and high risk in another, depending on how consequential its outputs are. A hiring tool that suggests candidates may be treated differently from one that decides autonomously. Classification depends on the system's purpose and the vulnerability of the people it affects.
For designers and PMs, understanding the tier structure means flagging early whether a feature carries compliance obligations. Getting that right at the design stage is far less costly than discovering it during a legal review.[4]
Auditability in AI
Auditability in AI refers to the ability to examine, trace, and verify how an AI system reached a particular decision. An auditable system keeps records sufficient for an independent reviewer to reconstruct what happened, identify where errors occurred, and determine whether the system behaved in accordance with its design and legal requirements.
Auditability matters because accountability without it is an empty promise. If an organization commits to responsible AI but cannot show how its systems made specific decisions, it cannot demonstrate compliance, investigate complaints, or learn from failures. When a loan is denied or a job application is rejected, an auditable system can show what data was used, what the model produced, and what threshold triggered the outcome.
For product teams, auditability is a design requirement, not an afterthought. A PM needs to specify what logs the system must keep and for how long. A designer needs to consider whether users can request an explanation of a decision that affected them. Building auditability from the start is far cheaper than retrofitting it when a regulator asks for records that do not exist.[5]
Pro Tip! If your product cannot answer "what data led to this decision?", it is not auditable. That is a product requirement, not a legal technicality.
AI policy
An AI policy is a documented set of rules and principles that governs how an organization develops, deploys, and uses artificial intelligence. It defines what is permitted, what is prohibited, what requires approval, and who is accountable for AI-related decisions across the organization.
AI policies vary in scope. Some set broad principles, committing to fairness, transparency, and human oversight. Others are operationally detailed, defining which use cases require a bias audit before launch, what data sources are approved for training, how model performance must be monitored after deployment, and what the escalation path is when a system produces unexpected outcomes. The most useful policies do both: establish principles that explain the why, and define processes that make those principles enforceable.
For designers and PMs, an organization's AI policy determines what is in scope for any given project and what decisions require sign-off from legal, ethics, or leadership. A PM who has read the policy can anticipate those requirements during planning rather than discovering conflicts during review.[6]
