“What’s the difference between a classic PM and an AI PO?” I get asked this regularly. The short answer: much more than most people think.
What stays the same
The fundamentals of product management remain:
- Understanding user needs
- Prioritizing the backlog
- Defining product vision
- Communicating with stakeholders
- Measuring results
If you’re a good PM, you already have 50% of the job. But the other 50% is genuinely different.
What changes: the nature of uncertainty
A classic PM manages uncertainty about what to build. The team generally knows how to build what’s decided.
An AI PO also manages uncertainty about whether it’s possible and to what degree. An AI agent that “works at 85%” — is that acceptable? It depends on the use case. But defining that threshold, measuring it, and deciding when it’s “good enough” for production — that’s a new type of decision.
What changes: specs
A classic spec describes deterministic behavior. “When the user clicks X, Y happens.”
A spec for an AI agent describes probabilistic behavior. “In most cases, the agent should do X. In these edge cases, Y. In these error cases, Z.” And you need to define what “most cases” means, and how to measure it.
Without a technical background, writing this type of spec is very difficult. You need to understand why an LLM might interpret your instruction differently depending on context.
What changes: evaluation
A classic PM evaluates product quality through business metrics (conversion, retention, satisfaction) and functional tests (works / doesn’t work).
An AI PO must also evaluate the quality of the model itself: precision rate, hallucination rate, output consistency, edge case behavior. These are skills close to what’s called “ML evaluation” — not dev, but not classic PM either.
What changes: the relationship with the technical team
A classic PM works with devs who implement specs. The boundary is relatively clear.
An AI PO works with devs who make model choices, prompt engineering decisions, and agent architecture choices that directly impact what the product can do. The boundary is blurry. If you don’t understand these choices, you can’t make good product decisions.
That’s where my dev background becomes a concrete advantage. I can participate in these decisions, not just be subjected to them.
What a good AI PO needs to master
No need to code agents. But you need to understand:
- How an LLM works (at a conceptual level)
- What a system prompt is and how it influences outputs
- How to evaluate an agent rigorously
- The costs and latency of different options
- AI-specific risks (hallucinations, bias, drift)
It’s a new profession. Not just a PM with a specialization.
Stéphanie Caumont
AI Product Owner · Learn more