Skills Vs. Duties in AI: The Imbalance That Will Cost Us

Almost anyone can build with AI now. The duties never spread with the skills.

A few years ago, building an AI system took a master's degree in applied math or data science. The work sat with a small group of practitioners, and so did everything that travelled with it. Checking that the output was actually right. Watching it over time to make sure it still worked. Naming a human who owns it when something goes wrong.

Those barriers are gone. Someone in your finance team can write a prompt, stand up a chatbot, and launch a use case in an afternoon. Most people read that as pure progress.

Paul Drennan reads it differently. He spent forty years turning data, analytics, and AI into enterprise capability, most recently as a Chief AI and Analytics Officer, and before any of that he was a Navy officer running engineering aboard a destroyer, where nobody separates what you are capable of from what you owe.

His argument is that the skills got democratized and the duties stayed where they were. The obligations that used to come bundled with the experts never made the trip. That imbalance is already producing consequences, and most of the leaders living with it have not named it yet.

Why he stopped calling it AI governance and started calling it safe enablement

Drennan is careful to say governance itself is not the problem. It is critical, in this domain more than most. What went wrong is the reputation.

Over time, governance acquired a negative sentiment. It became the thing that shows up late in the process and interrupts progress. So he changed the words, and then changed the sequence underneath them.

Governance makes people anxious, as he puts it. Everybody wants safe enablement.

The substance behind the relabel is a shift left. Rather than a post hoc inspection at the end, the people whose duty is to protect the firm from legal consequence, reputational harm, and security breaches get brought in at the beginning and asked to co-author the system. If the system is followed, safety is a property of the build rather than a verdict delivered afterward.

His tagline for it: safety through design, not supervision.

Team Yes: turning risk, legal, and audit into co-authors instead of blockers

Drennan calls that group the guardians. Audit, risk, legal, compliance, and often procurement, security, and architecture. Every one of them has a real obligation to protect the enterprise, and every one of them has learned, over years, that the safe answer is no.

When he formed his governing committee, he told them so directly. He knew they were all born on Team No. He was asking them to join him on Team Yes.

The pitch was not that the guardians should get out of the way. It was that these capabilities serve customers, shareholders, and each other, so yes is what everybody actually wants to say. Team Yes is not a rubber stamp. The question simply changes direction. Instead of asking the guardians for approval, the builders ask them what they need in order to be able to say yes.

Jeff Carson's read on it in the conversation was that most corporations have no Team Yes at all, and that the absence is what quietly caps adoption.

How publishing the safety boundary gets governance off the critical path

In the early days, every use case went in front of every guardian, because everything was novel and everything was first of its kind. That does not scale, and Drennan did not try to make it.

What he watched for instead was repetition. Someone wants a retrieval solution for legal. Someone wants the same thing for claims. The technical platform underneath them converges, the standard work in the build cycle converges, and the requirements for trust maintenance, observability, monitoring, and repair start looking like a pattern rather than a series of one-off judgments.

So the team pushed the boundary conditions upstream. They defined formally what safety looks like, named the things that would trigger anxiety, and then published the boundary set for everyone to see.

Two things followed. Most use cases turned out to sit inside the safe perimeter, where deep guardian review was adding very little. And builders who could see the boundary started building to it deliberately, because staying inside it made them faster. Once enough of the portfolio was flowing through the safe box, approval inside that box could be automated.

It took roughly eighteen months from forming the team to reaching that level of clarity. What it bought was not only speed for the builders. Guardians are specialized and scarce, they do not scale, and the builder population is expanding far faster than they are. Getting them out of routine review is what lets them spend their attention on novelty, which is exactly where their judgment is worth the most.

The AI literacy gap, and why humans are hallucinating too

Everyone worries about models hallucinating. Drennan's more uncomfortable observation is that the humans deploying them are doing something similar.

Something profound has emerged, everyone instinctively understands the future looks different, and enormous pressure has been applied to get on board immediately with the implication that anyone who does not is in real trouble. Fear like that produces two responses, and he has watched executive teams swing between both.

On one side, near paralysis. I do not understand it, so I do not know how to use it, and I am going to be held accountable for the outcome anyway. On the other side, a jump into the deep end without checking whether there is water in the pool.

His analogy for what we have done is deliberately uncomfortable. We travelled up the river, found people hunting with bows and arrows, handed out machine guns, and told everyone that refusing to use one meant a bleak future. What we did not do was teach anyone how the machine gun works.

So people try things that do not make sense, the tool does the job poorly, confirmation bias fires, and the verdict comes back that it is not ready. Then the pressure resumes and the cycle starts again. None of that motion builds trustworthiness.

Underneath the literacy problem sits an older one. Most executive teams cannot trace how data becomes an insight that becomes a decision, and AI widens that gap rather than creating it. Drennan has always found the hardest part of this work is not building the system. It is asking a person to rewrite their job description around a tool they cannot fully see inside while retaining full accountability for what happens next.

That climb got steeper recently. Five years ago a deployed algorithm was deterministic, and the same ten inputs produced the same output every time. With a generative solution that guarantee is gone. Prompt and context engineering restrict how creative the system gets, but they cannot promise an identical answer twice, and confirming the validity of a response is genuinely harder than it used to be.

The asset owner: the AI role most companies do not have

Once Drennan accepted that the hardest part of the job was recruiting belief rather than building models, he noticed something uncomfortable. Nobody on his team had that in their job description.

So he created a role and called it the asset owner. Their entire mission is to close the gap between the people who build the system and the people who have to stake their work on it.

The instruction he gave them was to convert a leap of faith into a hop of faith. Educate, expose, and make as much of the system transparent as it can be made, while acknowledging that some of it cannot be, because the goal is to recruit belief from users rather than compliance.

The skill blend is unusual. He wanted the sales engineer who can both use a tool and explain it. He wanted the strategy consultant who looks around the corner on a client's behalf and brings them things they did not know to ask for. He wanted the product owner who can hold the week to week progression and keep every stakeholder informed. The specific behaviors evolved a great deal over time. The personality he was hiring for did not: creative, passionate, helpful, humble.

The result was the fastest growing job family in his department. The business valued them as their person inside the builder shop. The builders valued them as the person carrying the complexity outward and building the confidence that got the work adopted. His only regret is that he did not connect those dots ten years earlier.

Why AI pilots really fail

The failure explanation Drennan does not accept is that the technology was not ready. That is rarely the root cause.

Some pilots fail on context. An initial view of the problem was correct at the time, nobody safeguarded how that context would evolve, and the system drifted away from the world it was built for.

But in his experience most of them fail somewhere less technical. The user was asked to rewrite their job description around the tool, their anxiety about doing that was never addressed by anyone whose job it was to address it, and adoption quietly never happened. Money got spent. The gap between the builders and the users never closed.

AI drift is microscopic damage, and safe to own is the repair

The line Drennan opens the episode with is the one worth sitting with. Unsupervised AI will betray you, and it is only a question of when. The enemy is drift.

The mechanism is straightforward once you see it. We do not write this code. The machine learns it from examples, including the foundation models, whose ability to predict the next token rests on the examples they were trained on. Tomorrow will not look like those examples.

Drift arrives from outside and inside at once. Climate, geopolitics, the price of oil. A new product launch, a different route to market, a changed appetite for a customer segment, a fresh set of incentives for the sales force this quarter because growth is behind. Every one of those moves the world away from the examples the model learned from, and as the distance widens, the odds of a safe inference fall.

Drennan thinks of it as microscopic damage, which means the response is repair. That requires observing the damage, and performance monitoring alone will not find it. Millisecond response times can be perfect while the answers quietly change. You have to watch what the same inputs produce over time and notice when the output is no longer what it used to be.

There is a second kind of damage that has nothing to do with accuracy. The algorithm can be performing exactly as designed while user confidence erodes anyway, because mistakes happen and the accountable human is keeping score. In his experience one bad miss carries the weight of a dozen good outcomes. Eventually there is one more mistake than a user can tolerate, and trust is gone.

The phrase his teams used for the answer was safe to own. It does not just work on day one, it will be working for you three years from now, because we will be vigilant on your behalf and do the work to keep it accurate. That promise is what a monitoring and repair mechanism actually buys.

What a Chief AI Officer is actually for

Drennan is amused by the idea that the role did not exist five years ago. He had the same job fifteen years ago. The term AI simply did not carry the usage it does now.

He is equally clear that there is no boilerplate job description that works everywhere, and he starts by naming the two extremes that fail.

A thousand points of light, where everyone goes and does their own thing, does not scale and handles the obligations poorly. Strict centralization, where one team builds and controls everything, protects the qualities you care about and strangles participation. That model was workable when this work required a data scientist and a machine learning engineer, and that world is gone.

The word he uses is federated. Build a system that supports a distributed builder community while holding on to safe enablement and to scalability, extensibility, affordability, and everything else that dies in a free for all. The fundamental duty of the Chief AI Officer is to navigate that balance and to keep teaching people what the balance looks like, knowing the definition keeps moving.

The method he trusts is what he calls the first wrong answer. Deploy something workable, invite critique, expect revision, gain operational experience, and reapply what you learn. As patterns become predictable, migrate them into the system. What does not work is sitting in a room, defining the answer, and arriving to promulgate it.

So when he evaluates someone for the role, he looks at attributes before duties. There is real platform engineering and data governance work in the job, and token budgets do have to be calculated. But the person has to be able to build an environment that invites participation, shares accountability for the guiding principles, and stays humble about a learning curve everybody is on together. Nobody arrives with a finished cookbook.

How do you audit generative and agentic AI

Carson pressed him on the auditing problem, and Drennan called it the single biggest challenge to scaling safety and value creation in generative AI.

The old toolkit assumed determinism. Five years ago there was labeled training data, validation cycles, and a clean comparison between the new model's output and the old model's output to see where the variances were. Confirmation was relatively straightforward. That is gone.

He is not hopeless about it. He expects solutions to move toward much more deliberate context engineering and expert system style design, where an ontology surrounds a business domain and world models orient the system to the boundaries that apply. Naively firing a few examples at a large language model and shipping the answer to production is not the future. Narrowing the space is what restricts the opportunity for strange behavior.

That approach creates two dependencies most programs have not staffed. The first is dedicated, professional curation of the knowledge base the system reasons over. If you are building AI and have not engineered that into the life cycle, Drennan considers the discovery of that gap essentially guaranteed. The second is validation, and validation needs people who know the domain well enough to spot an error.

SME validation is a protected duty

This is the part of the conversation Drennan is most concerned about, because the resource involved is invisible until it is missing.

Right now the workforce is full of subject matter experts who learned their jobs without an AI in their pocket. They can look at an output and know it is wrong. That creates a comfortable illusion that validation capacity is not constrained.

Picture the same organization ten years out. Those people have retired or moved on, and the knowledge that made them capable validators left with them. The people who replaced them grew up with the AI in their back pocket, and their ability to catch the same errors will not be the same.

His conclusion is that subject matter expert validation has to be treated as a protected duty and planned for explicitly in workforce planning, staff development, job descriptions, and how new hires are onboarded. You cannot hire your way out of it after the fact, and once the capability is gone, rebuilding it is a nightmare.

The economics make it harder. Validation carries both scarcity and high opportunity cost at the same time. If someone is a brilliant underwriter, you want them underwriting, not checking the model's homework. Drennan is careful to say none of this is new. Automation has always had this problem, and so has robotics. What is different now is magnitude, because nondeterministic output means the checking never stops being necessary.

Proud ownership, and what boards should be asking for

Carson asked what proud ownership looks like for a C-suite that did not come up through data science. Drennan's answer is that it is the summary result of everything above rather than a separate activity.

You did the upfront work so safety was designed in. You built user confidence and maintained it. You put observability, monitoring, repair, and validation in place and kept them running. What you get is a portfolio you can defend under internal and external challenge, and the ability to say that when something went wrong, you already knew, you had observed it, you mitigated the effects, and you fed the learning back in so that the only future mistakes are new ones.

The report card he cares about is not thirty percent productivity. There are budget assumptions and a cost benefit case behind that, and he wants those too. But the final exam is whether users adopted it and whether they are still using it a year later, and three years after that.

For boards, his advice starts with acknowledgment. Nobody has fully solved this, every organization sits somewhere different on the maturity curve, and the honest starting position is to accept that. Then keep it on the agenda as a standing priority rather than an incident response.

What he would want to see from the C-suite is regular reporting on how learnings are being reinvested, and he is deliberate that the system in question is more than the technology. It is the platform, the standard work, and the operating principles together. A system that is hungry for observations, with its antennae out, run by a culture that treats mistakes as material to reinvest rather than moments of consequence, is what makes people candid about what they are actually seeing.

What he would not do is build a slide with twelve metrics and score it once a quarter. That measures what is measurable and creates a false sense of confidence about everything else, and most of the levers that matter here are behaviors, communication, and humility about where everyone sits on the learning curve. He has no idea how to write a KPI for those, and he is suspicious of anyone who claims to.

Carson closed the conversation on the line the whole hour was building toward. The skills got easy. The duties and responsibilities did not. And until every leader owns that gap, the losses keep coming.

About Paul Drennan

Paul Drennan spent four decades turning data, analytics, and AI into working enterprise capability, most recently as a Chief AI and Analytics Officer, with earlier work across insurance, healthcare, financial services, and consulting. He built the governance model he calls safe enablement, formed the cross-functional group he named Team Yes, and created the asset owner role to close the trust gap between AI builders and the people whose work depends on them.

He is a United States Naval Academy graduate who served as a Navy officer running engineering aboard a destroyer, and holds a master's degree from the Naval Postgraduate School. He is a named inventor on multiple issued patents covering the infrastructure that makes large enterprises run. He now advises insurtech startups and internal enterprise teams on leadership, AI safety at scale, and practical solution building. Outside the work he is a blacksmith and leatherworker, and he builds the tools he uses.

About TheAIAudit

TheAIAudit is a patent-filed AI governance platform built on a time-tested financial framework, rebuilt to reflect the rapid adoption of AI by companies. It measures an organization's AI Investments across Governance, Impact, and Risk Flow and returns an AI Health Score, a 0 to 100 number built on over a thousand source metrics. It answers the three questions a board asks. Is it under control? Is it paying off? And what could it cost us?

Learn more about TheAIAudit

Know what your AI Investments return.

TheAIAudit measures your organization's AI Investments across Governance, Impact, and Risk Flow, and returns a single score executives and boards can act on.

Unsupervised AI will betray you. It is only a question of when. The enemy here is drift.

Paul Drennan