AI is already making decisions your leaders can't explain: Chief AI Officer, Ensono

Artificial intelligence has become remarkably good at making decisions. The bigger question is whether anyone can still explain them.

Across modern enterprises, AI is already prioritising IT incidents, recommending remediation paths, flagging suspicious transactions, reshuffling customer workflows and deciding where scarce computing resources should go. Most of these decisions happen behind the scenes. Individually, they appear routine. Collectively, they are reshaping how organisations operate.

In an exclusive interaction with People Matters, Jim Piazza, Chief AI Officer, Ensono Product, Technology & Consulting, says the greatest risk is no longer whether AI can automate work. It is whether leaders understand why those automated decisions are being made in the first place.

The problem starts with thousands of tiny decisions

The conversation around AI often centres on spectacular breakthroughs or worst-case scenarios. Piazza believes the real challenge is far less dramatic and far more common.

"The hardest decisions to explain are often not the dramatic ones," he says.

Instead, enterprises are dealing with thousands of operational decisions AI makes continuously across infrastructure, business applications and workflows. One system may prioritise an incident over another. Another may recommend how to resolve an issue, alter a customer journey or allocate computing capacity across hybrid environments.

Each decision may seem logical on its own. Over time, however, understanding how those conclusions were reached becomes increasingly difficult. Organisations may struggle to identify which data influenced a recommendation or whether the decision-making logic has gradually shifted.

Piazza describes this as "decision opacity by accumulation."

As AI becomes embedded across automation platforms, analytics tools and enterprise infrastructure, the decision chain grows longer and more interconnected. No single individual may fully understand every step involved. Leaders often know what happened, but not always why it happened.

The challenge, he says, will only intensify as organisations begin deploying AI agents capable of coordinating multiple tasks across enterprise applications with limited human intervention. Explainability will need to evolve alongside autonomy, particularly as AIOps platforms continue optimising hybrid infrastructure, reprioritising workloads and recommending remediation actions automatically.

When outcomes affect customers, employees, revenue, security or operational resilience, limited visibility becomes a business concern rather than simply a technical one.

When confidence quietly replaces judgement

Piazza believes one of AI's biggest operational risks comes from human behaviour rather than technology itself.

AI systems often present recommendations with remarkable confidence. Over time, operational teams begin trusting those recommendations because previous outcomes proved accurate, not because they fully understand the reasoning behind each new decision.

According to Piazza, this is where oversight slowly turns into rubber-stamping.

He also stresses that accountability never shifts from humans to machines. When automated systems make poor decisions, leaders cannot simply point to the model.

"Accountability does not transfer to the technology."

Responsibility continues to rest with the organisation and its leadership.

Limited visibility creates another challenge. If automated systems have adjusted priorities, changed configurations or triggered multiple downstream actions, reconstructing events during an incident becomes significantly harder. In highly complex enterprise environments, losing operational context can quickly turn relatively small issues into much larger disruptions.

Automation works best when governance grows alongside it

Piazza does not believe organisations should place humans in the approval chain for every AI-driven action. Doing so would undermine automation's value and limit scalability.

Instead, oversight should reflect the level of business risk.

He suggests low-risk, reversible actions can be highly automated, while decisions involving security, customer outcomes, financial exposure, regulatory obligations or critical infrastructure require stronger controls, clearer evidence and defined escalation paths.

Equally important is maintaining a complete decision trail. Organisations should capture:

• The data used to make the decision
• The model or rule applied
• The recommendation produced
• The action taken
• Who remained accountable throughout the process

Good governance, Piazza says, should make automation safer to scale rather than slowing business down. Human involvement is not required in every decision, but human accountability remains essential for the system as a whole.

He also encourages organisations to build governance into AI systems from the beginning instead of treating it as a compliance exercise after deployment.

His advice is straightforward: Do not treat AI as a bolt-on. Make it part of rationalised business processes.

Trust depends on transparency, not promises

For leaders managing AI transformation, technology alone will not determine success.

Piazza believes organisations lose trust when adoption moves faster than communication.

Employees, customers and stakeholders want to understand where AI is being used, what decisions it influences and what safeguards exist. Without clear explanations, ambiguity quickly creates suspicion.

Rather than presenting AI as flawless, leaders should openly acknowledge both its capabilities and its limitations. They should explain where human judgement remains essential and how organisations respond when automated systems make mistakes.

According to Piazza, trust is not built by claiming AI is safe. It is built by showing the organisation remains firmly in control.

As enterprise AI adoption accelerates, boards and regulators are also expecting governance practices to keep pace with deployment.

The future belongs to organisations that can explain their AI

Piazza sees autonomous IT operations changing the role of people rather than replacing them.

As repetitive infrastructure tasks become increasingly automated, human judgement shifts towards understanding business context, recognising unusual patterns and deciding when automated responses are no longer appropriate. Managers will increasingly oversee not only people and processes but also the behaviour of automated systems operating within their organisations.

Preventing overdependence on AI, however, requires more than policies. Piazza says enterprises need practical operating controls, including clear ownership for production AI systems, risk classifications, model monitoring, auditability, escalation procedures and the ability to quickly override automated actions. Preserving human expertise also remains critical because organisations become operationally fragile if employees no longer understand systems without the AI layer.

Looking ahead, Piazza believes successful organisations will treat AI as part of their operating model rather than simply another technology deployment. Those struggling with AI adoption are likely to scale automation faster than governance, remove human oversight too early or assume more data automatically produces better decisions.

Ultimately, he says, competitive advantage will not come from deploying the most AI. It will come from deploying AI that remains explainable, resilient and trusted as enterprise environments become increasingly complex. In the years ahead, organisations may discover their biggest AI differentiator is not how quickly machines can make decisions, but how confidently leaders can explain them.