Authored by Matt Johnson, Principal Technologist at MongoDB

Over the past few years, AI has rapidly moved from experimentation to a boardroom priority as businesses feel more pressure to integrate AI into products, services and internal operations. With boardrooms becoming increasingly focused on the competitive advantages that may be unlocked by AI integration, there are growing concerns around transparency, accountability and risk which are becoming harder to ignore, especially with the introduction of the new EU AI Act regulations.  

The competitive argument for moving quickly is legitimate. In retail and consumer products, 75 per cent of executives now consider AI a top strategic priority, with 82 per cent planning to increase investment over the next 12 months. Yet only 16.5 per cent can currently quantify a return on that investment. In manufacturing, only 14 per cent of firms studied had implemented AI in production, despite relatively strong levels of AI readiness. The pressure to adopt is therefore understandable, but businesses are still working out how to turn investment into sustainable value across industries. 

Businesses understandably fear being left behind if competitors capture the productivity, efficiency and customer benefits AI can provide first. Investors, customers and boards are also increasingly expecting organisations to have a clear AI strategy. However, moving quickly does not necessarily mean moving well. As AI becomes embedded into more critical processes, the cost of an unexplained, inaccurate or inappropriate output becomes much greater. The challenge is no longer simply how quickly businesses can adopt AI, but whether they can scale it responsibly. 

Building employee confidence in AI adoption 

Many organisations are discovering that speed alone is not enough. The people using AI need to understand the systems behind it, while the organisation itself needs visibility into the data feeding those systems, the models generating outputs and the applications acting upon them. Only 11 per cent of employers provided AI training in the previous 12 months, while just 21 per cent of employees reported feeling confident using AI at work with more than half believing that understanding AI risks and threats, as well as keeping information safe and private, is an important skill, which may add to the reluctance to adopt AI without specific training. Without this understanding, businesses risk scaling systems that their own people cannot fully interrogate, challenge or ultimately claim accountability for. 

Healthcare provides a particularly clear illustration. AI-enabled tools are already delivering tangible productivity benefits, with 42 per cent of UK clinicians saying they save at least 132 hours a year through AI-enabled tools, while 36 per cent say AI enables them to see more patients. Yet 74 per cent report that AI training within their organisation is inadequate, inconsistent or unavailable, while 56 per cent use personal AI tools when workplace solutions do not meet their needs. The value of AI is increasingly evident, but so is the risk of adoption getting ahead of organisational understanding and control. 

This is where the conversation needs to move beyond experimentation and into governance. As AI agents become capable of interacting with data, systems and other applications, businesses need to know what those systems can access, what information they are using, which model produced an output and what happened afterwards. This is not simply a compliance exercise. It is a technological requirement for accountability. As agentic AI develops, organisations will need increasingly sophisticated ways of monitoring activity, identifying unexpected behaviour and understanding how decisions are being made. 

Where does regulation come into play? 

Regulation is reinforcing this direction. Article 12 of the EU AI Act requires high-risk AI systems to have automatic logging capabilities across their lifecycle, supporting traceability, risk identification and ongoing monitoring. The wider emphasis on documentation, data, transparency and human oversight demonstrates that organisations will increasingly be expected to understand and evidence how their AI systems operate. 

There are legitimate concerns that regulation could introduce additional complexity for businesses trying to scale AI. Smaller organisations in particular may lack dedicated governance teams or the resources to interpret changing requirements. But regulation also creates an opportunity to establish better practices before problems occur. More than 88% of organisations now report using AI in at least one business function, yet only 39% of Fortune 100 companies disclosed any form of board oversight of AI as of 2024. Adoption is moving faster than governance, and that imbalance cannot continue indefinitely. 

Striking the balance between accountability and innovation 

The answer is not to create a rigid governance structure that slows every AI initiative. Businesses need foundations that allow governance and innovation to develop together. That means maintaining visibility across data, models and applications, creating traceable systems and ensuring organisations can understand and audit AI activity as it scales. There will be no one-size-fits-all approach, but there is a common requirement which is visibility. 

This will become even more important as the technology evolves. Further regulation and guidance are likely to emerge, while AI models, agents and applications will continue to change. Organisations should therefore avoid building governance around today’s technology alone. Instead, they should build adaptable systems that can evolve alongside it, ensuring visibility and accountability remain embedded as their AI infrastructure develops. 

Businesses do not have to choose between innovation and accountability. In fact, the two should be considered mutually reinforcing. The organisations that scale AI successfully will not necessarily be those that deploy the most systems or move the fastest. They will be those that build the technological visibility to understand what their AI is doing, the governance to intervene when necessary and the flexibility to adapt as the technology changes. 

AI is an opportunity, but accountability cannot be an afterthought. If businesses want to move fast with confidence, they need to know what is happening underneath the AI. Governance should not be the brake on innovation. It should be part of the infrastructure that makes sustainable innovation possible.  

Photo courtesy of MongoDB