top of page
Search

When AI goes rogue, the problem is not simply the technology. It is the authority we gave it.


We have been hearing so many reports recently about “rogue” AI which is adding to the fear and definitely should make every organisation’s board members and leadership teams pause, think and review the controls they have in place, aka Governance!


These reported incidents are unlikely to be the last, sadly, and they demonstrate what can happen when powerful AI encounters gaps in permissions, controls and containment.

In once incident, an AI coding agent reportedly delete a company’s production database and its backups in nine seconds. Clients of ours have reported similar, potentially business destructive, activity.


These activities have been labelled “rogue”. But what is really going on?

  • Who authorised the AI system?

  • What data, tools and infrastructure was it able to access?

  • Were its permissions limited to what was genuinely needed?

  • Which actions required human approval, and was there a process in place for that?

  • Could its behaviour be monitored and stopped? By whom?

  • Was there a tested response plan if it exceeded its authority?

  • Who remained accountable?


The UK’s National Cyber Security Centre now advises organisations to start with tightly “bounded”, low risk use, to apply limited access and privilege, to monitor agent behaviour and, very importantly, maintain meaningful human oversight. It also advises we should all plan for failure. 😮


NCSC states, wisely, that “if an organisation cannot understand, monitor or contain an AI agent’s actions, that agent is not ready for deployment”.


This is where AI TrustAssure and R AI D  ( Responsible AI Deployment) helps.


When we at The Trustbridge developed AI TrustAssure (with 224Protect) we wanted to provide a practical, structured way to assess whether AI is being introduced and used with effective governance around it.

Usoing ouir ICO approved GDPR frameworks, we added teh AI elements to help organisations:

  • identify AI systems, uses, owners and dependencies

  • evaluate risks to data, security, people and operations (using R AI D)

  • identify gaps in policies, accountability, access controls and oversight

  • establish proportionate controls and an implementation plan

  • create evidence that those controls exist and operate in practice

  • prepare for independent audit and demonstrate responsible AI governance.


AI TrustAssure does not promise that an AI system will never fail. No credible governance framework should or could.


What is does do is help ensure that organisations have asked the difficult questions, restricted the potential fallout and established human accountability before an incident occurs.


The lesson from recent events is not that organisations should stop using AI. It is that it must never be deployed without authority limits, monitoring and a reliable means of intervention.


AI can act in nano seconds. Governance needs to be in place before it does.

For more information https://www.thetrustbridge.co.uk/general-7 and start your AI Governance process with a R AI D assessment of the risks

 

 
 
 

Comments

bottom of page