Following proclamations of the inevitability of AI as a vital future part of IT support services, some of the biggest companies in the field of large language models have suddenly urged significant caution about using the tools they have aggressively been selling.
The most recent of these is Microsoft, which published a proposed Code of Conduct based on the principle that AI tools must always be in service of people, amidst concerns raised by major figures in the field that more advanced progress without safety barriers could be apocalyptic.
Whilst the claims are extreme, they do raise a valid question of whether AI tools are safe for businesses to use, or whether their propensity to make elementary mistakes and factual errors could potentially open a business up to legal liability.
Are AI-powered tools safe for your business to use? Are the calls for concern valid or an attempt to generate interest and funding in the wake of a potential AI bubble bursting? And why should businesses be careful with tools that seem to be innocuous productivity aids?
Why Have AI Companies Raised The Alarm?
The alarm was raised following an incident where an experimental AI model being trained by OpenAI (part-owned by Microsoft) broke its containment using a series of exploits before breaking into the machine learning platform Hugging Face to access internal datasets.
Whilst far from the first malfunction, which included an infamous moment where an agent powered by Anthropic’s Claude system deleted a firm’s database and all of its backups and admitted it ignored every instruction given, it highlights the extent to which a breach can damage a business.
Given the many issues that businesses have already faced through the ineffective use of LLMs for content production, marketing and coding, many people already know the problems caused by attempting to integrate AI without human oversight and potential security risks.
However, this particular incident has been described as a wake-up call for Silicon Valley, with two reasons why this may be the case.
Apocalyptic Fear
Part of it is based on the overall goals behind many AI companies, built upon the goal of creating “Artificial General Intelligence”, an imagined theoretical superintelligence that would theoretically be more powerful than any possible human way of containing it.
This belief, as well as similar theories such as Roko’s Basilisk, suggests the end-goal of AI tools is less about the productivity gains that are marketed by tools such as Copilot and more about a fear of a Skynet-styled intellectual singularity which conquers mankind.
Sam Altman, head of OpenAI, once noted that the company’s strategy to become profitable was to create AGI and then ask the superintelligent computer system how to make the company profitable.
Whilst these fears are closer to science fiction than anything else, they have driven both the rapid advances in AI and also this sudden fear after one managed to breach containment.
However, given that the cause of the Hugging Face incident was a prototype agent being given limited security safeguards and access to the Internet, this particular situation was based on very human failures rather than artificial advances.
Fear-Based Marketing
A far less outlandish theory is that the claims of a science-fiction style apocalypse are an attempt to aggressively market AI tools to venture capitalists and major investors.
A claim made in Jacobin magazine is that the aim of what it calls an “extinction fantasy” is to centre the importance of the biggest companies involved with AI tools (Microsoft, Nvidia, Google, OpenAI, Anthropic and others), and ensure that they are provided with significant additional funding.
However, as Cal Newport notes in his article, the system in question only did what it was instructed to do, and the fault was with OpenAI in how it controlled these parameters.
In the words of Aristotle, the stick moves the stone but is itself moved by the hand that is moved by the man.
Amidst fears that the actual userbase for AI is nowhere near large enough to avoid a financial bubble orders of magnitude the size of the dotcom bubble bursting, this could be an attempt to change the narrative and emphasise its importance through fear rather than an Industry 4.0-style future.
The True Risks Of AI Software For Businesses
The entire Hugging Face fiasco and the doommongering that resulted did highlight the true reason why AI tools are far less safe to use for businesses from a security and liability standpoint; they are exceptionally sloppy in their production and security.
LLMs are inherently unpredictable and often come to conclusions using utterly bewildering logic, if there is any logic at all to an agent’s decision-making.
This is why using AI such as Microsoft Office’s built-in Copilot to produce reports leads to it sometimes generating nonsensical or even made-up results, why posters made using AI image generation often have bizarre graphical and text artefacts, and why it can take longer to correct AI code than it would have taken to write it yourself.