Nvidia has launched a new software platform designed to help companies monitor and control artificial intelligence agents as concerns grow over AI systems acting beyond human instructions.
The chipmaker announced on Monday that it had introduced the NVIDIA Open Agent Safety Platform, developed with more than 100 industry partners.
Jensen Huang, Nvidia CEO, said the platform is aimed at addressing growing concerns around AI safety as businesses increasingly deploy autonomous AI agents capable of taking actions on their own.
“AI’s extraordinary potential for society will only be realized if we solve AI safety,” Huang said in a statement. “As we continue to discover the frontier of AI capabilities, we must accelerate discovery at the frontier of AI safety.”
According to Nvidia, the platform will allow companies to monitor the actions of AI agents and enforce policies governing what those systems can and cannot do.
The launch comes amid a series of incidents involving AI agents accessing computer systems and information without explicit authorisation.
OpenAI has disclosed several incidents involving experimental AI systems that moved beyond their intended environments. In one case, an AI model accessed the systems of AI developer Hugging Face while attempting to circumvent restrictions during a cybersecurity test.
Australian authorities also said an OpenAI agent accessed public and non-public files in the country’s Medicare statistics database in June.
More recently, OpenAI disclosed that its agents accessed publicly available information from the US Census Bureau using login credentials found online. The agents also accessed public information from the Securities and Exchange Commission and unsuccessfully attempted to obtain data from the US Education Department.
In another incident disclosed over the weekend, OpenAI said it had paused development of its latest models after an agent accessed the open internet without authorisation, despite additional security measures introduced following the earlier incidents.
The incidents have intensified debate over how companies should control increasingly autonomous AI systems.
OpenAI CEO Sam Altman and Anthropic CEO Dario Amodei have called for international rules to govern the development and deployment of advanced AI systems. Some researchers have also warned about potential risks from increasingly autonomous AI.
Huang, however, has argued that concerns about AI posing an existential threat are overstated. He has also said regulation should support the growth of the AI industry rather than unnecessarily restrict it.
“If they believe their company is out of control, then get the company under control,” Huang told CNN’s Anderson Cooper last week.
Nvidia’s new platform places the emphasis on that control challenge by giving organisations tools to observe AI-agent activity and apply rules to their behaviour.
The development comes as AI agents evolve beyond systems that simply generate text or images to software capable of browsing the internet, accessing databases, executing code and carrying out tasks with limited human intervention.
