6 days ago
21
Follow ZDNET: Add us as a preferred source on Google.
ZDNET's key takeaways
- Agentic AI technology is marked by a lack of disclosure about risks.
- Some systems are worse than others.
- AI developers need to step up and take responsibility.
Editor's note: This article has been updated with responses from Perplexity, OpenAI, and IBM.
Agentic technology is moving fully into the mainstream of artificial intelligence with the announcement this week that OpenAI has hired Peter Steinberg, the creator of the open-source software framework OpenClaw.
The OpenClaw software attracted heavy attention last month not only for its enabling of wild capabilities -- agents that can, for example, send and receive email on your behalf -- but also for its dramatic security flaws, including the ability to completely hijack your personal computer.
Also: Is Perplexity's new Computer a safer version of OpenClaw? How it works
Given the fascination with agents and how little is still understood about their pros and cons, it's important that researchers at MIT and collaborating institutions have just published a massive survey of 30 of the most common agentic AI systems.
The results make clear that agentic AI is something of a security nightmare at the moment, a discipline marked by lack of disclosure, lack of transparency, and a striking lack of basic protocols about how agents should operate.
A lack of transparency
The biggest revelation of the report is just how hard it is to identify all the things that could go wrong with agentic AI. That is principally the result of a lack of disclosure by developers.
"We identify persistent limitations in reporting around ecosystemic and safety-related features of agentic systems," wrote lead author Leon Staufer of the University of Cambridge and collaborators at MIT, University of Washington, Harvard University, Stanford University, University of Pennsylvania, and The Hebrew University of Jerusalem.
Across eight different categories of disclosure, the authors pointed out that most agent systems offer no information whatsoever for most categories. The omissions range from a lack of disclosure about potential risks to a lack of disclosure about third-party testing, if any.
A table featuring all the omissions of disclosure of agent systems in red.
The 39-page report, "The 2025 AI Index: Documenting Sociotechnical Features of Deployed Agentic AI Systems," which can be downloaded here, is filled with gems about just how little can be tracked, traced, monitored, and controlled in today's agentic AI technology.
For example, "For many enterprise agents, it is unclear from information publicly available whether monitoring for individual execution traces exists," meaning there is no clear ability to track exactly what an agentic AI program is doing.
Also: AI agents are already causing disasters - and this hidden threat could derail your safe rollout
"Twelve out of thirty agents provide no usage monitoring or only notices once users reach the rate limit," the authors noted. That means you can't even keep track of how much agentic AI is consuming of a given compute resource — a key concern for enterprises that have to budget for this stuff.
Most of these agents also do not signal to the real world that they are AI, so there's no way to know if you are dealing with a human or a bot.
"Most agents do not disclose their AI nature to end users or third parties by default," they noted. Disclosure, in this case, would include things such as watermarking a generated image file so that it's clear when an image was made via AI, or responding to a website's "robots dot txt" file to identify the agent to the site as an automation rather than a human visitor.
Some of these software tools offer no way to stop a given agent from running.
Alibaba's MobileAgent, HubSpot's Breeze, IBM's watsonx, and the automations created by Berlin, Germany-based software maker n8n, "lack documented stop options despite autonomous execution," said Staufer and team.
"For enterprise platforms, there is sometimes only the option to stop all agents or retract deployment."
Finding out that you can't stop something that is doing the wrong thing has got to be one of the worst possible scenarios for a large organization where harmful results outweigh the benefits of automation.
Also: OpenClaw is a security nightmare - 5 red flags you shouldn't ignore (before it's too late)
The authors expect these issues, issues of transparency and control, to persist with agents and even become more prominent. "The governance challenges documented here (ecosystem fragmentation, web conduct tensions, absence of agent-specific evaluations) will gain importance as agentic capabilities increase," they wrote.
Staufer and team also said that they attempted to get feedback from the companies whose software was covered over four weeks. About a quarter of those contacted responded, "but only 3/30 with substantive comments." Those comments were incorporated into the report, the authors wrote. They also have a form provided to the companies for ongoing corrections.
HubSpot, n8n, and Alibaba did not immediately respond to a request for comment.
A Perplexity spokesperson replied to ZDNET via email, stating that the report "contains significant factual inaccuracies" and that "we are working with the researchers to have these corrections made immediately, and strongly reject their characterization."
OpenAI responded to ZDNET in an email through a spokesperson with a list of bullet points about the risks and limitations of the Agent function of its Atlas browser, noting the program "is currently available only in preview, and carries some risk."
(Disclosure: Ziff Davis, ZDNET's parent company, filed an April 2025 lawsuit against OpenAI, alleging it infringed Ziff Davis copyrights in training and operating its AI systems.)
IBM responded to ZDNET via email offering point-by-point refutations with links to various supporting IBM documents. IBM remarked in totality that "The MIT study's assertions around IBM's agentic AI offering, watsonx Orchestrate, are inaccurate," adding:
IBM provides robust public documentation detailing agent observability, deterministic controls and guardrails, usage monitoring, reasoning transparency, and evaluation frameworks around our agentic offerings. We are currently reaching out to the MIT research team to address these inaccuracies and are committed to advancing trusted, governable AI for the enterprise.
An expanding landscape of agentic AI
Agentic artificial intelligence is a branch of machine learning that has emerged in the past three years to enhance the capabilities of large language models and chatbots.
Rather than simply being assigned a single task dictated by a text prompt, agents are AI programs that have been plugged into external resources, such as databases, and that have been granted a measure of "autonomy" to pursue goals beyond the scope of a text-based dialogue.
Also: True agentic AI is years away - here's why and how we get there
That autonomy can include carrying out several steps in a corporate workflow, such as receiving a purchase order in email, entering it into a database, and consulting an inventory system for availability. Agents have also been used to automate several turns of a customer service interaction in order to replace some of the basic phone or email, or text inquiries a human customer rep would traditionally have handled.
The authors selected agentic AI in three categories: chatbots that have extra capabilities, such as Anthropic's Claude Code tool; web browser extensions or dedicated AI browsers, such as OpenAI's Atlas browser; and enterprise software offerings such as Microsoft's Office 365 Copilot. That's just a taste: other studies, they noted, have covered hundreds of agentic technology offerings.
Most agents, however, "rely on a small set of closed-source frontier models," Staufer and team said. OpenAI's GPT, Anthropic's Claude, and Google's Gemini are what most of these agents are built on.
The good and the bad of agents
The study is not based on testing the agentic tools directly; it is based on "annotating" the documentation provided by developers and vendors. That includes "only public information from documentation, websites, demos, published papers, and governance documents," they said. They did, however, establish user accounts with some of the agentic systems to double-check the actual functioning of the software.
The authors offered three anecdotal examples that go into greater depth. A positive example, they wrote, is OpenAI's ChatGPT Agent, which can interface with websites when a user asks in the prompt for it to carry out a web-based task. Agent is positively distinguished as the only one of the agent systems they looked at that provides a means of tracking behavior by "cryptographically signing" the browser requests it makes.
In its emailed response, OpenAI notes several caveats about Agent. The company states, "We've conducted thousands of hours of red teaming with third parties and have active monitoring in place," and, "No system eliminates all risk, and our safeguards will not stop every attack that emerges as AI agents grow in popularity."
By contrast, Perplexity's Comet web browser sounds like a security disaster. The program, Staufer and team found, has "no agent-specific safety evaluations, third-party testing, or benchmark performance disclosures," and, "Perplexity […] has not documented safety evaluation methodology or results for Comet," adding, "No sandboxing or containment approaches beyond prompt-injection mitigations were documented."
Also: Gartner urges businesses to 'block all AI browsers' - what's behind the dire warning
The authors noted that Amazon has sued Perplexity, saying that the Comet browser wrongly presents its actions to a server as if it were a human rather than a bot, an example of the lack of identification they discuss.
Perplexity told ZDNET that the Comet incidents are grossly mischaracterized. As Perplexity described the findings:
The MCP API and Brave prompt injection entries were vulnerability disclosures sent to our security team. Our vulnerability response process worked as designed: issues were reported through responsible channels, patched quickly, and no users were harmed. This is exactly how bug bounty and disclosure programs are supposed to work. The Amazon entry is a commercial Terms of Service dispute involving no safety failure or data breach. Further, no court has ruled on the matter. Including a pending lawsuit allegation as a documented safety incident treats one party's litigation claims as established fact.
The third example, by Staufer and team, is the Breeze set of agents from the enterprise software vendor HubSpot. Those are automations that can interact with systems of record, such as customer relationship management. The Breeze tools are a mix of good and bad, they found. On the one hand, they are certified for many corporate compliance measures, such as SOC 2, GDPR, and HIPAA.
On the other hand, HubSpot offers nothing when it comes to security testing. It states the Breeze agents were evaluated by third-party security firm PacketLabs, "but provides no methodology, results, or testing entity details."
The practice of demonstrating compliance approval without disclosing actual security evaluations is "typical of enterprise platforms," Staufer and team noted.
Time for the developers to take responsibility
What the report doesn't examine are incidents in the wild, cases where agentic technology actually produced unexpected or undesired behavior that resulted in undesirable outcomes. That means we don't yet know the full impact of the shortcomings the authors identified.
Also: These top 30 AI agents deliver a mix of functions and autonomy
One thing is absolutely clear: Agentic AI is a product of development teams making specific choices. These agents are tools created and distributed by humans.
As such, the responsibility for documenting the software, for auditing programs for safety concerns, and for providing control measures rests squarely with OpenAI, Anthropic, Google, Perplexity, and other organizations. It's up to them to take the steps to remedy the serious gaps identified or else face regulation down the road.
English (US) ·