Anthropic Wants Claude AI to Use Computers the Way You Do

Startup Anthropic is adding a series of upgrades to its Claude AI model — including the ability to interact with computer interfaces much like a human does. 

That ability, which the company calls simply "computer use," could automate complex tasks by teaching the AI the skills you use to interact with the screen, such as moving a cursor, clicking buttons and typing in text. It could also analyze your computer screen and use a range of standard tools and software programs to act on your behalf.

Anthropic emphasized that the tool in its current stage is far from perfect, with some actions in particular, such as scrolling, dragging and zooming, presenting bigger challenges. It's tied to Anthropic's Claude 3.5 model and is currently available only in a public beta.

"At this stage, it is still experimental — at times cumbersome and error-prone," the company wrote in a blog post Tuesday. "We're releasing computer use early for feedback from developers, and expect the capability to improve rapidly over time."

Generative AI tools like Claude, ChatGPT, Copilot and Gemini have been wowing people over the last couple of years with their ability to quickly and (mostly) credibly produce text and images. Now the makers of those tools are looking to take gen AI even further into the realm of human skills. For instance, in September ChatGPT maker OpenAI introduced a new model — called o1 — that's designed to perform analytical "reasoning" to solve complex problems.

In a blog post, Anthropic said the computer use technology will allow developers to better automate repetitive processes, build and test software and conduct open-ended tasks like research.

The effort is part of Anthropic's broader goal to create advanced AI that can teach itself, better align with human needs and push its models beyond a chat box into true AI assistants.

Anthropic said companies including Asana, Canva, Cognition, DoorDash, Replit and The Browser Company are already using the new Claude 3.5 computer use capability to handle tasks with many steps. Replit, for example, is trying out the model's ability to navigate user interfaces, to create a tool that evaluates apps.

But the upgrade also highlights growing safety concerns around AI tools, especially as they might gain more control over sensitive systems like computers and private data. To mitigate risks like misinformation, fraud and misuse, Anthropic said it is implementing safeguards and monitoring for malicious behavior.

Claude 3.5 Sonnet and Claude 3.5 Haiku

Anthropic on Tuesday also talked up more-routine updates to Claude 3.5 Sonnet and introduced a new model, Claude 3.5 Haiku.

Those upgrades to 3.5 Sonnet, available immediately, deliver improved performance overall, and especially in coding and tool use, Anthropic said. The company introduced Claude 3.5 Sonnet earlier this year.

Meanwhile, 3.5 Haiku is the next generation of Anthropic's fastest model, and it will become available later this month. Anthropic said it's "well suited for user-facing products" and for "generating personalized experiences from huge volumes of data — like purchase history, pricing, or inventory records."