Huawei unveils Claw-Anything benchmark, revealing AI agents’ limitations in personal assistant tasks

Here’s a humbling number for the AI hype cycle: GPT-5.5, one of the most advanced language models on the planet, scored just 34.5% when asked to function as an always-on personal assistant in a realistic digital environment. Claude Opus 4.7 fared even worse at 31.8%.

Those results come from Claw-Anything, a new benchmark published by Huawei researchers in collaboration with Beijing Institute of Technology and Peking University. The paper, released on May 25, 2026, doesn’t just test whether AI can answer questions. It tests whether AI can actually run your digital life.

What Claw-Anything actually measures

The benchmark simulates a complete digital life, then asks AI assistants to manage it across long-horizon event streams and multiple interdependent backend services. Instead of asking the AI to summarize an email, you’re asking it to monitor your inbox, calendar, messaging apps, and file systems simultaneously, then take appropriate action without being told to.

The complexity is substantial. Tasks involve an average of 10.1 interdependent services, with some scenarios reaching up to 18. The benchmark includes 200 human-verified task environments with an average of 191.7k context words per environment.

The benchmark evaluates both graphical user interface and command line interface interactions across multiple devices. It also tests proactive behavior: can the AI notice something needs doing before you ask?

The training pipeline offers a glimmer of hope

The research team built an automated pipeline that generated 2,000 training environments for fine-tuning AI models on these complex assistant tasks. Qwen3.5-27B, a smaller open-source model, showed a 23.7% performance improvement after being fine-tuned on successful task trajectories from these environments.

ClawBench and WildClawBench, which test similar multi-step practical tasks within the broader OpenClaw ecosystem, show top AI models scoring somewhere between 33% and 62%.

Why crypto investors should pay attention

The 34.5% pass rate for GPT-5.5 is particularly notable because many crypto AI projects are built on top of OpenAI’s models. The fine-tuning results with Qwen3.5-27B suggest that specialized training on domain-specific successful trajectories can meaningfully improve performance. That means the crypto AI projects most likely to deliver real value are probably the ones investing heavily in curating high-quality training data from actual on-chain interactions.

Huawei’s involvement in open-source AI benchmarking, alongside the broader OpenClaw framework, signals that the race to build reliable AI assistants is increasingly global. The benchmark specifically tests the kind of complex, multi-step, multi-service coordination that crypto AI agents would need to perform reliably: managing DeFi portfolios across multiple protocols, monitoring governance proposals, rebalancing based on market conditions, and bridging assets between chains.

Disclosure: This article was edited by Editorial Team. For more information on how we create and review content, see our Editorial Policy.