AI lab staff need leadership backing to make oversight work, Bloomberg says

Human oversight of artificial intelligence systems can fail when reviewers lack the time, context and authority needed to identify subtle errors, Bloomberg reported.

Former Google DeepMind researchers Rishub Jain and Joshua Jacob have raised $6.5 million to build a hybrid human-AI “judge” intended to oversee models and reduce the risk of them going rogue.

Their nonprofit, Sampura Research, is based on work suggesting that human-AI oversight can outperform AI-only systems, although the improvement was modest. The researchers plan to compare their system with AI-only models and human reviewers working alone.

A DeepMind employee told Bloomberg that engineers and researchers are spending less time understanding code and more time verifying AI outputs. The employee said pressure to move quickly could erode the careful review needed to detect hidden errors.

More than 1,100 employees at OpenAI, Anthropic, Google and Meta have signed a letter asking the government to regulate the pace of AI development. The signatories argued that individual labs cannot slow development on their own.

Sarah Myers West of the AI Now Institute said human-in-the-loop systems can place responsibility on reviewers who have little context and little time to evaluate an AI system. She compared that risk with content moderation, where overworked staff can absorb blame for automated decisions.

Jain left Google after the company did not support the work he wanted to pursue. Bloomberg reported that the broader question is whether AI companies will give safety reviewers the working conditions and authority needed to challenge the systems they oversee.

Disclosure: This article was edited by Editorial Team. For more information on how we create and review content, see our Editorial Policy.