Geekbench 6 warns about inconsistent benchmarking performance from new Core Ultra 200S Plus chips — says Intel's IPC boosting Binary Optimization Tool modifies scores in 'unclear' fashion

One of the most noticeable upgrades Intel made to its latest Core Ultra 7 270K Plus and 250K Plus CPUs was the introduction of its Binary Optimization Tool that manipulates instructions at the hardware level to boost IPC. The tool is highly beneficial for squeezing extra performance out of the Arrow Lake architecture, but it has led to concerns over benchmarking accuracy and consistency with these chips. John Poole from Geekbench posted a warning to its users that Intel's latest tool can't be trusted at this time, and there's no way to identify when the tool is enabled or disabled during a benchmark run.

Pool revealed that Intel does not have any public documentation on the techniques the Binary Optimization Tool (or iBOT) uses to optimize code, making it difficult to determine how effective iBOT's techniques are when applied to a variety of different applications. Furthermore, this problem makes it impossible for Primate Labs (the makers of Geekbench) and its userbase to understand how iBOT is boosting performance compared to benchmarks that run without it. According to Poole, Geekbench 6 workload scores on the chips increase by up to 40% with iBOT enabled, with overall scores improving by up to 8%. "Since the tool modifies the benchmark, and it is unclear to both Primate Labs and the general public how these changes occur," he warned.

To deal with this problem, Geekbench will provide a warning on all Geekbench benchmark listings featuring iBOT-supported chips with the following description: “This benchmark result may be invalid due to binary modification tools that can run on this system.”

Article continues below

Unfortunately, the Geekbench developers don't have any other choice but to treat iBOT in this manner. Benchmarking consistency from run to run is very important for both users and the teams that create benchmarking apps, such as Geekbench 6, and losing this capability removes a benchmarking application's usefulness and credibility.

Thankfully, Intel is aware of these discrepancies; Intel told Jake Roach, one of Tom's Hardware's editors and main CPU reviewer, that it is cautious about rolling out the feature and wants to avoid any claims of playing dirty tricks to look better on benchmarks. Currently, iBOT is supported only in a handful of PC games, with the exception being Geekbench 6.3. Intel is using iBOT in Geekbench 6.3 as a proof of concept for how iBOT could potentially perform in non-gaming workloads. For instance, in our review of the 270K Plus, we tested iBOT in Geekbench 6.3 and saw performance improve by around 10% compared to the 265K.

The Core Ultra 7 270K Plus and Core Ultra 5 250K Plus debuted this week as potentially Intel's last hurrah for the LGA 1851 platform before Nova Lake arrives later this year. In our reviews of both chips, we found both make significant improvements on the lackluster Core Ultra 200S series parts that preceded them. Pricing for both SKUs is substantially cheaper than their non-Plus counterparts, while providing noticeably superior multithreading performance thanks to the addition of four extra E cores.

Follow Tom's Hardware on Google News, or add us as a preferred source, to get our latest news, analysis, & reviews in your feeds.