A black day for the gaming industry
Futuremark (the makers of 3DMark2003) have confirmed that NVidia is cheating on the benchmark. In addition, they found a probability that ATI was cheating too, although not as badly (or they were just better at it). This calls into question all benchmarks on recent cards. How can I be sure that I can trust the benchmark numbers? All the benchmarks are well known, and obviously it’s worth it for the companies to cheat on the benchmarks. How are we to know how cards really perform? All the higher numbers really show is which company is a better cheater. What we need now is for benchmarkers to play actual games for some amount of time, and give average/best/worst scores for the various cards. That’s a much better benchmark.
This also calls into question all of the scores comparing various processors. It has seemed that various processors “liked” various games better. Did this really mean that the driver cheats on those games worked better on that processor? Suddenly, I can’t really evaluate how good any hardware is for gaming. This is very bad, and could potentially cause an industry wide backlash or even a class action lawsuit.
Take the recent release of the Matrox Paherlia, for example. It scored pretty bad against both the NVidia and ATI cards. But, what if Matrox just wasn’t cheating? The sores were 24% worse on the NVidia cards when Futuremark disabled the driver cheats. This is enough to make up the difference in many of the benchmarks, suddenly making the Parhelia a competative card in real games, rather than in benchmarks. But, no-one is buying the Parhelia, because it scored so low.
I’m personally not planning on buying a new graphics card for 4-6 months. Hopefully, this will be enough time for the benchmarkers to find a way to do non-cheating benchmarks, and enough for NVidia and ATI to remove the various cheats from their drivers, and maybe I’ll be able to actually decide what card to buy. The only problem is that I probably won’t be buying a card that has come out in that time, so all I’ll have to work with is likely old benchmarks, all of which have to be assumed invalid. I’m not sure what I’ll do. Maybe I can buy one of each card, and return the one that performs worse on the games I care about? I don’t know. All I can say is that my faith in the industry is shaken. At least with a console, I know what I’m getting…
Discussion Area - Leave a Comment