Notes on the Graphs
Below is a list of factors to take in consideration when viewing or using the benchmark graphs.
Results are averages and vary over time
While we do perform some in house benchmarking, nearly all of the results in the graphs are averages of data sent to us by users of the PerformanceTest software. Approximately 500 new benchmark results arrive daily and each night we incorporate all the new results into the graphs. This means that the CPUMark values for each CPU model will vary over time. Popular older models have very stable scores but newer models can vary a lot, especially during the first couple of weeks following their release on the market. If you are planning on using the results as part of a tender, then either ask your suppliers to run the benchmark software on the exact hardware you are getting or take a print-out of the graphs on the day the tender is released (to avoid the values changing over time).
Minimum Sample Size
For a CPU type to be included on the charts it needs to have had at least one submitted results. (For the overclocked CPU chart, the minimum sample size required is 5. This is to help avoid a single really good or really bad result from throwing a CPU type into a completely incorrect place in the chart.) Due to the final score being an average of all benchmarks submitted for that CPU it is possible for one or more incorrect results to throw off the whole result when there is a small sample size. For this reason we have included information in the graph about how many CPUs of that type have been benchmarked and the margin for error.
For CPUs with...
- ...5 or less submissions, it is classified as High margin for error.
- ...20 or less submissions, it is classified as Medium margin for error.
- ...21 or more submissions, it is classified as Low margin for error.
To see this information simply move the mouse over the bar of the CPU you are interested in. A higher sample size means the data is more likely to be accurate.
Overclocking is when a CPU is made to run faster than it was designed for. Often people who like to test their computers and submit results are also the kind of people who like to overclock their computers. As such there are several benchmark results where the CPU has been overclocked.
In the past, it is easy to see when some of the CPUs in the sample have been overclocked as the average CPU speed will be higher than the design speed. However, with the introduction of the Turbo Boost feature in Intel i5 and i7 CPUs new ways of overclocking are now available. It has become more difficult to find all these results. (It is also much harder for AMD CPU’s where the CPU name does not explicitly state the speed it is meant to run at.) As much as possible we have separated out the overclocked results, compiling them into a separate overclocked CPU chart. Follow the forum post for further discussions on determining how a CPU is considered overclocked.
To help viewers of the chart spot the performance increase from overclocking, we have included the the average performance increase in the tool tip for the overclocked chart. For the purposes of the graphs overclocked CPUs a CPU is considered overlocked if the measured CPU speed or turbo speed is greater than 90Mhz difference from the factory base speed or maximum turbo speed respectively.
64-Bit and 32-Bit
There are 64bit CPUs and 32bit CPUs. There are also 64bit operating systems and 32bit operating systems. 64bit CPUs running a 64bit OS tend to have slightly higher performance than both 32bit CPUs and 64bit CPUs running a 32bit OS. Results from both 32bit and 64bit operating systems are mixed together in the charts. 64bit machines tend to get higher CPU results when running native 64bit applications partly because more RAM is available but mostly because they process large numbers quicker due to a more efficient instruction set and due to additional registers.
Different Operating Systems and different environments
The PerformanceTest software is designed to run on several different versions of Windows. These different OS have different levels of efficiency and as such a specific computer might perform better or worse depending on what OS is installed. Many of these results were submitted by our users. In some cases some of the PC's might have had configuration issues leasing to sub-optimal results. When a large number of samples are available this is not an issue but when only a small number of results are available it can sway the results.
In the case of CPUs that support hyperthreading the graph gives no indication as to how many of the tests were performed with it enabled. Typically CPU’s will perform slightly better with hyperthreading enabled.
Real Life Performance Comparison
The rating the CPU’s are given here represents their peak performance and not necessarily there real world performance with any specific software application. This is especially true in the case of CPU’s with multiple cores or benchmarks of systems with multiple physical CPUs. While PerformanceTest was designed to make use of all cores/CPU’s at once, many real world applications are not designed with this in mind, especially older applications. Unlike single core/CPU systems these systems will only show their full potential in high-performance situations such as Web-serving or when heavily multitasking, the average user might not see nearly as much improvement from them.
Naming of CPUs
Due to the fact that these graphs are automatically generated the names of the CPUs have been taken straight from the CPUs themselves. However, some older CPUs did not include very much detail about themselves or their specific type. For instance there is one CPU type simply listed as "Pentium III". This type comprises several different models of Pentium III's across a range of different speeds. Again the average CPU speed listed in the tool-tip can give a good guide on how to interpret these results.
In some cases we have picked up names of rare or low volume CPUs. Just because you haven't heard of a particular CPU type, doesn't mean it doesn't exist. For example, some people claim there was no such thing as a Pentium 5. But it seems there was, it was just an early release of the Pentium D CPUs, that was only on the market for a very short time.
Version of PerformanceTest
The results have been collected over several years using PerformanceTest V7.0., V8.0 and V9.0
Duplication of CPU types in the graph
Some people have contacted us to tell that we have messed up and duplicated various CPUs in the graph . To the best of our knowledge this is not the case. What we have done is separate out CPUs by their model, their clock speed and the number of CPUs in the system being tested. So a Dual CPU Intel Core 2 2.6Ghz system, is on a different line in the graph from a Single CPU Core 2 2.6Ghz system. This is done deliberately so that you can get some idea of the performance difference between dual and single CPU systems. (note that 'dual CPU' is NOT the same as 'dual core').
In other cases, there may appear to be a duplication, such as with the Intel Core2 Quad CPU @ 2.40GHz. and Intel Core 2 Quad CPU Q6600 @ 2.40GHz.. In this case we believe Intel release two different versions of this same CPU with slightly different names (Intel part numbers SL9UM (B3) and SLACR (G0)). Savvy purchasers got the G0 version, as it ran cooler and could be overclocked higher.