Performance Comparison of iOS Devices
Benchmark results ("Baselines") were gathered from users' submissions to the PassMark web site as well as from internal testing. PerformanceTest Mobile conducts 17 different tests spanning five standard test suites (CPU, Memory, Disk, 2D & 3D Graphics). The results are combined to determine the PassMark rating for a device. The PassMark rating takes into account the results from all five standard test suites and provides a general overall system rating.
The CPUMark, which is the result from the CPU test suite, is the biggest contributor to the overall PassMark rating. The CPUMark offers insight into the performance the CPU of the device being tested. Since the PassMark rating takes into account varied device setup (i.e. external memory cards), some user may find that the CPUMark can offer a better gauge of raw performance.
To ensure that the full CPU power of a device is realized, PerformanceTest Mobile runs each CPU test on all available CPUs/Cores. Specifically, PerformanceTest Mobile runs one simultaneous CPU test for every logical CPU; physical CPU core (dual core) or physical CPU package (multiple CPU chips). So hypothetically, if you have a device that has two CPUs, each with dual cores then PerformanceTest Mobile will run four simultaneous tests.
The following list describes each of the tests performed.
The Integer Math Test aims to measure how fast the CPU can perform mathematical integer operations. An integer is a whole number with no fractional part. This is a basic operation in all computer software and provides a good indication of 'raw' CPU throughput. The test uses a large set of random 32-bit and 64-bit integers and adds, subtracts, multiplies and divides these numbers.
The Floating Point Math Test performs the same operations as the Integer Math's Test however with floating point numbers. A floating point number is a number with a fractional part (i.e. 12.568). These kinds of numbers are handled quite differently in the CPU compared to Integer numbers as well as being quite commonly used, therefore they are tested separately.
The Prime Number Test aims to test how fast the CPU can search for Prime numbers, reported as operations per second. A prime number is a number that can only be divided by itself and 1. For example 1, 2, 3, 5, 7, 11 etc. This algorithm uses loops and CPU operations that are common in computer software, and determines the speed at which numbers can be compared with other numbers, the speed large floating numbers (eg. 26739754.5369) can be manipulated and the speed complex computations like square root can be performed.
The String Sorting Test uses the qSort algorithm to see how fast the CPU can sort strings. A very common task in many applications.
The Encryption Test encrypts blocks of random data. For Versions V0.4.2 and earlier, PerformanceTest Mobile App uses the Blowfish algorithm. For later versions, the App uses iOS CommonCrypto AES128 implmentation. Encrypting data only allows the resulting data to be accessed by someone with the encryption key. This test uses many of the techniques in the math tests, but also uses a large amount of binary data manipulation and CPU mathematical functions like 'to the power of'. Encryption is a very useful benchmark, as it is now very widely used in software applications, ranging from Internet browsers, communications software and many different business applications.
The Compression Test measures the speed that the CPU can compress blocks of data into smaller blocks of data without losing any of the original data. The result is reported in Kilobytes per Second. This test uses complex data structures and complex data manipulation techniques to perform a function that is very common in software applications, ranging from backup software to Email software. The compression test uses an Adaptive encoding algorithm based on a method described by from Ian H. Witten, Radford M. Neal, and John G. Cleary in an article called "Arithmetic Coding for Data Compression". The system uses a model which maintains the probability of each symbol being the next encoded. It reports a compression rate of 363% for English text, which is slightly better than the classic Huffman method.
For each of the disk tests, a test file is created in the respective storage location. Each test file is 90MB in size with read and write block size set to 16KB. If there is not enough space on the device, the test is skipped. All file I/O operations are sequential. Due to the variability seen for disk tests, its contribution to the System Rating is weighted less in the overall score.
On iOS, the read test uses the method readDataOfLength: (NSFileHandle) to access the the file. The write test uses writeData: (NSFileHandle) to write to the file.
For both the memory read and write test, the test will take one quarter of available physical memory. The actual size of the memory allocated is determined by the amount of ram currently available. A minimum allocation of 16MB is requred for the test to be preformed. If the system does not have enough available ram, the test is skipped. The read and write block size used is 16KB. All I/O operations are sequential.
The Solid Vectors test draws a rectangle filled with a gradient, a solid filled ellipse, a line, and Bezier and a solid filled star as many times as possible. Every 32nd cycle the colors are randomized.
The Transparent Vectors test is similar to the Solid Vectors test however all color fills are partially transparent.
The Complex Vectors test draws a jungle scene using a number of complex vector images of trees and animals. This scene will be drawn as many times as possible within the time limit and always finishes on a complete drawing of the scene.
The Image Rendering test measures the rate at which a bitmap image can be rendered to the screen at various sizes. The test always finishes on a complete cycle of rendering smallest to largest.
The Image Filters test applies built-in rotation, brightness and greyscale filters to an image. Counts the number of filters applied per second.
Twenty bouncing balls are presented in the scene. Each ball has an unique texture and each ball object has 840 vertices and 838 triangular polygons. Lighting is enabled.
Three helicopters travel around the scene featuring 150 trees on a terrain with water. The terrain is formed with 5,000 triangular polygons (15,000 vertices). The trees and water consist of 2 polygons (6 vertices) and each helicopter object consists of 1,120 triangular polygons (3,360 vertices). Lighting & alpha blending are enabled.