- Tech Junkie - https://www.techjunkie.com -

2015 VM Benchmarks: Parallels 11 vs. Fusion 8 vs. VirtualBox 5

We’re back with the final component of our yearly analysis of virtualization software for OS X. We’ve already looked at the recently updated offerings from Parallels [1] and VMware [2], and now it’s time to compare them directly.

Today, we’ll be looking at a performance comparison of Parallels Desktop 11 [3], VMware Fusion 8 [4], and Oracle’s VirtualBox 5 [5]. Although Parallels and Fusion are more popular options for OS X users looking to run Windows and other x86 operating systems on their Macs, we always like to keep an eye on VirtualBox to see just how well this free open source alternative can keep up with its commercial competitors.

Part of our goal with this analysis is not to simply determine which virtualization solution is the fastest, we also want to see how they compare to “native” Windows performance on the same hardware. We’ve therefore run all applicable tests in Boot Camp as well, which gives us an idea of how close these options are to eliminating the need for something like Boot Camp altogether, at least for certain tasks.

Also new this year is the addition of a “high end” host for some tests. As we’ll describe in more detail in the next section, all of our tests were performed on 2014 15-inch MacBook Pro, a system that we consider to be in the “mid-to-high” range of Mac configurations. But we were also curious about just how well Fusion and Parallels would perform if given access to clearly “high-end” resources. We therefore ran select CPU- and GPU-focused tests on a 2013 Mac Pro, and we have those numbers available in their own dedicated section later on.

Our benchmark tests and results are divided into the sections identified below. You can browse all results in order by using the “Next” and “Previous” buttons below, or you can jump directly to a specific test using the Table of Contents, which is found at the bottom of every page. Some tests required that we cram a lot of data into a single chart, and some of these charts may be difficult to read on smaller or low-resolution screens. To see any chart in its full-sized Retina glory, just click or tap on it to load the full image.

Table of Contents

[one_half padding=”0 5px 20px 0″]
1. Introduction
2. Test Setup & Methodology [40]
3. Geekbench [41]
4. 3DMark [42]
5. FurMark OpenGL [43]
6. Cinebench R15 [44]
7. PCMark 8 [45]
8. Passmark PerformanceTest [46]
[/one_half]

[one_half_last padding=”0 0px 20px 5px”]
9. Video Encoding [47]
10. File Transfers [48]
11. USB 3.0 Speed [49]
12. Virtual Machine Management [50]
13. Battery Life [51]
14. Mac Pro: Gaming [52]
15. Mac Pro: CPU [53]
16. Conclusions [54]
[/one_half_last]

parallels vs fusion test setup [55]

Test Setup & Methodology

As mentioned in the introduction, we ran our Parallels 11 vs. Fusion 8 benchmarks on two hardware platforms: a MacBook Pro and a Mac Pro. The MacBook Pro is a Mid-2014 15-inch model, with a 2.5GHz Intel Core i7-4870HQ [56], 16GB of RAM, 2GB NVIDIA GeForce GT 750M, and 512GB PCIe flash storage. The Mac Pro is a Late 2013 model with a 6-core Intel Xeon E5-1650 v2 [57], 64GB of RAM, dual 3GB AMD D500 GPUs, and 256GB PCIe flash storage. The MacBook Pro was our primary test platform, used for all results unless otherwise noted.

Our host operating system for all virtualized tests was OS X Yosemite 10.10.5, the latest publicly available version as of the date of this article. Even though both Fusion 8 and Parallels 11 support OS X El Capitan, we’re reluctant to perform tests on beta software where bugs or future changes could play a factor in skewing the results. We’ll revisit El Capitan once it launches later this year and we’ll let you know if it provides any performance boosts that would alter the benchmark results reported here.

Our guest operating system for all tests was Windows 10 Pro 64-bit, which was installed separately in four configurations: native to the Mac’s hardware via Boot Camp, in a virtual machine powered by Parallels 11, in a virtual machine powered by Fusion 8, and in a virtual machine powered by VirtualBox 5.

Regarding our decision to use the 64-bit version of Windows, it’s true that the 32-bit version can be easier to virtualize and therefore may offer slightly better performance in certain circumstances. Unfortunately, the latest version of Boot Camp requires the use of a 64-bit version of Windows, so we were forced to use 64-bit VMs as well in order to maintain consistency between the platforms.

As for our virtualization software, we used the latest available version of all applications, which at the time of publication is Fusion 8.0.0, Parallels 11.0.0, and VirtualBox 5.0.3, which is admittedly a test build but is required to provide Windows 10 compatibility.

Each of our Parallels and Fusion Windows 10 virtual machines was configured for maximum performance, with 8 assigned virtual CPUs, 12GB of RAM (the maximum recommended amount in order to ensure that enough is reserved for OS X), and 1GB of graphics memory configured for each platform’s most advanced DirectX and OpenGL graphics support. The VirtualBox VM was also configured to its maximum supported hardware level, with matching CPU and RAM, but with only 256MB of graphics memory, the most it allows. For all platforms, all features that could possibly impact performance, such as error logging or an expanding virtual disk, were disabled.

All operating systems and testing software were updated to their most recent versions as of the date of this article. More information about each benchmark application or test can be found on its respective results page.

As is standard practice here at TekRevue, all tests, unless otherwise noted in the results, were performed three times for each Windows installation, and the results were averaged. Our normal procedure in the event of a discrepancy greater than 5 percent is to re-run the tests until the issue can be identified. The only test that required this additional analysis was battery life, and the issue is described further in that section.

Table of Contents

[one_half padding=”0 5px 20px 0″]
1. Introduction [58]
2. Test Setup & Methodology

3. Geekbench [41]
4. 3DMark [42]
5. FurMark OpenGL [43]
6. Cinebench R15 [44]
7. PCMark 8 [45]
8. Passmark PerformanceTest [46]
[/one_half]

[one_half_last padding=”0 0px 20px 5px”]
9. Video Encoding [47]
10. File Transfers [48]
11. USB 3.0 Speed [49]
12. Virtual Machine Management [50]
13. Battery Life [51]
14. Mac Pro: Gaming [52]
15. Mac Pro: CPU [53]
16. Conclusions [54]
[/one_half_last]

parallels vs fusion benchmarks geekbench [59]

Geekbench

Our Parallels 11 vs. Fusion 8 benchmarks start with the popular cross-platform benchmarking tool, Geekbench [60]. Available for virtually every modern computing platform — Windows, OS X, Linux, Android, and iOS — Geekbench aims to provide universally comparable scores of relative performance across multiple device types. It’s important to note, however, that Geekbench only tests a device’s CPU and memory performance, and doesn’t look at other important areas such as graphics or storage.

We ran the 64-bit benchmark test three times on each Windows installation using Geekbench 3.3.2, the latest version [61] as of the date of publication. Geekbench reports two sets of results: one for single-core performance and one for multi-core performance. We’ll start with single-core results, below:

parallels vs fusion geekbench single core [62]

If you read our earlier benchmarks focused on Parallels 11 [63] and Fusion 8 [64] individually, you know that CPU performance is quite good, with both products able to offer near-native speed in the types of single-core tests run by Geekbench. With the race close, Fusion 8 pulls out a narrow victory in the CPU-bound tests by scoring between 1 and 2.5 percent higher than Parallels 11, although Parallels enjoys about a 3 percent advantage in the memory test.

parallels vs fusion benchmarks geekbench multi-core [65]

Multi-core Geekbench results reveal a similar trend, with Fusion 8 narrowly winning in all categories. The most notable aspect of this test is how poorly optimized VirtualBox is for multi-core performance. Although configured with the same virtual resources as Parallels and Fusion, and able to post respectable numbers in the single-core test above, VirtualBox falls far behind when its vCPU is put to the test.

Table of Contents

[one_half padding=”0 5px 20px 0″]
1. Introduction [58]
2. Test Setup & Methodology [40]
3. Geekbench
4. 3DMark [42]
5. FurMark OpenGL [43]
6. Cinebench R15 [44]
7. PCMark 8 [45]
8. Passmark PerformanceTest [46]
[/one_half]

[one_half_last padding=”0 0px 20px 5px”]
9. Video Encoding [47]
10. File Transfers [48]
11. USB 3.0 Speed [49]
12. Virtual Machine Management [50]
13. Battery Life [51]
14. Mac Pro: Gaming [52]
15. Mac Pro: CPU [53]
16. Conclusions [54]
[/one_half_last]

fusion 8 benchmarks 3dmark header [66]

3DMark

Futuremark’s 3DMark [67] is the latest in a long line of industry-standard gaming benchmark suites. 3DMark offers a range of tests that can evaluate the performance of everything from a low-power tablet to a $15,000 quad-SLI gaming PC, and it looks at a CPU’s ability to handle gaming-related physics calculations in addition to pure GPU performance.

3DMark offers multiple tests of increasing complexity that support DirectX 9, DirectX 10, and DirectX 11. However, Fusion 8 and Parallels 11 only support up to DirectX 10, allowing us to run only the entry-level and mid-range 3DMark tests. Of note, VirtualBox technically supports up to DirectX 11, and its graphics driver reports that compatibility to the guest OS, enabling access to the higher-end tests. Graphics performance, however, is so poor in VirtualBox at these higher levels that it is unusable for any practical purpose.

We’ll start with Ice Storm, the lowest-end 3DMark test that uses DirectX 9 APIs.

parallels vs fusion benchmarks 3dmark ice storm [68]

As we noted in our Fusion 8 benchmarks [69] earlier this week, VMware made graphics performance a major goal of this new release, and that work pays off with Fusion 8 enjoying a significant lead over Parallels 11 in GPU-focused portions of the Ice Storm test. The CPU-bound physics test, however, reveals the much closer competition between these two products, with Parallels narrowly beating Fusion and both approaching near-native performance.

The next test, Ice Storm Extreme, is another DirectX 9 benchmark that runs the same basic script as the standard Ice Storm test above, but increases the resolution to 1080p and utilizes higher quality textures and lighting effects, making it harder overall on the GPU.

parallels vs fusion benchmarks 3dmark ice storm extreme [70]

When confronted with a more demanding DirectX 9 scenario, Fusion 8 still easily wins, but sees its margin over Parallels 11 fall from about 46 percent to just under 25 percent. Again, we see that Parallels has a slight edge over Fusion when it comes to CPU-focused physics calculations.

The Cloud Gate benchmark is a DirectX 10 test, and the most advanced 3DMark benchmark that’s compatible with our virtualization software.

parallels vs fusion benchmarks 3dmark cloud gate [71]

As expected, neither platform offers anywhere near native performance when it comes to the still-developing DirectX 10 APIs, but Parallels’ implementation of the feature gives it about a 3 percent lead over Fusion 8.

3DMark06

The introduction of DirectX 10 support in Fusion 8 means that we no longer need to rely on older benchmarks in order to have cross-compatibility between the various Parallels and Fusion updates. Since we included it in our individual looks at Fusion 8 and Parallels 11, however, here’s how the two compare in 3DMark06 [72] performance, although this will likely be the last year that this benchmark is included in our testing suite.

parallels vs fusion benchmarks 3dmark06 [73]

The performance margins are much closer in 3DMark06 than they were in the Ice Storm tests, with Fusion performing better in terms of pure graphics power and Parallels taking the lead in CPU-focused tests. Of note, our VirtualBox VM would not complete the 3DMark06 test, and so it is omitted from the results.

Table of Contents

[one_half padding=”0 5px 20px 0″]
1. Introduction [58]
2. Test Setup & Methodology [40]
3. Geekbench [41]
4. 3DMark
5. FurMark OpenGL [43]
6. Cinebench R15 [44]
7. PCMark 8 [45]
8. Passmark PerformanceTest [46]
[/one_half]

[one_half_last padding=”0 0px 20px 5px”]
9. Video Encoding [47]
10. File Transfers [48]
11. USB 3.0 Speed [49]
12. Virtual Machine Management [50]
13. Battery Life [51]
14. Mac Pro: Gaming [52]
15. Mac Pro: CPU [53]
16. Conclusions [54]
[/one_half_last]

parallels vs fusion benchmarks furmark opengl [74]

FurMark OpenGL Benchmark

OpenGL support has been relatively limited in previous versions of Fusion and Parallels, but Fusion 8 surprised us with the inclusion of OpenGL 3.3 support, up from version 2.1 last year. We’ve therefore added a new benchmark to our test suite: FurMark [75]. FurMark is a demanding OpenGL benchmark and stress test that measures GPU performance and OpenGL compliance. It offers several benchmark presets based on resolution, but due to the nature of our mobile hardware and the limited GPU capabilities in our VMs, we stuck with the 720p and 1080p tests. Results are reported as a proprietary “score” with a higher number equaling better performance.

parallels vs fusion benchmarks furmark opengl [76]

Fusion 8’s improved graphics performance and OpenGL support give it a big lead over Parallels 11, scoring 28 percent higher at 720p and 48 percent higher at the more demanding 1080p resolution. For those who need to use virtualized OpenGL-based applications, it’s worth noting that Fusion 8 is only about 5-10 percent slower than native performance.

Table of Contents

[one_half padding=”0 5px 20px 0″]
1. Introduction [58]
2. Test Setup & Methodology [40]
3. Geekbench [41]
4. 3DMark [42]
5. FurMark OpenGL
6. Cinebench R15 [44]
7. PCMark 8 [45]
8. Passmark PerformanceTest [46]
[/one_half]

[one_half_last padding=”0 0px 20px 5px”]
9. Video Encoding [47]
10. File Transfers [48]
11. USB 3.0 Speed [49]
12. Virtual Machine Management [50]
13. Battery Life [51]
14. Mac Pro: Gaming [52]
15. Mac Pro: CPU [53]
16. Conclusions [54]
[/one_half_last]

cinebench benchmark [77]

Cinebench R15

We continue our Parallels 11 vs. Fusion 8 benchmarks with a look at the latest version of the Cinebench test. Based on the powerful Cinema 4D [78] animation software, Cinebench R15 [79] is a cross-platform benchmark that tests a computer’s 3D rendering capabilities.

Cinebench looks at three primary categories: multi-thread CPU rendering, single-thread CPU rendering, and OpenGL 3D performance. There’s a problem with that last category, however, as there’s a bug in Fusion 8 that produces abnormally low results, and the resulting score of the test is not indicative of actual OpenGL performance. We’ve asked VMware about this bug and they told us that their priority for this release was to optimize OpenGL performance for productivity applications, but they’re investigating the cause of the issue in Cinebench. While we don’t think that this Cinebench bug as it stands today is an indictment of Fusion’s potential overall, it does suggest that users who are interested in OpenGL rendering via apps like Cinema 4D may wish to test performance via the Fusion and Parallels free trial programs to ensure adequate performance with their required apps.

All of that said, the CPU-based rendering tested by Cinebench performed without issue in all platforms. The scores in the chart below are based on comparative Cinebench “points,” with a higher number indicating better performance.

parallels vs fusion benchmarks cinebench [80]

All virtualization platforms score relatively well in single-core rendering, and Parallels and Fusion are within 5 percent of native performance. When it comes to multi-core rendering, however, Fusion 8 again scores extremely close to native performance, while Parallels isn’t quite as efficient and falls about 10 percent short. As we saw earlier in the Geekbench tests, VirtualBox just can’t compare to its commercial competitors when it comes to multi-core efficiency, and trails Fusion by about 20 percent.

Table of Contents

[one_half padding=”0 5px 20px 0″]
1. Introduction [58]
2. Test Setup & Methodology [40]
3. Geekbench [41]
4. 3DMark [42]
5. FurMark OpenGL [43]
6. Cinebench R15
7. PCMark 8 [45]
8. Passmark PerformanceTest [46]
[/one_half]

[one_half_last padding=”0 0px 20px 5px”]
9. Video Encoding [47]
10. File Transfers [48]
11. USB 3.0 Speed [49]
12. Virtual Machine Management [50]
13. Battery Life [51]
14. Mac Pro: Gaming [52]
15. Mac Pro: CPU [53]
16. Conclusions [54]
[/one_half_last]

pcmark 8 [81]

PCMark 8

Whereas the earlier 3DMark benchmarks are focused on gaming, Futuremark’s PCMark [82] benchmark aims to measure overall system performance in a variety of “real world” usage scenarios. The test automates tasks such as Web browsing, video chatting, creating complex spreadsheets in Excel, and editing images in Photoshop. We conducted three of PCMark’s tests: the Home test, the Microsoft Office test, and the Adobe Creative Cloud test, hoping to give us a look at some workflows that are more commonly virtualized. The results are reported in arbitrary “points,” with a larger number of points representing better performance.

Of note, PCMark by default runs each test three times per benchmark cycle, so each test was only manually initiated once per Windows installation.

parallels vs fusion benchmarks pcmark 8 [83]

With PCMark, we see how close in performance Parallels 11 and Fusion 8 truly are when it comes to real world usage scenarios. In the Microsoft Office test, which focuses on creating and modifying documents and spreadsheets in Word, Excel, and PowerPoint, Parallels scores about 2.5 percent higher, while the scores for the Creative Cloud test, which looks at Photoshop, InDesign, Illustrator, and After Effects, are effectively equal (a 0.2 percent difference).

There is a wider gap (about 17 percent) between Parallels and Fusion in the Home test. This test includes portions of light 3D gaming, and Fusion’s lead here is primarily due to its superior performance this year with games, as we saw in the earlier 3DMark and FurMark tests. VirtualBox unsurprisingly takes a distant last place in all three tests, owing mostly to its poor 3D capabilities and low efficiency when scaling up to multithreaded workloads.

Table of Contents

[one_half padding=”0 5px 20px 0″]
1. Introduction [58]
2. Test Setup & Methodology [40]
3. Geekbench [41]
4. 3DMark [42]
5. FurMark OpenGL [43]
6. Cinebench R15 [44]
7. PCMark 8
8. Passmark PerformanceTest [46]
[/one_half]

[one_half_last padding=”0 0px 20px 5px”]
9. Video Encoding [47]
10. File Transfers [48]
11. USB 3.0 Speed [49]
12. Virtual Machine Management [50]
13. Battery Life [51]
14. Mac Pro: Gaming [52]
15. Mac Pro: CPU [53]
16. Conclusions [54]
[/one_half_last]

Passmark PerformanceTest 8.0

Similar to PCMark, Passmark PerformanceTest 8.0 [84] is another benchmark that aims to take an overall look at system performance. It measures CPU, 2D graphics, 3D graphics, and memory performance to calculate an overall Passmark rating, which forms the basis of the Passmark benchmark database [85]. Results are reported both as an overall score and as individual scores within each testing category. A higher score equals better performance.

Note that our VirtualBox VM would not initiate ether of the Passmark graphics tests, and crashed during the CPU test, leaving us with little in the way of meaningful results. VirtualBox is therefore omitted from the chart below.

parallels vs fusion benchmarks passmark [86]

Parallels 11 and Fusion 8 achieve similar scores in all tests except 3D graphics, and this gap is the primary factor in Parallels’ overall score advantage. Unlike some benchmarks that end immediately if the system being tested fails or cannot complete a certain part of the benchmark, the Passmark test will allow the benchmark process to continue and merely penalize the score of the system in the failed or incomplete test. In the case of the Passmark 3D graphics test, the Fusion 8 VM wouldn’t enable anti-aliasing when the test called for it, and so Passmark downgraded Fusion’s score, which is the main cause of the difference.

It should be noted that Passmark PerformanceTest also includes a disk test, but we have excluded it from these charts due to an issue that incorrectly deflates disk performance in Boot Camp and our observation of wild inconsistencies in the disk numbers reported for our virtual machines with, for example, one test reporting a result five times higher than a subsequent test, even when all efforts were made to disable or minimize OS caching that could interfere with the test. For this reason, we feel that the Passmark PerformanceTest disk benchmark is unreliable for testing purposes, and we’ve therefore omitted it from our result chart. Those interested in disk performance can check out our dedicated File Transfer testing [48] on page 10.

Table of Contents

[one_half padding=”0 5px 20px 0″]
1. Introduction [58]
2. Test Setup & Methodology [40]
3. Geekbench [41]
4. 3DMark [42]
5. FurMark OpenGL [43]
6. Cinebench R15 [44]
7. PCMark 8 [45]
8. Passmark PerformanceTest
[/one_half]

[one_half_last padding=”0 0px 20px 5px”]
9. Video Encoding [47]
10. File Transfers [48]
11. USB 3.0 Speed [49]
12. Virtual Machine Management [50]
13. Battery Life [51]
14. Mac Pro: Gaming [52]
15. Mac Pro: CPU [53]
16. Conclusions [54]
[/one_half_last]

big buck bunny [87]

x264 Encoding

Although there are many apps available for OS X that can perform video encoding using the x264 codec, and therefore little reason to use a Windows-based encoder via a virtual machine, the x264 encoding process is extremely CPU-heavy, and serves as a good benchmark for measuring performance between VMs and native Windows installations.

Our x264 test is based on the popular Handbrake [88] app and encodes a roughly 10-minute clip of the animated short Big Buck Bunny [89] using the default “Apple TV 3” preset. The results are the average frames per second achieved during the encoding process, as reported by the Handbrake log file, with a higher number indicating faster performance.

parallels vs fusion benchmarks x264 handbrake [90]

As was evident from our initial individual looks at Parallels 11 [91] and Fusion 8 [92], both virtualization platforms have nailed it when it comes to video encoding. Our earlier tests showed that Fusion and Parallels were very close to native performance when it comes to purely CPU-bound tasks, but the results here are practically identical to native performance, indicating that you could, if you wanted, use the Windows version of Handbrake in a virtual machine on your Mac and encode video at virtually the same speed as the native Handbrake app for OS X.

x265 Encoding

The x264 codec tested above is solid, established, and reliable. But that’s not exciting, is it? The future is all about x265, the open source version of the relatively new H.265/HEVC codec. With x265, content creators and consumers can achieve quality levels equivalent to x264, but at a fraction of the bit rate and, thereby, file size. The x265 codec and its commercial counterparts are going to be key to future online delivery of 4K video content, and encoders like Handbrake are already offering basic support. The only problem? x265 encoding and decoding is extremely CPU-intensive, and can bring even the highest-performing CPUs to their knees.

We wanted to see how the latest versions of Parallels and Fusion could handle such an intense workload, and how the free VirtualBox compared. So we turned to the pre-configured x265 Benchmark [93] test to give us an answer. Each encoding test runs five passes and, like the x264 benchmark, reports the average frames per second. You should expect the numbers to be much lower than with x264.

parallels vs fusion benchmarks x265 [94]

While the overall speed is a fraction of that achieved with x264, the relative results are virtually the same. Fusion 8 practically matches native performance, while Parallels 11 is about 8 percent behind, and VirtualBox safely trails the pack.

As we mentioned, with a host of video encoding options available for OS X, it’s probably not worth running a dedicated x264 or x265 encoder in a virtual machine, even though Fusion and Parallels are both amazingly close in performance. But these results are definitely good news for Mac-owning content creators who may need to use more advanced software that is only available for Windows. In such a case, the user should expect near-native video encoding performance with either Fusion 8 or Parallels 11.

Table of Contents

[one_half padding=”0 5px 20px 0″]
1. Introduction [58]
2. Test Setup & Methodology [40]
3. Geekbench [41]
4. 3DMark [42]
5. FurMark OpenGL [43]
6. Cinebench R15 [44]
7. PCMark 8 [45]
8. Passmark PerformanceTest [46]
[/one_half]

[one_half_last padding=”0 0px 20px 5px”]
9. Video Encoding
10. File Transfers [48]
11. USB 3.0 Speed [49]
12. Virtual Machine Management [50]
13. Battery Life [51]
14. Mac Pro: Gaming [52]
15. Mac Pro: CPU [53]
16. Conclusions [54]
[/one_half_last]

computer folders [95]

File Transfers

An important factor in the user experience of any virtual machine is file transfer performance. As we discovered last year [96], free virtualization software like VirtualBox really lags behind commercial options from Parallels and VMware when it comes to file management, and depending on your workflow, that could be a deal breaker.

This test (like the Virtual Machine Management [50] test later on) is a little different in that it doesn’t include Boot Camp performance. This is because we’ll be testing the process of copying files between the VM and the host operating system, and that activity has no equivalent for a native Windows installation.

Our file tests are divided into two groups: a large file transfer (a single 4GB Windows 10 installation ISO) and a small file transfer (4,096 4KB files). The numbers in the chart represent the time in seconds that it took for each transfer, so a lower score is better here.

Note that unlike our standard tests that rely on computer-generated results, these tests were measured manually with a stopwatch. As a result, we performed each test five times instead of the usual three iterations, in order to minimize the impact of human error. Also note that we rebooted the VMs in between each test to avoid any operating system caching that could artificially improve the times.

parallels vs fusion benchmarks file transfers [97]

Starting off with the large file transfer, we see that the platforms trade victories based on the type of transfer. In the “Within VM” test, which measures a transfer from one location on the VM’s virtual drive, Parallels beats Fusion by a little more than 1 second. When it comes to file transfers between the host and guest operating systems, the “To Host” test (a file transfer from the Windows VM to OS X) shows Fusion and Parallels to be about equal, while Fusion wins by about 3 seconds in the “To VM” test (a file transfer from OS X to the Windows VM).

But file transfer speed can vary greatly depending on the number and types of files, so we wanted to see what would happen if we threw a huge number (4,096) of small 4KB files at our VMs.

parallels vs fusion benchmarks small file transfer [98]

With small files, there’s no competition. Parallels 11 wins by a landslide. Things are mostly competitive for transfers within a VM, with Fusion taking about 2.5 seconds longer than Parallels, but when it comes to transfers between the host and the guest operating systems, things fall apart for Fusion. Fusion 8 takes a lengthy, but tolerable, 9 seconds longer to transfer our small files from OS X to the Windows VM, but an astounding 36 seconds longer (a 335 percent increase) when transferring from the Windows VM to OS X. This score is 11 seconds worse than VirtualBox.

We first thought this result was due to a bug or configuration issue, but we verified that there were no configuration factors at play — e.g., encryption, indexing, caching — that would explain it as abnormal. Factor in the poor performance of Fusion 7 on the same test [99] and the conclusion is that Fusion just isn’t very good at transferring lots of small files to the host operating system. Your unique workload and needs will determine if this is a deal breaker when it comes to Fusion vs. Parallels.

It’s important to note that the specific times reported here are all based on the relatively fast flash storage found in all recent MacBooks. The performance of virtual machines stored on traditional mechanical hard drives [100], or on external storage arrays, was not tested, and may reveal different trends or limitations between the various virtualization platforms.

Table of Contents

[one_half padding=”0 5px 20px 0″]
1. Introduction [58]
2. Test Setup & Methodology [40]
3. Geekbench [41]
4. 3DMark [42]
5. FurMark OpenGL [43]
6. Cinebench R15 [44]
7. PCMark 8 [45]
8. Passmark PerformanceTest [46]
[/one_half]

[one_half_last padding=”0 0px 20px 5px”]
9. Video Encoding [47]
10. File Transfers
11. USB 3.0 Speed [49]
12. Virtual Machine Management [50]
13. Battery Life [51]
14. Mac Pro: Gaming [52]
15. Mac Pro: CPU [53]
16. Conclusions [54]
[/one_half_last]

USB 3.0 Speed

Both Fusion 8 and Parallels 11 support USB 3.0 devices with USB 3.0-compatible host hardware. In theory, this lets the user access their flash drives and external USB hard drives directly from within the VM without a significant loss in speed. But the VM still has to translate the USB 3.0 signal from the host hardware to its own virtual hardware, and there are likely to be performance compromises in that process.

To test just how well our virtual machines could handle USB 3.0 transfers, we used a 1TB Samsung 850 Pro SSD [101] and connected it to our test setup via the StarTech USB 3.1 Single-Bay Dock [102] (our MacBook Pro does not have USB 3.1 support, but the dock is fully backwards-compatible with USB 3.0 devices). With a theoretical bandwidth limit of 5Gbps (and a real-world limit even lower), no USB 3.0 interface can accommodate the full speed of our Samsung SSD. Therefore, this isn’t a test of the maximum speed of the SSD itself, but rather a comparative test of how fast each virtual machine can perform sequential read and write operations.

We used ATTO Disk Benchmark [103] to measure transfer speed in megabytes per second at transfer sizes between 512 bytes and 64 megabytes. A higher number on the y-axis indicates faster speed. To give us a baseline result, we also tested this setup natively via Boot Camp. First up is read performance:

parallels vs fusion benchmarks usb 3 reads [104]

Natively via Boot Camp, we achieved maximum USB 3.0 speeds of about 460MB/s. Of our virtualization platforms, Fusion 8 clearly wins on USB 3.0 reads, coming in not far behind Boot Camp with max speeds of about 425MB/s. VirtualBox surprisingly holds its own in this race, offering similar performance to Parallels 11, albeit with a performance drop-off at the largest transfer sizes.

parallels vs fusion usb 3 writes [105]

USB 3.0 write performance is a little more complicated. Boot Camp gives us a performance baseline of around 420MB/s, and Fusion 8 achieves the best speed at large transfer sizes of about 350MB/s, but both Parallels 11 and VirtualBox offer faster speeds at small transfer sizes, with the tipping point right around 2MB/s. Therefore, if you need your VM to transfer lots of small files like text documents and spreadsheets to an external USB 3.0 device, you may see better performance with Parallels (or VirtualBox), but for large files like movies and games, Fusion 8 will be your better option.

Table of Contents

[one_half padding=”0 5px 20px 0″]
1. Introduction [58]
2. Test Setup & Methodology [40]
3. Geekbench [41]
4. 3DMark [42]
5. FurMark OpenGL [43]
6. Cinebench R15 [44]
7. PCMark 8 [45]
8. Passmark PerformanceTest [46]
[/one_half]

[one_half_last padding=”0 0px 20px 5px”]
9. Video Encoding [47]
10. File Transfers [48]
11. USB 3.0 Speed
12. Virtual Machine Management [50]
13. Battery Life [51]
14. Mac Pro: Gaming [52]
15. Mac Pro: CPU [53]
16. Conclusions [54]
[/one_half_last]

windows 10 shut down [106]

Virtual Machine Management

Next up in our Parallels 11 vs. Fusion 8 benchmarks is a look at virtual machine management. If you plan to use just a single virtual machine and leave it running most of the time, this section probably holds relatively little value for you. But for those users who frequently run or access lots of virtual machines, we wanted to see how Parallels, Fusion, and VirtualBox compare when it comes to boot, suspend, resume, and shut down times.

As with the earlier section for File Transfers [48], these tests were measured manually with a stop watch and conducted five times each. The numbers in the chart below represent seconds so, again, lower is better.

parallels-vs-fusion-2015-vm-management [107]

Once again, we see Parallels and Fusion trading places based on the type of activity. Parallels 11 boots Windows 10 insanely fast, in about 10 seconds, and can shut it down even quicker, at around 8 seconds. These speeds, which would seem ludicrous to a Windows user from a just a few years ago, are a product of both Parallels’ efficiency at handling boots and shutdowns, and also Microsoft’s efforts, begun with Windows 8 [108] and continuing in Windows 10 [109], to significantly improve boot times.

When it comes to suspend and resume functions, however, Fusion is the champ, and can get you up and running again in a suspended VM in just over 4 seconds. VirtualBox, for its part, really underscores the adage “you get what you pay for,” and takes a frustratingly long time to boot, suspend, and resume its virtual machines, with the only bright spot being a respectable second-place finish in shutdown times.

While Parallels’ fast boot times or VirtualBox’s minute-long suspend times may be noteworthy enough to sway a consumer towards (or away from) one product or another, remember that these factors only make a big difference if you use multiple virtual machines throughout the day and need to shut down or suspend one, and then boot or resume another, as quickly as possible. If you only use a single VM, booting it in the morning and suspending it at night, you may not want to shell out for Parallels or Fusion if, for example, VirtualBox’s other performance limitations aren’t a problem for you. Taking that course will save you between $50 and $80, which is a hefty price to otherwise pay for an extra minute of waiting while you suspend your VirtualBox VM.

Finally, note that, like our file transfer tests [48], these boot numbers are based on expected performance from a modern Mac CPU and fast flash storage. If you have an older Mac, or otherwise store your virtual machines on mechanical hard drives, you’ll see longer times for the tests above.

Table of Contents

[one_half padding=”0 5px 20px 0″]
1. Introduction [58]
2. Test Setup & Methodology [40]
3. Geekbench [41]
4. 3DMark [42]
5. FurMark OpenGL [43]
6. Cinebench R15 [44]
7. PCMark 8 [45]
8. Passmark PerformanceTest [46]
[/one_half]

[one_half_last padding=”0 0px 20px 5px”]
9. Video Encoding [47]
10. File Transfers [48]
11. USB 3.0 Speed [49]
12. Virtual Machine Management
13. Battery Life [51]
14. Mac Pro: Gaming [52]
15. Mac Pro: CPU [53]
16. Conclusions [54]
[/one_half_last]

Battery Life [110]

Battery Life

If you read our initial looks at Parallels 11 [1] and Fusion 8 [2], you know that battery life was one key factor we didn’t include in our results. That’s not because we don’t think it’s important; it’s simply that it takes so darn long to run our tests, with three full tests performed for each platform and each usage scenario (that’s 24 complete drain and recharge cycles, for those who are counting). We also hit an unusual issue, described later on, that required another few rounds of testing and pushed this final article out a few days. But never fear! We’re back with battery life benchmarks for Parallels, Fusion, and VirtualBox, and we also have native performance in Boot Camp as a baseline for comparison.

With the ever increasing use of laptops and mobile computing devices, battery life is one of the most important areas of product development and the customer experience. Although energy efficiency and battery capacity have improved dramatically in recent years, it still takes a lot of power to run a complex modern operating system, and with any of the virtualization platforms we’re looking at today, we’re asking our MacBook to go even further and run two operating systems (at least!).

We obviously expect that our MacBook will experience shorter running times when using a virtual machine in OS X, but the question is how much shorter. To test that question, we turned to Futuremark Powermark [111], a battery life benchmark for Windows that’s based on tests similar to those conducted by its sister app, PCMark [45]. Unlike PCMark, however, which runs a particular scenario just three times in order to determine a system’s performance level, Powermark runs the designated usage scenario repeatedly until the battery is drained.

Powermark has its own internal battery life calculator that reports both actual running time and estimated battery life remaining, but we didn’t want to take the chance that the virtualization process could introduce inaccuracies in that counter. We therefore turned to our old standby for measuring battery life in OS X: Automator [112]. Using a basic Automator workflow, we told the script to simply write a timestamp to a text file on the OS X desktop once every 30 seconds. We then ran our Powermark tests until the battery was drained, plugged the Mac back in, booted up, and calculated the total running time based on the difference between the first and last timestamp in the file generated by Automator. With this method, which has virtually no impact on battery life, we get a reliable result of total running time that has, at most, a 30-second margin of error.

We ran two Powermark benchmarks: Productivity and Entertainment. The productivity benchmark is likely to be the most common usage scenario for a real user’s virtual machine, and tests activities like Web browsing and office apps. It should be relatively light on our Mac’s battery and offer usable running time when out and about. The Entertainment test, on the other hand, is much harder on the battery, with tasks like video playback and entry-level 3D gaming. The results from this test should give you a good idea of how long your MacBook’s battery will last while running more demanding apps in your virtual machines.

The results reported for each test, below, are measured in minutes, so longer bars are better. All tests were performed with the MacBook’s screen set to 50 percent brightness and no other apps open except for the Powermark suite in the Windows VM and our Automator script in OS X.

parallels vs fusion battery life [113]

It should first be noted that Windows running in Boot Camp isn’t as energy efficient as OS X natively, due to the fact that Apple, which makes its own hardware, can fine-tune and optimize every aspect of hardware and software to maximize battery life. For example, we achieved an average time of 6 hours and 43 minutes while running the lightweight productivity test in Boot Camp. That’s about 16 percent less than the roughly 8 hours [114] that we could expect from running a similarly demanding test natively in OS X. Therefore, the Boot Camp results here aren’t meant to represent the maximum battery life potential of the MacBook Pro overall, but rather serve only as a baseline for comparing native Windows performance to that of the virtual machines on the same hardware.

That said, Parallels and Fusion again trade places depending on the workload. All VMs fall notably short of the battery life attained by Boot Camp, but Parallels 11 hits an average of 5 hours and 49 minutes on the Productivity test, which is 13 percent shorter than Boot Camp and about 5 percent longer than Fusion 8.

The roles reverse for the more demanding Entertainment test. Here, even Boot Camp only hits an average of 3 hours and 25 minutes, while Fusion 8 takes second place with an average running time of 2 hours and 46 minutes, about 12 percent longer than Parallels 11.

As for VirtualBox, you shouldn’t expect to get much more than a partial afternoon’s work done, even in lightweight “productivity” scenarios, with the VirtualBox VM only hitting 3 hours and 7 minutes at best. When you add more complexity from the Entertainment test, you’re down to just 1 hour and 44 minutes.

Parallels 11 Travel Mode

One of the new features in Parallels 11 is “Travel Mode [115],” an option that takes over Windows’ power management settings and can automatically pause or disable resource intensive services and processes in order to reduce energy usage and extend battery life. When we set out to conduct the tests in the first chart above, we naturally enabled travel mode for our Parallels VM, as we generally want to run our tests under conditions that take advantage of each app’s unique features and settings. But when the initial results came in, we found the battery life achieved by Parallels to be surprisingly, even suspiciously, low.

parallels vs fusion battery life travel mode [116]

As part of our troubleshooting process to figure out exactly what was going on, we eventually tried a test without Travel Mode and, a few hours later, received a more expected result like those recorded for Parallels in the first chart. Some additional testing led us to the conclusion that Parallels’ Travel Mode really shouldn’t be enabled if you’re actively trying to use your virtual machine. It seems that our battery life benchmark tests, like any running application, were constantly calling on Windows to deliver power and performance, and Parallels Travel Mode was constantly fighting that request, causing an inefficient ramping up and down of resources that actually drained the battery much quicker and resulted in slow performance from our benchmark apps (e.g., word processor windows were slow to resize, the gaming part of the benchmark ran at single-digit frame rates).

parallels 11 travel mode [117]

So what’s Travel Mode for, then? Well, as we continued to experiment with this issue over the past few days, we discovered that Travel Mode really can come in handy, but only when your Windows VM is in an idle state. We accidentally discovered this when one of our benchmark runs crashed at the start of the benchmark, but after we had left the room, resulting in a “test” that was nothing more than Automator writing a timestamp every 30 seconds. We returned the next morning to find a timestamp-calculated result indicating significantly better performance than any other test. After a few more runs to verify this, we concluded that the key to Travel Mode is not to fight it, and that you shouldn’t use it at all if you’re running anything in Windows other than possibly a simple text editor.

Following that advice gives us the much more interesting results, with all systems tested in the same “idle” mode:

parallels 11 travel mode idle [118]

That’s right, when your VM and Mac are both in an idle state, Parallels 11 in Travel Mode gets even better battery life than native Boot Camp (likely owing to the aforementioned energy efficiency advantages of OS X over Windows, and remembering that while in Travel Mode, the Parallels VM is just like any other energy efficient OS X app). The bad news? Your VM and Mac are in an idle state.

It’s true that you want your virtual machine to be as energy efficient as possible, especially when it’s not in use, and that may have been Parallels’ motivation for introducing this feature. But if your VM is going to be idle long enough for Travel Mode to make a difference (and, remember, it has to be completely idle, without even moderately-intensive applications running), you might as well just suspend the VM, something that Parallels is pretty good [50] at, to preserve battery life even further. In short, Travel Mode works great in this idle testing scenario (that’s a 20 percent improvement over Parallels 11 without Travel Mode enabled), but we honestly have no idea who will find this feature useful in its current state.

Table of Contents

[one_half padding=”0 5px 20px 0″]
1. Introduction [58]
2. Test Setup & Methodology [40]
3. Geekbench [41]
4. 3DMark [42]
5. FurMark OpenGL [43]
6. Cinebench R15 [44]
7. PCMark 8 [45]
8. Passmark PerformanceTest [46]
[/one_half]

[one_half_last padding=”0 0px 20px 5px”]
9. Video Encoding [47]
10. File Transfers [48]
11. USB 3.0 Speed [49]
12. Virtual Machine Management [50]
13. Battery Life
14. Mac Pro: Gaming [52]
15. Mac Pro: CPU [53]
16. Conclusions [54]
[/one_half_last]

Mac Pro: Gaming Benchmarks

As we mentioned earlier, a new part of our tests this year is a quick look at how Parallels and Fusion perform on faster hardware. We’re interested not only in which one is faster, but also how they compare to native Boot Camp performance and if the addition of more cores or more powerful GPUs could reverse the trends we saw in the MacBook Pro tests.

Unlike our earlier tests, we’re only looking at Parallels 11 and Fusion 8 here, as we’re interested in maximum performance and VirtualBox isn’t capable of competing at that level. We configured our VMs on the Mac Pro with 12 virtual CPUs, 32GB of RAM, and 2GB of graphics memory and then set out to run a selection of our benchmark tests that focus on graphics and processing power.

We’ve already described these tests in earlier pages for the MacBook Pro, so we won’t repeat our descriptions again for the Mac Pro. We’ll only make a brief comment or two for each section on the results from the tests. First up is 3DMark:

parallels vs fusion mac pro ice storm [119]

parallels vs fusion mac pro 3dmark ice storm extreme [120]

With DirectX 9, we see similar results to the Ice Storm tests performed on the MacBook Pro, with Fusion 8 enjoying a huge lead when it comes to graphics, and just about equal to Parallels in CPU-heavy physics calculations. Still, neither Parallels nor Fusion can come close to the native performance of two D500 GPUs.

parallels vs fusion mac pro 3dmark cloud gate [121]

Once again similar to the MacBook Pro results, both Parallels and Fusion struggle when it comes to DirectX 10, indicating that, even though both products now support the API, they have a long way to go to perfect and optimize performance.

parallels vs fusion mac pro 3dmark06 [122]

Here’s a result that provides one of the interesting angles we were looking for. While Fusion 8 beats Parallels 11 in 3DMark06 on the MacBook Pro, the order is flipped when Parallels is given more power to play with, and it comes in just ahead of Fusion 8 in every category.

parallels vs fusion mac pro furmark [123]

We end with a quick look at OpenGL performance in FurMark, and see the same basic trend as that revealed in the MacBook Pro tests. Fusion 8’s superior OpenGL graphics implementation results in a score that’s 66 percent higher than Parallels 11 at 720p, and 92 percent higher at 1080p.

Table of Contents

[one_half padding=”0 5px 20px 0″]
1. Introduction [58]
2. Test Setup & Methodology [40]
3. Geekbench [41]
4. 3DMark [42]
5. FurMark OpenGL [43]
6. Cinebench R15 [44]
7. PCMark 8 [45]
8. Passmark PerformanceTest [46]
[/one_half]

[one_half_last padding=”0 0px 20px 5px”]
9. Video Encoding [47]
10. File Transfers [48]
11. USB 3.0 Speed [49]
12. Virtual Machine Management [50]
13. Battery Life [51]
14. Mac Pro: Gaming
15. Mac Pro: CPU [53]
16. Conclusions [54]
[/one_half_last]

Mac Pro: CPU Benchmarks

As a companion to the Mac Pro’s gaming-focused benchmarks, we’ll now see how Parallels 11 and Fusion 8 perform on more processor-intensive tasks.

parallels vs fusion mac pro geekbench single core [124]

parallels vs fusion mac pro geekbench multi-core [125]

Here again we see a slight reversal of fortunes. While Fusion 8 once again comes in slightly ahead of Parallels 11 in single-core tests, Parallels reverses the result from the MacBook Pro tests and narrowly bests Fusion in multi-core tests.

parallels vs fusion mac pro cinebench [126]

Our tests conclude with Cinebench, where Fusion 8 narrowly beats Parallels 11 in both multi-core and single-core rendering. One interesting result from this test, however, is the difference in performance between Boot Camp and the VMs as compared to the same test on the MacBook Pro. In that test [44], Fusion also came out on top, but trailed Boot Camp by only 0.6 percent in the multi-core test, and by 4.1 percent in single-core. Here, Fusion trails Boot Camp by 13.8 percent in multi-core and 19.2 percent in single-core, suggesting that there’s a limit to how efficient Parallels and Fusion can be with so many cores at their disposal.

Table of Contents

[one_half padding=”0 5px 20px 0″]
1. Introduction [58]
2. Test Setup & Methodology [40]
3. Geekbench [41]
4. 3DMark [42]
5. FurMark OpenGL [43]
6. Cinebench R15 [44]
7. PCMark 8 [45]
8. Passmark PerformanceTest [46]
[/one_half]

[one_half_last padding=”0 0px 20px 5px”]
9. Video Encoding [47]
10. File Transfers [48]
11. USB 3.0 Speed [49]
12. Virtual Machine Management [50]
13. Battery Life [51]
14. Mac Pro: Gaming [52]
15. Mac Pro: CPU
16. Conclusions [54]
[/one_half_last]

Conclusions

If you look at our Parallels vs. Fusion benchmark results simply by the numbers, then Fusion 8 is the clear winner, taking 11 out of 18 tests and pushing on three more. But the reason we try to test so many different areas (and divide them up individually for easy access via the Table of Contents), is because everyone’s needs are different. Some users are looking for a virtualization app to play Windows-only games, or crunch data in a Windows-only CPU-intensive app. Those folks should probably go with Fusion 8 this year.

Others may want the absolute best battery life for lightweight workloads, need to transfer lots of small files out of their VM, or appreciate a VM that boots before they can even get their first sip of coffee down. In that case, Parallels 11 is probably your best bet.

Still others only need a Windows VM to run that one old productivity app that their job requires, and in that case VirtualBox can likely suffice without having to buy anything other than a Windows license.

That’s why it’s often difficult to answer in any universal way when we’re frequently asked “should I buy Parallels or Fusion?” And compounding that question further is the fact that the scenarios we just outlined above are only valid today. Both the Parallels and VMware engineers are working hard to one-up each other every year, introducing new features (like Windows Quick Look support in Parallels) or making huge performance leaps (like the big improvement in Fusion 8’s graphics engine). While both apps have occasionally maintained one type of performance advantage in consecutive years, it’s a safe bet that at least one significant performance crown will change hands each fall.

That said, it’s clear that Fusion 8 made much larger and more impressive performance gains this year, both compared to its predecessor and in a direct comparison with Parallels. On the other side, however, Parallels 11 brings a few new features that we think Mac users will really love, like the aforementioned Quick Look support.

Therefore, our advice for those new to virtualization software, or for existing users thinking about upgrading or crossgrading, is to start small and cheap. Check out the free VirtualBox [5] first to see if it meets your performance and capability needs. If not, grab the free trials of both Fusion 8 [127] and Parallels 11 [128] and give them a shot (Parallels offers a 14-day trial period, while you can use Fusion for up to 30 days free). You may find that you need less power or fewer capabilities than you thought, or you may prefer the particular features of one app over another. Either way, we hope these benchmarks can give you an idea of what to expect, and where to start on your own OS X virtualization journey.

If there is a benchmark or scenario that you’d like us to test that isn’t covered here, let us know in the comments and we’ll do our best to accommodate your requests.

Table of Contents

[one_half padding=”0 5px 20px 0″]
1. Introduction [58]
2. Test Setup & Methodology [40]
3. Geekbench [41]
4. 3DMark [42]
5. FurMark OpenGL [43]
6. Cinebench R15 [44]
7. PCMark 8 [45]
8. Passmark PerformanceTest [46]
[/one_half]

[one_half_last padding=”0 0px 20px 5px”]
9. Video Encoding [47]
10. File Transfers [48]
11. USB 3.0 Speed [49]
12. Virtual Machine Management [50]
13. Battery Life [51]
14. Mac Pro: Gaming [52]
15. Mac Pro: CPU [53]
16. Conclusions
[/one_half_last]