28

2015 VM Benchmarks: Parallels 11 vs. Fusion 8 vs. VirtualBox 5

Posted by Jim Tanous on September 4, 2015
fusion parallels virtual box

Battery Life

Battery Life

If you read our initial looks at Parallels 11 and Fusion 8, you know that battery life was one key factor we didn’t include in our results. That’s not because we don’t think it’s important; it’s simply that it takes so darn long to run our tests, with three full tests performed for each platform and each usage scenario (that’s 24 complete drain and recharge cycles, for those who are counting). We also hit an unusual issue, described later on, that required another few rounds of testing and pushed this final article out a few days. But never fear! We’re back with battery life benchmarks for Parallels, Fusion, and VirtualBox, and we also have native performance in Boot Camp as a baseline for comparison.

With the ever increasing use of laptops and mobile computing devices, battery life is one of the most important areas of product development and the customer experience. Although energy efficiency and battery capacity have improved dramatically in recent years, it still takes a lot of power to run a complex modern operating system, and with any of the virtualization platforms we’re looking at today, we’re asking our MacBook to go even further and run two operating systems (at least!).

We obviously expect that our MacBook will experience shorter running times when using a virtual machine in OS X, but the question is how much shorter. To test that question, we turned to Futuremark Powermark, a battery life benchmark for Windows that’s based on tests similar to those conducted by its sister app, PCMark. Unlike PCMark, however, which runs a particular scenario just three times in order to determine a system’s performance level, Powermark runs the designated usage scenario repeatedly until the battery is drained.

Powermark has its own internal battery life calculator that reports both actual running time and estimated battery life remaining, but we didn’t want to take the chance that the virtualization process could introduce inaccuracies in that counter. We therefore turned to our old standby for measuring battery life in OS X: Automator. Using a basic Automator workflow, we told the script to simply write a timestamp to a text file on the OS X desktop once every 30 seconds. We then ran our Powermark tests until the battery was drained, plugged the Mac back in, booted up, and calculated the total running time based on the difference between the first and last timestamp in the file generated by Automator. With this method, which has virtually no impact on battery life, we get a reliable result of total running time that has, at most, a 30-second margin of error.

We ran two Powermark benchmarks: Productivity and Entertainment. The productivity benchmark is likely to be the most common usage scenario for a real user’s virtual machine, and tests activities like Web browsing and office apps. It should be relatively light on our Mac’s battery and offer usable running time when out and about. The Entertainment test, on the other hand, is much harder on the battery, with tasks like video playback and entry-level 3D gaming. The results from this test should give you a good idea of how long your MacBook’s battery will last while running more demanding apps in your virtual machines.

The results reported for each test, below, are measured in minutes, so longer bars are better. All tests were performed with the MacBook’s screen set to 50 percent brightness and no other apps open except for the Powermark suite in the Windows VM and our Automator script in OS X.

parallels vs fusion battery life

It should first be noted that Windows running in Boot Camp isn’t as energy efficient as OS X natively, due to the fact that Apple, which makes its own hardware, can fine-tune and optimize every aspect of hardware and software to maximize battery life. For example, we achieved an average time of 6 hours and 43 minutes while running the lightweight productivity test in Boot Camp. That’s about 16 percent less than the roughly 8 hours that we could expect from running a similarly demanding test natively in OS X. Therefore, the Boot Camp results here aren’t meant to represent the maximum battery life potential of the MacBook Pro overall, but rather serve only as a baseline for comparing native Windows performance to that of the virtual machines on the same hardware.

That said, Parallels and Fusion again trade places depending on the workload. All VMs fall notably short of the battery life attained by Boot Camp, but Parallels 11 hits an average of 5 hours and 49 minutes on the Productivity test, which is 13 percent shorter than Boot Camp and about 5 percent longer than Fusion 8.

The roles reverse for the more demanding Entertainment test. Here, even Boot Camp only hits an average of 3 hours and 25 minutes, while Fusion 8 takes second place with an average running time of 2 hours and 46 minutes, about 12 percent longer than Parallels 11.

As for VirtualBox, you shouldn’t expect to get much more than a partial afternoon’s work done, even in lightweight “productivity” scenarios, with the VirtualBox VM only hitting 3 hours and 7 minutes at best. When you add more complexity from the Entertainment test, you’re down to just 1 hour and 44 minutes.

Parallels 11 Travel Mode

One of the new features in Parallels 11 is “Travel Mode,” an option that takes over Windows’ power management settings and can automatically pause or disable resource intensive services and processes in order to reduce energy usage and extend battery life. When we set out to conduct the tests in the first chart above, we naturally enabled travel mode for our Parallels VM, as we generally want to run our tests under conditions that take advantage of each app’s unique features and settings. But when the initial results came in, we found the battery life achieved by Parallels to be surprisingly, even suspiciously, low.

parallels vs fusion battery life travel mode

As part of our troubleshooting process to figure out exactly what was going on, we eventually tried a test without Travel Mode and, a few hours later, received a more expected result like those recorded for Parallels in the first chart. Some additional testing led us to the conclusion that Parallels’ Travel Mode really shouldn’t be enabled if you’re actively trying to use your virtual machine. It seems that our battery life benchmark tests, like any running application, were constantly calling on Windows to deliver power and performance, and Parallels Travel Mode was constantly fighting that request, causing an inefficient ramping up and down of resources that actually drained the battery much quicker and resulted in slow performance from our benchmark apps (e.g., word processor windows were slow to resize, the gaming part of the benchmark ran at single-digit frame rates).

parallels 11 travel mode

So what’s Travel Mode for, then? Well, as we continued to experiment with this issue over the past few days, we discovered that Travel Mode really can come in handy, but only when your Windows VM is in an idle state. We accidentally discovered this when one of our benchmark runs crashed at the start of the benchmark, but after we had left the room, resulting in a “test” that was nothing more than Automator writing a timestamp every 30 seconds. We returned the next morning to find a timestamp-calculated result indicating significantly better performance than any other test. After a few more runs to verify this, we concluded that the key to Travel Mode is not to fight it, and that you shouldn’t use it at all if you’re running anything in Windows other than possibly a simple text editor.

Following that advice gives us the much more interesting results, with all systems tested in the same “idle” mode:

parallels 11 travel mode idle

That’s right, when your VM and Mac are both in an idle state, Parallels 11 in Travel Mode gets even better battery life than native Boot Camp (likely owing to the aforementioned energy efficiency advantages of OS X over Windows, and remembering that while in Travel Mode, the Parallels VM is just like any other energy efficient OS X app). The bad news? Your VM and Mac are in an idle state.

It’s true that you want your virtual machine to be as energy efficient as possible, especially when it’s not in use, and that may have been Parallels’ motivation for introducing this feature. But if your VM is going to be idle long enough for Travel Mode to make a difference (and, remember, it has to be completely idle, without even moderately-intensive applications running), you might as well just suspend the VM, something that Parallels is pretty good at, to preserve battery life even further. In short, Travel Mode works great in this idle testing scenario (that’s a 20 percent improvement over Parallels 11 without Travel Mode enabled), but we honestly have no idea who will find this feature useful in its current state.

Table of Contents

[one_half padding=”0 5px 20px 0″]
1. Introduction
2. Test Setup & Methodology
3. Geekbench
4. 3DMark
5. FurMark OpenGL
6. Cinebench R15
7. PCMark 8
8. Passmark PerformanceTest
[/one_half]

[one_half_last padding=”0 0px 20px 5px”]
9. Video Encoding
10. File Transfers
11. USB 3.0 Speed
12. Virtual Machine Management
13. Battery Life
14. Mac Pro: Gaming
15. Mac Pro: CPU
16. Conclusions
[/one_half_last]

28 thoughts on “2015 VM Benchmarks: Parallels 11 vs. Fusion 8 vs. VirtualBox 5”

s says:
Hello, can I use the 2015 VM Benchmark Showdown picture in my Sr. Design project documentation? I wanted to compare different virtual machine throughput.
Reply
Anton Iakimov says:
Would be nice to see also
* ESXi with macOS guest
Reply
Janto Lenherr says:
i like how you go into detail, but it lacks audio software, that is what i would be interested about.
Reply
Michael M says:
I am late to this game. I do agree that Parallels offers a more “friendly” interface than Fusion but beyond that, they seem to function pretty much the same except for one nagging item that I have noticed,

Has anyone else notices that Fusion runs a bit “hotter” than Parallels? I find that opening up Fusion to Win 7 or Win 10 automatically gets the temp up at least 20 degrees (F) or so on my rMBP (2015) set for 2 of 4 cores and 8 of 16 gigs of the RAM allocated. This is very disappointing. I have zero issue with the VMware offering (used it years ago) save for this need to use more energy than Parallels. I only use Windows for a couple of apps but I would be worried about adding a more CPU intensive app if heat is generated like this at a greater scale.

Any thoughts or confirmation of Fusion running hotter?

Reply
Paul Bartlett says:
Excellent and thorough review, many thanks for this. It seems to mostly reflect what I have experienced. I have been a VMWare Fusion fan for many years and it has never let me down. Recently I tried Virtual Box and it is much better that it used to be, but too many problems, such as weird mouse pointers on Win10/retina display. I have not tried Parallels for several years. Last time I tried it also had a bunch of annoying issues. I am a developer often interfacing with USB devices so for me one of the most important things is USB support and Fusion has been great there. Several years ago Parallels did not have such good USB support either and it was quite expensive. I must try Parallels again one day as this review makes it look like it is up there with VMware, which it was not when I last used it.
Reply
Simone says:
When Parallels 12 vs Fusion 8.5 comparison?
Reply
niico100 says:
A very common use case is Visual Studio / SQL Server developers. So if you could do any kind of development benchmark that would be useful. (I have found on a retina 12″ MacBook VMWare runs significantly smoother – and Parallels has been locking up in retina mode).
Reply
Paul Bartlett says:
me too, would love to see Visual Studio and even scanning and printing. This benchmark shows ‘perfect world’ whereas I use Fusion because its the only one that doesn’t have beyond annoying issues.
Reply
Robert says:
Hoping to see a comparison of the newer Vmware Fusion 8.5.0 free upgrade and Parallels 12 and and the latest VirtualBox.
Reply
Paul Charlesworth says:
For me, using Parallels over Fusion has always come down to the fact that Fusion requires a password every time I want to access the bootcamp partition. Parallels also has Parallels Access, although the integration between their two products could be better.
Reply
RW says:
Excellent and exhaustive review, however I typically run linux guest appliances (primarily headless). It would have been help to see some linux guest performance comparisons in the future. Thank you.
Reply
Ksix says:
Do you recommend one over the other for running ArcGIS software?
Reply
Stephen says:
I am really interested in understanding the performance for disk access speed to the file system when compared between virtualized machines and boot camp. I am almost certain I will get some improvement by switching from Parallels to Boot Camp, but I don’t want to go through the effort/force a change in my workflow without solid numbers.
On my current setup it seems to take an eternity for the memory on my Windows VM to be filled with all the data it needs (I need to have a SQL server and a related application running on my Windows OS). Are there any good metrics on how long it takes for the OS to load up files from the drive into memory?
PS: Hard drive is a traditional drive, system report describes as APPLE HDD ST1000DM003
Reply
daniael says:
i own both vmware 8 pro and parallels 11 pro, parallels is faster by far. small files, large files, medium high res, real life work (compiling, resource editing, various ide(s), git, office, etc) there is no comparison between the two, in my experience. run them in 2 machines, a macbook pro i7 2.7ghz, 1tb ssd and a mac pro 24 cores 3.33ghz, also 1tb ssd. i am not game-ing nor watching videos in vm, i do that in host machine, when i do (not often)
Reply
Tien Dinh says:
Me too. I use Parallel desktop for years and recently switched to VMWare 8 Pro due to this article. I found the performance of VMWare 8 Pro to be unbearable. After wasting tons of time trying to determine the causes, i found that the disk performace is extremely slow compare to Parallel desktop or native boot camp speed. I’m swiching back to Parallel Desktop now.
Reply
DanialThom says:
This would be more useful if you outlined the tuning settings. You don’t even mention the number of cpus in the “multicore” test
Reply
TekRevue says:
The configuration options for the virtual machines are explicitly mentioned on the “Test Setup & Methodology” page. Quoting directly from that page:

Each of our Parallels and Fusion Windows 10 virtual machines was configured for maximum performance, with 8 assigned virtual CPUs, 12GB of RAM (the maximum recommended amount in order to ensure that enough is reserved for OS X), and 1GB of graphics memory configured for each platform’s most advanced DirectX and OpenGL graphics support. The VirtualBox VM was also configured to its maximum supported hardware level, with matching CPU and RAM, but with only 256MB of graphics memory, the most it allows. For all platforms, all features that could possibly impact performance, such as error logging or an expanding virtual disk, were disabled.

Reply
Samir Abdo says:
Can you use the Mac hot corners and gestures in Fusion 8?
daniael says:
1gb graphics memory is NOT the recommended size for parallels. since you mentioned 12gb ram, then you most likely run it on a modest macbook pro with 16gb ram and mediocre video card. when it comes to 24 core cpu (2 physical processors, 6 cores each with enough L2 cache) 64+GB RAM and super fast ssd(s) and video cards with 4gb and 1500 cores, fusion is way inferior (cannot even compare). most professional places use similar machines. this does not apply to home end-users or non technical enterprises…
TekRevue says:
You’re a bit confused, Dan. Let me clarify for you:

1) The exact models and specifications for the testing hardware are listed in the article, along with the methodology used for the tests. You’ll find this information on the ironically named “Test Setup & Methodology” page.

2) We submitted our results and methodology to both Parallels and VMware to give each company a chance to respond. Neither company claimed that our testing procedure or the configuration of our virtual machines were incorrect or unfair.

3) Other than providing licenses for both Fusion 8 and Parallels Desktop 11 for our tests, TekRevue was not remunerated by either company in any way, nor were the tests conducted or guided by anyone outside of TekRevue.

4) These products, and our review, are primarily targeted at consumers. That’s why we used both a MacBook Pro and mid-level Mac Pro in the tests. If you’d like us to conduct future tests on your custom Mac Pro (I say “custom” because the Mac Pro you describe in your other comment — 2 x 6-core hyperthreaded 3.33GHz — is not a configuration Apple ever shipped), then please send it to us and we’ll be happy to do the additional testing. In general, however, I hope you’ll agree that your specific configuration isn’t applicable to ~99% of users.

daniael says:
they probably ran it with default settings on a modest macbook pro
Reply
Mike S. says:
Thought I’d try out both on my Mac Pro 2013 (8-core CPU, 32gb RAM, 1tb PCIe SSD) and while both perform admirably, Parallels bested Fusion on Windows boot/startup times (as this articles mentions). For me, boot time is top priority, so Parallels was the winner for me.
I didn’t try any hard-core games, as I don’t use the machine for that purpose. However, multiple version of Windows (XP, 7, 10) ran faster and smoother overall in Parallels for general use in my experience.
Having said that, I had some trouble with Parallels recognizing certain USB devices that Fusion did not have trouble with. Complained about it to Parallels, and they’ve recently pushed out a new release that addressed and fix those problems, so I appreciate their customer support.
Good article, thanks!
Reply
NativeCalifornian says:
I opted for VMware Fusion and ran into a problem or 2. When I tried to upgrade my W7P VM to W10 it said the graphics driver was out dated. When I tried to run SoftPlan 2016 in my W7P it wouldn’t because it got errors installing DirectX 11. I think all things being equal Fusion is a better buy especially because I don’t have to buy a new copy for each machine in my house.
Reply
Ryh says:
I’m Parallels user for 6 years but I think I’ll switch to Fusion. With every free OS X upgrade I’m forced to upgrade Parallels for $50…
Reply
ceeioo says:
It’s the main reason i don’t choose Parallels
Reply
Simon says:
still confused which is better, according to parallels comparison http://www.parallels.com/landingpage/pd/fusion-compete/ parallels is the best, though it would be strange to see smth. else here 🙂 if you check vmware tests https://www.vmware.com/products/fusion/compare you will see another picture, but the strange thing is that both companies has different digits for the same features/options, eg. check VM capabilities, other comparisons show better results for VMware Fusion http://www.softocoupon.com/comparisons/parallels-desktop-11-for-mac-vs-vmware-fusion-8.php so which is the best?

VM Capabilities
VM Capabilities

Reply
wimver says:
I’ve been a Parallels 9 user and was about to upgrade to VMWare Fusion 8 after reading several reviews. Also because I did not like it I had to pay to upgrade my Parallels license if I wanted to upgrade Mac OS to OS X El Captain… I don’t like this kind of “you have to”‘s. So I was holding my credit card ready in my hand, ready to pay for Fusion 8. Especially with their Parallels to VMWare upgrade offering. But I thought: let’s try it fist; so I installed the trial of Fusion 8… This was so disappointing for me… First I imported the existing Parallels virtual machine and converted it which went really smooth. But then… Starting up time: disappointing by all means. I first thought: OK, maybe VMWare tools have to be installed first; but no: even after rebooting several times: boot-up time kept on being disappointing. Then… opening a Visual Studio project: disappointing. I’m not even talking about compiling some code: very disappointing. It just took so much much much longer time then it did in Parallels 9! So now; I just upgrade to Parallels 11: I love it! Boot-up time, Visual Studio performance, … I love it. It’s even better than Parallels 9. The only thing I don’t love is that I had to upgrade. But for my point of view it’s obvious: Parallels beats VMWare Fusion.
Reply
simmerkatt says:
I think it’s unfair to include VirtualBox with Fusion and Parallels.

You are forgetting one major thing: Fusion and Parallels are designed to integrate Windows with OS X. VirtualBox is designed to virtualise an entire machine as if it were 100% separate, no integration at all.

Reply
Max Coplan says:
I think BootCamp and VirtualBox are the two extremes in this comparison. VirtualBox virtualizes everything, without any integration. And BootCamp has zero virtualization at all, aka complete integration.
Reply
Marco Schirrmeister says:
Great post. I am a Fusion user since day 1 and really like it, but now that VMware has doubled the price I will stay with 7 as long as I can.
Reply
No Name says:
Great review. Thanks for writeup.
Reply
Br. Bill says:
Another thing to consider is the licensing. Yes, Virtualbox is free. Fusion gives you licenses for 3 hosts. Parallels gives you a license for one machine only. So Parallels effectively costs 3x what Fusion does, if you want to host VMs on multiple machines.
Reply
JBacTee says:
Any chance you could gain access to a multi-processor (read: 2 physical processor chips) Mac Pro for a similar comparison? Virtualbox has a blatant issue ( https://www.virtualbox.org/ticket/12742 ) negotiating multicore setups in a multiple CPU host (experienced this myself with the dual-CPU 2008 Mac Pro), where the virtualized guest suffers a huge performance hit if you’ve set it up with more than one virtual core. Both Windows 7 & attempts at running the Windows 10 preview were so terrible I gave up trying to run them with any more than a single allocated core on the Mac Pro (whereas my i7 MacBook Pro with a single-chip 8-thread CPU can easily virtualize up to 8 cores with no evident performance hit).

It would be worth seeing how Parallels & Fusion (or even Virtualbox, but I assume it’ll look absolutely terrible as far as performance goes) similarly act under a multi-chip host environment to see how they negotiate allocations between multiple physical chips, as opposed to threads all being sent to the same chip.

Reply
James Nguyen says:
I had a similar problem with my MPB mid 2009. Disgusting performance when virtualizing windows 7 or 10. Did you find a solution to this problem?
Reply
Fair game says:
Awesome! Review is great. Fusion is great for biz, PD 11 is good for personal/home use, use bootcamp if your windows friends bug you but you don’t need virtualization, use VitrualBox if you just need to visit the other worlds (Win/Linux), if you want something reliable go for fusion and hey if you have extra money, throw that shit at Parallel they deserve it, all good products but hey Oracle has screwed they entire industry, just like CUNYfirst, but trust me they don’t need your petty cash! Fair ‘n square.
Reply
Henrique Tavares says:
Really well made review, thank you for the excelent work!
Reply
Enrico G. Lupo says:
Many thanks for this full review.
I’m an enthusiast of fusion 8 and a user fro version 5. This last release is a great goal!
Reply
Zero1 says:
Really nice review. The first that I have seen online of such sorts.
Overall content on Tekrevue is very solid. I enjoy reading OS X tips mostly.
Thanks guys!:)
Reply

Leave a Reply

Your email address will not be published. Required fields are marked *


Disclaimer: Some pages on this site may include an affiliate link. This does not effect our editorial in any way.