For Mac OS X users, there are several choices when it comes to running Windows on an Intel-based Macintosh. The three top contenders are Parallels, VMware Fusion and Apple’s Boot Camp.

This article is the first of many benchmarking the latest versions of each solution.

Several months ago, MacTech published an article benchmarking the three. The tests run surprised me. The majority of tests run, were not only for situations I haven’t used a virtualization solution for, but were also not applicable for the majority of people I’ve talked to. I’ve set out to do more real-world tests.

Although the MacTech article is thorough and tests seem to have been run well, they started the testing (and thus completed the testing) with older versions of Boot Camp, Parallels and Fusion. They ran over 2500 tests.

There are a few weaknesses in their tests and I think more information about their test parameters should have been disclosed. I’m sure some people will find the results are very useful. Their tests focus around Microsoft Office, Internet Explorer and Network & File I/O. I’ve asked quite a few Mac users what they use Windows for, and none of them have mentioned needing to use Office. From my research, Mac users most often use Windows for relatively obscure software that doesn’t have a Mac counterpart, for playing PC games, or for browsing web sites designed specifically for Windows users. Often times Windows is used by Mac users to support legacy hardware.

Unfortunately, for me to do all the real-world testing, I would have to buy tens of thousands of dollars of hardware and software. Then I would have to learn how to use it. So I’m focusing this article on other common uses of Windows for Mac users.

That becomes quite a challenge as finding an application to benchmark between the three solutions is quite difficult. For example, I thought benchmarking the Retouch Artists’ Photoshop action using CS3 would be a good method. However, booting via Boot Camp is different “hardware” than from Parallels, so the trial promptly expired. (That’s simply a result of not having the resources to get a full version of Photoshop for this testing, so obviously Parallels shouldn’t be faulted for that.)

Not only that, but finding a benchmark method that is accurate in virtualization also comes into play. MacTech makes a comment about it possibly being a caching issue. I’ve asked around and one response I received suggests it may be an issue with time tracking in the virtual machine – that as the VM is brought to a crawl by the test, time also slows within the VM and thus results in a skewed outcome. This has proven itself true in testing. For example, while running Parallels the Windows system clock runs slow when the system is under stress.

In my spare time over the last four months, I’ve been working on tests that I could run in all three solutions and gain accurate results. The issue was compounded by my interest in testing Vista and XP. Only after yesterday’s update from Parallels have I been able to test a Boot Camp partition running Vista SP1 via Parallels.

Here are the initial results:

What’s next

I’ve already run hundreds, if not thousands of tests. I’ve had to scrap a lot of those. So now I’ve plotted a course to redo all the test that work. They’ll be done in a specific order. I’ll release the results as the tests are completed.