Problem :
I am in the process of running some benchmark tests on a variety of RDBMSes, and I am testing three different client operating systems.
Would it be terrible to partition the HDD and install Ubuntu, Windows 7 and Solaris 10? If I do not do this, I will likely have to re-install the O.S numerous times due to changes in the database server OS as well. The same question goes for the server; would it drastically affect the performance to install three server OSes on one server?
Solution :
The client-side software is likely to be fine sharing a single disk between three separately bootable o/s. You might even be able to use VMs sensibly here.
The server-side software is much more fraught – you will be in grave danger of comparing apples and oranges and melons because if you partition the server disk into 3 sections, you may end up with different performance for the inner vs middle vs outer section of the disk. That said, the seek time is probably most important, and since you’d be sweeping just one third of the disk instead of the full disk, you might get slightly better performance in the test environment (with more limited capacity) than in a production environment. If you reinstall the different o/s each time on the machine, you might still get different performances because of the default layouts of the systems on the disk. Benchmarking is very hard when done properly.
Note that any of the client systems should be able to communicate with any of the server systems; you could have 9 combinations to test.
There’s nothing wrong with repartitioning a laptop – I’m currently doing it, and it runs fine.
One alternative physically swapping between disks (easy on some laptops, like thinkpads, hard on others). It’s probably your best option if you intend to do a lot of swapping around, and want to save time.
A second option I’d suggest is getting each OS up to a baseline install and imaging it, then restoring it as needed – if there’s not a huge amount of data per OS this should at most take 15-20 minutes per partition. You can then modify/update on that system, reimage and work on that as needed.
On a server, if you have the space and resources, you could probably get away with multiple hard drives.
The best practice while benchmarking is to recreate the production environment and keep environment identical for each option .What I would suggest is that you do a fresh install of all the OS and carry out all the benchmarking processes.(Only one OS at one time on the machine)
This may sound tedious but this is the only way you are going to get genuine benchmarking reults .
You can check you nhinkle’s blog entry Spring 2011 Browser Roundup to see how he sets up the testing environment.
There is a good chance you cannot do this at all accurately with what you have. I’m assuming in the end the RDBMS will be deployed on a baremetal server. Or perhaps on a VM. Regardless, if you are not testing the same environment, there will be all kinds of caching, I/O, CPU latencies that will drastically warp your benchmark.
So if, and ONLY if, you plan to run this OS and the database Server on this laptop will the benchmarking be accurate. Otherwise it may be “interesting” but not even necessarily accurate for relative testing, let alone absolute testing.