Our Geek Week: VDI Shootout methodology (and how you can help)

Geek Week is here. We've got conversations going with VMware, Citrix, Microsoft, Quest, and Red Hat about who they're sending out and when

The Super Bowl is over (congrats, Saints…hopefully the Browns can do that someday), and we're getting into full swing planning our VDI shootout, otherwise known as Geek Week.  We've got conversations going with all of the companies involved (VMware, Citrix, Microsoft, Quest, and Red Hat) about who they're sending out and when, but we've yet to formalize a test plan.  I know we've asked before, but we now have a bit clearer idea of what we want to do, so I want to spell that out and see what your suggestions are.

The Plan

First off, we're not trying to do a scalability test or to pit any single hypervisor against another.  In fact, we're not even out to thoroughly pit one solution against another, because we can't test scalability and performance for the many different use cases for five different vendors in one week's time. 

For this week, our goal is to expose everyone to the top solutions on the market, so that people with experience with only a few solutions (or none at all) can get a taste of what else is out there. Then, people can dig in deeper with anything that seems interesting and apply it to their specific use case, rather than some catch-all that we would try to put together.

The plan is to go through an installation of the products, which will hopefully take half a day or less.  After that, we'll take a look at the configuration and management of the product before moving on to the user experience.  Finally, if we haven't already hit on them, we'll take a look at any other features that people say they want to see.  If we hear "I want to see Quest EOP," we'll do our best to get that in there (that's a poor example, of course we're going to look at EOP, but you get the point).

We'll rely on screen captures and video pretty heavily, especially in the case of user experience, where an over-the-shoulder video in real time will give us the best vantage point for what's actually going on.

As for timing, we're going to start working with the products on March 8th, at which time we'll tweet about it and maybe write a few things or post interesting things to the website.  Then, we'll take a couple weeks to assemble all of the raw video and notes into something more concise.  On March 29th, we'll begin posting articles, videos, and whatever else we come up with, one vendor per day.

What we've got to work with

Our hardware is modest, but not bleeding edge.  We've got a handful of different thin clients to use, which will allow us to show off the capabilities of each solution without getting piles of thin clients to sift through.  Maybe someday we'll do a Geek Month full of thin clients, but don't count on it.

The servers we've got are Dell t605's, one with dual six-core AMD processors and 16 GB of memory, and the other with dual quad-core AMD processors and 12 GB of memory.  Each server has four 160 GB SATA drives.  Certainly nothing fancy, but it should get the job done. 

What we need from you

Now that you know what we want to accomplish and what we've got to work with, we need to know a few things from the community. Things like:

Storage - even though we're not doing scalability, will we need something more robust (maybe I shouldn't say "more," since four SATA drives could hardly be considered "robust")?

WAN Emulation - I've talked with a few vendors about simulating WAN conditions, and I think it's an important thing to show for each product. Everyone seems to have their favorite simulator, and I'm curious what everyone else uses.  So far, the suggestions have been Network Nightmare, WANem, and Apposite Technology's Linktropy Mini2.  If you have any endorsement for one of those products or would like to throw another one out there, we'd love to hear it.

Applications - We want to keep this pretty simple, so the apps we do are probably just going to be simple things like Office, IE, and such.  If there's anything specific that you'd like to see that won't require a huge amount of extra work that would take time away from the actual VDI products, let us know what they are.

Actual test methodology
- Since we're not doing scalability testing, we don't need to go nuts with something like SysMark or anything like that.  We just need to make sure we do as much the same between vendors as possible.  If you have any additions, subtractions or suggestions to make about the plan outlined above, let us know.

That's all I can think of at the moment, but (this is the last time I'll ask) please let us know what you'd like to see.  If you don't want to comment, shoot me an email. We want all the thoughts and suggestions that we can get so that we can make the best use of our limited time with each vendor.  If this is successful, we can then decide how to proceed with future Geek Weeks, maybe diving deeper into one product or doing the same high-level taste of some other technology (client hypervisors, layering, etc…).

Join the conversation


Send me notifications when other members comment.

Please create a username to comment.

1) Please consider testing alternative client platforms, like Linux, CE, WinEmbStd, Mac, iPhone, etc...  Many of these vendors are attempting to support 'connect from anywhere & anything' but fall short on execution.

2) Security and encryption capabilities

3) Documentation and support


I'll post a wish list later but I agree with Rodd...

Client connectivity testing would be good to see. OS X is my primary focus outside of Windows. Quest seem to be leading in this space - out of the 5 vendors chosen for this project anyway. Citrix's client for OS X still lags behind in performance and features. VMware haven't made any real effort. I guess the same goes for MS. Can't speak about red hat as I've not used it.

I would also like to see USB connectivity testing, media re-direction, etc, from different client OS's.  


I really think this scenario is doomed to failure because of the limited time constraints you've set... 24 hours to install a virtualization solution and then get VDI going... and then actually doing something?

Perhaps you're very skilled on the installs of all of them so you won't run into any pitfalls, but I still think giving yourselves more time will reward you in the long run and you can try out more things with the extra time.


My primary on other OS support would be OS X also.

I would combine the apps/HDX equivalent features for testing.

USB Connectivity - Scanner and software or somethin equiv. or maybe multi-function printers.

Two way Audio/Video Conferencing- OCS equivalent like Skype or something that wouldn't require an infrastructure setup testing over both a LAN and WAN/Internet connection.

Streaming from youtube/etc over both LAN and WAN/Internet connection.

Graphics intensive apps (don't know what to suggest but for me it would be a PACS/radiology app).


Not sure what the budget or space allows, but this 'roll your own' storage recipe should provide some nice bang for ya buck, along with some 'enterprise grade' protection features....


- Add LSI MegaRAID SAS 9260-8i controller to server

- Toss in 2 or 3 Crucial RealSSD C300 256GB drives (Tier 1)

- Combine with 2 or 3 Seagate Cheetah 15k.7 600GB drives (Tier 2)

- Then stir in pre-prepared SATA drives (Tier 3) (Or get new Seagate 2TB XT drives)

Now for the magic sauce....

Pull all this together with NexentaStor 2.2 NAS/SAN unified storage software solution that utilises ZFS, leverage the VM Data Center 2.0 and/or Citrix StorageLink plug-ins, protect with RAID-Z, share volumes via NFS, and indicate the presence of SSD's for maximum zest.

Whammo! With a 45 day free trial available, it should do the trick nicely.

Or, if the cake doesn't bake, just buy a NetApp FAS ! ;)


I Agree with Scott (above) I think you might be a wee bit opptimistic on the time scales? But I hope it all comes together ;-)



On the Wan Simulation side of the house - they're all ok. Just make sure you're throwing packet-loss into the mix as you push up the latency. Another, often overlooked, piece is to get congestion and bandwidth restrictions in the mix - like the real world ;)


Ok my wish list (so far) in no order...

- Different client OS connectivity (discussed above).

- IT Administration ACLs (granular admin, helpdesk, etc).

- User/session remote support options (session shadowing/remote control).

- Desktop entitlement methods? – can it be done just on AD user/group or can it do the good stuff based on end-point location (IP range, etc).

- Ability to deploy packages to the hosted desktop? Stream, MSI, etc, from the broker/management console.

- Auto-expansion of managed desktops pools – can they all do this?

- Integration with Flexclone, link clone and whatever it’s called in Hyper-V :) - or any other storage cloning technology.

- Single image/template (golden image) options – if available.

- Peripheral based policies (USB, drive mappings, re-direction) based on end-point location.

- Other location awareness goodies – things like the end-points, MAC, IP, etc being passed to the hosted VM or TS session. This can be useful for certain package license nightmares.

- Bandwidth throttling for both VM/TS session and/or peripheral devices.

- Database integration? – Just MS SQL or do the others get a look in?

- End-point agent/plug-in “pass-thru authentication” support. Kerberos, etc?

- High latency performance.

- Bi-directional audio support.

- Just hosted desktops and TS session or can they support physical kit – including desktop streaming?

- User/group profile management features, folder re-direction, login times, etc. Anything that doesn't require a third party product such as Appsense (though I like it).

That will do for now – I might have a proper think after breakfast.




@ Scott Dowdle - I don't think the plan is to unpack & rack servers, install the hypervisor, configure storage, networking...and install and test the management components (broker) and clients.

I think they plan on having the platforms, networking and storage ready, so the day with each vendor will be spent installing and testing them management components/broker and clients.

You would be correct that doing all of this in one day would not be leaving much room for error.


I would consider keeping this very simple given the time and test the basic assumptions first before jumping into the advanced management capabilities that some above are talking about.

Goal 1: Can you run on the WAN?

Can you run rich video content at, low bandwidth, latency and packet loss, does the audio stay in synch?

What happens to the bandwidth when you run these tests. (no use killing BW to get it to work).

Goal 2: Can you do basic desktop stuff?

Run Office apps across the network, what happens when the network degrades

Print to a network printer

1-2 USB peripherals

Connect to both XP and Windows 7 hosted desktops?

The setup etc does not matter, if the basics don’t work.


What AppD said, but add a Wireshark trace so you can show bandwidth graphs ;)


if anyone on the post works for Shunra they should offer to loan you a box to get test latency and packet loss (great tool.)

Some big app use requests seem to be graphic (large still images, 3d and image manipulation.)  ERP, Print, peripherals, multimonitor.  

Gabe, folks also want to know the real world total cost for deploying each of the vdi solutions.  ie if one must run on a specific tool then include the cost in the total.