VDI and TS performance on ESX, Hyper-V, Xen, and bare-metal results: Project Virtual Reality Check!

Jeroen van de Kamp and I are proud to finally announce the official launch of project "Virtual Reality Check" (VRC). This is a independent research joint venture between our companies--Login Consultants and PQR.

Jeroen van de Kamp and I are proud to finally announce the official launch of project "Virtual Reality Check” (VRC). This is a independent research joint venture between our companies--Login Consultants and PQR. The primary purpose of VRC is to release multiple whitepapers to provide information about the scalability and best practices of virtualized Terminal Server and desktop workloads. The first phase of Project VRC compares the performance of virtualizing Windows XP and 32-bit Windows 2003 Terminal Services on ESX, XenServer, Hyper-v, and bare-metal hardware.

The four resulting white papers can be downloaded for free from www.virtualrealitycheck.net. (free registration required)

The goal of Project VRC is to investigate, validate, and find answers to the following questions:

  •     How do various Microsoft Windows Client OSes scale when used for virtual desktops?
  •     How does a VDI infrastructure scale in comparison to (virtualized) Terminal Server?
  •     Which performance optimization on the host and guest virtualization level can be configured, and what is the impact of these settings on user density?
  •     With the introduction of the latest hypervisor technologies, can we now recommend running large-scale Terminal Server or Citrix workloads on a virtualization platform?
  •     What is the performance impact of adding Citrix XenApp to Terminal Server?
  •     How do x86 (32-bit) and x64 TS platforms compare in scalability on bare metal and virtualized environments?
  •     What is the best way to partition (memory and vCPU) the Virtual Machines and the hypervisor host to achieve the highest possible user density?

All together, we've carried out over 150 tests! That said, project VRC is not finished, and probably never will be.

Additional publications are planned about virtualizing x64 workloads and the other (Vista and Windows 7) client OSes. We're also looking forward to evaluating new innovations in the hypervisor (ESX v4.0, XenServer 5.1, Hyper-V 2.0, etc. ) and hardware arenas.

Keep a close look on BrianMadden.com for news about when additional white papers will be released,.

Join the conversation

9 comments

Send me notifications when other members comment.

By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy

Please create a username to comment.

Jeroen and Ruben, this is AMAZING! I can't believe the amount of work your two teams have put into this. For all people reading this, each white paper has spreadsheets and graphs from like 60 different tests.


Again, wow!! This is unreal...


Cancel

Nice work guys.  Impressive content.


Cancel

Very interesting reading literature, they confirm several performance tests we have been conducting internally as well (but as you know, ESX EULA forbids the publishing of performance results without VMware's authorization).


I do have a suggestion: please include the measurement units on the axes in the whitepaper. This makes it easier to see what is actually on a graph (performance in procent? in virtual machines? in concurrent users? ... )


Cancel

The VMware paper is authorized by VMware.


About your suggestion, you are right, this can be confusing. The vertical axis stands for concurrent users. We will clarify this when we update the Whitepaper series.


Cancel

Jeroen,


Sorry for the confusion, I forgot to add that we didn't release our internal performance tests because of the EULA restriction. I didn't want to suggest the lack of authorization.


Thanks again for the excellent work, I have almost completed reading everything and am left with one question: in each whitepaper, you end with describing each individual test (chapter 7: "test details"). The graphs shown there typically show a lot of fluctuation. I would expect a much smoother behaviour for the increase of the response time as the number of active sessions increase. Do you have an explanation for that? And also, how many samples did you average each graph over?


Thanks for the answer & best regards,


Tim


Cancel

This is fantastic! I've been looking for this type of benchmark comparison for a long time. I'm looking forward to going over this in detail.


One thing that isn't mentioned in the various performance index whitepapers is whether or not RVI was enabled (CPUs are AMD Quad core and AMD claims huge performance gains for high context switching loads like TS, SQL, and Exchange).


I would have loved to have seen proof that RVI yields performance gains TS/XenApp VMs (32-bit and 64-bit). RVI could sway me to switch hardware platforms if the performance gains are drastic. Intel seems to be dragging their feet with the release of Nehalem to compete with AMD on this front.


Cancel

I've just spent half an hour reviewing the results of the testing (not nearly enough time to fully understand it), but I'm completely blown away by the findings.


On first blush, it would appear that ESX 3.5 absolutely kills XenServer 5 when it comes to user density for VDI with more than double the number of VMs due to the page sharing capabilties of ESX (RAM quickly maxed out with XenServer).


However, when it comes to XenApp VMs, XenServer 5 nearly DOUBLED user density over what ESX 3.5 could do! Have I read this correctly!!


If so, as a VMware user I'm bitterly disappointed with these results as it means that I've picked the wrong platform for virtualizing XenApp servers. Ouch!


Cancel

I really enjoyed reading these documents, specifically the VMware document because I am responsible for VDI and XenApp in my professional life. I would also like to thank “VRC” for putting these together. As a community we need more standardized docs like this besides just opinions on each of our blogs. 


Below are a few points that stuck out for me because they reinforce best practices and plus they just make sense. I have added my comments/opinions for the purpose of  discussion.


1. Pre-booting VMs in phases makes so much sense. In a production environment with a high density VDI load one should take care to ensure his or her startup sequence is long enough to allow for proper startup as to not over tax the host at first boot. The default is 120 secs. What about the “Continue Immediately if the VMware Tools start” option, what effect will that have on a host with 80 Workstation VMs resident?


2. “Page sharing is most effective when each VM uses the same application-dataset.” What a great statement, however I am sure there are a lot of admins out there that feel ESX hosts that are used for Workstation VMs are a one size fits all suit. Based on the underlying technology itself it is clear that when designing a large VDI environment, a cluster should be used for like VM personas.


3. I liked this bit regarding Virtualized TS/XenApp because this is always a hot debate.  â€œâ€¦as long as the total amount of vCPUs in all VMS combined do not exceed the physical amount of CPU cores. “ I think that says it all. There is a statement “You learned everything you need to know in life in Kindergarten” I think the same holds true here. Provisioning hardware and software to do a specific purpose with realistic expectations. This is nothing new.


4. “Disabling Page sharing can only be recommended when enough physical memory is available for each individual VM and the host is dedicated for high-density Terminal Server workloads”. Simply put but I have seen times where XenApp has been placed on a box that is also hosting a DC, etc, etc..


5. The only criticism (and it is not even really criticism, more like a suggestion)  I have regarding these documents, besides the picture of the DL 385s with the 1 U space in between  , is that although I can see the requirements and output of the data for each test, I feel that we are missing a big factor in the overall scope of VM performance, not just the head-to-head test, and that is STORAGE!!! For me it would be great to see if other storage technologies would have faired differently in user density with the tested products. I understand 100% that for the purpose of this comparo that the same storage be used and in this case it just happened to be DAS. But storage is such a big part of HOST/VM performance and I feel there is a direct relationship to user density. When I first saw the R5 DAS I immediately thought, “Ok, so they are presenting 1 LUN to the host which is made up of 8 drives and the hypervisor itself is also installed on that same RAID 5 set”. (I am guessing since the partition config was not supplied) Again, I know the purpose was the comparison but I feel that the VM (VDI) density and performance would be impacted  with each additional powered on VM because there will no doubt be contention for this one LUN (one raid card, once SCSI bus, on RAID array). I am curious to see if the results would been the same if we placed 30 VMs on one LUN and 30 on another. Or some variation of load balancing LUNS per VM set.


Thanks again for the documents!!!


Shane


Cancel

Can someone please assist with creating an account on www.virtualrealitycheck.net as it complains about the password not being valid.  Despite any combinations of length, characters, upper and lowercase for some reason one cannot create an account.  Don't know if there is something wrong with the interface.


Thank you.


Cancel

-ADS BY GOOGLE

SearchVirtualDesktop

SearchEnterpriseDesktop

SearchServerVirtualization

SearchVMware

Close