Citrix XenServer vs. VMware ESX: head-to-head video

Now that all the April Fool's Day silliness is over, we want to make up for it. No more fake companies and no more Rick Rolling (that jumped the shark this year anyway), what we have now has some real value!

Now that all the April Fool's Day silliness is over, we want to make up for it. No more fake companies and no more Rick Rolling (that jumped the shark this year anyway), what we have now has some real value!

Today, we're releasing our new ad-supported public video player. As you probably already know, the videos from BriForum 2005, BriForum US 2006, and BriForum Europe 2006 have been available for free for some time. The videos from both BriForums in 2007 are still under lock and key (unless you've attended the conference or purchased a DVD set). In the coming weeks, we'll be releasing some topical videos from the past two BriForums that relate to current events, as well as any other videos we think are cool or relevant.  If you have any requests, let me know.

To get off on the right foot, we're releasing the much-requested "Xen versus ESX: A performance head-to-head comparison" session presented at BriForum 2007 Amsterdam by Ron Oglesby. Interest in this video was spurred when XenDesktop was released back in February, so we set out to make it happen.

To get started, click on the image below:

Xen vesus ESX head to head - Ron Oglesby - BriForum 2007 Amsterdam

Of course, if you like what you see here, you'll love attending this year's BriForum at the Navy Pier in Chicago, June 16-18.  For more information, check out www.briforum.com

Join the conversation

24 comments

Send me notifications when other members comment.

Please create a username to comment.

...I'm going to head off some of the comments before they come... People who attend BriForum can still (and have always been able to) login to BriForum.com and access the videos ad-free. The "pre-roll" advertisements are only shown when these videos are shown to the public (i.e. non-attendees). So if you don't like the videos, then either come to BriForum or buy a DVD. :) The sponsors of BriForum make it possible to have the event, so no sponsors = no videos.
Cancel
how meaningful this is given it is a year old. Any plans to update these comparisons with the latest available ESX and Xen versions.
Cancel
This isn't even 6 months old (Briforum 2007 Amsterdam was in mid-October), and aside from the rebranding by Citrix, is still quite relevant. 
Cancel
Sorry, some of his slides had much older dates on it. The availability concern, what happens when an adapter fails has been addressed for ethernet connectivity because NIC bonding has been added to XenServer. So network and iSCSI traffic can survive a failed nic/cable/port. Certainly the I/O issues will be addressed, if not in the 4.1 release of xenserver, in upcoming releases given Microsoft's, Citrix's, and Novell's alliance against VMware.
Cancel
85$ per writable gig, some salesman did a dang good job selling those guys their storage.
Cancel
Everything in here about ESX vs. Xen has to be taken with a grain of salt because it only compares ESX 3 against XenServer 3...much has changed. ESX 3.5 is out and XS 4.x is out, and there really should be a new comparison done. This was current six months ago, now it is historical data. His off-the-cuff comments about various topics are very useful though. The main takeaway is that the big difference between ESX and XS is in disk I/O, ESX was much better, we now need to know if XS has closed this significant gap yet.
Cancel

Depends... Lets assume you use a DMX with replication and mirrored setups, plus you build in cost of support etc (ie a total cost) 85 can happen. I have been places that 100 would be the price per writable, and as low as 35 dollars per without the replication.


Of course there are lower cost disk solutions but 85 (while on the high side) is not unheard of.

Cancel
I can update it whenever I get a chance. Have to realize these are done (by most speakers that present at briforum) for free. Updating as new releases come out generally doesnt happen as we all ahve other jobs. Basically they get updated when we do the next presentation.
Cancel

First of all a hotfix for XenServer 4.0.1 was released last year to address some performance issues with TS/CPS: http://www.citrixxenserver.com/Pages/Download_4.0.1_Hotfix-2007-004.aspx


With 4.1 there are some specific new memory tweaks, built into the XenApp VM Template, which makes the performance even better. Also I/O drivers (PV drivers and Dom0) are improvide and optimized.


Stay tuned for a performance whitepaper on the Citrix Knowledge Center in a couple of weeks on this topic, although it probably will not include the VMware results, because of VMware's (ridiculous) policy on publishing performance results (what are they afraid of).


I'm not sure what the idea is behind this article? Promoting BriForum maybe? I think that if you post old news that you also should clarify that things have changed and how.

Cancel

The primary comparison points in this presentation is that in the testing environment, they found two things:

1) Regarding CPU (and I think memory usage) and # of possible sessions, ESX 3 and XS 3 are quite similar, with ESX having perhaps a 3-5% lead in this area at most. Graphs etc. are quite similar.

2) Regarding disk I/O, IOPS and similar topics, ESX 3 is about 50% better than XS 3, so the question is whether one is willing and able to accept lower disk/data throughput to save money on licensing and put that money toward storage, which is the most expensive aspect of virtualization (that's why the $45-85 per writeable gig).

I think the primary update and more information that most people REALLY want is to know whether in XS 4.1 Citrix has closed the gap to say, 20-25% slower than ESX 3/3.5?? This much improvement would definitely make XS worthwhile for the majority of SMBs, since this would be "good enough."

HTH (it is worth watching) 

 

 

Cancel
There's a lot to I/O than just disk access.  All traffic going in and out of your NIC is I/O.  All keyboard and mouse activity and screen updates are considered I/O. Ron knows he could have gone further to illustrate the severity of Xen's shortcomings by instumenting some scripts around client/server apps, Exchange (send/receive emails, edit calendar, etc).
Cancel

One important thing to note is that ESX server will cache guest disk reads and writes, while XenServer will not.  IOMeter is trying to measure throughput from user-space reads to physical medium, and so disables caching in the Guest OS. Because XenServer doesn't cache guest reads and writes, if you run IOMeter in the guest you get the actual uncached throughput and latency.  But if you run IOMeter in ESX, even if you turn off caching in the guest, you still get caching.  So running in ESX you're always seeing cached throughput and latency, which can't really be meaningfully compared with XenServer's uncached throughput.

Cancel
The talk mentioned that there was almost a 50% hit from running Terminal Services inside a virtual machine.  The new XenServer release coming up will have optimizations targeted specifically to optimize running XenApp inside of XenServer, that according to our tests make it far out-perform ESX server when CPU is the bottleneck.  It's likely to have a similar effect on Terminal Services.  There are several other key optimizations as well, so I'd love to see the results again on the forthcoming XenServer release.
Cancel

These two responses have good points. However, they seem to assume that Citrix is not making any effort to improve I/O -- disk or otherwise, and they don't state what effort has gone to improving this.

Furthermore, the effect of I/O varies with size of one's environment. An Exchange server with 100-150 mailboxes is very different and smaller than an Exchange server with 1000 - 10,000 mailboxes. SMBs have different usage characteristics than do enterprises.

Many VMware fans are enterprise users, and for them ESX makes a lot of sense. When you have a much smaller environment, where the need for speed is not paramount, then XS might be possible.

Citrix would not have bought XS if they did not think it could be a contender, Microsoft did or could not buy them because of anti-trust, probably.

Talking about how good ESX vs. XS really depends on versions being compared, I was asking for a comparison of XS 4.x with ESX, which would be more current and up to date. 

Cancel

Then maybe Xen needs to play catch-up by implementing caching.

Cancel

every version of ESX i have head the EXACT same thing. a perf increase of 3 or 5 % when taking a 50% perf hit is not impressive from a cost perspective. I'll beleive it when I see it.


 

Cancel

So since Xen doesnt cache ESX should be penalized...


 that is kidna like comparing two physical systems with built in scsi controls and one has cache on the control and one doesnt. Then saying that comparison isnt valid... end user performance is different between the two so why not show it....

Cancel

You're not getting the point.  Why does IOMeter turn off caching in the guest in the first place?  It turns it off because it wants to make sure that it's measuring the cost of round-trip going from user-space all the way to the disk and back.  If we changed IOMeter to enable caching in the *guest*, and then allocated the maximum amount of memory to the guest, XenServer and ESX would get similar performance numbers -- which would be useless, because once the cache was warm, you'd be measuring the speed of a memory-to-memory copies, not the speed of passing requests on to the physical hardware.

The comparison with a SCSI controller would only be valid if ESX had a fixed amount of dedicated memory that would always be used for caching requests.  But it doesn't.  The amount of cache ESX has depends on the total memory of the system, and the total amount of memory in use by VMs.  When you have a relatively idle system, with only half of the available host memory used by a VM, you'll have a relatively gigantic huge cache in ESX (do SCSI controllers ever have 2G of cache?), and your IOMeter numbers will look pretty sweet.  But when you have a fully-loaded server doing gigabytes of disk transfers, and all available memory in use by VMs, the little tiny bit of memory ESX in the buffer cache won't help a bit; and your overall disk performance won't in any way reflect the inflated numbers you got from IOMeter. 

Cancel
Based on Xen resource, using the paravirtualizaiton driver for network and storage can reduce the overhead from 30% down to 5%. So does your result shows the difference with that driver installed?
Cancel

So the tests run with mulple VMs isnt valid... ok. Of course if the test showed Xen whooped up on ESX the Citrite's would be fine with it....

Cancel
Xen tools were installed
Cancel

In the future I would recommend that if the folks at BrianMadden.com want to maintain some sort of credibility, you should be much more scientific in your reporting of test results. My sources tell me that this testing was done in April 2007 and published a full year later.


Can you give any details on the testing methodologies?

Cancel

...the relevant issue is not that it's only six months old, but that it's two full releases old.

 Did VMware grant permission under their EULA for these numbers to be presented, btw?

Cancel
Now that Ron Oglesby has tested ESX 3.5 and XenServer 4.1 and presented his findings at BriForum 2008... and XenServer tied ESX on all tests and beat it on many... it seems like time to update this...
Cancel

-ADS BY GOOGLE

SearchVirtualDesktop

SearchEnterpriseDesktop

SearchServerVirtualization

SearchVMware

Close