Qumranet testing Day 1: a new method for benchmarking. Also, XenDesktop hangs and VMware withdraws.

Gabe and I are in Sunnyvale, Calif., this week working with Qumranet.

Gabe and I are in Sunnyvale, Calif., this week working with Qumranet. Last week we wrote a blog post about our project describing our plans and asking the community for input as to how we should conduct our testing. (The short version is that we are evaluating Qumranet, Citrix, and VMware in the VDI space, and we had some questions about the best way to conduct our tests. Please read that original article before you read this article.)

Yesterday was the first day of our project, and today's article shares the final test plan that we put together (based on everyone's feedback) as well as an update on yesterday's progress.

Our (revised) testing methodology

Last week we wrote that our tests would focus on stressing the three vendors' respective hypervisors: KVM for Qumranet, Xen for Citrix, and ESX for VMware. Since we made that original plan, we've talked to several more people and received a LOT of feedback and comments on the article. After sleeping on this for a few days, we decided (or agreed with the people who said) that there really isn't that much value in testing just the hypervisor.

The problem that many people pointed out in the comments (and many people we talked to told us) is that it's easy to make a hypervisor that can crank through these AutoIT scripts as fast as possible, but what does that actually mean for end-user experience? Who cares if a script can run 10% faster if the VM host is ignoring clients and not sending screenshots down?

Therefore we decided that we'll look at several components of the VDI solutions, including:

  • The density of users / VMs on a hypervisor running a desktop workload
  • The thin provisioning process, performance, and management
  • The remote display protocol performance

The user density thing is what we were originallly planning to test, so nothing new there. The thin provisioning process is interesting, because a lot of people asked us whether we would evaluate Citrix Provisioning Server this week. Qumranet actually has thin provisioning themselves (where many users can share the same master image, but each with their own diffs), so this would be a great test to do. And then of course there's Qumranet's spice protocol, which as we wrote last week can support multiple displays with near-perfect quality, even with stuff like Skype and multimedia.

So this new plan is exciting for us because we'll be testing three components of the solution instead of just one. Let's look a bit more at how we'll run each of these tests.

The user density / hypervisor test

Last week we'd planned on writing our own AutoIT-based scripts to simulate office workers. Since then, someone pointed out that the Business Application Performance Corporation (Bapco) has an industry standard benchmark called SysMark that does almost this exact same thing. (They even have an "Office Productivity" benchmark.) So rather than try to write all of our own scripts, we decided that we'd just use the industry standard benchmark.

We can launch SysMark from the command-line in each VM. We're planning to just loop the SysMark tests and add more and more VMs to the test. Then about every five VMs or so, we'll run a single SysMark test that we'll get certified with Bapco. We're also planning to video record this process, so that we can publish videos that show things like "here's the experience with a SysMark score of 100. And here's the experience with 120. Etc.

We also decided that we're not going to include multimedia in these hypervisor loading tests. (We still plan on evaluating multimedia, just not as part of this test set.) The reasons for this decision are twofold. First, no one can agree on what percentage of users should run multimedia in our tests. Some want 50%. Some want 20%. Some say that at any given time, it won't be any more than 1 or 2%. So we decided that we'd evaulate the experience of different works with office apps as our baseline, and then we'd see how multimedia affects that later.

The second reason we decided not to do multimedia at the same time as the hypervisor loading is because different vendors accelerate and enhance multimedia in different ways. For example, VMware uses a multimedia redirection technology where they just send the original media file to the client where it's rendered locally. The problem is that you actually have to have a client device connected to the VM in order for this to work. If you just fire up a VM and run a script with no client, the host VM won't be able to leverage the multimedia redirection technology which means it will be forced to render the media stream within the host VM. This would artificially lower the performance or number of VMs we could get on a server.

So if we connect clients to the hosts for our hypervisor testing, can those clients be VMs? If so, do they have to have real screens displaying, or can then run headless? This issue becomes really complex really fast. These are all great questions that I hope someone can answer, but we simply don't have the time in our one-week evaluation to explore all of these options.

Therefore for hypervisor loading, we're going to focus on office apps running in a headless way. (Again, keep reading to learn about how we're going to test multimedia.)

Thin provisioning

Citrix's Provisioning Server has been the leading product in this space for years, although as I said Qumranet does thin provisioning and can stream these images on-demand to clients via NFS. VMware previewed a similar technology they called "SVI" (Scalable Virtual Images) at VMworld in Cannes earlier this year, but unfortunately that product is not out yet. So for this week's testing, we can only analyze Citrix Provisioning Server and Qumranet Solid ICE.

Remote display protocol performance

In the past I've written about how awesome Qumranet's spice protocol is, and I've linked to our YouTube video showing it running on a client with four displays in a heavy multimedia environment. Since then, a lot of people have criticized spice, saying things like, "sure that test is cool, but it requires 60 Mb per second of bandwidth. ICA could do that too."

My response to this is that I honestly don't know? Citrix ICA connecting to a XenDesktop client with four screens, multimedia, Skype, nick.com, etc. with unlimited network bandwidth? How well would ICA perform? And on the flipside of that, how about Qumranet spice running just regular business apps on a single display? How close is that bandwidth usage to ICA? And what about user experience? Bandwidth is only part of this equation. How good is the experience with certain apps at certain bandwidth levels?

We decided to answer all of these questions this week. We're going to compare ICA and spice head-to-head in many different scenarios: business apps, multimedia, single displays, multiple displays, etc. We'll look at bandwidth numbers and we'll also use an external video camera to record the performance as experiences by the user.

For these multimedia and protocol tests, we'll also run them with plain RDP too, just for fun. (Although that will be of course limited to just a single screen.) But this should be pretty cool: spice, ICA, and RDP, all head-to-head in different scenarios, all with network loads tested, and all recorded on video for head-to-head comparisons.

VMware's new version of VDI enhances the out-of-the-box RDP with multimedia redirection. We'd like to test that too, but we haven't been able to get a license from them yet, so it looks like that will remain untested. (More on that in a next.)

VMware withdrawals from testing

Many of you probably have heard that VMware "does not allow people to publish benchmarks." This is somewhat true. The license agreement in VMware products states that all performance testing results must be approved by VMware before they can be published. To me this doesn't seem like they can legally enforce that, but personally I don't want to sued by them to find out. So for this project, we did the "right" thing and contacted VMware and shared our plans with them.

We went back-and-forth several times over the past few weeks, discussing our test plan and methodology and all sorts of things. As you know, our plans have changed quite a bit, and I think VMware was starting to get a bit frustrated with us since we couldn't seem to make up our minds about what exactly we'd test and how we'd run the tests.

Yesterday we told VMware that we would not be doing a full VMware VDI test (since VDI does not include thin provisioning and it just uses the RDP protocol). Instead we'd focus on Citrix XenDesktop versus Qumranet Solid ICE. By this point we still hadn't received our evaluation licenses from VMware (even though they said we should have had them last week). So yesterday we told them that we might still be able to test ESX with XenDesktop if they could still get our licenses. (Our thought process was that since Citrix XenDesktop integrates so well with XenServer and ESX, why not run the "SysMark" portion of our tests on ESX and Xen?)

VMware responded by telling us that since we're not testing the full VDI product, they're pulling our evaluation license request. They told us that maybe we could re-engage in the future.

We responded and to let them know that we still think evaluating VMware with XenDesktop makes sense, especially since Citrix really leads the market in the provisioning and protocol areas, but VMware leads in the hypervisor area. So really running XenDesktop with ESX-based guests is probably a very popular choice. Unfortunately VMware never wrote back, so I guess we're done with them for now.

VMware is very weird to work with. The license agreement of their products state that you're not allowed to publish the results of benchmark testing on their software unless you first get their permission / approval. There are a lot of people who have publicly questioned whether that restriction is even legal, but we figured that for this testing we'd do the "right" thing and try to get their approval.

On the flip-side, Citrix has been really awesome to work with. Anyone can easily download a 99-user, 90-day, fully featured evalution of the entire XenDesktop suite right from their website, and anyone is allowed to say whatever they want about their products.

Citrix XenDesktop Hangs

Building our environment and getting everything set up yesterday went really well, except for a few little problems with XenDesktop that we need to sort out.

The first is that we simply cannot get our session to use all four of the displays that we have on our Windows XP client workstation. All displays are 1024x768, and they're configured in a straight row from left-to-right. If we maximize the ICA session, it just fills one screen. If we view the display properties within the session, we do see the four virtual display adapters, but displays 2, 3, and 4 disabled. If we click the box to enable them, they light up, but as soon as we hit the “apply” button then they switch back to disabled. We’re using version of the desktop receiver client that came with the XenDesktop ISO. We tried to look online for newer clients, but man, Citrix's new download website makes it really hard to find things. We're not sure whether there is a newer client or whether we just screwed something up.

The other problem we're having with XenDesktop is that we found a technique that can consistently hang a VM. Our XenDesktop setup is very simple. We have a domain controller, then another 2003 server running all of the XenDesktop components (the desktop broker, web interface, licensing, and farm database). Our desktop host is a Dell 1950, 8 cores, 16 GB, with XenServer 4.1.

One of the tests we wanted to do was to see just how well ICA could do with multimedia and a lot of action over four displays. Since we couldn't get four displays working, we decided to just work with one display for the time being. If we click our movie, it plays no problem. It will play as long as we want. But when we grab the top of the media player window and start dragging the window around, our session / display freezes. When this happens, the audio stops too. We had netmon open on one of our other displays (because hey, we we're not using that for ICA :), and when this freeze occurs, the network utilization drops to zero. Since this was so repeatable we actually captured a Wireshark network trace. It's weird.. The Windows XP VM just stops sending packets.

If we look at the stats of that VM via XenCenter, we'll see it up around 45-50% CPU utilization during playback. But when this freeze occurs, the CPU drops down to 2%.

So we're completely stumped??? We're hoping to get on the phone with someone at Citrix today to work through these two issues. We did record a video which we posted to YouTube showing this whole process. If anyone has any ideas about this problem, we'll promise you lots of cool coverage on BrianMadden.com!

(A side note about YouTube: If you visit the actual YouTube web page for this video, there is a "watch in high quality" link under the video so you can see it at its native resolution.)

Join the conversation


Send me notifications when other members comment.

Please create a username to comment.

Do you guys think you could throw in some WAN tests? I am very interested in VDI from the datacenter and until we can get a kickass WAN protocol for VDI its going to hold us back. I setup XenDesktop and it worked great locally; however over the WAN it just locks/stutters

Based on some intial feedback, we applied the brand-new (as of July 14) Hotfix 2 patch to XenServer, and we were able to immediately reproduce the hang. (We launched a desktop connection, launched the video, moved the the screen around, and the hang occured.) So we're still stumped?

Great segment; Ill be following this testing closely

We talked about that last night, and honestly we will probably run out of time before we can do that. But if we have extra time, we'll definitely throw in a quick WAN simulator and limit the bandwidth and increase latency just to see what happens. But I'm really not sure we'll have time.
I just love how VMware has thier licensing to not allow people to publish scalability numbers, basically because they don't want bad things said about them. Now there is something I bet all politicians would love, no one is allowed to publish anything bad about them.  VMware makes things way too difficult, which has us looking at replacing our vmware servers with XenServer or Hyper-V

from here : http://support.citrix.com/forums/thread.jspa?forumID=187&threadID=103761&tstart=0

""It is possible that these issues you are seeing are related to the multimonitor hook (in particular the XenCenter console).
As a way to try this, first up disable the multimonitor hook.
Go to c:\program files\citrix\icaservice and rename mmhook.dll.
Then try out the applications you are having issues with.
If you find this fixes it, then you can make use of the multi monitor exclude list to make sure the hook doesn't attach to the executables causing you issues - while still allowing the hook to take effect for apps that do not interact badly with the multimonitor hook.
Go to
HKEY_LOCAL_MACHINE\SOFTWARE\Citrix\CtxHook\AppInit_DLLs\Multiple Monitor Hook:Exclude
add the name of the executables you don't want the hook to apply to. "

Also I would try to disable ICA features like compression, speedscreen and so to see if at some time you are not able to reproduce.


Hi Brian

We are using XenDesktop with 2 monitors.  We also had issues expanding the session over both screens.  What we have to do is resize the Desktop Receiver window, drag it between 2 screens and then maximize.  We are then informed that the "Virtual Desktop may not exactly fit your" desktop but just click "Yes" to resize.  Not sure if this will apply to 4 monitors, perhaps resizing the window over all 4 monitors and then maximizing will do the trick.

Cheers, Derek


Ok, we did a few more tests, and every one of these hangs the VM:

  • We updated the XenTools in the Windows XP VM
  • We got it to hang with a flash-intensive website (nick.com) in additionto our WMP movie.
  • We tried this with one display instead of four (even though we can't get the VM to actuall use all four displays, it still saw them).
  • We tried from two different clients (Mac OSX running Mac ICA client and another Win32 client running the Citrix Desktop Receiver client)

BTW, the XenDesktop admin MMC will still show the VM as "in use," even after this hang. If we give it a few minutes, it will eventually switch from "in use" to "not registered." But the only way we can get it back to "idle" (and back to being able to receive ICA connections) is to reboot the VM. (We can use XenCenter to do a gentle OS reboot though... we don't have to do the "force reboot" thing.)

And oh by the way, if you want to see crappy performance, try the Mac ICA client vs. the Windows one. :) As a Mac user myself, I find that using ICA is M-U-C-H faster / better / smoother by running the Win32 ICA client in a Fusion VM on the Mac.

Cool! That worked.. Is this a "feature" or a "bug"? It was kind of strange because we had to first stretch our window across all four desktops, then maximize, but at least it's working now. YouTube video coming shortly....
Renaming mmhook.dll did not work. As for speedscreen, there is no speedscreen multimedia redirection in XenDesktop. :(
I'd second on this request...SPICE sounds awesome, but is optimized for 20-30milliseconds or less...I'd really like to see a comparison on the effectiveness of remote graphics presentation by SPICE/RDP/ICA as latency increases. I think it's probably a given that ICA will win - but by how much?

Ok, the 4-screen ICA / XenDesktop video is up on YouTube now.


Note that this also has the "high quality" button

Okay, just got an update from Citrix on this... This behavior is by design. It will maximize only to the window it's in, with the idea being that some people will have full remote desktops running on some monitors while others are local. The good news is that if you do expand it across multiple displays, it will remember the layout settings for next time. Thanks to Simon Crosby for taking the time to help with this!
Ok, I just got off the phone with Simon Crosby. He said this was a known bug that's been logged. For now, the workaround is to either (1) disable "show window contents while dragging," or (2) disable session reliability's caching. For our tests we're just going to disable show window contents while dragging and press ahead. Again, thank you Simon!
We found the source of this problem and have a workaround. See the comment thread a few levels above this one for details.

No problem, glad I could help.


 Derek Darling


Hey Brian, wrong Simon.  I phoned you earlier.

Simon Plant, Senior Product Manager, XenDesktop :-)  Glad we got it sorted.



OOOHHHH... "Simon from Citrix in the UK"... Ha!Man, that makes so much more sense.. I mean I was wondering (1) why Simon Crosby would call me, and (2) why he would know so much about XenDesktop...

Man.. too funny. So, Simon Plant, I owe you a beer for sure!

Correction. That was Simon Plant who helped me, not Simon Crosby. Oops!

Brian, shouldn't this be 60Mb, not 60MB?

Hi Ormond, so it sounds like SPICE is really optimized just for LAN only? Is that the case?

Yep.. great catch Patrick. I just changed the article.



here is something else you may want to apply on your xp images and Provisioning Server to  see if it helps... Also are you planning on using the ICA  11.00 beta client for xendesktop testing?




Tony Sanchez Citrix System SE




lets say you are wanting to transition into virtual desktops. And you know you need something that delivers fast graphics as well. Xendesktop and Spice. Vmware just does not sound like it can meet the needs. You really do not want to purchase alot of hardware. Do you just think by default that SPICE is what you would want?

The other side, My question is, would you buy the spice for instance.... wait for the updates until you finally get what you want?

Also, As for transitioning local workstations to a sort of VDI solution. Would you just purchase cheap desktops / notebooks or thinclients (I dont like that thinclients have a cd burner),

I dont know... Whats your thoughts?



Just wondering if Qumranet has engaged you in a pay for play scenario?  I just can't believe that the former "Mr. Citrix" would even consider a protocol that needs a minimum of 60 Mbps and < 30 ms latency.  Have you sold your soul to the devil.

 Will you produce your test plan?  Did VMWare really withdraw?  Or are you not sharing everything.  I smell something that's less than the truth in your title.  I don't work for VMWare but I know them very well.

Brian, 60 with an M after it is bad.  What the hell happened to you?

I didn't say that spice requires a minimum of 60 Mbps. I said that is the (incorrect) impression that people got because they heard it required that. But the demo of spice was with a session running four monitors, with high def video, Skype, pollypocket.com, and Word all running at the same time. Part of the tests we're doing this week are running those exact same scenarios via ICA and RDP, while video recording the client experience and using camtasia to capture the client side network. We'll get all these videos up next week, but let me tell you, the spice experience with multimedia is amazing, and using it for regular office is like ICA or RDP in terms of bandwidth.

As for the test plan, yes, we've outlined a lot of that over these past few days, and the white paper that we write will include everything, including our setup, our scripts, our plan, etc. It will be 100% reproducible by anyone.

Finally, yes, VMware did withdraw. What I wrote today was an exact quote from their email. They felt that since we weren't testing VDI, they didn't want to participate at just the hypervisor level. Then again, I was engaged with their VDI folks, so they personally probably don't have as much of a stake in the hypervisor space. But VMware has been cool about some stuff too. They have a guy there who was one of the developers of SYSmark, so once they learned we were planning on using SYSmark, they got us in touch with him and he's been a big help.

As for less than truth, of course you're entitled to your own opinion, and when we write about contentious stuff then there will definitely be people who don't believe what we say. But such is life. All I can say we've been blogging for a long time, so there is definitely plenty to read when forming your opinion of us.

You mean the "B" is bad, like MB versus Mb? Yeah, I guess that's a bad use of the shift key!
Is Qumranet paying you in any form for these tests?

Hey, you didn't read the first blog entry that I asked you to read before reading this one. :)

But the answer is YES. Qumranet hired us to analyze their product and compare it to other VDI solutions on the market. Our deliverables to them include performing the actual analysis as well as writing a paper about our results.


Well, spice as a protocol is only available with Qumranet's Solid ICE VDI product. So if you want XenDesktop you can't have spice.

But the rest of your questions are so hard to answer. I would check out this blog post for a starting point.

Hi Brian, if you still have the issue with the hanging VM at some point, you could try ditching the Broadcom nics and replace those with some Intel NICS....good luck with testing...
Is the Qumranet solution compatible with the ESX hypervisor and VirtualCenter?

The Solid ICE whitepaper on the Qumranet website suggests that the SPICE protocol is superior to RDP in a LAN environment but that RDP can be used with Solid ICE for connections over a WAN.  It worries me slightly that they suggest to reverting back to RDP when using a WAN connection. 

I know, for one, that my company has more concerns about how a protocol performs over the WAN since both RDP and ICA work well enough on the LAN.  However, I do accept that SPICE will provide a better user experience for graphics intensive applications.

Most customers are testing this for the WAN, if you can do test that, it would be very helpful.
Fair enough, I didn't read the first article.  Always good to have full disclosure.  Amazed at how many decision makers don't realize white papers are paid for.  Looking forward to seeing the test plan.
Agreed, while I might not jump out and replace all of our VMware ESX boxes just yet, as a VMware customer, I'm extremely unhappy they weren't cooperating with you guys on this test.  I'll be talking to my VMware Sales rep today...
No. Qumranet Solid ICE only works on the KVM hypervisor. I don't have enough room in this comment to explain, but it has to do with the architecture of spice, and the components that run on the host but outside of the guest VMs.
We found the cause of the hanging VMs. Read the comments higher up in this thread list for details.

We'll have to discuss WAN in another article to give it the full attention it deserves. The short answer is that running a protocol over a WAN equates to SBC from the central location, but there are other ways to solve that problem, like running VMs locally at the WAN location (either on local servers or directly on the clients). But whether you can do that depends on the specific use case and where the data is and all that stuff...

With regards to spice, Qumranet specifically built this to be a LAN protocol, so I don't think there's any value in testing over a LAN, specifically when they themselves recommend using RDP on the LAN. Testing spice on the LAN would be like testing a Ferrari on a dirt track--it's just not designed for it and we already know it would not be good. As for testing RDP on the WAN, there is mounds and mounds of data available on that already, so probably no value we can add.

oops.. I used the word LAN a few times instead of WAN.. That second part should have read " I don't think there's any value in testing over a WAN, specifically when they themselves recommend using RDP on the WAN."
Guest summed it up, but to quote their whitepaper... "SPICE is currently optimized for LAN users and environments where the client is reasonably closeto the data center (typically 20-30 msec or less)."
You would think by now companies would realize (especially when there is a viable FOSS alternative) that they can't get away with Gustapo tatics.  Combine that with pretty expensive licensing, and other than their head start in the market, I don't see how VMware is a leader (for long).

testing it over the WAN is actually is the make or break test. take my company for example. we have currently 15 offices worldwide, and we're working to consolidate it to 2 datacenters, u.s. east coast and west coast. we're going to have VDs in each datacenters. how is spice going to help a company like us if we have many clients that will be connecting via WAN?

i'm not quite familiar with qumranet, but does it have its own file system like vmware does with vmfs? i'm curious what the IOPS are if connecting to its native file format vs raw disk mappings or xen's. we currently have vmware esx for our servers, and we had IOPS issues before with SQL servers when using VMFS compared to raw disk mappings.

I can't wait for the final results of the test. too bad that WAN tests are not going to be included. any chances you'll be testing developer apps with this test? good luck!

too bad you didn't include Provision with their latest RDP Experience Optimization stuff and specially with Parallels underneath.

This also depends on how it reads your native monitor configuration, and the software your local graphics card uses.  Because it's as far from intuitive (or even explained), I keep leaning towards oversight vs. by design, but as long as it works at all, it's a plus...

For example, if you have a regular old card, with regular old support, and no fancy configuration, this appears to be the behavior, and you sit scratching your head until you play some games with maximizing and resizing.  But if you have a high-end card, with more configurable local software that will let you quickly switch between span, or monitor awareness, or ten other modes, XenDesktop mirrors whatever configuration it finds there from a driver/configuration perspective.

With the differences, you'll see this show up in two ways:

1.  Either the scenario where when you go to display properties, you see it mirror the number of monitors you actually have, and curse at the screen when you check the use both monitors button and nothing appears to happen.

2. With the more advanced setup, you have it working, but are twice as confused, as going to display properties won't show all those client monitors -- instead it will show one, and instead list the card as something like "Citrix Virtual Display 0/0" or something to that effect.


In regard to Qumranet and WAN scenarios, you'd have to wait for multi-site and SPLICE (search on this site) for the open beta.  Maybe someone who attended BriForum can comment on any details shared.

I'm curious of what the architecture looks like, and hoping that's it's more than just moving VM's to the other side of the WAN and maintaining centralized control -- providing benefit to the user experience, and detriment to application performance through moving the "client" farther from any application backends.  I'm assuming it's not this way if SPLICE is more protocol modification in addition to architecture change, but...

As for WAN scenarios in general, I 100% agree, which compliments one of Brian's recent articles "Citrix's ICA problem is not as bad as the RDP problem...." about VDI needing to cover 100% in order to be mainstream.  Exceptions are the rule, but that will always be the case, and that's both from a product and technology limitation as well as an often, internal limitation. 

I would be content with "good enough" with the flexibility of LAN and WAN scenarios, ideally in providing capability, but also providing control of that protocol, which leads to the internal limitation factor.  Even with current and future protocols, we still have the bandwidth factor.  They will always exist.  With a LAN, give me the capability to do almost anything, but also the ability to control and throttle that experience a bit.  Give me the capability and let me model it to my network rather than me having to model my network based on your protocol...for "good enough."  With a WAN, even moreso.  Give me a usable user experience for common things, but even if the graphics intensive scenario mentioned above could be done over the WAN in 1/4 the quality, I probably wouldn't want to see that hit my network...

But we also have people looking at VDI for different reasons, or all reasons.  Whether it's specifically for high-latency networks, or if it's a local desktop consolidation strategy and everything in between.  To me, the sweet spot for protocols is the in between that catches the bulk.  Not the < 5ms or > 150 ms environments, nor the tiny or enormous pipes.  Give me good enough for 90% of the dead center, in-between for common scenarios, or the ability to variably control that experience, and we're headed closer to usability.

Basic Requirement: You should always be able to watch a small, but decent quality video that talks about all of the wonders of how VDI will save the world....in a VDI session, without the audio breaking up, and the video looking like a flip book.


Brian - I'm guessing it's way too late, but since VMWare withdrew, and you mentioned multimedia redirection, any chance it getting Provision/Quest in quickly, even just for MMR?  Since this was originally about density and the hypervisor, in RDP/MMR scenarios, at least for multimedia (and supported streams), you'd be able to show offloading of CPU locally, which would be complimentary to the comparison since Qumranet can do adaptive CPU offloading, RDP keeps it on the server, and RDP with MMR offloads (supported) multimedia, and keeps the remaining functions on the server.  This would relate directly to the scaling and hypervisor test, and only exclude one scenario, that being those protocols that offload the majority of processing to a server-side GPU.

I'm stunned....

Perhaps when calling someone out, asking for full disclosure, and questioning methodology and even ethical behavior you should log in with a real user instead of hiding behind an anonymous  guest login.

I don't work for Brian Madden and actually, I don't even know him, but something smells a little less the truthful in you post

This is a very good article. I have XenDesktop running on VI 3.5. Performance is great with all apps running from XenApp server. I am running with 4 X 20' monitors with 1600X1200 from the WAN. Sound and Audio are great from YouTube. :)

Brian, Hanging sessions on XenDesktop is someththing I've seen before. Try to shutoff TCP offload on your NIC. Also Session Reliability should be turned off.


Greeting Koen


Hi Brian, I agree with you about spice being developed for LAN environments, but on the other side ICA has been optimized for WAN, I would say for poor wan connection. So, would it be fair to compare ICA with spice only on LAN?



We got "Day 1" and never heard anything new...just curious to what the status of the benchmark is?
Yeah Brian and Gabe got fired or the results were not presentable. These gusy are just to far away for from the real production world and don't have the time to mess with it anymore... I guess?! :-) Reach out for the quick buck!
Hi Brian - still no update? XenDesktop hangs, VMware withdraws and Solid ICE melted due to global warming?
What happen to the test results? 

Wow, this is a really ignorant sub-thread.  Use the free VMware hypervisor for gosh sake.  It'd been out for weeks by the time this was posted.  Testing VMware's ESX agains XenDesktop or Hyper-V is like testing a Cadillac against a Taurus.  Good cars both, but one simply has many more features than the other.  FOSS?  You get what you pay for.

Come on guys... we're now more than a month past day 1... where's the results?


if you are not even prepared to give an interim update!!!

The lack of results being published may have something to do with the quote from the original article 'Therefore, Qumranet has hired Gabe and me to conduct performance tests of their product and to compare it to other leading VDI products'.

If results are seen as not favourable to the sponsor of the tests then they woudl be less then enthusiastic to loosen the purse strings for an unfavourable result. SPICE may go towards addressing the issue of the user experience issues experienced with RDP in areas such as video performance and multi media, but the cost looks to significently outway the benefit.

The VDI user experience is is on the constant path of improvement with large multinational vendors such as Wyse, Citrix, HP, Sun, Microsoft and VMware all actively working on alternate transports or extensions to existing protocols to extend the offerings or address the weakness's where does SPICE fit into the long term VDI vision.

Sun with ALP (although it si only a wrapper of RDP) and HP with RGS have both shown that at the expense of bandwidth the experience can be significently improved. Not sure how other companies feel but he thought of a single station generating traffic levles exceeding 20MB for average quality video feeds scares the hell out of me when you are looking at hosting large numbers of desktops.

It would be hard to explain to the higher powers why your solution brought the corporate network to it's knees during the Olympics!




I guess the silence has to do with the announcment that Qumrnet was just acquired by Red Hat. Would appreciate to hear anything from Brian that he can say - even if it's only that he can't say anything.  The results of the testing would have been awesome, and still would be ... but killing the silence on this topic , right now, will help the community understand what's happening...
don't be naive....

Looks like Brianmadden needs to be updated; https://www.brianmadden.com/disclosures

What we won't do with vendors:

We will not accept money to do a product review.
We will not accept money to write about a vendor.
We will not give a vendor any special treatment just because they bought an ad or something.
We do not have any programs where vendors can pay us to be in their club.