The ultimate VDI storage option: "local" virtual storage. Does this product exist?

Just my quick thoughts/ramblings.  I'm a fan of "shared image / non-persistent" VDI, which, similar to what you mentioned, seems to remove complexity for storage.  We're working through our Virtual Bridges VERDE deployment of 550 concurrent session and are halfway finished.  So far we haven't run into storage or performance issues.  I'm not anywhere near as knowledgable about VDI as you or Gabe, but Virtual Bridges sure seems to know how to do it.  Also taking into consideration VERDE's disconnected use, you're using local storage and processing on the client, which is then sync'd with the host infrastructure, eliminating the need for a complex storage setup.  While disconnected use doesn't make sense for every VDI client, it does allow for breaking away from any performance limits imposed by the host infrastructure.

It sounds like you're looking for a Virtual Drive in RAM (I think PvS had this years ago called RAMDisk) where an image would be deployed into RAM. If you take this a step further a whole server could be provisioned into RAM including a vDisk and you would have the possible solution.

Anyone know if this feature is still available?

I also think it would be a great feature for Client-side hypervisors!

I've never used it myself but there is such a product:

HP StorageWorks P4000 Virtual SAN Appliance

h18000.www1.hp.com/.../index.html

It's a VMware virtual appliance and can take local storage presented by VMware and apply the LeftHand feature set to it.

Something like this might help a bit:

www.adaptec.com/.../hybrid-raid

Lacking a lot of the intelligence you want (as in choosing what goes where) and a bit of a pain if your server vendor doesn't bundle it in with support, etc. This kind of stuff with ZFS is pretty slick when not under significant load (=at home), I've never tried it with hardware controllers or in a meaningful environment.

Brian,

This may not do everything you mention in this post, but this is the closet solution I can think about (I have not read all the details though).

Adaptec is starting to mix and match SSD / magnetic drives on their controllers ->

http://tinyurl.com/2flvjy8

On the downside it seems to be limited to Raid1 / Raid10.

Atlantis ILIO is available as a virtual appliance to do exactly what you described either with a combination with SAN/NAS/File server (for user data) and local disk (for master images) or entirely with local disk (for both user data and master images).  The hypervisor sees Atlantis ILIO, which is a virtual appliance running on each VDI hypervisor, as an NFS data store. When IO is generated by VDI desktops, Atlantis ILIO deduplicates it before writing to storage, caches frequently used image data and then writes unique data through to the appropriate backend storage (i.e. SAN/NAS/Local Disk) based on the type of data.

The result is a very high performance VDI architecture that reduces the amount of storage required for VDI by up to 90% (by offloading 90% of IOPS) and enables customers to choose between SAN/NAS/Local Disk as storage for their user data and master images. With this architecture, it is also possible to use a persistent VDI disk without the high storage costs because Atlantis ILIO is deduplicating up to 90% of the image data before writing to storage. This means that customers can choose between a persistent and non-persistent VDI model based on functionality and aren’t forced to go with non-persistent to save on storage.  

Our largest customer uses an Atlantis ILIO virtual appliance on each VDI server with two local disks per server for image data, a non-persistent VDI desktop and SAN for user data. You can check out the case study here: www.atlantiscomputing.com/.../Atlantis_Financial_Service_Case_Study-9-22-10.pdf

At VMworld 2010, Atlantis Computing sponsored an event where Daniel Beveridge, Sr. View Architect in the Office of the CTO at VMware recommended this same architecture using a Virtual Storage Appliance (i.e. Atlantis ILIO as a virtual appliance), tiered storage with VMware View 4.5, Local SSD for image data and “enterprise storage” for user data. At the same event, John Premus of GlassHouse Technologies demonstrated live the GlassHouse vLab which uses Atlantis ILIO as a virtual appliance to reduce storage (80% IOPS offload, 90% storage capacity reduction) and provision all of VDI Lab components. You can check out the case study here: www.atlantiscomputing.com/vLabCaseStudy

In Mid-October, there is a large and detailed VDI reference architecture coming out that will highlight Atlantis ILIO running both in a virtual appliance and “Top-of-Rack” (Atlantis ILIO installed on a dedicated hardware appliance) configurations. The VDI reference architecture will include details about how the Atlantis ILIO virtual appliance model works, IOPS offload metrics and performance benchmarking. I will update everyone with a link to the document when it is released.  

I think the reality here is no desktop virtualization with the economics storage solution that works. I don't like the Atlantis storage appliance approach and don't buy it is anywhere near production ready unless you are willing to invest a ton of time to get it right. Even them it's a layer in my desktop infrastructure that I am very nervous to give to a startup that is not proven just yet given the start stop history with them. Anyway that's my opinion, I wish them on ill will. I also think distributed file system approaches like www.gluster.com are interesting. I like the idea of a file system that is abstracted to create a global name space to get to my commodity hardware. I not an expert in this area, but am clawing my way through options, and I still conclude this is an area ripe for innovation to take place. That said, most of the real implementations are simply using 1-1 desktops and those don't care about massive density. It's all manually managed using regular techniques and some custom work. For broader adoption, the guys that want cheap cheap cheap, they just go TS, and use local disk as they always have. I don't know of anybody who has done shared VDI at scale, or anybody who has not written custom management scripts etc for local disk deployments.

Has anyone seen this reference architecture from VMware yet?

VMware Reference Architecture for Stateless Desktops - http://bit.ly/94fTWb

I'm pretty interested in this because it would drive down the cost for disposable virtual desktop environments. There would be no need for expensive 100K+ Sans for these type of scenarios. I ordered some SSDs to throw in some servers and test out soon. Hopefully SSDs drop in price soon though....

I most of the time refrain on commenting on the VDI topic, the “paradox”, 1 to 1, 1 to many, streaming, storage, hypervisors, IOPS etc. you get the drill

I certainly see the merits of doing the VDI and rapidly it’s becoming hard to avoid. I even agree to the rationality at times. Problem of course is that I don’t see VDI as a mature enough solution yet. On a personal note see VDI as both confining and infringing.  

In my rhetoric I usually try to contrast the VDI (VDI+, VDI Next) with traditional computers and try to advocate to a balanced mix of remoting and local computing. A main thing to ask is “how do you compute”, that question is rhetoric as well because all of us that frequent this site or others alike are fully aware of that we all are running local OS:s with full ownership.

Sounds like another layer to far to me and another level of management and cost.

I share AppDetectives "concern" with startups. I will only invest it products that have a sustainable future (and history). Maybe in years to come though.

We've just started phase 2 of our VDI and RDS project expanding to a further 2500 - 3000 Windows 7 desktops and 1500 - 2000 RDS connections all running on Hyper-V and Good old reliable LOCAL storage.

We're successfully achieving the 1 to many model (not 1 - 3000 though :) but large numbers non the less).

Once SSD disks becomes the norm then a lot of VDI specialist storage technology vendors "could" struggle to stay in Business!

@Brian ???

I may be missing the point, but wouldn't it just make management of storage more complex you go down this decentralized path? If the high-performance storage interfaces in a SAN is too costly, then why not utilize a NAS appliance for this purpose and have a unified approach to manage the connection? After all, you have to buy a SAN anyways, correct?

Diskless servers can be acheived by utilizing provisioning services so you can boot off of SAN/NAS, therefore removing decentralized storage where possible.

IMO, management of distributed computing/storage makes sense out of the datacenter but not within it.

@ Daniel  - You make a valid point!  I agree any additional layer equals more management, more costs, and more risk.  In the end it boils down to what the benefit is and is it worth the risk. We have customers in production and a lot more doing proof-of-concepts that have offloaded 70-90% of IOPS and as a result dropped the cost of storage for VDI significantly while eliminating VDI performance problems cause by IO bottlenecks. In my opinion, that type of cost savings and performance improvement is worth the risk of testing the Atlantis in your environment. We don’t expect customers to take our word for it, which is why we use tools such as Liquidware Labs, Lakeside, Veeam to prove the results by measuring the IOPS and performance in the actual VDI environment with and without Atlantis ILIO in the picture. Using this methodology, customers will quickly know whether it make sense to use a VDI architecture with Atlantis.

In answer to Brian’s original question - The ultimate VDI storage option: "local" virtual storage. Does this product exist?

Yes it does and it is called Atlantis ILIO.

As an independent consultant (i.e. Atlantis do not pay me) I have been working with, tested and have had exposure of some major financial customers who run this in PRODUCTION today. I am sure some of the people posting on this forum would like to have them on their CV!

VDI is now taking off big time, but ask yourself this question – If tried and trusted storage vendors were able to service VDI at scale in a cost effect manor today, why are major financial companies using a start-up such as Atlantis ILIO (in production or POC) now?

In answer to Brian’s original question - The ultimate VDI storage option: "local" virtual storage. Does this product exist?

Yes it does and it is called Atlantis ILIO. As an independent consultant (i.e. Atlantis do not pay me) I have been working with, tested and have exposure of some major financial customers who run this in PRODUCTION today. I am sure some of the people posting on this forum would like to have them on their CV!

VDI is now taking off big time, but ask yourself this question – If tried and trusted storage vendors were able to service VDI at scale in a cost effect manor today, why are major financial companies using a start-up such as Atlantis ILIO (in production or POC) now?

As GavinT said above

HP StorageWorks P4000 Virtual SAN Appliance

(previously known as LefHand Networks VSA)

I use this regularly...works a treat and simple to use

Update from the SSD world...

www.engadget.com/.../ocz-ibis-touts-2gbps-high-speed-data-link-vanquishes-ssd-compet

Very fast. But how will it be used? Locally or in a SAN?

Anyone from a storage vendor care to take a look and comment?

RAM conquers all. Faster than any RAID configuration will ever be, cheap enough compared to SSD. Load the master image once from SAN -> RAM and go. Catastrophic hardware failure is meaningless, because the static image is secure on the SAN.

(of course, this is predicated on the nirvana of a single golden image... multiple images = screwed up economies from >8GB RAM sticks).

Not an answer to your question, but related to the topic.

The upcoming release of XenServer for which the beta started today (Project Cowley) has some enhancements targeted at (a future release of) XenDesktop which will utilize local storage.

More info on the beta: www.citrix.com/.../beta and in the webinar next week I will briefly explain what we've done.

Joel

Actually what you described is something that StorMagic has been doing for sometime now.

They Virtualize DAS disk on ESXi servers so that they can share and participate as a cluster.

Basically imagine that you have 3 ESX nodes. Each nodes has internal disk as you described. you create a small DataStore that hosts the VSA (Virtual SAN Appliance) from Stormagic. Then the rest of the DAS in the server is presented as RDM's to the StorMagic VSA which then shares the capacity to the local ESX server as well as the other ESX servers in the cluster over iSCSI.

It really is an ingeneous way to unlock the DAS in your servers.

Full Disclosure...I do not work for StorMagic, Sell StorMagic, or Use StorMagic. But I do know their VP of Sales and  Marketing Mike Stoltz.

Good people, a good product and a good price.

It's probably going to be XenServer 5.6 Feature Pack 1

Joel

Well you could use FusionIO's PCI-E product but that's not offering you a lot of space...

Really F the SAN?  When the costs of SAN's are getting so cheap these days...

Hell an IBM DS3400 or the newer DS3512/24 can get you on your way pretty cheaply.

Or an older HP MSA2100 or 2200 series is going to get you there too.

I think the SAN is for DR and Cluster needs... If your VDI is mission critical why do you want to trust this to something less?

Even if I was someone that might want a server setup in my home so I can throw thin clients all over the place...  I would use at least 2 node cluster with disk from something like the DS3400 or HP MSA.   Because stuff breaks...

Firstly can all the sales guys from Atlantis just F off and stop posting marketing BS. The ONE financial customer you have JP Morgan, has been baby sitting your product for years and handholding it to make it work to help with perceived user responsiveness. It does not mean the product is ready for prime time, and I doubt you will get traction much further than JPM, who have a cluster F implementation that includes every vendor under the sun designed by idiots who have no clue. They even used SunRay first and I wonder how that is going?

It makes no sense to put yet another layer of storage appliances to add yet more unreliability and risk into production infrastructure. A caching layer at the hypervisor makes a lot more sense. A virtual appliance can help (that's where Atlantis may have some legs to be fair). But longer term I am still a huge fan of local storage. I saw the Cowley announcement, and think it is very significant potentially as the hypervisor is now doing something smart and finally there may be a real reason to get off ESX for the desktop eventually. I see this trend as having legs as it enable use of cheap storage in server already and makes horizontal scale out much easier. Will be interesting to watch this space emerge, as it potentially mean optimized hypervisor stacks for different workloads. If that happens, having a strategy to support multiple hypervisors is going to even more critical than many people think.

I think that everyone is over complicating the issue and there is no need to add layers to the architecture.

The reason we have this conversation is because SAN hard drives (HDD) are too slow and are not getting any faster.  Instead of adding caching solutions or other technology to continue to enable this antiquated hardware, simply using SSD as primary storage will do it.  And, I think most enterprises will prefer that it be shared storage for easier HA/DR and management purposes.

The problem with SSD has been cost, endurance, and write speed (where VDI is 80% write).  The WhipTail Racerunner OS on our solid-state arrays optimizes the writes and leverages MLC flash so we eliminate these three obstacles and keep the architecture clean.

Your next challenge from there is trying not to require too much capacity.  That problem is related first to whether the business will need persistent or non persistent desktops, and then to what type of solution you are seeking to reduce those capacity requirements (PVS, Composer, Unidesk, Atlantis, etc).

But for $30/user, WhipTail can provide SSD performance as primary storage with the cleanest architecture.  Simple as that.

This reference architecture from CIsco provides clarity on how to deploy Atlantis with Citrix on  local disk.

www.cisco.com/.../cisco_ucs_atlantis_citrix.pdf