Did Nutanix just create the ultimate server/storage big data combo hardware for VDI?

Hardware startup Nutanix unveiled their first product last night, a server/storage highly-scalable combination that just might be the perfect VDI host. And just like that, Nutanix is my current favorite startup vendor.

Hardware startup Nutanix unveiled their first product last night, a server/storage highly-scalable combination that just might be the perfect VDI host. And just like that, Nutanix is my current favorite startup vendor. (I can't remember who I last had a crush on, but some of the past vendors have been Ardence, Atlantis, and Unidesk.)

The DNA of Nutanix is based on the "big data" trend of combining computing and storage capabilities into single entities. Nutanix sells server nodes with local storage built-in, but their magic is in the software that combines all the storage of all the nodes into a single giant storage pool, with any data from any node available from any server. They have a master-less architecture with no concurrency locking, and they can support advanced VMware features like vMotion. Oh, and they repeatedly assured me that when loading up VDI VMs on these things, IOPS will not be a bottleneck (even for unique persistent 1-to-1 disk images).

To understand the what, why, and how, let's take a step back.

Storage trend: combining compute and storage into one system

In the world of "big data" (which is just a fancy yet generic term that describes when there's so much data that traditional methods for dealing with it don't work), there's a trend to combine storage and compute.

In the traditional computing model, your computation happens on one system (the server), and your data is stored on another system (the SAN). But as things get faster and bigger, this split architecture becomes a bottleneck. One way to handle this (and a growing trend) is to move some of the computing capability onto the storage system. I mean after all, if a SAN has Intel processors, why can't those guys use some of their excess capacity to help out the computing layer?

This concept began with simple things. Maybe instead of a SQL query running on a database server, the application layer could pass the query directly to the storage tier which could run it locally?

There have been hundreds of companies which built solutions like this. A team of engineers who helped Oracle accelerate some of their highly scalable databases thought, ""Hey, why not ask some of the storage cells in the EMC to do some basic work?" And in 2005, that idea turned into Aster Data Systems, a massively parallel processing database that was about pushing the app logic down into the database.

Meanwhile Google was realizing the same thing. The team that originally built the Google File System originally built it in two tiers, but eventually they realized "Why not smash these together?"

And you can find examples of this everywhere now. Amazon Dynamo, Hadoop, the systems that run Facebook, Google, Bing, MySpace, Akamai… they all do this. But those systems were all built for very specific purposes. Is it possible to bring that architecture to the masses? That's exactly what Nutanix hopes to do. The three cofounders all come from Aster Data, including their VP of Engineering. And before that one was the lead designer of the Google File System, while another architected massively scalable databases for Oracle.

Enter Nutanix

Nutanix's goal is to leverage the trend of consolidating compute and storage but for a general-purpose platform of regular VMs. Their goal is to keep the system completely transparent to VMs. Nutanix is just providing block level access that's really no different (from the VM's standpoint) than any existing storage system.

This is different than today's "consolidated" systems from Cisco, HP, Dell, and VCE. Those systems are essentially just bundles of traditional storage and computing. Nutanix is doing is true convergence.

The Nutanix hardware

Nutanix has 2U enclosures with four server nodes. Each node is a dual socket with 48-192GB of RAM, a Fusion-io controller with SSD, plus 5TB of SATA storage.

Nutanix rendering

All of the nodes are completely seamless. The fact there are four per 2U appliance is just a form factor. Each node runs VMware ESXi and acts as your VM host, and then a controller VM running on each node acts as the iSCSI interface to the storage and basically turns the whole thing into a distributed SAN. There's a 10gig Ethernet connection for the storage traffic which is separate from the regular network traffic.

The controller VM decides where in the system to place the data. There's always one copy local plus another copy somewhere else in the cluster. Nutanix calls this "Cluster RAID," and it's fully compatible with VMware HA and vMotion. There's a distributed cache using the Fusion-io with SSD, as well as a persistent SATA tier.

Then the distributed MapReduce system does all the maintenance for them. Everything is completely transparent, and the whole system is lock-free and everything can be concurrent. There's no single master and no shared cache. They have true scale-out with their storage metadata (which lives on every node), and the system continues to scale as you add more nodes.

Nutanix views themselves as a serious enterprise alternative to NetApp and EMC. They're not just about taking two nodes and turning them into a SAN. This is not for small or medium business. This is serious enterprise class storage. (And thankfully they haven't done any hardware engineering. They sell the hardware for the sake of testing and consistency, but the hardware itself is nothing special.)

Next steps

I want to reiterate that Nutanix absolutely feels that IOPS will not be the bottle for VDI environments running on their systems. I asked that about whether they support single instance block level storage (a.k.a. "inline dedupe), and their answer was "no." Although after they said no, they spent a considerable amount of time discussing key issues such as variable block length and strong versus weak hashes, so it was obvious they were familiar with the concept.

At this point I'm looking forward to getting some experience with one of these in the real world. If they can build modular-yet-scalable essentially unlimited IOPS environments for VDI, then this is a huge thing. My only real fear is that they'll be swallowed up by some huge company and more-or-less destroyed.

So that's my crush for now. What do you think?

Join the conversation


Send me notifications when other members comment.

Please create a username to comment.

Very interesting solution!

How is decided which traffic goes on the Fusion IO card? Is IO Turbine build in?

I would like to know if this solution makes sense for XenDesktop 5 with MCS on vSphere since MCS does not allow storage tiering?


At first this seemed quite exciting until I thought about it....

"each node runs VMware ESXi" - So no Hyper-V or XenServer then? So that ties you in to a VMware world...

Looks okay if you want to run standard VDI VM's but doesn't help if you need GPU expansion? (RemoteFX, etc)... - Not that RemoteFX runs on VMware anyway! - ok GPU expansion is a limited use case at the moment but its growing and is still a use case)

I do agree though SAN's make little sense for VDI...

Be interesting to see a comparison Vs something like the HP C7000 loaded with blades and fusion  cards (inc costs)... Probably takes the same rack space per host.

Cosmetically nice though ;) Be interesting to see if they get any traction...

what are the RRP costs?

3/5 and a A+ for effort :)


Just like to add....

I actually quite like this product....

My comment above is just my devil's advocate side of me...


I must be getting old as computing seems to be repeating itself. CPUs and storage in the same box - wow, that's new. Not. One needs to look back to pioneers when every byte and block on disk was incredibly precious. They knew how to screw performance out of things. Okay, I can see how the single data pool is a neat trick but it's "Perfomance 101" that if the nearer data is to the computing engine, the faster things will be.


Of course, this will all be about cost.  Not just for VDI, but the possibilities for remote offices that need a bit of redundant compute will be endless... if the price is right.


Indeed - I'd love to see some hard spec's and realistic prices.

I can see this being a great turnkey option for VDI deployments - unless they require GPU's or APEX 2800 PCOIP decoders... seems a bit shortsited if they havent planned for some sort of optional PCI-Ex expansion...

They could always offer an option to switch out the Fusion IO card I guess - but given the form factor of the blades I expect they will be regular half height iodrives. and the PCOIP accelerator cards (and any high memory density GPU for RemoteFX) are likely to be full height.

Wouldn't it be nice if someone came out with a ribbon cable to extend a PCI-Ex slot to a 3.5" HD sized PCOIP/RemoteFX accelerator card / headless high memory density GPU? Then you could bang them in just about any rackmount server...


Disclaimer - I work for Dell

Dell has the C6100 in the PowerEdge-C line that is essentially the same thing and has been out for almost 2 years.  In response to @Wonderdog, using a HIC card and iPASS cable, you can extend the PCI bus to a C410x and have 8 PCI x16 slots for GPU, Fusion-io, etc.

At the end of the day, Nutanix, Dell, Cisco, HP, whatever - you still need to manage the ESX host at the hardware layer.  I'm curious how Nutanix does Systems Management.  Its all great that I can have 1000 VDI users in a 2U footprint with consolidated storage, how do I know when a node fails?  Regardless is vMotion moves workloads around, eventually I need to fix the lost "+1"


@Erik We use Fusion-io for caching and also as a fast persistent tier of storage. Our controller VM intercepts all IO and writes it to underlying Fusion-io, SATA SSDs and hard disks (tiering off cold data to hard disks). We don't use IOTurbine.


@Daniel Our architecture is hypervisor agnostic but we are starting with VMware. If you compare us with a bunch of blades filled with Fusion-io, each of those blades will be an island of storage. You'll need to give up on high availability, backup, replication, DR, snapshots, etc. - features that you get from your SAN/NAS vendor.


@Tony To clarify, we aren't providing you just 4 servers in a box. If you use Dell6100, each node is a regular server. We can cluster multiple boxes like Dell6100, with Fusion-io, SATA SSDs and hard disks, and turn them into a server+SAN combo. For example, if you took 5 Nutanix Blocks, it is equivalent to 20 server nodes and a SAN with 6.5 TB of FusionIO, 6 TB of SATA SSDs and 100TB of hard disk capacity, with all the storage looking like a shared storage. I hope that clarifies. Thanks.


@Ajeet Singh

Good to know your not just sticking at VMware...

However with VDI.... I don't think HA, backup, DR, etc is important if you get the other layers right (software and hardware). If thats the case then "islands of storage" is still a good option - if you have good management layers in place.

Are you looking at GPU expansion?

Also now we all know you're reading this..... What sort of prices are we looking at? RRP?



HA, DR, etc. are needed if you are doing VDI for knowledge workers. Why push these basic storage capabilities up the stack and force your apps/management layers to bear the burden? These should come from the data management layer and should be abstracted from layers above.

Our list prices start at 75K. We eliminate SAN from the equation and are much more cost effective than other server+SAN alternatives for enterprise virtualization, VDI included.


I understand the difference to Dell's solution but what about HP's P4000 - Virtual San Appliance?

The P4000 Virtual SAN Appliance allows you to transform your local disks into a SAN and isn't this exactly what Nutanix is doing?



Cheers for the RRP.... what does that get me? how many nodes, storage, cores, mem?

"HA, DR, etc. are needed if you are doing VDI for knowledge workers" - I really think that's the old way of thinking about it... 1-1 relation ships yeah.... if you get the layers right (and at a sustainable cost), 1-many can and does work and means you don't have to worry about HR, etc.



There's HP's P4000 and there are also quite a few other Virtual San Appliances out there, including the one launched by VMware recently. These are  targeted for smaller businesses, converting a few servers into a SAN. Our system is aiming to meet the needs of mid to large enterprises and differs from these VSAs in that:

- It is highly scalable. You can start with a 4 node cluster running 200 VMs and 20 TB of storage and grow to thousands of VMs and hundreds of TB of storage, all within a single cluster

- It has ground up optimizations for flash, inlcuding Fusion-io. We integrate Fusion-io and SATA SSDs in every node of the cluster

- It has rich data management features (some unannounced at this point) that you need for mid to large enterprises




For 75K, you get our starter kit with 3 nodes with dual socket Intels and 48 GB RAM on each to run your VMs and a total  1 TB of Fusion-io, 1 TB of SATA SSD and 15 TB of hard disks to store your data. And you get the ability to grow as your needs grow, even to thousands of VMs and hundreds of TB of storage. The performance scales linearly and you don't have to scale up at any point (i.e. with traditional systems, you have to throw away "lower end" product if you hit a wall).

Looking at this from a virtualizatio guy's lense, you get to run up to 300 VMs and you get very high performance 17TB of storage in a package which is very easy to deploy (literally under 30 min), easy to manage, easy to grow (grow one node at a time if you want) and gives you everything you want from an enterprise-class system (HA, backup, DR, etc.).


@ajeet which markets will be served, and what are the support options?



Mid to large enterprises. We offer standard enterprise support options, including field support through our partners.


@ajeet - sorry - meant which regions/countries.



Initially US.


It certainly looks like a magic bullet for flexible enterprise Virtualisation deployments - There is enough scope in the given kit design to cope with the majority of situations I can think of. Just because most people don't need this kind of kit doesnt mean there isnt a market for it!

There are doubtless much cheaper (note - I didn't say better value/more cost effective as such factors are dependant on the use case) ways to do away with dedicated SAN hardware - the aforementioned P4000 or vmware appliance approachs for example.

I guess if the powers that be called me up and asked me to build a new datacentre as quickly as possible, with the constraints being that

A) it must perform like a beast

B) it must be extremly flexible in terms of workload C) it must be built from immeadiately available vendor inventory components

D) it must be able to scale rapidly and be supported through a single pane of glass

E) I'd have to stake my professional reputation on delivering all of the above

- then this is the kind of kit I'd use.

It might cost more, but in a situation where you have some or all of the above constraints and price isnt one of them; and don't have the luxury of months to investigate, design, order, setup and tweak a custom built multi tiered SAN solution then I'd be pretty confident about something like this being plugged in and just... working - come what may. There is definately value in that.

One other point - while BM,com is a VDI focussed site, and this kind of kit could certainly eat up and spit out almost any VDI scenario or software product you might choose to throw on it, Its hasn't been engineered to host one type of VDI deployment at the expense of all other uses, so in my eyes its fine for Nutanix to tout benefits such as out of the box support for features like HA, DRS etc as they are very much relevent to many non VDI use cases (and knowledge worker persistant VDI deployments for that matter).

I'd still love to see an option for the forthcoming offload card onslaught for PCOIP/RemoteFX etc - then it would truly be ready to host even the highest strain VDI deployments.



Thanks for the feedback. The scenario you describe (A-E) is indeed a solid mix of needs we can solve for.


@Ajeet - I understand that you start with a cluster that containing 3 nodes (starter pack), or the stardard pack that includes 4 nodes.  Can you help me understand a bit more about how you scale after that?  After I get my starter cluster, what is the standard for node scaling?  Can I continue to add in one node increments or do I have to add an entire cluster of 4 nodes each time?



You have the option of scaling one node at a time.



I can answer that one.

You can scale one node at a time. The four nodes per enclosure is just arbitrary. It's all about the nodes. So at nodes one at a time, two at a time, six at a time.. whatever.

Since the architecture of the storage is that the bits are distributed throughout and physically in more than one place, you can pop-in a node and live migrate VMs over to it right away. Or you can just start using it. BTW, when you do this, the system will automatically move some data around in the background just keep everything nicely distributed, but that's something that happens in the background and as system load allows. But it's not like it's in a degraded state before that or anything.

So all that said, really you can just pop these nodes in even one at a time.. you just have to buy a new enclosure for every fourth node.


Thanks Brian!  

Next question, It has been mentioned that stripping is use for some form of parity.  I'm assume parity levels are not configurable (that is probably a good thing based on the intellectual property Nutanix is bringing to market.  With respect to usable storage capacity per node.  What am I looking at?


It's perplexing to me why you're promoting Fusion-io rather than commodity SSDs like Intel, Samsung, Micron, OCZ.  Like Eric B., I thought IO Turbine was the software used - definitely see major overlap with your stuff.  I'd rather just buy a Fusion-io card with IO Turbine software on top of VMware for VDI.  Fusion is getting free publicity and will eat your lunch!



Thanks for all the information.

I can see this solution selling very well.

What are the plans for the other regions? Especially APAC & ANZ?




I run the WW sales @Nutanix. We are currently planning to roll out APAC/ANZ in a phased manner. Start in Japan late this year and rest in Jan-Mar 2012.



Thanks for answering @fbillford.


You can use from half to all of usable capacity, depending on the striping policy you use. It's configurable.



I meant half to full raw capacity.



IOTurbine and Nutanix are different solutions. IOTurbine allows you to cache VM data (for Windows VMs) on Fusion-io, which is a part of what Nutanix provides. But Nutanix allows you to eliminate the need for a SAN at the back-end. With IOTurbine, you still need a SAN. The choice depends on what you want to do.


@Ajeet or Nutanix team:

I am not sure how you distinguish your self from other VSAs such as LeftHand etc. I had some questions for clarification ->

a) You do provide a block interface (via iSCSI) from your controller VM to ESX clusters (based on Brian's post). Is this correct or do you additionally provide NFS?

b) Since you are first exposing local disks & Fusion IO SSDs to each controller VM/node, I am assuming this may be some form of pass through RDM to the controller VM in VMware environment? Is this correct? The intelligence of how to leverage the SSD cache, distribute MD between controller VMs is all software inside the VM.

c) Do you have a fast path for IOs from VM to the local disk rather than every IO bouncing from the controller VM - whether remote (less likely based on this article) or local?

d) Do you expect consumers in the VMware virtualized environment to then carve out a separate  iSCSI LUN for each VM from this virtualized Nutanix storage? Or do you envision, customers to layer an abstraction on top of your block interface (iSCSI) by formatting these devices with VMware's clustered filesystem such as VMFS. I am assuming you would prefer the former, because it wouldn't potentially give you visibility into VM storage (vmdks) with the latter and also from performance aspect its another layer. So then what is your story around VM snapshots? Are these workflows driven by your custom management interface or via VMware's management product?

e) Do you have an offering coming up for Hyper-V?


@Ajeet Bravo!  Genius!!! I can see why Brian has a crush on you...  I'll be watching your product closely...  Too late for my VDI project but may be my next purchase next year or two..  Can you support hybrid architecture with your software? For instance If I have 3 HP DL380 ESXi hosts can your software make use these servers?


Too expensive, price/seat too high at initial buy in before you scale.  Could put a solution together much cheaper and perform just fine.  Running Fusion-io cards will do that to you.



Yes - our intelligence in the cluster of controller VMs which manages the IO. Our vDisk architecture is protocol-agnostic. Customers can do it either way - use our storage directly or put VMFS on top, depending on what they want. Our architecture is also  hypervisor agnostic - we'll be adding support for more hypervisors later.



We provide a complete hardware and software solution - we don't provide our software to run on your existing servers. That said, we can co-exist within your VMware environment with other hardware to the extent that you can manage us like a cluster of standard hosts from your VMware management interface.



Indeed, FusionIO does push up the price a lot, and for VDI deployments how much better is it that much cheaper SAS/SATA SSD?

As I alluded to earlier, It'd be great to to see an option to replace the expensive FusionIO boards with high memory GPU's or PCOIP accelerators instead, for a more affordable and suited "VDI" node version.

I still really like the form factor though.



If you are looking to run a few VMs on one host and don't care about features like HA, replication, backup, snapshots, scalability, etc,, we won't be the right fit. If you are looking for anything more serious than that, we can provide 40-60% lower CAPEX than most server+SAN combos.



Fusion-io may seem expensive on a $/GB basis but it's much cheaper on a $/IO basis. The main issue is that if you want both (performance and capacity), you need a solution that can use Fusion-io to deliver IOPs and SATA to deliver capacity, without making them an island, and without breaking the features like HA, snapshots, replication, etc. We use Fusion-io as a cache for hot data and also as a fast persistent store. And we transparently move cold data to SATA. This gives you high performance and high capacity at an attractive price point.


Vmware should buy you guys.  This would be perfect VDI on a box.  Server Virt on a box too..  You guys simplified the architecture, as long as your software is not buggy then this is a solid solution...  Cost is justified when product can deliver...  =), I really like it...


@ajeet - what are the stats regarding IOPS each module can do?

I am thinking as I scale this type of solution I may not need each module to include the expensive Fusion IO component.

If I know I'm using only 10% of my IOP capacity, but I want to add a bunch large file server type VMs, I would want the option to go all SATA.

Does Nutanix address this type of requirement now?

Will it?




I work in a K-12 public school where file service is just as important as block level storage for VMs. My current solution is a Netapp box that does both NAS and SAN.

Can I run Windows File Server(s) as VMs allowing staff and students to save files/folders onto the Nutanix? Do you have customers doing this?

The Nutanix box seems purpose built for server and desktop virtualization and certainly if I purchase one I would use it for that purpose. It would be nice if I could also store files/folders on it giving me one box that does it all for compute/storage.