Microsoft RemoteFX is now available via public beta. Here's our in-depth guide to how it works. - Brian Madden - BrianMadden.com
Brian Madden Logo
Your independent source for desktop virtualization, consumerization, and enterprise mobility management.
Brian Madden's Blog

Past Articles

Microsoft RemoteFX is now available via public beta. Here's our in-depth guide to how it works.

Written on Jul 13 2010 27,173 views, 18 comments


by Brian Madden

Yesterday Microsoft made a beta version of the Windows 7 SP1 / Server 2008 R2 SP1 available for public download. One of the new features in SP1 is RemoteFX, an enhancement to RDP that will give LAN users near-perfect remote desktop experiences (complete with multiple monitors, 3D effects, HD video, etc.)

I've written about RemoteFX quite a bit in the past, including:

A lot of the talk about RemoteFX over the past few months has been based on leaks and speculation. But now that the beta is public, and now that we've seen presentations from Microsoft RemoteFX engineers at TechEd (video link) and BriForum, we can confidently talk about what this "v1" RemoteFX capability will and will not be.

RemoteFX Basics

In Service Pack 1 for Windows 7 and Windows Server 2008 R2, Microsoft will "rev" the version of RDP from 7.0 to 7.1. The biggest new feature of RDP 7.1 is RemoteFX. RemoteFX is a LAN-based, host-side rendering graphics capability. The "host side rendering" distinction is important and something we'll dig into more later. But it's important to know that host-side rendering means that all of the graphical compositing happens on the remote host (the remote VDI VM or the remote Terminal Server), and all content is sent down to the client as compressed bitmaps. (This is similar to Teradici's PCoIP, and the opposite of client-based redirection schemes like multimedia redirection or Flash redirection.)

Since Microsoft uses RDP for two purposes—VDI to single user VMs, and Terminal Server / Remote Desktop Session Host—RemoteFX also supports remote hosts that are either VDI or Terminal Server (as long as they're running Win7 SP1 or Win2008 R2 SP1 of course).

The purpose of RemoteFX is to be able to deliver the "full" Windows experience to a client device. This means whatever a user can experience locally should also be experiences remotely via RDP with RemoteFX. So we're talking about multiple monitors, Aero glass, 3D games, multimedia, Flash, Silverlight, porn, video editing... everything! Anyone who's ever tried to use RDP as a "real" desktop replacement knows that there are still some limitations, and even though it's come a long way, if you maximize a Hulu video via regular RDP you'll quickly remember that you're not running Windows locally.

How does RemoteFX work?

The exact way RemoteFX works varies depending on whether you're using it with VDI or Terminal Server. Let's take a look at the VDI scenario first.

RemoteFX for VDI on Hyper-V

As you know, if you want to do HD video and 3D applications and Areo glass and everything, you have to have a GPU. Period. Prior to SP1 for Windows Server 2008 R2, there has never been a way for a VM host to "share" a GPU among multiple guest VMs. So that's actually the first big change in SP1. If you're using Hyper-V for Server 2008 R2 SP1, Microsoft has added the capability for the physical GPU(s) to be exposed to the guest VM. (In other words, it's virtualized and shared just like the CPU, memory, disk, etc.) Then if your guest VM is Win7 SP1, Microsoft created a new WDDM-based "virtual" GPU (vGPU) driver that lets the VM access the GPU. The actual brand of the GPU doesn't matter (since the hypervisor hides that from the VM), but at this point that VM sees a fully DirectX 9-compliant GPU, and that VM can run whatever GPU-requiring apps it needs.

In the land of RemoteFX, there's a component called the "RCC," for "Render, Capture, Compress." The RCC runs in the parent partition, although there's a separate instance of it for every VM using RemoteFX. The "render" stage is what I just described—it's the GPU which was virtualized and shared by the hypervisor that's now available as a normal GPU in the VM that's used to render whatever the VM needs just like any physical PC with a physical GPU.

Once the display content is rendered, it's passed over to the "capture" component. The capture component analyzes what was rendered in the previous frame and what's been rendered in the current frame. It divides the screen content into a grid and figures out which elements have changed and therefore what must be sent down to the client. The capture component continuously communicates with the RDP stack (remember it's running in the VM too) to understand what the network is doing and which buffers and frames have been acknowledged by the client. The capture component understands that if the network is congested it doesn't make sense to keep shoving screen data down, so instead it will start sampling and not sending every frame to ensure the user has the best realtime experience. Of course the flip side of that is also true—if you have a fast network then the capture engine can send more and more frames for an even better user experience.

The final component of the RCC is the "compress" engine which is responsible for compressing the frame data before it's sent to the RDP engine. The exact way the compress engine works depends on whether you have a regular server (with a GPU of course since you're running 3D apps and stuff) or whether you have a server with a custom-built RemoteFX add-in card.

If you have a regular server with a regular GPU, the compressor runs in "software mode." First it leverages some shader programming which accesses the GPU via DirectX 10 calls, and then it passes the encoding over to the CPU to finish it up. (In other words, the compress cycle uses both the GPU and CPU.) From there the compressed RemoteFX content is passed on to the RDP stack where it will go through the "standard" processing (bulk compression, encryption, prioritization, etc.)

If you have a RemoteFX add-in card (which do not yet exist as far as I know), then the compress processing is offloaded to the ASIC on the card and does not use the CPU or GPU. Microsoft feels that using the add-in card will be the way to go moving forward since it will mean that users will get more predictable frame rates and higher density since one user launching an intense 3D app won't steal GPU and CPU compress cycles from the other users. It's also probably worth pointing out that the hardware add-in RemoteFX cards will only be used in the Compress phase of the RCC; in other words you'll still need a GPU in your server to be shared as the vGPU which will enable each VM to actually run DirectX 9 stuff to begin with. (Although presumably you'd be able to getaway with fewer / smaller GPUs since they'll only be used for rendering and not compression.)

Ok, so now that we're through that, let's look at a couple more relevant points. First, as we've discussed before, RemoteFX is only going to work if your Win7 SP1 VMs are running on Hyper-V, since the whole vGPU and RCC engines hook in there. As for whether it will ever work on other hypervisors—probably not. Microsoft is very clear that one of the goals of RemoteFX is to create differentiating value for Hyper-V and it's a way to get more Hyper-V VMs in the world.

Second, the capture and compress engines act the same way regardless of the type of content. This is different than PCoIP which uses multiple slicing and compression algorithms (even within a single frame) depending on whether a particular screen region is images, computer graphics, or text. Of course Teradici and VMware claim that's a huge advantage for them. Certainly it is in theory but we'll have to see what the impact is in the real world. (My personal guess is that this is part of what helps PCoIP work with less than LAN bandwidth, while the "v1" version of RemoteFX is targeted to the LAN.)

Third, RemoteFX can be integrated into third-party remoting protocols (as long as the guest VMs are Win7 SP1 running on Hyper-V). Citrix has announced that HDX/ICA will support RemoteFX, and Ericom and Quest have announced that they'll support via their clients and brokers too.

Fourth, RemoteFX doesn't care what brand of GPU you use. They're not accessing it in a proprietary way (like Nvidia CUDA) or anything. Instead it's just normal DirectX 9 for the rendering and DirectX 10 (under the hood) for the compression.

Fifth, you can install up to four GPUs in a single host, and Hyper-V / RemoteFX will load-balance the workloads across them. (There was talk that an early private beta only supported one, but the current beta supports up to four.) The only requirement is that all the GPUs in a single system are the same brand. [UPDATE: Multiple GPUs in the same server must be the same model, not just the same brand.] Actually you'll also be able to live-migrate Win7 SP1 VMs with RemoteFX across physical hosts as long as the GPUs in both hosts are the same brand, and System Center will be "aware" of this requirement and will only let you live-migrate VMs using RemoteFX where it's possible.

RemoteFX for Remote Desktop Session Host (Terminal Server)

You'll also be able to use RemoteFX with Terminal Server sessions. The main difference between TS and VDI is that the Terminal Server implementation won't require Hyper-V. (In other words, you can use RemoteFX with servers running on bare metal as long as they're running 2008 R2 SP1.) Logically that makes sense, since a single instance of Terminal Server won't need to have a virtual GPU since all sessions can access the native hardware.

What's weird though is that the Terminal Server will NOT require a GPU. In that case the RCC engine will use the CPU for its compression (which again will probably impact user density, depending on the apps).

RemoteFX clients

From the protocol standpoint, RDP 7.1 traffic with RemoteFX-enabled graphics is always the same on the network—regardless of whether your host is VDI or TS, software-compressed or add-in card compressed. This means that your clients are fully interchangeable and interoperable with whatever hosts you use.

Microsoft will release a new version of the Remote Desktop Connection software (7.1) that will be able to display graphics based on RemoteFX. From the client's perspective, all RemoteFX traffic is 2D, since the RCC on the remote host essentially "flattens" 3D graphics down into 2D images that the client displays. This means that the client requirements are fairly light (which again makes sense since you've essentially "moved" your GPU into your datacenter.

In addition to the software RDP clients, Microsoft has also created a specification for a client-side ASIC that thin client makers can incorporate into WinCE or Linux thin clients. The idea is that these clients will be ultra-thin, based on non-x86 CPUs running at 200-400MHz with less than 256mb RAM consuming less than 5 watts. Microsoft has been showing off RemoteFX ulta thin clients from an Australian company called ThinLinx, but I know that HP and Wyse will soon have these types of offerings too. (The idea for these ultra thin clients is that they'll be really cheap, like less than $100.)

The final aspect of the clients that's important to know is that while RemoteFX is all about host-side rendering, keep in mind that it's still RDP, and RDP supports multimedia direction and other client-side capabilities. So if you connect to a VDI environment with RemoteFX enabled to watch a Windows Media video, if your client has the capability to do multimedia redirection then it will do it, even with RemoteFX. And of course this makes sense. I mean why not use it if it's there?

RemoteFX Performance and Scalability

This is one of the great unknowns of RemoteFX. Even Microsoft hasn't publicly shared scalability numbers yet, though they claim that they'll have sizing guides available once SP1 gets to RTM. So far we're seeing that the software encoding (where the RemoteFX add-in cards are not used) requires some pretty beefy GPU cards, like these double-wide power-hungry things that cost $2,000 each! And right now it looks like your best case is going to be about 12 users! (Again the exact number of users depends on how many displays and which resolutions people use. It also depends on what apps they use, since graphically-intense apps will require more compression which is also done by the GPU.)

Really as it stands today there aren't any GPUs that are perfect for this, since VDI admins probably don't want to put consumer gaming cards in their servers. (Actually how many servers can even fit double-wide PCI x16 cards anyway?) Microsoft feels that GPU and server vendors will release hardware that's more appropriate for RemoteFX in the future, and I would imagine this will come true since they're Microsoft and if they say that the vendors need to do this, the vendors will do it.)

If you haven't read them yet, definitely check out Teradici CTO Randy Groves' comments on this from his BrianMadden.com post a few months ago. There's a lot of posturing going on now, but the bottom line is that if you want awesome rich intense graphics for your VDI users, you're going to need the processing power somewhere. Whether that's in the datacenter or on the client, and whether it's via CPU, GPU, or custom chips—that's what we're all waiting to see.

If you want to start playing with RemoteFX, you can do so now. The easiest way is to pop a Direct X 10 compatible GPU in a server (or test workstation) install Hyper-V running on 2008 R2 SP1 beta, and then build a VM running Win7 SP1 beta. Then install the Remote Desktop Connection 7.1 beta software on another workstation and make a direct RDP connection to the WIn7 VM. You won't find any configuration options to enable RemoteFX from the client. Instead you just go into the connection properties and set the network slider all the way over to the "LAN" setting, and the client will automatically try to enable RemoteFX.

Next Steps?

I guess the next thing is to go out and start using it. I'll be moving my work desktop (a Win7 physical machine) to a VM as soon as I can find a server-GPU combination that will work. What about you? Will you be testing RemoteFX? Do you even care? Will you wait for Citrix to support it?

 
 




Our Books


Comments

Daniel Bolton wrote re: Microsoft RemoteFX is now available via public beta. Here's our in-depth guide to how it works.
on Tue, Jul 13 2010 3:25 AM Link To This Comment

"Terminal Server won't need to have a virtual GPU since all sessions can access the native hardware."

What happens if you virtualize RDS/TS as a VM? - There are many benefits to this and many organizations that do this. Does the RDS VM get treated like a Windows 7 VDI desktop?

I'm looking forward to getting some budget in August to order a GFX card (or 4) for testing though.

Hopefully Quest will release support (even if classed as "experimental") soon.

Tally-Ho!

Kimmo Jernstrom wrote re: Microsoft RemoteFX is now available via public beta. Here's our in-depth guide to how it works.
on Tue, Jul 13 2010 9:51 AM Link To This Comment

Thanks. this artice is a great summary of RemoteFX functionality. With the beta pubicy out it's fun times ahead. I hope there wil be some scaability reports soon out.

At the topic of detcated GPU boards for server class hardware, what do you guys think of NVidia:s announcment/blog post here > blogs.nvidia.com/.../nvidia-and-microsoft-enhancing-the-virtual-desktop-user-experience-with-microsoft-remotefx.html

Aaron Parker wrote re: Microsoft RemoteFX is now available via public beta. Here's our in-depth guide to how it works.
on Tue, Jul 13 2010 9:54 AM Link To This Comment

Interestingly, it appears RemoteFX may require it's own CALs? Hopefully that's not how it works. Information on licensing with RemoteFX is a little thin at the moment.

cfg wrote re: Microsoft RemoteFX is now available via public beta. Here's our in-depth guide to how it works.
on Tue, Jul 13 2010 10:26 AM Link To This Comment

Apparently the nvidia quadro plex might be initially the way to go:

http://tinyurl.com/ybtg9tt

Not sure if Nvidia has a refresh (base on the 2050 / 2070 gpu) in the pipe.

See the following blog from Dell ->

http://tinyurl.com/2fxoyuq

VMguy wrote re: Microsoft RemoteFX is now available via public beta. Here's our in-depth guide to how it works.
on Tue, Jul 13 2010 11:43 AM Link To This Comment

I know it is just me, but I still do not see too much value in having Aero Glass and 3D effects on corporate desktops.

In the LAN HDX and PCoIP can display Hi-Def video, complex flash content, and both can pretty much fulfill corporate requirements.

To not mention you probably will never be able to use blade servers, and even on regular rack servers, specific GPUs will still need to be released.

Would you buy additional hardware, and decrease scalability, just for your users to have fancy effects on their desktops ?

How much corporate applications really needs this ? There might be some for designers or something like that, but looks like a niche.

VMguy wrote re: Microsoft RemoteFX is now available via public beta. Here's our in-depth guide to how it works.
on Tue, Jul 13 2010 11:45 AM Link To This Comment

Brian, do you know if OpenGL will also be supported ? I believe many CAD applications are based on OpenGL instead of DirectX.

edgeseeker wrote re: Microsoft RemoteFX is now available via public beta. Here's our in-depth guide to how it works.
on Tue, Jul 13 2010 12:13 PM Link To This Comment

@VMguy - Good question. Hopefully, openGL "should" work because a virtualized GPU is present inside the VM. Calls to opengl32.dll are then passed down through a thunk layer to the GPU.

Having said that, an ICD (Installable Client Driver) must exist. This is normally developed by the hardware manufacturers in the case of physical machines. I don't know if Microsoft has developed this for the latest virtual machine spec. If not, opengl32.dll will sadly revert to its software rendering mode, which uses the CPU and eeks out crappy results.

appdetective wrote re: Microsoft RemoteFX is now available via public beta. Here's our in-depth guide to how it works.
on Tue, Jul 13 2010 1:28 PM Link To This Comment

I don't care because I have to get Windows 7 rolled out soon. That means for desktop virtualization use cases, I want to drop the cost in the datacenter as much as possible. That means cheaper servers. Add to that the servers to really take advantage of RemoteFX are not available today, so my costs are being sunk into commodity low costs servers today and it will stay that way for years. I don't believe in thin or zero clients unless I want zero client side flexibility, so I think it is far smarter to use the GPU client side as opposed to the server side. It's cheaper for me to upgrade the clients mid Win 7 lifecycle than it is upgrade the GPU's in the datacenter for RemoteFX. I am not that concerned about scalability, because to me gang banging too many VMs into a single server offers a poor desktop experience. For shared use cases XA is still the way to go. I also have ZERO intention of locking into Hyper-V which is another reason I hate PCoIP. At best this will be a marginal niche use case over time. Client side execution with central management I see as a far more compelling future.

Brian Madden wrote re: Microsoft RemoteFX is now available via public beta. Here's our in-depth guide to how it works.
on Tue, Jul 13 2010 2:33 PM Link To This Comment

Answers to questions raised in the comments:

For RemoteFX Licensing, I've confirmed with Microsoft that RemoteFX does NOT require it's own CAL. (i.e. it just uses the normal RDS or VDI CAL, the same requirement as today)

For support for apps that require OpenGL, they're supporting apps that use OpenGL v1.4 and below to work in the VM, but they don't expect that apps that use a higher version of OpenGL will work (unless of course they have a DirectX or CPU fallback mode).

Daniel Bolton wrote re: Microsoft RemoteFX is now available via public beta. Here's our in-depth guide to how it works.
on Tue, Jul 13 2010 6:36 PM Link To This Comment

@VMguy "do you know if OpenGL will also be supported ? I believe many CAD applications are based on OpenGL instead of DirectX"... Good question we had this discussion in the office today... Edgeseeker pretty much summed up the outcome (he must of been spying). The RemoteFX "team" live and breath GFX... I would imagine that they have this covered.

@VMguy (again) "How much corporate applications really needs this ? There might be some for designers or something like that, but looks like a niche." Do you presume that VDI is just for corporate users? What about education, healthcare, military, etc, etc. 3D apps are one of the things halting many adoptions of VDI/RDS in these areas. Yes HDX and the like can help with some of these but not all.

Aero maybe a tad gimmicky but it's what the users have on their own devices. Replicating the same "sort of" experience will put users in a comfort zone, increase productivity and stop them nagging the helpdesk all the time.

Connecting to a Win7/RDS desktop from a Vista or Win7 end-point can already provide Aero.

@appdetective... I sort of agree... I would like RemoveFX to work the way I like my women - doing it from both ends... Use local GPU if it's present, if not, server GPU.

appdetective wrote re: Microsoft RemoteFX is now available via public beta. Here's our in-depth guide to how it works.
on Tue, Jul 13 2010 8:38 PM Link To This Comment

@danielbolton LMAO. Fallback yes, but I'm not paying extra for it ;-) That's the problem I have with Server side GPU vs, just a cheap server. In time I agree would like to have the option but the economics have to make sense, which I am sure an OEM will figure. However IMO the Windows 7 train has already left the station for this to make a huge in the short to medium timeframe. I hope I am proven wrong as I think this will help many use cases.

Daniel Bolton wrote re: Microsoft RemoteFX is now available via public beta. Here's our in-depth guide to how it works.
on Wed, Jul 14 2010 7:07 PM Link To This Comment

This is not me - I repeat this is not me!

www.youtube.com/watch

The guy may give the word stereotype a bad name but it's good stuff.

And if your him... Sorry :)

Dan.

Rob wrote re: Microsoft RemoteFX is now available via public beta. Here's our in-depth guide to how it works.
on Sun, Jul 18 2010 1:57 PM Link To This Comment

@Microsoft: Great software.

This software show's that Microsoft can build up your complete workspace environment with full user experience without (extra costs) Citrix.

As i mentioned before, Citrix just uses the features that Microsoft delivers. There’s simple reason for that, Citrix lays on top of an OS so it's depending on the capability off it......

Daniel Bolton wrote re: Microsoft RemoteFX is now available via public beta. Here's our in-depth guide to how it works.
on Wed, Jul 21 2010 2:02 AM Link To This Comment

A link that works this time: ecn.channel9.msdn.com/.../VIR305.wmv

Good presentation!

AG4IT wrote re: Microsoft RemoteFX is now available via public beta. Here's our in-depth guide to how it works.
on Tue, Jul 27 2010 9:57 AM Link To This Comment

RemoteFX will certainly deliver a great experience for LAN users accessing rich media content and 3D applications.  For access over slow WANs though, organizations that want to enable access for remote users should seriously consider complementing it with Ericom Blaze, a software-based RDP acceleration and compression product. It accelerates RDP by up to 25x and is ideal for remote users connecting over slow WANs who need to access graphics-rich content and applications.

For a free eval, visit:

www.ericom.com/ericom_blaze.asp

Adam

Sean McPartlin wrote re: Microsoft RemoteFX is now available via public beta. Here's our in-depth guide to how it works.
on Thu, Sep 30 2010 10:33 AM Link To This Comment

RemoteFX is simply bringing a Microsoft "free" solution to help give Hyper-V and edge over VMware.

As I hate VMware and see it as a $ leach I say bring it on.

It would have been nice if they supported XP sp3 but let's face it Microsoft wants to sell more copies of Windows 7....  So RemoteFX will help that a little.

I think XP uses less resources then Windows 7... To the point that you could get get more XP VM's running on a server then Windows 7.

But oh well.  RemoteFX should make more of a diffrence for multimedia then other applications.

They showed off playing a 1080P video at Tech ED over RDP.  1080P people... who what's their users doing that?  

I see this as the solution for kiosk computers.. or putting thin-clients out in remote places...

This is about keeping the user closer to the data or app servers they need to access.  Allowing for a crappy to t1 link to a location where they just "view" the PC they were controling.  We have been doing this for years.

The GPU part is the new sauce to me.

1. It would seem the Quadro Plex 2200 is 13K... far too expensive.  However the little I read from Dell made it seem like you can cable it to more then one server.   Anyone know the details about this?

2. I plan on dropping a single FX 5800 into each hyper-v node.  No idea about special drivers... couldn't find them.  Anyone what to post a link?

3. This is just another step forward for Hyper-V to putting the "big boy" pants on.

Sean McPartlin wrote re: Microsoft RemoteFX is now available via public beta. Here's our in-depth guide to how it works.
on Thu, Sep 30 2010 11:10 AM Link To This Comment

I also wanted to add that Remote FX is offered more as an alterntive to :

Citrix (ICA-HDX) or VMware (PCoIP).

ojutan wrote re: Microsoft RemoteFX is now available via public beta. Here's our in-depth guide to how it works.
on Sat, Sep 21 2013 8:12 AM Link To This Comment

Citrix uses CUDA cores for compression in Xenapp 6.5

Most apps like Word, Excel, some CAE engineering tools they do only 2D don't put any load on GPU, I did monitoring on that years ago and I've found nothing, absolutely nothing. Relation between CPU and GPU usage was 100:1

Exceptions from this are programs they do real Direct3D overlays for having rendered 3D-output on screen. And THAT leaded to software rendering (below 2008R2 SP1) or just empty windows. For people doing this I recommend VDI instead :-)

The only loss the users have is the loss of the speed of GPU memory, because VMware and Terminal services do that in OS memory. And I hope to get that back with RemoteFX.

HyperV and ESX can do passthrough, they can also share a GPU among more virtual machines, but what sense does it make if it doesn't speed up things?

The CUDA acceleration for XA6.5 features might be the only reason to have a fat GPU in a Citrix server. I just ordered a couple of GPUs in different classes like Nvidia GT210 (16 CUDA cores), GS250 ( 96-128 CUDA cores), GTX 640 / GT 640 (384 CUDA cores) for demystifying the use. Vmware propsed a Dual GPU card having 4096 CUDA cores in total.... one of my big customers (I do some IT architecture design for them) already baught it and it didn't fit into any server they had :-(

In a tech blog I've found that Microsoft does not publish the impact of GPU in Terminal servers, VDI servers or HyperV hosts so my approach is to profile that I have - our CAE tool and the third party environment (Office 2010, Acrobat reader, some PDF printer solutions).

(Note: You must be logged in to post a comment.)

If you log in and nothing happens, delete your cookies from BrianMadden.com and try again. Sorry about that, but we had to make a one-time change to the cookie path when we migrated web servers.