I had another reminder today that SIDs *do* matter in a virtual Windows environment!
We used to religiously change SIDs with a variety of third-party tools (e.g. Ghostwalker, newSID, etc.), but between Mark Russinovich stating that SID changes were no longer necessary and the speed advantages of various “Quick prep” tools which I’ll define below, everyone kind of forgot why we bothered changing machine SIDs.
I’d like to start by talking about what happened to me today, and then expand a bit about the scenarios where SIDs matter very much indeed.
Never Clone a Domain Controller without Sysprep
I was working in a new sandboxed lab environment, where I had cloned an existing server, changed the server ID and IP address, and started building servers. The first server cloned became a domain controller, the next a management server, the third a file/print server. Logging on as the local or domain administrator, everything worked fine.
But when another domain administrator account logged on to the management or file/print server, things were decidedly odd. Stuff like trying to ping the DC failed with an “Unable to contact IP driver. General failure” error. A lot of network stuff just failed and yet you could do an nslookup. It turned out to be a SID issue which was resolved by changing the machine SID (management and file/print) with sysprep. There’s a lesson here somewhere because if I’d sysprep’d the DC beforehand I wouldn’t have noticed.
So why did this happen and why don’t we see this problem normally?
When you dcpromo a machine to turn it into a domain controller, the first DC uses the machine SID as the domain SID, whereas subsequent DCs pick up the same SID as the first DC. In the good old days you could create a remote DC by using newSID (Sysinternals) to change a machine SID to match the SID of an existing domain controller. In this case, all the clones had the same SID so the domain SID and the local machine SIDs were identical.
A user SID consists of two parts, a domain/machine part and then a user number or more correctly relative ID (RID), for example S-1-5-21-636461855-2365528612-2953867313-67402, where S-1-5-21-636461855-2365528612-2953867313 is the domain SID (or machine SID if logged on as local user) and 67402 is the RID. It’s important to stress that this is normally a unique identifier for that user.
Well as you’ve probably figured out, this is not quite true, because we have a special case here. The administrator account always has a user id of 500, so in our sandbox lab, both the domain administrator and local administrator account on all the machines had the same SID. Where it gets more interesting is when a domain admin (not administrator) logs on to one of these clones, because the domain SID is the same as the machine SID. So the operating system thinks this is a local user, but the RID isn’t in the local SAM database and gets very confused and peculiar things start happening. Run sysprep and things work as they should.
Cloned “Quick prep’d” Virtual Desktops and Local Administrator Security
Desktop virtualization (VDI specifically) is another scenario where having identical SIDs can potentially be a problem. There are a number of technologies, which I’ll call quick_prep as a generic term because a whole lot of other word combinations are proprietary. In the virtual machine (VM) provisioning context, quick_prep is a way of cloning or creating new virtual machines (VMs) quickly without running sysprep. Quick_prep generally involves cloning an existing image that is domain joined, changing the machine ID and hostname of the target VM and re-joining the target VM to a domain using pre-created domain machine accounts.
Early versions of Windows had issues with identical SIDs and security equivalence between machines, but I didn’t think this would still be true for Windows 7. After all, both Mark Russinovich and Pete Downing (Citrix and formerly Ardence) suggested it wasn’t a problem from a security viewpoint. Both VMware View and Citrix XenDesktop (Provisioning Services) offer “quick_prep” methods and while the technology is a bit different, the end result is the same. So, armed with this knowledge, and despite the errors above that I got from cloning a Domain Controller, I thought we would still be on safe ground when cloning Windows 7 virtual desktops with quick_prep.
I should point out that today in Quest vWorkspace we use Microsoft sysprep, because it’s the only method officially supported by Microsoft. (Note that even sysprep has a restricted list of supported configurations.)
I’m going to present an example, which happens to be XenDesktop because it was handy, but any quick_prep technology is likely to give you the same result. The screen shot below is from a newly-provisioned Citrix XenDesktop 5.0 Windows 7 VM. Both machines are domain members but the logged-in user is the local (not domain) administrator (xdwin7-008\administrator).
Note that I was able to open the c$ share on another newly cloned XenDesktop 5 Windows 7 VM for read/write access without authentication! This isn’t supposed to happen! Windows 7 is a lot more secure?
While I had assumed that additional user credentials would be required, they weren’t. What we see here is user equivalence between two VMs having the same SID, because both users have the same name, SID+RID and password. If I change the local administrator’s password, log out and back in again I need credentials to get to the other VM’s C$ share.
That isn’t so bad because even if you have local administrator rights and can change the common admin account password, that doesn’t give you open access to other VM’s. But how easy is it to find the local admin password?
Most of the password cracks for Windows 7 depend on accessing the Windows 7 system disk after booting to a linux CD with SAM cracking tools. That’s not going to work in this scenario, but if you could get a copy of the security account manager (SAM) hive to crack offline, then it wouldn’t be that hard.
There’s an old NT 4 resource kit tool called regback that used to do a nice job of backing up and restoring registry hives. It turns out that regback still works with Windows 7, provided you have administrator rights.
Once you’ve got a copy of the SAM, it’s just a matter of using a password cracking tool to crack the NT hash.
That gives you the credentials to get network access (via default C$ and admin$ shares) to any other quick_prep’d VM made from the same template. Sounds bad, but when you considered that any provisioning technology, sysprep included, generally has a common local administrator account, quick_preped machines are from this viewpoint no more or less secure. Except for one area that is sufficiently important that it has to be highlighted. Anything running in the local administrator context has unsecured access to any other VM made from the same parent. That has some really interesting ramifications when it comes to a virus/worm that infects one VM and then has full access to all the other VMs because of the security equivalence. If that happens I hope you’ve got a good antivirus!
SCCM and KMS
Of course sysprep does a lot more than just change a machine’s SID, it also changes the CMID (client machine ID) which is used by KMS for licensing, and so tracking the number of Microsoft licenses in use might get a whole lot harder too. (See http://support.microsoft.com/kb/929829)
The last, rather broader SID-related issue comes about because a number of machine management products use the local machine SID or a derivative as an identifier in their inventories. Even Microsoft has SID dependencies.
If we look at SCCM it doesn’t look like a good an example because it uses a non-SID identifier, an SMS GUID. Provided the SCCM client is installed after cloning, you don’t have a problem. If you’re unfortunate enough to have the SCCM client already installed on the VM before you converted it to a template, you will have a major house-keeping problem resolving duplicate GUIDS.
Of course, there is a mechanism to change the SMS GUID if any of the following parameters change:
- The SMBIOS serial number (no change in quickprep)
- The machine SID (no change in quickprep)
- The hardware ID (see below)
The HardwareID may change if any 3 of the parameters below are changed:
- FirstdriveSerial (no change in quickprep)
- MAC Address (yes – will change)
- CDROM device (no change in quickprep)
- DisplayAdapter (no change in quickprep)
- HWIDVersion (no change in quickprep)
- ProcessorSerial (not sure here)
- DiskDevice (no change in quickprep)
- SCSIAdapter (no change in quickprep)
- DiskAdapter (no change in quickprep)
- ProcessorType (no change in quickprep, in homogeneous cluster)
- RAMSizeMB (no change in quickprep)
Since we’ve only got 1 or 2 changes it’s probable that the HardwareID won’t change so our VM will have the same SMS GUID as the parent template. So unless you’re careful, quickprep and SCCM aren’t a good pairing if you want to manage your persistent VMs.
So, do SIDs matter? I guess it depends. In a scenario of non-persistent VMs where you can blow away and re-provision the whole fleet if there’s a problem, where you don’t let users have local administrator rights, the answer is a slightly qualified “no”. If you’ve got persistent VMs, let users have local administrator rights and intend to use something to manage the VMs, quick_prep may not be a very good idea and you may need to stick with the full “sysprep”.