"Advanced Format" 4k disk drives and performance: How will this impact your virtual disks?

Oh those disk drive guys... you'd better pay attention to what they are doing!

Oh those disk drive guys... you'd better pay attention to what they are doing! (Especially with virtualization, there are significant performance and potential failure impacts on your system when you replace a hard drive.) Got your attention? Yeah, you really need to watch for these new "Advanced Format" 4k disk drives!

Why?

Hard drives traditionally use a 512-byte sector size. On the platter, when you write 512 bytes of information, it takes up not only that 512 bytes, but also some sector overhead. A major source of that overhead to error correction--around 50 bytes. The error correction block (ECB) is what allows windows to “fix” a bad sector. Additional info is written in the ECB that magically allows software to figure out which byte is wrong and what the right value should be. (The sector is then marked as "bad" and the good data is moved to another sector.) The ECB is 50 bytes for a 512-byte sector, meaning that more than 8% of the platter space is taken by this overhead.

To increase capacity, the hard drive manufacturers want to move to 4k sectors. Today this isn’t about how big of a drive is possible due to sector size--it’s really about how much usable data can be stored on a given platter size. With a 4k sector size, that overhead for the ECB (now 100 bytes) drops to just over 2%. This 6% yield is important to the manufacturer. And since the overhead is more than just the ECB, the yield might is greater. For example, Western Digital is talking about 7 to 13% improvements in yield in their EARS drives. Keep in mind this doesn't mean you get to store more data on a 1GB drive--these are just internal efficiencies. This is a chance to store more data for less money, and that is a good thing as long as it doesn’t hurt performance or break things.

When?

Now!

While some work began on moving to 4KB sectors back in 1998, drives didn't start to appear on the market until the end of 2009. I've heard that the move to “Advanced Format” drives will be rapid, possibly as fast as the vendors can change out their lines. All of the major drive manufacturers claim that any new drive lines release after January 2011 will be “Advanced Format.” The ability to even purchase 512-byte sector drives has the potential to dry up pretty quickly, although it's unclear if that will really happen. I've seen a report that it may happen by the beginning of 2014, but if there's a need, you can bet someone will fill it.

There are many of these drives are on the market today from multiple manufactures--you just need to look at the label. They should be marked “Advanced Format.” For example, the Western Digital Green drives marked with “EARS” are so labeled. Seagate is not quite as clear, but I think that some (or all?) of their “Momentus” drives are also.

Technical Terms to Know

First, you need to be aware that some of these drives are shipping now. The hard drive industry, through International Disk Drive Equipment and Materials Association, has issued a campaign at the end of June, called "Are You Ready?". As part of this joint effort, they've coined the term Advanced Format which is how they're labeling the drives.

You need to be aware that there are two potential types of “Advanced Format” drives using 4k sectors. The first, are “4k native”, meaning they do everything with a 4k sector, just like the 512-byte sector does it today. The second--and this seems to be what is shipping today--is a transitional technology referred to as “512e” (the “e” is for “emulation”).

With a 512e drive, the interface looks like a 512-byte sector drive to the controller and operating system.  The drive internally uses a 4k sector, packing eight “logical” sectors in it. 512e allows the manufacturer to gain the yield while hiding potential issues with the OS, drivers, and applications, from seeing a sector size other than 512.

Of Performance and Problems

The 512-byte disk sector has been around a long time--something like 30 years. So it shouldn't be surprising that there just might be some software out there that hard codes 512 bytes. When it comes to applications, such a hard coding probably isn’t going to break anything since they don’t normally deal with the sector directly, but send chunks of data to the OS which then deals with the disk and the NTFS format. For applications, we're generally going to be okay at 4k sectors, with the application's performing the same (except with 512e as described later) but not necessarily gaining potential improvements.

But the OS, or driver, might do something bad with 4k. Today, for example, we are aware of “alignment issues”. Some NTFS format utilities, such as comes with Windows XP or Windows Vista operating systems, offset the NTFS partition by a sector count that isn’t a multiple of 8, such as LBA 63 for XP. This means that even when the OS tries to write a 4k block it thinks might be aligned, it isn’t. On a native 4k disk, I think this probably(?) wouldn’t work, but on 512e it's just really bad performance.

The 512e solution, while helping to keep things from breaking, induces some interesting performance differences. Consider two cases:

 

  • The OS writes a 512-byte logical sector. The hard disk must read a 4k physical sector, update 512 bytes, and write 4k out to the disk. Clearly this will hurt performance.
  • The OS reads a 512-byte logical sector. The hard disk must read a 4k physical sector and send the 512 bytes. While probably a little slower you’ll never notice the difference. But should the OS now read the next 512-byte logical sector that was in the 4k sector, it probably is already in the disk controller cache. So this will probably help performance a little.

 

In addition to operating systems, WinPE, cloning, and backup software may see the most performance impacts with 512e. Database applications would likely be susceptible also. And of course, in particular interest to this community, are OS virtualization solutions.

Microsoft’s VHD format is clearly optimized for 512 byte-sectors today, and you might see significant performance issues with 512e drives. I’m not sure about Citrix or VMware. Microsoft is also rumored to be producing a hot fix for 2008 R2, (you can search for KB 982018 981208 and find references, but it looks like that KB has been pulled for some reason).

Real World Examples

Users report that it takes up to 6 hours to format a drive on Windows 7. After formatting, the drive performance is reported as reasonable.

Manufacturers are releasing extra utilities to help. Basically you run these utilities after installing XP or Vista. For Western Digital, they call it the “WD Align” utility. For Seagate it's the “SmartAlignT” utility. These utilities essentially move the partition after it's created to make it more naturally align and may take 20 minutes or more to run. Windows 7, Mac, and Linux do not seem to need this alignment. How important is this alignment to performance on XP? Check out these benchmarks at HotHardware.

I also note that I recently purchased a hybrid drive (Momentus 500GB hard drive with 4GB Flash) that might be one of these “Advanced Format” 512e drives, although it doesn't say so on the label. (Possibly because they consider it hidden by the flash interface?) Formatting definitely took a long time, although the flash interface might have helped keep it from being 6 hours.

So Now What?

Make sure you know what you're buying. Keep an eye out for “advanced format”, “512e”, or “4k” designations. This will be especially important if replacing drives in a raid array. In fact, before using the new drives in a raid array (even matched 512e sets) I would want to contact the RAID manufacturer and get their advice.


    Author: Tim Mangan is a Microsoft MVP for App-V and a CTP with Citrix.  He holds the position of "Kahuna" at TMurgent Technologies and now considers himself a "researcher" (which means he web searches everything twice). He has spoken at every BriForum and other venues. Read more from Tim at his home blog or website.

Join the conversation

6 comments

Send me notifications when other members comment.

By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy

Please create a username to comment.

Great article Tim!


And a very interesting point about the VHD issue.


Would you have any idea how quickly these drives will make their way into servers and storage from the big three OEMs?


Also, a minor correction. SmartAlign is actually not a utility but a firmware feature in the Segate drive firmware. :) Segate likes to call it a 'utility-free' solution. www.seagate.com/.../tp615_smartalign_for_af_4k.pdf


Cancel

Thanks for that correction.  I think someone (I can't remember who - maybe it was them?)  added a jumper to the drive to activate a firmware compatibilty change also.


As to the question - I suggest asking whoever you buy from.  They should know.


Cancel

I found the correct article number for the Windows 7 hotfix, that improves performance with Advanced Format drives:


support.microsoft.com


It looks like you had a little switcheroo with digits. ;)


Cancel

Thanks for finding this.  It seems Microsoft originally posted it as 982018, which is the number originally given to me by Microsoft before it was posted.  A number of kb watch sites have reference to that number but then can't find the article.  


Any way, it can now be found as 981208 and has the same title.  Now that we can see the detail, the KB addresses a performance issue with large file transfers (writes) to such a drive.  It applies to both Windows 7 and 2008 R2.  This is a requestatable hotfix, and one would presume it should show up the the SP1 releases.


Cancel

Hi Brian,


Thanks for your coments on this very interesting topic.


Can you comment on why we haven't seen any native 4k drives yet when windows 7 and vista are supposed to support them?


Even the new 3tb seagate drive presents a 512e interface...


Cancel

@topsturvy


I think the hotfix is a great example of how those OSs don't support the drives well.  You would need the hotfix for a 4k native drive as well to keep performance from bogging down.  But it is so much more than just the OS from Microsoft.  


Any software that works with the disk at a low level, like backup software or 3rd party defragmentation, probably need to make changes.  I mean backup software will work and probably perform well when backing up, but if you ever had to restore it might take a day instead of hours.  Given how far Microsoft was behind on this one, guess how far behind other vendors will be?


This is kindof "wakeup call" time for vendors, but with this audience it is more of be carefull what you are buying.  We tend to just think in terms of spindle speed and interface and capacity; we need to look more closely or we may be surprised.


Tim


Cancel

-ADS BY GOOGLE

SearchVirtualDesktop

SearchEnterpriseDesktop

SearchServerVirtualization

SearchVMware

Close