It's quite easy to get swept up by Meltdown and Spectre, news of which broke last week and reached a boiling point as we were compiling our Friday Notebook. Though these vulnerabilities are real, and remediating them will be a challenge, the impact on performance has yet to be fully assessed or felt. Certain areas of IT will be affected more than others, so it's important to narrow down your thought process when it comes to how you'll handle this.
In other words: stay in your lane.
That's tough to do, because at every turn you find more information about how big of a deal this could be. We've all heard that the vulnerabilities affect Intel, AMD, and even ARM processors, and when you look around your office (or house), it's not hard to imagine some augmented reality view of the world that highlights everything with blinking lights as a vulnerability, including the imaginary AR glasses you're using. Here are a few devices that popped into my head over the weekend:
- Xbox One
- Thin clients
- Nest thermostat
- Skybell doorbell
- Apple TV
- Roku Stick
- Smart TV
These, of course, are in addition to all the laptops, desktops, phones, tablets, servers, and cloud platforms that affect our lives. The thought of patching every single thing is pretty overwhelming.
A healthy dose of the weekend has seen some of the FUD give way to more organized thoughts about just what, exactly, the impact is. For example, Microsoft has said the Xbox One's security model means that neither vulnerability is a threat. ARM released a list of affected CPUs, and, though the chips on the list are in a lot of devices, it doesn't come close to covering all of the devices that run on ARM processors. In other words, your wifi doorbell is safe. (Of course, the iPhone you connect to it with is affected, and your router might also b….ok, it's getting away from me again. See how easy that is?!)
The point is, there is so much speculation out there and it's too easy to react to fragments of information before looking at the big picture. With that in mind, here are a few things that have stood out to me when taking a step back:
People are mad at Intel for the wrong reasons
Intel's stock took a bit of a hit after news broke, and there's already a few class action lawsuits directed their way over the vulnerability. But why is Intel all of a sudden infallible, when Microsoft has a monthly event named after the endless stream of security updates they produce? If someone can show that Intel (and AMD, and ARM) knew that this was there all along and chose not to fix it, that's another story, but to think that hardware is somehow immune to bugs is silly.
Anger at Intel might be better directed at their CEO, Brian Krzanich, who allegedly sold nearly 900,000 shares of Intel stock in October and November. Since Intel was alerted to these vulnerabilities in June, this could become a problem with the SEC. Predictably, Intel and Krzanich have said the sale is unrelated, and, to be fair, the price he sold the stock for is less than a dollar more than what Intel is trading at as of this writing. Keep your eye on this story.
Performance hits are unknown
Initial reports indicated that the performance hit on some systems could be as high as 30% (some outlets further sensationalized an already sensationalized number, claiming as much as a 50% degradation!), and there were some charts to back this up. Nevertheless, those that have rolled out the patches thus far haven't noticed much, and Tom's Hardware showed through their own testing that any changes still fall within the normal margin of error of their benchmarking tools.
Admittedly, the bulk of the tests out there have been run on individual PCs, which can absorb inefficiencies better than VDI environments. We could very well learn that in a VDI environment, where every little bit of resource consumption is multiplied, these patches cause some reduction in density, but I wouldn't click the "Submit" button on that order to 50% more servers just yet.
On the cloud side, there are mentions of performance hits from various news sites, but I had trouble finding anything definitive. If there are problems, they're likely workload-dependent, but the details about which workloads haven't been nailed down. Or, it could be that your news organization was waiting for the first thing to go wrong so you could report on it with some snarky title.
The path to remediation is unknown
We know that there are OS patches to help deal with the problems introduced by Meltdown and Spectre, but some people are saying that there will also need to be BIOS and microcode updates as well. OS patches are one thing – we're pretty good at doing that by now, but even then you should probably wait. Evidently, there are other things that have to be done with antivirus before you can apply the OS patches. Separately, it appears that Microsoft has pulled their latest patch because some AMD systems won't boot after applying it. This is almost certainly due to the rushed nature of the updates, or, as Woody Leonhard called it in his excellent article, "Windows, Meltdown, and Spectre: Keep calm and carry on," the "change-the-blades-while-the-blender-is-running" approach to patching Windows.
Assuming Microsoft gets the updates right eventually (it appears Linux and Apple have already done so), deploying them won't be a problem. Updating firmware, though, will likely require touching the desktops and servers, which could become a bit of a nightmare. Like IT in the 90's. You might not need to do it at all, though, so you'll just have to cross that bridge when you get to it.
In the cloud, Azure and AWS customers have already been protected. This, along with an article from Microsoft that suggests patching Hyper-V and rebooting the VMs is enough to protect version 8 VMs (older VM versions need a simple registry change first), would indicate that the problem can be fixed at the hypervisor level without having to patch individual VMs.
I could spend days tracking down more information about Meltdown and Spectre, but through the work that went into this article, I've realized one important fact: This has all become sensationalized.
Some flaws are bigger than others, for sure. WannaCry was a huge deal, but, like anything else, if you followed proper security procedures and kept your machines up to date, you weren't at risk. While it's true that strict adherence to application security protocols isn't enough to completely protect you from Meltdown and Spectre (which, if you haven't heard, allow the pre-execution of code before the code is checked for security), there are two important things to remember:
- Following security and update procedures still limits your attack surface
- No malware exists to take advantage of these vulnerabilities right now
Look at it another way: With WannaCry, Microsoft did something they said they'd never do–they patched Windows XP, which reached End of Life three years earlier. In their blog post entitled "Protect your Windows devices and Spectre and Meltdown," Microsoft officially states that they "…will not be issuing updates for Windows Vista or Windows XP-based systems…" Though it’s not a clear barometer of the impact of the vulnerability, it sure appears that Microsoft considers WannaCry a much bigger deal than Meltdown and Spectre.
Reflecting on this entire hype cycle, I've noticed a few red flags. For example, when a vulnerability is announced along with matching websites, pre-created logos that are already licensed for use by everyone, and after months of the big companies secretly knowing about it, you can bet more effort was put into how this information was released than you put into your wedding.
Yes, it's a big deal that will affect everyone in some way, but we have to resist the impulse to do something for the sake of doing something or to fix problems that we don’t yet have (like performance degradation). Hardware can have bugs, too, and, like software, those bugs will get worked out. We'll find a way to deal with it now, and when the dust completely settles from this we'll have better hardware from chip manufacturers, and more secure operating systems.