Introduction from Jack: To wrap up 2019, we invited some of our frequent contributors to do a final piece on any topic they’re interested in right now. When Rachel mentioned Kubernetes, I thought it was great. I know I sat through parts of the VMworld 2019 keynote wishing I had time to learn more about it!
Kubernetes (often called K8s) is “an open-source container-orchestration system for automating application deployment, scaling, and management.” (This concise definition is from Wikipedia.) This basically means that it’s something that is of use to DevOps, delivery managers, and developers.
Kubernetes is not something that obviously or directly affects the end user or customer for most vendors, but can solve a lot of problems vendors face delivering high-quality products and responsive user experiences to their end customers.
Let’s start with a few real challenges vendors face that impact what customers actually experience:
- In-application-only and hardware-specific bugs. Most folks will be familiar with the scenario of a customer calling support with a software problem, but support simply can’t reproduce the issue. There may be some back and forth while support gets a video of the customer and a ton of system dumps and hardware information, and some running around to find some similar hardware, but nobody’s quite sure why the issue only shows up on the original system. At this stage it may get escalated into development, and a grumpy developer may well have a rant about support not providing enough tangible to source the issue, and a “can’t reproduce flag” may well be attached to the ticket.
- A vendor launches a web service-based application, e.g., a clickable map of their office locations, it gets promoted by a celebrity and demand surges. All of a sudden, the back-end of the application can’t cope and the hosting server crashes or hangs; the end user finds the map is slow, unresponsive, or perhaps not available.
- A financial institution or bank might run many applications to access customer accounts; these may well be called by a chain of applications, e.g., the application to issue a statement. Controlling and limiting which applications can access functionality is critical for security.
Before we specifically go into the details of Kubernetes, let’s first start with the basics of containers. Back in 2007, some Google engineers added features to the Linux kernel under the umbrella term cgroups; native and intrinsic to the kernel, these features provided functionality to limit and isolate the resources (CPU, memory, disk I/O, network, etc.) used by collections of processes. Leveraging these features allowed containers to appear—namely LXC (Linux Containers), an OS-level virtualization method for running multiple isolated Linux systems (containers) on a control host using a single Linux kernel. In addition to the cgroup resource control, functionality was provided for namespace isolation, as well as complete isolation of an application's view of the operating environment, including process trees, networking, user IDs, and mounted file systems. In other words, this was a way of securely virtualizing an application within the Linux Kernel itself, without the need to spin-up a virtual machine (VM).
In traditional virtualization, each VM has its own OS. This means that segregating applications by putting them in separate VMs has high overhead, as it involves spinning up the VM and resourcing it both in terms of time and hardware. In contrast, containers are very scalable and lightweight, and also very secure particularly, as they can only interact through defined interfaces. Often, containers are referred to as virtual environments (VE), and because it’s built into the kernel, there’s no emulation overhead and you get native performance.
There’s no single way of defining containers; and several genre-extending LXCs have evolved. Docker is probably the most-high profile; it abstracts the end user a bit further and automates the process of defining containers, how they communicate, etc. There’s a useful FAQ on the Docker site that explains this further, including the container configuration and templating functionality. One of the key features is that Docker provides a mechanism to generate containers that are portable across different machines, although numerous other mechanisms to do this already exist. It also worth noting that whilst the Kubernetes framework is open sourced, Docker is a proprietary product, indeed one that has very recently changed hands.
Once we have containers, we now have a solution to the problem of those in-application or “it works on MY machine” bugs. If an application is deployed within a container, its view of the world is always the same. The segregation and sandboxing means another application can’t overwrite the memory being used and so on. Containers can be useful for individual developers and managed and scripted on a small scale by numerous mechanisms.
Containers are strictly defined, and as such, how they can communicate can be very tightly controlled and limited. Applications can be built for very specific tasks, which are only allowed to communicate with a very restricted audience. This type of architecture allows for microservices, which are inherently secure and scalable. Containers support small, very specific applications which can do very bespoke tasks and provide specific services (e.g., print a bank statement for a customer). Microservice architectures have a vast array of benefits: low-attack surface, so it’s secure; and clearly defined functionality, so ownership and maintenance is easy.
Kubernetes, the popular option
Kubernetes comes in as one of the most popular container orchestration frameworks, again originating from Google, and now open sourced. There are alternatives (some have built their own in-house orchestration), and many products that consume Kubernetes to produce proprietary and supported orchestration systems, or even IaaS/PaaS products/services (in a similar way to how open-source KVM is consumed). Being able to orchestrate lots of containers provides a way to help with that problem caused by the busy map application, with orchestration to provide a highly available app. Several containers with identical configurations can be spun up associated with a load-balance, and in the event of a usage surge, more containers can be spun up very quickly. Or in the event of a failure, failover can be managed.
Kubernetes itself is only a framework though, which is probably why so many explanations are simply turgid as they plunge in discussing the structure of Kubernetes components (pods, namespaces, etc). As a framework, you usually have to add an awful lot of third-party tools or functionality to build a working system. It’s incredibly flexible, as the possibilities are endless, but to build your own system you need to understand the architecture. If you want to start going deeper – the most accessible starting point is probably “The illustrated Children’s Guide to Kubernetes,” available on YouTube. Docker is probably the most familiar container definition Kubernetes is used to orchestrate, but many other options are possible, and the use of Kubernetes is independent of the choosing Docker or an alternative. (Occasionally the two technologies are conflated/confused in explanations.)
The types of products and services that are added to Kubernetes give an idea of what is needed: storage and host management (AWS, Ceph, Terraform), container templating and service registration (Docker, Rancher, Helm, JFrog), logging and metrics (Grafana, Fluentd), code building and publishing (Jenkins, GitHub), and load balancing (NetScaler/Citrix ADC).
Containerization is almost certainly the way to go if you have products that need to run across enterprise and cloud infrastructure. So, who is likely to adopt container orchestration, who is likely to make money from the surging demand, and what markets (e.g., IoT) could it open for vendors that invest in Kubernetes-like technology streams?
In-house EUC vendors
EUC solutions usually include a vast number of interacting products from numerous vendors communicating via integrations, plug-ins and APIs. Componentized technologies naturally benefit from containerization within their development and testing infrastructure, raising product quality, and reducing support issues and calls. As such, if you’re an engineering manager at an EUC vendor, this is stuff you really need to be looking at.
Enterprise EUC customers
For those already invested in large-scale VDI, managed container deployments make sense for their own development and service products. Think of those large financial and federal customers developing apps and services as well as consuming VDI. If you are already a VMware shop, then getting your DevOps and container architecture from your existing enterprise supplier makes a lot of sense. Indeed, VMware seem very much aware of this and have a credible Kubernetes story with their Kubernetes management control plane VMware PKS. For the EUC reader, the information around VMware’s offering is probably one of the best insights into this market, VMware have put out some nice high-level (readable) whitepapers in this area, too: “Demystifying Kubernetes” and “Driving Digital Transformation with Containers and Kubernetes.”
Of the VDI vendors, VMware does seem to be the only serious player at the moment, with the largest cloud players being their most significant competition (Amazon AWS, Microsoft Azure, and Google GCP). The one small pocket of Citrix that seems to be very vested in the wider industry trend is their NetScaler/ADC (load balancing and similar functionality) team. It’s just an integration that allows those using Kubernetes to add Citrix ADC to a Kubernetes framework but there are no signs the wider organization is looking for a piece of the action, with a play to be involved in that management or infrastructure. They’ve some good blogs on the ADC integration though, “Why is everyone talking about Citrix and Kubernetes” and “Citrix ADC for Kubernetes knocking at Kubernetes Front Door.”
Existing EUC customers looking at VMware are likely to take a look at the increasingly mature offerings direct from the big clouds such as Amazon’s AWS EKS, Microsoft Azure’s AKS (Azure Kubernetes Service), and Google GCP’s GKE (Google Kubernetes Engine). With on-premises options and increasing cloud VDI options (particularly Microsoft’s WVD) between them, it’s clear why VMware might be keen to keep VMware shops happy rather than leave the large DevOps and application market of their traditional large enterprise base open.
Outside of EUC/VDI, traditional big on-premises enterprise providers are also going in on Kubernetes-like infrastructure, like IBM, Oracle, and SAP, so this is definitely a mainstream, enterprise trend that VMware is aligning to and with the overlap of their customer base makes a lot of sense for them.
Could container architected products push out those based on traditional virtualization?
Consider the economies of products using containers (VEs) instead of VMs: there are cloud products out there where traditional VDI can be used, but containers offer similar benefits with greater scalability and better economics. An obvious example is the remote browsing products we’ve seen a glut of and have reviewed. A few months ago, we looked at the container based offering from Menlo. For traditional VDI vendors offering similar but based around VMs, the higher costs of infrastructure to support VMs may make such products far less viable than those such as Menlo (can a product such as Citrix Remote Browser scale as easily and without higher costs?). There are some “cloud” offerings from EUC vendors where they’ve obviously taken their legacy VM technology and hacked it into the cloud—you can literally see (and wait for) VMs whirring up to support DDCs, etc. Products architected for the cloud may well have a competitive advantage; with containers you can often get 2-3x more users on a server than with traditional VM-based virtualization.
A good Kubernetes strategy could be good for IoT markets
The microservice nature of containerization also lends itself very well to IoT products. With potentially millions of smart and connected devices to update, the lightweight nature of containerization and the inherent scalability of Kubernetes are a good match and complement each other well. We wrote a while ago about whether there’s a place for EUC vendors in IoT, since then VMware has continued to invest in IoT, and in tandem with their managed Kubernetes play, are in a strong position to build an IoT platform, whilst Citrix’s OctoBlu IoT acquisition has since ceased to be supported. Again, this is an area where Citrix have made some noise and demos (Alexa command of your VM or a lightbulb type stuff) but little tangible has resulted that looks like an enterprise-grade, scalable product.
Beyond Kubernetes: Managed hosting providers and PaaS
Kubernetes is undoubtably an absolute beast, one which requires a significant commitment of budget, skilled DevOps, and administration; even with the Kubernetes management services on offer it’s a huge commitment. For many, particularly small and medium businesses, but also those large organizations with fairly homogenous needs (e.g., large-scale websites, web apps to manage) a higher-level product may be a better choice. There are rafts of organizations with hundreds if not thousands of websites and apps where container orchestration offers a lot of benefits, but building your own Kubernetes is way overkill.
Both managed hosting providers or smaller bespoke PaaS (Platform as a Service) may offer a more turnkey offering. Particularly with PaaS offerings, the vendor takes on the implementation and maintenance of Kubernetes/similar framework and all the component products within them, so your developers, application delivery people, content maintainers, and sys admins don’t have to worry about whether all the products within the framework are security patched or how they interact and they provide SLAs for high availability. If there’s any issue, there are enterprise-support agreements and helpdesks available without having to take on support internally. One name EUC may be familiar with in this area is DigitalOcean where Citrix’s former CEO, Mark Templeton moved onto for a while.
It’s telling that, under-the-hood, many of the vendors listed in Gartner Magic Quadrants, such as “Digital Experience Platforms” or “Web Content Management,” are essentially Kubernetes/container orchestration with a lot of out-of-the-box plumbing and tooling and some easy-to-use GUI/WYSIWYG baubles.
Although not on the radar of a 500-seat XenApp or Horizon user, the widespread uptake and industry-wide trend to Kubernetes is looking like a juggernaut with huge momentum. It’s definitely one to be aware of; even if you are unlikely to deploy K8s yourselves, it’s nice to know a little but about the underlying direction of cloud and enterprise infrastructure.