Overview and Scope

I wrote this after a colleague requested some things they should read to catch up on what’s been going on in the IT industry over the last few years.

Enterprise IT often moves at a much slower pace than other IT adopters, and we can be so inwardly focused that we don’t get a chance to see what’s going on outside. I imagine many others are in the same boat, so I thought I’d publish something for both internal and external consumption.

The intention is to summarise some general observations on the IT industry from a systems administration perspective; comment on where I think things are going; and then provide some links for further learning/research if you’re that way inclined. My focus of interest is mainly in the Virtualization, Cloud and Storage space so apologies for the lack of Windows-based innovation.

General observations

Virtualization is commoditized

In the last few years, virtualization (hypervisors) as a way of decoupling operating systems from their underlying hardware, has been largely commoditized. Hypervisors are so common as to be taken for granted, and understanding basic virtualization concepts is assumed. In fact, the concept of virtualizing compute is so normal now that vendors are looking to virtualize networking and storage as well as compute.

In the world of High Performance Compute, virtualization of computers is still fairly “exotic”. This is mostly because our main business needs to eke out every last CPU cycle to free up hideously expensive simulation licenses ASAP. Nowadays, when I tell vendors who solve problems caused by virtualization that we barely virtualize, I get funny looks 🙂

All-Flash Storage Arrays

The storage industry as a whole is moving away from regular spinning disk to flash memory, fast. AFAs (All Flash Arrays) are everywhere, it seems.

Cloudy cloud cloud

Cloud is everywhere, but it’s not just virtualization done in a data center run by someone else. Cloud is all of the value-add stuff that you want after you gain the flexibility of virtualization. Cloud is automation of workloads, it’s customer self-service VM deployments, it’s automated spin up of VMs to meet demands, it’s automated failover to a DR site in the case of a disaster. Cloud is not having to worry about if server Freddie is going to suffer a disk failure soon, whether I’m about to run out of storage space, or if I’m going to run out of network switch ports when I next need to roll out some hardware. It’s liberating! 🙂

Reduction of friction for developers

What’s the biggest problem for Devs? Pesky IT slowing them down!

Two things of interest in this area:

Where are things going?

Hyper-Convergence

Managing the Compute, Storage and Networking elements of a Virtualization stack can be bothersome. Some vendors are pushing Hyper-Convergence, which effectively enables compute nodes to cluster together, pool their storage, and then treat that storage as a shared pool. The idea here is that you focus less on managing your hardware stacks and more on managing VMs/adding value elsewhere. A worthy goal.

DevOps

Perhaps not as applicable to the Semiconductor industry as others, but still interesting, is the DevOps movement. DevOps is most common in companies that develop customer-facing web applications or mobile phone apps. Devs create the code and deploy it, Ops make sure the platform (server, OS etc) is stable and running. My take on DevOps is that its aim is to encourage Developers and Ops to co-operate and communicate with each other, rather than pointing the finger at each other, leading to a more stable platform and faster innovation. Magic.

Hybrid Cloud

In the last few years, everyone’s figured out that pure Public Cloud is not necessarily a panacea, and that a blended approach to cloud is needed. Cue: Hybrid Cloud, which is usually a combination of Public cloud and Private cloud with the ability to move workloads between.

Automate or die

There’s a general agreement that our roles as IT Professionals need to change, and that simply being a sysadmin/server hugger isn’t enough. The idea is to take steps towards automating much of our roles and then for us to focus on adding value to the business elsewhere (like, you know, having a relationship with the business). We should also start treating our resources as Cattle and not Pets. The adoption of “the Cloud” is quickly accelerating this. Scripting languages like Python and/or PowerShell can help to automate common tasks.

Who’s doing what (vendors)

While this list isn’t exhaustive, I’ve tried to summarise the main players I’m aware of in each niche. The likelihood of me missing someone is high.

Cloud platforms

Hyper-Convergence

Storage Arrays

Virtualization focused and/or All Flash Arrays

NAS

Filling the gaps

Virtualization

In HPC, some of us don’t get much exposure to virtualization tech, so I’d recommend checking out VMware’s free courses such as Data Center Virtualization. VMware ESXi (aka vSphere) is pretty much the defacto standard for hypervisors, but Hyper-V, and KVM (featured in OpenStack) are also pretty popular.

Where can I learn more?