Overview and Scope
I wrote this after a colleague requested some things they should read to catch up on what’s been going on in the IT industry over the last few years.
Enterprise IT often moves at a much slower pace than other IT adopters, and we can be so inwardly focused that we don’t get a chance to see what’s going on outside. I imagine many others are in the same boat, so I thought I’d publish something for both internal and external consumption.
The intention is to summarise some general observations on the IT industry from a systems administration perspective; comment on where I think things are going; and then provide some links for further learning/research if you’re that way inclined. My focus of interest is mainly in the Virtualization, Cloud and Storage space so apologies for the lack of Windows-based innovation.
Virtualization is commoditized
In the last few years, virtualization (hypervisors) as a way of decoupling operating systems from their underlying hardware, has been largely commoditized. Hypervisors are so common as to be taken for granted, and understanding basic virtualization concepts is assumed. In fact, the concept of virtualizing compute is so normal now that vendors are looking to virtualize networking and storage as well as compute.
In the world of High Performance Compute, virtualization of computers is still fairly “exotic”. This is mostly because our main business needs to eke out every last CPU cycle to free up hideously expensive simulation licenses ASAP. Nowadays, when I tell vendors who solve problems caused by virtualization that we barely virtualize, I get funny looks 🙂
All-Flash Storage Arrays
The storage industry as a whole is moving away from regular spinning disk to flash memory, fast. AFAs (All Flash Arrays) are everywhere, it seems.
Cloudy cloud cloud
Cloud is everywhere, but it’s not just virtualization done in a data center run by someone else. Cloud is all of the value-add stuff that you want after you gain the flexibility of virtualization. Cloud is automation of workloads, it’s customer self-service VM deployments, it’s automated spin up of VMs to meet demands, it’s automated failover to a DR site in the case of a disaster. Cloud is not having to worry about if server Freddie is going to suffer a disk failure soon, whether I’m about to run out of storage space, or if I’m going to run out of network switch ports when I next need to roll out some hardware. It’s liberating! 🙂
Reduction of friction for developers
What’s the biggest problem for Devs? Pesky IT slowing them down!
Two things of interest in this area:
Where are things going?
Managing the Compute, Storage and Networking elements of a Virtualization stack can be bothersome. Some vendors are pushing Hyper-Convergence, which effectively enables compute nodes to cluster together, pool their storage, and then treat that storage as a shared pool. The idea here is that you focus less on managing your hardware stacks and more on managing VMs/adding value elsewhere. A worthy goal.
Perhaps not as applicable to the Semiconductor industry as others, but still interesting, is the DevOps movement. DevOps is most common in companies that develop customer-facing web applications or mobile phone apps. Devs create the code and deploy it, Ops make sure the platform (server, OS etc) is stable and running. My take on DevOps is that its aim is to encourage Developers and Ops to co-operate and communicate with each other, rather than pointing the finger at each other, leading to a more stable platform and faster innovation. Magic.
In the last few years, everyone’s figured out that pure Public Cloud is not necessarily a panacea, and that a blended approach to cloud is needed. Cue: Hybrid Cloud, which is usually a combination of Public cloud and Private cloud with the ability to move workloads between.
Automate or die
There’s a general agreement that our roles as IT Professionals need to change, and that simply being a sysadmin/server hugger isn’t enough. The idea is to take steps towards automating much of our roles and then for us to focus on adding value to the business elsewhere (like, you know, having a relationship with the business). We should also start treating our resources as Cattle and not Pets. The adoption of “the Cloud” is quickly accelerating this. Scripting languages like Python and/or PowerShell can help to automate common tasks.
Who’s doing what (vendors)
While this list isn’t exhaustive, I’ve tried to summarise the main players I’m aware of in each niche. The likelihood of me missing someone is high.
- Microsoft Azure – a worthy contender. You can also play with this for free
- OpenStack – create a private cloud for free* (*requires time and patience)
- Nutanix – probably the most well-known in this space. They have a Community Edition of their software that you can play with for free.
- VMware VSAN
Virtualization focused and/or All Flash Arrays
Filling the gaps
In HPC, some of us don’t get much exposure to virtualization tech, so I’d recommend checking out VMware’s free courses such as Data Center Virtualization. VMware ESXi (aka vSphere) is pretty much the defacto standard for hypervisors, but Hyper-V, and KVM (featured in OpenStack) are also pretty popular.
Where can I learn more?
- Check out the links above, and:
- I cover a lot of free learning on IT topics in Completely free IT training resources to help diversify your IT career. I’ve fully updated it as part of this write-up so all the links should still be working.