Here we go again

Its the week before VMworld 2018.  The time when everyone is finalizing their plans for what should be another great week of innovation, learning, networking and reconnecting with friends, old and new. This will be my 7th consecutive year at VMworld and I must say it never gets old. I always look forward to the opportunity to be inspired by all the awesome companies out there bringing their best to this event. VMworld is the place to catch up with old friends from around the world as well as meet new friends.  It never fails though, every year I leave having missed out on something, whether a specific session or meeting with someone I’ve been interacting with on social media. This year I want to be proactive and share my activities for the week.  If you plan to attend VMworld and want to meet up, here are a few ways we can make that happen.

The Virtually Speaking Podcast

John and I were thrilled to be invited back to the HCI Zone to record podcast interviews and meet and greet listeners. If you listen to the Virtually Speaking Podcast we hope you’ll make time to come say to us.  We love chatting with listeners and hearing feedback. If you do come, you won’t leave empty-handed.  We’ll be sure to shower you with vSpeaking swag in the form of stickers and fancy t-shirts.  I know I know, just what every VMworld attendee needs…. more swag to take home. 🙂 but wait… there’s more. As we rapidly approach the 250K download mark, John and I thought it would be a good time to show some true vSpeaking appreciation.  So IF you are a subscriber to the podcast, and IF you are attending VMworld 2018, and IF you listen to a special episode that will air during VMworld, you will find out how to win a very nice prize.  I won’t spill the beans just yet but trust me, we’re not talking about a fidget spinner. So stay tuned and be sure to come and say hi at the HCI Zone in the solutions Exchange.


Podcast Interviews

If you’re interested in joining us on the podcast this week be sure to follow us on twitter and send us a message @virtspeaking.


I will present four sessions at VMworld that I am truly passionate about. I hope to see you at each of these.

  • HCI2810QU Virtual Volumes Technical Deep Dive: VVols is a technology that is near and dear to my heart.  It radically changes the way storage is managed in the data center from a storage-centric approach to an application-centric approach.  In this session, Patrick Dirks and I will walk you through why VVols is simpler, smarter, and faster than the traditional approach. We’ll also cover what you need to know to have a smooth migration experience.
  • HCI1768BU – What’s New in vSAN: This is a session you won’t want to miss.   There is a new version of vSAN about to be announced and my fellow podcast co-host and dear friend John have the privilege of sharing all the details with you. If you’ve ever listened to the Virtually Speaking Podcast you know that John is going to dive deep on this one.  You will leave fully informed and inspired. Extra credit for listeners of the podcast that heckle John with show quotes. Hope to see you there.
  • HCI1272BU – vSAN Data Placement and Availability: This is a session that I absolutely love presenting. Jeff Hunter and I will focus specifically on how VMware vSAN distributes data across drives and hosts to ensure availability. You will get a quick explanation of how vSAN stores virtual machine objects such as virtual disks, as well as a look at storage policy-based management, which is the primary method for managing vSAN objects. Whether you are new to HCI or already a vSAN customer this is a session not to be missed.
  • HCI1769BU – We Got You Covered: Top Operational Tips from vSAN Support Insight:  Long title but another really interesting session. VMware vSAN customers enrolled in the VMware Customer Experience Improvement Program (CEIP) can utilize their deployment data with vSAN Support Insight. vSAN Support Insight extends the vSAN health checks and enables support and engineering visibility into your cluster. Learn what the most common health checks are, how to resolve issues, and make sure that your clusters are ready for anything! John and I will again be co-presenting this session so all podcast heckling is welcome and possibly even rewarded (hint)


I probably won’t be attending many parties this year as my schedule is pretty full with customer meetings, podcast work, and sessions, but one party I will definitely be attending is the vSAN Customer Appreciation party on Monday evening. Hope to see you there.


Customer POCs



A Proof of Concept (or POC) project is a common approach used by companies to assess the viability of a software product for solving a particular business need. So what does it take to conduct a successful POC? This week on the Virtually Speaking Podcast we bring in Dave Morera and Sudhir Balasubramanian to walk us through this process and share details of a recent POC using Oracle RAC on vSAN.  READ MORE

Its Career Day



Trying to explain what I do for a living is always fun. If I say I work in IT the response usually is “Oh can you fix my computer”? If the person is more tech savvy and knows what VMware is, I usually get the response “Oh so you’re a programmer” News Flash: of the 20,000+ employees at VMware we are not all “programmers” Its actually quite surprising how many different roles there are behind each software product regardless of the company. Read more.

episode70-1Virtual Volumes (VVols) was designed to solve a lot of the pain points VMware and/or storage admins face today. This week on the Virtually Speaking Podcast we bring in one of the industry’s most authoritative VVol subject matter experts Cody Hosterman from Pure Storage.  Read more.


Recently I guest hosted a great post by Glenn Sizemore entitled vSAN from the eyes of a Storage Admin.  This post got so much visibility that I thought we should bring Glenn in to the podcast and expound on these ideas. Read more


vSAN is VMware’s software-defined storage solution, built from the ground up for vSphere virtual machines. It abstracts and aggregates locally attached disks in a vSphere cluster to create a storage solution that can be provisioned and managed from vCenter and the vSphere Web Client.

This week on the Virtually Speaking podcast we are pleased to welcome VMware CTO of the Storage and Availability Business Unit, Christos Karamanolis. Read more

Glenn Sizemore recently joined the vSAN team coming from one of the big blue storage vendors. His first hand knowledge of traditional storage and now also of vSAN gives him an interesting perspective. Check out Glenn’s most recent post as he discusses vSAN from the eyes of a Storage Administrator.


Today I would like to discuss vSAN from a slightly different perspective, that of the dedicated storage administrator.  In my experience working with storage admins, I sometimes will come across a group who in my opinion are overly dismissive of HCI solutions.  The subtle subtext being that HCI solutions are not capable of handling “real work.”  Having transitioned from a significant storage vendor to the Storage and Availability team here at VMware, I was pleasantly delighted to confirm that those preconceived notions are rooted in pride and tradition not technical distinction.  I don’t begrudge anyone who may share this out of date perspective, but I would like to confront several of the talking points often used within the industry when discussing HCI solutions and more specifically address how several of these criticisms have been mistakenly levied against vSAN.

HCI is fine for dev/test data, but it’s not reliable enough for production data.

Nope, this one is just FUD. Aimed at making the consumer overly cautious because as we all know losing data in this business is a resume generating event.   The facts are that vSAN had a different starting point when it comes to availability as it utilizes RAIN in place of the more traditional RAID.  While not genuinely unique the shared-nothing architecture of vSAN does enable an enticing set of powerful capabilities.  These capabilities are a direct result of the architectural choice that went into vSAN.

Fundamentally there are only two ways to make a piece of data highly available.  Either provide multiple redundant paths to a shared media source or store multiple copies of the data.   vSAN being a shared nothing architecture necessitates storing multiple copies of the data; however, it isn’t nearly as pedestrian as one may think.  You see, vSAN is an object store, not a distributed file system.  This means that availability and durability are implemented at a per-object (component) level and can be adjusted as the realities of a deployment change.

Speaking of durability; it’s worth calling out that vSAN implements all the bit-rot detection and self-healing sub-systems one would find in a dedicated storage OS.  At the end of the day as long as the failures to tolerate (FTT) policy settings are set higher than the number of nodes lost the data is still available.  The real maturation process that vSAN has undergone over the past three years and six releases has been a refinement in the underlying systems which oversee this process. This helps ensure administrators don’t put themselves in an exposed position unknowingly.

HCI is fine so long as you don’t need to scale.

This will sometimes be more directly stated as; vSAN is fine but doesn’t scale. This particular claim is a pet peeve of mine because it’s rarely an actual concern.   When we are honest, the question we ask is not will it scale, but instead if it scales sufficiently to solve the problem at hand.  To answer that question let’s take a brief look at the current scale points of a vSAN cluster.  The vSAN 6.6 release currently supports up to 5 disk groups per host.  Each disk group will contain a cache and up to 7 capacity drives.  Since only the capacity drives contribute to usable capacity that gives us a maximum of 35 capacity drives per vSAN node. Finally, we can combine up to 64 hosts into a single cluster.

Assuming 1.9TB flash drives that would give us roughly ~4PB of raw capacity in my theoretical cluster.    Usable capacity is dependent on the policies applied to the component objects being stored.  Therefore, I’d like to set that aside for a moment and instead continue to explore the scalability of vSAN.   The final point of which is that as of the date of this post, up to 2000 vSphere hosts can potentially be placed into a single management domain for a combined total of ~125PB of raw capacity.  That is a LOT of flash storage, and to be honest, there are other restrictions which kick in and limit a deployment before the 125PB maximum in my example. If additional capacity were needed, we can easily use a larger drives.  Suffice it to say I believe we can safely move past the capacity argument, while not limitless vSAN 6.6 is already able to support more hardware then all but a handful of deployments can afford/justify.

What about performance?

It depends and it’s not productive to try and address performance any further than that in this post as its just too involved.  Again though, there are sufficient proof points that vSAN is more than capable when it comes to performance, and can exceed the actual required performance of all but the most demanding customer workloads.

Object count?!

Again, this is a simple misunderstanding. As of vSAN 6.0, every node added to a cluster can support 9000 component objects.  A single component can be up to 255GB.  Therefore, a full 64-node All-Flash vSAN cluster would support 576,000 component objects.  Mind you this is merely the current upper limit.   Assuming all components were consumed in support of VMDK objects, there is ~140PB of addressable space in a VSAN cluster to spread across 6000 VMs.

So to circle back around to the beginning of the conversation just how much “scale” does one need out of a single vSphere cluster?  I wouldn’t be comfortable with having that many eggs in a single fault domain, regardless of the storage technology in use.

Space Efficiency then!  HCI solutions can’t compete with dedicated storage operating systems when it comes to space efficiency technologies.

You may have noticed how we’ve moved past questions and into accusations. That’s because that’s how these conversations, unfortunately, tend to go.  Never the less it’s worth addressing.

vSAN 6.2 introduced Erasure-Coding as a failure toleration mechanism.   This was a game changer as it enabled vSAN customers to realize the media efficiency of a RAID deployment with the flexibility and composable nature of HCI. It is a very compelling meet in the middle approach which allows customers to optimize their capacity pool as they deem optimal.  Customers who value raw performance can utilize traditional full copy based RAID-1 striping.

Heck, they can even sacrifice additional capacity to further strip a component across a RAID-0 under the RAID-1 granting additional queues and media to a given workload.

When capacity utilization is a priority, the customer can opt to implement a RAID-5/RAID-6 EC deployment allowing up to 50% raw media utilization with minimal capacity lost to parity.  The ability to configure how the data will be protected on a per object basis while the standard for an object storage system is unheard of in storage arrays.  Traditionally, storage arrays would build the parity mechanism into the media pool themselves and support multiple different availability targets would require separate media pools.

Oh yeah, and vSAN 6.2 also added support for Deduplication and Compression on a per-disk group level.  After spending way too much time arguing over who’s compression and dedupe works the best…  I would just like to summarize this by saying your results will vary but not that much from vendor to vendor.   Not all workloads are compatible, which is why the erasure coding RAID implementations are far more significant in my opinion.  However, if your workloads recieves a benefit from dedup and or compression, then you will see those benefits reflected in vSAN for a surprisingly minimal performance impact.

This post carried on a little longer then I had intended if you made it this far I would like to thank you for your time. If you disagree with any of the points made, please reach out and let me know.  I’m open to changing my mind.  I have come by my current opinion only after building and selling CI and HCI solutions for customers of all sizes for the past several years. While doing so, I watched vSAN from the outside as the engineering teams iterated through the problem space.  Now on the inside with a full view of the sausage factory, I’ve concluded that while vSAN is a new way of solving the VM storage problem. There are some very compelling capabilities in addition to the undeniable advantages around administration ease of use.  If you’re on the fence and unsure how vSAN satisfies a particular concern addressed by your current solution, please drop me a line. I’d love to continue this conversation.

After this post went live we invited Glenn on the Virtually Speaking Podcast to add some color to these points. Enjoy!