Cloud Cruiser Blog

May 2, 2017

Cloud Metrics That Matter: The Challenges

Filed Under:

By Deirdre Mahon  |  CMO and Head of Customer Success  |  @dbmahon


I don’t think I’ll get any argument around the merits of measuring your cloud usage and spend.  Measuring after the fact when it’s too late to take action is a problem I regularly hear about in the industry. Organizations struggle to get ahead of any problems and so keeping a close eye on key performance indicators (KPIs) to measure the performance of individuals, teams, and the products and services you deliver is critical. Getting to the point of optimized services so your cloud spending is aligned with organizational goals and budgets is what most strive for. Perhaps your KPIs are mere indicators of performance – “nice to knows” with pretty visuals that you share in the hope that someone someday will make real decisions driven by reliable intelligence.  I think we all know what the holy grail of business intelligence (BI) feels like, but getting there is really hard.


Put more succinctly, are your KPIs actual performance indicators for the business or does the “P” stand for possible indicator?  Agreeing on which metrics to track is probably the hardest thing to pin down.  Hurdle #1 is the fact that everyone comes with a personal agenda, focused on what will ease the pain of getting the job done, such as “what is possible without having to code our way around it?” or perhaps “how do we massage a plethora of numbers from disparate sources and systems to get close to the truth?” Getting from the point of “I have no idea what is going on with cloud usage and spend” to “I am so on top of this and have reliable data at my fingertips; I feel totally in control and empowered…” are very different states of being inside most orgs today.


It’s Complicated


Beyond the fact that most companies use multiple cloud services across multiple business groups, these services also come from multiple providers.  That’s a Rubik’s Cube or what I refer to as a mind-bender.  Add thousands of services across hundreds of business entities and your eyes start to blur. As stated in RightScale’s 2017 State of the Cloud Survey, ‘Public cloud users are already running applications in an average of 1.8 public clouds while experimenting with 1.8 more.” Pretty standard is AWS and Azure and in some cases, organizations dabble in Google Cloud but it’s certainly not mainstream.


With two different providers alone, confusion arises in what fields are being used to track usage and rates.  In the case of Azure, usage is by Account, by Subscription and then by Department and Resource Group.  For AWS, it is simpler and is largely by Account.  AWS also employs more tagging for resource usage tracking compared to Azure.  Understanding the field structures, naming, and hierarchies is key (excuse the pun) to figuring out what set of reports to extract so you can start to measure real business activity.


Getting on the Same Page


Normalizing data and fields across disparate schemas is a critical part of getting to a single report or KPI across multiple service types and that often requires a more detailed explanation.  It’s not always intuitive to the naked eye.  Sometimes you just need to go to the sausage factory once in order to see how it’s really made before taking a bite out of the final juicy report.  Well you get the picture.  (Apologies to vegetarian or vegan readers for this meaty metaphor).


If you gather the right business and technology teams around the table to agree on the right set of KPI’s, you often find that disparity abounds. For example, finance cares more about cloud spend by business group or entity and by various timeframes for budgets and forecasts.  By contrast, business owners or stakeholders care more about units of cloud services and types and, whilst they often have an eye on the money, they need deeper knowledge on cloud service type by group or project.


Finance may not know the difference between cloud providers, much less cloud service types, yet they need to be able to align cloud spend with the appropriate business units and measure performance against budget.  Likewise, seeing certain services left running at peak over a weekend or during business downtime might be a red-flag to a business stakeholder but mean nothing to a finance person signing off on spend that increased drastically over the past few months. These examples highlight the need to normalize data across various service providers and infuse it with the proper business context so it’s relevant and actionable by decision makers across finance, IT, and business stakeholders.


Stay tuned for the second blog in the Cloud Metrics That Matter series and view our on-demand webcast on the topic.