Dear dashboard obsessors,
I am concerned about your zeal and overall passion for the tools we call dashboards. Let me start by saying that I am not down on dashboards, I make dashboards all the time. I believe dashboards are an integral piece to the process of gaining insights from the data we collect. But I also believe that dashboards are not the only tool necessary for gathering insights from our data. My concern is we do not agree on this point.
As I hear you talk about dashboards, I get the sense that you expect amazing insights to pop out of the two-dimensional screen you sit in front of every day. You might not say it, but you believe that if you just had 10 more metrics in your amazing 15-page dashboard, you might uncover the next transformational insight and get yourself the promotion you have been wanting. You simply love dashboards, but as I will argue, perhaps a little too much.
Now, allow me to add that the undying affection for your dashboards is only one problem at play in this larger discussion of dashboards. There are many reasons why your dashboard practices may not be ideal. In order to discuss these “not-so-ideal” perception about dashboards, let me start by outlining the over-arching process of finding insights.
For the sake of understanding, let me simplify the process of insights discovery. Let’s say there are 10 steps to finding actionable insights in your data. The reason dashboards are helpful is because they can take you a few steps down the road in this journey of finding insights. I think it is fair to estimate that dashboards can get you through step three in this process.
However, not everyone thinks of dashboards in these terms. If you have dashboards and still do not find value in your data, you likely fall into one of these camps:
- “I believe dashboards get me from step 1 to step 10” – These are the dashboard addicts. These people wholeheartedly believe that dashboards are the one-stop solution for finding value in their data. *Side note – seemingly, people also seem to think this about Tableau. They think it can do everything under the sun when it comes to data. It cannot.
- “I don’t know what to do after step 3” – These people know dashboards only get them so far, but they lack an understanding of what comes after dashboards. Therefore, they stay with what they know, hoping that an insight might sneak itself into a dashboard.
- “My dashboard doesn’t even get me past step 1” – These people unfortunately have bad dashboards. They can’t get past step 1 because their dashboard is not designed for moving further into their data. Their dashboards are surface level and do not lead people to ask questions.
If you fall into one of these camps, allow me to talk about each one in further detail, hopefully showing you how you can get back on track with uncovering insights in your data.
I Believe Dashboards get me From Step 1 to Step 1o
If you are in this camp, let me start by making a point of connection. You and I have a shared value when it comes to analytics: speed. If you believe dashboards get you from step 1 to 10, you likely think this because you want to get to the insights as quickly as possible. You know the importance of making data-driven decisions, and you also want to utilize technology to its fullest potential. Therefore, you want to automate those dashboards and use them for the entire process. I like the way you are thinking, but this is simply not a good idea.
In order to show you why, let us walk through an example of arriving at an actionable insight. Let’s say you have a dashboard for your e-commerce website, measuring everything from incoming traffic to conversion rates. You notice that your revenue is 10% lower than normal, so you start to look at different metrics which influence revenue. You manage to find that your conversion rate (the percent of visits which place an order) has dropped by 15%. Now that you know your conversion rates are under-performing, how do you move forward?
First, you identify reasons as to why your conversion rate may be under-performing. You list the following possibilities off the top of your head:
- Your tracking ‘mechanism’ has somehow broke and you are not measuring conversions correctly
- Your marketing campaigns changed and are bringing in lower quality users
- The UX of your website has changed and is somehow making it harder to purchase a product
- There is a disconnect between your marketing and your product offerings
- Your payment funnel is broken, making it impossible for some users to purchase a product
From here you order these possibilities by priority and start to move down the list. You investigate your tracking and determine that everything is working correctly. You pour over your crash data, hoping to find some segment of users who are experiencing technical difficulties. You look at tons of metrics, but find nothing.
You then walk over to your product team and ask if the website has changed at all. They tell you some of the changes that have been made, but all of them seem pretty small. After much discouragement, you look at your marketing campaigns, and discover that even though the number of users who came to the site from Facebook has stayed the same, the number of users who purchased is dramatically lower. You run the numbers, and conversion rate for Facebook is 16% lower than last week.
You look at your campaign data for Facebook and determine your campaigns have slightly decreased.
It appears the phrasing of your title has changed; you look at the landing page for these campaigns and see that nothing has changed, so you wonder what is going on. After looking more at these two sources, you realize the disconnect that users are experiencing. Your title changed from “Check out our Products” to “Check out our Free Products”. You have always had both free and paid products, but the landing page for this campaign is the “all products page”.
When the messaging didn’t say free, people expected to see products, free or paid. When it did say free, well, they expected to see free things. You quickly inform the marketing team and have them change the messaging. You also embark on an A/B testing journey to figure out which phrasing will work best for your Facebook campaigns. Yay!
Ok, now that we have worked through an example, let’s recap. During this fairly simple analysis, you looked at four different data sources (behavior data, crash date, campaign data, website/content data) and probably worked through 60 metrics. You didn’t arrive at the answer right away, and most of the data you looked at was not helpful. But of course, you did not know what data would be helpful before you did the analysis. If you did, you would have the ability to see into the future. And if you have the ability to see into the future, please call me. I have some investing I want to do.
The point to make here is that insights are not something that you can predict. The process of discovering insights takes you on a journey of analyzing lots of data, most of which will not be helpful. If you wanted your dashboard to get you to step 10, it would need to include all of this data. We’re talking about hundreds of metrics, probably making your dashboard 20 pages long. Not only would this dashboard be insanely expensive to maintain (labor costs if it’s not 100% automated), you would be inundated with data and would not be able to do anything. A dashboard with too many metrics is overwhelming, it leads you down too many pathways, therefore not helping you make any long-term progress. A dashboard with all the data you need for analysis is simply not smart.
I Don’t Know What to do After Step 3
Not everyone is extremely passionate about dashboards, some just don’t know how they fit into the process of finding insights. Using our example above, some might see in their dashboard that revenue has gone down. They might be able to find the connection to a drop in conversion rate, and they might even have a graph showing them conversion rate broken down by marketing channel. The people who don’t know what to do after step 3 simply stop there, believing that is enough information for a proper analysis. Unfortunately, this is not the case!
This person will report to their boss and tell them that revenue is down. They will excitedly inform their boss about the cause, saying “our decrease in revenue is due to our Facebook campaigns. Their conversion rate dropped by 16%”. Their boss will ask them the question we are all expecting – “why did our conversion rates drop?”, and of course the analyst will not know why. Dashboards do not (and can not) provide us with all data necessary for insights discovery. We must use dashboards as a springboard for analysis, giving us direction into what data we will investigate.
For people in this camp, I cannot fully explain the analysis process after step 3 in one blog post. However, there is one key you can learn which will drastically help you in your process: segmentation. Every good analyst knows the power of segmentation, because segmentation will almost always lead you to deeper insights. For example, you found that your conversion rate for Facebook dropped, how about segmenting that by device type? Did this only happen for mobile phones? What about by date, did this start on a specific day? And what landing pages? Did all Facebook traffic come to the same page, or different pages? Did one specific page under-perform? Segmentation continually provides you with clues, getting you ever so close to your transformational insight.
If you want to learn more about this, checkout the following resources:
My Dashboard Doesn’t Even Get me Past Step 1
There are unfortunately many people in this camp, and they simply have poorly designed dashboards. Rather than belaboring this point, allow me to provide you a visual what good dashboard design looks like. I will go through the strategy of this dashboard and show how it gets us to step 3 in the insights process.
Main Business Question
Every dashboard should address a business question. If your dashboard does not address a business question, it does not have a goal. And dashboards without goals rarely help you achieve your goals. Put your business question at the top so everyone knows why this dashboard exists.
Every business question should be supported by a KPI (Key Performance Indicator). However, your business should not have more than 3 or 4 KPI’s. In this first section, your goal is to show your KPI with as much context as possible. How did this KPI perform compared to last week? Last month? Last year? Where is this KPI projected to go? Don’t overdue it, but context is key when attempting to move from step 1 to 2.
Your levers are what causes your KPI to go up or down. As the example shoes, average order value is a lever for revenue. Levers are what take you from step 1 to step 2, because they inform your thinking about your KPI. Is your KPI down? Which lever is down? Is your KPI up? Which lever is up? Your levers are the next logical step in the analysis process, which is why they are right below your main KPI.
Your supporting metrics are the levers of your levers, meaning they influence what your levers do. This is likely going to be your section with the most metrics, but its important to note that these are least important. Quite often we obsess over increasing visits to our website, but if we don’t make money off visits, we probably should focus on something else. You also don’t need to provide as much context around these metrics, simply because you have less space. It’s still a good idea to include at least 1 data point of context, for us this is change from last week.
If this dashboard is built correctly, you will have three levels of metrics which inform your business question. Each level is meant to inform the previous level, allowing the user to do basic analysis. “My KPI went down, why did it go down?” – see your levers section. “My second lever went down, why did this go down?” – see your supporting metrics. And by the time you have identified the supporting metrics which caused your KPI to decrease, you are ready to launch into a deeper analysis and move onto step 4.
Dashboards are a great tool, but they will not do our jobs for us. We still have to critically think about our business problems and use the tools within their intended context. Once we develop a tool kit wide enough to go through steps 1 through 10, then we can start to really add value through analytics.