The Two Things about Website Measurement

“Information is all around us, but so is what Richard Saul Wurman, an architect who, along with Tufte, shaped the field of information design in the 1970s and ’80s, calls ‘non-information,’ the flotsam and noise that are the by-products of a hyperactively quantitative culture.”

– Joshua Yaffa, in The Information Sage, #longreads piece in The Washington Monthly on information design guru Edward Tufte

Last month, I keyboard-mashed a quick brain-dump of things I wish I could tell everyone about analytics, which was pretty much a reaction to a couple of troubling trends I’ve noticed within the realm of website measurement. First and foremost, troubling-trend #1 is the notion that data are meaningful in and of themselves. No. Wrong. I’ll get to that in a moment. Troubling-trend #2 is the generally quantitative focus that really should be qualitative. That’s why I wanted to start out with quoting Joshua Yaffa’s piece on Edward Tufte—his (and Tufte’s) indictment of us is, I believe, spot on. We are a hyperactively quantitative culture. This is not good!

The Two Things About Measurement

So, in my introduction to the Two Things meme, I promised a forthcoming “two-things” post on measurement. Here it is. The two things about website measurement are:

1. Proper measurement is slavery infinitely cyclical.

2. There are no independently meaningful metrics.

Proper measurement is Slavery Infinitely Cyclical

When was the last time you went to your doctor for a physical exam? Over a year after moving to North Carolina, I found a new doctor and scheduled a physical. I was somewhat nervous, thinking that my new doctor would chide me for waiting so long. Instead, I was shocked when he concluded my visit with, “see you in a few years.” I had planned on scheduling one for a year later—I assumed it would be wise to lock-in to some kind of schedule. My doctor assured me that I was healthy and nothing really serious is likely to come up until I got closer to forty. I was 27. Doesn’t that just sound like bad advice to you? Don’t worry, nothing really serious is likely to come up for a long time. So is the smart thing to do to ignore watching something until it’s more statistically vulnerable? Of course not. I bet you’re thinking that I should have found a new doctor. You’re right. I should have, but I didn’t.

So what does this have to do with measurement? Well, I think that the story I just told about getting a physical bears a striking resemblance to what I have observed of website measurement behavior. In fact, there are three particular similarities between a typical approach to measurement and a typical approach to healthcare:

1. It’s outsourced (aka, someone else’s problem). As a result…

2. It’s a PITA (I’m sure you can figure out what that stands for). Which, in turn, results in…

3. It’s not as routine as it should be.

While it is appropriate to outsource our healthcare to qualified professionals, it is not appropriate to outsource our health to anyone. Your health is your problem. Health is the result of a lifestyle of well-being, not a one-time fix. When we consider it a one-time fix, it will definitely become a big pain-in-the-____. I think it was Michael Pollan who said something along the lines of infirmity is more expensive than well-being. That is a solid principle. So, applied to the health of your website, this means that well-being requires routine observation and measurement, not one-time diagnostics (usually brought about by some kind of emergency—infirmity) outsourced to experts.

I try to live a lifestyle of well-being. I won’t say it’s easy. There are days when I am lacing up my gym shoes and thinking to myself, “Really? I’m doing this again?” Yes, in those moments it feels like slavery—or if I allow myself an even more dramatic comparison, sisyphean tragedy. Afterward, with endorphins rushing through my body, I’m glad I stuck to my routine, and from a more long-term perspective, I certainly wouldn’t want to trade the hours I spend exercising for equivalent time in a sickbed. Believe it or not, the same feelings creep up when I pull up Google Analytics, the dashboard of our CMS, and several other data sources I refer to every day. But they, too, are assuaged by the long-term success of our site, which comes from an ongoing cycle of daily measurement, discovering strengths and weaknesses, making changes, reframing how we measure, and on and on…

There are No Independently Meaningful Metrics

Let’s stick with the doctor’s office for a moment. Imagine if my “physical” had amounted to a quick hop on the scale? Again, you’d probably suggest I see a different doctor. But that’s pretty close to the approach we take by default with website measurement. We look at basic stats, and we look at the same ones over and over again. And generally, we’re looking for more of something—more visits, more pageviews, more conversions. I imagine that if I kept returning a bigger number with each visit, even my hypothetically lazy doctor would eventually get concerned about me. So more isn’t enough.

We need to be asking questions that require us to look at more than one metric. In fact, any metric really should be qualified by another one: How many visits did your website get this week? But, how many of them converted? How many views did this page on your website get today? But, how many of them bounced? You see, every stat is clarified by another one. Statistics (*plural) describe an ecosystem of behaviors.

A year ago, I wrote our monthly newsletter on how to use advanced measurement techniques to tell the real story of your website. I still recommend it—the principles haven’t changed since then. Ultimately, I tried to provide a process that began with breaking the outsourcing instinct—in this case, taking data out of Google Analytics in order to create your own unique report that folds in data from other sources. (That alone will help you to formulate your own point of view toward what’s happening on your site.) Then, you can get back into the stats and answer the questions you have—particularly those related to clarifying the bounce and conversion activity on your website.