Metrics & Measurement

Metrics & Measurement — 1 Comment
25
Jun 12

Fun graph: New data shows true screen sizes

EDF got nice attention lately for our apparently ground-breaking use of responsive design. Ironically, that came just as we started to think about screen resolution in a new way.

Jakob Nielsen recently wrote about screens getting bigger over time (right). Useful data, but it only goes so far. Just because we can view Twitter feeds at 1920×1600, do we?

It turns out that we might not.

We don’t always keep our browser window at the maximum size. We change it as we click around. And some of us use toolbars, which shrink the actual space available to see sites. With all the variation, how are people are really seeing our sites?

I suspect the reason that Nielsen didn’t address this question is that it’s hard to get the data, particularly historically. Out of the box, Google Analytics tells you only the physical size of the screens your visitors have. When I asked EDF’s super-developer, Patrick Cranston, for help getting beyond this limitation, he turned up this method for capturing actual dimensions. Now, we can measure the true size of viewing area.

Our awesome analyst Mathew Grimm sliced some of the early data, and the results are fascinating (click for a larger version).

The larger the circle, the more visitors saw the site through that size. See below for a note on the red square.

The biggest takeaway — it would be almost impossible to design one site that works well at this huge variety of sizes. Responsive design looks like an increasingly important solution. Other observations:

  • The old idea of one “fold” that applies to everyone is dead — while people do see the site at (somewhat) dominant widths, that’s not at all true of heights. Even people who see the site at the same width see it at a wide variety of heights.
  • A surprising  number of people see less than 800 pixels of height. And our home page is the only page I know of that responds to height as well as width…
  • The dense cluster marked by the red square is an interesting challenge. The difference between 900×512 and 1280×896 is significant from a design point of view, and there aren’t clear dividing points in that mess. People see the site every which way in between.

There’s a lot more to explore here. For example, a substantial number of visitors re-size their browsers during their visit to our site, and we want to look into when and in what direction.

I’m looking forward the next generation of EDF’s responsive design, informed by this new generation of screen size data.

Metrics & Measurement — No Comments
23
Mar 11

Key Takeaways: 2011 NTC Segmenting Session

This session was called “A Scientist in Your Communications Department,” and  Jeff Shuck, the presenter, had a ton of energy and insights to share. (It looks like the whole video is posted so you can see for yourself!)

Sally Heaven over at Convio posted her takeaways from this session, too. All the points she listed were ones I had written down, too, but I had somewhat different top takeaways: Continue reading →

Management / Metrics & Measurement — No Comments
21
Mar 11

Key Takeaways: NTC Session “What Does the Data Say?”

I’m going to highlight some of the insights I starred in my notes at last weekend’s conference here in D.C.  First up is a session called “What Does the Data Say? Converting Analytics to Action,” moderated by George Weiner of DoSomething.org.

The underlying premise is that we should use data to drive decisions within our organizations. With that in mind, here are my takeaways: Continue reading →

Metrics & Measurement — 1 Comment
13
Jul 10

Analytics Exchange: Great for Both Nonprofits and Analysts

I just heard about the Analytics Exchange on an NTEN webinar by Eric Peterson, and it sounds like a solid deal all around. Here’s the deal:

  • An organization submits a project — a straightforward question to be answered through web analytics.
  • A “student” — anyone looking to improve their analytics skills — performs the analysis.
  • A mentor coordinates the project and offers advice.

I was surprised to hear Eric say the biggest limiting factor is the number of projects — they need more! It seems that organizations have more to gain from this than anyone else, so why isn’t there a mile-long wait list? Submit a project already!

The materials are super-clear and set good expectations about how the process works. I’m signing up and encouraging other folks at EDF to do so, too.

If anyone has participated in this, I’d love to hear about your experience.

Metrics & Measurement / Video — 6 Comments
15
Jun 10

Beyond YouTube Views: How Do You Measure Success?

A year or two ago, the online team at EDF focused on getting people comfortable with the idea of using video. We urged staff to consider the option and answered basic questions. Now, we have 175 videos on our YouTube channel, created by staff from La Paz, Mexico, to Boston. And we’re picking up steam — we have four more in the queue this week.

With all this video to work with, we’re increasingly looking at how to gauge whether videos are successful. In a conversation about another organization’s nicely done video series, a colleague commented:

…when you go to their YouTube channel, you’ll see that hardly any of their videos have more than a couple of hundred views. So, they are putting a lot of effort into something that doesn’t have a great promotion plan in place.

That’s an astute observation, but hidden in it is the thought that views are what define a video’s success. Is that really the best or only option?

I’ve been asking around, and consensus is that people are still figuring it out — and that it’s important to figure out. A counterpart at another large environmental organization suspects the lack of goals for video contributes to their lackluster results. It’s hard for them to get staff time to work on videos, and the videos they do produce tend to be fueled by enthusiasm rather than a clear communications purpose.

I also got suggestions for specific metrics, like this one from Jordan Gantz:

While we still haven’t figured out an effective way to know across the board if the videos are achieving the expected goals, one thing that has been interesting is looking into how long people are spending watching each video and at what point they leave.

When I asked Wendy Harman how the Red Cross, which produces huge amounts of video, grapples with this, she mentioned a time they knew what the metric was but that still didn’t help. They got great attention to a video they made right after the Haiti earthquake — so much attention, in fact, that the tracking mechanisms didn’t hold up to the traffic, so they couldn’t be sure exactly how much money it helped them raise.

Michael Hoffman from see3.net best said the view that I’ve been coming around to:

Generally, we think video should be measured in similar ways that you measure other content investments and to connect the ROI of video to broader organizational goals. Views are fine, just as website visitors are fine, but it only gets you a sense of the total amount of engagement.  (If, for example, you make a video designed to influence a small  group of elites (lawmakers, corporate decision makers, etc.) then views doesn’t seem so important at all. The metrics should reflect the purpose, like in any communications effort.)

He’s absolutely right — we don’t measure all our blog posts or web pages or actions by the same standards, so it follows that the same should be true of videos.

I’m looking forward to the challenge of figuring out the best ways to convert goals into metrics, so that for each video, we can answer the tough questions about what worked and what didn’t.

Have anything to add about how you do or don’t measure video? Please leave a comment!