Wednesday, June 21, 2006

The best metrics

I've been continuing to conduct surveys to learn how our communication efforts are going over. (Typically, I've done a couple where I've not presented the results. Not too busy to do the research; just too busy to share. Hope that makes all you measurement procrastinators feel better.)

But I've had a couple of recent experiences that illustrate the power of experience and instinct. I want to share one in particular. It's a reminder that research only tells part of the story.

We hold our town hall meetings in a space designed for them. A stage with a podium, three screens for slides or video, a sound system and control booth -- the works. Because it was built to accommodate larger groups, our team tends to feel a bit sparse there. However, the town halls earn good marks on surveys, by and large.

Recently, however, we had a major organizational change. Our team and several others met in the "big room" for the key announcement. We then immediately held a meeting for our team only, crowding 50 people into a conference room. Some sat around the table while the rest stood leaning against the walls while a few peeked in the doorway.

This second meeting was a revelation. It was a dialogue, with terrific give-and-take between the team and leadership, much humor and a general feeling of camaraderie. The informality of the setting -- the lack of distance between "presenters" and "audience" -- freed everyone.

So my next group meeting for this smaller team will be in a big conference room, with chairs at the table and more around the walls, and people crowding in, and everyone -- boss, admin, staff -- at the same level. I'll let you know how it goes.

OK, back to surveys. I've been conducting a lot of them lately. A post-Town Hall survey; a survey to measure the popularity of a newsletter I launched to support a short-term project; and another as a kind of focus group to see if we can predict the interest in a contest we're designing for employees.

The first one is part of my job -- how did people react to the meeting, the presenters, the boss and the message delivered? I like these because after you've done a few you have a nice baseline and you can tell when interest and support is soft and when people are truly engaged.

The newsletter survey was pure self-promotion. Yes, I needed to do it as part of good, disciplined communication. But the newsletter was a smash hit and I already knew that from comments, subscription requests and other indicators. But now I have numbers, charts and quotes that will still be there when my performance review rolls around. I put together a PowerPoint deck and sent it off to my boss just to make sure that the good news arrived. Also, the newsletter had one detractor, someone who isn't a big fan of transparency and has a good sense of the power of tightly controlled information. (Which the newsletter took from him.) His responses to the survey stick out like a sore thumb. He gets to make his points, but the numbers are with me.

The third survey is a new type for me. I had to invent a bunch of new questions, redesign them when they didn't work and even toss a few out. It's always eye-opening to preview a survey you've just slaved over and realize how misleading, muddy, confusing and wrong headed it is. So if you don't already, test them yourself and send them to some friends who won't go telling everyone how lousy your drafts can be.