Thursday, August 18, 2005

Celebrity Endorsements

Last time I wrote about plans to check read receipts from e-mails sent from leaders of the firm. I was expecting a boost in opened e-mails from those leaders, compared to those sent from our "generic" mailbox. I was surprised by the size of the leap. Here's the data from an e-mail from the North American leader to all North American employees. (By the way, does everyone know that you can click on the tables to open up a large version?)



It pays to have the right messenger -- these numbers are nearly twice as good as the average. Here's a comparison between this message and all the other e-mails for which I've gathered read receipt data.


(Not sure why this is one won't open to a large version. Working on it.)

Now, there are a lot of differences among these messages aside from the messenger. They were sent to different audiences and the content was different. It's likely that the senior leader message was simply more important and interesting than the others.

However, I'd argue that the differences among the lower four messages reinforce the similarity of readership. The four lower messages all received essentially identical attention despite variations in audience and content. The North American leader message response is starkly different.

I'll do another check of messages from the leader mailbox when I have a chance. I hope to get a less compelling message to check -- something that is less interesting in itself, to see how people respond.

So, what good is this information? Here are a few thoughts:

  • It helps manage expectations on message penetration.
  • If you can isolate readership levels you can do a better job of judging what other factors are effective in your messages. Let's say I send two different messages, each asking employees to take a survey. I know from read receipts that 40 percent of employees read each of them, but one message drove 30 percent of employees to comply and the other just 20 percent. I can explore the message content, timing or other factors to see what drove the higher compliance.
  • I was asked how many dial-in ports we might need if we asked all North American employees to attend a conference call with the North American leader. If I'd had only the earlier read receipt data I may have guessed that enough ports for half the employees would be plenty, since fewer than half of most e-mails are even opened. Because I had seen that more than 75 percent of North American employees opened the leader's most recent message, I increased my estimate.
  • It sets a baseline and a goal for all messages. If I know I can reach 75 or 80 percent of employees given the right message and the right messenger, then I can be a more strategic messenger.
  • The knowledge also creates a greater responsibility. Maybe I can reach all those people, but do I want to? Are all messages equal? Would my organization be more or less effective if I learned to get everyone to open and read every e-mail message sent? My role is also to protect my audience, so I think this data would drive better targeting of messages.
  • At the very least, I now have a sense of what constitutes good and average message penetration -- and I didn't really know that before.

I've recently had HR give me a list of all employees by e-mail address, level, location and other data. I can now map my read receipts to this data to discover different response rates based on those factors. Are they more or less likely to read it in Europe? Do managers read more messages than vice presidents? As I've said before, a little data goes a long, long way, and you never know how handy it will be until you have it.

My next post will provide the remaining data from my post-conference session survey. Then, I'll be helping my father-in-law conduct a survey for a small social-and-sports club he belongs to, using surveymonkey.com. That will probably be worth a few laughs here at standonabox.

After that, I'm not sure -- I'm moving to a new job at a different firm in September. I expect it to have a large measurement component, which you will be able to read about here. Wish me luck!

Thursday, August 11, 2005

The Devil's in the details

I'm jumping back to read receipts this time, to provide some new information... and a warning.

I was waiting eagerly to get some read receipt data from two messages we were planning. One from our North America leader and one from our global leader. These would provide contrast to our other e-mail messages, which have been sent from our generic corporate mailbox.

Unfortunately, I failed to anticipate a simple change in plans. Someone else did me the favor of sending out the global leader message and neglected to request read receipts. Since those kinds of chances don't come along every day, it's disappointing.

It also shows the value of not trying to do these kinds of studies on your own -- form a team of interested parties and include everyone who might have a role to play in the process.

Anyway, the North America leader message receipts were very interesting. Next time...

Wednesday, August 03, 2005

Aw, shucks.

In which we take a break from read receipts...

This blog grew out of a Ragan Conference session on metrics in communications. In that session I extolled the beauty of www.surveymonkey.com, a great site for creating and analyzing surveys. I asked attendees if they would take a post-session survey and gathered their e-mail addresses for that purpose.

(Here I must again cop to the fact that I managed to lose all those e-mail addresses, except the ones given to me on business cards. Let me publically state for the record, again, that I'm a dope. If any of you are out there, my abject apologies.)

Anyway, I ended up with only nine respondents to my survey, which I conducted using www.surveymonkey.com. It is extremely easy to use -- I was able to put together a fairly sophisticated survey on my first visit. Basic features are free, and advanced features are an extremely reasonable $20/month. I've even used it for fun, silly surveys to amuse friends and family.

I promised to share the post-session results when they were in. I've hesitated because, frankly, they are embarassingly positive and I am -- deep, deep, deep, deep, deep, deep, deep, deep down -- the modest type. I guess it's my small-town upbringing.

But you may be interested in the survey questions, and I'd be interested in your thoughts on them. So in this post, I'll cover the demographic questions, and what my nine new best friends answered. (The numbers after the questions are the percent and raw number responding.) My next post will cover the questions and responses about the session itself.

In the demographic section (enticingly titled "About You") I asked these questions:

1. How would rate your experience with using metrics in your job before attending my session?
  • No experience - I'd never done any real measurement (33.3% - 3)
  • Beginner - I'd taken some small steps toward measuring my activities (55.6% - 5)
  • Intermediate - I conduct regular measurement (0% - 0)
  • Advanced - I've done considerable measurement and it's a regular and valuable part of my activities (11.1% - 1)

2. How long have you worked in communications?

  • One to three years (33.3% - 3)
  • Four to seven years (22.2% - 2)
  • Eight to 10 years (11.1% - 1)
  • More than 10 years (33.3% - 3)

3. What do you consider your core skills as a communicator? Please rate the following choices.

  • Writing
    1 No experience or proficiency 0% (0)
    2 Beginner: I don't do it well and largely rely on others for this skill 0% (0)
    3 Competent: I'm OK -- maybe not the best around 0% (0)
    4 Proficient: Colleagues come to me for this skill 78% (7)
    5 Expert: I'm among the best around 22% (2)
    Response Average 4.22
  • Communications strategy
    1 No experience or proficiency 0% (0)
    2 Beginner: I don't do it well and largely rely on others for this skill 0% (0)
    3 Competent: I'm OK -- maybe not the best around 33% (3)
    4 Proficient: Colleagues come to me for this skill 56% (5)
    5 Expert: I'm among the best around 11% (1)
    Response Average 3.78
  • Project management/organization
    1 No experience or proficiency 0% (0)
    2 Beginner: I don't do it well and largely rely on others for this skill 0% (0)
    3 Competent: I'm OK -- maybe not the best around 33% (3)
    4 Proficient: Colleagues come to me for this skill 56% (5)
    5 Expert: I'm among the best around 11% (1)
    Response Average 3.78
  • Relationship building
    1 No experience or proficiency 0% (0)
    2 Beginner: I don't do it well and largely rely on others for this skill 0% (0)
    3 Competent: I'm OK -- maybe not the best around 44% (4)
    4 Proficient: Colleagues come to me for this skill 56% (5)
    5 Expert: I'm among the best around 0% (0)
    Response Average 3.56
  • Analytics, including metrics
    1 No experience or proficiency 33% (3)
    2 Beginner: I don't do it well and largely rely on others for this skill 33% (3)
    3 Competent: I'm OK -- maybe not the best around 22% (2)
    4 Proficient: Colleagues come to me for this skill 11% (1)
    5 Expert: I'm among the best around 11% (1)
    Response Average 2.11
  • Business knowledge - general/finance
    1 No experience or proficiency 0% (0)
    2 Beginner: I don't do it well and largely rely on others for this skill 22% (2)
    3 Competent: I'm OK -- maybe not the best around 56% (5)
    4 Proficient: Colleagues come to me for this skill 11% (1)
    5 Expert: I'm among the best around 11% (1)
    Response Average 3.11
  • Employee/internal communications
    1 No experience or proficiency 0% (0)
    2 Beginner: I don't do it well and largely rely on others for this skill 0% (0)
    3 Competent: I'm OK -- maybe not the best around 22% (2)
    4 Proficient: Colleagues come to me for this skill 67% (6)
    5 Expert: I'm among the best around 11% (1)
    Response Average 3.89
  • Public relations
    1 No experience or proficiency 11% (1)
    2 Beginner: I don't do it well and largely rely on others for this skill 0% (0)
    3 Competent: I'm OK -- maybe not the best around 33% (3)
    4 Proficient: Colleagues come to me for this skill 44% (4)
    5 Expert: I'm among the best around 11% (1)
    Response Average 3.44
  • Marketing/sales communications
    1 No experience or proficiency 0% (0)
    2 Beginner: I don't do it well and largely rely on others for this skill 25% (2)
    3 Competent: I'm OK -- maybe not the best around 25% (2)
    4 Proficient: Colleagues come to me for this skill 50% (4)
    5 Expert: I'm among the best around 0% (0)
    Response Average 3.25

Sounds like a pretty proficient group to me, bless 'em.

This is my list of communications proficiencies -- is it the right list? What's missing?

Next post -- the session questions and responses.