Here are my notes from the third session of the Enterprise 2.o Black Belt Workshop: Measuring Success and Business Value – Metrics and Analysis
Speakers:
- Ted Hopton, Wiki Community Manager, United Business Media
- Donna Cuomo, Chief Information Architect, the MITRE Corporation
Background:
[These are my quick notes, complete with (what I hope is no more than) the occasional typo and grammatical error. Please excuse those. Thanks!
From time to time, I’ll insert my own editorial comments – exercising the prerogatives of the blogger. I’ll show those in brackets. ]
Notes:
Ted Hopton:
- His company organized the Enterprise 2.0 conference
- They use Jive software for their enterprise 2.0 platform
- Focus first on participation
- Use the analytics module of your enterprise 2.0 tool to see who is visiting the site, where the activity is taking place, who is creating and viewing content, etc.
- Analyze active members by level of activity
- Problem: Make sure the metrics tie back to your project goals
- Use qualitative measures to improve your understanding
- Use a survey – ask how and how often people use the community
- List possible positive outcomes and ask users which of these outcomes they have experienced
- Ask why they don’t use the tool more
- Use blunt, negative statements
- Encourage them to tell you exactly how they feel
- Use this information to benchmark (and draw out the venom – otherwise it festers)
- Net Promoter Score – give users a scale of 1-10 and ask them how likely they are to promote your work. Scores above 6 indicate that they will promote rather than detract your promoter. Subtract your scores below 6 from your scores of 6 and above. This yields your “Net Promoter Score.” Obviously, the higher the better.
- Track your success stories and share them
- Lessons Learned
- While it’s good to have consistent metrics, be aware that metrics evolve and your methods should evolve too
- Beware of benchmarks (e.g., the 90-9-1 standard of participation). Make sure the benchmark you are using really applies to E2.0 projects.
Donna Cuomo:
- The Mitre Company runs four differently federally funded programs (including for the Dept. of Homeland Security and the Dept. of Defense)
- Use Case 1: Improve MITRE’s Research Program Selection Process
- They used Spiggot to be their “innovation management tool”
- They wanted to codify their research competition process
- They wanted to stop people further down the food chain from weeding out ideas too early
- They wanted to encourage broader participation (from a review perspective)
- They created an “Idea Market”based on a SharePoint wiki
- Their first-year metrics indicated broad participation
- They were able to create widespread transparency
- They used surveys to compare the new tools (and user satisfaction) against the old tools/methodologies
- Use Case 2: Social Bookmarking
- Hypothesized that social bookmarking would inmprove resource sharing, leveraging the research of others across teams and the corporation
- They also thought the tagging would help identify experts within the organization
- They used a tool similar to Delicious
- Bookmarks helped create a lightweight newsletter (this was an unexpected benefit)
- You don’t need many participants in order to provide real value to the entire organization
- Use Case 3: Babson SNA Study
- They identified super users of their internal social networks and social media (brokers) and then interviewed their colleagues
- They discovered that these super users tended to be innovative and provide huge value to their networks
- Frequency of interactions was not as important as the number of unique connections each broker had (indicative of their ability to have an impact on a wider range of people).
Exercise:
- What are the most important things you are NOW measuring?
- Number of communities
- Number of community members
- Percentage of contributors versus consumers
- Usage across geographies, business units, etc.
- Number of visits
- Dwell time (how long is each visit)
- Number of concurrent users at any one time
- Number of people editing (indicates collaboration)
- Number (and identity of ) lurkers
- Measuring conversion of lurkers to active participants
- Participation in community activities (who is sharing, who is editing, who is tagging, etc.)
- Utilization of the various social tools
- Success stories
- What are the most important things you should be measuring?
- Abandonment rate – when do visits/activity drop off
- Tracking against business goals
- Net Promoter Score
- Day/time of highest activity
- first and last page viewed
- business improvement metrics
- = correlation of usage to operating metrics
- = correlation of usage to improved business process
- Measuring cross-fertilization (the extent to which people choose to go outside their community for information)
- Number of new ideas/ rate of innovation
- What’s the reduction in other forms of overhead activities (e.g., now that the subject matter expert is posting answers on a social platform, what is the resulting decline in repetitive e-mail requests?)
- Percentage of profile completion
- Rating content
- Ability to determine a dollar value to participation
- Where was the content reused, how was it reused, and what were the results of the reuse (e.g., cost savings, process improvement, etc.)
- Presentations: http://www.e2conf.com/boston/2010/presentations/workshop
- User name: Workshop
- Password: Boston
- Presentations also on Slideshare: http://slideshare.net/20adoption
Great notes! Thanks for posting :-)Minor correction: NPS scale runs from 0-10, rather than 1-10 (just so you'll be consistent with the way it's used elsewhere and the research behind it).
Thanks very much for the clarification, Ted. And thanks for a great session today.- MaryVMaryAbrahamAboveandBeyondKM.com