Always measure your social media effort. Except when you shouldn’t.

Matt Ridings is one of the most intellectually challenging and entertaining friends I have made on Twitter. During a recent debate on {grow} about social media measurement he chimed in with such a smart counter-point that I wanted to provide it as a feature for the entire {grow} community …

Guest post by {grow} community member Matt Ridings

Social media measurement. This is a seemingly simple issue, with a complex point.  Or perhaps a “nuanced” point is more accurate.  I’m obviously a believer in data, it’s a big part of what I’ve done over my career.  So to be clear, trying to measure whether something has a meaningful impact is something I am in support of.  And you should always measure … unless you shouldn’t. Here is what I mean …

1) Spreadsheets should never be a substitute for business instincts.  Too many fall back on spreadsheets as means to avoid risk and accountability.  Thus using ROI is an excuse not to move forward with something they don’t understand.

2) Sometimes there is no ROI to measuring ROI.  Just because something *can* be measured doesn’t mean it *should* be measured.  If the cost of trying to measure an activity outweighs the gain of the activity itself, or eats too far into that gain then why would you measure it?  It doesn’t mean there was no gain, it means you have to make a decision based on more than spreadsheets as to whether you can mentally correlate enough benefit to quantifying the activity to continue doing it.

3) Understanding appropriate time horizons and objectives is critical. Numbers mean nothing if you aren’t balancing them against proper expectations.  If you don’t have a solid, educated theory as to how long an activity should take before it starts showing its full benefits (in a relationship driven economy the long tail activity becomes the norm so this becomes even more critical) then how can you know when to make a decision to stop or increase that activity?

4) Achieving a return on an activity is meaningless without a knowledge of the return on activities you *could* have been doing.  Companies have limited resources, they can’t do everything, so they have to maximize those that they undertake.  Getting a 20% return on something is a failure if you needed a 40% return to make it a viable alternative to some other activity that gets 30%.  So “achieving an ROI” isn’t the same thing as “achieving success.”

5) Understanding the impact of measurement, both positive and negative, is also critical.  Measurement itself impacts the behaviors and decisions inside the organization.  How does the measurement motivate or impact the individuals?  Does it do so in ways that are beneficial to the customer or in ways that benefit the company in the short term but cut its throat in the long term?

I’ll stop there, but the point is basically “It’s not whether you measure, it’s whether you understand what to do with it.”

Agree?  Would love to hear your thoughts on “rational measurement!”

Matt Ridings, aka @techguerilla, is the co-Founder and CEO of SideraWorks, a new Social Business consultancy founded with Amber Naslund.

 

Related Posts Plugin for WordPress, Blogger...

All posts

  • A quality article. Being a Master of Science student at GCU,  I am always curious about  formulas. 

  • It’s as if you were inside my head. What a great post, Matt.

    The thing that is most interesting to me about measurement is what it tells you about your strategy… whether it’s working or not in terms of desired outcomes, not just numbers. Because you’re right; numbers by themselves don’t mean anything. They need context, otherwise they can be very impressive (or disappointing) and are completely useless.

    I teach a course for Hopkins’ MA/Communication program every year for professionals either in the non-profit space or hoping to work in it, on smart communication in the digital age. You’d think it would focus a lot on the tools, etc. – and we certainly go through those – but measurement is front and center of the course, probably to the point of driving my students insane. But I love it when I see from their discussions (the course is completely online) that they are really starting to get the importance of integrating measurement right from the start into their overall communications. That usually happens around mid-semester, and it’s as if someone turned all the lights on in the room.

  • Ketan Raval

    its like just dive in.. forget about roi.. it is important to have social face of the brand .. someone has to take a shot and if no one is there for your business its yourself to do that.. 

  • This is great Matt! I am a big fan of data as well in the sense of measuring activities and spend (not so much the big data kind from third party research firms but I digress…). It continues to amaze me how many people are still just throwing money at stuff and hoping it sticks. No measurement process in sight, no assumptions on the outcome, it’s just spaghetti being thrown at a wall which is not a great form of measurement by any means.

    Your final point here hits it head-on and it’s great; “It’s not whether you measure, it’s whether you understand what to do with it.” Maybe some of us just dont know what to do with it so we dont bother at all. Better than the other way around of forcing a justification on something when it just isnt there to begin with.

  • No, you definitely don’t want to forget about ROI – you just want to make sure you’re measuring the stuff you should be measuring and then making sure you know how to act on that information.

  • Measurements come in various forms – more or less, ratios, % and interval scales.  Too often one will look for a ratio scale without realizing that the underlying data doesn’t provide ratio scale significance.  Here is a simple example.  Suppose that a person mapped the average football jersey numbers to a team’s score and calculated the ratio.  Since the numerator contains numbers that are largely conventional, this ratio is not just stupid but meaningless.  

    Starting off, you are better with a number of ordinal measurements – more or less measures- rather than looking for an interval scale measure – like temperature.  As for weights, a ratio scale- it is unlikely that the data supports the informational requirement needed for a ratio scale.

    My takeaway – focus on finding improvements that are more or less in the right direction.

  • Hi Matt,

    This article explains precisely why the advice to “just jump in” that you see so often in the world of social media is so misleading. The question marketers face with anything they do is not answering the question, “Is there an ROI to this.” The questions they should be asking are:

    What am I investing? And b-t-dubs, time = money

    Should I be investing here? What’s my rationale? And how much will I invest based on that information?

    What kind of return am I looking for? If there isn’t likely to be a lot of monetary gain, you might want to strategize more carefully how you spend your time. Prioritize money-makers first.

    The list goes on and on. 

    I hope, for the sake of businesses, that we get beyond the whole “Oh, ROI – that’s Return on Ignorance” and actually start dealing with these significant business issues. That would be awesome.

  • Pingback: Always measure your social media effort. Except when you shouldn’t. | fabriziofaraco()

  • Anonymous

    Hi Matt,

    I’m the social media manager at my job, and we’ve been measuring our activities since December. I’ve honestly been surprised at how long it takes me to get the data for our dashboard and then fill it out. The good news is, it is really informing us of what people want us to give them. So even though it takes quite a bit of time, I’d say the ROI of measuring ROI is there.

    Thanks for a great graphic with your post, too.

    Jenn

  • Thanks Christina, I’m torn about this because while it may be great advice (and it is) I’ve seen companies go too far the other way as well.  Throwing money at stuff and seeing what sticks sometimes has another name…innovation.  And we don’t want to kill that off by a misguided focus on efficiency.  The key I think, is having great leaders in companies who know how to properly strike that delicate balance between hope, efficiency, and revenue generation.  We always see those latter two (efficiency and revenue gen) being discussed, but ‘hope’ is a dirty word in business…and I think that’s a shame.

    The primary piece of advice I would tell people where measurement is concerned (whether they are choosing to measure at that moment or not) is to be diligent about setting baselines.  There’s only one certainty when it comes to gauging the success of something,  and that’s that you can’t know how far you’ve come if you don’t where you started from.

    Thanks for the great comment.

    Matt Ridings – @techguerilla:disqus 

  • One thing is for sure Shonali, you DON’T want me rattling around inside your head 🙂

    Great comment.

    Matt Ridings – @techguerilla:disqus 

  • All credit for that graphic goes to Mark, I liked it too.  Glad to hear you’ve been able to determine the value of measurement in your social media activities!

    Matt Ridings

  • Thanks for the comment Marjorie, I left you a reply that seems to have been eaten by Disqus.  But rest assured it was very flattering to you, included several mentions of rainbows and unicorns, and contained gratuitous amounts of textual fluvium 🙂

  • Mark Limbach

    I notice increasing reluctance on the part of marketing executives to use judgment; they are coming to rely too much on research, and they use it as a drunkard uses a lamp post for support, rather than for illumination.
    David OgilvyWe don’t ask research to do what it was never meant to do, and that is to get an idea.William Bernbach

  • Love both those quotes, thanks.

  • Oh SURRRRRRRRE 🙂

    mmmmmm, fluvium. nom nom nom.

  • “Throwing money at stuff and seeing what sticks sometimes has another name…innovation”

    Pure brilliance right there.

  • Hi Matt, thanks for some thought provoking points here that can help marketers avoid chasing their tales.  I’ll comment on two of them:
    1.) On the title of the post: I wish that the title of this piece was “How to decide what to measure” — because I do think social media marketing should be measured.  The only question is WHAT should you spend time and effort and money measuring.  For some, the answer to that question might be navigational metrics only (i.e., the metrics that help the marketer optimize for increasing levels of engagement, influencer WOM recommendations, or some other desired outcome).  For others, the answer to that question might be metrics that serve as indicators of some form of return. 

    2.) On the idea of time horizons:  I’m really glad you brought this up.  The problem is that with any new thing (social media itself isn’t new, but new channels and tactics emerge almost daily), knowing the time horizon required to get results just isn’t possible.  The only way to get information on what the time horizon should be is to do it and track it.  One of the things I like about Social Snap is that every metric we report on is shown over time and against goal.  This isn’t very helpful if you are just starting out, but over time, seeing those trends can really provide some very helpful insights, especially when viewed alongside contextual data (like where your competitors are on the same metric).

    I also really enjoyed your comment about the impact of measurement internally.  From my experience, this is something that gets overlooked 99.9 of the time when an organization takes on a measurement project.

    Thanks for an insightful post.

  • This post rings true on so many levels Matt… I just read a parallel piece by Avinash Kaushik that opens “The single biggest mistake [web analysts] make is working without purpose.” http://twrt.me/kcawx4 >

    Substitute [marketers] and it wraps around to the same point you make – If you don’t know the right questions to ask, you’ll invariably throw up a bunch of data and hope some of it casts the right light on the subject.

    I think the overlay of instinct you’ve added is important – not every question reveals a single answer; if we’re smart we’ll see the array; if we’re clever, we’ll shape a chosen path into the best answer.

  • Hi Matt,
    This a topic that should have engaged marketers for a while now, as it certainly excites most CFO’s when looking for cost savings. The onus to support an investment with ROI data falls on the initiator, and in the absence of justification, it will just get cut.
    Here is my approach, http://strategyaudit.wordpress.com/2011/12/02/return-on-exposure/
    which works in concert with the analytical tools.
    Cheers
    Allen Roberts

  • This reminds me of some of that good ole down home country wisdom I learned from John Wooden – the hall of fame basketball coach of the UCLA Bruins . . .

    “Don’t mistake movement for achievement”.

    Thanks for shedding light on something to focus on beyond the momentary stats Matt! 

  • Pingback: Best of B2B Marketing Zone for March 5, 2012 « Sales and Marketing Jobs()

  • Your final point is, for me, perhaps the most salient. “Its not what you measure, its whether you understand what to do with it.” This is perhaps the biggest challenge for many businesses. There are few guarantees in the world, even if you ARE measuring it, but at some point, someone has to have the courage to make a decision.  The challenge with ROI, is that for real understanding to take place, it takes more than 30 days of measurement and sometimes, the decisions take too long, where an instinctual, business professional could have predicted the outcome, but the spreadsheet addicted  demanded another look and another reworking. I’m not saying I’m against data, far from it. But its to you point – data collection isn’t the end game.   

  • I found this article on Twitter and debated on reading it because I figured it would be just fluff.  So much conversation about ROI or not ROI is.  Personally, I think most Social Media professionals are too extreme one way or the other.  Never track it.  Always track it.  Neither is right.  Well written.  Well balanced.  Glad I clicked on it.  🙂

  • Thanks Bill, and thanks for the pointer to the analysts post.

  • As I’m a good ole down home country boy, I can relate 🙂

  • So 1) this is EXACTLY a question I’m going to ask my panel at SXSW, because it’s a great one: 
    http://schedule.sxsw.com/2012/events/event_IAP9012 (#shamelessplug #humblebrag #otherapplicabledisclaimers)

    2) While I completely understand and agree there’s a point of diminishing returns with any kind of measurement, including social ROI, I worry that this is a crutch for bad business habits rather than a true choice of the best opportunity cost.  As businesses slowly metamorphosize into social businesses, the opportunity to identify an ROI from social should be increasingly greater, as should the potential returns.  And, as we sort through the technologies and techniques to make that possible, the costs of defining that ROI in a manner that’s accepted across the business should decrease.  Right now, many social marketing programs are based on HIPPO (HIghest Paid Person’s Opinion), which is fine for getting started, but utterly unsustainable for the long term.  Businesses that run their social marketing on dead reckoning alone because it’s expedient or cheaper lose the opportunity to fail fast early, without the increased scrutiny that WILL come later.  That lost opportunity to learn in a safe environment has its own cost – one that could offset the early struggles of learning how to measure a social ROI.

  • Balance.  In all things. 🙂  Try and be right more than you are wrong.  Remember that ‘being wrong’ has a different name, it’s called ‘experience’.  Applying the learnings from that experience makes you wrong less often.  Remember that waiting until you have everything possible to eliminate the risk from your decision means that you have either already waited too long, or you are too risk averse to be in a decision making position.  Measuring is a means to validate, or invalidate, decisions…not a substitute for making them.

  • I’m glad you did too Corey 

  • I’m certainly not saying “don’t measure”, I hope it didn’t come across that way.  

    What I am saying is when are we going to stop treating businesses like they’re idiots and giving them black/white choices for what are heavily nuanced gray issues?  These are professionals who have to make choices.  My preference is to educate, give them the tools, demonstrate the complexity of the options available, etc.  But once they grasp the variables in play, it’s not my place to presume to sit in their shoes and know the pressures and implications of their decisions.  I’m a trusted guide, that’s it, no more no less.  We have a tendency (speaking about consultants as a whole) to forget that sometimes that ‘right’ choice, is not a practical one.  I can tell you all day long that the absolute best way to get to London is via airplane, it’s the most efficient method available…but if you can only afford a dugout canoe and a paddle I’d be a pretty crap consultant if I told you you were wrong for taking the canoe no?  I want my clients to make the most educated choice they can make, I want to teach them to fish, I can’t do that by pretending that there is a one-size-fits-all answer.

    Will I be seeing you at SxSW?  Hope so!

  • Yes, will be there and definitely need to find a hotel bar table to pick your brain over. We’ll get Chuck and Tom to join us and call it an Analytics Salon. 🙂

    I couldn’t agree more with you on giving your client the tools to make the best decisions they can for their business.  We get very enamored with advanced solutions (particularly in social media), but often just because they’re complex, not because they’re effective.  We also do our clients a disservice if we don’t provide them with multiple ways to get the job done (if they’re all effective) at multiple price points.  I think the real measure of a good consultant (love your “trusted guide” metaphor – I’m gonna steal that one) is when they will present options that don’t necessarily earn them more money.  

    See you in Austin!

  • We can all do with more fluvium

  • Chris

    I spend time trying to figure out ways to tap into peoples emotions and cultivate their brand love.  I’ve always found it interesting how many demand an ROI on this nebulous thing… emotional connectivity.  I loved Gary Vaynerchuk’s answer to a question at an event about measuring ROI… “How do you measure the ROI of your mother?”.  It’s quite similar to the modern marketers dilemma.  We try to be really clever with our ideas, tapping into those deep emotional connections, forge meaningful relationships for the long haul that will net us little rewards along the way… and now you want to measure that?  Against all the other variables that exist?  I’m starting to lean more towards instinct these days and LESS (not eliminate) towards analytics.

  • Unless you are a Maths professor or that dude from Star Trek – Data is never the full solution.  It may well be part of the solution (sometimes a very useful part) – but good decisions are born from a combination of factors.  The skill is knowing how to use all your asstes collectively to make the best choice…..

  • This reminds me of the debates from my physics and philosophy courses in college! One cannot measure anything without changing it! consequently, the act of measurement in itself needs to be factored into the reults..
    Anyway, Ive always found that the problem is the absence of a plan, rather than getting the right plan!
    Pretty much any data can be utilised to support or undermine anything your business does. Due in large part to what Matt talks about here and the inability of those measuring to understand both the impact of the measurement and what the measurement means!
    Great post, really enjoyed it, and enjoying the debate even more so!

  • Matt, I don’t think anyone can argue with what you are saying.  You summary point is absolutely right on.  Fundamentally, before any company tries to build an ROI matrix for Social Media, they really need to understand what it is first. Given the hype cycle we’re still in (and will be for quite some time), companies need guidance to help figure this all out. As an example, plowing a lot of money into Facebook advertising (just because everybody is doing it) is not necessarily a smart thing to do (as many companies are now discovering).  I think the real point is “use valid business judgement”. 

  • Awesome post! I agree that at some point social media needs to be measured. If you’re paying someone exclusively to handle your SMM at your business you need to find out if your ROI is there. However I think some business owners, especially small business owners become so preoccupied with ROI that they forget the intangibles that can’t be measure with social media. If I may, here are some basics for the novice small business owner to keep in mind when getting started with their SMM strategy http://ow.ly/9x71T

  • Pingback: Always measure your social media effort. Except when you shouldn’t. | Metiers du Web | Scoop.it()

  • Pingback: Developdot – Social Media ROI: It’s Not One Size Fits All()

  • Pingback: Rational Social Media Measurement | Redes Socia...()

The Marketing Companion Podcast

Why not tune into the world’s most entertaining marketing podcast that I co-host with Tom Webster.

View details

Let's plot a strategy together

Want to solve big marketing problems for a little bit of money? Sign up for an hour of Mark’s time and put your business on the fast-track.

View details

Close