Obama’s Town Hall Meeting versus Twitter Truth

The premiere social media event of the week is U.S President Obama’s “Twitter town hall,” today.  The president will take questions posed through the #askobama hashtag during a discussion moderated by Jack Dorsey, a co-founder of Twitter.

The White House said it wants the talk to focus on the economy, but Mr. Obama is sure to get asked all sorts of things when he takes questions from the Twitterverse.  But is this really going to reflect what the Twitter Nation wants to hear, or is this just going to be a political dog and pony show?

I wondered … would it be possible to use analytical tools to actually predict what questions would be used today?  Of course it all gets down to the judgment of a few people actually setting the agenda, but at least based on the volume of suggested questions coming in, what should the Town Hall discussion be about?

Our loyal {grow} community member Dan Holowack was up to the task. He offered to put his proprietary software tool, TwitSprout, to the test to try to predict the agenda today.  It will be interesting to see if the actual questions in today’s meeting line up with Dan’s predictions.  Here is his analysis:

The Town Hall and Twitter Truth

My team has literally stayed up all night analyzing well over 10,000 unique questions in the hours leading up to the deadline. As you can see, there has been a frenzy of increasing activity:

 

Looking for tweet themes

It’s still an open problem in artificial intelligence for computers to “understand” human language, even in constrained settings like this one. Separating the real questions from the garbage is therefore more of an art than a science — we were forced to use a bunch of heuristics for this initial pass, but our tests indicate that they work quite well. According to this measure, just under 60% of all #AskObama tweets are genuine questions.

Once we identified the “legitimate” questions, they were passed through further filters to combine tweets that are just retweets or quotes of one another, to bring the quantity down to a more manageable level. Finally, these tweets were sorted in descending order by popularity and influence, and one of our team members went through them manually, grouping them into high-level categories (“Jobs”, “Taxes”, etc) by hand. By adding together all of the tweets in these categories, we can get a real sense of which topics are on the minds of Americans.

Our Predictions

To predict which questions Mr. Obama will answer today, it would be nice to know how they’re being chosen. According to Twitter, the team at Mass Relevance (plus Jack Dorsey) are the ones entrusted with curating and selecting the lucky tweets. Nobody can be sure exactly what their selection process will look like, but it’s safe to say it’ll be some combination of the following:

1. Random selection: In true democratic fashion, all tweets might be treated equal and have the same chance of being picked at random. While nobody can complain about the fairness, it might lead to some really strange results.

2. Sheer numbers: The best questions are being retweeted hundreds of times, and the cream is really rising to the top. The team could select the most retweeted and RT’ed tweets as the questions the most Americans care about.

3. Popularity contest: There are many ways of measuring influence on Twitter — whether it’s Klout, follower count, or your TwitSprout dashboard. However they choose to measure it, the questions asked by the most important people might be the lucky ones.

4. Politics as usual: Of course, it’s not impossible that the White House has already “vetted” a series of questions from their own talking points, and the team is simply looking for people on Twitter who asked the same things.

We can’t do much about #1 and #4, but the information we’ve gathered gives us everything we need for the other cases. Questions will still be coming in, but assuming nothing dramatic happens, these are the most popular and the most influential questions so far.

“Most Popular”

If all that matters is quantity, these are the tweets that have gathered the most retweets, quotes, and RTs:

#1 (with 2,907 retweets): “Would you consider legalizing marijuana to increase revenue and save tax dollars by freeing up crowded prisons, court rooms?”

#2 (with 1,567 retweets): “You’ve said many times that the Bush Cuts for the 2% Should Expire. Can you promise to let them in 2012?”

#3 (with 750 retweets): “Mr. President, why should you not be held responsible for your silly prediction that unemployment would stay below 8%?”

“Most Influential”

If we look for the tweets made by the most influential users — those with the most Klout and followers — we get a different set (although they have one tweet in common!)

#1 “Tech and knowledge industries are thriving, yet jobs discussion always centers on manufacturing. Why not be realistic about jobs?”

#2 “Mr. President, why should you not be held responsible for your silly prediction that unemployment would stay below 8%?”

#3: “Why do we have 1.5 million fewer jobs than we did before the stimulus when the # of ATMs is unchanged?”

“By Category”

Thanks to our manual grouping of tweets, we can also figure out which categories are the most popular (even if the team chooses some other specific tweet to represent it). There’s always a bit of subjectivity involved when making some of the decisions here, but most of our results should surprise nobody:

#1 Legalization of Marijuana (4,911 total tweets and retweets)

#2 Jobs (2,024 total tweets and retweets)

#3 Taxes (1,800 total tweets and retweets)

#4 Economy and Debt Ceiling (442 total tweets and retweets)

The one surprise is the winner of this category — marijuana laws.  Regardless of which side of this debate his opinion happens to fall, if the president doesn’t address this point during tomorrow’s discussion, somebody is avoiding the data. By a landslide, this is what the Twitterverse wants to know.

The post-mortem

This section of the blog post was added AFTER the Town Hall event. How did it turn out?

THE GOOD

What did we get right? We feel we matched the major categories of things Obama focused on — tax cuts, the debt ceiling, and job creation — but that wasn’t very difficult to do. More specifically, we nailed the third tweet Obama answered:

Tech and knowledge industries are thriving, yet jobs discussion always centers on manufacturing. Why not be realistic about jobs?

This one was right on our dashboard, and we had singled it out because @Kim, a very high-Klout user, was one of the many who put their weight behind it. The White House chose the earliest version of the tweet, by @dmscott (with slightly lower Klout), but the logic is clear — this tweet was chosen for the high-level support it received.

Another one we expected, but for a completely different reason, was Obama’s 8th:

Mr. President, In several states we have seen people lose their collective bargaining rights. Do you have a plan to rectify this?

According to our calculations, at the time of the Town Hall, this tweet had been mentioned, retweeted, or RT’ed over 182 times — making it the 4th-most active tweet in the Jobs category, and 14th overall. This is high enough to be on our list of likely predictions, although not high enough to earn one of the few spots on the Dashboard itself. Even though the tweeter, @pmglynn, is not very influential (with only around 100 followers), the question itself generated a lot of discussion and interest. It was an excellent candidate, and both TwitSprout and Obama appear to think so.

THE BAD

We definitely stand by all the predictions we made — there was solid reasoning behind each of them, and we’re confident they would have made excellent questions; however, there are a couple of predictions we didn’t make that, in hindsight, probably would have made sense.

The most obvious omission is @johnboehner’s provocative tweet:

After embarking on a record spending binge that’s left us deeper in debt, where are the jobs?

At the time of the Town Hall, this question had been propagated 79 times: a nontrivial amount, but only enough to place it in 10th place in the Jobs category, and much lower overall. There are two things that should have tipped us off about it, however:

  • Even though @johnboehner’s Klout is only 69, his influence on President Obama is considerably higher. His position as Speaker of the House undoubtedly played a role in Mass Relevance’s decision. Of course, there’s no automated tools that would detect this. We simply should have thought of it.
  • Even though the full form of the tweet had only 79 propagations, the snappier version “Where are the jobs?” by itself had 341 of them — enough to place it in 2nd place in the Jobs category, and 6th place overall, which would certainly have gotten our attention — except that this tweet didn’t make sense to us, since we didn’t see it in context with Boehner’s tweet (they were far apart in our rankings). Of course, we had algorithms to detect subtle rewordings between tweets, but these two didn’t trigger them.

THE UGLY

We might as well come out and address the elephant in the room. It was obvious to anyone who had followed our predictions, those of Simply Measured, or paid even casual attention to the #AskObama stream, that there was one topic getting a large share of the buzz: the legalization of marijuana. Although the topic of the Town Hall was ostensibly about the economy, it was becoming clear that people did think of this as an economic issue. By our measurements, the two most popular tweets of the entire stream, were:

Would you consider legalizing marijuana to increase revenue and save tax dollars by freeing up crowded prisons, court rooms?

Pres. Obama: Why can’t we discuss legalizing cannabis to create jobs and save millions annually on enforcement?

These two messages accounted for more impressions than every question about taxation put together! An event that emphasizes popular opinion as a tool for accountability could surely not fail to address these questions in the depth they deserve; but, of course, the topic was notable only for its absence. The President discussed the war on drugs merely tangentially, for less than half a minute.

Ah, politics as usual : )

Thanks to Dan and and his team for pulling two all-nighters to produce this post!

Related Posts Plugin for WordPress, Blogger...

All posts

  • Two observations:

    Once such an analysis is done, it can be done by those preparing the townhall — which means that we’re still going to get massively canned answers. (There will be no curveballs, and a sense of faux spontaneity.)

    Second — How much of the discussion on twitter has already been seeded and primed by mainstream news outlets?

    This will be an experiment, to be sure, but I don’t even know how interesting.

  • this was interesting, much more so than the actually “event” will be I am sure

  • I agree with Ike that the Obama team will be evaluating the tweets to prep – and will surely have a larger team than Dan did!

    I think the thing to learn from this post is WHAT Dan’s team did – analyzing and grouping to predict themes.  This process can be applied to many social programs to identify themes, customer service issues, product feature requests, etc.

  • Hi Ike. Thanks for the comment.
    This is exactly what we are attempting to monitor.
    The purpose of Twitter Town Hall is not necessarily to spur Presidential spontaneity. The purpose, in my opinion, is to encourage Obama to face the tough questions, the topics put forward by the people on Twitter. Yes, we can perform ongoing analysis – but its more about accountability during the event. All this data is public, it’s more difficult to hide from the highly retweeted topics.

    It would be exceedingly difficult for mainstream news to plant questions and propel them to the top (this requires 1000s of retweets). We have tight integration with Klout to help weed out new ‘seeding’ accounts. To your point, any news outlet could initiate the conversation (similar to an individual) but it requires the collective interest of hundreds (or thousands) of Twitter users to bubble to the top.

    All will be revealed at 2PM EST. Stay tuned! http://Obama.TwitSprout.com
    Cheers,
    Dan and the TwitSprout Team

  • Hi Mark! Thanks for using TwitSprout’s analysis to support this post!
    Our Co-Founder, Adrian http://twitter.com/#!/apetresc, has been working long hours to uncover these trends & insights that support our latest predictions.
    We’re here to answer any questions related to our free TwitSprout beta and ongoing analysis of the Obama Dashboard – http://Obama.TwitSprout.com

    Cheers!
    Dan & team

  • Precisely, Jody! This exercise has helped hone our TwitSprout algorithms & visualizations for tracking performance of business hashtag campaigns in the future.

    I’m very curious to see if Obama avoids certain topics or popular questions which clearly represent a large portion of our data set. Time will tell.
    -dan

  • Just to clarify — I don’t think that mainstream media would bother trying to proactively seed the discussions… but we’ve seen a lot of research that indicates that most of the RTs being shared are from mainstream outlets.  They are still big dogs in a 140-world.

    I’m just saying that the agenda probably won’t be much different than that…

  • Got it, Ike. I don’t see how news outlets would be interesting in promoting other people’s political interests. RTing a question to Obama stats your interest in the topic. Most outlets tweet general news about the event – but we filter the majority of this noise from the analysis.
    -dan

  • Do I smell a follow up post? 😉

  • Wow, I found all this fascinating! I might make an effort to attend, just to see it happening.

  • This is an awesome “before” picture for the event. I’d love to see the “after” results. Will you be running an analysis to see what was asked and how much it matched up with what “should have been” asked based on the numbers? Because if I were to predict, I bet it won’t be matching up very well. I’d be very surprised if the ATM question gets asked :).

  • Ok. I want to know whether the marijuana question reflects the general population or whether the Twitter population is on the fringe and its demographic is particularly interested in marijuana!

  • Way to go Dan! I really appreciate the work that was put into this. It’s interesting to see the most popular questions and the questions that are actually addressed. Very neat post! I can’t wait to see more like this 🙂

  • I meant way to go Dan and team!

  • My hunch is fringe, but who knows??

  • Hi Eugene, the post has been updated with our results. A few of the predictions were right on target. The selection of questions during the events did not align well with our Retweet prediction at all. In fact, some of the questions were from users with less than 10 followers and zero RTs! Therefore, there must have been some random selection or alignment with pre-set speaking topics. Still, we can only speculate. The team is working on a final “Results” infographic that ties all this together – with some interesting discoveries. We’ll continue to post content to http://obama.twitsprout.com
    Thanks for reading! Cheers. -dan

  • Thanks Tara! -dan

  • You’re likely correct, Neicole. We would need aggregate data from other sources to compare. Either way, it had an overwhelming majority – and every person that tweeted also has a vote. Obama gave the drugs topic 20 seconds of his time and the space program (with less than 200 tweets) nearly 2.5 minutes! Now that’s a huge imbalance!
    Thanks! -dan

  • Pingback: One Big Natural Extension » @Josh_Muirhead | Digital Marketing Specialist()

  • Great job Dan!!!! I believe you’ve successfully rationalized your results and with some additional data could have likely picked up on anything “overlooked”. What would have been really interesting (although I’m not sure possible) would be to consider the political agenda to anticipate what would likely be answered to move that agenda forward and see the delta between what the people want verses what the political machine is prepared to do.

  • Real Estate Web Design Delhi

    Very interesting post! real estate web design Delhi

  • Pingback: Blast Off!! Save the Space Program by.. Legalizing #Marijuana? « So Genius()

The Marketing Companion Podcast

Why not tune into the world’s most entertaining marketing podcast that I co-host with Tom Webster.

View details

Let's plot a strategy together

Want to solve big marketing problems for a little bit of money? Sign up for an hour of Mark’s time and put your business on the fast-track.

View details

Close