Archive for year 2011

The premiere social media event of the week is U.S President Obama’s ”Twitter town hall,” today.  The president will take questions posed through the #askobama hashtag during a discussion moderated by Jack Dorsey, a co-founder of Twitter.

The White House said it wants the talk to focus on the economy, but Mr. Obama is sure to get asked all sorts of things when he takes questions from the Twitterverse.  But is this really going to reflect what the Twitter Nation wants to hear, or is this just going to be a political dog and pony show?

I wondered … would it be possible to use analytical tools to actually predict what questions would be used today?  Of course it all gets down to the judgment of a few people actually setting the agenda, but at least based on the volume of suggested questions coming in, what should the Town Hall discussion be about?

Our loyal {grow} community member Dan Holowack was up to the task. He offered to put his proprietary software tool, TwitSprout, to the test to try to predict the agenda today.  It will be interesting to see if the actual questions in today’s meeting line up with Dan’s predictions.  Here is his analysis:

The Town Hall and Twitter Truth

My team has literally stayed up all night analyzing well over 10,000 unique questions in the hours leading up to the deadline. As you can see, there has been a frenzy of increasing activity:

Looking for tweet themes

It’s still an open problem in artificial intelligence for computers to “understand” human language, even in constrained settings like this one. Separating the real questions from the garbage is therefore more of an art than a science — we were forced to use a bunch of heuristics for this initial pass, but our tests indicate that they work quite well. According to this measure, just under 60% of all #AskObama tweets are genuine questions.

Once we identified the “legitimate” questions, they were passed through further filters to combine tweets that are just retweets or quotes of one another, to bring the quantity down to a more manageable level. Finally, these tweets were sorted in descending order by popularity and influence, and one of our team members went through them manually, grouping them into high-level categories (“Jobs”, “Taxes”, etc) by hand. By adding together all of the tweets in these categories, we can get a real sense of which topics are on the minds of Americans.

Our Predictions

To predict which questions Mr. Obama will answer today, it would be nice to know how they’re being chosen. According to Twitter, the team at Mass Relevance (plus Jack Dorsey) are the ones entrusted with curating and selecting the lucky tweets. Nobody can be sure exactly what their selection process will look like, but it’s safe to say it’ll be some combination of the following:

1. Random selection: In true democratic fashion, all tweets might be treated equal and have the same chance of being picked at random. While nobody can complain about the fairness, it might lead to some really strange results.

2. Sheer numbers: The best questions are being retweeted hundreds of times, and the cream is really rising to the top. The team could select the most retweeted and RT’ed tweets as the questions the most Americans care about.

3. Popularity contest: There are many ways of measuring influence on Twitter — whether it’s Klout, follower count, or your TwitSprout dashboard. However they choose to measure it, the questions asked by the most important people might be the lucky ones.

4. Politics as usual: Of course, it’s not impossible that the White House has already “vetted” a series of questions from their own talking points, and the team is simply looking for people on Twitter who asked the same things.

We can’t do much about #1 and #4, but the information we’ve gathered gives us everything we need for the other cases. Questions will still be coming in, but assuming nothing dramatic happens, these are the most popular and the most influential questions so far.

“Most Popular”

If all that matters is quantity, these are the tweets that have gathered the most retweets, quotes, and RTs:

#1 (with 2,907 retweets): “Would you consider legalizing marijuana to increase revenue and save tax dollars by freeing up crowded prisons, court rooms?”

#2 (with 1,567 retweets): “You’ve said many times that the Bush Cuts for the 2% Should Expire. Can you promise to let them in 2012?”

#3 (with 750 retweets): “Mr. President, why should you not be held responsible for your silly prediction that unemployment would stay below 8%?”

“Most Influential”

If we look for the tweets made by the most influential users — those with the most Klout and followers — we get a different set (although they have one tweet in common!)

#1 “Tech and knowledge industries are thriving, yet jobs discussion always centers on manufacturing. Why not be realistic about jobs?”

#2 “Mr. President, why should you not be held responsible for your silly prediction that unemployment would stay below 8%?”

#3: “Why do we have 1.5 million fewer jobs than we did before the stimulus when the # of ATMs is unchanged?”

“By Category”

Thanks to our manual grouping of tweets, we can also figure out which categories are the most popular (even if the team chooses some other specific tweet to represent it). There’s always a bit of subjectivity involved when making some of the decisions here, but most of our results should surprise nobody:

#1 Legalization of Marijuana (4,911 total tweets and retweets)

#2 Jobs (2,024 total tweets and retweets)

#3 Taxes (1,800 total tweets and retweets)

#4 Economy and Debt Ceiling (442 total tweets and retweets)

The one surprise is the winner of this category — marijuana laws.  Regardless of which side of this debate his opinion happens to fall, if the president doesn’t address this point during tomorrow’s discussion, somebody is avoiding the data. By a landslide, this is what the Twitterverse wants to know.

The post-mortem

This section of the blog post was added AFTER the Town Hall event. How did it turn out?

THE GOOD

What did we get right? We feel we matched the major categories of things Obama focused on — tax cuts, the debt ceiling, and job creation — but that wasn’t very difficult to do. More specifically, we nailed the third tweet Obama answered:

Tech and knowledge industries are thriving, yet jobs discussion always centers on manufacturing. Why not be realistic about jobs?

This one was right on our dashboard, and we had singled it out because @Kim, a very high-Klout user, was one of the many who put their weight behind it. The White House chose the earliest version of the tweet, by @dmscott (with slightly lower Klout), but the logic is clear — this tweet was chosen for the high-level support it received.

Another one we expected, but for a completely different reason, was Obama’s 8th:

Mr. President, In several states we have seen people lose their collective bargaining rights. Do you have a plan to rectify this?

According to our calculations, at the time of the Town Hall, this tweet had been mentioned, retweeted, or RT’ed over 182 times — making it the 4th-most active tweet in the Jobs category, and 14th overall. This is high enough to be on our list of likely predictions, although not high enough to earn one of the few spots on the Dashboard itself. Even though the tweeter, @pmglynn, is not very influential (with only around 100 followers), the question itself generated a lot of discussion and interest. It was an excellent candidate, and both TwitSprout and Obama appear to think so.

THE BAD

We definitely stand by all the predictions we made — there was solid reasoning behind each of them, and we’re confident they would have made excellent questions; however, there are a couple of predictions we didn’t make that, in hindsight, probably would have made sense.

The most obvious omission is @johnboehner’s provocative tweet:

After embarking on a record spending binge that’s left us deeper in debt, where are the jobs?

At the time of the Town Hall, this question had been propagated 79 times: a nontrivial amount, but only enough to place it in 10th place in the Jobs category, and much lower overall. There are two things that should have tipped us off about it, however:

• Even though @johnboehner’s Klout is only 69, his influence on President Obama is considerably higher. His position as Speaker of the House undoubtedly played a role in Mass Relevance’s decision. Of course, there’s no automated tools that would detect this. We simply should have thought of it.
• Even though the full form of the tweet had only 79 propagations, the snappier version “Where are the jobs?” by itself had 341 of them — enough to place it in 2nd place in the Jobs category, and 6th place overall, which would certainly have gotten our attention — except that this tweet didn’t make sense to us, since we didn’t see it in context with Boehner’s tweet (they were far apart in our rankings). Of course, we had algorithms to detect subtle rewordings between tweets, but these two didn’t trigger them.

THE UGLY

We might as well come out and address the elephant in the room. It was obvious to anyone who had followed our predictions, those of Simply Measured, or paid even casual attention to the #AskObama stream, that there was one topic getting a large share of the buzz: the legalization of marijuana. Although the topic of the Town Hall was ostensibly about the economy, it was becoming clear that people did think of this as an economic issue. By our measurements, the two most popular tweets of the entire stream, were:

Would you consider legalizing marijuana to increase revenue and save tax dollars by freeing up crowded prisons, court rooms?

Pres. Obama: Why can’t we discuss legalizing cannabis to create jobs and save millions annually on enforcement?

These two messages accounted for more impressions than every question about taxation put together! An event that emphasizes popular opinion as a tool for accountability could surely not fail to address these questions in the depth they deserve; but, of course, the topic was notable only for its absence. The President discussed the war on drugs merely tangentially, for less than half a minute.

Ah, politics as usual : )

Thanks to Dan and and his team for pulling two all-nighters to produce this post!