With yet another stream of data to analyse from social media, there’s a budding industry of companies promising killer tools to keep tabs on whats being said about your brand. However, there are some real question marks regarding how some of the most interesting metrics are computed and presented.
I’m not going to name and shame, but I recently received a sample report from a social media monitoring company to demonstrate the level of insights they could provide. Reading through the report, it presented some pretty standard metrics and charts such as number of mentions and searches.
However, when it comes to social media, the more intriguing aspect is not merely how frequently your brand is being featured but what people are actually saying; so-called sentiment analysis.
To my surprise, the first example of a “positive mention” in the report refered to the former CMO of the company in question as greedy. I’m not sure I would categorise that as a positive mention. Curious about how this seemingly inaccurate analysis happened (and even made it into the top of a sample report), I clicked through to the original page in the search for clues. Except for the fact that the text was long and used some complex words, there wasn’t any dead giveaway to how the error occurred.
While I appreciate that you can’t expect any machine to interpret something as complex has human language (with a pinch of irony and sarcasm thrown in) with hundred percent accuracy, it highlights the fact that measuring the softer aspect of social media is a whole other beast than your traditional web metrics.
So what are marketers to do? As with most tools, as long as you’re aware of its short comings, you can still use it. Web analytics tools typically present different absolute figures, but the trends should be fairly similar which is most important for optimisation. However, if the natural language engine miss-interprets the true sentiment of posts and tweets, you might be in dangerous waters.
Related posts:
{ 5 comments… read them below or add one }
Technology should serve to assist human analysis, not replace it. Your example is a great one on why there should be serious reservations on trusting the results of automated sentiment analysis. And to compound the issue, this all assumes that the content being analyzed is comprehensive and qualified. If the human factor is left out of this stage, the “machine” may not even be analyzing the right conversations. I’m biased but to answer your question, marketers may be better served to skip automation and rely on human analysis in the first place.
Thanks for sharing, Mike.
I’m with you: automated sentiment analysis isn’t close to 100% perfect yet. As you mentioned, Magnus, sarcasm and irony will throw things into the ditch pretty quickly. But it sure seems like a big step in a very interesting direction. If these tools get me even a little way down the analysis road, there’s value for me there. The best tools will enable me to make decisions about which posts to keep and which to toss from within the admin. It’s going to be interesting to watch how these tools evolve.
Thanks, man.
Margaret (@blocheads)
Hi Mangus – yes good points which evokes the overused cliche caveat emptor.
> However, if the natural language engine miss-interprets the true sentiment of posts and tweets, you might be in dangerous waters.
But it’s more than just NLP i.e. you need the right mix of embedded semantics which of course is keen bias of mine
I just started taking a look at http://www.socialtality.com/ which looks intriguing ( disclosure: do not have a working relationship with them in case you’re wondering )
- Steve
Thanks for the comments. Some interesting links there, will have a look.
Think Mike’s comment is spot on that it’s simply key to involve human analysis.
Hi Magnus,
absolutely agree with the points raised.
Over at Brandtology, this is precisely the ‘pain point’ that we seek to address. We are one of the largest Business and Brand Online Intelligence service providers that combines technology, processes and trained professionals to deliver accurate and relevant intelligence to global organizations.
With more than 100 Social Media Specialists in 10 locations around the world, we are able to verify and enhance our automated machine analysis in more than 9 different languages. This ensures very high accuracy and relevancy of the analysis reports provided to our clients who do not have to waste a moment sieving through irrelevant data.
Ashley Lim
Social Media Consultant
Brandtology