The ups and downs of measuring sentiment

Sentiment analysis is one of the oldest and most enduring aspects of article assessment. Over time, sentiment analysis is also one of the most useful guides to determine how a company, brand, or issue is perceived. By examining sentiment, you can easily see how PR efforts might be changing public perception, and it’s one of the simplest ways to set a benchmark.

Challenges

Although sentiment analysis seems straightforward, it has a number of challenges.

• Subjectivity– If you’re manually assigning sentiment to content and you have more than one person working on the account, you’re bound to have some significant differences when assessing a piece. What one person views as a positive mention might be viewed as neutral by another. Over the long term, with enough content analyzed, this will probably not matter as the volume will “smooth out” differences. However, if you have either a small data set or are looking at a limited time period, these differences may have an outsized effect.
• Context– Context always matters when assigning sentiment. The Red Cross and other relief agencies are frequently mentioned in articles about natural disasters. The articles may have a negative bent and tone due to the loss of life and destruction, but the direct mentions of the relief agencies providing assistance would be considered positive.
• Language variation and sarcasm– References to something being “sick” could be positive or negative, and sarcasm can sometimes be hard to detect. These types of linguistic quirks need to be watched out for when conducting sentiment analysis manually.
• Reverse logic – This comes into play if you are monitoring for your brand along with competitors. An article that carries a negative mention of a competitor and a neutral mention of your brand might best be categorized as either neutral or potentially even as a positive. Another example of this is when the client prefers not to have the brand mentioned in the news at all—in which case any mention is a negative mention, even if the article itself was neutral in tone.

Approaches

There are several things to keep in mind when tackling the above challenges.

• Consistency– When monitoring for sentiment over time, consistency will be your best friend and will provide a more reliable foundation on which to base decisions on sentiment. If your brand or client is frequently mentioned in articles as a matter of reporting, decide at the outset if you’ll be categorizing that type of mention as positive or neutral, and then stick with it. For example, if the client is a restaurant that frequently hosts a regular Rotary Club meeting, every time there’s an article recapping the meeting, your client will likely be mentioned. Decide how you feel that should be rated, and then stick with it.
• Automation– If you have a lot of content to rate, you may want to consider using automated sentiment analysis. If it’s a feature of your monitoring tool, talk to their customer service to see how you can adjust the sentiment rating to address any of the above challenges. If you do choose to use automated sentiment analysis, it’s a good idea to occasionally spot-check on important topics to make sure the correct sentiment is being applied.

Whether you use automated or manual sentiment analysis, keeping track of the tone of content is a good and effective way to measure your PR results. Because sentiment is such a logical way to assess PR effectiveness, it makes sense to make sure it’s as accurate as possible.

About the Author

Jennifer Zingsheim Phillips is a Strategic and creative communications executive with more than 20 years of experience providing PR counsel to clients across a wide range of industries. Dynamic, adaptable, and dedicated to superior client service. Published in business magazines, web content creation experience, podcasting, and fiction writing. Freelance writing with focus on public affairs (outreach campaigns, grassroots, coalition building and general communications) and content marketing.