Making big data real for businesses (part – II): Leveraging social media data to identify key consumer insights
Big data analytics, more specifically in this case social media analytics can give companies a competitive edge if the insights can be made available to key decision-makers at the right time. Metrics like number of followers, shares and visitor count are no longer sufficient to monitor brand health and many firms struggle with the real time implementation of tools which translate big data into pertinent insights and metrics.
As mentioned in part 1 of this blog series, it is important for organizations to assess how investments into social media analytics can help them meet business objectives. An organization can gather social data by:
- Buying data from vendors
- Training an in-house team to capture data from publicly available APIs like Twitter Public API
- Leveraging digital assets like CRM websites, mobile apps/sites and other trade & ad related sources
Partnering with data analytics companies and leveraging consulting services will help businesses speed up the process while remaining cost-effective. Alternatively, they can create an in-house integrated ecosystem that has the right talent and infrastructure to handle analytical requirements internally. Once the required infrastructure is in place, social media “listening posts” can be set up.
However, social media analysis is fraught with a range of issues such as:
- Most of the data from social media is unstructured and takes the form of text, images and other media. Analyzing this on a real time basis is a challenging task
- There is a lack of obvious demographic markers, which are usually masked for privacy reasons and makes it difficult to identify and classify users into designated segments
- There is high level of inextricable noise content in the data due to misleading conversations and non-standardized templates which lead to the capture of irrelevant data. There are no accurate means to derive the context of a word/phrase. ‘Hate’ can differ depending on the context. For e.g.; “I hate it when we run out of ice cream in the middle of a party.” The usage of the adjective ’hate’ here, does not translate into an opinion about the brand
By deep-diving into the nuances of the individual challenges and their interrelations, organizations can create an effective social media analytics roadmap to enhance business decision making. To make the most of social media data, they need to employ an interdisciplinary approach by combining mathematical expertise, a data analytics platform, technological knowhow, apply business context along with an understanding of behavioral sciences.
Managing unstructured data using the right technology:
Modern technology platforms such as Hadoop or pay-as-you-go cloud services like Amazon web services can help manage unstructured data to a great extent. Organizations can also establish a sandbox environment to explore the capabilities of unfamiliar software like R, Python or Java.
Reducing noise by capturing a core set of social conversations from users: An understanding of the online lingo of the customers can help businesses compile detailed search strings and build comprehensive dictionaries around topics to flag the relevant conversations. Techniques like Topic modelling that use packages from R or Python can also be implemented to generate common topics and associated words. Irrelevant topics can be excluded and authors sharing relevant content can be classified, retained and studied. Over time, successive iterations of this process will improve the classification of users into the segments, while adapting itself to changes in the data.
Fostering an integrated decision support ecosystem:
The final road block is the inherent operational model of most present day businesses where the data gathering process and the actual analysis occur far away from the organization’s business decision making process. To draw maximum benefits and keep pace with changing times where real time situations require real time decisions, organizational barriers will need to be re-thought keeping business decision makers involved in the actual data analysis and insight generation process.