Automated Data Integration and Machine Learning for Enhanced Social Media Marketing Analysis

https://drive.google.com/file/d/18MhI005pq-IMCbSNZDwqi-C7DgELMEcw/view?usp=drive_link

In today’s digital era, the abundance of data generated on social media platforms presents a valuable resource for extracting marketing insights. This thesis, born from an internship project at Mediamente Consulting s.r.l., addresses the pressing need of a customer for efficient data integration and automation in the context of social media marketing campaign analysis. The primary objective is to develop a robust Data Integration model to streamline the visualization of social media marketing campaign performance. The traditional approach of manually downloading and aggregating standard reports from YouTube and LinkedIn analytics tools is time-consuming and errorprone. To replicate the company workflow, our thesis outlines a comprehensive framework comprised of four key stages. The initial stage, Data Ingestion, involves the extraction of data from various sources with the utilization of REST APIs provided by the respective social media platforms. In cases where APIs are not available, a web scraper is employed, ensuring a comprehensive data collection process. Following the Extract, Transform, Load (ETL) design pattern, the data is prepared for analysis. This phase involves the integration of data from diverse sources into a unified model, ensuring consistency and coherence for subsequent analyses. Leveraging supervised and unsupervised Machine Learning techniques, data points are clustered into meaningful groups. Each record is then assigned a label based on the corresponding marketing campaign, enriching data and facilitating richer insights into campaign performance. The reconciled and enriched data is stored in a Google BigQuery dataset, serving as a centralized Data Warehouse. This repository allows for complex querying and facilitates both Data Visualization and further Machine Learning analyses. The thesis acknowledges the potential for future enhancements to the developed system, including the incorporation of additional data sources such as Google Ads for comprehensive cost and revenue analysis. Moreover, consideration is given to the prospect of a full migration to a serverless cloud platform solution to enhance scalability and reliability, ensuring the system’s long-term viability. In summary, this thesis presents an innovative approach to addressing the automation of data integration challenges inherent in social media marketing campaign analysis. By optimizing the collection, preparation, and enrichment of data from wide-ranging sources and with the advantage of Machine Learning techniques, it offers a powerful decision-support resource for the customer who is seeking deeper insights and greater efficiency in their campaigns.