BEHAVIOURAL INSIGHTS FROM BIG DATA MULTIMODAL ANALYSIS
Valerio Rughetti, Blerina Sinaimeri
Instructional goals
This course aims to equip students with the ability to extract, integrate, and interpret behavioural insights from diverse data modalities, such as text, images, audio, and structured digital traces, to inform marketing decisions. Students will learn how to apply machine learning and data visualization techniques to real-world behavioural data, critically assess ethical implications, and communicate findings effectively through dashboards and storytelling. The course emphasizes hands-on analysis, critical thinking, and the integration of behavioural theory with advanced data techniques.
Prerequisites
Prior knowledge on basic data manipulation, and data visualization. Basic proficiency in data-driven reasoning is essential.
Intended learning outcomes
By the end of the course, students will be able to identify and integrate multimodal data sources relevant to behavioural analysis in marketing contexts. They will be able to apply appropriate machine learning techniques to extract behavioural insights, evaluate the ethical implications of their analyses, and communicate results effectively using advanced data visualization and storytelling methods. Students will also develop the ability to critically assess dashboards and predictive models, design basic behavioural experiments, and formulate data-driven marketing strategies grounded in consumer behaviour evidence.
Course Contents
The course covers the collection, integration, and analysis of behavioural data from multiple modalities, including text, image, audio sources. Topics include multimodal data fusion, sentiment and emotion analysis, visual content analytics, behavioural clustering, and recommender systems. Emphasis is placed on ethical and legal considerations, such as GDPR and responsible AI use. The course also addresses advanced data visualization, storytelling techniques, and the design of behavioural experiments. Case studies and hands-on project work will guide students in applying these methods to real-world marketing problems.
Reference Books
All the material needed for the course will be available on the website of the course.
Teaching Methods
The course consists of lectures complemented by practical lab sessions and group project works.
Assessment Method
Individual assessment during course - 1/3 of the final grade Final individual exam + project discussion - 2/3 of the final grade
Thesis assignment criteria
To be discussed with the professors.
Week 1
* Course Introduction and Expectations
-Overview of course objectives, structure, grading
- Examples of multimodal behavioural data in marketing * Behavioural Science Meets Big Data
- Behavioural marketing frameworks: biases, decision processes
- Mapping psychological insights onto digital traces - Intro on KNIME / Orange
Week 2
* Types of Data in Behavioural Analysis
- Structured (CRM, survey), unstructured (text, image), semi-structured * Digital Footprints and Tracking
- Sources: clickstream, app usage, social media, sensors, eye-tracking
Week 3
* What Is Multimodal Data?
- Definitions, challenges (synchronization, sparsity), relevance - Hands on KNIME / Orange * Integration Techniques
- Feature-level vs. decision-level fusion, entity resolution, schema alignment
Week 4
* Text Mining in Marketing
- Preprocessing, keyword extraction, topic modeling - Hands on KNIME / Orange * Sentiment and Emotion Analysis
- Lexicon-based vs. ML-based sentiment classification in KNIME/Orange or Python
Week 5
* Visual Content and Engagement: Computer vision basics: image tagging, feature extraction, branding cues * Case Study: Instagram, Pinterest, YouTube: Analyzing campaign images and consumer response through visual features
Week 6
* Behaviour Prediction with Machine Learning: Predicting churn, purchase, clicks using multimodal features * Recommendation Systems for Behavioural Targeting: Collaborative filtering, hybrid models, explainability
Week 7
* Customer Behavioural Clustering: Sequence mining, time-series patterns, path analysis * Beyond RFM: From Descriptive to Predictive Clustering
: Unsupervised learning with multimodal inputs
Week 8
* A/B Testing and Behavioural Experiments: Test design, KPIs, interpretation pitfalls * Uplift Modeling and Optimization: How to identify treatment-sensitive groups
Week 9
* Designing Insightful Dashboards: Dashboard best practices, misleading visualizations, persuasion techniques * Critiquing and Improving Visualizations: Student-led critique session on real-world dashboards
Week 10
* Voice and Audio Cues in Behavioural Analytics: Emotion from tone, call center analysis, acoustic signal features * Multimodal Emotion Recognition
: Combining voice, facial expressions, and text (overview and demo)
Week 11
* Ethics in Behavioural Data Use: Dark patterns, manipulative nudging, transparency, bias * Legal Guidelines and Responsible AI : GDPR, AI Act, consent mechanisms, fairness in predictions
Week 12
Exam preparation: Practice oral exams, integration of concepts, Q&A