Tracking Emotional Dynamics in Chat Conversations: A Hybrid Approach using DistilBERT and Emoji Sentiment Analysis
arXiv preprint arXiv:2408.01838, August 2024
Ayan Igali, Abdulkhak Abdrakhman, Yerdauit Torekhan, Pakizar Shamoi
School of Information Technology and Engineering, Kazakh-British Technical University
Abstract
Computer-mediated communication has become more important than face-to-face communication in many contexts. Tracking emotional dynamics in chat conversations can enhance communication, improve services, and support well-being in various contexts. This paper explores a hybrid approach to tracking emotional dynamics in chat conversations by combining DistilBERT-based text emotion detection and emoji sentiment analysis. A Twitter dataset was analyzed using various machine learning algorithms, including SVM, Random Forest, and AdaBoost. We contrasted their performance with DistilBERT. Results reveal DistilBERT's superior performance in emotion recognition. Our approach accounts for emotive expressions conveyed through emojis to better understand participants' emotions during chats. We demonstrate how this approach can effectively capture and analyze emotional shifts in real-time conversations. Our findings show that integrating text and emoji analysis is an effective way of tracking chat emotion, with possible applications in customer service, work chats, and social media interactions.
Method
Two channels, one signal
Chat text and emoji are treated as separate carriers of the same emotional state. A DistilBERT classifier reads the text; a sentiment analysis of the emoji reads what the text leaves out. In chat, these frequently disagree — the words stay neutral while the emoji does the emotional work — and a text-only model misses exactly that.
Baselines on a Twitter dataset
The text side is validated against classical baselines — SVM, Random Forest, and AdaBoost — trained on the same Twitter dataset, so the gain from the transformer is measured rather than assumed.
Results
DistilBERT wins, emoji adds what it misses
DistilBERT outperforms the classical baselines on emotion recognition. Layering emoji sentiment on top of it captures emotional shifts as they happen across a conversation, which is what makes the method usable in live settings — customer service, work chats, and social media threads — rather than only on archived text.
Key points
- Hybrid model: DistilBERT for text, sentiment analysis for emoji.
- DistilBERT beats SVM, Random Forest and AdaBoost baselines on the same data.
- Captures emotional shifts in real time, not just per-message labels.
- Targets customer service, work chats and social media.
Cite
@article{igali2024tracking,
title = {Tracking Emotional Dynamics in Chat Conversations: A Hybrid
Approach using {DistilBERT} and Emoji Sentiment Analysis},
author = {Igali, Ayan and Abdrakhman, Abdulkhak and Torekhan, Yerdauit and
Shamoi, Pakizar},
journal = {arXiv preprint arXiv:2408.01838},
year = {2024}
}