Skip to content

Investigating Semantic Roles for Emotion Role Prediction

Tushar Dhyani, Maximilian Wegge

Institut für Maschinelle Sprachverarbeitung, University of Stuttgart

A detailed scientific report on this project is available on Researchgate. Please feel free to read and share your feedback with me over email

Emotion analysis primarily focuses on classifying, predicting and retrieving emotions and their related properties from text. However, only few research was conducted towards analyzing the semantic roles of emotions, i.e. who is experiencing which emotion, what caused it and what or whom is it directed towards. This project investigate the influence of semantic role labels on emotion role prediction. Building on top of previous approaches and resources, I've implemented a framework for predicting emotion roles using different features with co-researcher Maximilian Wegge. We find that semantic role label features have no significant influence on the task and identify two possible reasons for that.

This project was conducted under the supervision of Dr. Roman Klinger

What is Semantic Role Labelling?

In natural languages, we understand the who, why, which, what, etc. in a sentence by understanding the words and phrases and their semantic meaning. This semantic meaning helps a lot in understanding why emotion is triggered, towards whom it is directed, or what gets affected. The process of pointing these individual components from a sentence is called Semantic Role Labelling or is sometimes analogous to Emotion Role labeling.

Semantic role labeling provides us with fine-grained control over emotion classification or prediction as we get a clear understanding of it.

emotion_roles Here, who (experiencer) feels which emotion (indicated by cue), which object, person, or instance the emotion is directed towards (target), and what evoked the emotion in the feeler (cause).

Datasets used:



We evaluate our approach by calculating the Jaccard value and reporting the F1 score of the overlapping spans. The Jaccard index is defined as $$ J(A,B) = \frac{| A \cap B |}{| A \cup B |} = \frac{| A \cap B|}{|A| + |B| - |A \cap B|} $$

In our case, we take A as our prediction span and B as the target span. We calculate a span to be correct only if the predicted and the target span have 80% overlap.


We run our experiments on the testing split of the following corpora and compare the performances using jaccard score. For our evaluation, we set the overlap threshold to 0.8 and consider only the spans that have atleast 80% overlap compared to the training spans.

Dataset Method Exp Tar Cue Cause
REMAN HMM 0.228 0.021 - 0.011
biLSTM-emb 0.435 0.051 - 0.101
biLSTM-emb+srl(all) 0.494 0.025 - 0.078
biLSTM-emb+srl(slct) 0.465 0.115 - 0.139
Table 1: Demonstrate the results of our experiments on Reman corpus. Our model achieves higher scores compared to our baseline using HMM model.
Dataset Method Exp Tar Cue Cause
GNE HMM 0.370 0.062 0.293 0.321
biLSTM-emb 0.595 0.228 0.441 0.654
biLSTM-emb+srl(all) 0.591 0.312 0.443 0.638
biLSTM-emb+srl(slct) 0.553 0.278 0.408 0.613
Table 2: The results of our experiments on Good News Everyone corpus. Our model's performance still remains higher compared to baseline HMM model.
Dataset Method Exp Tar Cue Cause
ES HMM - - - 0.215
biLSTM-emb - - - 0.593
biLSTM-emb+srl(all) - - - 0.591
biLSTM-emb+srl(slct) - - - 0.553
Table 3: Results of our model on Emotion Stimuli dataset for cause only. The results show that SRL features do not really impact the performance on respective corpus.
Dataset Method Exp Tar Cue Cause
ET HMM 0.0 0.228 0.122 0.124
biLSTM-emb 0.0 0.383 0.170 0.047
biLSTM-emb+srl(all) 0.0 0.434 0.134 0.085
biLSTM-emb+srl(slct) 0.0 0.443 0.136 0.070
Table 4: Results of our model on Electoral Tweet dataset. The results still suggests that the model does not achieve any improvement using the SRL features.
Dataset Method Exp Tar Cue Cause
ECA HMM - - - 0.025
biLSTM-emb - - - 0.155
biLSTM-emb+srl(all) - - - 0.206
biLSTM-emb+srl(slct) - - - 0.152
Table 5: Results demonstrate no improvement using the SRL all and selected features in improvement of emotion detection.

A detailed scientific report on this project is available on Researchgate. Please feel free to read and share your feedback with me over email