Emotion detection through smartphone typing

Abstract

Keystroke analysis or typing analysis for detecting human emotion refers to the method of recognizing emotion based on the typing pattern by checking the various features like typing speed, typing mistakes, usage of special characters, number of sessions, number of characters in a session, number of words etc. This approach can be considered desirable in human computer interaction because the data used is rather non-intrusive and easy to obtain. However, there were only limited investigations about the phenomenon itself in previous studies. This work aims to examine the source of variance in keystroke typing patterns caused by emotions. As it is smartphone typing based emotion detection, first I have designed a custom keyboard in android which can collect these typing pattern features. Then to get these typing data in csv file for feature extraction firebase analytics cloud service is used. Along with typing data self-reported emotion states (happy, sad, excited, stressed) are also collected. After that in python various features are extracted to train multiclass classification model Like decision tree, random forest, K-Nearest Neighbors. The experiment is performed on three different users. Here physiological, psychological and behavioural modalities are combined that is to check whether using heartrate along with typing data improves the accuracy or not two user's heartrate data along with typing data is collected and then compared. As collected emotion classes are imbalanced, to check the accuracy precision, recall and f1 score are measured. The result shows that using heartrate along with typing data increases the accuracy for emotion detection. The result also shows that emotional influence on keystroke pattern changes based on person's characteristic. So, by this way, this work is a small step towards answering the question that how strongly is smartphone typing correlated to our perceived emotion.