digital divination

Last year, as part of my thesis, I created several trackers in python and javascript to track various metrics: expressions, blinks, attention values from an emotion recognition API; word counts, backspace counts, and keystroke dynamics from a keylogger; sentiment analysis on those logs; browser activity through a Chrome extension.

Basically, I collected data on anything I could think of that might even be slightly emotionally charged, because I wanted to try to predict depression-adjacent emotions: mood, morale, stress, and fatigue. I ended up training RNNs for this task, which now make predictions every 15 minutes or so, as the data comes in. Before any of that happened, I did a quick analysis of this data (for my own benefit) to see if there were any correlations emerging. At the time, there was only about five months of data, so it wasn’t very meaningful, but since I was extremely short on time this week (due to thesis and this data science class at Steinhardt that’s way out of my league—sorry), I decided it would be appropriate/least fatal to revisit my dusty old Jupyter Notebook for homework this week—presently, I’m almost at one year’s worth of data.

Visualizations of the correlation matrix (at five months) and slightly more recent ML results can be viewed here. I’ll update it at some point… after I graduate.

Mood:

Another one of my trackers is an hourly self-reported questionnaire on my present emotional state (the data from which was used as labels in the training process), including an item that is a 1-5 scale rating of general “mood”. According to Pearson correlation, these are the digital behaviors that could prophesy positive mood:

  • avg_engagement (0.214871): this “engagement score” is from the emotion recognition API Affectiva
  • expressions (0.198212): my tracker counts the number of expressions that Affectiva registers in the minute that it records every hour
  • Joy_score (0.150085): this “joy score” is a sentiment analysis result from IBM Watson’s Tone Analyzer API on my hourly keylogs
  • stepCount (0.120612): not a digital behavior, but tracked digitally by the Google Fit app
  • avg_valence (0.101866): another Affectiva score on overall positivity of facial expression

These are the behaviors indicative of negative mood:

  • Fear_score (-0.107625): IBM Watson’s sentiment analysis score for fear
  • Sadness_score (-0.128840): IBM Watson’s sentiment analysis score for sadness
  • feedly_activity (-0.133028): I oftentimes mindlessly turn to Feedly, my RSS aggregator, when I’m exhausted an in need of a break
  • heart_rate (-0.145422): not a digital behavior, but calculated by openCV
  • tv_activity (-0.222593): I only really watch TV when I’m exhausted and want to dissociate from the world

Morale:

Another item from my self-reported questionnaire; these are the indicators for positive morale:

  • productivity_score (0.167960): a score calculated by RescueTime
  • backspace_count (0.123373): from my Keylogger
  • Joy_score (0.114208): sentiment analysis score from IBM Watson

Indicators for negative morale:

  • uniqueword_ratio (-0.127617): keylogger calculation of unique words/total word count
  • heart_rate (-0.151213)
  • feedly_activity (-0.167261)
  • tv_activity (-0.352525): I wasn’t kidding about the dissociation thing

Stress:

Positive indicators for self-reported stress:

  • current_tabCount (0.500686): hourly count of total tabs by my Chrome extension; I’m a crazy tab hoarder in general, but the tendency definitely is exacerbated by anxiety/existential angst…
  • current_windowCount (0.481969): see above
  • windows_created (0.106337): I wasn’t kidding about this browser thing either
  • tv_activity (-0.157775)
  • heart_rate (-0.255155)

Compulsions:

Positive indicators for self-reported compulsive behavior:

  • tv_activity (0.146173)
  • uniqueword_ratio (0.094941): this is weird…

Negative indicators (also weird):

  • tabs_activated (-0.128082)
  • tabs_created (-0.133961)