going foward

For the last week of Rest of You, I tried to automate all of the trackers that I made in the past seven weeks, and feed them into a locally hosted visualization. I didn’t get quite that far, but am getting there:

I liked the viz I created last week, so my obvious first step was to automate the python scripts that retrieved the data. The productivity scores are requested from the RescueTime API, and my Chrome history is just a JSON converted from a SQL file that Chrome locally maintains. Both python scripts update their datasets hourly. The JSONs are linked to a local d3.js visualization, so I can refresh any time to see an update. The design could use some cleaning up, but the data is there.

The next step was to automate the entrainment process as well. Rather than having to monitor my own mind-wandering, then muster the self-discipline to open up my six-minute entrainment meditation, a python script checks my productivity levels and opens the webpage if they dip below 50%:


If I “meditate”, the event gets encoded as a bright teal line in the viz, so I can see how the entrainment affects my behavior. I was apparently pretty productive this week, and didn’t trigger the entrainment script at all, so I had to waste some time this morning to see it in action:

Finally, I also got around to hooking up my python keylogger to IBM Watson’s Tone Analyzer API. I was hoping to feed its analysis into my visualization as well, to see if mood was correlated to mind-wandering, but the results were a little questionable:

Moving forward, I’d also like to integrate Affectiva’s attention, engagement, and valence variables into the viz—I just need to figure out how to open the locally-hosted demo in the background, so it captures a candid moment for a few minutes every hour. I would also like to figure out how to automatically count the number of windows and browser tabs I have open, as I’m pretty confident these are reliable markers for stress and mind-wandering. Finally, I also plan to create a questionnaire that prompts me every hour, to have corroborate the correlations of all my tracked data.

QH W5 HW: Quant Self Intervention

I’ve been in grave danger ever since discovering Sex and the City on Amazon Video.

I normally avoid all media like the plague, partially because I find the content objectionable for SJW reasons, partially because of my own insecurities which I will probably intellectualize away with SJW reasons forever, but mostly because I can’t trust my addictive personality to watch even one video without immediately downward-spiraling for ten hours into a pit of shame and self-loathing.

The one exception I make is for SATC, for no good reason other than the fact that I was able to cobble together the entire series on the cheap during my phase as a Housing Works regular (this is generally how I consume media: once everyone’s completely over it). Despite this DVD collection’s efficacy as a coping mechanism, its inconvenient—and now obsolete, thanks to Apple’s sanctimony— physical form was never a threat to my daily functioning.


By some loving grace of God, I only discovered this year that SATC was included with Amazon Prime, but somehow I’ve already watched three seasons of it, plus the first movie, and 1.5 seasons of SJP’s new show, Divorce, which I highly do not recommend, and only watched while nursing the sugar headache that SJP’s younger self tends to cause (I’m Team Kim).

So this week, I decided to put an end to this nonsense. RescumeTime, a tracker that I installed near the beginning of QH, converts my activity into a handy “productivity” score—one that I get to define by categorizing any website or application I use on a scale from “Very Productive” to “Very Distracting”:

I decided to use python to grab this score every hour through RescueTime’s API, and open my slightly meditative, mostly masochistic, brain entraining p5 sketch if the score dips under 50%:

Here it is in action:


The idea of this intervention is to allow myself the room to indulge in “very distracting” activity if I need it, but to catch myself before I spiral out of control and have to live with the concomitant guilt forever. The “entrainment” part—regardless if it actually entrains my brain or not—is, at the very least, a way of resetting myself and my OCD.

This d3 sketch illustrates how my activity changes before and after “entrainment”, and I’m working on automating the chrome history collection so that I can have a viz that automatically updates in real(-ish) time.

talking to the elephant

This week, I did a small survey on talking to my subconscious. I thought the assignment was a perfect excuse to try a sensory deprivation tank—at least, the Brooklyn version of one:

Unfortunately, the session for me was merely an hour’s worth of boredom and discomfort; not at all the spiritual experience one guest described, in which “God spoke to [her] in [her] own voice.”

The assignment was also a good excuse to break out my entrainment studies from last semester, and I created a viz of my chrome history (a good metric of mind-wandering for me) one hour before and after “entrainment”.

Lastly, I made an appointment with Dr. Donatone over at the Health and Wellness Center across the street. After a short interview, she decided I required hypnosis for my hypodermic needle phobia. The hypnosis lasted about ten minutes, and was simply a form of meditation in which, after a short body scan, I was forced to imagine cartoon drawings of needles and tourniquets and report aloud how my body responded to them (spoiler: my response was absolute fear and loathing across the board, even when she asked me to image a syringe made out of balloons).

QH W4 HW: what i learned this week


Several sources of inspiration for this week’s assignment:

Humanist service design guidelines: Ie, rather than keeping users hooked on a product, we should be designing products that facilitate our human needs: sleep, human connection, engagement with the physical world, etc. By the Center for Humane Technology.

This Vox article also galvanized me to reconsider the importance of social connection on mental illness, while reminding me of the town of Geel, which treats mental disorders with social inclusion.

Anytime you consider anything less than everything you are missing something: J. Paul Neeley’s beautiful talk on considering everything really validated my insatiable impulse to collect the data possible.



two more years

I wanted to continue my analysis of my flickr photostream by comparing last week’s recently created dataset to an older set of photos. Since there was no way (that I could find) of “walking” backwards through a photoset with the python flickrapi module, I thought I would “walk” through my entire photostream of 20,000 photos while pushing necessary information into a list as I went, then reverse-feeding that list into the Clarifai API for analysis:

After a few false starts (in retrospect, this was probably flickr warning me off), I was able to walk through about 11,000 photos before flickr shut me down for exceeding their rate limit. Now that I know flickr has a rate limit, I suppose the ideal way to get this job done would be to pay away Clarifai’s operations limit, and run my entire photostream through its API in one go. The one downside is that this would take hours, since simply looping through half my photostream with flickr’s api alone took almost two.

The other obstacle is that I’m an inveterate miser with zero present income, and so am still waiting to hear back from Clarifai on a student “collaboration” discount. A girl can dream.

In the meantime, I crudely copied the last four thousand URLs that my aborted python script had printed and threw them into Clarifai. This second “dataset” accounts for a time period from October 2015 until July 2013. I then loaded the results into the same d3 sketch as last week.

One thing to note is that for these new visualizations, I gathered more “concepts” per photo than last week, in hopes that more interesting/abstract concepts—like “togetherness”, which may inherently inspire less confidence than an objective prediction like “people”—might emerge. And they did:


Top 10 concepts in 2017: 1) people, 2) outdoors, 3) no person, 4) adult, 5) travel, 6) woman, 7) indoors, 8) man, 9) portrait, 10) nature

Top 10 concepts in 2014: 1) no person, 2) outdoors, 3) travel, 4) people, 5) nature, 6) one, 7) architecture, 8) old, 9) portrait, 10) sky

At least for the more recent set of photos. One thing that’s immediately obvious: I photographed way fewer “concepts” in 2014 than I did in 2017. I also took fewer photos.

Another striking observation is that “no person” is byfar the most common concept of the 2014 photoset, while “people”—literally the opposite—is the most common for 2017. Looking at the top 10 concepts, one could definitely speculate that I had more company in 2017 than I did in 2014.

While this visualization does its job as an abstract overview of the data, I wanted the photos themselves to tell the story. So on my click, I had the page spit out the photos that were trapped inside their respective bars.

“People” in 2017: 12 total unique individuals out of ~126 photos

“People” in 2014: 8 total unique individuals (not including office parties!) out of ~220 photos

Comparing the “people” category for both photosets, I clearly saw fewer unique people over a longer period of time (ie, how many photos fit in the window) in 2014, while in 2017, I saw more unique individuals over a shorter period of time, even while half the photos displayed were repeats.

Also notable was that the 2014 sample seemed to be entirely processed in instagram, which may be coincidence; I probably just happened to have choosen a period where I backed up all my instagram files at once? Will have to look into that one, but it’s amazing to me that I bothered to process so many mundane photos through instagram, though they would never be posted publicly. Perhaps I truly thought a filtered reality looked better, or maybe I was just constantly looking for an opportunity for public validation.

So what’s with the discrepancy? It will surprise no one to learn that I was extremely depressed from 2013-2014, a period which overlaps with the second photoset in this experiment. This analysis truly corroborates the idea that mental health is a social issue.

For my next steps, I’d like to train a custom model to identify selfies, which I believe is a strong marker (at least, personally) of depression. I’d also like to incorporate Clarifai’s color model to the workflow, run my Instagram history through it, and display it as a time series visualization. I’m absolutely certain this will be able to map my history of depression with excruciating accuracy.

what I wanted to remember

For this week’s Rest of You assignment, I decided to run my Flickr photostream through Clarifai, an image recognition model that you can use with an extremely effortless API. Thank God for that, because the Flickr API was anything but.

This was a really good (and hard-earned) moment for me.

I’ve basically been backing up my photos to Flickr ever since I started using smart phones in 2013; I’m also an adamant Samsung Galaxy user solely because of their superior cameras. As such, I figured my photo backups would be a rich database to find trends in, and considering that I had more than 20,000 photos backed up on Flickr, I decided to try to automate the process by making both APIs talk to each other.

Sadly, they only got through 4,800 photos before I hit my Clarifai processing limit—ie, somewhere in the middle of 2016. Unfortunately, the really telling data is around 2013-2015, so I’ll have to sign up with my other e-mail accounts to finish the rest of my history, or slowly work through it for the next five months.

Here’s a screenshot of Clarifai’s response to my API call, along with my code:

Tragically, I didn’t consider JSON-fiying these responses as they came in (who knew there was a difference between single and double quotes in JSON world? Probably a lot of people.), so there was a subsequent and painful formatting process to follow. Thanks, boyfriend! Note to self: learn VIM for data processing.

Below is a more legible example of their response, in JSON format, along with the respective image:

After the scary Python part was over, it was time to speak my slightly more native language, d3.js. I isolated the concept objects, threw out all but Clarifai’s top two predictions for each photo, and applied a rollup to count the unique keys. As you can see from the screenshot below, over the past 1.5 years, I photographed 451 unique concepts:

Some of these concepts were redundant (like “people” and “adult”), which will have to be dealt with later, as the foregoing work took ages to do! Below, the top twenty concepts sorted in descending order:

And here is the same data visualized in a quick d3 sketch:

ML4A with Gene Kogan


paperspace or colab

ml4a suite:

  • ConvnetOSC meant for wekinator
    • ml4a ConvnetOSC + wekinator + processing or p5 w/ OSC
  • KeyboardOSC: controls keyboard with wekinator
  • AudioClassifier: classify sounds that you make

more on http://ml4a.github.io/guides


pupil labs is awesome

Pupil Labs detects your pupil size and gaze position pretty well, and lets you export excessively granular data. Here’s a map of my tracked gaze as I got a tutorial from Cristobal:

According to the Thinking, Fast and Slow chapter I read last week, pupil dilation correlates with the exertion of mental effort. With this in mind, I decided to do another reading experiment in two vastly different environments: 1) on the floor on pb&j day, and 2) alone at home. I was particularly interested to see my eye movements, as my recently developed deficit in attention requires me to read sentences, and even entire paragraphs, multiple times after realizing that I’ve looked at the words without actually processing them in the slightest.

Here’s the map of my eye movements in the second environment:

Looks cool, but is not very informative, so I decided to throw together a quick p5 sketch (with d3 support) to animate the movements over time, and add in the corresponding diameter data.

Here’s a new visualization with the same at-home dataset:

And here’s one for pb&j day:

So the image positioning for both are eyeballed, but it’s pretty clear by the density of the movement data for the latter set that sitting next to the pb&j cart between classes upsets my concentration, and forced me to reread the same lines an embarrassing number of times. Pupil diameter (concomitantly encoded in the position tracking lines, as well as supplementally represented with the circles the lower-right corner) was also on average larger at school than in my quiet home environment, suggesting that more effort was required at the former.

That is, if you can ignore the extremely anomalous data that came in at the end of the at-home dataset, which explains the huge circle left behind in the supplemental diameter visualization.

I tried uploading the sketches to gist, and you can try loading the at-home viz here, but the huge dataset will probably cause the browser to freeze at some point. Will try to clean up the data later and re-upload.

QH W3 HW: Dear Data

For week 2’s assignment, my boyfriend and I logged our feelings every hour for about four days. We had recently realized just how differently we perceived and experienced the same relationship, and thought it would be interesting to do a comparison.

The variables we manually tracked on a Google spreadsheet were time, reaction, whether it was a positive or negative feeling, description of the event that triggered the reaction, and an overall “satisfaction score”:

On Monday, he forfeited his log and I mapped both datasets on the postcards Matthew gave us in class:

First dataset is his, bottom is mine

I choose a simple bar graph in order to flatten an extremely nuanced and qualitative dataset into something visually digestible. The bars were encoded with two colors: pink for the score, and a secondary color that indicated the type of trigger. The fluctuations in bar height in my boyfriend’s postcard illuminated just how much my boyfriend’s anxiety and sensitivity affects his experience of our relationship, while my rather uniform results illustrated how generally unperturbed I am, and/or how oblivious I am to his emotions.