Neural Aesthetic Class Notes wk8

limitations of feed-forward NNs:

  • static, does not change over time
  • does not take advantage of context
  • inputs and outputs are fixed length

sequence to sequence: language translation

unit to sequence: image captioning

skip-thought vectors: arbitrary sequences of words (image to story)

dense captioning: multiple captioning for objects within images

text to image (stackGAN): https://arxiv.org/abs/1612.03242

Talking and Storytelling w1 reading notes

8 CLASSIC STORYTELLING TECHNIQUES FOR ENGAGING PRESENTATIONS

  1. Monomoyth (the hero’s journey):
    • Structure:
      • leaves home to a threatening, unknown place
      • overcomes great trail
      • returns home with newfound wisdom
    • Good for:
      • explaining what has brought you to the wisdom you’re sharing
      • bringing the message alive
      • showing the benefits of taking risks
  2. The Mountain
    • way of mapping the tension and drama in a story
    • doesn’t necessarily have a happy ending
    • Structure
      • scene is set
      • series of small challenges and rising action
      • climatic conclusion
    • like a TV series: each episode has ups and downs, all building up to a big season finale
    • Good for:
      • showing how you overcame a series of challenges
      • slowly building tension
      • delivering a satisfying conclusion
  3. Nested Loops
    • three or more narratives are layered within each other
    • Structure
      • the center = the most important story with the core of your message
      • outside layers elaborate or explain the central principle
      • the first story you begin with is the last story you finish, the second story you start is the penultimate you finish, etc
    • Good for:
      • explaining the process of how you were inspired/came to a conclusion
      • using analogies to explain a central concept
      • showing how a piece of wisdom was passed to you
  4. Sparklines
    • way of mapping presentation structures
    • very best speeches succeed because they contrast our ordinary world with an ideal, improved world—comparing what is with what could be
    • Good for:
      • inspiring the audience to action
      • creating hope and excitement
      • creating a following
    • MLK’s I Have a Dream speech
  5. In Media Res
    • Structure
      • narrative begins in the heat of the action
      • starts over at the beginning to explain how you got there
    • try hinting at something bizarre or unexpected, something that needs more explanation, to hook the audience
    • only works for shorter presentations
    • Good for:
      • grabbing attention from the start
      • keeping an audience craving resolution
      • focusing attention on a pivotal moment in your story
  6. Converging Ideas
    • shows the audience how different strands of thinking came together to form one idea
    • can be used to show the birth of a movement, explain how a single idea was the culmination of several minds working towards one goal
    • Good for:
      • showing how great minds came together
      • demonstrating how a development occurred at a certain point in hitsory
      • showing how symbiotic relationships formed
  7.  False Start
    • begin to tell a seemingly predictable story, before unexpectedly disrupting it and beginning it over again
    • good for talking about failures where you were forced to go back to square one and reassess; ideal for talking about the things that you learned from the experience, or some innovative way you solved a problem
    • quick attention hack which will disrupt your audience’s expectations
    • Good for:
      • disrupting audience expectations
      • showing the benefits of a flexible approach
      • keeping the audience engaged
  8. Petal Structure
    • organizes multiple speakers or stories around one central concept
    • useful if you have several unconnected stories you want to tell, or things you want to reveal, that all relate back to a single message
    • each petal should be a complete narrative in itself; evidence around your central theory
    • Good for:
      • demonstrating how strands of a story or process are interconnected
      • showing how several scenarios relate back to one idea
      • letting multiple speakers talk around a central theme

https://www.sparkol.com/en/Blog/8-Classic-storytelling-techniques-for-engaging-presentations

citizen science final proposal

Frontal alpha symmetry neurofeedback:

  1. test positive memory recall for alpha idling in right frontal area
  2. test negative memory recall for alpha idling in left frontal area
  3. test neurofeedback protocols for frontal alpha symmetry
  4. implement best protocol as regular training to test influence on mood

 

Other methods of altering brain activity

  • active:
    • alpha/theta training
    • meditation (mindfulness vs focused attention)
    • gratitude logging
    • positive autobiographical recall
  • passive (alpha band vs gamma):
    • photic driving
    • binaural beats
    • aromatherapy

Neural Aesthetic w6 class notes

  • Generative models synthesize new samples that resemble the training data
    • applications: visual content generation, language models (chatbots, assistants, duplexes), music, etc
    • Models the probability distribution of all possible images; images that look like the dataset have a high probability
  • PCA projects down in lower dimensions and back out
  • latent space: space of all possible generated outputs
  • later layers can be used as a feature extractor because it is a compact but high-level representation
    • distance calculations between feature extractors can be used to determine similarities between images
    • transfer learning
    • can use PCA to reduce redundancies, then calculate distances
      • images (points) can then be embedded in feature space
        • vectors between points imply relationships
  • Autoencoders reconstruct its inputs as its outputs; networks learns an essential representation of the data via compression through a small middle layer
    • first half encoder, second half decoder
    • can throw in labels for a conditional distribution
    • can encode images and get their latent representation to project outward
      • smile vector
  • GANs: circa 2014
    • hard to train
    • hard to evaluate
    • can’t encode images directly
    • structured like a decoupled autoencoder
      • generator > discriminator
        • generator: basically like the decoder in an autoencoder
          • takes in random numbers, not images
          • tries to create images to trick the discriminator into thinking they’re real
        • discriminator: takes in an input image (from generator), decides if it is real or fake
        • “adversarial”: trained to work against each other
  • DC GANs
    • unsupervised technique, but can give it labels
      • interpolations through latent space AND through labels
        • labels are one hot vectors
        • MINST: glyph between integers
  • Deep Generator Network
    • similar to deep dream; optimizes an output image to maximize a class label
  • Progressively grown GANs
    • super high res, super realistic

Hello, Computer Final

As a person with no inherent self-preservation instinct, nor the self-discipline to enact a learned one, it’s been a tough transition to independent adult life. I nearly immediately fell into an acute depression after moving to NYC in 2012 for my first job, shortly after completing my BFA. Since then, I’ve learned the hard way to take care of myself physically, mentally, and emotionally, and while I’ve skimmed countless research papers, books, and MOOCs about psychology over the years, making a regular practice of what I’ve learned still proves a challenge.

So for my Hello, Computer final project, I created a self-care assistant. Named Life Support, it’s a relatively sprawling/unfocused endeavor that attempts to intervene generally on common/modern emotional discomforts. While I plan to build this out even more as part of my thesis, Life Support presently consists of the following features:

  1. A guided breathing exercise. When you tell Life Support that you’re feeling some variety of agitated (anxious, nervous, etc), it will guide you through a breathing exercise: five counts in, five counts out. When a body is stressed, the fight or flight response (ie the sympathetic nervous system) kicks in; breathing exercises promote the opposite response (ie the parasympathetic nervous system) and indicate to the body that there is no threat. Breathing exercises/mindfulness practices in general have also been shown to reduce mild depressive symptoms. Since the assistant’s response seems to be capped at about 30-40 seconds, Life Support will ask whether to continue the exercise every six breaths.
  2. A guided physical exercise. When you indicate that you’re feeling some variety of burnt out (sluggish, unfocused, foggy, etc), Life Support will guide you through NYT’s Scientific Seven Minute Workout. As a person who naturally loathes exercise from the depths of my being, this is the rare workout that’s actually within my tolerance threshold, and since their interactive guide stopped working I’ve definitely become even lazier. Luckily, it was quite easy to recreate using SSML. Similar to the breathing exercise, Life Support will ask after each exercise whether or not the user wants to continue. Physical exercise has been shown to have countless benefits on a person’s mental and physical health, and is particularly helpful to prevent burn out.

  3. Guided cognitive behavioral therapy. When you tell Life Support that something is bothering you, it will help you CBT. CBT makes good on the belief that affect, behavior, and cognition directly influence (and therefore, exacerbate) each other by empowering the individual to targeting self-defeating cognitive distortions. Because cognitive distortions are based on fears and not facts, they can be effectively abated by gathering evidence for and against them. The individual then reassesses their beliefs based on these facts. While easily executed with the help of a therapist and some distance from the triggering event, it’s much more difficult to practice CBT in the moment, by oneself, while emotionally agitated. Life Support can help you challenge your triggering thoughts in the moment, thereby de-escalating emotional turmoil.There’s another branch in the conversation that the user can go to if they’re catastrophizing, which helps them challenge the extreme belief that is triggering them. Life Support asks them to consider 1) the worst case scenario, 2) the best case scenario, 3) the most probable case scenario, and 4) a purposeful action based on that probable scenario.

  4. Gratitude and good things log. In addition to helping users with their unpleasant emotions, Life Support also helps them prioritize positive emotions by keeping a log for good things that happen, as well as a log for things that the user is grateful for. According to positive psychology, a bias towards optimism and the refocusing on positive emotions allows people to change their subjective reality, and eventually their objective reality when their behavior changes as a result.Another benefit of keeping track of such things is the ability to revisit them; apparently, merely recounting positive moments from one’s past is enough to elicit those positive emotions in the present. When the user tells Life Support that they are feeling sad, Life Support will respond with something from their log, which is stored on a Firebase database.
  5. Knowledge base fallback. In addition to the foregoing intents, I also trained Dialog Flow’s Knowledge base on the transcripts of UPenn’s positive psychology MOOCs, as well as Barbara Oakley’s Learning How to Learn MOOC. As a result, Life Support will attempt to give advice when the user asks about anything related to positive psychology (coping/resilience skills, how to feel better about life, etc) or learning.

hello computer w6 hw

!!!!!!

 


Notes from the reading  (actually, it was so well-written that I just copy/pasted excerpts that I particularly wanted to remember)

Effectiveness of Alexa’s responses is inferred by these metrics:

  • Is the same question uttered again? (Did the user feel heard?)
  • Was the question reworded? (Did the user feel the question was understood?)
  • Was there an action following the question? (Did the interaction result in a tracked response: a light turned on, a product purchased, a track played?)
“With each interaction, Alexa is training to hear better, to interpret more precisely, to trigger actions that map to the user’s commands more accurately, and to build a more complete model of their preferences, habits and desires. What is required to make this possible? Put simply: each small moment of convenience – be it answering a question, turning on a light, or playing a song – requires a vast planetary network, fueled by the extraction of non-renewable materials, labor, and data.”
Extractive system of AI:
  • As Liam Young and Kate Davies observe, “your smart-phone runs on the tears and breast milk of a volcano. This landscape is connected to everywhere on the planet via the phones in our pockets; linked to each of us by invisible threads of commerce, science, politics and power.” 8
  • Our exploded view diagram combines and visualizes three central, extractive processes that are required to run a large-scale artificial intelligence system: material resources, human labor, and data… 
  • Vincent Mosco has shown how the ethereal metaphor of ‘the cloud’ for offsite data management and processing is in complete contradiction with the physical realities of the extraction of minerals from the Earth’s crust and dispossession of human populations that sustain its existence. 
  • Sandro Mezzadra and Brett Nielson use the term ‘extractivism’ to name the relationship between different forms of extractive operations in contemporary capitalism, which we see repeated in the context of the AI industry. 10
  • Thinking about extraction requires thinking about labor, resources, and data together.
  • The Echo user is simultaneously a consumer, a resource, a worker, and a product.
Statua citofonicaStatua citofonica by Athanasius Kircher (1673)

On environmental resource extraction:
  • Reflecting upon media and technology as geological processes enables us to consider the profound depletion of non-renewable resources required to drive the technologies of the present moment. Each object in the extended network of an AI system, from network routers to batteries to microphones, is built using elements that required billions of years to be produced. Looking from the perspective of deep time, we are extracting Earth’s history to serve a split second of technological time, in order to build devices than are often designed to be used for no more than a few years.
On human labor extraction:
  • Amazon CEO Jeff Bezos, at the top of our fractal pyramid, made an average of $275 million a day during the first five months of 2018, according to the Bloomberg Billionaires Index. 17 A child working in a mine in the Congo would need more than 700,000 years of non-stop work to earn the same amount as a single day of Bezos’ income
On the immensity of the supply chain:
  • As a semiconductor chip manufacturer, Intel supplies Apple with processors. In order to do so, Intel has its own multi-tiered supply chain of more than 19,000 suppliers in over 100 countries providing direct materials for their production processes, tools and machines for their factories, and logistics and packaging services. 20
  • Dutch-based technology company Philips has also claimed that it was working to make its supply chain ‘conflict-free’. Philips, for example, has tens of thousands of different suppliers, each of which provides different components for their manufacturing processes. 21 Those suppliers are themselves linked downstream to tens of thousands of component manufacturers that acquire materials from hundreds of refineries that buy ingredients from different smelters, which are supplied by unknown numbers of traders that deal directly with both legal and illegal mining operations.
  • Apple’s supplier program reveals there are tens of thousands of individual components embedded in their devices, which are in turn supplied by hundreds of different companies. In order for each of those components to arrive on the final assembly line where it will be assembled by workers in Foxconn facilities, different components need to be physically transferred from more than 750 supplier sites across 30 different countries. 24This becomes a complex structure of supply chains within supply chains, a zooming fractal of tens of thousands of suppliers, millions of kilometers of shipped materials and hundreds of thousands of workers included within the process even before the product is assembled on the line.

On environmental damage/effects on human health:

  • In recent years, shipping boats produce 3.1% of global yearly CO2 emissions, more than the entire country of Germany. 27 In order to minimize their internal costs, most of the container shipping companies use very low grade fuel in enormous quantities, which leads to increased amounts of sulphur in the air, among other toxic substances. It has been estimated that one container ship can emit as much pollution as 50 million cars, and 60,000 deaths worldwide are attributed indirectly to cargo ship industry pollution related issues annually. 28Typically, workers spend 9 to 10 months in the sea, often with long working shifts and without access to external communications. The most severe costs of global logistics are born by the atmosphere, the oceanic ecosystem and all it contains, and the lowest paid workers.
  • David Abraham describes the mining of dysprosium and Terbium used in a variety of high-tech devices in Jianxi, China. He writes, “Only 0.2 percent of the mined clay contains the valuable rare earth elements. This means that 99.8 percent of earth removed in rare earth mining is discarded as waste called “tailings” that are dumped back into the hills and streams,” creating new pollutants like ammonium. 31 In order to refine one ton of rare earth elements, “the Chinese Society of Rare Earths estimates that the process produces 75,000 liters of acidic water and one ton of radioactive residue.” 32Furthermore, mining and refining activities consume vast amount of water and generate large quantities of CO2 emissions.

AI Extractivism:

  • Availability of open-source tools for doing so in combination with rentable computation power through cloud superpowers such as Amazon (AWS), Microsoft (Azure), or Google (Google Cloud) is giving rise to a false idea of the ‘democratization’ of AI. While ‘off the shelf’ machine learning tools, like TensorFlow, are becoming more accessible from the point of view of setting up your own system, the underlying logics of those systems, and the datasets for training them are accessible to and controlled by very few entities. In the dynamic of dataset collection through platforms like Facebook, users are feeding and training the neural networks with behavioral data, voice, tagged pictures and videos or medical data. In an era of extractivism, the real value of that data is controlled and exploited by the very few at the top of the pyramid.
  • Every form of biodata – including forensic, biometric, sociometric, and psychometric – are being captured and logged into databases for AI training. That quantification often runs on very limited foundations: datasets like AVA which primarily shows women in the ‘playing with children’ action category, and men in the ‘kicking a person’ category. The training sets for AI systems claim to be reaching into the fine-grained nature of everyday life, but they repeat the most stereotypical and restricted social patterns, re-inscribing a normative vision of the human past and projecting it into the human future.

  • While Shiva is referring to enclosure of nature by intellectual property rights, the same process is now occurring with machine learning – an intensification of quantified nature. The new gold rush in the context of artificial intelligence is to enclose different fields of human knowing, feeling, and action, in order to capture and privatize those fields. When in November 2015 DeepMind Technologies Ltd. got access to the health records of 1.6 million identifiable patients of Royal Free hospital, we witnessed a particular form of privatization: the extraction of knowledge value. 53 A dataset may still be publicly owned, but the meta-value of the data – the model created by it – is privately owned.

 


Response to the reading:
So I had been the president of my high school’s Environmental Club, obsessively sorting the building’s recycling bins, running e-waste collection drives, and begging my parents to buy low-flow shower heads and compact fluorescent lightbulbs. To their despair, I applied to colleges with the intent of majoring in Environmental Science, and spent the first semester at UMich loading up on such courses and joining every environmentally-themed club I could find. All this led to nothing but my rapid disillusionment—most people just don’t care. Like, at all. And just like that, I completely lost hope that humanity could ever curb its materialism for the sake of our planet’s health, and transferred to art school the next year.

Like the article pointed out, not just economies but even early forms of trade are based on the extraction of resources for gain. Capitalism fundamentally depends on the pillaging of the planet and exploitation of large swaths of society. The tech industry may seem relatively innocuous, providing mostly intangible services rather than physical products, but they still pillage resources—namely, our time, attention, and data, as well as the materials necessary for the server farms and physical products they do sell—and profit to a degree that is astronomical.

Unfortunately, all of it is here to stay, and it’s still painful for me to be complicit in such an extravagantly wasteful system when I was so sensitive to it before, but the reality is that opting out would effectively disable me from engaging and participating in society in any meaningful way—technology is the platform on which our society organizes itself nowadays. Even in 2013, before I finally acquiesced to buying my first smart phone, I already had the feeling that I slipping to the margins. To even attempt to opt out is to render yourself irrelevant.