Final Project Presentations
MIT 6.S191
February 1, 2019

6.S191 ProjectGroup 1
Varnika Sinha
Julia Wang
Emily Zhang

Problem
Current security
systems are notrobust enough toadequately protectuser data.
Group 1Cybersecurity Challenges:❖ Unprecedented attacks❖ Cyber espionage❖ Data theft

Solution & Algorithms
cat:mouse::attacker:defense❖ Reinforcement Learning❖ Reward system❖ Policy gradientGroup 1

Applications & Impact❖ Password protection❖ Cloud security❖ Protection against malware& viruses❖ Voting systems security❖ Internet of things security
Group 1

Identification of Heartbeat
Systems in Photometric
Surveys
Baichuan Mo, Erik Tamre,
Prajwal Niraula, Yunpo Li
6.S191
Feb 1, 2019

Heartbeat Systems

Thompson et al. 2012

Scientific Motivations

Probing the Internal
Stellar Structure
As method of finding
exoplanet
Tidal Circularization

Distinguishing the HeartBeat Systems
HB System

Non - HB

Proposed method
Deep Learning! But why?
Method one: 1-D Convolutional Neural Network
Motivation: Extract local features in the time domain (e.g., peaks, valleys, etc.)

Method Two: Recurrent Neural Network (LSTM)
Motivation:
1. Time series input data
2. Different duration of each photometric survey : Handle variable length input

We will try to run the models on both time domain and frequency domain

Challenges and path forward
Challenges at current stage
More than 100,000 unlabeled observed data, small amount of labeled data

To push forward this work
Manual labeling vs. model labeling
Introduce data other than photometric survey

Advanced Scoliosis
Detection with
Deep Neural Nets
Group 3Sandra Liu
Eric Magliarditi
Nathan Rebello

Scoliosis is an abnormal lateral spinal curvature●Starts before 15 yrs. old●600,000 patient visits/year●

30,000 children fitted w/brace●

40,000 undergo spinal fusion
surgeryBrace
May 2014-July 2015
https://www.aans.org/Patients/Neurosurgical-Conditions-and-Treatments/Scoliosis

How do we map early signs to advanced scoliosis?
●No predictive methods●Costs:●○$5K on bracing○$100K/surgery○$1K/year on checkups

If detected, preventative
measures can improve posture

https://health.costhelper.com/scoliosis.html

Mild>10°

Severe>40°

Use Convolutional Neural Nets & Supervised
Learning to predict severe scoliosis
Input Data:

Convolution Neural NetClassification

X-Ray of patient with earlysigns of Scoliosis: Curvature~10-20°

1. Learns features present
in patients who had mildscoliosis but became severe
after a time, T
3. Assigns weights tofeatures
4. Outputs classification

Class 1:
Mild Scoliosis after time T

Labels:Future Scoliosis Severity(Mild, Moderate, Severe)*Classification Subject to Change

Class 2:Moderate Scoliosis after time TClass 3:
Severe Scoliosis after time T

Goal: CNN Detect Subtle Features that Trigger
Advanced Scoliosis●Severe:○●Spinal curving

Subtle:○Uneven Shoulders○Ribs at different heights○Head not centered abovepelvis○Uneven waist○

Hips raised highSevereWhat are the challenges?
● Validation● Data Acquisition○ 600,000 patient visits/year○ Potential privacy issues with hospitals

Further Applications●●●

Individual-specific physical therapy treatments
Potential using physician/therapist data with the deep neuralnet to develop effective therapy treatments
Detecting features in X-ray images that might lead to severe
scoliosis

https://www.webmd.com/back-pain/what-are-the-symptoms-of-scoliosis#1

Thank you!

One case every 3 seconds.

131.5 million by 2050.
Costs above a $1 trillion in 2018.

Music can help ...

...but which music works best?

Wristbands offer:- Activity (calories)- Temperature- Electrocardiogram

LSTM

Feedback

Infer Mood

RecommendersystemPersonalizedmusic fortherapy

Pred1ct: Predicting Useful CRISPR-Cas9 OutcomesDavid Li, Joshua Park, Akshaj Kadaveru
6.S191 Group 5

CRISPR/Cas9 allows for targeted gene editing●●CRISPR/Cas9 cuts at specific locations given a guide RNA
Cells have own mechanisms to repair cuts in DNAExisting Approaches●●●

Library of guide RNAs (target different areas)
Treated cells containing target sequences with Cas9
Sequence and compare

Pred1ct Network Architecture-Feedforward neural network
Input: 20 dimensional vector; each dimension can be one of four values--

Output: Percentage of insertions/deletions that are one nucleotide insertions--(A, C, T, G)20
Value between 0 and 1

Existing training set of ~42,000 sequences

...

...

...

...

Consequences●●●

Small frameshifts make up 24% of mutations that manifest in currently
recognized genetic disease.
Accurate prediction of +1 frequencies allows for the design of useful guide
RNAs that would allow correction of these diseases
For example:○ Cystic Fibrosis○ Crohn's Disease○ Tay-Sachs DiseaseRational Design ofElectrocatalysts

Nathan Ricke and Eric Alt
6.S191
Group 6Electrocatalysts for Storing and Recovering
EnergyComputationally Generate and Test Catalysts

LASSORegression

G

LSTM Neural NetworkG
Smiles String
FC2ccc1nc(G)c(G)nc1c2

[2]
[1]

[3]

Group 7: Yiwen Huang, Greg Allan, Michael Schmid

GANs for Automotive
Exterior DesignAutomotive Design - State of the Art

Attractive design requires creativity
BUT there are no new elements on cars!

BMW 8 series concept sketch [4]BMW 8 series concept car [5] BMW 8 series final design [6]6.S191GANs for Automotive Exterior DesignIdea: GANs in Automotive Design
Idea from lecture:
Exploit the creativity ofGenerative Networks[11]

GenerativeNetwork[7], [8], [9]6.S191

Slowly increase or
decrease single latentvariables (luxury, sports,race)
GANs for Automotive Exterior Design?

design suggestions[10]

Thank you for your attention!

[12]6.S191

GANs for Automotive Exterior DesignDeep Reinforcement Learning forRadiation Therapy PlanningGroup 8: Susu Yan (Listener), Michelle Jiang (Credit)
MIT 6.S191 presentation

Radiation Treatment Plan: Tumor and Organs-at-risk

Esophagus

Tumor

Spinal cord

Radiation Treatment plan: Dose Distribution

3D distribution of dose in patient shown on CT

Actions

Actions (cont.)

There are thousands of parameters that can be modified togenerate a radiation therapy plan.

Reward: Minimizing or Maximizing DoseValues and Meeting Clinical Goals

The goal is to kill all tumor cells and minimize radiation
damage to healthy tissues.

Challenges

Challenges●●●●Requires years of training andexperience
Time consuming
Never sure if a better planexists
Patient needs to be treated assoon as possible

Reinforcement Learning
Treatment
Planning System

State:
Dose ineachvoxel

Actions:
Changeparameters

Reward:Meetingclinical goals

CT Scan of Patient withsegmentation of contours

Image adapted from https://adeshpande3.github.io/Deep-Learning-Research-Review-Week-2-Reinforcement-LearningThank you!
Group 8: Susu Yan (Listener), Michelle Jiang (Credit)Diagnosing Defects in 2D
Materials with Deep Learning
Nina & Jovana Andrejevic

Project Group 9

Introduction
2D materials exhibit tunable electronic and opticalproperties, exciting for development of next-generationelectronic and optoelectronics devices
Quality is critical, but challenging to monitor
Raman spectroscopy provides one signature of material quality
Need a high-throughput technique for rapidly identifying andquantifying defects to satisfy industry-scale growth and processing

Project Group 9

Proposal
Can unsupervised deep learning automate the screeningof “defect fingerprints”?

We use an autoencoder to learn a compressedrepresentation of materials’ signatures that- is resistant to artifacts produced by defects- distinguishes different materials in an unsupervised manner

Project Group 9Data Generation
Generate 3 different materials
classes with increasing complexity

Example Spectrum

Defect Fingerprints

shiftasymmetry
Project Group 9Project Group 9Preliminary Results

Asymmetriccluster bymaterial

ShiftedPristine

Material 1

Project Group 9

Material 2Material 3

Conclusion
Our preliminary results show:- the suitability of autoencoders for recovering salient features ofRaman signatures corrupted by defects- the network’s ability to learn a well-separated representation ofdifferent materials’ signatures in an unsupervised manner
Next steps:- Train on experimental data, possibly supplemented by simulation- Quantify defect concentration
Project Group 9References[1] “Raman Spectroscopy Quality Control of New 2D Materials.” Spectroscopy Europe/Asia, 12 July
2017, www.spectroscopyeurope.com/news/raman-spectroscopy-quality-control-new-2d-materials.
[2] “Keras: The Python Deep Learning Library.” Keras Documentation, https://keras.io/.

Thank you!

Project Group 9Using GANs in Filmmaking
to replace traditional VFX
Baptiste, Nick, Suraj, Brandon - Group 10

Current CGI Implementation- Generate 3D models- Texture, lighting, and color- Animate the CGI- Avatar- Music videos

Current Uses of Machine Learning in VFX- CGI overlaid on real images- Wire removal & green screen detection- Rotoscoping- Deepfakes- Human faces created out of nothing- Already large dataset

Group 11...

THANKS FOR LISTENING!
Group 11

AI-assistedparentingZhenhua (Ray) Rui&Kai JinGroup 12

parenting is a sophisticated job
Kids development
Language
Motor skills
MentalHabits...

learn how kids learn and suggest the next move“en”, “a”, ...

“en”, “a”, “mama”“en”, “a”, “mama”

dancedancetempo

“car”
...

“car”,“pickup truck”
choose books...

6 month

12 month

18 month

24 monthchallenge

Can AI help parents
raise better humans?

Using Deep Learning toassist Colorblind people
Victor Horta
Luis Covatti

AgendaWhat is Color blindness, and why it matters?
What is the problem we are trying to solve?
Proposed solution
Potential applications

What is Color blindness, and why it matters?

8% of men are colorblind [1]
1 in every 200 women is colorblind [1]What number do you see?
Source: L. T. Sharpe, A. Stockman, H. Jagle, and J. Nathans, “Color vision: From genes to perception
chapter Opsin genes cone photo pigments,” in Color Vision and Color Blindness. Cambridge, U.K.:
Cambridge Univ., 1999, pp. 3–51.

What is Color blindness, and why it matters?

8% of men are colorblind [1]
1 in every 200 women is colorblind [1]Source: L. T. Sharpe, A. Stockman, H. Jagle, and J. Nathans, “Color vision: From genes to perception
chapter Opsin genes cone photo pigments,” in Color Vision and Color Blindness. Cambridge, U.K.:
Cambridge Univ., 1999, pp. 3–51.

The problem
Colorblind people usually candistinguish standalone colors

RedGreen????

Easy!

But things get harder whencertain shades are very small
and close to each otherSurroundings seem to matter

How can we generate image
filters that increase the world
readability by slightly tilting
colors, while still maintaining
them as true to their original aspossible?

What has been done so far?
Mechanical solutions

Recoloring algorithms

Enchroma glasses

Aim to improve color differentiation
Drastic change in original colors (unnatural)
Do not preserve image details
Original

Adaptive Fuzzy[2]“The results show that the glasses introduce
a variation of the perceived color, butneither improve results in the diagnosis
tests nor allow the observers with CVD tohave a more normal color vision.” [1]Deep Correct[3]Source: [1] https://www.osapublishing.org/oe/abstract.cfm?uri=oe-26-22-28693[2] Jimmi Lee, Wellington P. dos Santos. An Adaptive Fuzzy-Based System to Simulate, Quantify and Compensate Color Blindness[3] Gajo Petrovic, Hamido Fujita. Deep Correct: Deep Learning color correction for color blindness

Corrected

The model/system[Illustrative]

Input
image

Outputimage

Loss-function
dependent on:
Interpretability bycolorblind people

max

Assessed by mathematicaltransformation and fieldtests (feedback)

CNN

Using colors within
the domain ofvisualization forcolorblind people

Distortion fromoriginal image

minPotential applications
Improve accessibility to digital content

Visual Arts
(paintings, movies)

Entertainment(games)

Internet(maps readability, web design)Thank you

All Dolled Up:
How Deep Learning Can
Teach Children to LoveThemselves
By: DIna Atia, Faduma Khalif, Yousef Mardini
Group 17Background● National Black Doll Museum● Topsy Turvy Dolls● Mattel Barbie: 1968, 1980

All Dolled Up:● Learns what you look like
● Maps your features todoll feature set● Doll that looks like you!

Methodology
collecting ground truth data
training a set of Classifiers
applying the most accurate classifier
to raw, unannotated data.
making the necessary corrections to focus
on the weak points of the classifierThe ClassifierA CNN

Thanks For Listening!!
Please Give Us Prizes

Background● National Black Doll Museum● Topsy Turvy Dolls● Mattel Barbie: 1968, 1980

Customized interior designusing generative models
Group 18: Keran Rong ; Mia Hong

Customized interior design using generative models
Interior design Current situation:• A high demand in market• High expertise is needed• Fashion – sensitive•Customized design is expensiveCustomized interior design using generative models

Thank you! Any Questions?

Neural Networks as an early-stage
Architecture Design & Sustainability Tool

6.S191Group 23
Yu Qian Ang (credit)
Klo’e Ng (listener)Group 23

PROBLEM
THE BUILT ENVIRONMENT:LARGE AMOUNT OF WASTE>3 Billion tonnes of raw materials
consumed annually

ENERGY INTENSIVE
Built environment consumes >30% to 70%of total primary energy use

CARBON EMISSIONS
Buildings are key contributor togreenhouse gas emissions around theworld

Group 23

PROPOSED APPLICATIONEARLY STAGE DESIGN IN BUILT ENVIRONMENTWHY MACHINE LEARNING / NEURAL NETWORK:-Human design inherently subjective-Opportunity for impact downstream(enhance sustainability)-Insufficient time/manpower toexplore many design options-Human error, blind-spots, and bias

Group 23

PROPOSED APPLICATIONGENERATIVE ADVERSARIAL NETWORKS (GAN)

B Design and develop GAN model
A Identify key parameters/features to optimise(Low) carbon
footprint

Cost efficiency(Low) energy
use/wastage(Low) materialwastageGroup 23

PROPOSED APPLICATIONGENERATIVE ADVERSARIAL NETWORKS (GAN)

1 Data | Samples
Data/samples from past
projects, or design options
developed manually

2GENERATIVENETWORKMODEL

Latent Space

3Feed data into model to train iteratively,
aim to minimize loss

MODEL GENERATESCONCEPTUAL DESIGN OPTIONSGroup 23

IMPACT & CHALLENGES
POTENTIAL IMPACT

more/betterbuilding design
options

EnhancedsustainabilityLess wastage

Less humanerror/bias

CHALLENGES

Mode collapse:
generator keeps
generating similar
designs(limited diversity ofsamples)

Validation of GAN
outputs

Contextualizing
the GAN outputs(may need to run
physics based
simulations)(architecture issometimes highly
contextualized)

Group 23

SIMILAR APPROACHES/APPLICATIONS

1GeneratingAnime GirlsJin et al (2017)(Fudan & Carnegie Mellon)

2Generating
Pose-guidedApparel

Ma et al (2018)(KU-Leuven & ETH Zurich)

3

CycleGAN:
Generating
photos frompaintings etc

Zhu et al (2017)(UC Berkeley)

THANK YOU

DEEP DOODLE
DEEP LEARNING METHODS TO GENERATE SKETCHES FROM LABELSWHAT DEEP LEARNING CONCEPTS
ARE WE USING?

Semantic segmentation of one
cat image from a dataset used totrain image recognition neuralnets

Some cat doodles from Google
QuickDraw

WHAT DEEP LEARNING CONCEPTS
ARE WE USING?

POSSIBLE APPLICATIONS• Used in cognitive interviews• Animating children’s books in real time.

• Assistive technology for teaching kids to draw.

Demohttps://colab.research.google.com/drive/1hHMR3Q2-I
ugs8fb5GXG3SGmwXkMpo2FC

https://magenta.tensorflow.org/assets/sketch_rnn_de
mo/index.html

Deep Learning in Major
League Baseball

Maximilian Porlein and Jack Phifer, MIT 2022

Inspiration
MLB Beat the Streak: choose up to two Major League players
daily. String together a 57-game hit streak to beat Joe DiMaggio’s
record of 56 games and you win $5.6 million dollars. If either ofyour players goes hitless, you start all over.

Why not just pick the top hitters in the MLB?

Why not just pick the top hitters in the MLB?
- Too many factors can influence a player’s ability to make a hit

Why not just pick the top hitters in the MLB?
- Too many factors can influence a player’s ability to make a hit- Pitcher:

Source: FanGraphs PITCHF/x, Feb 2011

Why not just pick the top hitters in the MLB?
- Too many factors can influence a player’s ability to make a hit- Weather:- Air temperature can change a baseball's trajectory- Air density can play a role in how far a ball travels- High and low temperatures can affect a pitcher's grip
- Cloud coverage can affect how players see the ball- Windy conditions
Source: Alan Nathan, University of Illinois Department of Physics

Why not just pick the top hitters in the MLB?
- Too many factors can influence a player’s ability to make a hit- Location:- Home team advantage

Source: SB Nation, Jan 2011

Our Model - Formulation●16 Different Variables○ Data compiled from several different sources including mlb.com andbaseball-reference○ Most categories didn’t directly exist but were compiled in python○ All categories were scaled to a value between 0 and 1

Our Model - Implementation● Used 1 hidden layer○ Consisted of 28 nodes● Outputted probability of a hit● All active players are fed into
NN and player with the highest
output is selected for that nightOur Model - Conclusion●●●

Model was trained using data from 2016 and 2017 season○ 2018 was used as the testing data
Model was largely unsuccessful as it could never put together high streaks○ Highest streak was 9Reasons the model fell short○ Baseball is an imperfect game with human variation○ Not enough testing data (data for previous years wasn’t as accessible)○Not enough training timeExtensions of Our Model- Allows individual teams to choose rosters before playing
specific teams- Teams can use to determine their most consistent players- Can be used in recruitment for colleges and teams- See which players are most consistentExtensions of Our Model
MLB Beat the Streak:- Even if the model is not completely perfect…
- MLB frequently gives “off-day” exceptions to streaks longer
than 10-15 days
- Prizes (like merchandise) are still awarded to streaks asshort as 5 days

Thank you!
Maximilian Porlein - mporlein@mit.edu
Jack Phifer - jphifer@mit.edu

Final Project Presentations
MIT 6.S191
February 1, 2019