Source Discovery and Time Series Classification Using
Berkeley's Transient Classification Pipeline
Berkeley Astronomy: Dan Starr, Josh Bloom, Justin Higgins, Dovi Poznanski, Maxime Rischard, Nat Butler, Chris Klein, Rachel Kennedy, Adam Morgan
San Francisco State University: John M. Brewer
Berkeley Statistics: Noureddine El Karoui, John Rice
Berkeley CS: Massoud Nikravesh, Martin Wainwright Lawrence Berkeley Lab: Peter Nugent, Horst Simon Los Alamos Nat. Lab. / UC Santa Cruz: Damian Eads

The Berkeley Transients Classification Pipeline (TCP) is aparallelized source identification and classification pipeline
which ingests several data streams and emits classified
science events.

{ {Currently, the TCP ingests and classifies subtracted sourcesfrom the Palomar Transient Factory 48” telescope survey. The
7.8 sq-deg, 100 Mpix instrument samples the sky at 120
seconds, producing ~1 Million source detections per night.

2D imageclassification:
Machine-Learningwith human input(in future)

SDSS

stripe-82
archived data

SASIR LSST
(future)PTF / LBL
subtraction
pipeline(future)

Survey XSurvey Y(real-time survey (static surveytelescope)
repository)

Transients
Classification
Pipeline
Database containing“sources”• features for a source• data epochs associatedwith a source>1000:1 rejection of
bogus candidates(prelim. cuts+ machine learning)

Broadcast “sources”• interesting or transientsource• include classifications

The GroupThink web-app displays thumbnails of astrophysicalas well as defective and poorly subtracted sources from the
PTF pipeline and allows a dozen users to judge whether thesources are astrophysical or not.
The resulting dataset from GroupThink is used to train amachine-learning based filter. This filter and itʼs associatedmetrics effectively model the user judgments, allowing theTCP to automatically filter out most non-astrophysicaldetections without the help of a human.

Databasefeaturegeneration
sourceclassification

Follow-up
telescope
observations

This “Noisification” software also adds simulated instrument
and limiting magnitude offsets to better represent real PTF
data.
Machine-Learned
Classification

Classify
Database
Broadcast“Object”
Datastream

TCP

N sources

sourcegeneration

Before training the TCPʼs classifiers, we re-sample the well
sampled lightcurves found in DotAstro.org to better match thePTF surveyʼs observing cadence.

junk ... good{ {Humanjudgments vary
due todifferent
reasoning andexperiences

Confusion Matrix

This confusion matrix shows how well a single machinelearned classifier distinguishes between 30 different scienceclasses. A perfect classifier would contain 100% along itʼsdiagonal.

Broadcast

Each new observation is first associated with a source in a
database, then “feature” attributes are regenerated whichcharacterize that source, and finally science classifications are
made using those features.
This triplet of steps for a single observation are parallelized
over 22-110 CPU cores using IPython.

Here, the classifierʼs effectiveness was evaluated using crossvalidation with the training-set data. This means the heat-map
colors represent successful classification percentages whengiven ideal lightcurves. Even when training and testing withthe same cross-validated data, we can gain insight on how
effective this classifier is with a set of lightcurve attributes.

The real-time TCP is generally distributed over 6 clustercomputers, although a 96 core beowulf cluster is occasionally
included for re-evaluation purposes.
Additionally, weʼve recently started
reclassification and exploration of classifier andattribute parameter spaces using Hadoop on
Yahoo!ʼs 4000 core M45 cluster. This wasmade possible once we ported WEKA to theHadoop environment.

+To train the TCPʼs science classifiers, the TCP
references ~20,000 lightcurves which arecontained in the (http://dotastro.org) lightcurve
warehouse. DotAstro.org contains 150 variable
and transient classes which were derived from87 papers.

These are a subset of PTF sources
identified and classified by the TCP as
potential SN/AGN during a test run. This
preliminary test identified 84% of thealready known and spectroscopicallyconfirmed Super Novae as an SN/AGN
science class. Mis-classifications wereprimarily faint and nearly-ambiguous
sources.

Contacts:
Josh Bloom (PI)jbloom@astro.berkeley.edu
Dan Starr(primary developer)
dstarr@astro.berkeley.edu

http://DotAstro.org

Dan Starr

Links:EC2

DotAstro.org

http://dotastro.org(TCP links)

http://is.gd/5mA4
http://tinyurl.com/tcp123