A Functional Perspective on
Machine Learning
via Programmable
Induction and Abduction
Steven W. T. Cheung
Victor Darvariu
Dan R. Ghica(University of Birmingham)

Reuben N. S. Rowe
(University of Kent)

Koko Muroya(University of Birmingham& RIMS, Kyoto University)
FLOPS(Nagoya),
9 May
2018
Muroya(U. B’ham.
& RIMS,KyotoU.)

Machine Learning

https://xkcd.com/1838/2

Muroya (U. B’ham. & RIMS, Kyoto U.)

Machine Learningmodelisinghttps://xkcd.com/1838/3

Muroya (U. B’ham. & RIMS, Kyoto U.)

Machine Learning

using /predicting

https://xkcd.com/1838/4

Muroya (U. B’ham. & RIMS, Kyoto U.)

Machine Learninglearning /training

https://xkcd.com/1838/5

Muroya (U. B’ham. & RIMS, Kyoto U.)

Machine Learningmodelisingparameterised modelusing /
predicting

learning /training

answers

parameters
6

Muroya (U. B’ham. & RIMS, Kyoto U.)

Machine Learning with TensorFlow
https://www.tensorflow.org/
https://github.com/sherrym/tf-tutorialmodelisingparameterised modelusing /
predicting

W = tf.Variable(...)
b = tf.Variable(...)
y = W * x_data + b

sess = tf.Session()sess.run(init)
y_initial_values = sess.run(y)learning /training
sess = tf.Session()sess.run(init)
sess.run(train)answers

parameters
7

Muroya (U. B’ham. & RIMS, Kyoto U.)

TensorFlow● shallow embedded DSL○ lack of integration with host language
TensorFlow○ cannotlibraries inlimited
graphshostuselanguage(computation graphs
integration(programs)○ difficult to debug / type graphs+ “sessions”)○● imperative parameter (“variable”) update

Proper functional language?
● simple & uniform programming language○ well-defined operational semantics● functional parameter update
8

Muroya (U. B’ham. & RIMS, Kyoto U.)

Machine Learningmodelisingparameterised modelusing /
predicting

learning /training

answers

parameters
9

Muroya (U. B’ham. & RIMS, Kyoto U.)

Machine Learningmodelisingparameterised modelusing /
predicting

learning /training

answers

parameters
10

Muroya (U. B’ham. & RIMS, Kyoto U.)

functional programming language for using & learning

realisability style“proofs as programs”

logical framework of science methodology
C. S. Peirce “Illustrations of the Logic of Science”

11Muroya (U. B’ham. & RIMS, Kyoto U.)

Peirce’s Logical Inferences● analytic rule, believable results○ deduction● synthetic rules, tentative results○ induction○ abduction

12

Muroya (U. B’ham. & RIMS, Kyoto U.)

Peirce’s Logical Inferences● analytic rule, believable results○ deduction● synthetic rules, tentative results○ induction○ abduction

13

Muroya (U. B’ham. & RIMS, Kyoto U.)

Peirce’s Logical Inferences● analytic rule, believable results○ deduction● synthetic rules, tentative results○ induction○ abduction14

Muroya (U. B’ham. & RIMS, Kyoto U.)

Peirce’s Logical Inferences● analytic rule, believable results○ deduction● synthetic rules, tentative results○ induction○ abduction

15

Muroya (U. B’ham. & RIMS, Kyoto U.)

Peirce’s Logical Inferences, adapted to ML● analytic rule, believable results○ deduction● synthetic rules, tentative results○ induction
create a modelfrom data○ abduction

16

Muroya (U. B’ham. & RIMS, Kyoto U.)

Peirce’s Logical Inferences, adapted to ML● analytic rule, believable results○ deduction● synthetic rules, tentative results○ induction
create a modelfrom data○ abduction
find the best parameter ofa model to explain data
17

Muroya (U. B’ham. & RIMS, Kyoto U.)

“Proofs as Programs”● analytic rule, believable results○ deduction● synthetic rules, tentative results○ induction
create a modelfrom data○ abduction
find the best parameter ofa model to explain data
18

Muroya (U. B’ham. & RIMS, Kyoto U.)

“Proofs as Programs”● analytic rule, believable results○ deduction● synthetic rules, tentative results○ induction
create a modelfrom data○ abduction
find the best parameter ofa model to explain data
19

Muroya (U. B’ham. & RIMS, Kyoto U.)

“Proofs as Programs”● analytic rule, believable results○ deduction● synthetic rules, tentative results○ induction
create a modelfrom data○ abduction
find the best parameter ofa model to explain data
20

Muroya (U. B’ham. & RIMS, Kyoto U.)

Towards Programmable Induction & Abduction
interpolation
exterpolation

generic optimisation(e.g. gradient descent)21

Muroya (U. B’ham. & RIMS, Kyoto U.)

Towards Programmable Induction & Abduction
interpolation
exterpolation
metric space

generic optimisation(e.g. gradient descent)vector space

22

Muroya (U. B’ham. & RIMS, Kyoto U.)

Machine Learning with Induction & Abduction
modelisingparameterised modelusing /
predicting

learning /training

answers

parameters
23

Muroya (U. B’ham. & RIMS, Kyoto U.)

Machine Learning with Induction & Abduction
modelisingparameterised modelusing /
predicting

learning /training

answers

parameters
24

Muroya (U. B’ham. & RIMS, Kyoto U.)

Machine Learning with Induction & Abduction
modelisingparameterised modelusing /
predicting

learning /training

answers

parameters
25

Muroya (U. B’ham. & RIMS, Kyoto U.)

Machine Learning with Induction & Abduction
modelisingparameterised modelusing /
predicting

learning /training

answers

parameters
26

Muroya (U. B’ham. & RIMS, Kyoto U.)

Machine Learning with Induction & Abduction
modelisingparameterised modelusing /
predicting

learning /training

answers

parameters
27

Muroya (U. B’ham. & RIMS, Kyoto U.)

Machine Learning with Induction & Abduction
modelisingparameterised modelusing /
predicting

learning /traininganswers

parameters
28

Muroya (U. B’ham. & RIMS, Kyoto U.)

Machine Learning with Induction & Abduction
modelisingparameterised modelusing /
predicting

learning /training

answers

parameters
29

Muroya (U. B’ham. & RIMS, Kyoto U.)

Machine Learning with Induction & Abduction
modelisingparameterised modelusing /
predicting

learning /training

answers

parameters
30

Muroya (U. B’ham. & RIMS, Kyoto U.)

Parameter Management in Modelising
let pl v x = v[0] * x + v[1]let pci v = (pl v[0,1], pl v[0,2])
let pwr v = (pl v[0,1], pl v[2,3])

linear regression withconfidence interval

weighted regression

31

Muroya (U. B’ham. & RIMS, Kyoto U.)

Parameter Management in Modelising
let pl v x = v[0] * x + v[1]let pci v = (pl v[0,1], pl v[0,2])
let pwr v = (pl v[0,1], pl v[2,3])

error-prone
linear regression withconfidence interval

weighted regression

32

Muroya (U. B’ham. & RIMS, Kyoto U.)

Modelising with Embedded Parameters
let pl’ a b x = a * x + b
let ci = let a = {0} in(pl’ a {1}, pl’ a {2})let wr = (pl’ {0} {1}, pl’ {2} {3})linear regression withconfidence interval

weighted regression

33

Muroya (U. B’ham. & RIMS, Kyoto U.)

Modelising with Embedded Parameters
let pl’ a b x = a * x + b
let ci = let a = {0} in(pl’ a {1}, pl’ a {2})let wr = (pl’ {0} {1}, pl’ {2} {3});; pl’: F -> F -> F -> F;; ci,wr : (F -> F) * (F -> F)

let pl v x = v[0] * x + v[1]let pci v = (pl v[0,1], pl v[0,2])
let pwr v = (pl v[0,1], pl v[2,3]);; pl: P -> F -> F;; pci,pwr : P -> ((F -> F) * (F -> F))
34

Muroya (U. B’ham. & RIMS, Kyoto U.)

Modelising with Embedded Parameters
let pl’ a b x = a * x + b
let ci = let a = {0} in(pl’ a {1}, pl’ a {2})let wr = (pl’ {0} {1}, pl’ {2} {3});; pl’: F -> F -> F -> F;; ci,wr : (F -> F) * (F -> F)...and Decoupling
let pl v x = v[0] * x + v[1]let pci v = (pl v[0,1], pl v[0,2])
let pwr v = (pl v[0,1], pl v[2,3])dec(ci) ~ (pci, [0;1;2])
dec(wr) ~ (pwr, [0;1;2;3]);; pl: P -> F -> F;; pci,pwr : P -> ((F -> F) * (F -> F))
35

Muroya (U. B’ham. & RIMS, Kyoto U.)

Machine Learning with Induction, Abduction
and Decoupling
modelisingparameterised modelusing /
predicting

learning /training

answers

parameters
36

Muroya (U. B’ham. & RIMS, Kyoto U.)

DecML, or Idealised TensorFlow
modelising● as an extended simply-typed lambda-calculus○ type soundness & beta-law, using graph-rewriting
operational semantics [-, Cheung & Ghica, LICS ‘18 to appear]○ graph-rewriting visualiser
https://cwtsteven.github.io/GoI-TF-Visualiser/CBV-with-CBN-embedding/index.html● as a PPX extension of OCaml○

https://github.com/DecML/decml-ppx(to be merged)○ (non-trivial) translation to OCaml in pre-processing
https://github.com/reubenrowe/ocaml-decml
37Muroya (U. B’ham. & RIMS, Kyoto U.)

Machine Learning with Induction, Abduction
and Decoupling

library

host language

https://xkcd.com/1838/38

Muroya (U. B’ham. & RIMS, Kyoto U.)

Machine Learning with Induction, Abduction
and Decoupling

Let them beunited :-)https://xkcd.com/1838/

39

Muroya (U. B’ham. & RIMS, Kyoto U.)

Machine Learning with Induction, Abduction
and Decoupling

Let them beunited :-)probability
TBA
https://xkcd.com/1838/40

Muroya (U. B’ham. & RIMS, Kyoto U.)