Classifying Spin-Interactions 

Using Reinforcement Learning

Kate Meuse (Cornell University)

Collaborators: Jessica Opsahl-Ong (Rice University), 

Paul Kienzle, William Ratcliff

Xufeng Zhang/Yale University
2

Multitudinous Magnets

● Magnetoresistive RAM (MRAM), 

hard drives depend on magnets

● Need to examine magnetic 

structure of new materials to 

find those with useful 

properties

Stock Image
3

Magnet Math

● Ground state orientation of 

spins in a magnetic structure 

is derivable

● Don’t know which 

interactions create the 

ground state

J

i,j = strength of interaction

S

i

 & Sj= spins
4

Nearest neighbor Next nearest neighbor

Interaction Models

J

1

J
5

Magnet Math ● Inelastic neutron scattering 

excites the system to create 

a spinwave that is dependent 

on the types and strengths of 

the interactions

● Want to measure the energy 

of the spinwave along 

directions of the structure
6

SpinW/bumps

Certain commercial products are mentioned for information purposes, but are not endorsed by NIST

● MATLAB bound to Python

● Generates dispersion graph 

given structure, interactions

● Bumps can fit dispersion to 

get J values (strengths of 

interactions)

● Compare goodness of fit to 

pick model [1 1 0]

J1 = -5

(meV)
The Problem

● Beam time is valuable – 

limited access

● Select more efficient 

measurements as not all 

are required

● Determine how to 

quickly select 

measurements that 

describe both type and 

strength of interactions

7
8

The Solution: Reinforcement Learning

AlphaGo Deepmind

Teaching a computer to 

make optimal decisions 

using rewards
Reinforcement Learning

9

● Observes environment to make 

decisions

● Each step it receives a reward

● Next decision is based on that 

reward and new environment 

information

Massimiliano Patacchiola
Reinforcement Learning

10

Learns each step and each episode to create the fastest navigation

Massimiliano Patacchiola
Applying Reinforcement Learning

● Action: select a direction to 

calculate the dispersion

● State: all previously measured 

directions

● Reward: low chi squared, low 

uncertainty, and high 

difference between models

● Ends episode with low chi 

squared and low uncertainty 

11

(meV)
12

Results: Nearest Neighbor

Reward Per Episode Steps Per Episode
13

Unfittable

(meV) (meV)

(meV) (meV)

(meV)

Omega (meV)
14

It’s Essentially A Square

● Out of the plane of the 

square, not much is 

happening

● Valuable information 

only found in directions 

with some a or b 

component

3 Å

3 Å

18 Å
15

Correctness: Nearest Neighbor

● Slightly higher at first, 

probably not significant

● Likelihood of picking bad 

measurement compared 

to any other option much 

decreased

Measurements
16

Correctness: Nearest Neighbor

● Always chose the correct 

model

● Due to the process of 

model selection, this is 

unsurprising

● Model with fewer 

parameters and same fit 

will be chosen

~100%
17

Next Step: Next-Nearest Neighbor
Future Steps

● Calculate and fit neutron 

intensities instead of 

merely dispersion

● Add finite resolution of 

the instrument

● Attempt same problem 

with Gaussian processes

● Publish!

18
Acknowledgements

Thanks to the SURF program 

directors, interns and mentors.

Special thanks to William Ratcliff, 

mentor.

We are grateful for funding from the 

Center for High Resolution Neutron 

Scattering.

19
Questions