Quantifying Spin Interactions 

Using Reinforcement Learning

Jessica Opsahl-Ong

Collaborators: 

Kate Meuse, Paul Kienzle, William Ratcliff 
Magnetism

Many fields with current 

interest in magnetic 

structures 

• Hard drives, MRAM, 

quantum computing, etc.

In order to utilize magnetic properties, we 

must be able to identify the specific 

interactions that give way to magnetism 
Magnetic Properties 

The magnetic s pin ground s tate is determined by the minimization of 

its total energy (repres ented by the Heis enberg-Hamiltonian) 

However, knowing the ground s tate (i.e. the s pin vectors ) does not 

uniquely determine the s et of interactions (i.e. the coupling cons tant 

J ) which gave ris e to the Hamiltonian 
Finding J

• In order to quantify J , we mus t make a magnetic excitation 

• Simply flipping the direction of one s pin is too high-energy 

• Ins tead, s hare the s ingle s pin revers al among many s pins 

creates a s pin wave  meas ure the energy of this wave

• Create and meas ure s pinwave us ing inelas tic neutron s cattering
Simulating Spinwaves with 

• SpinW is a MATLAB library that 

can numerically s imulate 

magnetic s tructures and their 

s pinwave dis pers ions 

• We us ed pySpinW which binds 

SpinW to Python 

• Tes ted how accurate this 

interface is by comparing 

pySpinW and SpinW res ults 

ω (meV)

Q

Dispersion along [1 1 0] direction 
Square Lattice Ferromagnet

J1 J1

J2

Nearest neighbor model Next-nearest neighbor model
Distinguishing Models

• Can fit for J1 and J2 using BUMPS

• Use Bayesian Information Criterion 

(BIC) to distinguish between models 

• Based on fit + number of 

parameters 

Ques tion: what are the mos t 

informative meas urements to fit 

for J s and dis tinguis h between 

models ? 

next-nearest

nearest

ω (meV)

Q

Dispersion along [1 1 0] direction 
Reinforcement Learning

Defined:

Teaching a computer to make 

optimal decis ions us ing rewards

How does it work?

1. The agent is in an environment

2. The environment returns a state

3. Agent makes action based on 

state

4. Agent is rewarded after action

5. Algorithm learns how to best 

make actions based on rewards
Reinforcement Learning

https://mpatacchiola.github.io/blog/2017/01/15/dissecting-reinforcement-learning-2.html
Applying Reinforcement Learning

• Action: choosing measurement 

• State: all measurements chosen thus far

• Ends episode when chi squared & uncertainty is low 
Applying Reinforcement Learning

• Action: choosing measurement 

• State: all measurements chosen thus far

• Ends episode when chi squared & uncertainty is low 

-----------Reward function --------------------------------------------------------------------------

-100 per measurement taken

+150 when BIC difference > 10, otherwis e 10 * (BIC difference) 

+50 when chi-s quared < 1 and uncertainty < 100

-----------------------------------------------------------------------------------------------------------------
J1

J2

Next-nearest neighbor model

Run

next-nearest

nearest

next-nearest

nearest

Dispersion along [1 1 0] direction 

Dispersion along [1 0 0] direction 

ω (meV)

ω (meV)

Q

Q
J1

J2

Next-nearest neighbor model

Run

next-nearest

nearest

next-nearest

nearest

Dispersion along [1 1 0] direction 

Dispersion along [1 0 0] direction 

ω (meV)

ω (meV)

Q

Q

Diagonal

Not diagonal
Results - Rewards 

• Run with real model as 

next-nearest neighbor 

• General upward trend in 

rewards – means it’s 

learning!

• Need to run for longer
Results – Overall Goals

Number of measurements per episode Model chosen at the end of an episode
Results – Measurement Distribution

Measurement Measurement

First 1000 Episodes Last 1000 Episodes
Next Steps

1.Calculating neutron intensities rather than just the dispersion

• current difficulty with python bindings to SpinW

2.Including the finite resolution of the instrument

3.Exploring the use of alternative methods

• Gaus s ian Proces s es 

4. Publishing
Thanks to the SURF program 

directors, interns and mentors.

Special thanks to William Ratcliff.

We are grateful for funding from 

the Center for High Resolution 

Neutron Scattering.

Acknowledgements 
Any Questions?
Magnet Lattice Example

J1

J2

Spin

Coupling 

interaction
Goals & 

Impact

Beam time is valuable –

limited acces s

Want more efficient 

meas urements as not all 

are required

Software to be implemented 

on ins truments us ing NICE
Magnetic 

Crystal 

Structures

Square lattice s tructure
Magnetic 

Crystal 

Structures

Neares t neighbor model 
Magnetic 

Crystal 

Structures

Next-neares t neighbor model 
Key Parameters 

J1 – coupling cons tant for 

neares t neighbor interactions 

J2 – coupling cons tant for nextneares t neighbor interactions
Defining the Problem

You know the bas ic s tructure  how do you know which interactions 

there are & their coupling cons tants ?
Spinwaves

Nearest neighbor model Next-nearest neighbor model 

[0, 0, 1]
GOAL

Minimize the number of measurements 

necessary by implementing a way to 

determine the most useful measurements. 

-- useful: distinguish between different 

models & find correct values
Results
Results

Conclusion