Automatically aligning crystal lattices to neutron 

diffraction instruments using reinforcement 

learning

Dylan Zhang

Mentor - Dr. William Ratcliff

University of 

Michigan

1
Crystals

● A type of solid material composed of 

atoms or groups of atoms that are 

arranged in a three-dimensional pattern 

that is very ordered

● Lattice determines the different 

diffraction patterns

2

https://www.expii.com/t/crystal-lattice-structure-formation-7999
Bravais Lattice

● Symmetrical three-dimensional structural 

arrangements

● One of the 14 different types of unit cells 

that a crystal structure can be made up 

of

● Determines the different diffraction 

patterns

3

https://www.chm.bris.ac.uk/webprojects2003/cook

/latticetypes.htm
Crystallographic Planes

Miller Indices (HKL)

● Represent the orientation of 

crystallographic planes within the 

crystal lattice

4

https://www.researchgate.net/figure/Miller-indices-indicating-the-planeperpendicular-to-the-vector-given-for-the-cubic_fig7_302838100
Neutron Diffraction

● Neutron Diffraction is the application of neutron scattering to determine the 

atomic and magnetic structure of a material

● A sample is placed in a beam of neutrons to obtain a diffraction pattern that 

provides information about the structure of the sample

5

https://nmi3.eu/neutron-research/techniques-for-/structuralresearch.html
Objective

● Automatically align crystal 

structures to instruments 

● Assist in fully automating the 

crystal diffraction process

6

https://www.ill.eu/users/instruments/instruments-list/orientexpress/howit-works/principle-of-neutron-laue-diffraction
Alignment

1. Choose a particular plane

2. Choose two projection vectors corresponding to the plane

3. Rotate table

4. Take alignment scans

5. Analyze scans

6. Repeat 3, 4, and 5 until aligned

7

Dr. Leland Harriger
Reinforcement Learning - Key Elements

● Agent

● Environment

● State

● Action

● Reward

8

Area of machine learning where agents take 

actions in an environment in order to maximize 

the notion of cumulative reward

https://towardsdatascience.com/reinforcement-learning-101-e24b50e1d292
Reinforcement Learning - advantages

● Can be used to solve very complex problems 

● Can outperform humans in many tasks

● It is similar to the learning of human beings and can correct errors during 

the training process

● It doesn't require large labeled datasets

9
Reinforcement Learning - examples

● DeepMind

● AlphaFold

● AlphaGo

10

https://venturebeat.com/ai/deepminds-big-lossesand-the-questions-around-running-an-ai-lab/
Q-Learning

● Model-free reinforcement learning 

algorithm

● Q-table that stores the expected 

rewards for state-action pairs

● Q-values updated over multiple 

episodes 

● Agent selects the action with the 

highest Q-value

● Effective for small state spaces

● Different from Value function

11

https://towardsdatascience.com/a-beginners-guide-to-q-learningc3e2a30a653c
Q-learning with tic tac toe

12

https://towardsdatascience.com/how-to-play-tic-tac-toe-usingreinforcement-learning-9604130e56f6
Implementation

● Training:

○ Reward

■ Punished for each rotation

■ Awarded for finding peak

● Decides how much to rotate 

● Rotates automatically until finds diffraction peak

13
Hyperparameters

● Values that control the learning process

● Learning rate, discount factor, episodes

14

https://www.researchgate.net/publication/343594992/figure/tbl2/AS:932830192341025@159941545

7664/Reinforcement-learning-RL-training-hyper-parameters.png
Why/How

● automate some of the 

crystallography process 

● significant because correct 

alignment is necessary 

1. Build the q learning algorithm

2. Train the algorithm using simulated data

3. Evaluate/test the algorithm

4. Tune hyperparameters if needed

5. Evaluate again

Why How

15
Results

16
What’s Next

17

● Mcstas needs to be implemented 

● Implement into different types of 

instruments such as triple-axis 

spectrometer
Acknowledgements ● Dr. Ratcliff ● CHRNS ● Dr. Borchers ● Dr. Harriger ● Mr. Kienzle ● Mr. Krzywon ● Dr. Liu

18