Reinforcement Learning 

Algorithms for Taking Efficient 

Crystallographic Measurements

Ryan Cho and Telon Yan

Montgomery Blair High School

Abigail Wilson, SURF Student

Steven G. Satterfield, Mentor

Judith E. Terrell, Mentor

William D. Ratcliff, Mentor
Introduction

Crystallography: Study of crystalline 

structures and their properties

Neutrons are a scarce resource:

Neutron diffraction is time 

consuming and there are few 

facilities to perform it

Goal: Apply Artificial Intelligence to 

reduce beam time 

https://serc.carleton.edu/research_education/

geochemsheets/techniques/SXD.html - 

Single Crystal Diffraction Schematic
Neutron Diffraction

● Powder Diffraction

○ Faster

○ Cheaper

○ Less Information

● Single Crystal Diffraction

○ Slower

○ More information

Our project is based on Single 

Crystal Diffraction

https://upload.wikimedia.org/wikipedia/commo

ns/2/20/Diffractometre_berceau_Euler.png - 

Eulerian Cradle
Unit cell: Smallest repeating structure; 

resonant of the whole structure

Miller Indices: Coordinate representations 

of planes in a unit cell

Instruments for rotating crystals allows us 

to target specific Miller indices and take 

measurements

Miller Indices (hkl’s)

https://en.wikipedia.org/wiki/Miller

_index - Miller Indices

https://chem.libretexts.org - Crystal 

Structures
Reinforcement Learning (RL)

Machine Learning: Repeated 

Interaction with Data → “Learning”

Agent: Interacts with Environment

Environment: Returns Information

Examples: OpenAI, Deepmind, 

AlphaGo, etc. 

Various Distinct Algorithms 

https://becominghuman.ai/the-very-basics-of-reinf

orcement-learning-154f28a79071 - Becoming 

Human
Two Processes

● Exploitation: Capitalize on 

known information to make 

best choice

● Exploration: Learn more about 

the environment by sacrificing 

performance

Epsilon: Probability of exploration

Epsilon Greedy

https://imaddabbura.github.io/blog/data%20sc

ience/2018/03/31/epsilon-Greedy-Algorithm.

html - Bandit Algorithms: epsilon-Greedy 

Algorithm
Methods

Agent: Instance of Epsilon Greedy 

Environment: All Possible Miller Indices

Crystals: Pr2NiO4, MoTe2

Software and Libraries: Python, PuTTY, 

BLAND (pycrysfml, bumps), Docker, Linux 

Terminal, VR Cave

Variables: Z-coordinate approximation, 

starting values

http://iopscience.iop.org/a

rticle/10.1088/0953-8984/

24/40/405504 - Analysis 

of structural and 

electronic properties of 

Pr2NiO4 through 

first-principles 

calculations

https://www.nature.com/articles/n

comms13552 - Raman signatures 

of inversion symmetry breaking 

and structural phase transition in 

type-II Weyl semimetal MoTe2
Our Model
Methods

We develop and run agents using pycrysfml and bumps to efficiently select 

Miller Indices while finding a viable location approximation for the 

praseodymium or molybdenum atom. 

Data Collection

● Z-approximations for Praeseodymium/Molybdenum

● Divergence from actual Z-value (Residuals)
Run Specs

50 Sets x 800 simulations for each starting value and crystal

CRYSTAL Pr2NiO4 MoTe2

Expected 

Z-Approximation

Praseodymium Z 

Coordinate = 0.356

Molybdenum Z 

Coordinate = 0.449

Agent’s Starting Values {0.2, 0.3, 0.4} {0.3, 0.4, 0.5}
Results - MoTe2
Results - Pr2NiO4
Results - Starting Points

The closer the starting point was to the actual approximation, the faster and 

more accurately the agent converged

At some distances, there was significant improper convergence, meaning 

there is likely a threshold beyond which RL cannot function

Pr2NiO4 (Z = 0.356)

Init. Z 0.2 0.3 0.4

Convergence 

Rate

0% 42% 60%

MoTe2 (Z = 0.449)

Init. Z 0.3 0.4 0.5

Convergence 

Rate

2% 100% 100%
Virtual Reality Cave (ITL)
Movie Time!
Visualizing Our Algorithm

MoTe

2 Pr

2NiO4
Conclusion

● Reinforcement Learning is a viable mechanism for planning efficient 

crystallographic measurements

● Starting parameters for RL agents are crystal-specific, and need to be 

adjusted per crystal

● Epsilon Greedy has limited functionality, so exploration into more 

advanced algorithms is warranted
Acknowledgements

Special thanks to mentors Dr. William D. Ratcliff, Steven G. Satterfield, and 

Judith E. Terrill. 

Much thanks to Abigail Wilson, Terrence Griffin, Paul Kienzle, and Yang 

Zhao for their assistance

We would like to acknowledge the SHIP Program, NCNR, and ITL, as well as 

funding from CHRNS, and Angelique Bosse for her help with obtaining this 

opportunity.