Improving the 

Robustness of Bumps 

using Parallel Tempering

KENNETH JIANG

UNDER PAUL KIENZLE
• Data fitting and uncertainty analysis package for Python

• Takes in a set of data and an equation format

• Searches for the best parameters that will fit

• Utilized for reflectometry, small angle neutron 

scattering, and crystallography analysis

http://bumps.readthedocs.or

g/en/latest/_static/logo.png

Bumps
Example

-1

-0.5

0

0.5

1

1.5

2

2.5

3

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6
Markov Chain Monte Carlo

• Technique for sampling from the 

posterior distribution in very complex 

models

• Acceptance: 

• Problems

• Determining step size and direction

P= ee−∆EE

Parameter 1

Parameter 2
Differential Evolution Adaptive Metropolis 

(DREAM)

• Runs multiple chains in parallel

• Utilizes the differential evolution 

metaheuristic

1. Take two points in the population

2. Use them to compute a vector

3. Use that vector to compute a step for 

a third point http://cdn.iopscience.com/images/0067-0049/210/1/11/Full/apjs488859f1_lr.jpg
The problem

• Problems with complex distributions are slow to converge

https://upload.wikimedia.org/wikipedia/commons/thumb/d/d4/Correlation_examples2.svg/2000px-Correlation_examples2.svg.png
Parallel Tempering

• Temperature Parameter

• High temperatures – cover large areas

• Low temperatures – explore local 

minima High Temperature

Low Temperature

P= ee

−∆EE

TT

Parameter

Energy
Example – Gaussian Mixture Model
Parallel Tempering

• Multiple chains in parallel

• Chains are occasionally swapped

http://www.iearth.org.au/images/PT-scheme.png

PP = ee∆EE∆ββ

∆ββ =

1

TTii+1

− 1

TTii
Optimizations

• Step Scaler

• Holds average acceptance rate around 

23.4%

• Minimizes the step size issue

• Step Switcher

• Switches between 3 steppers based on 

performance

• DE

• Jiggle (temperature-dependent)

• Subspace jiggle (when chains are 

completely stuck)

• Dynamic Temperatures

• Reallocate temperatures to critical levels
Tests

• 10 Runs

• Problems

• Model Functions (40,000 steps)

• Spinal Reconstruction Problems (40,000 steps)

• Reflectivity Test Case (70,000 steps)
Testing - Model functions

• 10 parameters

• Find the global minima

Rosenbrock Function Ackley Function
Testing - Spline Reconstruction

• 16 parameters

• Find the control points for a B-spline

• Data is randomly generated each run
Testing – Reflectivity Analysis

• 18 parameters

• Density and Thickness of each layer
Data

1

0.5

1 1

0.9

0

1

0

0.4

1 1

0

0

0.5

1

REFLECT ACKELY ROSENBROCK SPLINE

CONVERGENCE PERCENTAGE

Parallel Tempering

Dream

DE
Discussion

• Parallel Tempering performed very well in three out of the four tests

• Faltered on the Ackley Function

• Due to large amount of local minima 

• Adaptive Stepper seemed to perform well on other reflectivity 

problems

• No Free Lunch Theorem

• Not one algorithm will be able to solve every problem
Future Work

• Maximum and minimum temperature adjuster

• Increase exploration if needed

• Reweighted histograms/full implementation of MCMC

• Implementing an adaptive stepper in DREAM
Acknowledgements

• Paul Kienzle 

• Julie Borchers and Yamali Hernandez

• SHIP program 

• NSF/CHRNS 

• All of my teachers