Original: The town also used to have a much bigger population back when the neighboring DOE site was bigger.
Simplified: The town had a bigger population when the nearby DOE site was bigger.