Turns out you do need to have .rst extensions for GitHub to interpret files correctly.