Only restrict the buttons instead of blocking all player commands
Fix restrictions on ladders and actually restrict movement again