Turns out I was sending a ton of packets to the entire player list on the server when it wasn't actually necessary at all, switched to new sendToReceivers() method