A kidney cyst may become infected, which can cause fever and pain. People who have a kidney cyst that bursts feel a lot of pain in their back or side.