Non-prescription pain relievers like ibuprofen or acetaminophen can be used for temporary relief from kidney pain, but consulting a doctor before taking any medication is important.