I recommend staying in Pest, preferably in the quiet 8th District. You're close enough to join the party, but far enough away to get some peace and quiet. The Matild Palace is my pick and is perfect for this Budapest itinerary.