We arrived very late and only stayed in Agrigento for one night.
We were on our way from Palermo to Agrigento to visit the temples and the scala dei Turchi.
Didn't have/take the time to visit the city center but don't think we missed out on anything either. (nothing special)
Gabriele and Yuki were nice and welcoming hosts.
I would not recommend to stay in the city center if you are not visiting the center anyway. Streets are very steep and narrow to drive through.
Plus was the parking spot near the B&B and the pass to park there for free (given by the hosts)
But if you are only in Agrigento to visit the temples and scala dei Turchi, there are other nice b&b's outside of the city center so much easier to reach.