We got lucky twice. 1. Bayonne is an extremely nice and romantic place, we would have never expected this. 2. this place is right in the center of the old town, so you just get out and have great bars and restaurants, all around. All within walking distance.
And if you have a car, within 20 mins you are in Biarritz, which has an awesome beach, and within 40 mins you are in San Sebastian, which is awesome all around.