I always recommend to choose the size which fits your chest width. The patterns have enough seam allowances on the side seams to adjust the garment accordingly. Once you have sewn your zipper in place, you should try it on and adjust – for example – the waistline.