Vinegar is too weak to use on brick mortar. You'll have to step it up and use hydrochloric acid. Muriatic acid is the common name of this acid when you look for it at a hardware store, building-supply store or home center.