Absolutely yes, there is no law that prohibits you from doing that! And it looks good too. You can throw on a topcoat or a blazer over your puffer vest for the best results. This does create an edgy smart look. There are other options available too, a preppy bomber or varsity jacket should look just as good.