The famous fish market was closed on Sunday, but some vendors still set up near the wharf