the only possible way around this is to look into possible grandfather clause. However if you had not moved to build or taken any steps this may not apply. unfortunately ordinances can be changed and sometimes situations like this will happen.I am not admitted in WI so I would suggest you call a local attorney who specializes in this area of law. Tell them your exact location, show them the paperwork that you obtained stating it was legal to build, show them any other efforts made on your part to take the steps to do so. With all the paperwork they will be able to determine if you can make a motion for an exception, but you will have to accept a possible no although its worth pursing. good luck