The attorney is right.  Under the Fair Use doctrine, even taking it down immediately does not mean you can not be hit with a suit and a fine.  I would spend your money wisely here and seek legal help in negotiating the matter.  It is a very expensive lesson learned.  Good luck.