<PAGE>
 
                   EXHIBIT A TO AGREEMENT AND GENERAL RELEASE
Exhibit 10.32



                              SEVERANCE AGREEMENT
                              -------------------

                                        
     By and between WILLIAM J. TAAFFE ("Mr. Taaffe"), a resident of New York and
IRON AGE CORPORATION, a Delaware corporation ("Iron Age")

                                  WITNESSETH:
                                  -----------
     1.  Mr. Taaffe hereby resigns as President and Chief Operating Officer of
Knapp Consumer Brands Division of Iron Age Corporation as of the date hereof.

     2.  Mr. Taaffe shall begin a terminal leave of absence effective as of the
date hereof, subject to the following terms and conditions:

     (i) Mr. Taaffe shall continue to receive his current gross salary of
$4,038.46 payable bi-weekly through July 1, 1999.

     (ii) Mr. Taaffe shall be released of all further duties and shall not
report to work on and after the date hereof.

     (iii)  Mr. Taaffe shall remain fully covered by all Company benefits plans
as in effect from time to time from the date hereof through July 1, 1999;
provided, however, that if Mr. Taaffe becomes covered by a medical plan of
another employer his medical coverage with Iron Age shall cease as of such date.

     (iv) Mr. Taaffe shall continue to participate in the Iron Age Corporation
Profit Sharing Retirement Plan through July 1, 1999 and on such date shall be
deemed to have completed 1,000 Hours of Service during the Plan Year ended
December 31, 1999.  It is the
<PAGE>
 
intent of this paragraph that Mr. Taaffe shall be fully vested in his entire
account in the Profit Sharing Retirement Plan on July 1, 1999.

     (v) Mr. Taaffe shall have the right to exercise any or all of his vested
options under the Iron Age Holdings Corporation 1997 Stock Option Plan through
and until July 30, 1999 at which time all his nonexercised options shall be
deemed terminated and canceled.

     3.  Mr. Taaffe shall return to the Company all proprietary information,
including trade secrets, relating to the Company or any of its affiliated
companies currently in his possession, including original agreements and any
copies thereof.

     4.  Mr. Taaffe shall keep confidential and shall not disclose to any third
party other than his attorneys or other personal advisors any proprietary
information, including trade secrets, relating to the Company or any of its
affiliated companies which he obtained during his employment with the Company.

     5.  It is understood and agreed that any violation by Mr. Taaffe of this
Severance Agreement shall entitle the Company to terminate and cancel this
Severance Agreement without any further liability to Mr. Taaffe.

     IN WITNESS WHEREOF, and intending to be legally bound hereby, each of the
parties hereto has caused this Severance Agreement to be executed as of the
dates indicated.

                                IRON AGE CORPORATION

Date:  _____________________    By:_______________________
                                     Chairman and CEO

                                /s/  
Date:  _____________________    __________________________
                                     WILLIAM J. TAAFFE

                                      -2-