<PAGE>   1
                                                                  EXHIBIT (10)h.


                           SIXTH AMENDMENT TO MODIFIED
                           AND RESTATED LOAN AGREEMENT

         THIS SIXTH AMENDMENT TO MODIFIED AND RESTATED LOAN AGREEMENT (the
"Sixth Amendment") dated as of October 4, 2000, is to that Modified and Restated
Loan Agreement dated as of September 24, 1997, as amended January 30, 1998,
March 31, 1998, August 1, 1998, December 11, 1998 and November 5, 1999
(hereinafter, such Loan Agreement as amended hereby, and as further amended or
modified from time to time, the "Loan Agreement"; all terms used but not
otherwise defined herein shall have the meanings provided in the Loan
Agreement), by and among GENESCO INC. (the "Borrower"), the banks and financial
institutions on the signature pages hereto (the "Banks"), BANK ONE, NA (formerly
known as The First National Bank of Chicago), as Co-Agent for the Banks (the
"Co-Agent"), and BANK OF AMERICA, N.A. (formerly known as NationsBank, N.A.), as
Agent for the Banks (in such capacity, the "Agent").

                              W I T N E S S E T H:

         WHEREAS, the Borrower has requested certain modifications to the Loan
Agreement; and

         WHEREAS, the Banks have agreed to the requested modifications on the
terms and conditions herein set forth;

         NOW, THEREFORE, IN CONSIDERATION of these premises and other good and
valuable consideration, the receipt and sufficiency of which is hereby
acknowledged, the parties hereto agree as follows:

         A.       The Loan Agreement is amended and modified in the following
respects:

                  (1) Section 7.5.4 shall be amended in its entirety so that
such Section now reads as follows:

                           7.5.4 Capital Expenditures. The Borrower will not,
                  and will not permit any of its Subsidiaries to, purchase or
                  otherwise acquire, or commit to purchase or otherwise acquire,
                  any fixed or capital asset or otherwise make or incur
                  obligations for Capital Expenditures by the expenditure of
                  cash or the incurrence of Indebtedness, the cost of which (or,
                  in the case of any acquisition not in the nature of an
                  ordinary purchase, the book value of the consideration given
                  for which), when aggregated with the costs of all other such
                  assets purchased or otherwise acquired by the Borrower and its
                  Subsidiaries taken as a whole during such Fiscal Year, would
                  exceed $36,000,000 during any Fiscal Year (commencing with the
                  Fiscal Year ending January 31, 2001); provided, that, if
                  during any Fiscal Year Capital Expenditures are less than
                  $36,000,000, the lesser of (i) the difference between
                  $36,000,000 and the actual Capital Expenditures for such
                  Fiscal


<PAGE>   2

                  Year, or (ii) $3,000,000 (such lesser amount being referred to
                  as the "Excess Capital Expenditures Allowance") shall be
                  carried forward so as to increase the maximum Capital
                  Expenditures which may be made in accordance with this
                  Subsection 7.5.4 for the immediately succeeding Fiscal Year,
                  but not for any other subsequent Fiscal Year, except to the
                  extent permitted by the next succeeding sentence. Capital
                  Expenditures made in any such succeeding Fiscal Year shall be
                  applied first to the Excess Capital Expenditures Allowance
                  carried forward until such Allowance is exhausted and shall
                  then be applied to the maximum Capital Expenditures specified
                  above for such Fiscal Year in determining whether an Excess
                  Capital Expenditure Allowance is available to be carried
                  forward to the next succeeding Fiscal Year in the manner
                  described in this Subsection 7.5.4.

                  (2) Section 7.6 shall be amended in its entirety so that such
         Section now reads as follows:

                           7.6 Restrictions on Fundamental Changes. The Borrower
                  will not, and will not permit any of its Subsidiaries to (i)
                  enter into any transaction of merger or consolidation, or
                  liquidate, wind up or dissolve itself (or suffer any
                  liquidation or dissolution), or (ii) convey, sell, lease,
                  transfer or otherwise dispose of subsequent to the Closing
                  Date, in one or more transactions, all or any portion of its
                  business, properties or assets (real and personal, tangible
                  and intangible) or any stock or other Securities of any of its
                  Subsidiaries, whether now owned or hereafter acquired,
                  constituting in the aggregate for all of such transactions
                  consummated on or after the end of the second fiscal quarter
                  of Fiscal Year 2001 more than 10% of Consolidated Tangible
                  Assets as of the end of the second fiscal quarter of Fiscal
                  Year 2001; provided, that, so long as no Event of Default or
                  Potential Default has occurred and is continuing or would
                  occur as a result thereof, (x) any Subsidiary of the Borrower
                  may be merged or consolidated with or into the Borrower or any
                  direct wholly-owned Subsidiary of the Borrower, or be
                  liquidated, wound up or dissolved, or all or substantially all
                  of its business, properties or assets (real and personal,
                  tangible and intangible) may be conveyed, sold, leased,
                  transferred or otherwise disposed of, in one transaction or a
                  series of transactions, to the Borrower or any direct
                  wholly-owned Subsidiary of the Borrower; and (y) the Borrower
                  or any of its Subsidiaries may acquire any Person by merger or
                  consolidation, provided that the Borrower or such Subsidiary
                  is the corporation surviving such merger or consolidation, in
                  any transaction that would not cause an Event of Default or
                  Potential Default under this Loan Agreement.

         B.       The Borrower hereby represents and warrants that:



                                       2
<PAGE>   3

                  (i) any and all representations and warranties made by the
         Borrower and contained in the Loan Agreement (other than those which
         expressly relate to a prior period) are true and correct in all
         material respects as of the date of this Sixth Amendment; and

                  (ii) No Default or Potential Default currently exists and is
         continuing under the Loan Agreement simultaneously with the execution
         of this Sixth Amendment.

         C.       The Borrower will execute such additional documents as are
reasonably requested by the Agent to reflect the terms and conditions of this
Sixth Amendment.

         D.       Except as modified hereby and except for necessary
modifications to exhibits to bring such exhibits in conformity with the terms of
this Sixth Amendment, all of the terms and provisions of the Loan Agreement (and
Exhibits) remain in full force and effect.

         E.       The Borrower agrees to pay all reasonable costs and expenses
in connection with the preparation, execution and delivery of this Sixth
Amendment, including without limitation the reasonable fees and expenses of the
Agent's legal counsel.

         F.       This Sixth Amendment may be executed in any number of
counterparts, each of which when so executed and delivered shall be deemed an
original and it shall not be necessary in making proof of this Sixth Amendment
to produce or account for more than one such counterpart.

         G.       This Sixth Amendment and the Loan Agreement, as amended
hereby, shall be deemed to be contracts made under, and for all purposes shall
be construed in accordance with the laws of the State of Tennessee.





                  [Remainder of Page Intentionally Left Blank]




                                       3
<PAGE>   4

         IN WITNESS WHEREOF, each of the parties hereto has caused a counterpart
of this Sixth Amendment to be duly executed under seal and delivered as of the
date and year first above written.

BORROWER:

                                    GENESCO INC.,
                                    a Tennessee corporation



                                    By /s/ James S. Gulmi
                                       -----------------------------------------

                                    Title Senior Vice President - Finance
                                          --------------------------------------

BANKS:

                                    BANK OF AMERICA, N.A.,
                                    individually in its capacity as a
                                    Bank and in its capacity as Agent



                                    By /s/ Timothy H. Spanos
                                       -----------------------------------------

                                    Title Managing Director
                                          --------------------------------------

                                    BANK ONE, NA (Main Office - Chicago,
                                    formerly known as The First National Bank of
                                    Chicago),
                                    individually in its capacity as a Bank and
                                    in its capacity as a Co-Agent



                                    By /s/ Catherine A. Muszynski
                                       -----------------------------------------

                                    Title Vice President
                                          --------------------------------------






                                       4