<PAGE>

                                                                   Exhibit 10.4h

                  AMENDMENT NUMBER THREE TO FOURTH AMENDED AND
                            RESTATED CREDIT AGREEMENT

         This Amendment Number Three to Fourth Amended and Restated Credit
Agreement ("Amendment") is entered into as of December 17, 2001, by and between
Jos. A. Bank Clothiers, Inc., a Delaware corporation ("Company"), the various
financial institutions that are or may from time to time become parties to the
Agreement referred to below (collectively, the "Lenders" and each individually a
"Lender"), and Foothill Capital Corporation, a California corporation as Agent
for the Lenders ("Agent") in light of the following:

         A.   Company, Wells Fargo Bank, National Association ("Wells Fargo")
and Lenders have previously entered into that certain Fourth Amended and
Restated Credit Agreement, dated as of April 30, 1996 (as amended from time to
time the "Agreement").

         B.   Wells Fargo, Lenders and Company entered into four amendments to
the Agreement between September 3, 1996 and June 1, 1998 (collectively, the
"Wells Fargo Amendments").

         C.   Wells Fargo has assigned its interest in the Agreement (as amended
by the Wells Fargo Amendments) and related Loan Documents to Foothill pursuant
to that certain Assignment and Acceptance, dated as of June 2, 1999 and Foothill
was appointed Agent under the terms of the Agreement.

         D.   Agent, Lenders and Company have entered into that certain Fifth
Amendment to the Agreement as of December 8, 1999, and that certain Sixth
Amendment to the Agreement as of July 28, 2000 (collectively, the "Foothill
Amendments").

         E. Agent, Lenders and Company have entered into that certain Combined
Amendment Number One to the Fourth Amended and Restated Credit Agreement as of
December 15, 2000 (the "Combined Amendment Number One").

         F.   Agent, Lenders and Company have entered into that certain
Amendment Number Two to the Fourth Amended and Restated Credit Agreement as of
March 27, 2001 (the "Amendment Number Two").

         G.   Company, Lenders, and Agent desire to amend the Agreement as
provided for and on the conditions herein.

         NOW, THEREFORE, Company, Lenders, and Agent hereby amend and supplement
the Agreement as follows:

         1.   DEFINITIONS.  All initially capitalized terms used in this
              -----------
Amendment shall have the meanings given to them in the Agreement unless
specifically defined herein.

<PAGE>

         2.   AMENDMENTS.
              ----------

              (a)  The definition of "Eligible Inventory Borrowing Base" in
     Section 1.1 of the Agreement is amended in its entirety to read as follows:

                   "Eligible Inventory Borrowing Base" means: (i) the lesser of:
         (a) 70% (increasing to 75% during the months of September, October and

         November) of the Value of Eligible Landed Inventory; or (b) 85%
         (increasing to 90% during the months of September, October and
         November) of the Net Liquidation Value of Eligible Landed Inventory;
         plus (ii) the lesser of: (a) $3,000,000; (b) 70% (increasing to 75%
         during the months of September, October and November) of the Value of
         Eligible L/C Inventory; and (c) 85% (increasing to 90% during the
         months of September, October and November) of the Net Liquidation Value
         of Eligible L/C Inventory; minus (iii) the Inventory Reserve.
                                    -----

              (b)  Section 1.1 of the Agreement is amended by adding the
following definition:

                   "Inventory Reserve" means a reserve against the Eligible
         Inventory Borrowing Base in the amount of $5,000,000.

              (c)  The definition of "Maximum Revolving Amount" in Section 1.1
of the Agreement is amended in its entirety to read as follows:

                   "Maximum Revolving Amount" means $60,000,000.

              (d)  The definition of "Maturity Date" in Section 1.1 of the
Agreement is amended in its entirety to read as follows:

                   "Maturity Date" means April 30, 2005.

              (e)  Section 2.4(A.)(ii) of the Agreement is amended in its
entirety to read as follows:

                   (ii) an unused line fee equal to: (a) $55,000,000, minus the
         average of the daily amount of Revolving Loans; times (b) 0.375% per
         annum calculated on the basis of a 360-day year and the actual number
         of days elapsed and payable monthly in arrears on the first day of each
         calendar month through the Maturity Date so long as this Agreement has
         not terminated, provided, however, that if this Agreement terminates on
         a date other than the 1st of a month, the unused line fee will be
         prorated for the partial month on which the termination occurs and
         payable on the date of termination;

              (f)  Section 5.1(i) of the Agreement is amended in its entirety to
read as follows:

<PAGE>

              (i)  Borrowing Base Certificates. If Company has Revolving Loan
                   ---------------------------
         borrowing availability of less than $5,000,000, prior to noon,
         California time, on each Tuesday, a borrowing base certificate, in form
         and detail acceptable to Agent, as of the close of Company's business
         on the immediately preceding Saturday; or if Company has Revolving Loan
         borrowing availability of $5,000,000 or more, prior to noon, California
         time, on the third Business Day of any fiscal month, a borrowing base
         certificate, in form and detail acceptable to Agent, as of the close of
         Company's business on the last day of the immediately preceding fiscal
         month;

         (g)  Section 5.14 of the Agreement is hereby deleted in its entirety
and replaced with a new Section 5.14 as follows:

              5.14 [Intentionally Deleted.]
                    ---------------------

         (h)  Section 6.6 of the Agreement is amended in its entirety to read as
follows:

              6.6  Financial Covenants
                   -------------------

              Agent shall set a minimum EBITDA covenant for each fiscal month
         commencing with the fiscal month ending on or about March 31, 2001 and
         a maximum Capital Expenditure covenant for each fiscal year commencing
         with the fiscal year ending on or about January 31, 2002. The covenants
         shall be based upon Company's projections for the fiscal year
         containing that fiscal month and fiscal year, which projections must be
         delivered to Agent prior to the commencement of such fiscal year. The
         projections shall be satisfactory to Agent in its reasonable credit
         judgment. Agent shall set the future periods EBITDA covenant based on
         85% of the expected performance set forth in the projections delivered
         to Agent. The covenants set forth in this Section will not be
         applicable to Company in any fiscal month during which Company
         maintains a daily average of Revolving Loans borrowing availability of
         $5,000,000 or more, provided, no Event of Default has occurred and is
         continuing.

         (i)  National Tailoring Services, Inc., a Delaware corporation ("NTS")
and Company have previously entered into that certain Agreement and Plan of
Merger, dated as of August 5, 2001 (the "Merger Agreement"). Pursuant to the
terms of the Merger Agreement: (i) NTS ceases to exist as a separate corporate
entity; (ii) NTS merges with and into Company; and (iii) Company assumes all of
the assets and liabilities of NTS. Company, Lenders, and Agent hereby agree that
all references to "National Tailoring Services, Inc., a Delaware corporation" or
to "NTS" in the Agreement and all other Loan Documents shall be and mean, and
shall be revised to say, "Jos. A. Bank Clothiers, Inc., a Delaware corporation"
or "Company", respectively.

         (j)  Company acknowledges and agrees that the Loan Documents to which
NTS is a party are valid, enforceable and in full force and effect and that: (i)
Company assumes the full payment and performance of all Obligations thereunder;
and (ii) all rights of Lenders and

<PAGE>

Agent under the Loan Documents to which NTS is a party and all liens upon any
property of NTS, are valid, enforceable and in full force and effect against
Company.

         3.   REPRESENTATIONS AND WARRANTIES.  Company hereby affirms to Agent
              ------------------------------
and the Lenders that all of Company's representations and warranties set forth
in the Agreement are true, complete and accurate in all respects as of the date
hereof.

         4.   NO DEFAULTS.  Company hereby affirms to Agent and the Lenders that
              -----------
no Event of Default has occurred and is continuing as of the date hereof.

         5.   CONDITIONS PRECEDENT.  The effectiveness of this Amendment is
              --------------------
expressly conditioned upon the receipt by Agent of a copy of this Amendment
executed by Company and Requisite Lenders.

         6.   COSTS AND EXPENSES.  Company shall pay to Agent all of Agent's
              ------------------
out-of-pocket costs and expenses (including, without limitation, the fees and
expenses of its counsel, which counsel may include any local counsel deemed
necessary, search fees, filing and recording fees, documentation fees, appraisal
fees, travel expenses, and other fees) arising in connection with the
preparation, execution, and delivery of this Amendment and all related
documents.

         7.   LIMITED EFFECT.  In the event of a conflict between the terms and
              --------------
provisions of this Amendment and the terms and provisions of the Agreement, the
terms and provisions of this Amendment shall govern. In all other respects, the
Agreement, as amended and supplemented hereby, shall remain in full force and
effect.

         8.   COUNTERPARTS; EFFECTIVENESS. This Amendment may be executed in any
              ---------------------------
number of counterparts and by different parties on separate counterparts, each
of which when so executed and delivered shall be deemed to be an original. All
such counterparts, taken together, shall constitute but one and the same
Amendment. This Amendment shall become effective upon the execution of a
counterpart of this Amendment by each of the parties hereto.

         IN WITNESS WHEREOF, the parties hereto have executed this Amendment as
of the date first set forth above.

                              FOOTHILL CAPITAL CORPORATION,
                              a California corporation, as Agent and as the sole
                              Lender


                              By:   /s/ THOMAS F. MORGAN
                                   -----------------------------
                              Title:  Vice President
                                      --------------------------

<PAGE>

                              JOS. A. BANK CLOTHIERS, INC.,
                              a Delaware corporation


                              By:   /s/
                                   -----------------------------------
                                   David E. Ullman, EVP-CFO

<PAGE>

         Each of the undersigned has executed a Continuing Guaranty in favor of
Foothill Capital Corporation, as Agent for the Lenders (as defined in the
Amendment) ("Foothill") respecting the obligations of Jos. A. Bank Clothiers,
Inc., a Delaware corporation ("Company") owing to Agent and Lenders. Each of the
undersigned acknowledges the terms of the above Amendment and reaffirms and
agrees that: its Continuing Guaranty remains in full force and effect; nothing
in such Continuing Guaranty obligates Agent or any Lender to notify the
undersigned of any changes in the financial accommodations made available to
Company or to seek reaffirmations of the Continuing Guaranty; and no requirement
to so notify the undersigned or to seek reaffirmations in the future shall be
implied by the execution of this reaffirmation.

                              JOSEPH A. BANK MFG. CO., INC.,
                              a New Jersey corporation


                              By:   /s/
                                   ----------------------------------
                                   David E. Ullman, EVP-CFO


                              RS SERVICING CO., INC.,
                              a Delaware corporation


                              By:  /s/
                                  ----------------------------------
                                  David E. Ullman, President


                              IS SERVICING CO., INC.,
                              a Delaware corporation


                              By:  /s/
                                  ----------------------------------
                                  David E. Ullman, President