<PAGE>   1
                                                                     EXHIBIT 4.9



                               FIRST AMENDMENT TO
                              TERM CREDIT AGREEMENT

                      (C $75,000,000 Term Credit Agreement)


                  THIS FIRST AMENDMENT TO TERM CREDIT AGREEMENT dated as of
September 14, 1999 (this "Amendment"), by and among GOLDEN MOORES FINANCE
COMPANY, (the "Borrower"), THE MEN'S WEARHOUSE, INC. (the "Parent"), the
financial institutions listed on the signature pages hereto (the "Banks") and
BANK OF AMERICA, N.A., formerly known as NationsBank, N.A., in its capacity as
agent (the "Agent") and in its individual capacity as a Bank hereunder.

                  WHEREAS, the Borrower, the Parent, the Agent and the Banks
have entered into that certain Term Credit Agreement dated as of February 5,
1999 (together with any and all amendments and modifications thereof, the
"Credit Agreement"); and

                  WHEREAS, the Credit Agreement provides that upon the
designation, formation or acquisition of any Restricted Subsidiary, the Parent
shall cause such Restricted Subsidiary to deliver to the Agent for the benefit
of the Banks a guaranty of the obligations of the Borrower in the form of a
guaranty supplement set forth as Exhibit A to the Affiliate Guaranty
(capitalized terms used herein but not otherwise defined herein shall have the
meanings assigned to them in the Credit Agreement); and

                  WHEREAS, the Parent has acquired K&G of Ohio, Inc., a Georgia
corporation ("K&G Ohio"), and Gares Cigars, LLC, a Georgia limited liability
company ("Gares"), as Subsidiaries; and

                  WHEREAS, K&G Ohio and Gares are de minimis Subsidiaries, and
the Parent has requested that the Credit Agreement be amended to provide that
such Subsidiaries shall not be required to deliver a guaranty supplement unless
there is a substantial increase in the net worth of such Subsidiaries and in
several other respects as provided for herein;

                  NOW, THEREFORE, in consideration of the premises, and for
other good and valuable consideration, the receipt and sufficiency of which are
hereby acknowledged, the parties hereto agree as follows:

                  Section 1. Amendments to Credit Agreement.

                  (a) The definition of "Restricted Subsidiary" set forth in
Section 1.2 of the Credit Agreement is hereby amended by deleting the sentence
"Except for directors' qualifying shares and the Exchangeable Shares, each
Restricted Subsidiary shall be directly or indirectly wholly-owned by the
Parent." and by inserting in lieu thereof the sentence "Except for directors'
qualifying shares, the Exchangeable Shares and minority interests, if any, in
K&G of Ohio, Inc. and Gares Cigars, LLC, each Restricted Subsidiary shall be
directly or indirectly wholly-owned by the Parent; provided, however, that in
the event K&G of Ohio, Inc. or Gares Cigars, LLC shall become directly





<PAGE>   2

or indirectly wholly-owned by the Parent, such entity shall thereafter remain
directly or indirectly wholly-owned by the Parent."

                 (b) Section 9.7 of the Credit Agreement is hereby amended by
adding at the end thereof the following language:

                 It is furthermore agreed and understood that upon the
       acquisition by the Parent of K&G of Ohio, Inc. and Gares Cigars, LLC,
       each such Subsidiary shall be a de minimis Subsidiary, and therefore,
       each such Subsidiary shall be a Restricted Subsidiary but shall not be
       required to execute and deliver a guaranty of the obligations of the
       Borrower as of the date of such Subsidiary's acquisition. If there is a
       substantial increase in the net worth of K&G of Ohio, Inc. or Gares
       Cigars, LLC, as applicable, after the date of such Subsidiary's
       acquisition, the Parent agrees to cause such Restricted Subsidiary to
       become an Affiliate Guarantor upon the request of the Agent.

                 (c) Schedule 7.17 to the Credit Agreement is hereby amended by:

       (i)    under the caption "I. Guarantors", deleting the language
              "Value Priced Clothing, Inc. (100% owned by The Men's
              Wearhouse, Inc.)."

       (ii)   adding the following entities and related information under
              the caption "I. Guarantors":

              K & G Men's Center, Inc. (100% owned by The Men's Wearhouse, Inc.
              K&G of Indiana, Inc. (100% owned by K & G Men's Center, Inc.)
              K&G Associates of New Jersey, Inc. (100% owned by K & G Men's
                  Center, Inc.)
              T & C Liquidators, Inc. (100% owned by K&G Men's Center, Inc.)
              Value Priced Clothing, LLC (100% owned by The Men's Wearhouse,
                  Inc.) (formed July 7, 1999; Value Priced Clothing, Inc. merged
                  into Value Priced Clothing, LLC effective as of July 30,
                  1999); and

       (iii)  adding the following entities and related information under
              the caption "II. Non-Guarantors":

              K&G of Ohio, Inc. (On June 1, 1999, 60% owned by K & G Men's
                  Center, Inc. As of September 3, 1999, 100% owned by K & G
                  Men's Center, Inc.)
              Gares Cigars, LLC (at least 70% owned by K & G Men's Center,
                  Inc.).

                  (d) Schedule 10.2 to the Credit Agreement is hereby amended by
adding the following language at the end thereof:



                                      -2-
<PAGE>   3


         D.       K & G Men's Center, Inc. made a loan, payable on demand, to
                  K&G of Ohio, Inc. in the amount of $270,000, bearing interest
                  thereon at a rate of 6% per annum, as evidenced by that
                  certain Promissory Note dated June 10, 1994.

         E.       Edward C. Dworetz made a loan, payable on demand, to K&G of
                  Ohio, Inc. in the amount of $180,000, bearing interest thereon
                  at a rate of 6% per annum, as evidenced by that certain
                  Promissory Note dated June 10, 1994. Such loan was repaid
                  effective as of September 3, 1999.

                  Section 2. Conditions to Effectiveness. This Amendment shall
become effective as of June 1, 1999 (the "Effective Date") upon execution and
delivery by a duly authorized officer of each of the Borrower, the Parent, K&G
Ohio, Gares, the Agent and the Majority Banks; provided that the schedules
amended hereby shall reflect changes thereto occurring during the period from
June 1, 1999 through September 7, 1999.

                  Section 3. Ratification of Related Documents. The Credit
Agreement and each other Loan Document and any related document to which the
Borrower or the Parent is a party are hereby ratified and confirmed to be in
full force and effect.

                  Section 4. Limitations. The modifications set forth herein are
limited precisely as written, and shall not be deemed to (a) be a consent to, or
waiver or modification of, any other term or condition of the Credit Agreement
or any of the other Loan Documents, or (b) prejudice any right or rights which
the Banks may now have or may have in the future under or in connection with the
Credit Agreement or any of the other Loan Documents. Except as expressly
modified hereby, the terms and provisions of the Credit Agreement and the other
Loan Documents are and shall remain in full force and effect. In the event of a
conflict between this Amendment and any of the foregoing documents, the terms of
this Amendment shall be controlling.

                  Section 5. Representations and Warranties; No Default. The
Parent hereby represents and warrants that on and as of the date hereof, and
after giving effect hereto: (i) the representations and warranties of the Parent
made in Article 7 of the Credit Agreement shall be true and correct; and (ii) no
Default or Event of Default shall have occurred and be continuing.

                  Section 6. Payment of Expenses. The Parent agrees to pay and
reimburse the Agent for all its reasonable costs and out-of-pocket expenses
incurred in connection with the preparation, execution and delivery of this
Amendment and ancillary documents, including, without limitation, the reasonable
fees and disbursements of counsel to the Agent.

                  Section 7. Choice of Law. THIS AMENDMENT AND THE RIGHTS AND
OBLIGATIONS OF THE PARTIES HEREUNDER SHALL BE CONSTRUED IN ACCORDANCE WITH AND
BE GOVERNED BY THE LAWS OF THE STATE OF NEW YORK AND THE UNITED STATES OF
AMERICA.



                                      -3-
<PAGE>   4



                  Section 8. Descriptive Headings. The descriptive headings of
the several Sections of this Amendment are inserted for convenience only and
shall not be deemed to affect the meaning or construction of any of the
provisions hereof.

                  Section 9. Entire Agreement. THIS AMENDMENT AND THE DOCUMENTS
REFERRED TO HEREIN REPRESENT THE ENTIRE UNDERSTANDING OF THE PARTIES HERETO
REGARDING THE SUBJECT MATTER HEREOF AND SUPERSEDE ALL PRIOR AND CONTEMPORANEOUS
ORAL AND WRITTEN AGREEMENTS OF THE PARTIES HERETO WITH RESPECT TO THE SUBJECT
MATTER HEREOF. THERE ARE NO UNWRITTEN ORAL AGREEMENTS BETWEEN THE PARTIES.

                  Section 10. Counterparts. This Amendment may be executed in
any number of counterparts and by parties hereto on separate counterparts, each
counterpart, when so executed and delivered, constitute an original instrument,
and all such counterparts shall constitute but one and the same instrument.



             [THE REMAINDER OF THIS PAGE INTENTIONALLY LEFT BLANK]




                                      -4-
<PAGE>   5



                  IN WITNESS WHEREOF, the parties hereto have caused this
Amendment to be duly executed and authorized by their respective officers as of
the date first above written.



Parent                          THE MEN'S WEARHOUSE, INC.


                                By:   /s/ NEILL P. DAVIS
                                    ------------------------------------------
                                Name:   Neill P. Davis
                                      ----------------------------------------
                                Title:   Vice President and Treasurer
                                       ---------------------------------------


Borrower                        GOLDEN MOORES FINANCE COMPANY


                                By:   /s/ NEILL P. DAVIS
                                    ------------------------------------------
                                Name:   Neill P. Davis
                                      ----------------------------------------
                                Title:   Treasurer
                                       ---------------------------------------


Banks                           BANK OF AMERICA, N.A.
                                as a Bank and as Agent


                                By:   /s/ KIM WHITNEY
                                    ------------------------------------------
                                Name:   Kimberley A. Whitney
                                      ----------------------------------------
                                Title:   Principal
                                       ---------------------------------------


                                BANKBOSTON, N.A.


                                By:   /s/ STEPHEN J. GARVIN
                                    ------------------------------------------
                                Name:   Stephen J. Garvin
                                      ----------------------------------------
                                Title:   Director
                                       ---------------------------------------




                                      -5-
<PAGE>   6


                    UNION BANK OF CALIFORNIA, N.A.


                    By:   /s/ GAIL FLETCHER
                        ---------------------------------------------------
                    Name:   Gail L. Fletcher
                          -------------------------------------------------
                    Title:   Vice President
                           ------------------------------------------------


                    WELLS FARGO BANK, N.A.


                    By:   /s/ DONALD A. HARTMANN   /s/ CATHERINE M. WALLACE
                        ---------------------------------------------------
                    Name:   Donald A. Hartmann/Catherine M. Wallace
                          -------------------------------------------------
                    Title:   Senior Vice President/Vice President
                           ------------------------------------------------


                    CHASE BANK OF TEXAS, NATIONAL ASSOCIATION


                    By:   /s/ H. DAVID JONES
                        ---------------------------------------------------
                    Name:   H. David Jones
                          -------------------------------------------------
                    Title:   Vice President
                           ------------------------------------------------


                    FIRST UNION NATIONAL BANK


                    By:   /s/ RICHARD A. CLARKE
                        ---------------------------------------------------
                    Name:   Richard A. Clarke
                          -------------------------------------------------
                    Title:   Senior Vice President
                           ------------------------------------------------


                    BANK OF MONTREAL


                    By:   /s/ AMY K. DUMSER
                        ---------------------------------------------------
                    Name:   Amy K. Dumser
                          -------------------------------------------------
                    Title:   Director
                           ------------------------------------------------



                                      -6-