<PAGE>   1
                                                                    EXHIBIT 4.12

                               SECOND AMENDMENT TO
                              TERM CREDIT AGREEMENT

                      (C $75,000,000 Term Credit Agreement)


                  THIS SECOND AMENDMENT TO TERM CREDIT AGREEMENT dated as of
January 28, 2000 (this "Amendment"), by and among GOLDEN MOORES FINANCE COMPANY
(the "Borrower"), THE MEN'S WEARHOUSE, INC. (the "Parent"), the financial
institutions listed on the signature pages hereto (the "Banks") and BANK OF
AMERICA, N.A., formerly known as NationsBank, N.A., in its capacity as agent
(the "Agent") and in its individual capacity as a Bank hereunder.

                  WHEREAS, the Borrower, the Agent and the Banks have entered
into that certain Term Credit Agreement dated as of February 5, 1999, as amended
by that certain First Amendment to Term Credit Agreement dated as of September
14, 1999 (together with any and all amendments and modifications thereof, the
"Credit Agreement"); and

                  WHEREAS, the Credit Agreement provides that upon the
designation, formation or acquisition of any Restricted Subsidiary, the Parent
shall cause such Restricted Subsidiary to deliver to the Agent for the benefit
of the Banks a guaranty of the obligations of the Borrower in the form of a
guaranty supplement set forth as Exhibit A to the Affiliate Guaranty (a
"Guaranty Supplement") (capitalized terms used herein but not otherwise defined
herein shall have the meanings assigned to them in the Credit Agreement); and

                  WHEREAS, the Parent has formed (i) The Men's Wearhouse of
Michigan, Inc., a Delaware corporation, (ii) TMW Capital Inc., a Delaware
corporation, (iii) TMW Equity LLC, a Delaware limited liability company, (iv)
TMW Finance LP, a Delaware limited partnership, (v) TMW Realty Inc., a Delaware
corporation, (vi) TMW Texas General LLC, a Delaware limited liability company,
(vii) The Men's Wearhouse of Texas LP, a Delaware limited partnership, (viii)
TMW Merchants LLC, a Delaware limited liability company, (ix) TMW Purchasing
LLC, a Delaware limited liability company, (x) K&G Men's Center, Inc., a
Delaware corporation, (xi) K&G Men's Company Inc., a Delaware corporation, and
(xii) TMW Transition LLC, a Delaware limited liability company, as Subsidiaries
and has not designated any of them as Unrestricted Subsidiaries (collectively,
the "New Guarantors"); and

                  WHEREAS, in connection with the formation of the New
Guarantors, the Parent, the Borrower, the existing Guarantors and the New
Guarantors (collectively, the "Restructuring Parties") desire to enter into a
series of transactions among themselves, as hereinafter described, whereby
certain Investments will be made by the Restructuring Parties in the New
Guarantors, certain assets and rights of the Parent and the other Restructuring
Parties, including capital stock of and ownership interests in existing
Guarantors, will be assigned, contributed, transferred or otherwise disposed of
to the New Guarantors and existing Guarantors, certain existing Guarantors and
New Guarantors will merge into or become Subsidiaries of certain New Guarantors,
certain Restructuring Parties will incur Debt to other Restructuring Parties,
certain existing Guarantors will be dissolved and their



<PAGE>   2


respective assets will be distributed to their respective shareholders, the
Parent, who in turn may contribute such assets to one or more existing
Guarantors or New Guarantors and the Restructuring Parties will engage in other
transactions and enter into certain agreements with each other in relation to
the foregoing (collectively, the "Restructuring Transactions") and the Parent
has requested the consent of the Banks for such transactions as provided for
herein;

                  NOW, THEREFORE, in consideration of the premises, and for
other good and valuable consideration, the receipt and sufficiency of which are
hereby acknowledged, the parties hereto agree as follows:

                  Section 1. Agreements Regarding the Restructuring
Transactions.

                  (a) The Agent and the Banks hereby consent to the
Restructuring Transactions by and among the Parent, the existing Guarantors and
the New Guarantors whereby, at the completion thereof, certain assets of the
Parent shall be held by the existing Guarantors and New Guarantors in the manner
described below:

         (i) The Men's Wearhouse stores of Parent located in the State of
         Michigan and employees related thereto shall have been assigned to The
         Men's Wearhouse of Michigan, Inc.

         (ii) The membership interest of Parent in Value Priced Clothing, LLC
         shall have been assigned to The Men's Wearhouse of Michigan, Inc.

         (iii) The real property of Parent (not including any improvements
         thereon) located in Fremont, California shall have been assigned to TMW
         Capital Inc.

         (iv) The assets, operations and employees of the treasury department of
         the Parent shall have been assigned to TMW Finance LP.

         (v) The improvements on the real property of Parent located in Fremont,
         California shall have been assigned to TMW Realty Inc.

         (vi) The Men's Wearhouse stores of Parent located in the State of Texas
         and employees related thereto shall have been assigned to The Men's
         Wearhouse of Texas LP.

         (vii) 100% of the issued and outstanding capital stock of TMW Licensing
         I, Inc. shall have been assigned to TMW Marketing Company, Inc.
         (formerly known as TMW Licensing Company, Inc.)

                                       -2-

<PAGE>   3



         (viii) 100% of the issued and outstanding capital stock of TMW
         Licensing II, Inc. shall have been assigned to TMW Licensing I, Inc.

         (ix) The assets, operations and employees of the merchandising
         department of the Parent shall have been assigned to TMW Merchants LLC.

         (x) The assets, operations and employees of the inventory control
         department of the Parent shall have been assigned to TMW Purchasing
         LLC.

         (xi) The assets, operations and employees of the legal department of
         the Parent shall have been assigned to TMW Marketing Company, Inc.

                  (b) The Agent and the Banks hereby agree that this Amendment
constitutes notice to the Agent of the formation of the Subsidiaries
constituting the New Guarantors for all purposes under the Credit Agreement.

                  (c) The Agent and the Banks agree that none of the
Restructuring Transactions shall be deemed a utilization of any exception to any
covenant contained in any Loan Document, and that none of the amounts
attributable to such transactions shall cause the reduction of any such amounts
otherwise permitted under the Loan Documents, including any such exception of
any such covenant contained therein.

                  Section 2. Amendment to Credit Agreement. Effective as of
January 30, 2000, Schedule 7.17 to the Credit Agreement is hereby amended by
deleting it in its entirety and by replacing it with the Schedule 7.17 attached
hereto as Exhibit A.

                  Section 3. Conditions to Effectiveness. Except as set forth in
Section 2 above, this Amendment shall become effective as of January 28, 2000
(the "Effective Date") upon (i) execution and delivery of this Amendment by a
duly authorized officer of each of the Borrower, the Parent, the Agent and the
Majority Banks; and (ii) execution and delivery of a Guaranty Supplement by a
duly authorized officer of each of the New Guarantors.

                  Section 4. Ratification of Related Documents. The Credit
Agreement and each other Loan Document and any related document to which the
Borrower or the Parent is a party are hereby ratified and confirmed to be in
full force and effect.

                  Section 5. Limitations. The modifications set forth herein are
limited precisely as written, and shall not be deemed to (a) be a consent to, or
waiver or modification of, any other term or condition of the Credit Agreement
or any of the other Loan Documents, or (b) prejudice any right or rights which
the Banks may now have or may have in the future under or in connection with the
Credit Agreement or any of the other Loan Documents. Except as expressly
modified hereby, the terms and provisions of the Credit Agreement and the other
Loan Documents are and shall


                                       -3-
<PAGE>   4


remain in full force and effect. In the event of a conflict between this
Amendment and any of the foregoing documents, the terms of this Amendment shall
be controlling.

                  Section 6. Representations and Warranties; No Default. The
Parent hereby represents and warrants that on and as of the date hereof, and
after giving effect hereto: (i) the representations and warranties of the
Borrower made in Article 7 of the Credit Agreement shall be true and correct
(other than those representations and warranties limited by their terms to a
specific date); and (ii) no Default or Event of Default shall have occurred and
be continuing.

                  Section 7. Payment of Expenses. The Parent agrees to pay and
reimburse the Agent for all its reasonable costs and out-of-pocket expenses
incurred in connection with the preparation, execution and delivery of this
Amendment and ancillary documents, including, without limitation, the reasonable
fees and disbursements of counsel to the Agent.

                  Section 8. Choice of Law. THIS AMENDMENT AND THE RIGHTS AND
OBLIGATIONS OF THE PARTIES HEREUNDER SHALL BE CONSTRUED IN ACCORDANCE WITH AND
BE GOVERNED BY THE LAWS OF THE STATE OF NEW YORK AND THE UNITED STATES OF
AMERICA.

                  Section 9. Descriptive Headings. The descriptive headings of
the several Sections of this Amendment are inserted for convenience only and
shall not be deemed to affect the meaning or construction of any of the
provisions hereof.

                  Section 10. Entire Agreement. THIS AMENDMENT AND THE DOCUMENTS
REFERRED TO HEREIN REPRESENT THE ENTIRE UNDERSTANDING OF THE PARTIES HERETO
REGARDING THE SUBJECT MATTER HEREOF AND SUPERSEDE ALL PRIOR AND CONTEMPORANEOUS
ORAL AND WRITTEN AGREEMENTS OF THE PARTIES HERETO WITH RESPECT TO THE SUBJECT
MATTER HEREOF. THERE ARE NO UNWRITTEN ORAL AGREEMENTS BETWEEN THE PARTIES.

                  Section 11. Counterparts. This Amendment may be executed in
any number of counterparts and by parties hereto on separate counterparts, each
counterpart, when so executed and delivered, constitute an original instrument,
and all such counterparts shall constitute but one and the same instrument.



              [The remainder of this page intentionally left blank]

                                       -4-

<PAGE>   5




                  IN WITNESS WHEREOF, the parties hereto have caused this
Amendment to be duly executed and authorized by their respective officers as of
the date first above written.



Parent                                     THE MEN'S WEARHOUSE, INC.



                                           By:   /s/ NEILL P. DAVIS
                                              ---------------------------------
                                           Name:   Neill P. Davis
                                                -------------------------------
                                           Title:  Vice President and Treasurer
                                                 ------------------------------


Borrower                                   GOLDEN MOORES FINANCE COMPANY


                                           By:   /s/ NEILL P. DAVIS
                                              ---------------------------------
                                           Name:   Neill P. Davis
                                                -------------------------------
                                           Title:   Treasurer
                                                 ------------------------------


Banks                                      BANK OF AMERICA, N.A.
                                           as a Bank and as Agent


                                           By:   /s/ KIM WHITNEY
                                              ---------------------------------
                                           Name:   Kimberley A. Whitney
                                                -------------------------------
                                           Title:   Principal
                                                 ------------------------------


                                           BANKBOSTON, N.A.


                                           By:   /s/ JUDITH C.E. KELLY
                                              ---------------------------------
                                           Name:   Judith C.E. Kelly
                                                -------------------------------
                                           Title:   Vice President
                                                 ------------------------------


                                      -5-
<PAGE>   6



                                           UNION BANK OF CALIFORNIA, N.A.


                                           By:  /s/ GAIL FLETCHER
                                              ---------------------
                                           Name:   Gail L. Fletcher
                                                -------------------
                                           Title:   Vice President
                                                 ------------------


                                           WELLS FARGO BANK, N.A.


                                           By:  /s/ CAROL POLASKY
                                              ---------------------
                                           Name:   Carol Polasky
                                                -------------------
                                           Title:   Vice President
                                                 ------------------


                                           CHASE BANK OF TEXAS, NATIONAL
                                           ASSOCIATION


                                           By:  /s/ H. DAVID JONES
                                              ---------------------
                                           Name:   H. David Jones
                                                -------------------
                                           Title:   Vice President
                                                 ------------------


                                           FIRST UNION NATIONAL BANK


                                           By:  /s/ WILLIAM F. FOX
                                              ---------------------
                                           Name:   William F. Fox
                                                -------------------
                                           Title:   Vice President
                                                 ------------------


                                           BANK OF MONTREAL


                                           By:  /s/ BRUCE A. PIETKA
                                              ---------------------
                                           Name:   Bruce A. Pietka
                                                -------------------
                                           Title:   Director
                                                 ------------------




                                       -6-
<PAGE>   7
                                    EXHIBIT A

                       SCHEDULE 7.17 (CANADIAN TERM LOAN)
                                  SUBSIDIARIES

1.   Guarantors:

     The Men's Wearhouse of Michigan, Inc. (100% owned by The Men's Wearhouse,
          Inc.)
     Value Priced Clothing, LLC (100% owned by The Men's Wearhouse of Michigan,
          Inc.)
     TMW Capital Inc. (100% owned by The Men's Wearhouse, Inc.)
     TMW Equity LLC (100% owned by TMW Capital Inc.)
     TMW Finance LP (1% general partner interest owned by TMW Equity LLC) (99%
          limited partner interest owned by TMW Capital Inc.)
     TMW Realty Inc. (100% owned by The Men's Wearhouse, Inc.)
     TMW Texas General LLC (100% owned by TMW Realty Inc.)
     The Men's Wearhouse of Texas LP (1% general partner interest owned by
          TMW Texas General LLC) (99% limited partner interest owned by TMW
          Realty Inc.)
     Renwick Technologies, Inc. (100% owned by The Men's Wearhouse, Inc.)
     TMW Marketing Company, Inc.(formerly TMW Licensing Company, Inc.) (100%
         owned by The Men's Wearhouse, Inc.)
     TMW Licensing I, Inc. (100% owned by TMW Marketing Company, Inc.)
     TMW Licensing II, Inc.(100% owned by TMW Licensing I, Inc.)
     TMW Merchants LLC (100% owned by TMW Marketing Company, Inc.)
     TMW Purchasing LLC (100% owned by TMW Merchants LLC)
     Golden Moores Company (100% owned by The Men's Wearhouse, Inc.)
     K&G Men's Center, Inc. (100% owned by The Men's Wearhouse, Inc.)
     K&G Men's Company Inc. (100% owned by K & G Men's Center, Inc.)
     T&C Liquidators, Inc. (100% owned by K&G Men's Center, Inc.)
     K&G of Indiana, Inc. (100% owned by K&G Men's Center, Inc.)
     K&G Associates of New Jersey, Inc. (100% owned by K&G Men's Center, Inc.)

2.   Non-Guarantors

     Golden Moores Finance Company (100% owned by The Men's Wearhouse, Inc.)
          ("Borrower")
     Moores Retail Group Inc. (100% of the voting Common Shares owned by Golden
          Moores Company, while 100% of the nonvoting Exchangeable Shares are
          owned by the shareholders and optionholders of Moores as they existed
          prior to the Effective Date)
     Moores The Suit People Inc. (100% owned by Moores Retail Group Inc.)
     Golden Brand Clothing (Canada) Ltd. (100% owned by Moores Retail Group
          Inc.)
     Moores The Suit People U.S., Inc. (100% owned by Moores Retail Group Inc.)
     Gares Cigars, LLC (at least 70% owned by K & G Men's Center, Inc.)
     K&G of Ohio, Inc. (100% owned by K&G Men's Center, Inc.)

3.   Unrestricted Subsidiaries
     None

                                       -7-