<PAGE>

                                                                  EXHIBIT 10.16B

                   SECOND AMENDMENT TO EMPLOYMENT AGREEMENT
                   ----------------------------------------

  THIS SECOND AMENDMENT (this "Amendment"), made this 25th day of May, 2001, is
made to that certain Employment Agreement by and between ROBERT N. WILDRICK
("Executive") and JOS. A. BANK CLOTHIERS, INC. ("Employer"), dated as of
November 1, 1999, as amended by that certain First Amendment to Employment
Agreement, dated March 6, 2000 (as so amended, the "Agreement").

  WHEREAS, Executive and Employer are the sole parties to the Employment
Agreement; and

  WHEREAS, Executive and Employer have agreed to amend the Employment Agreement,

  NOW THEREFORE, for good and valuable consideration, the receipt and adequacy
of which are hereby acknowledged, Executive and Employer hereby amend the
Employment Agreement and agree as follows:

     1. Section 2 of the Agreement is hereby amended by deleting from the third
line thereof the date "February 2, 2002" and inserting in lieu thereof the date
"January 29, 2005".

     2. Section 3.2 of the Agreement is hereby deleted in its entirety and the
following is hereby inserted in lieu thereof:

     3.2  Location of Executive Offices. The Company will maintain its principal
          -----------------------------
     executive offices at a location in the Baltimore, Maryland metropolitan
     area and additional executive offices at a location in the Palm Beach,
     Florida metropolitan area. Executive may conduct the affairs of the Company
     from the Palm Beach office, subject to spending such amounts of his time as
     shall be reasonably necessary in the Baltimore Office. Executive shall not
     be required to perform services for the Company at a location other than
     the Baltimore office and the Palm Beach office, except for services
     rendered in connection with reasonably required travel on the Company's
     business.

     3. Section 4.1 of the Agreement is hereby deleted in its entirety and the
following is hereby inserted in lieu thereof:

     4.1  Base Salary. Employer shall pay to Executive during the Employment
          -----------
     Period an annual base salary (the "Base Salary") in accordance with this
     Section 4.1. Subject to increases as set forth in the immediately following
     sentence, the Base Salary for the period from the Start Date through
     February 2, 2002 shall be $450,000 and the Base Salary for the period from
     February 3, 2002 through January 29, 2005 shall be $750,000. Beginning in
     Fiscal 2001, the Board shall increase the Base Salary at least once each
     fiscal year on the date on which general salary increases within the
     Company for such year take effect (the "Annual Increase Date"). The annual
     increase shall be in an amount not less than the percentage increase in the
     consumer price index over the most recently reported 12-month period.  In
     the event the Board fails to so increase the Base Salary in any fiscal year
     by the Annual Increase Date, the Base Salary shall automatically be
     increased on such date by an amount equal to the percentage increase in the
     consumer price index over the most recently reported 12-month period. The
     Base Salary for each calendar year shall be payable in installments in
     accordance with the Company's policy on payment of executives in effect
     from time to time.
<PAGE>

     4. Effective for Fiscal 2002 (and for each fiscal year thereafter during
the Employment Period), Section 4.2.A of the Agreement is hereby amended by:

     a. deleting from the third line thereof the number "100%" and inserting in
lieu thereof the number "125%";

     b. deleting from the fifth line thereof the word "Committee" and inserting
in lieu thereof the phrase "Compensation Committee of the Board (the
`Committee')"; and

     c. adding at the end thereof the following: "The Company and Executive
acknowledge that the Company has in the past established additional bonus
programs wherein participants' bonus rates are more or less than the otherwise
applicable maximum bonus rate for such participants. For example, in Fiscal 2000
(during which Executive's bonus rate was 100% of Base Salary), the Company
established three bonus programs- the "max" bonus program, wherein Executive's
bonus rate was 50% of Base Salary; the "supermax" bonus program, wherein
Executive's bonus rate was 100% of Base Salary; and the "turnaround" bonus
program, wherein Executive's bonus rate was 200% of Base Salary. Without
limiting the Executive's right to receive a Bonus as set forth in the first
sentence of this Section 4.2.A, if and to the extent the Company shall establish
additional bonus programs in the future, without obligating the Company to do
so, Executive's bonus rate in such programs shall be proportionately increased
or decreased based upon Executive's otherwise applicable bonus rate of 125%. For
example, if Executive's contractual bonus rate in Fiscal 2000 had been 125%, his
bonus rates for the three bonus programs then in effect would have been: 62.5%
of Base Salary for the "max" program; 125% of Base Salary for the "supermax"
program; and 250% of Base Salary for the "turnaround" program.

     5. Section 4.3 of the Agreement is hereby deleted in its entirety and the
following is hereby inserted in lieu thereof:

     4.3  Car and Club Allowance. Employer shall pay to Executive throughout the
          ----------------------
     Employment Period a car and club allowance equal to $1,750 per month, which
     shall be in lieu of any expense reimbursement related to a car (purchased
     or leased) or a club.

     6. Section 4.7 of the Agreement is hereby amended by inserting at the end
thereof the following:

          Employer shall propose to the shareholders of the Company the creation
     of an additional option plan for at least 350,000 shares of the Company's
     common stock (the "Stock"). If the shareholders do not approve such
     proposal, Employee shall be entitled to terminate this Agreement as of
     February 2, 2002 by notice to the Company not later than August 3, 2001 and
     the Company shall pay to Mr. Wildrick severance in the amount of $900,000
     pursuant to Section 6.4 hereof. Provided the shareholders approve such
     proposal, the Company will grant to Mr. Wildrick as of the date of
     shareholder approval an option to purchase 350,000 shares of the Stock as
     hereinafter set forth. The grant shall consist of a first tranche of
     150,000 options, a second tranche of 100,000 options and a third tranche of
     100,000 options (respectively, "Tranche One", "Tranche Two" and "Tranche
     Three"). The Tranche One options shall vest immediately upon issuance and
     shall be exercisable at a price equal to the closing price of the Stock on
     the date of grant. The Tranche Two options shall vest on February 2, 2003
     and shall be exercisable at a price equal to the greater of the exercise
     price of the Tranche One options or $8.00 per share. The Tranche Three
     options shall vest on February 1, 2004 and shall be exercisable at a price
     equal to the greater of the exercise price of the Tranche One options or
     $9.00 per share. Options shall be exercisable not earlier than the date of
     vesting. If the parties fail to agree upon a renewal or extension of the
     Employment Period and the Employment

                                       2
<PAGE>

     Period expires on its then stated expiration date, vested options may be
     not be exercised later than three months after the expiration of the
     Employment Period. Subject to the foregoing provisions, the agreement
     pursuant to which the options shall be granted shall be generally in the
     form of option agreement attached as an exhibit hereto.

     7. Section 5.3 of the Agreement is hereby amended by deleting therefrom
clause (z) and inserting in lieu thereof the following "(z) any transfer of the
Company's principal or additional executive offices outside the respective
geographic areas described in Section 3.2 of this Amendment or requirement that
Executive principally perform his duties outside of the Palm Beach, Florida
metropolitan area."

     8. Section 6.1 of the Agreement is hereby deleted in its entirety and the
following is hereby inserted in lieu thereof:

          6.1  Termination Without Cause by Employer or for Good Reason or
               -----------------------------------------------------------
     Change of Control by Executive. If the Employment Period is terminated by
     ------------------------------
     Executive at any time pursuant to the provisions of Section 5.4 hereof,
     Employer will pay to Executive on the last day of the Employment Period the
     sum of (a) $1,800,000, plus (b) if applicable, the Bonus for the last full
     Bonus Year pursuant to Section 4.2. If the Employment Period is terminated
     by Employer pursuant to the provisions of Section 5.1 hereof or by
     Executive pursuant to the provisions of Section 5.3 hereof, Employer will
     pay to Executive on the last day of the Employment Period the sum of (y)
     $1,500,000, plus (z) if applicable, the Bonus for the last full Bonus Year
     pursuant to Section 4.2. Employer shall have no obligation to continue any
     other benefits provided for in Section 4 past the date of termination.

     9. Section 6.3 of the Agreement is hereby deleted in its entirety and the
following is hereby inserted in lieu thereof:

          6.3  Termination for Cause Following a Change of Control. In the event
               ---------------------------------------------------
     Employer terminates the Employment Period for cause pursuant to Section 5.2
     at any time within 90 days following a change in control of the Company,
     Employer shall pay to Executive on the last day of the Employment Period
     the sum of (a) $1,800,000, plus (b) if applicable, the Bonus for the last
     full Bonus Year pursuant to Section 4.2. Employer shall have no obligation
     to continue any other benefits provided for in Section 4 past the date of
     termination.

     10. Section 6.4 of the Agreement is hereby amended by deleting therefrom
the phrase "his then annual Base Salary plus his maximum Bonus for the year in
which the termination occurs and" on the third and fourth lines of page 8 of the
Agreement and inserting in lieu thereof the phrase "(a) $900,000 and (b)".


     Except as specifically amended hereby, the Employment Agreement shall
remain in full force and effect according to its terms. To the extent of any
conflict between the terms of this Amendment and the terms of the remainder of
the Employment Agreement, the terms of this Amendment shall control and prevail.
Capitalized terms used but not defined herein shall have those respective
meanings attributed to them in the Employment Agreement. The word "Company" as
used herein and in the Agreement shall mean and refer to "Jos. A. Bank
Clothiers, Inc." and is used herein and in the

                                       3
<PAGE>

Agreement interchangeably with the word "Employer". This Amendment shall
hereafter be deemed a part of the Employment Agreement for all purposes.

     IN WITNESS WHEREOF, the parties hereto have executed this Amendment as of
the date first above written.

JOS. A. BANK CLOTHIERS, INC.

By: /s/: Andrew A. Giordano              /s/: Robert N. Wildrick
    ------------------------             -----------------------
    Andrew A. Giordano,                  ROBERT N. WILDRICK
    Chairman of the Board

                                       4