Exhibit 10.1

EMPLOYMENT AGREEMENT

This Employment Agreement (“Agreement”) is made as of January 30, 2006 between
CASUAL MALE RETAIL GROUP, INC., a Delaware corporation with an office at 555
Turnpike Street, Canton, Massachusetts, 02021 (the “Company”), and Wayne Diller
(the “Executive”) having an address at 233 Country Club Way, Kingston MA 02364.

WITNESSETH:

WHEREAS, the Company desires that Executive serve as Senior Vice President,
Divisional Merchandise Manager & Visual Merchandising and Executive desires to
be so employed by the Company.

WHEREAS, Executive and the Company desire to set forth in writing the terms and
conditions of the Executive’s employment with the Company from the date hereof.

NOW, THEREFORE, in consideration of the promises and the mutual promises,
representations and covenants herein contained, the parties hereto agree as
follows:

1. EMPLOYMENT

The Company hereby employs Executive and Executive hereby accepts such
employment, subject to the terms and conditions herein set forth. Executive
shall hold the office of Senior Vice President, Divisional Merchandise Manager &
Visual Merchandising.

2. TERM

The term of employment under this Agreement shall begin on January 30, 2006 (the
“Employment Date”) and shall continue for a period of two (2) years from that
date (the “Term”), subject to prior termination in accordance with the terms
hereof.

3. COMPENSATION

(a) As compensation for the employment services to be rendered by Executive
hereunder, the Company agrees to pay to Executive, and Executive agrees to
accept, payable in equal bi-weekly installments in accordance with Company
practice, an annual base salary of two hundred seventy thousand dollars
($270,000).

(b) In addition to the annual base salary, Executive is eligible to participate
in the Company’s Annual Incentive Plan. Such incentive shall be determined and
payable in accordance with the Company’s incentive program in effect at the
time, subject to change from year to year in the Company’s sole discretion.
Executive will participate in the Company’s incentive program at a rate of 28%
at target (35% max) (FY 2006) of Executive’s actual annual base earnings. The
actual award under the incentive program, if any, may be more or less than the
target and will be based on Executive’s performance and the performance of the
Company and payment will be made in accordance with the terms of the incentive
program then in effect.

--------------------------------------------------------------------------------

(c) In addition, Executive is eligible to participate in the Company’s Long Term
Incentive Plan (LTIP). Such incentive shall be determined and distributable in
accordance with the terms and conditions as described in the Long Term Incentive
Plan (LTIP) documents in effect at the time of the award, subject to change from
year to year in the Company’s sole discretion. Executive will participate in the
Company’s Long Term Incentive Plan at an incentive rate of 70%, at target, of
Executive’s combined actual annual base salary, for the incentive period, as
defined in the Long Term Incentive Plan documents in effect at the time of the
award. However, given the three year term of the Long Term Incentive Plan and
the eighteen (18) month vesting period for distribution, Executive would have to
be employed with the Company beyond the term of this Agreement to receive
benefits under the Plan.

(d) In addition, Executive will receive a twenty five thousand dollar ($25,000)
Stay On Incentive, payable in a single lump sum (subject to all federal and
state taxes and withholding). Payment will be made on the first regularly
scheduled payroll following the execution of this Employment Agreement. In the
event Executive’s notice of resignation or effective date of resignation occurs
within the first twelve (12) months of the effective date of this agreement of
employment (on or prior to January 30, 2007), Executive agrees to repay Company
the full twenty five thousand dollar ($25,000) Stay On Incentive and agrees to
enter into a reasonable repayment agreement with the Company in the event of
such resignation, with repayment to be made within twelve (12) months of
resignation.

(e) In addition, the Company shall grant to the Executive twenty-five thousand
(25,000) Incentive Stock Options under the Company’s 1992 Stock Incentive Plan,
which are exercisable at the purchase price per share equal to the opening price
of the Common Stock on the Agreement Date (the “Grant Date”). The options will
vest pro-rata over a three (3) year period commencing on the first anniversary
of the Grant Date, with one third of the total vesting and becoming exercisable
on each of the first, second and third anniversaries of the Grant date, subject
to the terms of the Stock Option Agreement, which the Executive must execute in
connection with the grant of the options.

4. EXPENSES

The Company shall pay or reimburse Executive, in accordance with the Company’s
policies and procedures and upon presentment of suitable vouchers, for all
reasonable business and travel expenses, which may be incurred or paid by
Executive in connection with his employment hereunder. Executive shall comply
with such restrictions and shall keep such records as the Company may reasonably
deem necessary to meet the requirements of the Internal Revenue Code of 1986, as
amended from time to time, and regulations promulgated thereunder.

5. OTHER BENEFITS

(a) Executive shall be entitled to such vacations and to participate in and
receive any other benefits customarily provided by the Company to its management
(including any profit sharing, pension, 401 (k), short and long-term disability
insurance, medical and dental insurance and group life insurance plans in
accordance with the terms of such plans), all as determined from time to time by
the Compensation Committee of the Board of Directors in its discretion.

 

2

--------------------------------------------------------------------------------

(b) The Company will, during the term of Executive’s employment hereunder,
provide Executive with an automobile allowance in the total amount of seven
thousand two hundred dollars ($7,200.00) annually, in equal bi-weekly payments
in accordance with the company’s normal payroll practices. Executive shall pay
and be responsible for all insurance, repairs and maintenance costs associated
with operating the automobile. Executive is responsible for his gasoline, unless
the gasoline expense is reimbursable under the Company’s policies and
procedures.

(c) Executive will be eligible to participate in the Company’s annual
performance appraisal process.

6. DUTIES

(a) Executive shall perform such duties and functions consistent with his
position as Senior Vice President, Divisional Merchandise Manager & Visual
Merchandising and/or as the Chief Executive Officer of the Company shall from
time to time determine and Executive shall comply in the performance of his
duties with the policies of, and be subject to the direction of, the above named
officer of the company.

(b) During the Term of this Agreement, Executive shall devote substantially all
of his time and attention, vacation time and absences for sickness excepted, to
the business of the Company, as necessary to fulfill his duties. Executive shall
perform the duties assigned to him with fidelity and to the best of his ability.
Notwithstanding anything herein to the contrary, and subject to the foregoing,
Executive shall not be prevented from accepting positions in outside charitable
organizations so long as such activities do not interfere with Executive’s
performance of his duties hereunder and do not violate Section 10 hereof.

(c) The principal location at which the Executive shall perform his duties
hereunder shall be at the Company’s offices in Canton, Massachusetts or at such
other location as may be temporarily designated from time to time by the Chief
Executive Officer. Notwithstanding the foregoing, Executive shall perform such
services at such other locations as may be required for the proper performance
of his duties hereunder, and Executive recognizes that such duties may involve
travel.

7. TERMINATION OF EMPLOYMENT; EFFECT OF TERMINATION

(a) Executive’s employment hereunder may be terminated by the Company at any
time:

(i) upon the determination that Executive’s performance of his duties has not
been fully satisfactory for any reason which would not constitute justifiable
cause (as hereinafter defined) or for other business reasons necessitating
termination which do not constitute justifiable cause, in either case upon
thirty (30) days’ prior written notice to Executive; or

 

3

--------------------------------------------------------------------------------

(ii) upon the determination that there is justifiable cause (as hereinafter
defined) for such termination.

(b) Executive’s employment shall terminate upon:

(i) the death of Executive;

(ii) the “total disability” of Executive (as hereinafter defined in Subsection
(c) herein) pursuant to Subsection (h) hereof; or

(iii) Executive’s resignation of employment.

(c) For the purposes of this Agreement, the term “total disability” shall mean
Executive is physically or mentally incapacitated so as to render Executive
incapable of performing the essentials of Executive’s job, even with reasonable
accommodation, as reasonably determined by the Company, which determination
shall be final and binding.

(d) For the purposes hereof, the term “justifiable cause” shall mean: any
failure or refusal to perform any of the duties pursuant to this Agreement or
any breach of this Agreement by the Executive; Executive’s breach of any
material written policies, rules or regulations which have been adopted by the
Company; Executive’s repeated failure to perform his duties in a satisfactory
manner; Executive’s performance of any act or his failure to act, as to which if
Executive were prosecuted and convicted, a crime or offense involving money or
property of the Company or its subsidiaries or affiliates, or a crime or offense
constituting a felony in the jurisdiction involved, would have occurred; any
unauthorized disclosure by Executive to any person, firm or corporation of any
confidential information or trade secret of the Company or any of its
subsidiaries or affiliates; any attempt by Executive to secure any personal
profit in connection with the business of the Company or any of its subsidiaries
and affiliates; or the engaging by Executive in any business other than the
business of the Company and its subsidiaries and affiliates which interferes
with the performance of his duties hereunder. Upon termination of Executive’s
employment for justifiable cause, this Agreement shall terminate immediately and
Executive shall not be entitled to any amounts or benefits hereunder other than
such portion of Executive’s annual base salary and reimbursement of expenses
pursuant to Section 5 hereof as have been accrued through the date of his
termination of employment.

(e) If the Company terminates this Agreement without “justifiable cause” as
provided in Subsection 7 (a)(i) the Company shall pay Executive the base salary
for the remainder of the Term not to exceed an amount equal to one half of
Executive’s annual base salary, payable in equal payments in accordance with the
Company’s customary payroll practices. However, if Executive is employed or
retained, as an employee, independent contractor, consultant or in any other
capacity or if he is offered another position by the Company at a comparable
salary (“New Employment”) prior to or during the time he receives payment under
this Subsection or Subsection 3 (b), the Company is entitled to a credit for all
sums paid or earned by Executive during this period of time or which he

 

4

--------------------------------------------------------------------------------

could have earned had he accepted the comparable position by the Company. The
Executive must make a good faith effort to find New Employment and mitigate the
amount of money to be paid by the Company to Executive under this Subsection or
Subsection 3(b). Executive also agrees to immediately notify the Vice President
of Human Resources of the Company at 555 Turnpike Street, Canton, Massachusetts,
02021, if and when he is offered another position and/or accepts another
position. The Company will pay any amount due and owing in accordance with the
payment schedule in 3(a), until paid in full. Any payment pursuant to this
paragraph 7(e) is contingent upon Executive’s execution of a general release and
separation agreement in a form acceptable to the Company and will be in lieu of
payments to which Executive might have been entitled under any other severance
plan of the Company.

(f) If Executive shall die during the term of his employment hereunder, this
Agreement shall terminate immediately. In such event, the estate of Executive
shall thereupon be entitled to receive such portion of Executive’s base annual
salary and reimbursement of expenses pursuant to Section 5 as have been accrued
through the date of his death.

(g) Upon Executive’s “total disability”, the Company shall have the right to
terminate Executive’s employment. Any termination pursuant to this Subsection
(f) shall be effective on the earlier of (i) the date 30 days after which
Executive shall have received written notice of the Company’s election to
terminate or (ii) the date he begins to receive long-term disability insurance
benefits under the policy provided by the Company pursuant to Section 6 hereof.

(h) Upon the resignation of Executive in any capacity, that resignation will be
deemed to be a resignation from all offices and positions that Executive holds
with respect to the Company and any of its subsidiaries and affiliates. In the
event of Executive’s resignation, he shall be entitled only to receive such
portion of his annual base salary and reimbursement of expenses pursuant to
Section 5 as have been accrued through the date of his resignation.

(i) Change of Control. In the event the Executive’s employment with the Company
is terminated by the Company during the Term as a result of a Change of Control
of the Company occurring during the Term then, in such event, the Company shall
pay Executive an amount equal to twelve (12) months of base annual salary in
effect at the time of the termination, which amount will be subject to
mitigation in accordance with Section 7(e) above. For the purposes of the
foregoing, Change of Control shall have the meaning set forth in the Designs,
Inc. 1992 Stock Incentive Plan (without regard to any subsequent amendments
thereto). Any payment pursuant to this paragraph 7(i) is contingent upon
Executive’s execution of a general release and separation agreement in a form
acceptable to the Company and will be in lieu of payments to which Executive
might have been entitled under paragraph 7(e) of this Agreement or under any
other severance plan of the Company

8. REPRESENTATION AND AGREEMENTS OF EXECUTIVE

(a) Executive represents and warrants that he is free to enter into this
Agreement and to perform the duties required hereunder, and that there are no
employment contracts or understandings, restrictive covenants or other
restrictions, whether written or oral, preventing the performance of his duties
hereunder.

 

5

--------------------------------------------------------------------------------

(b) Executive agrees to submit to a medical examination and to cooperate and
supply such other information and documents as may be required by any insurance
company in connection with the Company’s obtaining life insurance on the life of
Executive, and any other type of insurance or fringe benefit as the Company
shall determine from time to time to obtain.

(c) Executive represents and warrants that he has never been convicted of a
felony and he has not been convicted or incarcerated for a misdemeanor within
the past five years, other than a first conviction for drunkenness, simple
assault, speeding, minor traffic violations, affray, or disturbance of the
peace.

(d) Executive represents and warrants that he has never been a party to any
judicial or administrative proceeding that resulted in a judgement, decree, or
final order (i) enjoining him from future violations of, or prohibiting any
violations of any federal or state securities law, or (ii) finding any
violations of any federal or state securities law.

(e) Executive represents and warrants that he has never been accused of any
impropriety in connection with any employment;

Any breach of any of the above representations and warranties is “justifiable
cause” for termination under Section 7(d) of this Agreement.

9. NON-COMPETITION

(a) Executive agrees that during his employment by the Company and during the
two (2) year period following the termination of Executive’s employment
hereunder (the “Non-Competitive Period”), Executive shall not, directly or
indirectly, as owner, partner, joint venturer, stockholder, employee, broker,
agent, principal, trustee, corporate officer, director, licensor, or in any
capacity whatsoever, engage in, become financially interested in, be employed
by, render any consultation or business advice with respect to, accept any
competitive business on behalf of, or have any connection with any business
which is competitive with products or services of the Company or any
subsidiaries and affiliates, in any geographic area in the United States of
America and Puerto Rico where, at the time of the termination of his employment
hereunder, the business of the Company or any of such subsidiaries and
affiliates was being conducted or was proposed to be conducted in any manner
whatsoever; provided, however, that Executive may own any securities of any
corporation which is engaged in such business and is publicly owned and traded
but in an amount not to exceed at any one time one percent (1%) of any class of
stock or securities of such corporation. In addition, Executive shall not,
during the Non-Competitive Period, directly or indirectly, request or cause any
suppliers or customers with whom the Company or any of its subsidiaries and
affiliates has a business relationship to cancel or terminate any such business
relationship with the Company or any of its subsidiaries and affiliates or
otherwise compromise the Company’s good will or solicit, hire, interfere with or
entice from the Company any employee (or former employee) of the Company.

 

6

--------------------------------------------------------------------------------

(b) If any portion of the restrictions set forth in this Section 10 should, for
any reason whatsoever, be declared invalid by a court of competent jurisdiction,
the validity or enforceability of the remainder of such restrictions shall not
thereby be adversely affected. For the purposes of this Section 10, a business
competitive with the products and services of the Company (or such subsidiaries
and affiliates) shall include, a specialty retailer which primarily distributes,
sells or markets so-called “big and tall” apparel of any kind for men or which
utilizes the “big and tall” retail or wholesale marketing concept as part of its
business.

(c) Executive acknowledges that the Company conducts business throughout the
United States and Puerto Rico, that Executive’s duties and responsibilities on
behalf of the Company are nationwide (including Puerto Rico) in scope, that its
sales and marketing prospects are for continued expansion throughout the United
States and therefore, the territorial and time limitations set forth in this
Section 10 are reasonable and properly required for the adequate protection of
the business of the Company and its subsidiaries and affiliates. In the event
any such territorial or time limitation is deemed to be unreasonable by a court
of competent jurisdiction, Executive agrees to the reduction of the territorial
or time limitation to the area or period which such court shall deem reasonable.

(d) The existence of any non-material claim or cause of action (a “non-material”
claim or cause of action is defined as a claim or cause of action which results
from something other than a material breach of the terms and provisions of this
Agreement by the Company) by Executive against the Company or any subsidiary or
affiliate shall not constitute a defense to the enforcement by the Company or
any subsidiary or affiliate of the foregoing restrictive covenants, but such
claim or cause of action shall be litigated separately.

10. INVENTIONS AND DISCOVERIES

(a) Upon execution of this Agreement and thereafter, Executive shall promptly
and fully disclose to the Company, and with all necessary detail for a complete
understanding of the same, all existing and future developments, know-how,
discoveries, inventions, improvements, concepts, ideas, writings, formulae,
processes and Methods (whether copyrightable, patentable or otherwise) made,
received, conceived, acquired or written during working hours, or otherwise, by
Executive (whether or not at the request or upon the suggestion of the Company)
during the period of his employment with, or rendering of advisory or consulting
services to, the Company or any of its subsidiaries and affiliates, solely or
jointly with others, in or relating to any activities of the Company or its
subsidiaries and affiliates known to him as a consequence of his employment or
the rendering of advisory and consulting services hereunder (collectively the
“Subject Matter”).

(b) Executive hereby assigns and transfers, and agrees to assign and transfer,
to the Company, all his rights, title and interest in and to the Subject Matter,
and Executive further agrees to deliver to the Company any and all drawings,
notes, specifications and data relating to the Subject Matter, and to execute,
acknowledge and deliver all such further papers, including applications for
copyrights or patents, as may be necessary to obtain copyrights and patents for
any thereof in any and all countries and to vest title thereto to the Company.
Executive shall assist the Company in obtaining such copyrights or patents
during the term of this Agreement, and at any

 

7

--------------------------------------------------------------------------------

time thereafter on reasonable notice and at mutually convenient times, and
Executive agrees to testify in any prosecution or litigation involving any of
the Subject Matter; provided, however, that Executive shall be compensated in a
timely manner at the rate of $250 per day (or portion thereof), plus
out-of-pocket expenses incurred in rendering such assistance or giving or
preparing to give such testimony if it is required after the termination of this
Agreement.

11. NON-DISCLOSURE OF CONFIDENTIAL INFORMATION

(a) Executive acknowledges that the Company possesses certain confidential and
propriety information that has been or may be revealed to him or learned by
Executive during the course of Executive’s employment with the Company and that
it would be unfair to use that information or knowledge to compete with or to
otherwise disadvantage the Company. Executive shall not, during the term of this
Agreement or at any time following termination of this Agreement, directly or
indirectly, disclose or permit to be known (other than as is required in the
regular course of his duties (including without limitation disclosures to the
Company’s advisors and consultants), as required by law (in which case Executive
shall give the Company prior written notice of such required disclosure) or with
the prior written consent of the Board of Directors of the Company), to any
person, firm, corporation, or other entity, any confidential information
acquired by him during the course of, or as an incident to, his employment or
the rendering of his advisory or consulting services hereunder, relating to the
Company or any of its subsidiaries and affiliates, the directors of the Company
or its subsidiaries and affiliates, any supplier or customer of the Company or
any of their subsidiaries and affiliates, or any corporation, partnership or
other entity owned or controlled, directly or indirectly, by any of the
foregoing, or in which any of the foregoing has a beneficial interest,
including, but not limited to, the business affairs of each of the foregoing.
Such confidential information shall include, but shall not be limited to,
proprietary technology, trade secrets, patented processes, research and
development data, know-how, market studies and forecasts, financial data,
competitive analyses, pricing policies, employee lists, personnel policies, the
substance of agreements with customers, suppliers and others, marketing or
dealership arrangements, servicing and training programs and arrangements,
supplier lists, customer lists and any other documents embodying such
confidential information. This confidentiality obligation shall not apply to any
confidential information, which is or becomes publicly available other than
pursuant to a breach of this Section 12(a) by Executive.

(b) All information and documents relating to the Company and its affiliates as
herein above described (or other business affairs) shall be the exclusive
property of the Company, and Executive shall use commercially reasonable best
efforts to prevent any publication or disclosure thereof. Upon termination of
Executive’s employment with the Company, all documents, records, reports,
writings and other similar documents containing confidential information,
including copies thereof then in Executive’s possession or control shall be
returned and left with the Company.

12. SPECIFIC PERFORMANCE

Executive agrees that if he breaches, or threatens to commit a breach of, any of
the provisions of Sections 9, 10 or 11 (the

 

8

--------------------------------------------------------------------------------

“Restrictive Covenants”), the Company shall have, in addition to, and not in
lieu of, any other rights and remedies available to the Company under law and in
equity, the right to have the Restrictive Covenants specifically enforced by a
court of competent jurisdiction, it being agreed that any breach or threatened
breach of the Restrictive Covenants would cause irreparable injury to the
Company and that money damages would not provide an adequate remedy to the
Company. Notwithstanding the foregoing, nothing herein shall constitute a waiver
by Executive of his right to contest whether a breach or threatened breach of
any Restrictive Covenant has occurred. The Company shall be entitled to recover
from Executive all attorneys’ fees and expenses reasonably incurred in
establishing a breach of this agreement. The damages, attorneys’ fees and costs
shall be in addition to and not in lieu of any preliminary injunctive relief
that may be available to the Company.

13. AMENDMENT OR ALTERATION

No amendment or alteration of the terms of this Agreement shall be valid unless
made in writing and signed by both of the parties hereto.

14. GOVERNING LAW

This Agreement shall be governed by, and construed and enforced in accordance
with the substantive laws of The Commonwealth of Massachusetts, without regard
to its principles of conflicts of laws.

15. SEVERABILITY

The holding of any provision of this Agreement to be invalid or unenforceable by
a court of competent jurisdiction shall not affect any other provision of this
Agreement, which shall remain in full force and effect.

16. NOTICES

Any notices required or permitted to be given hereunder shall be sufficient if
in writing, and if delivered by hand or courier, or sent by certified mail,
return receipt requested, to the addresses set forth above or such other address
as either party may from time to time designate in writing to the other, and
shall be deemed given as of the date of the delivery or at the expiration of
three days in the event of a mailing.

17. WAIVER OR BREACH

It is agreed that a waiver by either party or a breach of any provision of this
Agreement shall not operate, or be construed as a waiver of any subsequent
breach by that same party.

18. ENTIRE AGREEMENT AND BINDING EFFECT

This Agreement contains the entire agreement of the parties with respect to the
subject matter hereof and shall be binding upon

 

9

--------------------------------------------------------------------------------

and inure to the benefit of the parties hereto and their respective legal
representatives, heirs, distributors, successors and assigns and supersedes any
and all prior agreements between the parties whether oral or written including
the Letter Agreement dated August 26, 2004. This Agreement may not be modified
except upon further written agreement executed by both parties. Executive agrees
that the Company may in its sole discretion, during the term of Executive’s
employment with the Company and thereafter, provide copies of this Agreement (or
excerpts of the Agreement) to others, including businesses or entities that may
employ, do business with, or consider employing Executive in the future.
Executive further agrees that any subsequent change or changes in his duties,
compensation or areas of responsibility shall in no way affect the validity of
this Agreement or otherwise render inapplicable any of the provisions of
paragraphs 9 through 12 of this Agreement, which shall remain in full force and
effect except as may be modified by a subsequent written agreement.

19. SURVIVAL.

Except as otherwise expressly provided herein, the termination of Executive’s
employment hereunder or the expiration of this Agreement shall not affect the
enforceability of Sections 5, 8, 9, 10, 11 and 12 hereof.

20. RESOLUTION OF DISPUTES

Any and all disputes arising under or in connection with this Agreement shall be
resolved in accordance with this Section 20.

The parties shall attempt to resolve any dispute, controversy or difference that
may arise between them through good faith negotiations. In the event the parties
fail to reach resolution of any such dispute within thirty (30) days after
entering into negotiations, either party may proceed to institute action in any
state or federal court located within the Commonwealth of Massachusetts and each
party consents to the personal jurisdiction of any such state or federal court.

21. FURTHER ASSURANCES

The parties agree to execute and deliver all such further documents, agreements
and instruments and take such other and further action as may be necessary or
appropriate to carry out the purposes and intent of this Agreement.

22. HEADINGS

The Section headings appearing in this Agreement are for the purposes of easy
reference and shall not be considered a part of this Agreement or in any way
modify, amend or affect its provisions.

23. COUNTERPARTS

This Agreement may be executed in one or more counterparts, each of which shall
be deemed an original and all of which together shall constitute one and the
same agreement.

 

10

--------------------------------------------------------------------------------

IN WITNESS WHEREOF, the parties hereto have executed this Agreement, under seal,
as of the date and year first above written.

 

CASUAL MALE RETAIL GROUP, INC.         By:  

/s/ David A. Levin

    Date:                      Name:   David A. Levin     Its:   President,
Chief Executive Officer     By:  

/s/ Dennis R. Hernreich

    Date:                      Name:   Dennis R. Hernreich     Its:   Executive
VP, COO, CFO    

/s/ Wayne P. Diller

    Date: 6/29/2006     Wayne P. Diller    

 

11