<PAGE>   1
EXHIBIT 10.6

                              HANOVER DIRECT, INC.
                        DIRECTORS CHANGE OF CONTROL PLAN


                                    SECTION 1

                            ESTABLISHMENT OF THE PLAN

        Hanover Direct, Inc. ("Hanover") has established this Hanover Direct,
Inc. Directors Change of Control Plan (the "Plan") effective as
of May 3, 2001 (the "Effective Date").

                                    SECTION 2

                                   DEFINITIONS

        2.1     "Board of Directors" means the Board of Directors of Hanover.

        2.2     "Change of Control" means the first to occur of any of the
events described in subsections (i) through (iii) below, following the Effective
Date:

                (i)     When any Person becomes, through an acquisition, the
beneficial owner of shares of Hanover having at least fifty percent (50%) of the
total number of votes that may be cast for the election of directors of Hanover
(the "Voting Shares"); provided, however, that the following acquisitions shall
not constitute a Change of Control:

                        (a)     if a Person owns less than fifty percent (50%)
of the voting power of Hanover and that Person's ownership increases above fifty
percent (50%) solely by virtue of an acquisition of stock by Hanover, then no
Change of Control shall have occurred, unless and until that Person subsequently
acquires one or more additional shares representing voting power of Hanover; or

                        (b)     any acquisition by a Person who as of the date
of the establishment of the Plan owned at least thirty-three percent (33%) of
the Voting Shares.

                (ii)(a) Notwithstanding the foregoing, a Change of Control will
occur when the shareholders of Hanover approve any of the following (each, a
"Transaction"):

                        (I)     any reorganization, merger, consolidation or
other business combination of Hanover;

                        (II)    any sale of fifty percent (50%) or more of
Hanover's assets; or

                        (III)   a complete liquidation or dissolution of
Hanover.

                (b)     Notwithstanding subsection (ii)(a) above, shareholder
approval of either of the following types of Transactions will not give rise to
a Change of Control:

                        (I)     a Transaction involving only Hanover and one or
more of its subsidiaries; or

                        (II)    a Transaction immediately following which the
shareholders of Hanover immediately prior to the Transaction continue to have a
majority of the voting power in the resulting entity.


<PAGE>   2

                (iii)   When, within any twenty-four (24) month period, persons
who were directors of Hanover (each, a "Director") immediately before the
beginning of such period (the "Incumbent Directors") shall cease (for any reason
other than death or disability) to constitute at least a majority of the Board
of Directors or the board of directors of any successor to Hanover. For purposes
of this subsection (iii), any Director who was not a Director as of the
Effective Date shall be deemed to be an Incumbent Director if such Director was
elected to the Board of Directors by, or on the recommendation of, or with the
approval of, at least a majority of the members of the Board of Directors or the
nominating committee who, at the time of the vote, qualified as Incumbent
Directors either actually or by prior operation of this subsection (iii), and
any persons (and their successors from time to time) who are designated by a
holder of thirty-three percent (33%) or more of the Voting Shares to stand for
election and serve as Directors in lieu of other such designees serving as
Directors on the effective date of the Plan shall be considered Incumbent
Directors. Notwithstanding the foregoing, any director elected to the Board of
Directors to avoid or settle a threatened or actual proxy contest shall not,
under any circumstances, be deemed to be an Incumbent Director.

        2.3     "Change of Control Payment" means the payment described in
Section 5 of the Plan.

        2.4     "Company" means Hanover and its successors.

        2.5     "Director's Fee" means the sum of the retainer fee, meeting fees
and per diem fees paid to a Director for his/her service on the Board of
Directors for the twelve-month period immediately preceding the effective date
of the Change of Control.

        2.6     "Exchange Act" means the Securities Exchange Act of 1934, as
amended.


        2.7     "Participant" means a Director, provided, however, that the term
Participant shall not be deemed to include (i) any Director who is also an
employee of Hanover for purposes of the Federal Insurance Contributions Act; or
(ii) any persons (and their successors from time to time) who are designated by
a holder of thirty-three percent (33%) or more of the Voting Shares to stand for
election and serve as Director.

        2.8     "Person" means any "person" (as defined in Section 3(a)(9) of
the Exchange Act and as used in Sections 13(d) and 14(d) thereof), including a
"group" as defined in Section 13(d)(3) of the Exchange Act, but excluding (i)
the Company, (ii) any employee benefit plan sponsored or maintained by the
Company (including any trustee of such plan acting as trustee), and (iii) any
person who, as of the date of the establishment of the Plan, owned at least
thirty-three percent (33%) of the Voting Shares (but excluding any unaffiliated
successor or assignee of such thirty-three percent (33%) or greater holder).

        2.9     "Plan Administrator" means the Board of Directors or its
delegate or delegates.

                                    SECTION 3

                       ELIGIBILITY FOR PLAN PARTICIPATION

        Prior to a Change of Control, a Participant shall cease to be a
Participant in the Plan on the earliest to occur of the following dates: (i) the
date of the Participant's death; (ii) the date on which the Participant ceases
to be a Director for any reason; or (iii) the effective date of the termination
of the Plan.

                                    SECTION 4

                    CONDITIONS FOR CHANGE OF CONTROL PAYMENT

        A Participant shall be entitled to receive a Change of Control Payment
as long as he/she is a Director on the effective date of a Change of Control.


                                        2
<PAGE>   3



                                    SECTION 5

                            CHANGE OF CONTROL PAYMENT

        5.1     A Change of Control Payment shall be an amount equal to the
greater of (i) $40,000 or (ii) 150% of the Participant's Director's Fee.

        5.2     A Change of Control Payment shall be paid to a Participant in
one lump sum payment within thirty (30) days following the effective date of the
Change of Control.


                                    SECTION 6

                      CLAIMS FOR CHANGE OF CONTROL PAYMENTS

        In the event that a Director desires to make a claim with respect to a
Change of Control Payment, the Director shall submit evidence thereof
satisfactory to the Plan Administrator. Any claim with respect to a Change of
Control Payment shall be made in writing within ninety (90) days of the event
that the Director is asserting entitles him/her to a Change of Control Payment
under Section 4. Failure by the Director to submit his/her claim within such
ninety (90) day period shall bar the Director from any claim for payment under
the Plan as a result of the occurrence of that event.


                                    SECTION 7

                    ADMINISTRATION AND FINANCING OF THE PLAN

        7.1     Subject to Section 7.2, the Plan Administrator shall have the
discretion, authority, duty, power and responsibility to decide all factual and
legal questions under the Plan, including without limitation, (i) the
interpretation and construction of the provisions of the Plan as it deems
appropriate, necessary or advisable and any ambiguous or unclear terms within
the Plan document, (ii) the adoption, establishment and revision of rules,
procedures and regulations relating to the Plan, (iii) the determination of the
conditions subject to which a Change of Control Payment may be payable, (iv) the
resolution of all questions concerning the status and rights of Participants
under the Plan, whether a Participant is eligible for a Change of Control
Payment under the Plan, and (v) any other determinations which it believes
necessary or advisable for the administration and operation of the Plan. All
determinations of the Plan Administrator shall be final and binding on all
Directors and Participants. The Plan Administrator may appoint a committee or an
agent or other representative to act on its behalf (the "Committee"), and may
delegate to such Committee any of the powers of the Plan Administrator. Any
action that the Committee takes in connection with the Plan shall be considered
to be the action of the Plan Administrator, when the Committee is acting within
the scope of the authority that the Plan Administrator delegates to it, and the
Plan Administrator shall be responsible for all such actions.

        7.2     In the event that a claim is made by a Director pursuant to
Section 6 with respect to his/her eligibility to participate in the Plan or
his/her entitlement to receive a Change of Control Payment, the Board of
Directors shall be required to designate in good faith an independent
third-party to decide such dispute or contest (the "Independent Third-Party').
All determinations of the Independent Third-Party shall be final and binding on
all Directors and Participants.

        7.3     The Company shall pay all Change of Control Payments, if any,
from its general assets.


                                        3
<PAGE>   4




                                    SECTION 8

                            AMENDMENT AND TERMINATION

        The Company in accordance with applicable corporate law reserves the
right at any time to amend or terminate the Plan; provided, however, that the
Plan shall not be amended or terminated in a manner that would adversely affect
any then Participant, if such amendment or termination would be effective during
the period commencing 90 days prior to a Change of Control.


                                    SECTION 9

                            MISCELLANEOUS PROVISIONS

        9.1     The failure of the Company or the Plan Administrator to enforce
any of the provisions of the Plan shall in no way be construed as a waiver of
any such provision, nor in any way to affect the validity of the Plan or any
part hereof, or the right of the Plan Administrator thereafter to enforce every
other provision of the Plan.

        9.2     Any Change of Control Payments received by a Participant shall
be in lieu of and not in addition to and shall supersede and replace severance
benefits, change of control benefits or any similar payments or benefits a
Participant might be eligible for under any other practice, plan, policy,
program, agreement or arrangement of the Company.

        9.3     No Change of Control Payment payable to any Participant shall be
subject in any manner to anticipation, alienation, sale, transfer, assignment,
pledge, encumbrance or charge, other than pursuant to the laws of descent and
distribution, and any attempt to do so shall be null and void.

        9.4     The Plan shall not give any Director or Participant any right or
claim except to the extent that the right is fixed specifically under the Plan.
The establishment of the Plan shall not be construed to give any Participant a
right to be continued as a Director or as interfering with the right of the
Company to remove a Director in accordance with the by-laws of the Company,
articles of incorporation of the Company or other applicable law.

        9.5     Section headings are for convenience only and the language of
the Plan itself shall be controlling.

        9.6     The Plan shall be unfunded. Any liability of the Company under
the Plan shall be based solely on contractual obligations, if any, that are
created hereunder. No such liability of the Company shall be deemed to be
secured by any property of the Company.

        9.7     The Plan Administrator, each member of any Committee designated
under the Plan and the Independent Third-Party shall, except as prohibited by
law, be indemnified and held harmless by the Company from any and all
liabilities, costs and expenses (including legal fees), to the extent not
covered by liability insurance, arising out of any action taken (or the omission
of any action) by such person with respect to the Plan, unless the liability,
cost or expense arises from the person's (i) claim for his or her own Change of
Control Payment, (ii) gross negligence, (iii) bad faith, (iv) reasonable belief
his or her conduct was unlawful, or (v) conviction of any criminal act or
criminal misconduct. This indemnification shall continue as to a person who has
ceased to be the Plan Administrator, an Independent Third-Party, a Committee
member or a member of the Board of Directors, and shall inure to the benefit of
the heirs, executors and administrators of any such person.


                                        4
<PAGE>   5

        9.8     Whenever any Change of Control Payment become payable under the
Plan, the Company shall have the right to withhold such amounts as are
sufficient to satisfy any federal, state or local tax withholding requirements.

        9.9     The Plan shall be construed and administered under the laws of
the State of New York.



        IN WITNESS WHEREOF, Hanover has caused the Plan to be executed on
May 11, 2001.

                                          HANOVER DIRECT, INC.

                                          By: /s/ Charles F. Messina
                                              -------------------------------

                                          Title: Executive Vice President
                                                 Chief Administrative Officer



                                        5