<PAGE>   1
                                                          Exhibit 10.32

                             PERFORMANCE YEAR OPTION

            This STOCK OPTION AGREEMENT (this "Agreement") is made as of August
23, 1996 between Hanover Direct, Inc., a Delaware corporation (the "Company"),
and Rakesh K. Kaul (the "Executive").

            WHEREAS, the Compensation Committee of the Company's Board of
Directors (the "Compensation Committee") has heretofore adopted and the
Company's shareholders have heretofore approved and ratified the Long-Term
Incentive Plan for Rakesh K. Kaul (the "Plan"); and

            WHEREAS, the Plan provides for the granting of a performance year
option subject to the terms set forth herein.

            NOW, THEREFORE, in consideration of the premises and the mutual
agreements herein set forth, the parties hereto agree as follows:

            1. The Company hereby evidences and confirms the grant to the
Executive on the date hereof (the "Date of Grant") by the Compensation Committee
of an option (the "Option") to purchase 1,000,000 shares of Common Stock (the
"Shares") at an option price of $1.15625 per share, representing the fair market
value of the Common Stock on the date hereof. The Option shall expire on March
7, 2006 (the "Expiration Date"), subject to earlier cancellation or termination
as provided herein.

            2. Subject to the other provisions contained herein regarding the
exercisability of the Option, this Option shall become exercisable only as
provided in this Section 2.

          (a) On or before December 1, 1996 and March 31 of each of the three
     successive years (four years if required to implement the carryover
     provisions described below), the Compensation Committee shall establish
     written performance goals, including a performance target, with respect to
     such year ("performance year"). The performance goals shall be expressed in
     terms of one or more of the following objective financial criteria with
     respect to the Company: earnings per share, earnings before interest and
     taxes, earnings before interest, taxes, depreciation, and amortization, or
     inventory management. The Compensation Committee shall promptly certify
     following the end of each performance year whether the preestablished
     performance target with respect to such performance year has been attained.
     The Option


                                       -1-


<PAGE>   2

     shall become exercisable with respect to 250,000 Shares on the Vesting Date
     for the performance year ending December 31, 1996, and with respect to an
     additional 250,000 Shares on the respective Vesting Dates for each of the
     three succeeding performance years, provided in each case that the
     Compensation Committee certifies that the preestablished performance target
     with respect to such performance year has been attained. The Vesting Date
     for a performance year means the March 7 that next follows the end of such
     performance year.

          (b) If the performance target for 1996, 1997, or 1998 is not attained,
     the number of Shares as to which the Option would otherwise have become
     exercisable on the Vesting Date for such performance year but for such
     failure (as determined after applying the carryover provisions of this
     Section 2(b)) shall be carried over and added to the number of Shares as to
     which the Option shall become exercisable on the Vesting Date for the next
     performance year in the event the performance target for such next
     performance year is attained. If the performance target for 1999 is not
     attained, the number of Shares as to which the Option would otherwise have
     become exercisable on the Vesting Date for such performance year but for
     such failure (as determined without applying the carryover provisions of
     this Section 2(b)) shall be carried over and become exercisable on the
     Vesting Date for the performance year 2000 in the event the performance
     target for the performance year 2000 is attained.

          (c) Notwithstanding the foregoing, the Option shall immediately vest
     and become exercisable in full upon the Executive's termination of
     employment by reason of death or permanent disability (as determined by the
     Compensation Committee), or upon the occurrence of a change in control (as
     defined in the Employment Agreement) during the term of the Employment
     Agreement or within six months following the end of such term. For purposes
     hereof, a permanent disability means the Executive's inability to engage in
     any substantial gainful activity by reason of any medically determinable
     physical or mental impairment which can be expected to result in death or
     which has lasted or can be expected to last for a continuous period of not
     less than 12 months.

            3. In the event of a termination of the Executive's employment with
the Company while any portion of the Option remains unexercised, the Executive's
rights to exercise the Option shall be exercisable only as follows:


                                       -2-

<PAGE>   3

          (i) Involuntary Termination. If the Executive's employment is
     involuntarily terminated by the Company other than for cause, the Executive
     may, until the later of (i) 12 months following the date of such
     termination or (ii) in the event such termination occurs during a
     performance year, three days after the Compensation Committee has
     certified, and communicated to the Executive, whether the performance
     target for such performance year has been met, exercise the Option with
     respect to such number of Shares as to which the Option is exercisable (or
     would be exercisable if his employment had not terminated) on the date of
     exercise, as determined pursuant to Section 2. For purposes hereof, the
     provisions of the Employment Agreement shall apply in determining whether
     the Executive's employment has been involuntarily terminated by the Company
     other than for cause.

          (ii) Death. If the Executive's employment terminates by reason of
     death, his Option may be exercised during the 12- month period following
     such termination.

          (iii) Disability. If the Executive's employment terminates by reason
     of permanent disability (as determined by the Compensation Committee), the
     Option may be exercised during the three-month period following such
     termination.

          (iv) Termination in Other Circumstances. If the Executive's employment
     terminates in circumstances not described in clauses (i) through (iii), the
     Executive may, within 30 days following such termination, exercise the
     Option with respect to such number of Shares as to which the Option is
     exercisable (or would be exercisable if his employment had not terminated)
     on the date of exercise, as determined pursuant to Section 2.

Notwithstanding the foregoing, the Option shall in no event be exercisable in
whole or in part after the Expiration Date.


     4. (a) Except as provided in paragraph (b), the Option is not transferable
by the Executive other than by will or the laws of descent and distribution and
is exercisable, during the Executive's lifetime, only by the Executive.

        (b) Notwithstanding the provisions of paragraph (a), if the current
transferability restrictions imposed by the Securities and Exchange Commission
under Rule 16b-3 are eliminated or modified, the following provisions shall
apply if and to the extent that they will not adversely affect the Option's
status under such

        
                                       -3-

<PAGE>   4

rule:

          (i) In the event of the Executive's incapacity, the Option may be
     exercised by a conservator, guardian, or the agent under a Durable Power of
     Attorney;

          (ii) Upon the Executive's death, the Option is transferable by will,
     by a revocable or irrevocable trust established by the Executive, or by a
     written beneficiary designation executed by the Executive and delivered to
     the Company prior to the Executive's death;

          (iii) The Executive may transfer the Option to the Executive's spouse
     and/or issue or trusts for the benefit of the Executive, the Executive's
     spouse, and/or the Executive's issue.

            5. In order to exercise the Option, in whole or in part, the
Executive shall give written notice to the Company, specifying the number of
Shares to be purchased and the purchase price to be paid, and accompanied by the
payment of the purchase price. Such purchase price may be paid in cash, a
certified check, or a bank check payable to the Company, or in whole shares of
Common Stock evidenced by negotiable certificates, valued at their fair market
value on the date of exercise, or in a combination of the foregoing.
Alternatively, the Option may be exercised, in whole or in part, by delivering a
properly executed exercise notice together with irrevocable instructions to a
broker to deliver promptly to the Company the amount of sale or loan proceeds
necessary to pay the purchase price, and such other documents as the
Compensation Committee may require. Upon receipt of payment, the Company shall
deliver to the Executive (or to any other person entitled to exercise the
Option) a certificate or certificates for such Shares. If certificates
representing shares of Common Stock are used to pay all or part of the purchase
price of the Option, separate certificates shall be delivered by the Company
representing the same number of shares as each certificate so used and an
additional certificate shall be delivered representing the additional shares to
which the Executive is entitled as a result of exercise of the Option.

            6. The Option shall be exercised only with respect to full Shares;
no fractional Shares shall be issued.

            7. As a condition to the issuance of Shares under the Option, the
Executive agrees to remit to the Company at the time of exercise any taxes
required to be withheld by the Company under the applicable laws or other
regulations of any governmental authority, whether federal, state or local, and
whether domestic or foreign. The Company shall promptly remit


                                       -4-

<PAGE>   5

such taxes to the applicable governmental authority.

            8. If the Executive so requests in writing, shares purchased upon
exercise of the Option may be issued in the name of the Executive and another
person jointly with the right of survivorship, or in the name of a revocable
trust of which the Executive is the grantor.

            9. The Option does not qualify as an incentive stock option under
Section 422 of the Internal Revenue Code.

            10. This Option shall be binding upon and inure to the benefit of
any successor or assignee of the Company and to any executor, administrator,
legal representative, legatee, or distributee or transferee entitled by law or
the provisions of the Plan to the Executive's rights hereunder.

            11.  The Option is subject in all respects to the terms
of the Plan, the provisions of which are incorporated in this
Agreement by reference.

            12. This Agreement is entered into, and shall be construed and
enforced, under the laws of the State of New York, and shall not be modified
except by written agreement signed by the parties hereto.

            IN WITNESS WHEREOF, the parties have executed this Agreement as of
the date first above written.

                                                 HANOVER DIRECT, INC.



                                                 By: ________________________




                                                
                                                 ____________________________
                                                 Rakesh K. Kaul


                                       -5-