<PAGE>   1
                       SECURITIES AND EXCHANGE COMMISSION
                             WASHINGTON, D.C. 20549

                                  FORM 10-K/A1

                                  AMENDMENT TO
                                  ANNUAL REPORT
                         PURSUANT TO SECTION 13 OR 15(d)
                     OF THE SECURITIES EXCHANGE ACT OF 1934
                   for the fiscal year ended December 28, 1996

                         Commission file number 1-12082

                              HANOVER DIRECT, INC.
--------------------------------------------------------------------------------
              (Exact name of registrant as specified in its charter

        DELAWARE                                          13-0853260
----------------------------------     -----------------------------------------
(State of incorporation)                    (I.R.S. Employer Identification No.)

1500 HARBOR BOULEVARD, WEEHAWKEN, NEW JERSEY                             07087
--------------------------------------------                          ----------
  (Address of principal executive offices)                            (Zip Code)

Registrant's telephone number, including area code: 201-863-7300

Securities registered pursuant to Section 12(b) of the Act:

                                                              name of each
                                                                exchange
      Title of each class                                   which registered

Common Stock, $.66 2/3 par Value                         American Stock Exchange

Securities registered pursuant to Section 12(g) of the Act: None

Indicate by a check mark whether the registrant (1) has filed all reports
required to be filed by Section 13 or 15(d) of the Securities Exchange Act
of 1934 during the preceding 12 months (or for such shorter period that the
registrant was required to file such reports), and (2) has been subject to
such filing requirements for the past 90 days. YES X  NO

Indicate by a check mark if disclosure of delinquent filers pursuant to Item 405
of Regulation S-K is not contained herein, and will not be contained, to the
best of registrant's knowledge, in definitive proxy or information statements
incorporated by reference in Part III of this Form 10-K or any amendment to this
Form.

As of March 24, 1997, the aggregate market value of the voting stock held by
non-affiliates of the registrant was $38,280,286 (based on the closing price of
the Common Stock on the American Stock Exchange on March 24, 1997).

As of March 24, 1997, the registrant had 144,318,452 shares of Common Stock
outstanding.

                       DOCUMENTS INCORPORATED BY REFERENCE

The Company's definitive proxy statement to be filed by the Company pursuant to
Regulation 14A is incorporated into items 10, 11, 12 and 13 of Part III of this
Form 10-K.


<PAGE>   2
                                Explanatory Note

     This Form 10-K/A1 is being filed by Hanover Direct, Inc., a Delaware
corporation (the "Company"), as an amendment to its Annual Report on Form 10-K
for the fiscal year ended December 28, 1996, filed March 28, 1997, to add
certain additional exhibits in Part IV thereof.


<PAGE>   3
                                   SIGNATURES

     Pursuant to the requirements of Section 13 or 15(d) of the Securities
Exchange Act of 1934, the registrant has duly caused this amendment to report to
be signed on its behalf by the undersigned, thereunto duly authorized.

                                         HANOVER DIRECT, INC.
                                             (registrant)

Dated: April 11, 1997                    By: /s/ Rakesh K. Kaul
                                             -------------------------------
                                             Rakesh K. Kaul
                                             President and Chief
                                             Executive Officer

     Pursuant to the requirements of the Securities Exchange Act of 1934, this
amendment to report has been signed below by the following persons on behalf of
the registrant and in the capacities indicated and on the date indicated below.

Principal Financial Officer:

/s/ Larry J. Svoboda
---------------------------------
Larry J. Svoboda
Senior Vice President and
Chief Financial Officer

Board of Directors:

/s/ Ralph Destino                              /s/ Edmund R. Manwell
---------------------------------              ---------------------------------
Ralph Destino, Director                        Edmund R. Manwell, Director


/s/ J. David Hakman                            /s/ Jan du Plessis
---------------------------------              ---------------------------------
J. David Hakman, Director                      Jan du Plessis, Director


/s/ Rakesh K. Kaul                             /s/ Alan G. Quasha
---------------------------------              ---------------------------------
Rakesh K. Kaul, Director                       Alan G. Quasha, Director

                                               /s/ Howard M.S. Tanner
---------------------------------              ---------------------------------
Theodore H. Kruttschnitt,                      Howard M.S. Tanner,
Director                                       Director

/s/ Elizabeth Valk Long                        /s/ Robert F. Wright
---------------------------------              ---------------------------------
Elizabeth Valk Long, Director                  Robert F. Wright, Director


Dated: April 11, 1997



<PAGE>   4
                                  EXHIBIT INDEX

<TABLE>
<CAPTION>
EXHIBIT NUMBER      
ITEM 601 OF         DESCRIPTION OF DOCUMENT AND INCORPORATION
REGULATION S-K      BY REFERENCE WHERE APPLICABLE
--------------      -----------------------------
                                                                                     PAGE
                                                                                     NO.
                                                                                     ---
<S>                 <C>                                                              <C>    
     2.1            Asset Purchase Agreement dated as of December 1, 1994 among
                    the Company, LWI Holdings, Inc., Bankers Trust Company,
                    Leichtung, Inc. and DRI Industries, Inc. Incorporated by
                    reference to the Company's Annual Report on Form 10-K for
                    the year ended December 31, 1994.

     2.2            Stock Purchase Agreement dated as of February 16, 1995 among
                    the Company, Hanover Holdings, Inc., Aegis Safety Holdings,
                    Inc., F.L. Holdings, Inc., Roland A.E. Franklin, Martin E.
                    Franklin, Jonathan Franklin, Floyd Hall, Frederick Field,
                    Homer G. Williams, Frank Martucci, Norm Thompson Outfitters,
                    Inc. and Capital Consultants, Inc. (as agent) (collectively,
                    the "Aegis Sellers"). Incorporated by reference to the
                    Company's Annual Report on Form 10-K for the year ended
                    December 31, 1994.

     2.3            Stock Purchase Agreement dated as of May 19, 1995 by and
                    among the Company, Austad Holdings, Inc. ("AHI"), The Austad
                    Company ("TAC"), David B. Austad ("DBA"), Denise Austad
                    ("DA"), David Austad, as custodian ("DBAC"), Oscar Austad,
                    Dorothy Austad, Randall Austad, Kristi Austad, Lori Miller,
                    Robin Miller, Kerri Derenge, Sharon Stahl, Lori Miller, as
                    custodian, Dorothy Austad, as attorney-in-fact, and
</TABLE>
<PAGE>   5
                    Kara Miller (collectively, the "Austad Individuals").
                    Incorporated by reference to the Company's Annual Report on
                    Form 10-K for the year ended December 30, 1995.

     2.4            Agreement and Plan of Corporate Separation and
                    Reorganization dated as of February 16, 1996 by and among
                    the Company, AHI, TAC, DBA, DBAC, and DA. Incorporated by
                    reference to the Company's Annual Report on Form 10-K for
                    the year ended December 30, 1995.

     3.1            Restated Certificate of Incorporation.***

     3.2            By-laws.***

     4.1            Warrant Agreement dated as of October 25, 1991 ("NAR
                    Warrant") between the Company* and NAR Group Limited ("NAR")
                    for 279,110 shares of Common Stock. Incorporated by
                    reference to the Company's* Current Report on Form 8-K dated
                    October 25, 1991.

     4.2            Registration Rights Agreement dated as of July 8, 1991 among
                    the Company*, NAR and Intercontinental Mining & Resources
                    Limited ("IMR"). Incorporated by reference to the Company's*
                    Current Report on Form 8-K Dated July 10, 1991.

     4.3            Warrant Agreement dated as of January 1, 1994 between the
                    Company and Sears Shop At Home Services, Inc. ("Sears").
                    Incorporated by reference to the Company's Annual Report on
                    Form 10-K for the year ended December 31, 1994.
<PAGE>   6
     4.4            Registration Rights Agreement dated as of February 16, 1995
                    among the Company and the Aegis Sellers. Incorporated by
                    reference to the Company's Annual Report on Form 10-K for
                    the year ended December 31, 1994.

     4.5            Warrant Agreement dated as of July 8, 1991 between the
                    Company and IMR for 1,750,000 shares of Common Stock.
                    Incorporated by reference to the Company's Current Report on
                    form 8-K dated July 10, 1991.

     4.6            Warrant Agreement dated as of October 25, 1991 between the
                    Company and NAR for 931,791 shares of Common Stock.
                    Incorporated by reference to the Company's Current Report on
                    form 8-K dated October 25, 1991.

     4.7            Second Amendment to Warrant Agreement and Warrant
                    Certificate for 931,791 shares of Common Stock, between the
                    Company and NAR dated as of November 13, 1995. Incorporated
                    by reference to the Company's Annual Report on Form 10-K for
                    the year ended December 30, 1995.

     4.8            First Amendment to Warrant Agreement and Warrant Certificate
                    for 1,750,000 shares of Common Stock, between the Company
                    and IMR dated as of November 13, 1995. Incorporated by
                    reference to the Company's Annual Report on Form 10-K for
                    the year ended December 30, 1995.

     4.9            First Amendment to Warrant Agreement and Warrant Certificate
                    for 279,110 shares of Common Stock, between the Company and
                    NAR dated as of November 13, 1995. Incorporated by
                    reference to the Company's Annual Report on Form 10-K for
                    the year ended December 30, 1995.
<PAGE>   7

     4.10           Second Amendment to Warrant Agreement between the Company
                    and IMR dated as of August 23, 1996. FILED HEREWITH.

     4.11           Second Amendment to Warrant Agreement between the Company
                    and NAR dated as of August 23, 1996.***

     4.12           Third Amendment to Warrant Agreement between the Company and
                    NAR dated as of August 23, 1996.***

     10.1           Stock Option Plan, as amended. Incorporated by reference to
                    the Company's* Annual Report on Form 10-K for the fiscal
                    year ended December 28, 1991.

     10.2           Account Purchase Agreement dated as of December 21, 1992
                    among the Company*, Hanover Direct Pennsylvania, Inc.
                    ("HDPI"), Brawn of California, Inc. ("Brawn") and General
                    Electric Capital Corporation ("GECC"). Incorporated by
                    reference to the Company's* Annual Report on Form 10-K for
                    the fiscal year ended December 26, 1992.

     10.3           Amendment No. 1 to the Account Purchase Agreement dated as
                    of July 12, 1993 among the Company*, HDPI, Brawn, Gump's By
                    Mail, Gump's, Gump's Holdings and GECC. Incorporated by
                    reference to the Company's* Current Report on Form 8-K dated
                    July 12, 1993.
<PAGE>   8
     10.4           Amendment No.2 to the Account Purchase Agreement dated as of
                    June 1, 1995 among the Company, HDPI, Brawn, Gump's, Gump's
                    By Mail, Gump's Holdings and GECC. Incorporated by reference
                    to the Company's Annual Report on Form 10-K for the year
                    ended December 30, 1995.

     10.5           Waiver and Amendment No. 3 to the Account Purchase Agreement
                    dated as of December 14, 1995 among the Company, HDPI, Brawn
                    and GECC. Incorporated by reference to the Company's Annual
                    Report on Form 10-K for the year ended December 30, 1995.

     10.6           Amendment No. 4 to the Amended and Restated Account Purchase
                    Agreement dated as of June 28, 1996 among the Company, HDPI,
                    Brawn, Gump's, Gump's by Mail, Gump's Holdings and GECC.
                    FILED HEREWITH.

     10.7           Form of Stock Option Agreement between the Company* and
                    certain Directors of the Company, as amended. Incorporated
                    by reference to the Company's* Annual Report on Form 10-K
                    for the fiscal year ended December 28, 1991.

     10.8           Termination of Employment Agreement and Employment and
                    Consulting Agreement dated as of December 31, 1995 between
                    the Company and Jack E. Rosenfeld.***

     10.9           Registration Rights Agreement between the Company and 
                    Rakesh K. Kaul, dated as of August 23, 1996. FILED HEREWITH.
<PAGE>   9
     10.10          Form of Indemnification Agreement among the Company* and
                    each of the Company's directors and executive officers.
                    Incorporated by reference to the Company's* Current Report
                    on Form 8-K dated October 25, 1991.

     10.11          Letter Agreement dated May 5, 1989 among the Company*,
                    Theodore H. Kruttschnitt, J. David Hakman and Edmund R.
                    Manwell. Incorporated by reference to the Company's* Current
                    Report on Form 8-K dated May 10, 1989.

     10.12          Hanover Direct, Inc. Savings Plan as amended. Incorporated
                    by reference to the Company's Annual Report on Form 10-K for
                    the year ended January 1, 1994.

     10.13          Restricted Stock Award Plan. Incorporated by reference to
                    the Company's* Registration Statement on Form S-8 filed on
                    February 24, 1993, Registration No. 33-58760.

     10.14          All Employee Equity Investment Plan. Incorporated by
                    reference to the Company's* Registration Statement on Form
                    S-8 filed on February 24, 1993, Registration No. 33-58756.

     10.15          Executive Equity Incentive Plan, as amended.***

     10.16          Form of Supplemental Retirement Plan. Incorporated by
                    reference to the Company's Annual Report on Form 10-K for
                    the year ended January 1, 1994.
<PAGE>   10
     10.17          1996 Stock Option Plan. Incorporated by reference to the
                    Company's 1996 Proxy Statement.

     10.18          Loan and Security Agreement dated as of November 14, 1995 by
                    and among Congress Financial Corporation ("Congress"),
                    Hanover Direct Pennsylvania, Inc. ("HDPA"), Brawn of
                    California, Inc. ("Brawn"), Gump's by Mail, Inc. ("Gump's by
                    Mail"), Gump's Corp.("Gump's"), The Company Store, Inc.
                    ("The Company Store") , Tweeds, Inc. ("Tweeds"), LWI
                    Holdings, Inc.("LWI"), Aegis Catalog Corporation ("Aegis"),
                    Hanover Direct Virginia, Inc. ("HDVA") and Hanover Realty
                    Inc. ("Hanover Realty"). Incorporated by reference to the
                    Company's Annual Report on Form 10-K for the year ended
                    December 30, 1995.

     10.19          First Amendment to Loan and Security Agreement dated as of
                    February 22, 1996 by and among Congress, HDPI, Brawn, Gump's
                    by Mail, Gump's, The Company Store, Tweeds, LWI, Aegis, HDVA
                    and Hanover Realty. FILED HEREWITH.

     10.20          Second Amendment to Loan and Security Agreement dated as of
                    April 16, 1996 by and among Congress, HDPI, Brawn, Gump's by
                    Mail, Gump's, The Company Store, Tweeds, LWI, Aegis, HDVA
                    and Hanover Realty. FILED HEREWITH.


     10.21          Third Amendment to Loan and Security Agreement dated as of
                    May 24, 1996 by and among Congress, HDPI, Brawn, Gump's by
                    Mail, Gump's, The Company Store, Tweeds, LWI, Aegis, HDVA
                    and Hanover Realty. FILED HEREWITH.
<PAGE>   11
     10.22          Fourth Amendment to Loan and Security Agreement dated as of
                    May 31, 1996 by and among Congress, HDPI, Brawn, Gump's
                    by Mail, Gump's, The Company Store, Tweeds, LWI, Aegis, HDVA
                    and Hanover Realty. FILED HEREWITH.

     10.23          Fifth Amendment to Loan and Security Agreement dated as of
                    September 11, 1996 by and among Congress, HDPI, Brawn,
                    Gump's by Mail, Gump's, The Company Store, Tweeds, LWI,
                    Aegis, HDVA and Hanover Realty. FILED HEREWITH.

     10.24          Sixth Amendment to Loan and Security Agreement dated as of
                    December 5, 1996 by and among Congress, HDPI, Brawn, Gump's
                    by Mail, Gump's, The Company Store, Tweeds, LWI, Aegis, HDVA
                    and Hanover Realty. FILED HEREWITH.

     10.25          Seventh Amendment to Loan and Security Agreement dated as of
                    December 18, 1996 by and among Congress, HDPI, Brawn, Gump's
                    by Mail, Gump's, The Company Store, Tweeds, LWI, Aegis, HDVA
                    and Hanover Realty. FILED HEREWITH.

     10.26          Subordination Agreement, dated as of November 14, 1995,
                    among Congress, IMR, and the Trustee. Incorporated by
                    reference to the Company's Annual Report on Form 10-K for
                    the year ended December 30, 1995.

     10.27          Long-Term Incentive Plan for Rakesh K. Kaul. FILED HEREWITH.

     10.28          Short-Term Incentive Plan for Rakesh K. Kaul. ***

<PAGE>   12
     10.29          Employment Agreement dated as of March 7, 1996 between the
                    Company and Rakesh K. Kaul.***

     10.30          Tandem Option Plan dated as of August 23, 1996 between the
                    Company and Rakesh K. Kaul.***

     10.31          Closing Price Option dated as of August 23, 1996 between the
                    Company and Rakesh K. Kaul.***

     10.32          Performance Price Option dated as of August 23, 1996 between
                    the Company and Rakesh K. Kaul.***

     10.33          Six-Year Stock Option dated as of August 23, 1996 between
                    NAR and Rakesh K. Kaul.***

     10.34          Seven-Year Stock Option dated as of August 23, 1996 between
                    NAR and Rakesh K. Kaul.***

     10.35          Eight-Year Stock Option dated as of August 23, 1996 between
                    NAR and Rakesh K. Kaul.***

     10.36          Nine-Year Stock Option dated as of August 23, 1996 between
                    NAR and Rakesh K. Kaul.***

     10.37          Letter of Credit, dated December 18, 1996, from Swiss Bank
                    Corporation, New York Branch ("Swiss Bank") in favor of
                    Fleet National Bank, as trustee ("Fleet Bank").***

     10.38          Reimbursement Agreement, dated as of December 18, 1996, by
                    and among Swiss Bank and the Company.***
<PAGE>   13
     10.39          Hanover Indemnity Agreement, dated as of December 18, 1996,
                    between Richemont Finance S.A. ("Richemont") and the
                    Company, HDPI, Brawn, Gump's, Gump's by Mail, The Company
                    Store, Tweeds, LWI, Aegis, HDVA and Hanover Realty. ***

     10.40          Subordination Agreement, dated as of December 18, 1996,
                    between Congress and Swiss Bank. FILED HEREWITH.

     10.41          Subordination Agreement, dated as of December 18, 1996
                    between Congress and Richemont. FILED HEREWITH.

     10.42          Series A Note Agreement, dated as of November 9, 1994,
                    between the Company and Norwest Bank Minnesota, N.A.
                    ("Norwest"), as trustee. FILED HEREWITH.

     10.43          Placement Agreement, dated as of November 9, 1994, by and
                    between the Company and NationsBank of North Carolina, N.A.
                    FILED HEREWITH.

     10.44          Remarketing and Interest Services Agreement, dated as of
                    November 9, 1994, by and between the Company and NationsBank
                    of North Carolina, N.A. FILED HEREWITH.

     10.45          First Supplemental Series A Note Agreement, dated as of
                    December 29, 1995, between the Company and Norwest. FILED 
                    HEREWITH.

     10.46          First Amendment to Placement Agreement, dated as of December
                    29, 1995 by and between the Company and NationsBank of North
                    Carolina, N.A. FILED HEREWITH.
<PAGE>   14
     10.47          First Amendment to Remarketing and Interest Services
                    Agreement, dated as of December 29, 1995 by and between the
                    Company and NationsBank of North Carolina, N.A. FILED 
                    HEREWITH.

     10.48          Second Supplemental Series A Note Agreement, dated as of
                    December 18, 1996, between the Company and Norwest. *** 

     10.49          Second Amendment to Series A Note, dated December 18, 1996
                    made by the Company. FILED HEREWITH

     10.50          Second Amendment to Placement Agreement, dated as of
                    December 18, 1996 by and between the Company and NationsBank
                    of North Carolina, N.A. ***

     10.51          Second Amendment to Remarketing and Interest Services 
                    Agreement, dated as of December 18, 1996 by and between 
                    the Company and NationsBank of North Carolina, N.A. *** 

     10.52          Series B Note Agreement dated as of April 25, 1995, between
                    the Company and Norwest. FILED HEREWITH.

     10.53          [intentionally deleted]

     10.54          [intentionally deleted]

     10.55          First Amendment to Series B Note Agreement, dated as of
                    December 29, 1995, between the Company and Norwest. FILED 
                    HEREWITH.
<PAGE>   15
     10.56          Second Supplemental to Series B Note Agreement, dated as of
                    December 18, 1996, between the Company and Norwest. FILED 
                    HEREWITH.

     10.57          Second Amendment to Series B Note, dated December 18, 1996
                    made by the Company. ***

     10.58          Series B Letter of Credit, dated as of December 18, 1996,
                    issued by Swiss Bank. ***

     10.59          [intentionally deleted]

     10.60          NAR Promissory Note dated as of September 11, 1996. ***

     10.61          Series A Letter of Credit, dated as of December 18, 1996,
                    issued by Swiss Bank. ***

     10.62          First Amendment to Series A Note, dated as of December 29,
                    1995 made by Hanover Direct, Inc. FILED HEREWITH.

     10.63          $10,000,000 Series B Note, dated as of April 27, 1995 and
                    made by Hanover Direct, Inc. FILED HEREWITH.

     10.64          First Supplemental Series B Note Agreement, dated as of
                    December 29, 1995. FILED HEREWITH.

     10.65          $10,000,000 Series A Note, dated as of November 9, 1994 and
                    made by Hanover Direct, Inc. FILED HEREWITH.

     11             Computation of Per Share Earnings. ***

     21.1           Subsidiaries of the Registrant. ***
       
     23.1           Consent of Independent Public Accountants. ***

     27.1           Financial Data Schedule. **/***

--------------

   *  Hanover Direct, Inc., a Delaware corporation, is the successor by merger 
      to The Horn & Hardart Company and The Hanover Companies.

  **  EDGAR filing only.

 ***  Previously submitted with Hanover's Annual Report on Form 10-K, filed on
      March 28, 1997.