<PAGE>   1
                                                         Exhibit 10.51

                               SECOND AMENDMENT TO
                   REMARKETING AND INTEREST SERVICES AGREEMENT

     This SECOND AMENDMENT TO REMARKETING AND INTEREST SERVICES AGREEMENT (the
"Second Amendment"), dated as of December 18, 1996, by and between HANOVER
DIRECT, INC., a corporation organized and existing under the laws of the State
of Delaware (the "Borrower"), and NATIONSBANK, N.A., a national banking
association and the successor to NationsBank of North Carolina, N.A. (the
"Remarketing Agent");

                              W I T N E S S E T H :

     WHEREAS, the Borrower has previously issued and sold its interest bearing
flexible term notes in two separate series (the first series of such notes, as
amended as of the date hereof, being hereinafter referred to as the "Series A
Notes", the second series of such notes, as amended as of the day hereof, being
hereinafter referred to as the "Series B Notes", and the Series A Notes and the
Series B Notes being hereinafter collectively, referred to as the "Notes") each
in the aggregate principal amount of $10,000,000, pursuant to the provisions of
a Series A Note Agreement dated as of November 9, 1994 between the Borrower and
Norwest Bank Minnesota, N.A., as Trustee and Paying Agent (the "Trustee") (as
amended by that certain First Supplemental Series A Note Agreement dated as of
December 29, 1995 and that certain Second Supplemental Series A Note Agreement
dated as of December 18, 1996 between the Borrower and the Trustee and as
hereinafter amended, the "Series A Note Agreement"), and the Series B Note
Agreement dated as of April 27, 1995 between the Borrower and the Trustee (as
amended by that certain First Supplemental Series B Note Agreement dated as of
December 29, 1995 and that certain Second Supplemental Series B Note Agreement
dated as of December 18, 1996 and as hereinafter amended, the "Series B Note
Agreement", and together with the Series A Note Agreement being hereinafter
collectively referred to as the "Note Agreements"), respectively; and

     WHEREAS, the Borrower and the Remarketing Agent entered into the
Remarketing and Interest Services Agreement dated as of November 9, 1994 (the
"Remarketing Agreement") pursuant to which the Borrower appointed the
Remarketing Agent as its agent to perform certain services in connection with
the remarketing of the Notes tendered for purchase and the determination of the
interest rates and interest periods with respect to the Notes; and

     WHEREAS, the Borrower has this day delivered to the Trustee a Substitute
Series A Letter of Credit in substitution for the Series A Letter of Credit
(each as defined in the Series A Note Agreement) and a Substitute Series B
Letter of Credit in


<PAGE>   2

substitution for the Series B Letter of Credit (each as defined in the Series B
Note Agreement) pursuant to the terms of the respective Note Agreements; and

     WHEREAS, in order to more fully evidence the substitution of letters of
credit referenced above, the Borrower and the Remarketing Agent desire to amend
the Remarketing Agreement, subject to the terms and conditions set forth herein;

     NOW, THEREFORE, the parties hereto agree as follows:

     Section 1. Definitions. All capitalized terms used herein and not otherwise
herein defined shall have the meanings ascribed to them in the applicable Note
Agreement.

     Section 2. Amendment to Second Recital of the Remarketing Agreement. The
Second Recital of the Placement Agreement set forth on page 1 thereof is hereby
deleted in its entirety and replaced with the following:

          WHEREAS, the payment when due of the principal of, interest on and
     Purchase Price (as defined in the applicable Note Agreement) of the Series
     A Notes will be supported, to the extent provided therein, by the Series A
     Letter of Credit, and the payment when due of the principal of, interest on
     and Purchase Price of the Series B Notes will be supported, to the extent
     provided therein, by the Series B Letter of Credit (the Series A Letter of
     Credit and the Series B Letter of Credit are sometimes hereinafter
     collectively referred to as the "Letter of Credit"), each issued in favor
     of the Trustee by Swiss Bank Corporation, New York Branch (the "Bank") and
     each in an amount of not less than $9,638,541, representing the principal
     amount of the applicable series of Notes plus 35 days interest on such
     amount computed at the Maximum Rate (as defined in the applicable Note
     Agreement) on the basis of actual number of days elapsed in a year of 360
     days, all pursuant to the Reimbursement Agreement dated as of December 18,
     1996 (the "Reimbursement Agreement"), by and between the Borrower and the
     Bank, pursuant to which, among other things, the Bank has executed the
     respective Application pursuant to which the Series A Letter of Credit and
     the Series B Letter of Credit are issued by the Bank and delivered to the
     Trustee, and any and all modifications, alterations, amendments and
     supplements thereto any similar agreements between or among the Account
     Parties, the Borrower and the issuer of a Substitute Series A Letter of
     Credit or Substitute Series B Letter of Credit or the lender providing
     credit support to such issuer;

     Section 3. Amendment to Section 7 of the Remarketing Agreement. Subsection
(b) of Section 7 of the Remarketing


                                        2

<PAGE>   3

Agreement is hereby amended by deleting the reference to the "Lender" in the
third line thereof and replacing it with a reference to the "Bank".

     Section 4. Amendment to Section 11 of the Remarketing Agreement. Section 11
of the Remarketing Agreement is hereby amended by deleting the references to,
and the addresses of, the "Lender" and "L/C Issuer" therein and replacing them
with the following:

      If to the Bank:         Swiss Bank Corporation,
                               New York Branch
                              222 Broadway,
                              New York, New York 10038
                              Attention:
                              Telephone No.:
                              Fax No.:

     Section 5. Effect of Second Amendment. Except as modified hereby, all of
the terms and provisions of the Remarketing Agreement shall remain in full force
and effect.

     Section 6. Governing Law. This Second Amendment and the Remarketing
Agreement, as amended hereby, shall be deemed to be contracts made under, and
for all purposes shall be construed in accordance with, the laws of the State of
North Carolina.

     Section 7. Severability. If any provision of this Second Amendment shall be
determined to be unenforceable by a court of law, that shall not affect any
other provision of this Second Amendment.

     Section 8. Counterparts. This Second Amendment may be executed in several
counterparts, each of which shall be an original and all of which, taken
together, shall constitute one and the same instrument.

                         [Signatures on following page]


                                        3

<PAGE>   4

     IN WITNESS WHEREOF, the parties hereto have caused this Second Amendment to
be duly executed as of the day and year first above written.



                                    HANOVER DIRECT, INC.


                                    By:________________________________
                                    Name:______________________________
                                    Title:_____________________________



                                    NATIONSBANK, N.A.


                                    By:________________________________
                                    Name:______________________________
                                    Title:_____________________________


                                        4