<PAGE>
                                                             [Execution Version]
Exhibit 10.3

                             EIGHTEENTH AMENDMENT TO
                           LOAN AND SECURITY AGREEMENT

        THIS EIGHTEENTH AMENDMENT TO LOAN AND SECURITY AGREEMENT, dated as of
November 12, 2001, is entered into by and among CONGRESS FINANCIAL CORPORATION,
a Delaware corporation ("Lender"), HANOVER DIRECT PENNSYLVANIA, INC., a
Pennsylvania corporation ("HDPI"), BRAWN OF CALIFORNIA, INC., a California
corporation ("Brawn"), GUMP'S BY MAIL, INC., a Delaware corporation ("GBM"),
GUMP'S CORP., a California corporation ("Gump's"), LWI HOLDINGS, INC., a
Delaware corporation ("LWI"), HANOVER DIRECT VIRGINIA INC., a Delaware
corporation ("HDV"), HANOVER REALTY, INC., a Virginia corporation ("Hanover
Realty"), THE COMPANY STORE FACTORY, INC., a Delaware corporation ("TCS
Factory"), THE COMPANY OFFICE, INC., a Delaware corporation ("TCS Office"),
TWEEDS, LLC, a Delaware limited liability company ("Tweeds LLC"), SILHOUETTES,
LLC, a Delaware limited liability company ("Silhouettes LLC"), HANOVER COMPANY
STORE, LLC, a Delaware limited liability company ("HCS LLC"), DOMESTICATIONS,
LLC, a Delaware limited liability company ("Domestications LLC") and KEYSTONE
INTERNET SERVICES, INC., a Delaware corporation ("Keystone Internet"; and
together with HDPI, Brawn, GBM, Gump's, LWI, HDV, Hanover Realty, TCS Factory,
TCS Office, Tweeds LLC, Silhouettes, HCS LLC and Domestications, each
individually referred to herein as a "Borrower" and collectively, as
"Borrowers"), and HANOVER DIRECT, INC., a Delaware corporation, ("Hanover"),
AMERICAN DOWN & TEXTILE COMPANY, a Wisconsin corporation ("American Down"), D.M.
ADVERTISING, INC., a New Jersey corporation ("DM Advertising"), SCANDIA DOWN
CORPORATION, a Delaware corporation ("Scandia"), KEYSTONE LIQUIDATIONS, INC., a
Delaware corporation, formerly known as Tweeds of Vermont, Inc., HANOVER HOME
FASHIONS GROUP, LLC, a Delaware limited liability company ("HHFG LLC"), KITCHEN
& HOME, LLC, a Delaware limited liability company ("Kitchen & Home, LLC"),
DOMESTICATIONS KITCHEN & GARDEN, LLC, a Delaware limited liability company
("Domestications K&G, LLC"), ENCORE CATALOG, LLC, a Delaware limited liability
company ("Encore LLC"), CLEARANCE WORLD OUTLETS, LLC, a Delaware limited
liability company ("Clearance World"), SCANDIA DOWN, LLC, a Delaware limited
liability company ("Scandia Down, LLC"), ERIZON, INC., a Delaware corporation
("erizon, inc."), HANOVER BRANDS, INC., a Delaware corporation ("Hanover
Brands"), ERIZON.COM, INC., a Delaware corporation ("erizon.com"), LACROSSE
FULFILLMENT, LLC, a Delaware limited liability company ("LaCrosse, LLC"), and
SAN DIEGO TELEMARKETING, LLC, a Delaware limited liability company ("San Diego
LLC"; each individually a "Guarantor" and collectively "Guarantors").

                              W I T N E S S E T H:

        WHEREAS, Borrowers, Guarantors and Lender are parties to the Loan and
Security Agreement, dated November 14, 1995, as amended by First Amendment to
Loan and Security Agreement, dated February 22, 1996, Second Amendment to Loan
and Security Agreement, dated April 16, 1996, Third Amendment to Loan and
Security Agreement, dated May 24, 1996, Fourth Amendment to Loan and Security
Agreement, dated May 31, 1996, Fifth Amendment to Loan and Security Agreement,
dated September 11, 1996, Sixth Amendment to Loan and Security Agreement, dated
as of December 5, 1996, Seventh Amendment to Loan and Security Agreement, dated
as of December 18, 1996, Eighth Amendment to Loan and Security Agreement, dated
as of March 26, 1997, Ninth Amendment to Loan and Security Agreement, dated as
of April 18, 1997, Tenth Amendment to Loan and Security Agreement, dated as of
October 31, 1997, Eleventh Amendment to Loan and Security Agreement, dated as of
March 25, 1998, Twelfth Amendment to Loan and Security Agreement, dated as of
September 30, 1998, Thirteenth Amendment to Loan and Security Agreement, dated
as of September 30, 1998, Fourteenth Amendment to Loan and

<PAGE>

Security Agreement, dated as of February 28, 2000, Fifteenth Amendment to Loan
and Security Agreement, dated as of March 24, 2000, Sixteenth Amendment to Loan
and Security Agreement, dated as of August 8, 2000, and Seventeenth Amendment to
Loan and Security Agreement, dated as of January 5, 2001 (as so amended, the
"Loan Agreement"), pursuant to which Lender has made loans and advances to
Borrowers;

        WHEREAS, Borrowers and Guarantors have requested that Lender waive the
Event of Default arising from the failure of Hanover and its Subsidiaries to
maintain the cumulative minimum amount of EBITDA in accordance with Section
6.31(b) of the Loan Agreement for the cumulative fiscal quarter year to date
period commencing on December 30, 2000 through and including June 30, 2001, and
Lender has agreed to waive that Event of Default, subject to the terms and
conditions contained herein; and

        WHEREAS, the parties to the Loan Agreement desire to enter into this
Eighteenth Amendment to Loan and Security Agreement (this "Amendment") to
evidence and effectuate such waiver and certain amendments to the Financing
Agreements relating thereto, in each case subject to the terms and conditions
and to the extent set forth herein;

        NOW, THEREFORE, in consideration of the premises and covenants set forth
herein and other good and valuable consideration, the receipt and sufficiency of
which is hereby acknowledged, the parties hereto agree as follows:

1.              Definitions.

(a)                     Amendment to Definition of Consolidated Net Income.
Effective as of June 30, 2001, all references to the term "Consolidated Net
Income" in the Loan Agreement and the other Financing Agreements shall be deemed
and each such reference is hereby amended to mean, with respect to any Person
and its Subsidiaries for any period, the aggregate of the net income (loss) of
such Person and its Subsidiaries, on a consolidated basis, for such period
(excluding to the extent included therein any extraordinary or unusual and
non-recurring gains) after deducting all charges which should be deducted before
arriving at the net income (loss) for such period and, without duplication,
after deducting the Provision for Taxes for such period, all as determined in
accordance with GAAP; provided, that, (A) the net income of any Person that is
not a wholly-owned Subsidiary or that is accounted for by the equity method of
accounting shall be included only to the extent of the amount of dividends or
distributions paid or payable to such Person or a wholly-owned Subsidiary of
such Person; (B) except to the extent included pursuant to the foregoing clause,
the net income of any Person accrued prior to the date it becomes a wholly-owned
Subsidiary of such Person or is merged into or consolidated with such Person or
any of its wholly-owned Subsidiaries or that Person's assets are acquired by
such Person or by its wholly-owned Subsidiaries shall be excluded; and (C) the
net income (if positive) of any wholly-owned Subsidiary to the extent that the
declaration or payment of dividends or similar distributions by such
wholly-owned Subsidiary to such Person or to any other wholly-owned Subsidiary
of such Person is not at the time permitted by operation of the terms of its
charter or any agreement, instrument, judgment, decree, order, statute, rule or
governmental regulation applicable to such wholly-

                                       2
<PAGE>

owned Subsidiary shall be excluded. For the purposes of this definition, net
income excludes any gain (or loss) together with any related Provision for Taxes
for such gain (or loss) realized upon the sale or other disposition of any
assets that are not sold in the ordinary course of business (including, without
limitation, Asset Sales and dispositions pursuant to sale and leaseback
transactions) or of any Capital Stock of such Person or a Subsidiary of such
Person and any net income realized or loss incurred as a result of changes in
accounting principles or the application thereof to such Person, except, that,
solely for purposes of calculating the minimum amount of EBITDA during the
fiscal year of Hanover and its Subsidiaries ending December 29, 2001 pursuant to
Section 6.31 of the Loan Agreement, Hanover and its Subsidiaries may include the
net income derived from the sale of the HDPI Real Property permitted by the
letter, dated as of April 30, 2001, by Congress to HDPI and Hanover Re: Sale of
101 Kindig Lane Facility of Hanover Direct Pennsylvania, Inc. and from the sale
of the LWI Assets permitted by the letter, dated as of June 13, 2001, by
Congress to LWI, Keystone Internet, Hanover Brands and Hanover Re: Sale of
Certain Assets of the "Improvements" Mail Order Catalog Business of LWI
Holdings, Inc., the term "Provision for Taxes" shall mean an amount equal to all
taxes imposed on or measured by net income, whether Federal, State, Provincial,
county or local, and whether foreign or domestic, that are paid or payable by
any Person in respect of any period in accordance with GAAP, and the term
"Capital Stock" shall mean, with respect to any Person, any and all shares,
interests, participation or other equivalents (however designated) of such
Person's capital stock at any time outstanding, and any and all rights, warrants
or options exchangeable for or convertible into such capital stock (but
excluding any debt security that is exchangeable for or convertible into such
capital stock).
(b)
(c)                     Interpretation.  All capitalized terms used herein and
not defined herein shall have the meanings given to such terms in the Loan
Agreement.
(d)
2.              Waiver of EBITDA Covenant Default.
3.
(a)                     Lender hereby waives, subject to the terms and
conditions contained in this Amendment, the Event of Default arising under
Section 6.31(b) of the Loan Agreement as a result of the failure of Hanover and
its Subsidiaries to maintain EBITDA in the amounts required thereunder for the
cumulative fiscal year to date period measured from December 30, 2000 through
and including June 30, 2001.
(b)
(c)                     Lender has not waived and is not by this Amendment
waiving, and has no intention of waiving, any other Event of Default, which may
have occurred before the date hereof, or may be continuing on the date hereof or
any Event of Default that may occur after the date hereof, whether the same or
similar to the Event of Default described in Section 2(a) hereof or otherwise,
other than the Event of Default described in Section 2(a) hereof. Lender
reserves the right, in its discretion, to exercise its rights and remedies
arising under the Financing Agreements, applicable law or otherwise as a result
of any other Events of Default that may have occurred before the date hereof, or
are continuing on the date hereof, or any Event of Default that
<PAGE>
may occur after the date hereof, whether the same or similar to the Event of
Default described in Section 2(a) hereof or otherwise.
(d)
4.              EBITDA Loan Availability Reserve.
5.
(a)                     Effective on the date hereof, in addition to any then
existing availability reserves that may be in effect at such time, Lender shall
establish and maintain at all times thereafter an availability reserve against
the amount of Revolving Loans and Letter of Credit Accommodations otherwise
determined by Lender to be available to Revolving Loan Borrowers under the Loan
Agreement and the other Financing Agreements in the amount equal to $500,000.
(b)
(c)                     The availability reserve established in accordance with
this Section 3 is in addition to, and nots in limitation of, all other existing
availability reserves and the rights of Lender from time to time to establish
other and further reserves against the availability of Revolving Loans and
Letter of Credit Accommodations under the Loan Agreement and the other Financing
Agreements.
(d)

6.              Appraisals. Section 6.15 of the Loan Agreement is hereby deleted
and replaced with the following:
7.
8.                      "6.15   Appraisals
9.
                (a)     Borrowers shall, at Borrowers' expense, upon the request
        of Lender, obtain appraisals by the Appraiser of any or all of the
        Inventory of Borrowers, in form, scope and methodology acceptable to
        Lender, and including, but not limited to, a report as to the Net
        Orderly Liquidation Value of the Inventory of Borrowers, other than
        Inventory of Gump's at the Gump's Main Store, and of the Net GOB Value
        of the Inventory of Gump's at the Gump's Main Store.

                        (b)     Upon Lender's request, Borrowers shall, at their
        expense, once in any calendar year during the Term (excluding any such
        reports or appraisals delivered in connection with the closing
        hereunder) and at any time or times as Lender may request on or after an
        Event of Default, deliver or cause to be delivered to Lender written
        reports or appraisals as to their Real Property or Equipment, in form,
        scope and methodology acceptable to Lender and by an appraiser
        acceptable to Lender."

1.              Fees. In addition to all other fees, charges, interest and
expenses payable by Borrowers to Lender under the Loan Agreement and the other
Financing Agreements, Borrowers shall pay to Lender the following additional
fees:
2.
(a)                     Borrowers shall pay to Lender, contemporaneously
herewith, a closing fee in the amount of $375,000, which fee is fully earned as
of the date hereof and may be charged into the loan account(s) of any Borrower.
(b)                     Tranche B Term Loan Borrowers shall pay to Lender,
contemporaneously herewith, a closing fee in the amount of $125,000, which fee
is fully earned as of the date hereof and may be charged into the loan
account(s) of any Tranche B Term Loan Borrower.
(c)

<PAGE>

3.              Representations, Warranties and Covenants. Borrowers and
Guarantors represent, warrant and covenant with and to Lender as follows, which
representations, warranties and covenants are continuing and shall survive the
execution and delivery hereof, the truth and accuracy of, or compliance with
each, together with the representations, warranties and covenants in the other
Financing Agreements, being a condition of the effectiveness of this Amendment
and a continuing condition of the making or providing of any Revolving Loans or
Letter of Credit Accommodations by Lender to Borrowers:
4.
(a)                     This Amendment and each other agreement or instrument to
be executed and delivered by each Borrower or Guarantor hereunder have been duly
authorized, executed and delivered by all necessary action on the part of each
of Borrower and each Guarantor which is a party hereto and thereto and, if
necessary, their respective stockholders (with respect to any corporation) or
members (with respect to any limited liability company), and is in full force
and effect as of the date hereof, as the case may be, and the agreements and
obligations of each Borrower and Guarantor, as the case may be, contained herein
and therein constitute legal, valid and binding obligations of each Borrower and
Guarantor, as the case may be, enforceable against them in accordance with their
terms.
(b)
(c)                     No action of, or filing with, or consent of any
governmental or public body or authority, and no consent of any other party, is
required to authorize, or is otherwise required in connection with, the
execution, delivery and performance of this Amendment and each other agreement
or instrument to be executed and delivered pursuant to this Amendment.
(d)
(i)                     Neither the execution and delivery of this Agreement or
any other agreements, documents or instruments in connection therewith, nor the
consummation of the transactions therein contemplated, nor compliance with the
provisions thereof has violated or shall violate any Federal or State securities
laws or any other law or regulation or any order or decree of any court or
governmental instrumentality in any respect, or does, or shall conflict with or
result in the breach of, or constitute a default in any respect under any
mortgage, deed of trust, security agreement, agreement or instrument to which
any Borrower or Guarantor is a party or may be bound, or does or shall violate
any provision of the Certificate of Incorporation or By-Laws of any Borrower or
Guarantor.
(ii)
(e)                     All of the representations and warranties set forth in
the Loan Agreement as amended hereby, and the other Financing Agreements, are
true and correct in all material respects after giving effect to the provisions
of this Amendment, except to the extent any such representation or warranty is
made as of a specified date, in which case such representation or warranty shall
have been true and correct as of such date.
(f)
(g)                     After giving effect to the provisions of this Amendment,
no Event of Default or Incipient Default exists or has occurred and is
continuing.
(h)
5.              Conditions Precedent. Concurrently with the execution and
delivery hereof (except to the extent otherwise indicated below), and as a
further condition to the effectiveness of this Amendment and the agreement of
Lender to the modifications and amendments set forth in this Amendment:
6.

<PAGE>

(a)             Lender shall have received a photocopy of an executed original
or executed original counterparts of this Amendment by facsimile (with the
originals to be delivered within five (5) days after the date hereof), as the
case may be, duly authorized, executed and delivered by Borrowers and
Guarantors;
(b)
(c)             Lender shall have received, in form and substance satisfactory
to Lender, Secretary's or Assistant Secretary's Certificates of Directors'
Resolutions with Shareholders' Consent evidencing the adoption and subsistence
of corporate resolutions approving the execution, delivery and performance by
Borrowers and Guarantors that are corporations of this Amendment and the
agreements, documents and instruments to be delivered pursuant to this
Amendment; and
(d)
(e)                     each of Borrowers and Guarantors shall deliver, or cause
to be delivered, to Lender a true and correct copy of any consent, waiver or
approval to or of this Amendment, which any Borrower or Guarantor is required to
obtain from any other Person, and such consent, approval or waiver shall be in a
form reasonably acceptable to Lender.
(f)
7.              Effect of this Amendment. This Amendment constitutes the entire
agreement of the parties with respect to the subject matter hereof, and
supersedes all prior oral or written communications, memoranda, proposals,
negotiations, discussions, term sheets and commitments with respect to the
subject matter hereof. Except as expressly provided herein, no other waivers
changes or modifications to the Loan Agreement or any of the other Financing
Agreements, or waivers of or consents under any provisions of any of the
foregoing, are intended or implied by this Amendment, and in all other respects
the Financing Agreements are hereby specifically ratified, restated and
confirmed by all parties hereto as of the effective date hereof. To the extent
that any provision of the Loan Agreement or any of the other Financing
Agreements conflicts with any provision of this Amendment, the provision of this
Amendment shall control.
8.
9.              Further Assurances. Borrowers and Guarantors shall execute and
deliver such additional documents and take such additional action as may be
reasonably requested by Lender to effectuate the provisions and purposes of this
Amendment.
10.
11.             Governing Law. The validity, interpretation and enforcement of
this Amendment in any dispute arising out of the relationship between the
parties hereto, whether in contract, tort, equity or otherwise shall be governed
by the internal laws of the State of New York, without regard to any principle
of conflict of laws or other rule of law that would result in the application of
the law of any jurisdiction other than the State of New York.
12.
13.             Binding Effect. This Amendment shall be binding upon and inure
to the benefit of each of the parties hereto and their respective successors and
assigns.
14.             Counterparts. This Amendment may be executed in any number of
counterparts, but all of such counterparts shall together constitute but one and
the same agreement. In making proof of this Amendment, it shall not be necessary
to produce or account for more than one counterpart thereof signed by each of
the parties hereto.
15.
16.

<PAGE>

                  [REMAINDER OF PAGE INTENTIONALLY LEFT BLANK]


<PAGE>

        IN WITNESS WHEREOF, the parties hereto have caused this Amendment to be
executed on the day and year first written.


                                            CONGRESS FINANCIAL CORPORATION

                                            By: /s/ Janet S. Last

                                            Title: First Vice President


                                            HANOVER DIRECT PENNSYLVANIA, INC.
                                            BRAWN OF CALIFORNIA, INC.
                                            GUMP'S BY MAIL, INC.
                                            GUMP'S CORP.
                                            LWI HOLDINGS, INC.
                                            HANOVER DIRECT VIRGINIA INC.
                                            HANOVER REALTY, INC.
                                            THE COMPANY STORE FACTORY, INC.
                                            THE COMPANY OFFICE, INC.
                                            KEYSTONE INTERNET SERVICES, INC.
                                            TWEEDS, LLC
                                            SILHOUETTES, LLC
                                            HANOVER COMPANY STORE, LLC
                                            DOMESTICATIONS, LLC

                                            By: /s/ Brian C. Harriss
                                                -------------------

                                            Title: Vice President
                                                   ----------------

By their signatures below, the
undersigned Guarantors acknowledge
and agree to be bound by the
applicable provisions of this
Amendment:

HANOVER DIRECT, INC.
By:  /s/ Brian C. Harriss
    --------------------

Title: Executive Vice President and Chief Financial Officer
       ----------------------------------------------------

                       [SIGNATURES CONTINUE ON NEXT PAGE]

<PAGE>

                    [SIGNATURES CONTINUED FROM PREVIOUS PAGE]

AMERICAN DOWN & TEXTILE COMPANY
D.M. ADVERTISING, INC.
SCANDIA DOWN CORPORATION
KEYSTONE LIQUIDATIONS, INC.
HANOVER HOME FASHIONS GROUP, LLC
KITCHEN & HOME, LLC
DOMESTICATIONS KITCHEN & GARDEN, LLC
ENCORE CATALOG, LLC
CLEARANCE WORLD OUTLETS, LLC
SCANDIA DOWN, LLC
ERIZON, INC.
HANOVER BRANDS, INC.
ERIZON.COM, INC.
LA CROSSE FULFILLMENT, LLC
SAN DIEGO TELEMARKETING, LLC

By: /s/ Brian C. Harriss
    --------------------

Title: Vice President
       ------------------