<PAGE>

                                                                    Exhibit 10.3

                                                             [Execution Version]

             TWENTY-EIGHTH AMENDMENT TO LOAN AND SECURITY AGREEMENT

         THIS TWENTY-EIGHTH AMENDMENT TO LOAN AND SECURITY AGREEMENT (this
"Amendment"), dated as of November 4, 2003, is entered into by and among
CONGRESS FINANCIAL CORPORATION, a Delaware corporation ("Lender"), BRAWN OF
CALIFORNIA, INC., a California corporation ("Brawn"), GUMP'S BY MAIL, INC., a
Delaware corporation ("GBM"), GUMP'S CORP., a California corporation ("Gump's"),
HANOVER REALTY, INC., a Virginia corporation ("Hanover Realty"), THE COMPANY
STORE FACTORY, INC., a Delaware corporation ("TCS Factory"), THE COMPANY OFFICE,
INC., a Delaware corporation ("TCS Office"), SILHOUETTES, LLC, a Delaware
limited liability company ("Silhouettes LLC"), HANOVER COMPANY STORE, LLC, a
Delaware limited liability company ("HCS LLC"), DOMESTICATIONS, LLC, a Delaware
limited liability company ("Domestications LLC"), KEYSTONE INTERNET SERVICES,
LLC, a Delaware limited liability company ("KIS LLC"), and THE COMPANY STORE
GROUP, LLC, a Delaware limited liability company ("CSG LLC"; and, together with
Brawn, GBM, Gump's, Hanover Realty, TCS Factory, TCS Office, Silhouettes LLC,
HCS LLC, Domestications LLC and KIS LLC, collectively, "Borrowers" and each,
individually, a "Borrower"), HANOVER DIRECT, INC., a Delaware corporation
("Hanover"), HANOVER HOME FASHIONS GROUP, LLC, a Delaware limited liability
company ("HHFG LLC"), CLEARANCE WORLD OUTLETS, LLC, a Delaware limited liability
company ("Clearance World"), SCANDIA DOWN, LLC, a Delaware limited liability
company ("Scandia Down LLC"), LACROSSE FULFILLMENT, LLC, a Delaware limited
liability company ("LaCrosse LLC"), D.M. ADVERTISING, LLC, a Delaware limited
liability company ("DM Advertising LLC"), AMERICAN DOWN & TEXTILE, LLC, a
Delaware limited liability company ("ADT LLC"), and HANOVER GIFTS, INC., a
Virginia corporation ("Hanover Gifts"; and, together with Hanover, HHFG LLC,
Clearance World, Scandia Down LLC, LaCrosse LLC, DM Advertising LLC and ADT LLC,
collectively, "Guarantors" and each, individually, a "Guarantor").

                              W I T N E S S E T H:

         WHEREAS, Borrowers, Guarantors and Lender are parties to the Loan and
Security Agreement, dated November 14, 1995, as amended by the First Amendment
to Loan and Security Agreement, dated February 22, 1996, the Second Amendment to
Loan and Security Agreement, dated April 16, 1996, the Third Amendment to Loan
and Security Agreement, dated May 24, 1996, the Fourth Amendment to Loan and
Security Agreement, dated May 31, 1996, the Fifth Amendment to Loan and Security
Agreement, dated September 11, 1996, the Sixth Amendment to Loan and Security
Agreement, dated as of December 5, 1996, the Seventh Amendment to Loan and
Security Agreement, dated as of December 18, 1996, the Eighth Amendment to Loan
and Security Agreement, dated as of March 26, 1997, the Ninth Amendment to Loan
and Security Agreement, dated as of April 18, 1997, the Tenth Amendment to Loan
and Security Agreement, dated as of October 31, 1997, the Eleventh Amendment to
Loan and Security

<PAGE>

Agreement, dated as of March 25, 1998, the Twelfth Amendment to Loan and
Security Agreement, dated as of September 30, 1998, the Thirteenth Amendment to
Loan and Security Agreement, dated as of September 30, 1998, the Fourteenth
Amendment to Loan and Security Agreement, dated as of February 28, 2000, the
Fifteenth Amendment to Loan and Security Agreement, dated as of March 24, 2000,
the Sixteenth Amendment to Loan and Security Agreement, dated as of August 8,
2000, the Seventeenth Amendment to Loan and Security Agreement, dated as of
January 5, 2001, the Eighteenth Amendment to Loan and Security Agreement, dated
as of November 12, 2001, the Nineteenth Amendment to Loan and Security
Agreement, dated as of December 18, 2001, the Twentieth Amendment to Loan and
Security Agreement, dated as of March 5, 2002, the Twenty-First Amendment to
Loan and Security Agreement, dated as of March 21, 2002, the Twenty-Second
Amendment to Loan and Security Agreement, dated as of August 16, 2002, the
Twenty-Third Amendment to Loan and Security Agreement, dated as of December 27,
2002, the Twenty-Fourth Amendment to Loan and Security Agreement, dated as of
February 27, 2003, the Twenty-Fifth Amendment to Loan and Security Agreement,
dated as of April 21, 2003, the Twenty-Sixth Amendment to Loan and Security
Agreement, dated as of August 29, 2003, and the Twenty-Seventh Amendment to Loan
and Security Agreement, dated as of October 31, 2003 (as so amended, the "Loan
Agreement"), pursuant to which Lender has made loans and advances to Borrowers;

         WHEREAS, Borrowers and Guarantors have requested that Lender amend the
definitions of Consolidated Net Worth and Consolidated Working Capital to
account for the requirement to write off certain deferred tax assets and
deferred tax liabilities of Hanover not to exceed the net amount of $11,300,000;
and

         WHEREAS, the parties hereto desire to enter into this Amendment to
evidence and effectuate such amendments, subject to the terms and conditions and
to the extent set forth herein;

         NOW, THEREFORE, in consideration of the premises and covenants set
forth herein and other good and valuable consideration, the receipt and
sufficiency of which is hereby acknowledged, the parties hereto agree as
follows:

         1. Definitions.

                  (a) Amendments to Definitions

                           (i) Consolidated Net Worth. All references to the
term "Consolidated Net Worth" in the Loan Agreement and the other Financing
Agreements shall be deemed and each such reference is hereby amended by deleting
the word "and" after proviso (e), adding the word "and" after proviso (f), and
adding the following new proviso (g) as follows:

                           "and (g) solely for purposes of calculating
                  Consolidated Net Worth of Hanover and its Subsidiaries during
                  Hanover's fiscal year ending December 27, 2003, Hanover's net
                  deferred tax assets in the amount of $11,300,000 (consisting
                  of a non-current deferred tax asset in the amount of
                  $13,600,000 and a current deferred tax liability of
                  $2,300,000) that are required to be written off pursuant to
                  Financial

                                     - 2 -
<PAGE>


                  Accounting Standards No. 109 shall be added back for purposes
                  of determining the net amount of assets of Hanover and its
                  Subsidiaries."

                           (ii) Consolidated Working Capital. All references to
the term "Consolidated Working Capital" in the Loan Agreement and the other
Financing Agreements shall be deemed and each such reference is hereby amended
by adding the following new proviso prior to the period at the end of such
definition as follows:

                           "; provided, further, that, solely for purposes of
                  calculating Consolidated Working Capital of Hanover and its
                  Subsidiaries during Hanover's fiscal year ending December 27,
                  2003, current deferred tax liabilities of Hanover in the
                  amount of $2,300,000 that are required to be written off
                  pursuant to Financial Accounting Standards No. 109, shall be
                  added back for purposes of determining the amount of current
                  liabilities of Hanover and its Subsidiaries."

                  (b) Interpretation. All capitalized terms used herein and not
defined herein shall have the meanings given to such terms in the Loan
Agreement.

         2. Representations, Warranties and Covenants. Each Borrower and
Guarantor represents, warrants and covenants with, to and in favor of Lender as
follows, which representations, warranties and covenants are continuing and
shall survive the execution and delivery hereof, the truth and accuracy of, or
compliance with, each, together with the representations, warranties and
covenants in the other Financing Agreements, being a condition of the
effectiveness of this Amendment and a continuing condition of the making or
providing of any Revolving Loans or Letter of Credit Accommodations by Lender to
Borrowers:

                  (a) This Amendment and each other agreement or instrument to
be executed and delivered by Borrowers or Guarantors hereunder have been duly
authorized, executed and delivered by all necessary action on the part of
Borrowers and Guarantors which are a party hereto and thereto and, if necessary,
their respective stockholders (with respect to any corporation) or members (with
respect to any limited liability company), and is in full force and effect as of
the date hereof, as the case may be, and the agreements and obligations of
Borrowers or Guarantors, as the case may be, contained herein and therein
constitute legal, valid and binding obligations of Borrowers and Guarantors, as
the case may be, enforceable against them in accordance with their terms.

                  (b) No action of, or filing with, or consent of any
governmental or public body or authority, and no approval or consent of any
other party, including, without limitation, Richemont or Chelsey, is required to
authorize, or is otherwise required in connection with, the execution, delivery
and performance of this Amendment.

                  (c) All of the representations and warranties set forth in the
Loan Agreement, as amended hereby, and the other Financing Agreements are true
and correct in all material respects, after giving effect to the provisions of
this Amendment, except to the extent any such representation or warranty is made
as of a specified date, in which case such representation or warranty shall have
been true and correct as of such date.

                                     - 3 -
<PAGE>

                  (d) After giving effect to the provisions of this Amendment,
no Event of Default or Incipient Default exists or has occurred and is
continuing.

         3. Conditions Precedent. Concurrently with the execution and delivery
hereof (except to the extent otherwise indicated below), and as a further
condition to the effectiveness of this Amendment and the agreement of Lender to
the modifications and amendments set forth in this Amendment:

                  (a) Lender shall have received, in form and substance
satisfactory to Lender, a photocopy of an executed original or executed original
counterparts of this Amendment by facsimile (with the originals to be delivered
within five (5) Banking Days after the date hereof), as the case may be, duly
authorized, executed and delivered by each Borrower and Guarantor; and

                  (b) each Borrower and Guarantor shall deliver, or cause to be
delivered, to Lender a true and correct copy of any consent, waiver or approval
to or of this Amendment, which any Borrower or Guarantor is required to obtain
from any other Person, including, without limitation, Richemont or Chelsey, and
such consent, approval or waiver shall be in a form reasonably acceptable to
Lender.

         4. Effect of this Amendment. This Amendment constitutes the entire
agreement of the parties with respect to the subject matter hereof, and
supersedes all prior oral or written communications, memoranda, proposals,
negotiations, discussions, term sheets and commitments with respect to the
subject matter hereof. Except as expressly provided herein, no other changes or
modifications to the Loan Agreement or any of the other Financing Agreements, or
waivers of or consents under any provisions of any of the foregoing, are
intended or implied by this Amendment, and in all other respects the Financing
Agreements are hereby specifically ratified, restated and confirmed by all
parties hereto as of the effective date of this Amendment. To the extent that
any provision of the Loan Agreement or any of the other Financing Agreements
conflicts with any provision of this Amendment, the provision of this Amendment
shall control.

         5. Further Assurances. Borrowers and Guarantors shall execute and
deliver such additional documents and take such additional action as may be
reasonably requested by Lender to effectuate the provisions and purposes of this
Amendment.

         6. Governing Law. The validity, interpretation and enforcement of this
Amendment in any dispute arising out of the relationship between the parties
hereto, whether in contract, tort, equity or otherwise, shall be governed by the
internal laws of the State of New York, without regard to any principle of
conflict of laws or other rule of law that would result in the application of
the law of any jurisdiction other than the State of New York.

         7. Binding Effect. This Amendment shall be binding upon and inure to
the benefit of each of the parties hereto and their respective successors and
assigns.

         8. Counterparts. This Amendment may be executed in any number of
counterparts, but all of such counterparts shall together constitute but one and
the same agreement. In making

                                     - 4 -
<PAGE>

proof of this Amendment, it shall not be necessary to produce or account for
more than one counterpart thereof signed by each of the parties hereto.

<PAGE>

         IN WITNESS WHEREOF, the parties hereto have caused this Amendment to be
executed on the day and year first written.

                                  CONGRESS FINANCIAL CORPORATION

                                  By:    /s/ Eric J. Storz
                                     -------------------------------------

                                  Title: AVP

                                  BRAWN OF CALIFORNIA, INC.

                                  By:    /s/ Steven Seymour
                                     -------------------------------------
                                  Name:  Steven Seymour
                                  Title: President

                                  GUMP'S BY MAIL, INC.

                                  By:    /s/ Jed Pogran
                                     -------------------------------------
                                  Name:  Jed Pogran
                                  Title: President

                                  GUMP'S CORP.

                                  By:    /s/ Jed Pogran
                                     -------------------------------------
                                  Name:  Jed Pogran
                                  Title: President

                                  HANOVER REALTY, INC.

                                  By:    /s/ Doug Mitchell
                                     -------------------------------------
                                  Name:  Doug Mitchell
                                  Title: President

                                  THE COMPANY STORE FACTORY, INC.

                                  By:    /s/ David Pipkorn
                                     -------------------------------------
                                  Name:  David Pipkorn
                                  Title: President

                     [SIGNATURES CONTINUE ON FOLLOWING PAGE]

<PAGE>

                    [SIGNATURES CONTINUED FROM PREVIOUS PAGE]

                                  THE COMPANY OFFICE, INC.

                                  By:    /s/ David Pipkorn
                                     -------------------------------------
                                  Name:  David Pipkorn
                                  Title: President

                                  SILHOUETTES, LLC

                                  By:    /s/ Edward M. Lambert
                                     -------------------------------------
                                  Name:  Edward M. Lambert
                                  Title: President

                                  HANOVER COMPANY STORE, LLC

                                  By:    /s/ Edward M. Lambert
                                     -------------------------------------
                                  Name:  Edward M. Lambert
                                  Title: President

                                  DOMESTICATIONS, LLC

                                  By:    /s/ Edward M. Lambert
                                     -------------------------------------
                                  Name:  Edward M. Lambert
                                  Title: Vice President

                                  KEYSTONE INTERNET SERVICES, LLC

                                  By:    /s/ Edward M. Lambert
                                     -------------------------------------
                                  Name:  Edward M. Lambert
                                  Title: Vice President

                                  THE COMPANY STORE GROUP, LLC

                                  By:    /s/ Edward M. Lambert
                                     -------------------------------------
                                  Name:  Edward M. Lambert
                                  Title: President

                     [SIGNATURES CONTINUE ON FOLLOWING PAGE]

<PAGE>

                    [SIGNATURES CONTINUED FROM PREVIOUS PAGE]

By their signatures below, the
undersigned Guarantors acknowledge
and agree to be bound by the
applicable provisions of this
Amendment:

HANOVER DIRECT, INC.

By:    /s/ Edward M. Lambert
   --------------------------------
Name:  Edward M. Lambert
Title: Executive Vice President and
       Chief Financial Officer

HANOVER HOME FASHIONS GROUP, LLC

By:    /s/ Edward M. Lambert
   --------------------------------
Name:  Edward M. Lambert
Title: Vice President

CLEARANCE WORLD OUTLETS, LLC

By:    /s/ Edward M. Lambert
   --------------------------------
Name:  Edward M. Lambert
Title: President

SCANDIA DOWN, LLC

By:    /s/ David Pipkorn
   --------------------------------
Name:  David Pipkorn
Title: President

LA CROSSE FULFILLMENT, LLC

By:    /s/ Edward M. Lambert
   --------------------------------
Name:  Edward M. Lambert
Title: President

                     [SIGNATURES CONTINUE ON FOLLOWING PAGE]

<PAGE>

                    [SIGNATURES CONTINUED FROM PREVIOUS PAGE]

D.M. ADVERTISING, LLC

By:    /s/ Edward M. Lambert
   --------------------------------
Name:  Edward M. Lambert
Title: President

AMERICAN DOWN & TEXTILE, LLC

By:    /s/ David Pipkorn
   --------------------------------
Name:  David Pipkorn
Title: President

HANOVER GIFTS, INC.

By:    /s/ Doug Mitchell
   --------------------------------
Name:  Doug Mitchell
Title: President