<PAGE>   1

                                                                Exhibit 10.4

                SECOND AMENDMENT TO ACCOUNT PURCHASE AGREEMENT

     AMENDMENT, made and entered into as of June 1, 1995, by and among HANOVER
DIRECT, INC. ("HDI"), a Delaware corporation and the successor-in-interest to
The Hanover Companies, HANOVER DIRECT PENNSYLVANIA, INC. ("HDPI"), a
Pennsylvania corporation formerly known as Hanover Direct, Inc. and Hanover
Direct Fulfillment, Inc., BRAWN OF CALIFORNIA, INC. ("Brawn"), a California
corporation, GUMP'S BY MAIL, INC. ("Gump's By Mail"), a Delaware corporation,
GUMP'S CORP. ("GUMP'S"), a California corporation formerly known as GSF
Acquisition Corp., and GUMP'S HOLDINGS, INC. ("Gump's Holdings"), a Delaware
corporation (HDI, HDPI, Brawn, Gump's By Mail, Gump's and Gump's Holdings
being hereinafter collectively and individually referred to as "Hanover") and
GENERAL ELECTRIC CAPITAL CORPORATION ("GE Capital"), a New York corporation.

                             W I T N E S S E T H:

     WHEREAS, Hanover and GE Capital are parties to an Account Purchase
Agreement dated as of December 21, 1992, as amended by an amendment dated as
of July 12, 1993 and as amended, restated and renamed as of April 25, 1994 and
as further amended as of November 2, 1994 (collectively the "Purchase
Agreement");

     WHEREAS, it is the mutual desire of Hanover and GE Capital that the
Purchase Agreement be amended in accordance with the terms and conditions
hereinafter set forth;

     NOW, THEREFORE, in consideration of the mutual promises and subject to
the terms and conditions hereinafter set forth, the parties hereto hereby
agree as follows:

     1.   Capitalized terms used herein which are not otherwise defined shall
have the same meaning as in the Purchase Agreement.

     2.   All references to (i) "Hanover Companies" throughout the Purchase
Agreement shall be omitted and replaced with "HDI", (ii) "HDFI" and "Hanover
Direct" throughout the Purchase Agreement shall be omitted and replaced with
"HDPI", and (iii) "GSF" throughout the Purchase Agreement shall be omitted and
replaced with "GUMP'S".

     3.   The following definition shall be added after the definition of
"Collection Policy" in Section 1:

          "Commercial Paper Rate" shall mean the rate of so-called sixty day
high-grade unsecured notes sold through dealers by major corporations in
multiples of one thousand dollars
<PAGE>   2
($1,000) as published by The Wall Street Journal in its "Money Rates" section
under the heading "Commercial Paper" (or if such publication is discontinued,
such other publication of similar type designated by GE Capital), on any
Business Day that such rate is published, whether or not such rate is actually
charged or paid by an entity.

     4.   The following definition shall be added after the definition of
"Agreement" in Section 1:

          "Average Commercial Paper Rate" shall mean, for any period, the sum
of the Commercial Paper Rates for each Business Day on which such rate is
published during such period, divided by the number of Business Days on which
such rate is published during such period.

     5.   The following definition shall be added after the definition of
"Ineligible Indebtedness" in Section 1:

          "Insurance Program" shall mean any program which may be offered
through GE Capital pursuant to Section 2.10 under which GE Capital or any
insurance company or other third party makes available insurance coverage to
Account Debtors.

     6.   The definition of "Termination Date" in Section 1 is deleted and
substituted in its place is the following:

          "Termination Date" shall mean the last day of the Settlement Period
immediately after December 31, 2000 or after any subsequent fifth anniversary
thereafter, as the case may be, upon extension of the term of the arrangements
contemplated hereby as provided in Section 12.1 hereof.

     7.   The following definition shall be added after the definition of
"Total Equity Investment" in Section 1:

          "Value-Added Program" shall mean any products or services which may
be offered by or through GE Capital to Account Debtors pursuant to Section
2.11 that enhance the features of the Accounts including, without limiting the
foregoing, credit card protection plans, legal services, auto clubs and
extended warranties; provided, however, that the term shall not be deemed to
include credit insurance or any offerings falling within the definition of
"Insurance Programs."

     8.   The first sentence of Section 2.3 shall be deleted, and the
following shall be added at the beginning of such Section:




















                                      -2-
<PAGE>   3
          2.3.  Unit Fee.  On each Settlement Date (including each Settlement
Date during any renewal term), Hanover shall pay GE Capital a Unit Fee of
$2.168 for each Account that is an Active Account during the immediately
preceding Settlement Period.

     9.   Section 2.3 (a) is deleted and substituted in its place is the
following:

               (a)  For each Settlement Period, the Unit Fee shall be
increased or decreased, respectively, by adding an amount equal to .9075 cents
($.009075) for each dollar by which the Average Net Account Balance exceeds
$132.00, or by subtracting an amount equal to .9075 cents ($.009075) for each
dollar by which the Average Net Account Balance is less than $132.00.

     10.  The Section Title and the first sentence of Section 2.4 (a) (i)
shall be deleted, and the following shall be added at the beginning of such
Section:

          2.4.  Service Fee Adjustment.

               (a) (i)  During the first thirty (30) Settlement Periods after
execution of the Existing Agreement, if the Prime Rate in effect on the last
Business Day of any Settlement Period exceeds 6.0%, Hanover shall be charged
an additional fee calculated by multiplying the Aggregate Investment in such
Settlement Period (to the extent that such Aggregate Investment exceeds the
Funding Date Aggregate Investment) by 74.8% of such excess in the Prime Rate
over 6.0%, and dividing the product by 12.

     11.  Section 2.4 (b) is deleted, and substituted in its place is the
following:

               (b)  After the first thirty (30) Settlement Periods through and
including the thirty-sixth Settlement Period after execution of the Existing
Agreement, Hanover shall be charged each Settlement Period an additional fee
equal to (i) the amount by which the product of the Average Commercial Paper
Rate (calculated as of the end of such Settlement Period) and ninety-four
percent (94%) exceeds 5.04%, multiplied by (ii) the Aggregate Investment in
such Settlement Period (to the extent that such Aggregate Investment exceeds
the Funding Date Aggregate Investment) and (iii) divided by twelve (12).  Such
amount shall be payable on the next Settlement Date.  GE Capital shall credit
to Hanover each Settlement Period an amount equal to (i) the amount by which
5.04% exceeds the product of the Average Commercial Paper Rate (calculated as
of the end of such Settlement Period) and ninety-four percent (94%),
multiplied by (ii) the Aggregate Investment in such Settlement Period (to the
extent that such Aggregate Investment exceeds the Funding Date Aggregate
Investment) and (iii) divided by twelve (12).  Any amounts so due and owing to
Hanover shall be deducted from the Unit Fee due and owing to GE Capital on the
next Settlement Date.















                                      -3-
<PAGE>   4
     12.  The following new Subsection 2.4 (c) shall be added:

               (c)  After the first thirty-six (36) Settlement Periods after
the execution of the Existing Agreement, Hanover shall be charged each
Settlement Period an additional fee equal to (i) the amount by which the
product of the Average Commercial Paper Rate (calculated as of the end of such
Settlement Period) and ninety-four percent (94%) exceeds 5.04%, multiplied by
(ii) the Aggregate Investment in such Settlement Period and (iii) divided by
twelve (12).  Such amount shall be payable on the next Settlement Date.  GE
Capital shall credit to Hanover each Settlement Period an amount equal to (i)
the amount by which 5.04% exceeds the product of the Average Commercial Paper
Rate (calculated as of the end of such Settlement Period) and ninety-four
percent (94%), multiplied by (ii) the Aggregate Investment in such Settlement
Period and (iii) divided by twelve (12).  Any amounts so due and owing to
Hanover shall be deducted from the Unit Fee due and owing to GE Capital on the
next Settlement Date.

     13.  Section 2.10 is deleted, and substituted in its place is the
following:

          2.10.  Insurance Solicitation of Accounts.  GE Capital, or its
agents, may solicit Account Debtors for Insurance Programs.  GE Capital and
Hanover will, through solicitations, mailings and other advertising media,
jointly develop and cooperate with each other in the offering of such
Insurance Programs.  These Insurance Program offerings may be made in the form
of Credit Agreement application enrollments, direct mail or statement inserts.
GE Capital shall pay Hanover an expense reimbursement allowance (which
reimbursement may include but not be limited to direct out-of-pocket expenses)
for enrolling Account Debtors in any such Insurance Program.  The expense
reimbursement allowance will be paid once on each insured Account in the
amount of one dollar ($1.00) for each Account on which the Account Debtor
elects insurance.  At Hanover's option and subsequent to the date which
Hanover provides evidence of proper insurance licensing either on Hanover's
part or on behalf of one of its affiliated companies, GE Capital shall pay
Hanover forty percent (40%) of GE Capital's net annual income (exclusive of
any insurance proceeds) derived from such Insurance Programs, to the extent
allowable under applicable law.

     14.  The following new Section 2.11 shall be added:

          2.11  Value-Added Solicitation of Account.  GE Capital, or its
agents, may solicit Account Debtors for Value-Added Programs.  GE Capital
shall pay Hanover forty percent (40%) of the net annual income derived from
such Value-Added Programs.

     15.  The first sentence of Section 3.4 shall be deleted, and the
following shall be added at the beginning of such Section:

          3.4  Promotion.














                                      -4-
<PAGE>   5
               (a)  Hanover agrees to actively promote the credit program and
will promote the purchase of merchandise on credit, including without
limitation in Hanover's mail-order catalogues, to create Accounts for
submission to, or establishment and/or addition by, GE Capital, all
substantially in accordance with its past practices.

     16.  The following new Subsections 3.4 (b) and (c) shall be added:

               (b)  Beginning June 1, 1995 and fifteen (15) days after the end
of each Settlement Period, provided there exists no Event of Default on the
part of Hanover on any such date and neither party has delivered a notice of
termination in accordance with Section 12 hereof as of such date, GE Capital
shall contribute an amount equal to one-twelfth (1/12) of the product of the
Aggregate Investment for the prior Settlement Period, multiplied by .005.  The
amounts contributed pursuant to the immediately preceding sentence shall be
known as the "Promotion Fund".

               (c)  The costs of all mutually agreed upon marketing promotions
shall be paid on a matching basis, whereby Hanover shall direct GE Capital to
pay one-half (1/2) of the costs of any such promotions from the Promotion Fund
and Hanover shall pay directly one-half (1/2) of such costs.  Hanover shall
have the right to use the Promotion Fund on a matching basis until it is fully
expended and any unused portion can be carried over from year to year during
the term of this Purchase Agreement; provided, however, that it is the
intention of Hanover and GE Capital to fully utilize the Promotion Fund during
each year of the term of this Purchase Agreement; and, provided, further, that
amounts not so utilized in the calendar year after the year in which such
amounts were contributed shall thereafter be forfeited by Hanover.

     17.  The last sentence of Section 6.2 shall be deleted, and the following
shall be added at the end of the Section:

               During each Settlement Period thereafter the Required Reserve
has been or shall be adjusted as described in Sections 6.4 and 6.5 below to an
amount equal to the sum of:  (i) 10% of Eligible Indebtedness as of the
relevant Billing Dates, plus (ii) 100% of Ineligible Indebtedness as of such
Billing Dates.

     18.  Subsection 12.1 (a) is deleted, and substituted in its place is the
following:

               (a) Subject to the provisions of subsection (b) below and
Section 13 hereof, this Agreement shall be in effect until the Termination
Date and shall automatically renew itself for successive five-year terms
thereafter unless either party terminates such arrangement at the end of any
term by giving the other party written notice of such termination at least one
hundred eighty (180) days prior thereto, in which event termination of this
Agreement















                                      -5-
<PAGE>   6
shall be effective as of the last day of the Settlement Period ending
immediately after the Termination Date.

     19.  The last sentence of Section 12.2 shall be deleted, and the
following shall be added at the end of the Section:

               Without in any way limiting the generality of the foregoing,
upon such termination, (a) Hanover shall continue to be liable for (i) the
Unit Fee and other fees provided for in Sections 2.3 and 2.4 hereof and for
(ii) RPR Indebtedness to the extent specified in Section 2 5 hereof, (b) the
Unit Fee under 2.3 shall in no event be less than $2.168 plus adjustments for
postage and delinquency under Section 2.3(b) and (c), until such time as there
is no Indebtedness owned by GE Capital pursuant to Section 12.2 hereof,
whichever shall occur earlier, and (c) GE Capital shall continue to perform
its obligations under this Agreement, other than those set forth in Section
2.1 hereof and related functions including, but not limited to, new account
processing and promotions.

     20.  Each party shall pay its own out-of-pocket legal expenses in
connection with the execution and delivery of this Amendment and the closing
of the transactions relating thereto.

     21.  Except as specifically provided herein, the terms and conditions of
the Purchase Agreement shall continue in full force and effect and shall be
fully binding on the parties hereto.  Upon execution of this Amendment, each
reference in the Purchase Agreement to "this Purchase Agreement," "hereunder,"
"hereof," or words of like import, shall mean and be a reference to the
Purchase Agreement as amended hereby.  In the event of any conflict between
the terms of the Purchase Agreement and the terms of this Amendment, the terms
of this Amendment shall prevail.

     22.  This Amendment may be executed in any number of counterparts, all of
which taken together shall constitute one and the same amendatory instrument,
and any of the parties hereto may execute this Amendment by signing any such
counterpart.

     IN WITNESS WHEREOF, this Amendment has been duly executed as of the day
and year first above written.


                              HANOVER DIRECT, INC.


                              By: /s/ Edward J. O'Brien
                                 -----------------------------------------

                                   Name:   Edward J. O'Brien
                                           -------------------------------










                                      -6-
<PAGE>   7
                                   Title:  Vice President
                                           -------------------------------

                              HANOVER DIRECT PENNSYLVANIA, INC.


                              By:  /s/ Edward J. O'Brien
                                 -----------------------------------------

                                   Name:   Edward J. O'Brien
                                          --------------------------------
                                   Title:  Vice President
                                          --------------------------------

                              BRAWN OF CALIFORNIA

                              By:  /s/ Edward J. O'Brien
                                 -----------------------------------------
                                
                                   Name:   Edward J. O'Brien
                                          --------------------------------
                                   Title:  Vice President
                                          --------------------------------

                              GUMP'S BY MAIL, INC.

                              By:  /s/ Edward J. O'Brien
                                 -----------------------------------------

                                   Name:   Edward J. O'Brien
                                          --------------------------------
                                   Title:  Vice President
                                          --------------------------------

                              GUMP'S CORP.

                              By:  /s/ Edward J. O'Brien
                                 -----------------------------------------

                                   Name:   Edward J. O'Brien
                                         ---------------------------------
                                   Title:  Vice President
                                          --------------------------------


































                                      -7-
<PAGE>   8
                              GUMP'S HOLDINGS, INC.

                              By:  /s/ Edward J. O'Brien
                                 -----------------------------------------
                                   Name:   Edward J. O'Brien
                                         ---------------------------------
                                   Title:  Vice President
                                          --------------------------------

                              GENERAL ELECTRIC CAPITAL CORPORATION

                              By:  /s/ Richard C. McKenzie
                                 -----------------------------------------

                                   Name:   Richard C. McKenzie
                                         ---------------------------------
                                   Title:  Manager
                                           -------------------------------




















































                                      -8-