<PAGE>
                                                                     Exhibit 4.1

                       AMENDMENT NO. 2 TO RIGHTS AGREEMENT

         This AMENDMENT NO. 2 (this "Amendment"), dated as of June 25, 2003, is
between The Elder-Beerman Stores Corp., an Ohio corporation (the "Corporation"),
and Norwest Bank Minnesota, N.A., as Rights Agent (the "Rights Agent").

                                    Recitals

         WHEREAS, the Corporation and the Rights Agent are parties to a Rights
Agreement, dated as of December 30, 1997, as amended by Amendment No. 1, dated
November 11, 1998 (the "Rights Agreement");

         WHEREAS, Wright Holdings, Inc., a Delaware corporation ("Parent"),
Wright Sub, Inc., an Ohio corporation ("Sub"), and the Corporation propose to
enter into an Agreement and Plan of Merger (as it may be amended from time to
time pursuant to its terms, the "Merger Agreement"), pursuant to which Sub shall
be merged with and into the Corporation with the Corporation continuing as the
surviving corporation and a wholly-owned subsidiary of Parent (the "Merger");

         WHEREAS, the Board of Directors of the Corporation has approved the
Merger Agreement and the Merger; and

         WHEREAS, pursuant to Section 27 of the Rights Agreement, the Board of
Directors of the Corporation has determined that an amendment to the Rights
Agreement as set forth below is necessary and desirable in connection with the
foregoing, and the Corporation and the Rights Agent desire to evidence such
amendment in writing.

         NOW, THEREFORE, in consideration of the foregoing and the mutual
agreements set forth herein, the parties agree as follows:

         1. AMENDMENT OF SECTION 1(A). Section 1(a) of the Rights Agreement is
amended to add the following sentence at the end of such Section:

                  "Notwithstanding anything in this Agreement to the contrary,
                  neither Parent, Sub nor any Affiliate of either Parent or Sub
                  shall be deemed to be an Acquiring Person by virtue of (i) the
                  consummation of the Merger, (ii) the execution of the Merger
                  Agreement, or (iii) the consummation of the other transactions
                  contemplated in the Merger Agreement, or any or all of the
                  foregoing in combination."

         2. AMENDMENT TO SECTION 1(I). Section 1(i) of the Rights Agreements is
amended by striking the phrase:

                  "and (iii) the time at which all exercisable Rights are
                  exchanged as provided in Section 24."

at the end of such Section and substituting the phrase:

                  "and (iii) the time at which all exercisable Rights are
                  exchanged as provided in Section 24 hereof, or (iv) the
                  consummation of the Merger."

         3. AMENDMENT OF SECTION 1(CC). Section 1(cc) of the Rights Agreement is
amended to add the following sentence at the end of such Section:


<PAGE>
                  "Notwithstanding anything in this Agreement to the contrary, a
Triggering Event shall not be deemed to have occurred as the result of (i) the
consummation of the Merger, (ii) the execution of the Merger Agreement, or (iii)
the consummation of the other transactions contemplated in the Merger Agreement,
or any or all of the foregoing in combination."

         4. AMENDMENT OF SECTION 1. Section 1 of the Rights Agreement is amended
to insert the following additional definitions to the end of such Section:

                  (dd) "Merger" shall have the meaning set forth in the Merger
                  Agreement.

                  (ee) "Merger Agreement" shall mean that certain Agreement and
                  Plan of Merger, dated as of June 25, 2003, by and between
                  Parent, Sub and the Corporation, as amended from time to time
                  pursuant to its terms.

                  (ff) "Parent" shall mean Wright Holdings, Inc., a Delaware
                  corporation.

                  (gg)  "Sub" shall mean Wright Sub,Inc., an Ohio corporation.

         5. AMENDMENT OF SECTION 3(C). Section 3(c) of the Rights Agreement is
amended to add the following sentence at the end of such Section:

                  "Notwithstanding anything in this Agreement to the contrary, a
                  Distribution Date shall not be deemed to have occurred as the
                  result of (i) the consummation of the Merger, (ii) the
                  execution of the Merger Agreement, or (iii) the consummation
                  of the other transactions contemplated in the Merger
                  Agreement, or any or all of the foregoing in combination."

         6. AMENDMENT OF SECTION 11(A)(II). Section 11(a)(ii) of the Rights
Agreement is amended to add the following sentence at the end of such Section:

                  "Furthermore, notwithstanding anything in this Agreement to
                  the contrary, (i) the consummation of the Merger, (ii) the
                  execution of the Merger Agreement, and (iii) the consummation
                  of the other transactions contemplated in the Merger
                  Agreement, separately or in combination, shall not be deemed
                  to be any of the events of the type described in this Section
                  11(a)(ii) and shall not cause the Purchase Price, the number
                  and kind of securities issuable upon exercise of each Right or
                  the number of Rights outstanding to be adjusted, or any other
                  action to be taken or right, obligation or change herein to
                  arise pursuant to this Section 11(a)(ii)."

         7. AMENDMENT OF SECTION 13(A). Section 13(a) of the Rights Agreement is
amended to add the following sentence at the end of such Section:

                  "Furthermore, notwithstanding anything in this Agreement to
                  the contrary, (i) the consummation of the Merger, (ii) the
                  execution of the Merger Agreement, and (iii) the consummation
                  of the other transactions contemplated in the Merger
                  Agreement, separately or in combination, shall not be deemed
                  to be any of the events of the type described in this Section
                  13(a) and shall not cause the Purchase Price, the number and
                  kind of securities issuable upon exercise of each Right or the
                  number of Rights outstanding to be adjusted, or any other
                  action to be taken or right, obligation or change herein to
                  arise pursuant to this Section 13(a)."

         8. EFFECTIVENESS. This Amendment shall be deemed effective as of the
date first written above, as if executed on such date. Except as amended hereby,
the Rights Agreement shall remain in full force and effect and shall be
otherwise unaffected by this Amendment.


                                       -2-
<PAGE>
         9. MISCELLANEOUS. This Amendment shall be deemed to be a contract made
under the laws of the State of Ohio and for all purposes shall be governed by
and construed in accordance with the laws of such state applicable to contracts
to be made and performed entirely within such state. This Amendment may be
executed in any number of counterparts, each of such counterparts shall for all
purposes be deemed to be an original, and all such counterparts shall together
constitute but one and the same instrument. If any provision, covenant or
restriction of this Amendment is held by a court of competent jurisdiction or
other authority to be invalid, illegal or unenforceable, the remainder of the
terms, provisions, covenants and restrictions of this Amendment shall remain in
full force and effect and shall in no way be effected, impaired or invalidated.

         IN WITNESS WHEREOF, the parties hereto have caused this Amendment to be
fully executed, all as of the date and year first above written.


                                      THE ELDER-BEERMAN STORES CORP.


                                      /s/ Byron L. Bergren
                                      ------------------------------------------
                                      By: Byron L. Bergren
                                      Its: President and Chief Executive Officer


                                      NORWEST BANK MINNESOTA, N.A.
                                      as Rights Agent


                                      /s/ Jennifer Leno
                                      ------------------------------------------
                                      By: Jennifer Leno
                                      Its: Officer




                                       -3-