<PAGE>
 
                                                                 EXHIBIT NO. 4.1

                      AMENDMENT NO. 4 TO RIGHTS AGREEMENT


     This Amendment No. 4 dated May 14, 1999 hereby amends the Rights Agreement
originally dated as of October 13, 1995, as amended on June 29, 1998, August 18,
1998 and September 3, 1998 (the "Agreement"), between Arch Communications Group,
Inc., a Delaware corporation (the "Company"), and The Bank of New York, a
national banking association, as Rights Agent (the "Rights Agent").

                             W I T N E S S E T H:

     WHEREAS, no Person has become an Acquiring Person as such terms are defined
in the Agreement; and

     WHEREAS, the Company has directed the Rights Agent to enter into this
Amendment No. 4 pursuant to Section 27 of the Agreement;

     NOW, THEREFORE, in consideration of the premises and mutual agreements set
forth herein, the parties hereby agree as follows:

1.   Section 1(ii) of the Agreement is hereby deleted in its entirety and the
following substituted in lieu thereof:

          (ii) "Exempted Person" shall mean:  (A) Sandler Capital Partners IV,
          L.P. and Sandler Capital Partners IV FTE, L.P. (collectively,
          "Sandler"), unless and until the earlier of (I) such time as Sandler,
          together with its Affiliates, directly or indirectly, becomes the
          Beneficial Owner of more than 22.0% of the Common Stock then
          outstanding (or such greater percentage as may result solely from the
          acquisition of shares of the Company's Series C Convertible Preferred
          Stock or shares of Common Stock issued to effect the payment of
          dividends, conversion or redemption thereof) (the "Sandler Exempt
          Threshold") or (II) the Effective Time (as defined below), in which
          event Sandler immediately shall cease to be an Exempted Person; (B) W.
          R. Huff Asset Management Co., L.L.C., together with its Affiliates
          (collectively, "Huff"), unless and until such time as Huff, directly
          or indirectly, becomes the Beneficial Owner of Common Stock in excess
          of the Huff Exempt Threshold (as defined below), in which event Huff
          immediately shall cease to be an Exempted Person; (C) Credit Suisse
          First Boston Corporation, together with its Affiliates (collectively,
          "CS First Boston"), unless and until such time as CS First Boston,
          directly or indirectly, 
<PAGE>
 
          becomes the Beneficial Owner of Common Stock in excess of the CS First
          Boston Exempt Threshold (as defined below), in which event CS First
          Boston immediately shall cease to be an Exempted Person; (D)
          Whippoorwill Associates, Inc., together with its Affiliates,
          including, without limitation, any accounts and investment funds
          managed by it or its Affiliates (collectively, "Whippoorwill"), unless
          and until such time as Whippoorwill, directly or indirectly, becomes
          the Beneficial Owner of Common Stock in excess of the Whippoorwill
          Exempt Threshold (as defined below), in which case Whippoorwill
          immediately shall cease to be an Exempted Person; (E) The Northwestern
          Mutual Life Insurance Company, together with its Affiliates
          (collectively, "Northwestern Mutual"), unless and until such time as
          Northwestern Mutual, directly or indirectly, becomes the Beneficial
          Owner of Common Stock in excess of the Northwestern Mutual Exempt
          Threshold (as defined below), in which event Northwestern Mutual
          immediately shall cease to be an Exempted Person; and (F) Resurgence
          Asset Management, L.L.C., together with its Affiliates (collectively,
          "Resurgence"), unless and until such time as Resurgence, directly or
          indirectly, becomes the Beneficial Owner of Common Stock in excess of
          the Resurgence Exempt Threshold (as defined below), in which event
          Resurgence immediately shall cease to be an Exempted Person. For
          purposes of this Agreement, the Huff Exempt Threshold shall mean the
          lesser of (I) 33.0% of the Common Stock then outstanding or (II) a
          percentage of the Common Stock then outstanding equal to (x) that
          percentage of shares of Common Stock directly or indirectly
          Beneficially Owned by Huff immediately after the Effective Time plus
          (y) 5.0%; the CS First Boston Exempt Threshold shall mean the lesser
          of (I) 26.0% of the Common Stock then outstanding or (II) a percentage
          of the Common Stock then outstanding equal to (x) that percentage of
          shares of Common Stock directly or indirectly Beneficially Owned by CS
          First Boston immediately after the Effective Time plus (y) 5.0%; the
          Whippoorwill Exempt Threshold shall mean the lesser of (I) 27.0% of
          the Common Stock then outstanding or (II) a percentage of the Common
          Stock then outstanding equal to (x) that percentage of shares of
          Common Stock directly or indirectly Beneficially Owned by Whippoorwill
          immediately after the Effective Time plus (y) 5.0%; the Northwestern
          Mutual Exempt Threshold shall mean the lesser of (I) 15.5% of the
          Common Stock then outstanding or (II) a percentage of the Common Stock
          then outstanding equal to (x) that percentage of shares of Common
          Stock directly or indirectly Beneficially Owned by Northwestern Mutual
          immediately after the Effective Time plus (y) 5.0%; and the Resurgence
          Exempt Threshold shall mean the lesser of (I) 19.0% of the Common
          Stock then outstanding or (II) a percentage of the Common Stock then
          outstanding equal to that percentage of shares of Common Stock
          directly or indirectly Beneficially 

                                      -2-
<PAGE>
 
          Owned by Resurgence immediately after the Effective Time, rounded to
          the next highest whole integer. The Sandler Exempt Threshold, the Huff
          Exempt Threshold, the CS First Boston Exempt Threshold, the
          Whippoorwill Exempt Threshold, the Northwestern Mutual Exempt
          Threshold and the Resurgence Exempt Threshold are collectively
          referred to herein as the "Exempt Threshold." For purposes of this
          Section 1(ii), none of Huff, CS First Boston, Whippoorwill or
          Northwestern Mutual shall be deemed the Beneficial Owner of any shares
          of Common Stock that are Beneficially Owned by any other Person solely
          as a result of any such Person's execution and performance of any
          Standby Purchase Commitment (as such term is defined in the Plan and
          Agreement of Merger between the Company, Farm Team Corp., a wholly-
          owned Subsidiary of the Company, MobileMedia Corporation and
          MobileMedia Communications, Inc., dated as of August 18, 1998) (as it
          may be amended from time to time, the "Merger Agreement") and Common
          Stock shall be deemed to include shares of Common Stock issuable upon
          conversion of shares of the Company's Series C Convertible Preferred
          Stock, $.01 par value per share. For purposes of this Agreement the
          Effective Time shall have the meaning ascribed to such term in the
          Merger Agreement.

                                      -3-
<PAGE>
 
     IN WITNESS WHEREOF, the parties have caused this Amendment No. 4 to be duly
executed and their respective corporate seals to be hereunto affixed and
attested as of the day and year first written above.

                                        ARCH COMMUNICATIONS GROUP, INC.
Attest:


/s/ J. Roy Pottle                       By:  /s/ C. E. Baker, Jr.
----------------------------------      ----------------------------------------
Name:   J. Roy Pottle                   Name:   C.E. Baker, Jr.
Title:  Executive Vice President        Title:  Chairman of the Board and
        and Chief Financial Officer             Chief Executive Officer

     Seal

                                        THE BANK OF NEW YORK
Attest:


/s/ Luis C. Ortiz                       By:  /s/ Leslie A. Deluca
----------------------------------      ----------------------------------------
Name:  Luis C. Ortiz                    Name:  Leslie A. Deluca
Title: Assistant Vice President         Title: Vice President

     Seal

                                      -4-