<PAGE>   1



                                                                     EXHIBIT 4.1


                       THIRD AMENDMENT TO RIGHTS AGREEMENT

         THIS THIRD AMENDMENT TO RIGHTS AGREEMENT (this "Amendment") is made as
of the 13th day of January, 2000, by and between RAILAMERICA, INC., a Delaware
corporation (the "Company"), and AMERICAN STOCK TRANSFER & TRUST COMPANY (the
"Rights Agent").


                                    RECITALS

         WHEREAS, on January 6, 1998, the Board of Directors of the Company
authorized and declared a dividend distribution of one Right for each share of
common stock, $.001 par value, of the Company outstanding at the close of
business on January 20, 1998; and

         WHEREAS, the Company and the Rights Agent entered into a certain Rights
Agreement, dated as of January 6, 1998 as amended September 3, 1998 and April
18, 1999 (the "Rights Agreement"), providing, among other things, for the
issuance of the Rights (all capitalized terms used in this Amendment and not
otherwise defined herein shall have the meanings ascribed to such terms in the
Rights Agreement); and

         WHEREAS, the Board of Directors of the Company has approved the
amendment of the Rights Agreement as hereinafter set forth, pursuant to a
unanimous written consent dated as of January 13, 2000;

         NOW, THEREFORE, in consideration of the premises and for other good and
valuable consideration, the receipt and sufficiency of which are hereby
acknowledged, the parties hereto agree as follows:

         1. Section 1 of the Rights Agreement is hereby amended by deleting the
definition of "Acquiring Person" set forth in subsection (a) thereof in its
entirety and substituting therefor a new definition of "Acquiring Person" to
read as follows:

                  "`Acquiring Person' shall mean any Person who or which,
         together with all Affiliates and Associates of such Person, shall be
         the Beneficial Owner of fifteen percent (15%) or more of the shares of
         Common Stock the outstanding. Notwithstanding the foregoing, the term
         "Acquiring Person" shall not include (i) the Company, any Subsidiary of
         the Company, any employee benefit plan of the Company or of any
         Subsidiary of the Company, or any Person or entity organized, appointed
         or established by the Company for or pursuant to the terms of any such
         plan, (ii) EGS Associates, L.P., a Delaware limited partnership, EGS
         Partners L.L.C., a Delaware limited liability company, Bev Partners,
         L.P., a Delaware limited partnership, Jonas Partners, L.P., a Delaware
         limited partnership, William Ehrman, Frederic Greenberg, Frederick
         Ketcher, Jonas Gerstl, James McLaren or


<PAGE>   2


         William Lautman, unless such Persons shall be the Beneficial Owners,
         individually or in the aggregate, of thirty percent (30%) or more of
         the shares of Common Stock then outstanding, PROVIDED, HOWEVER, that
         such percentage shall be reduced to twenty percent (20%) upon the
         earlier to occur of (x) the completion of the merger of RailTex, Inc.
         with and into a wholly-owned subsidiary of the Company and (y) April
         30, 2000, (iii) Barclays Bank PLC and the lenders named in that certain
         Senior Secured Loan Facility and Guaranty Agreement (the "Facility")
         dated as of April 30, 1999 among Freight Victoria, the Company,
         RailAmerica Australia Pty Ltd., Barclays Bank PLC and the lenders named
         therein (such lenders collectively with Barclays Bank PLC to be
         referred to hereinafter as the "Lenders"), but only with respect to (A)
         warrants (the "Warrants") to purchase shares of Common Stock issued by
         the Company to the Lenders pursuant to the Facility and pursuant to
         that certain Warrant Agreement dated as of April 30, 1999 between the
         Company and the Lenders and (B) the shares of Common Stock issuable
         upon exercise of the Warrants, it being understood that all other
         shares of Common Stock acquired by the Lenders independent of (A) or
         (B) above shall be considered for purposes of determining Acquiring
         Person status of Lenders; or (iv) any Person who or which, together
         with all Affiliates and Associates of such Person, would be an
         Acquiring Person solely by reason of a reduction in the number of
         issued and outstanding shares of Common Stock of the Company pursuant
         to a transaction or series of related transactions approved by a
         majority of the Independent Directors, if any, then in office and
         approved by a Supermajority Vote; provided, further, however, that in
         the event that such Person described in the foregoing clause (iv) does
         not become an Acquiring Person by reason of the foregoing clause (iv),
         such Person shall nonetheless become an Acquiring Person in the event
         such Person thereafter acquires Beneficial Ownership of an additional
         one percent (1%) of the Common Stock of the Company, unless such
         additional Common Stock ownership results solely from a subsequent
         reduction in the number of issued and outstanding shares of Common
         Stock of the Company."

         2. Except as specifically amended hereby, the Rights Agreement is and
remains unmodified and in full force and effect and is hereby ratified and
confirmed.

         3. This Amendment shall be deemed a contract made under the laws of the
State of Delaware and for all purposes shall be governed by and construed in
accordance with the laws of such State applicable to contracts made and to be
performed entirely within such State.

         4. This Amendment may be executed in counterparts and both of such
counterparts shall for all purposes be deemed to be an original, and such
counterparts shall constitute but one and the same instrument.



                                       2
<PAGE>   3


         IN WITNESS WHEREOF, the parties hereto have caused this Amendment to be
duly executed as of the date first above written.

                                     RAILAMERICA, INC.


                                     By: /s/ Gary O. Marino
                                        ---------------------------------------
                                     Name: Gary O. Marino
                                     Its:  Chairman of the Board, President and
                                           Chief Executive Officer



                                     AMERICAN STOCK TRANSFER & TRUST COMPANY


                                     By: /s/ Herbert J. Lemmer
                                        ---------------------------------------
                                     Name: Herbert J. Lemmer
                                     Its:  Vice President















                                       3