<PAGE>   1


                                                                     EXHIBIT 4.1

                       THIRD AMENDMENT TO RIGHTS AGREEMENT

         THIS THIRD AMENDMENT TO RIGHTS AGREEMENT (this "Amendment") is made as
of the 13th day of January, 2000, by and between RAILAMERICA, INC., a Delaware
corporation (the "Company"), and AMERICAN STOCK TRANSFER & TRUST COMPANY (the
"Rights Agent").


                                    RECITALS

         WHEREAS, on January 6, 1998, the Board of Directors of the Company
authorized and declared a dividend distribution of one Right for each share of
common stock, $.001 par value, of the Company outstanding at the close of
business on January 20, 1998; and

         WHEREAS, the Company and the Rights Agent entered into a certain Rights
Agreement, dated as of January 6, 1998 as amended September 3, 1998 and April
18, 1999 (the "Rights Agreement"), providing, among other things, for the
issuance of the Rights (all capitalized terms used in this Amendment and not
otherwise defined herein shall have the meanings ascribed to such terms in the
Rights Agreement); and

         WHEREAS, the Board of Directors of the Company has approved the
amendment of the Rights Agreement as hereinafter set forth, pursuant to a
unanimous written consent dated as of January 13, 2000;

         NOW, THEREFORE, in consideration of the premises and for other good and
valuable consideration, the receipt and sufficiency of which are hereby
acknowledged, the parties hereto agree as follows:

         1. Section 1 of the Rights Agreement is hereby amended by deleting the
definition of "Acquiring Person" set forth in subsection (a) thereof in its
entirety and substituting therefor a new definition of "Acquiring Person" to
read as follows:

                  "`Acquiring Person' shall mean any Person who or which,
         together with all Affiliates and Associates of such Person, shall be
         the Beneficial Owner of fifteen percent (15%) or more of the shares of
         Common Stock the outstanding. Notwithstanding the foregoing, the term
         "Acquiring Person" shall not include (i) the Company, any Subsidiary of
         the Company, any employee benefit plan of the Company or of any
         Subsidiary of the Company, or any Person or entity organized, appointed
         or established by the Company for or pursuant to the terms of any such
         plan, (ii) EGS Associates, L.P., a Delaware limited partnership, EGS
         Partners L.L.C., a Delaware limited liability company, Bev Partners,
         L.P., a Delaware limited partnership, Jonas Partners, L.P., a Delaware
         limited partnership, William Ehrman, Frederic Greenberg, Frederick
         Ketcher, Jonas Gerstl, James McLaren or William Lautman, unless such
         Persons shall be the Beneficial Owners, individually or in the
         aggregate, of thirty percent (30%) or more of the shares of Common
         Stock then outstanding, PROVIDED, HOWEVER, that such percentage shall
         be reduced to twenty percent (20%) upon the earlier to occur of (x) the
         completion



<PAGE>   2

         of the merger of RailTex, Inc. with and into a wholly-owned subsidiary
         of the Company and (y) April 30, 2000, (iii) Barclays Bank PLC and the
         lenders named in that certain Senior Secured Loan Facility and Guaranty
         Agreement (the "Facility") dated as of April 30, 1999 among Freight
         Victoria, the Company, RailAmerica Australia Pty Ltd., Barclays Bank
         PLC and the lenders named therein (such lenders collectively with
         Barclays Bank PLC to be referred to hereinafter as the "Lenders"), but
         only with respect to (A) warrants (the "Warrants") to purchase shares
         of Common Stock issued by the Company to the Lenders pursuant to the
         Facility and pursuant to that certain Warrant Agreement dated as of
         April 30, 1999 between the Company and the Lenders and (B) the shares
         of Common Stock issuable upon exercise of the Warrants, it being
         understood that all other shares of Common Stock acquired by the
         Lenders independent of (A) or (B) above shall be considered for
         purposes of determining Acquiring Person status of Lenders; or (iv) any
         Person who or which, together with all Affiliates and Associates of
         such Person, would be an Acquiring Person solely by reason of a
         reduction in the number of issued and outstanding shares of Common
         Stock of the Company pursuant to a transaction or series of related
         transactions approved by a majority of the Independent Directors, if
         any, then in office and approved by a Supermajority Vote; provided,
         further, however, that in the event that such Person described in the
         foregoing clause (iv) does not become an Acquiring Person by reason of
         the foregoing clause (iv), such Person shall nonetheless become an
         Acquiring Person in the event such Person thereafter acquires
         Beneficial Ownership of an additional one percent (1%) of the Common
         Stock of the Company, unless such additional Common Stock ownership
         results solely from a subsequent reduction in the number of issued and
         outstanding shares of Common Stock of the Company."

         2. Except as specifically amended hereby, the Rights Agreement is and
remains unmodified and in full force and effect and is hereby ratified and
confirmed.

         3. This Amendment shall be deemed a contract made under the laws of the
State of Delaware and for all purposes shall be governed by and construed in
accordance with the laws of such State applicable to contracts made and to be
performed entirely within such State.

         4. This Amendment may be executed in counterparts and both of such
counterparts shall for all purposes be deemed to be an original, and such
counterparts shall constitute but one and the same instrument.



<PAGE>   3


         IN WITNESS WHEREOF, the parties hereto have caused this Amendment to be
duly executed as of the date first above written.

                                   RAILAMERICA, INC.


                                   By:    /s/ Gary O. Marino
                                         ------------------------------------
                                   Name: Gary O. Marino
                                   Its:  Chairman of the Board, President and
                                         Chief Executive Officer



                                   AMERICAN STOCK TRANSFER & TRUST COMPANY


                                   By:    /s/ Herbert J. Lemmer
                                         ------------------------------------
                                   Name: Herbert J. Lemmer
                                   Its:  Vice President