<PAGE>
                                                                   EXHIBIT 10.96





                              EMPLOYMENT AGREEMENT


                  This EMPLOYMENT AGREEMENT ("AGREEMENT") is made and entered
into as of August 1, 2004, by and between RailAmerica, Inc., a Delaware
corporation (the "COMPANY"), and Mr. Charles Swinburn (the "EXECUTIVE").

                              W I T N E S S E T H:

                  WHEREAS, the Executive is employed as Chief Executive Officer
of the Company;

                  WHEREAS, the Company wishes to enter into this Agreement with
the Executive to be assured of his services on the terms and conditions
hereinafter set forth;

                  WHEREAS, the Executive wishes to enter into this Agreement
with the Company and to perform and to serve the Company on the terms and
conditions hereinafter set forth; and

                  WHEREAS, the Compensation Committee (the "COMMITTEE") of the
Board of Directors of the Company (the "BOARD") has approved the terms of this
Agreement as of the date set forth above;

                  NOW THEREFORE, in consideration of the mutual terms,
covenants, agreements and conditions hereinafter set forth, the Company and the
Executive hereby agree as follows:

         1. EMPLOYMENT.

                  (a) The Company hereby employs the Executive to serve as a
full time employee of the Company, and the Executive hereby accepts such
employment with the Company, for the period set forth in SECTION 2 hereof. The
Executive shall be employed as Chief Executive Officer of the Company and shall
faithfully and competently perform such duties in such manner as the Company may
from time to time reasonably direct. The Executive shall report to the Board of
Directors and shall have overall senior executive responsibility for the
Company, and shall perform such other tasks and duties as may be assigned to him
from time to time.

                  (b) Except as may otherwise be approved in advance by the
Board, and except during vacation periods and reasonable periods of absence due
to sickness, personal injury or other incapacity, the Executive shall devote his
full time and efforts throughout the Employment Term to the services required of
him hereunder. The Executive shall render his services exclusively to the


<PAGE>

Company during the Employment Term and shall use his best efforts, judgment and
energy to improve and advance the business and interests of the Company in a
manner consistent with the duties of his position. The Executive shall observe
and comply with the Company's rules and regulations regarding the performance of
his duties and shall carry out and perform all orders, directions, and policies
given to him. The Executive shall at all times carry out the duties assigned to
him in a loyal, trustworthy and businesslike manner.

                  (c) The Executive's principal place of employment shall be at
the Company's headquarters in Boca Raton, Florida or at such other location as
shall be mutually acceptable to the Executive and the Company.

                  (d) The Executive shall be covered by Directors' and Officers'
liability insurance, provided that the terms and amounts of such insurance are
approved by the Board. Additionally, the Executive shall be indemnified by the
Company to the fullest extent permitted by law.

        2. TERM.

                  Unless earlier terminated or extended as provided in this
Agreement, the term of the Executive's employment under this Agreement shall be
for a period beginning on the date hereof (the "COMMENCEMENT DATE") and ending
on the second anniversary of the Commencement Date (such period being
hereinafter called the "INITIAL TERM"). At the end of the Initial Term, the term
of employment automatically shall renew for successive one (1) year terms
(subject to earlier termination as provided herein), unless the Company or the
Executive delivers written notice to the other at least three (3) months prior
to the expiration date of its or his election not to renew the term of
employment. References herein to the "EMPLOYMENT TERM" shall refer both to the
Initial Term and each successive one-year term for which this Agreement is
extended.

        3. SALARY, BONUSES AND BENEFITS.

                  (a) SALARY. In consideration of the services of the Executive
rendered to the Company hereunder, the Company shall pay the Executive a base
salary (the "BASE SALARY") at an annual rate of Five hundred thousand dollars
($500,000) during the Employment Term, payable in regular intervals in
accordance with the Company's current payroll practices, as the same may be
changed from time to time.

                  (b) RESTRICTED STOCK. In consideration of the services of the
Executive hereunder, the Company shall issue to Executive restricted shares
("Restricted Stock") of common stock, par value $0.01 per share ("Common
Stock"), of the Company as follows: (i) $100,000 of Restricted Stock on the date
hereof, one third of which shall vest on each of the three succeeding
anniversaries hereof, but none of which may be sold until the third anniversary
hereof except as required by Executive to pay any applicable income taxes; (ii)
$100,000 of Restricted Stock on the first anniversary hereof, one-half of which
shall vest on each of the two succeeding anniversaries thereof, but none of
which may be sold until the third anniversary hereof except as required by
Executive to pay any applicable income taxes; and (iii) provided there has been
no termination of this Agreement by the Company or the Executive, $100,000 of
Restricted Stock on the second anniversary hereof, all of which shall vest and
may be sold by Executive on the third anniversary hereof. For purposes of
determining the amount of Restricted Stock to be issued to Executive under this
SECTION 3(B), the Company shall divide (A) $100,000 by (B) the average closing
sales price for the Company's Common Stock on the New York Stock Exchange on the


                                       2
<PAGE>

last (5) business days immediately prior to the date of issuance. All
restrictions on the Restricted Stock shall lapse immediately upon any
termination of this Agreement.

                  (c) DISCRETIONARY BONUSES. In addition to the Base Salary, the
Company may pay to Executive such bonuses and other incentive compensation, if
any, as the Committee or the Board annually, in their sole discretion, deem
appropriate. This includes but is not limited to participation in the Annual
Incentive Plan with a target payment of 60% of salary subject to the attainment
of defined goals, participation in the Long Term Incentive Compensation Plan as
defined, and participation in the Deferred Compensation Plan, provided that the
Committee or the Board may alter, amend or terminate any of such plans or
Executive's level of participation, as the case may be, at any time, which may
result in a diminution in benefits or no benefits to the Executive.

                  (d) WITHHOLDINGS AND DEDUCTIONS. The payment of any salary,
bonus or other compensation hereunder shall be subject to income tax, social
security and other applicable withholdings, as well as such deductions as may be
required under the Company's employee benefit plans.

                  (e) BENEFITS. During the Employment Term, the Executive shall
be:

                           (i) eligible to participate in all employee fringe
                  benefits and any pension and/or profit sharing plans,
                  including 401k plan employer contributions, that may be
                  provided by the Company for its management employees generally
                  in accordance with the provisions of any such plans, as the
                  same may be in effect on and after the date hereof, subject to
                  any amendments or changes as shall be determined at any time
                  by the Committee or the Board;

                           (ii) eligible to participate in any medical and
                  health plans or other employee welfare benefit plans that may
                  be provided by the Company for its management employees
                  generally in accordance with the provisions of any such plans,
                  as the same may be in effect on and after the date hereof; and
                  supplemental coverage for any co-payments and deductibles,
                  provided that the annual premiums, fees and other costs paid
                  by the Company do not exceed $10,000 annually for the
                  Executive and his family, and provided further that this
                  supplemental coverage benefit shall terminate on December 31,
                  2004;

                           (iii) entitled to four (4) weeks of annual paid
                  vacation;

                           (iv) entitled to sick leave, sick pay and disability
                  benefits in accordance with any Company policy or plan that
                  may be applicable to employees on and after the date hereof;
                  and additional Long-Term Disability Insurance for the
                  Executive, provided that the annual premiums, fees and other
                  costs paid by the Company do not exceed $10,000;

                           (v) entitled to additional Term Life Insurance for
                  the Executive in the amount of $1,000,000 above and beyond the
                  life insurance provided by the Company in accordance with the
                  Company's policies that may be applicable to employees on and
                  after the date hereof, provided that the annual premiums, fees


                                       3
<PAGE>

                  and other costs paid by the Company do not exceed $10,000 for
                  such insurance; and

                           (vi) entitled to reimbursement for all reasonable and
                  necessary out-of-pocket business expenses incurred by the
                  Executive in the performance of his duties hereunder in
                  accordance with the Company's policies that may be applicable
                  to employees on and after the date hereof.

                  (f) Participation in stock option and restricted stock
programs as approved by the Board from time to time, subject to any alterations,
amendments or termination by the Board of any such programs at any time.

        4. TERMINATION.

                  (a) The Executive's employment hereunder shall be terminated
upon the occurrence of any of the following:

                           (i) the death of the Executive;

                           (ii) termination by the Company because of the
                  Executive's inability to perform his duties because of a
                  Disability (as hereinafter defined);

                           (ii) termination by the Executive at any time for any
                  reason whatsoever (including, without limitation, resignation
                  or retirement) other than for Good Reason (as hereinafter
                  defined);

                           (iv) termination by the Company at any time for Cause
                  (as hereinafter defined), such termination to take effect
                  immediately upon written notice from the Company to the
                  Executive; and

                           (v) termination (A) by the Company without Cause, or
                  (B) by the Executive for Good Reason.

                  (b) The following terms shall have the following meanings:

                           (i) "CAUSE" shall mean the Executive's: (A)
                  indictment, arrest or charge with any felony, (B) commission
                  of any act of dishonesty or moral turpitude, (C) failure to
                  obey the reasonable and lawful orders of the Board or any
                  senior executive officer of the Company or (D) negligence in
                  the performance of, or disregard of, his obligations
                  hereunder.

                           (ii) "DISABILITY" shall mean a physical or mental
                  incapacity that prevents the Executive from performing the
                  essential functions of his position with the Company for a
                  period of ninety (90) or more days, whether or not
                  consecutive, occurring within any period of twelve (12)
                  consecutive months as determined in accordance with any
                  long-term disability plan provided by the Company of which the
                  Executive is a participant or by the Board in its sole
                  discretion.


                                       4
<PAGE>


                           (iii) "GOOD REASON" shall mean: (A) a material
                  diminution of the Executive's responsibilities or authority
                  hereunder, or (B) a material (i.e., greater than ten (10)
                  percent) decrease in the Executive's Base Salary and benefits,
                  taken as a whole, other than a decrease applicable to all
                  other members of senior management.

                  (c) In the event that the Executive's employment is terminated
         during the Employment Term pursuant to clause (v) of paragraph (a)
         above, the Company shall pay to the Executive, as damages, the amount
         of Base Salary and discretionary bonus and incentive plan compensation,
         if any, which the Executive would have otherwise been entitled to
         receive pursuant to Section 3(a) hereof had the Executive's employment
         not been so terminated from the date of termination until twelve (12)
         months after the date of such termination (such amount being herein
         referred to as the "TERMINATION PAYMENTS" and such period being herein
         referred to as the "TERMINATION PERIOD"); PROVIDED, HOWEVER, that no
         Termination Payments shall be made unless and until the Executive
         executes a general release in a form reasonably satisfactory to the
         Company.

                  (d) While receiving salary continuation, Executive will be
         placed on a leave of absence status and entitled to all benefits for
         which Executive is eligible to participate. Stock options will continue
         to vest during the salary continuation period. Vested options will
         expire in accordance with their respective grant letters.

                  (e) Notwithstanding anything to the contrary expressed or
         implied herein, except as required by applicable law and as provided in
         this SECTION 4(C), the Company (and its Affiliates) shall not be
         obligated to make any payments to the Executive or on his behalf of any
         kind or nature by reason of the Executive's cessation of employment
         (including, without limitation, by reason of termination of the
         Executive's employment by the Company for Cause), other than (i) any
         Base Salary payable through the date of termination, and (ii) any
         previously awarded bonus that remains unpaid as of the date of said
         termination.

                  (f) No interest shall accrue on or be paid with respect to any
         portion of any payments hereunder, except as required by law.

                  (g) Notwithstanding anything to the contrary in this
         Agreement, an amount may not be paid under this Agreement to the extent
         that it, when considered along with other amounts paid or provided by
         the Company to Executive, would not be deductible by the Company
         because it would (a) exceed an amount constituting reasonable
         compensation, (b) constitute an excess parachute payment under Section
         280G of the Internal Revenue Code, or (c) exceed the limitations of
         Section 162(m) of the Internal Revenue Code, or if for any other reason
         it would not be allowable as a deduction by the Company under the
         Internal Revenue Code.

        5. DUTY OF LOYALTY.

                  (a) GENERAL. The Executive's employment hereunder creates a
relationship of confidence and trust between the Executive and the Company. The
purpose of the restrictions contained in this SECTION 5 is to protect the
goodwill and other legitimate business interests of the Company, and the Company
would not have entered into this Agreement in the absence of such restrictions.

                                       5
<PAGE>

                  (b) CONFIDENTIAL INFORMATION. The Executive hereby covenants,
agrees and acknowledges as follows:

                           (i) The Executive will have access to and will
                  possess Confidential Information that has been created,
                  discovered or developed by, or otherwise is maintained by the
                  Company and its Affiliates or in which property rights have
                  been or may be assigned or otherwise conveyed to the Company
                  and/or any Affiliate, which information has commercial value
                  in the business in which the Company and/or any Affiliate is
                  engaged and is treated by the Company and its Affiliates as
                  confidential.

                           (ii) The Executive will keep confidential all
                  Confidential Information and will not without the prior
                  written consent of the Board (A) use for his benefit or
                  disclose, except to the extent required by the performance by
                  him of his duties as an employee of the Company, any
                  Confidential Information, or (B) take any Confidential
                  Information with him upon leaving the employ of the Company.

                           (iii) The Executive agrees that upon termination of
                  his employment by the Company for any reason, or at such
                  earlier time as the Company may request, the Executive shall
                  forthwith return to the Company all documents and other
                  property in his possession belonging to the Company or any of
                  its Affiliates.

                  (c) NON-INTERFERENCE WITH BUSINESS RELATIONSHIPS. During the
Restricted Period, the Executive will not directly or indirectly, as a director,
officer, employee, manager, consultant, independent contractor, advisor or
otherwise:

                           (i) make any statements or perform any acts intended
                  or likely to interfere with any interest of the Company or any
                  of its Affiliates in its relationship and dealings with
                  existing or potential customers or clients;

                           (ii) make any statements or do any acts intended to
                  cause or having the effect of causing any customers or clients
                  of the Company or any of its Affiliates to make use of the
                  services of any business in which the Executive has or expects
                  to acquire any interest, if such statements or acts would
                  result or would likely result in such customers or clients
                  ceasing to do business with the Company or any Affiliates; or

                           (iii) engage in competition with, or be connected
                  with any business or organization which operates railroad
                  track within one-hundred fifty (150) miles of, any track
                  operated by the Company or any of its Affiliates (A) as of the
                  Commencement Date or (B) during the Restricted Period;
                  PROVIDED, HOWEVER, that the provisions of this SECTION 5(C)
                  shall not be deemed to prohibit the Executive's ownership of
                  not more than one percent (1%) of the total shares of all
                  classes of stock outstanding of any publicly held company.

                  (d) NON-SOLICITATION. During the Restricted Period, the
Executive will not directly or indirectly, as a director, officer, employee,
manager, consultant, independent contractor, advisor or otherwise:



                                       6
<PAGE>

                           (i) employ or solicit for employment, or advise or
                  recommend to any other person or entity that they employ or
                  solicit for employment, any employee of the Company or any of
                  its Affiliates; or

                           (ii) solicit or encourage any employee of the Company
                  or any of its Affiliates to leave the employ of the Company or
                  the Affiliate or to do any act that is disloyal to, or
                  inconsistent with the interests of, the Company or any of its
                  Affiliates.

                  (e) DEFINITIONS. For the purposes of this SECTION 6, the
following terms shall have the following meanings:

                           (i) "AFFILIATE" or "AFFILIATES" shall mean the
                  Company and any person, corporation or other entity directly
                  or indirectly under the common control of the Company.

                           (ii) "CONFIDENTIAL INFORMATION" shall mean all
                  nonpublic and/or proprietary information respecting the
                  business of the Company or any Affiliate, regardless of
                  whether such information has been reduced to documentary form.
                  Confidential Information also includes information concerning
                  the Company's or any Affiliate's clients, customers and
                  suppliers, including their identity, address and other
                  information kept by the Company or any Affiliate.

                           (iii) "RESTRICTED PERIOD" shall mean the twelve (12)
                  month period after the Executive's employment with the Company
                  ceases, regardless of the reason for the cessation.

                           (v) "COMPETE" shall mean when a person or entity
                  (including, without limitation, the Executive) conducts,
                  operates, carries out or engages in the short-line railroad
                  business.

                  (f) The Executive represents that his experience, capabilities
and circumstances are such that the provisions of this SECTION 5 will not
prevent him from earning a livelihood and further agrees that the limitations
set forth in this SECTION 5 are reasonable in duration, geographic area and
scope and are properly required for the adequate protection of the Company and
its Affiliates.

                  (g) The Executive acknowledges and agrees that any remedy at
law for any breach or threatened breach of the provisions of this SECTION 5
would be inadequate and, therefore, agrees that the Company and any of its
Affiliates shall be entitled to injunctive relief in addition to any other
available rights and remedies in cases of any breach or threatened breach by the
Executive.

        6. NON-ASSIGNABILITY.

                  (a) Neither this Agreement nor any right or interest hereunder
shall be assignable by the Executive, his beneficiaries, or legal
representatives without the Company's prior written consent; PROVIDED, HOWEVER,
that nothing in this SECTION 6(A) shall preclude the Executive from designating
a beneficiary to receive any benefit payable hereunder upon his death or
incapacity.



                                       7
<PAGE>

                  (b) Except as required by law, no right to receive payments
under this Agreement shall be subject to anticipation, commutation, alienation,
sale, assignment, encumbrance, charge, pledge, or hypothecation or to exclusion,
attachment, levy or similar process or assignment by operation of law, and any
attempt, voluntary or involuntary, to effect any such action shall be null, void
and of no effect.

         7. NOTICES. Any notice required or permitted to be given under this
Agreement shall be sufficient if in writing and either delivered in person, sent
by first class certified or registered mail, postage prepaid or sent by
overnight courier, if to the Company, at the Company's principal place of
business, and if to the Executive, at his home address most recently filed with
the Company, or to such other address or addresses as either party shall have
designated in writing to the other party hereto.

         8. SEVERABILITY. The Executive agrees that in the event that any court
of competent jurisdiction shall finally hold that any provision of this
Agreement is invalid, illegal or incapable of being enforced in whole or in part
by reason of any rule of law or public policy, that part shall be deemed to be
reformed to delete the offending language or completely severed from the
remainder of this Agreement for the purpose only of the particular proceedings
in question, and all other covenants and provisions of this Agreement shall in
every other respect continue in full force and effect.

         9. WAIVER. Failure to insist upon strict compliance with any of the
terms, covenants or conditions hereof shall not be deemed a waiver of such term,
covenant or condition.

         10. ENTIRE AGREEMENT; MODIFICATIONS. This Agreement constitutes the
entire and final expression of the agreement of the parties with respect to the
subject matter hereof and supersedes all prior agreements, oral and written,
between the parties hereto with respect to the subject matter hereof. This
Agreement may be modified or amended only by an instrument in writing signed by
both parties hereto. If any provision of this Agreement conflicts with any other
agreement, policy, plan, practice or other Company document, then the provisions
of this Agreement will control.

         11. GOVERNING LAW. This Agreement shall be construed and enforced in
accordance with the internal laws of the State of Delaware without regard to the
conflicts of law principles thereof.

         12. COUNTERPARTS. This Agreement may be executed in counterparts, each
of which shall be deemed an original, but both of which together shall
constitute one and the same instrument.

         13. INDEMNIFICATION.

                  a. Subject to limitations imposed by law, the Company shall
indemnify and hold harmless the Executive to the fullest extent permitted by law
from and against any and all claims, damages, expenses (including attorneys'
fees), judgments, penalties, fines, settlements, and all other liabilities
incurred or paid by him in connection with the investigation, defense,
prosecution, settlement or appeal of any threatened, pending or completed
action, suit or proceeding, whether civil, criminal, administrative or
investigative and to which the Executive was or is a party or is threatened to
be made a party by reason of the fact that the Executive is or was an officer,
employee or agent of the Company, or by reason of anything done or not done by
the Executive in any such capacity or capacities, provided that the Executive
acted in good faith, in a manner that was not grossly negligent or constituted


                                       8
<PAGE>

willful misconduct and in a manner he reasonably believed to be in or not
opposed to the best interests of the Company, and, with respect to any criminal
action or proceeding, had no reasonable cause to believe his conduct was
unlawful. The Company also shall pay any and all expenses (including attorneys'
fees) incurred by the Executive as a result of the Executive being called as a
witness in connection with any matter involving the Company and/or any of its
officers or directors.

                  b. The Company shall pay any expenses (including attorneys'
fees), judgments, penalties, fines, settlements, and other liabilities incurred
by the Executive in investigating, defending, settling or appealing any action,
suit or proceedings described in this Section 13 in advance of the final
disposition of such action, suit or proceeding. The Company shall promptly pay
the amount of such expenses to the Executive, but in no event later than ten
(10) days following the Executive's delivery to the Company of a written request
for an advance pursuant to this Section 13, together with a reasonable
accounting of such expenses.

                  c. The Executive hereby undertakes and agrees to repay to the
Company any advances made pursuant to this Section 13 if and to the extent that
it shall ultimately be found that the Executive is not entitled to be
indemnified by the Company for such amounts.

                  d. The Company shall make the advances contemplated by this
Section 13 regardless of the Executive's financial ability to make repayment,
and regardless whether indemnification of the indemnitee by the Company will
ultimately be required. Any advances and undertakings to repay pursuant to this
Section 13 shall be unsecured and interest free.

                  e. The provisions of this Section 13 shall survive the
termination of the Employment Term or expiration of the term of this Agreement.

         14. ACKNOWLEDGEMENT. The Executive represents and acknowledges the
following:

                  (a)      He has carefully read this Agreement in its entirety;

                  (b)      He understands the terms and conditions contained
                           herein;

                  (c)      He has had the opportunity to review this Agreement
                           with legal counsel of his own choosing and has not
                           relied on any statements made by the Company or its
                           legal counsel as to the meaning of any term or
                           condition contained herein or in deciding whether to
                           enter into this Agreement; and

                  (d)      He is entering into this Agreement knowingly and
                           voluntarily.




                                       9
<PAGE>




                  IN WITNESS WHEREOF, the Company and the Executive have duly
executed and delivered this Agreement as of the day and year first above
written.



                                          RAILAMERICA, INC.


                                          By: /s/ FERD C. MEYER, JR.
                                             --------------------------------
                                               Name:  Ferd. C. Meyer, Jr.
                                               Title: Chairman, Compensation
                                                      Committee of the Board


                                          EXECUTIVE

                                          /s/ CHARLES SWINBURN
                                          -----------------------------------
                                          Name: Charles Swinburn



                                       10