<PAGE>
                                                                   Exhibit 10.53

                                August 17, 2001

Mr. John G. Larkin
403 Somerset Road
Baltimore, MD 21210

Dear John:

     This letter (the "Agreement") will confirm our understanding concerning
your termination of employment with RailWorks Corporation (the "Company") and
your release of the Company from any liability to you, as follows:

     1. Pursuant to your resignation letter dated as of August 17, 2001,
effective as of the date hereof (the "Termination Date"), you will cease to be
an employee of the Company, the Chairman of the Board of the Company, a member
of the Board of Directors of the Company (the "Board") and a member of any
Committee of the Board; and except as explicitly provided herein will promptly
return any Company property in your possession.

     2. In recognition of your position with and service to the Company, the
Company is willing to provide the following so that an orderly and amicable
separation can be achieved:

         (i) Only if this Agreement becomes effective as provided in paragraph
     9. hereof, the Company shall pay to you the following amounts as set forth
     below, less legally required withholding for taxes and other deductions, as
     severance pay. All payments made under this Agreement shall, pursuant to
     present practice, be made by direct deposit to your account at First Union
     National Bank unless otherwise specified in writing by you.

               (a)  your base salary through June 30, 2002 in accordance with
                    the following schedule, and should the Company become the
                    subject of a bankruptcy proceeding, it will not, to the
                    extent legally permissible, object to the allowance of a
                    claim equal in amount to all unpaid installments or contest
                    a claim by you with respect to such payments:

<TABLE>
<CAPTION>
-------------------------------------------------------------------------------------
  DATE                                         PAYMENT AMOUNT
-------------------------------------------------------------------------------------
  <S>                                          <C>
  Company's last regular pay day in the        All salary due through August 2001
  month of August
-------------------------------------------------------------------------------------
  Company's last regular pay day in the        All salary due through September and
  month of September                           October 2001
-------------------------------------------------------------------------------------
  Company's last regular pay day in the        All salary due through November and
  month of November                            December 2001
-------------------------------------------------------------------------------------
  Pay periods from January 1, 2002 through     Company's regular salary payments
  through June 30, 2002                        therefor
-------------------------------------------------------------------------------------
</TABLE>

; and
<PAGE>

               (b)  the cash equivalent of your accrued unused vacation time
                    through the Termination Date (estimated to be three (3)
                    weeks), payable on the Company's last regular pay day in the
                    month of August.

         (ii)   As soon as reasonably possible after the effectiveness of this
                Agreement, the Company agrees to cause the transfer agent to
                remove all legends and stop-transfer orders on the 175,000
                shares of the Company's stock held by you which are not subject
                to Section 2(iv) hereof, so long as said action is compatible
                with all legal and contractual requirements relating thereto;

         (iii)  Any and all outstanding stock options granted to you under any
                Company incentive stock plan shall become immediately vested
                upon effectiveness of this Agreement and shall remain
                exercisable through August 16, 2002 so long as such arrangement
                does not subject the Company to variable accounting treatment;

         (iv)   Immediate cancellation of the outstanding principal balance and
                accrued unpaid interest on the Consolidated, Amended and
                Restated Note executed March 1, 2001, through foreclosure upon
                the stock of the Company held as collateral;

         (v)    The Company shall reimburse you for all of the reasonable legal
                fees incurred by you in connection with the restructuring of the
                management of the Company in the spring and summer of 2001
                through the negotiation of this Agreement;

         (vi)   The Company will provide for your benefit, at its expense, a
                program of reasonable outplacement services for a reasonable
                period of time of not less than six (6) months which the Company
                will arrange by mutual agreement with you;

         (vii)  The Company shall indemnify and hold you harmless to the fullest
                extent permissible under applicable law with respect to the good
                faith actions taken by you in your capacity as a director and/or
                officer of the Company. The Company will use commercially
                reasonable efforts to maintain directors' and officers'
                liability insurance in commercially reasonable amounts under
                which you shall be covered to the same extent as other directors
                and officers of the Company for a reasonable period of time
                following the Termination Date;

         (viii) The Company will furnish to you and your dependents, at its
                expense, the health insurance benefits, including without
                limitation the medical, dental, hospitalization and vision care
                plans, that are generally made available to persons in similar
                positions in the Company for a period of twelve months after the
                Termination Date followed by all rights afforded to you under
                COBRA, or until you obtain comparable health insurance coverage
                through other employment, whichever shall occur first;

         (ix)   The Company will continue to provide you with your existing life
                insurance benefit for a period of twelve months after the
                Termination Date;

         (x)    The Company will reimburse you for all business expenses
                incurred by you prior to the Termination Date for which
                appropriate documentation is presented to the Company; and

         (xi)   The Company will allow you to keep the personal laptop computer
                provided for your use as an employee after the Company is given
                sufficient time and access to confirm that there is no
                proprietary information of the Company on such computer.

     3. You understand and agree that by your acceptance of the terms of this
Agreement and by your receipt of the consideration described in paragraph 2
hereof, the receipt and sufficiency of which is hereby acknowledged, you and the
Company unconditionally, voluntarily and irrevocably agree:

         (i)    That you are no longer bound by any contractual provisions that
                prohibit you from accepting employment from a competitor of the
                Company;

         (ii)   That you will not disparage the Company, its business, its
                officers, directors and/or its employees and the Company will
                not disparage you in any way; and
<PAGE>
         (iii)  To release, acquit, and forever discharge the Company, and any
                affiliated entities, their officers, directors, agents,
                representatives, attorneys, servants, employees, predecessors,
                successors, assigns, and all persons acting herein specifically
                named or not from any and all claims, demands, liabilities,
                debts, judgments, damages, expenses, actions, causes of action
                or suits of any kind whatsoever which you or your heirs,
                personal representatives, and assigns, and each of them may have
                had or may now have, whether known or unknown, including but not
                limited to common law claims, statutory claims, contract claims,
                claims for wages or earnings or benefits, claims for overtime,
                claims or causes of action under Title VII of the Civil Rights
                Act, claims or causes of action pursuant to the Age
                Discrimination in Employment Act ("ADEA"), claims or causes of
                action pursuant to any other federal, state, or municipal
                statute, or local ordinance, incidental or consequential claims
                or damages, expenses incurred, litigation expenses, court costs,
                attorneys' fees, employment, reinstatement, and any and all
                other damages or statutory sums whatsoever known or unknown,
                compensatory or punitive, which arose out of or are connected in
                any way, directly or indirectly with your employment with the
                Company; provided, however, that nothing in this paragraph shall
                be construed to (i) limit your right to receive benefits under
                the Company's employee benefit plans in which you are vested as
                of the Termination Date in accordance with the terms of those
                plans, (ii) limit your right to receive indemnification under
                the Company's charter, by-laws, or any other agreement entered
                into between the Company and you, including this Agreement, or
                as provided by law nor any right to receive insurance coverage
                or proceeds with respect to such indemnification rights, or
                (iii) limit the ability of either party to sue for any act or
                omission which occurs subsequent to termination of the
                employment relationship as it relates to the ADEA. The
                consideration described herein contains additional amounts for
                your waiver of any and all ADEA claims arising out of your
                employment relationship with the Company prior to termination.

     4. The settlement specified in paragraph 2 hereof includes full and
complete compensation for any and all of your counsel or other fees, expenses
and costs incurred in connection with any claims you might possibly have had
against the Company.

     5. You expressly agree that the terms of your settlement with the Company
and the contents of this Agreement are to be kept confidential, and you promise
that you have not revealed and will not reveal such terms to any person other
than any appropriate taxing authorities, your attorney, your accountant and your
immediate family.

     6. If any provision of this Agreement or the application thereof is held
invalid, the invalidity shall not affect other provisions or application of the
Agreement which can be given effect without the invalid provisions or
application and to this end the provisions of this Agreement are declared to be
severable.

     7. Any disputes arising regarding the meaning or validity of this Agreement
shall be governed by the law of the State of Maryland which shall govern the
interpretation of the provisions contained in this agreement.

     8. By your signature on this letter, you hereby affirm and acknowledge that
you have read this Agreement, have been advised by the Company to consult with
an attorney prior to signing this Agreement, and in fact have had full
opportunity to consult with an attorney prior to signing this Agreement. You
further affirm that you are executing this Agreement free from duress, undue
pressure or influence, harassment, or intimidation and you are voluntarily
signing with full knowledge of its terms and conditions.

     9. You have been informed that you have twenty one (21) days from the date
you receive this Agreement to consider and sign it, and that if you sign the
Agreement it will not be effective for a period of seven (7) calendar days,
during which time you can revoke this Agreement as far as it extends to
potential claims under ADEA by written notice to the Company within seven (7)
calendar days following your execution of this Agreement. To be effective, a
notice under this paragraph must be sent within the relevant period specified
above and addressed to:

         Attention: John Kennedy
         Chief Executive Officer
<PAGE>

         RailWorks Corporation
         6225 Smith Avenue, Suite 200
         Baltimore, Maryland 21209.

     10. This Agreement shall not in any way be construed as an admission by the
Company that it has acted wrongfully with respect to you or any other person, or
that you have any rights whatsoever against the Company, and the Company
specifically disclaims any liability to or wrongful acts against you or any
other person, on the part of itself, its employees, or its agents.

     11. This instrument constitutes and contains the entire agreement and
understanding of you and the Company, with respect to the subject matters
addressed herein, and supersedes and replaces all prior negotiations and all
agreements, proposed or otherwise, whether written or oral, concerning the
subject matter hereof.

If the foregoing accurately sets forth our understandings, please so indicate by
signing in the space provided below and returning one signed copy to me.

                                             Very Truly Yours,

                                             RAILWORKS CORPORATION



                                             By: /s/ John Kennedy
                                                ---------------------
                                             Name:  John Kennedy
                                             Title: Chief Executive Officer


Accepted and agreed
this 17th day of August, 2001.


/s/ John G. Larkin
-------------------------------
John G. Larkin