<PAGE>
                                                                   Exhibit 10.54

                                August 17, 2001

Mr. Michael R. Azarela
12310 Cleghorn Road
Cockeysville, MD 21030

Dear Mike:

     This letter (the "Agreement") will confirm our understanding concerning
your termination of employment with RailWorks Corporation (the "Company") and
your release of the Company from any liability to you, as follows:

     1. Pursuant to your resignation letter dated as of August 17, 2001,
effective as of the date hereof (the "Termination Date"), you will cease to be
an employee of the Company, the President and Chief Operating Officer of the
Company, a member of the Board of Directors of the Company (the "Board") and a
member of any Committee of the Board; and except as explicitly provided herein
will promptly return any Company property in your possession.

     2. In recognition of your position with and service to the Company, the
Company is willing to provide the following so that an orderly and amicable
separation can be achieved:

          (i) Only if this Agreement becomes effective as provided in paragraph
              Item 12. hereof, the Company shall pay to you the following
              amounts as set forth below, less legally required withholding for
              taxes and other deductions, as severance pay. All payments made
              under this Agreement shall, pursuant to present practice, be made
              by direct deposit to your account at the Bank of America unless
              otherwise specified in writing by you.

               (a)  your base salary through June 30, 2002 in accordance with
                    the following schedule, and should the Company become the
                    subject of a bankruptcy proceeding, it will not, to the
                    extent legally permissible, object to the allowance of a
                    claim equal in amount to all unpaid installments or contest
                    a claim by you with respect to such payments:

<TABLE>
<CAPTION>
-------------------------------------------------------------------------------------
  DATE                                         PAYMENT AMOUNT
-------------------------------------------------------------------------------------
  <S>                                          <C>
  Company's last regular pay day in the        All salary due through August 2001
  month of August
-------------------------------------------------------------------------------------
  Company's last regular pay day in the        All salary due through September and
  month of September                           October 2001
-------------------------------------------------------------------------------------
  Company's last regular pay day in the        All salary due through November and
  month of November                            December 2001
-------------------------------------------------------------------------------------
  Pay periods from January 1, 2002             Company's regular salary payments
  through June 30, 2002                        therefor
-------------------------------------------------------------------------------------
</TABLE>

; and
<PAGE>

               (b)  the cash equivalent of your accrued unused vacation time
                    through the Termination Date (estimated to be four (4)
                    weeks), payable on the Company's last regular pay day in the
                    month of August.

(ii)   As soon as reasonably possible after the effectiveness of this Agreement,
       the Company agrees to cause the transfer agent to remove all legends and
       stop-transfer orders on the shares of the Company's stock held by you
       which are not subject to Section 2(iv) hereof, so long as said action is
       compatible with all legal and contractual obligations of the Company
       relating thereto (including without limitation the "Block Trade"
       restrictions);

(iii)  Any and all outstanding stock options granted to you under any Company
       incentive stock plan shall become immediately vested upon effectiveness
       of this Agreement and shall remain exercisable through August 16, 2002 so
       long as such arrangement does not subject the Company to variable
       accounting treatment;

(iv)   Immediate cancellation of the outstanding principal balance and accrued
       unpaid interest on the Consolidated, Amended and Restated Note executed
       March 1, 2001, through foreclosure upon the stock of the Company held as
       collateral;

(v)    The Company shall reimburse you for all of the reasonable legal fees
       incurred by you in connection with the negotiation and, for a period of
       fifteen (15) calendar days, the implementation and interpretation of this
       Agreement;

(vi)   The Company will provide for your benefit, at its expense, a program of
       reasonable outplacement services for a reasonable period of time of not
       less than six (6) months which the Company will arrange by mutual
       agreement with you;

(vii)  The Company shall indemnify and hold you harmless to the fullest extent
       permissible under applicable law with respect to the good faith actions
       taken by you in your capacity as a director and/or officer of the
       Company. The Company will use commercially reasonable efforts to maintain
       directors' and officers' liability insurance in commercially reasonable
       amounts under which you shall be covered to the same extent as other
       directors and officers of the Company for a reasonable period of time
       following the Termination Date;

(viii) The Company will furnish to you and your dependents, at its expense, the
       health insurance benefits, including without limitation the medical,
       dental, hospitalization and vision care plans, that are generally made
       available to persons in similar positions in the Company for a period of
       twelve months after the Termination Date followed by all rights afforded
       to you under COBRA, or until you obtain comparable health insurance
       coverage through other employment, whichever shall occur first;

(ix)   The Company will continue to provide you with your existing life
       insurance benefit for a period of twelve months after the Termination
       Date;

(x)    The Company will reimburse you for all business expenses incurred by you
       prior to the Termination Date for which appropriate documentation is
       presented to the Company; and

(xi)   The Company will allow you to keep the personal laptop computer provided
       for your use as an employee after the Company is given sufficient time
       and access to confirm that there is no proprietary information of the
       Company on such computer.

       3. You understand and agree that by your acceptance of the terms of this
Agreement and by your receipt of the consideration described in paragraph 2
hereof, the receipt and sufficiency of which is hereby acknowledged, you and the
Company unconditionally, voluntarily and irrevocably agree:

(i)    That you are no longer bound by any contractual provisions that prohibit
       you from accepting employment from a competitor of the Company;

(ii)   That you will not disparage the Company, its business, its officers,
       directors and/or its employees and the Company will not disparage you in
       any way; and
<PAGE>
         (iii) To release, acquit, and forever discharge the Company, and any
               affiliated entities, their officers, directors, agents,
               representatives, attorneys, servants, employees, predecessors,
               successors, assigns, and all persons acting herein specifically
               named or not from any and all claims, demands, liabilities,
               debts, judgments, damages, expenses, actions, causes of action or
               suits of any kind whatsoever which you or your heirs, personal
               representatives, and assigns, and each of them may have had or
               may now have, whether known or unknown, including but not limited
               to common law claims, statutory claims, contract claims, claims
               for wages or earnings or benefits, claims for overtime, claims or
               causes of action under Title VII of the Civil Rights Act, claims
               or causes of action pursuant to the Age Discrimination in
               Employment Act ("ADEA"), claims or causes of action pursuant to
               any other federal, state, or municipal statute, or local
               ordinance, incidental or consequential claims or damages,
               expenses incurred, litigation expenses, court costs, attorneys'
               fees, employment, reinstatement, and any and all other damages or
               statutory sums whatsoever known or unknown, compensatory or
               punitive, which arose out of or are connected in any way,
               directly or indirectly with your employment with the Company;
               provided, however, that nothing in this paragraph shall be
               construed to (i) limit your right to receive benefits under the
               Company's employee benefit plans in which you are vested as of
               the Termination Date in accordance with the terms of those plans,
               (ii) limit your right to receive indemnification under the
               Company's charter, by-laws, or any other agreement entered into
               between the Company and you, including this Agreement, or as
               provided by law nor any right to receive insurance coverage or
               proceeds with respect to such indemnification rights, or (iii)
               limit the ability of either party to sue for any act or omission
               which occurs subsequent to termination of the employment
               relationship as it relates to the ADEA. The consideration
               described herein contains additional amounts for your waiver of
               any and all ADEA claims arising out of your employment
               relationship with the Company prior to termination.

     4. The settlement specified in paragraph 2 hereof includes full and
complete compensation for any and all of your counsel or other fees, expenses
and costs incurred in connection with any claims you might possibly have had
against the Company.

     5. You expressly agree that the terms of your settlement with the Company
and the contents of this Agreement are to be kept confidential, and you promise
that you have not revealed and will not reveal such terms to any person other
than any appropriate taxing authorities, your attorney, your accountant and your
immediate family.

     6. If any provision of this Agreement or the application thereof is held
invalid, the invalidity shall not affect other provisions or application of the
Agreement which can be given effect without the invalid provisions or
application and to this end the provisions of this Agreement are declared to be
severable.

     7. Any disputes arising regarding the meaning or validity of this Agreement
shall be governed by the law of the State of Maryland which shall govern the
interpretation of the provisions contained in this agreement.

     8. By your signature on this letter, you hereby affirm and acknowledge that
you have read this Agreement, have been advised by the Company to consult with
an attorney prior to signing this Agreement, and in fact have had full
opportunity to consult with an attorney prior to signing this Agreement. You
further affirm that you are executing this Agreement free from duress, undue
pressure or influence, harassment, or intimidation and you are voluntarily
signing with full knowledge of its terms and conditions.

     9. You have been informed that you have twenty one (21) days from the date
you receive this Agreement to consider and sign it, and that if you sign the
Agreement it will not be effective for a period of seven (7) calendar days,
during which time you can revoke this Agreement as far as it extends to
potential claims under ADEA by written notice to the Company within seven (7)
calendar days following your execution of this Agreement. To be effective, a
notice under this paragraph must be sent within the relevant period specified
above and addressed to:

         Attention: John Kennedy
         Chief Executive Officer
<PAGE>

         RailWorks Corporation
         6225 Smith Avenue, Suite 200
         Baltimore, Maryland 21209.

     10. This Agreement shall not in any way be construed as an admission by the
Company that it has acted wrongfully with respect to you or any other person, or
that you have any rights whatsoever against the Company, and the Company
specifically disclaims any liability to or wrongful acts against you or any
other person, on the part of itself, its employees, or its agents.

     11. This instrument constitutes and contains the entire agreement and
understanding of you and the Company, with respect to the subject matters
addressed herein, and supersedes and replaces all prior negotiations and all
agreements, proposed or otherwise, whether written or oral, concerning the
subject matter hereof.

     If the foregoing accurately sets forth our understandings, please so
indicate by signing in the space provided below and returning one signed copy to
me.

                                             Very Truly Yours,

                                             RAILWORKS CORPORATION



                                             By: /s/ John Kennedy
                                                ---------------------
                                             Name:  John Kennedy
                                             Title: Chief Executive Officer


Accepted and agreed
this 20th day of August, 2001.


/s/ Michael R. Azarela
-------------------------------
Michael R. Azarela