<PAGE>
                                                                    Exhibit 10.4


                       COLLATERAL ASSIGNMENT OF TRADEMARKS
                              (SECURITY AGREEMENT)

            COLLATERAL ASSIGNMENT OF TRADEMARKS (SECURITY AGREEMENT) dated as of
February 4, 1997, between ATLANTIC EXPRESS TRANSPORTATION CORP., a New York
corporation with offices at 7 North Street, Staten Island, New York 10302
("Assignor"), and THE BANK OF NEW YORK, a New York banking corporation, with
offices at 101 Barclay Street-- 21W, New York, New York 10286, as the trustee
under the Indenture (defined below) ("Assignee").

                              W I T N E S S E T H:

            WHEREAS, Amboy Bus Co., Inc., Atlantic-Conn. Transit, Inc.,
Atlantic-Hudson, Inc., Atlantic Paratrans, Inc., Atlantic Paratrans of Kentucky
Inc., Atlantic Express Coachways, Inc., Atlantic Express of Missouri Inc.,
Atlantic Express of Pennsylvania, Inc., Brookfield Transit Inc., Courtesy Bus
Co., Inc., K. Corr, Inc., Merit Transportation Corp., Metropolitan Escort
Service, Inc., Raybern Bus Service, Inc., Raybern Capital Corp., Raybern Equity
Corp., Staten Island Bus, Inc., 180 Jamaica Corp., Block 7932, Inc., G.V.D.
Leasing Co., Inc., Metro Affiliates, Inc., Midway Leasing Inc., Temporary
Transit Service, Inc. (each a "Guarantor" and collectively, "Guarantors"),
Assignor, and Assignee have entered into an Indenture dated the date hereof
(together with all supplements and amendments thereto and all extensions,
renewals, restatements and replacements thereof, the "Indenture"), and a
Security and Pledge Agreement dated the date hereof (together with all
supplements and amendments thereto and all extensions, renewals, restatements
and replacements thereof, the "Security Agreement," and such Indenture and
Security Agreement together with all agreements, instruments and documents now
or hereafter entered into or delivered in connection therewith, collectively,
the "Financing Agreements"), pursuant to which Indenture Assignor has issued
certain Notes (as defined in the Indenture);

            WHEREAS, Assignor is the sole stockholder of each Guarantor;

            WHEREAS, Assignor owns all right, title and interest in and to,
among other things, certain United States and foreign trademarks and service
marks, trademark and service mark registrations, and trademark and service mark
applications and trade names, including, but not limited to, those set forth on
Exhibit 1 hereto (the "Trademarks"), which are used in the business of one or
more of the Guarantors, along with the goodwill of the business symbolized
thereby and the Licenses (as hereinafter defined);

            WHEREAS, in order to secure its Obligations (as defined in the
Indenture), Assignor has agreed to grant to Assignee a security interest and
continuing lien in, to and under the Trade-
<PAGE>

marks, the goodwill and the Licenses and certain other assets with respect to
the Trademarks, the goodwill and the Licenses, as further set forth herein, and
Assignee has requested Assignor to enter into this Agreement to evidence such
security interest;

            NOW THEREFORE, KNOW ALL MEN BY THESE PRESENTS, that for valuable
consideration received and to be received, as security for the full payment and
performance of the Obligations, and to induce Assignee to act as trustee
pursuant to the Indenture, Assignor hereby grants to Assignee a security
interest in and continuing lien on Assignor's right, title and interest in, to
and under the following property of Assignor to the extent, and only the extent,
that the following property is part of, is related to, or arises in connection
with or from any "Accounts" or "Inventory" (as such terms are defined in the
Security Agreement):

            (a)   the Trademarks;

            (b)   all applications and registrations of the Trademarks in any
                  state of the United States and any foreign countries and
                  localities;

            (c)   all trade names, trademarks and service marks and trademark
                  and service mark applications and registrations hereafter
                  adopted or acquired and used by Assignor or any Guarantor in
                  its business, including, but not limited to, those which are
                  based upon or derived from the Trademarks or any variations
                  thereof (the "Future Trademarks");

            (d)   all extensions, renewals, and continuations of the Trade marks
                  and Future Trademarks and the registrations and applications
                  referred to in clause (b) above;

            (e)   all rights to sue for past, present and future infringements
                  of the Trademarks and Future Trademarks, and any trade marks
                  and service marks covered by any licenses of trademarks,
                  trademark applications or registrations, or trade names used
                  in the business of one or more of Guarantors and under which
                  Assignor is licensee, to the extent that the assignment
                  thereof will not result in Assignor's loss of the benefits
                  thereof ("Licenses");

            (f)   all packaging, labeling, trade names, service marks, logos,
                  and trade dress including or containing the Trademarks, Future
                  Trademarks, and the trademarks and service marks covered by
                  the Licenses, or a representation thereof, or any variation
                  thereof;


                                       2
<PAGE>

            (g)   all licenses and other agreements under which Assignor is
                  licensor, and all fees, rents, royalties, proceeds or monies
                  thereunder, relating to the Trademarks, Future Trademarks, and
                  the trademarks and service marks covered by the Licenses, and
                  the use thereof, to the extent that the assignment thereof
                  will not result in Assignor's loss of the benefits thereof;

            (h)   all good will of Assignor's business connected with,
                  symbolized by or in any way related to the items set forth in
                  clauses (a) through (g) above; and

            (i)   all proceeds of the foregoing, including without limitation,
                  license royalties, income, payments, claims, damages and
                  proceeds of suit.

All of the foregoing items set forth in clauses (a) through (i) are hereinafter
referred to collectively as the "Collateral".

            Assignor hereby covenants with Assignee as follows:

            1. Defined Terms. As used herein, capitalized terms defined in the
Security Agreement and not otherwise defined herein are used herein as so
defined.

            2. Assignor's Obligations. Assignor agrees that, notwithstanding
this Agreement, it will perform and discharge and remain liable for all its
covenants, duties, and obligations arising in connection with the Collateral and
any licenses and agreements related thereto. Assignee shall have no obligation
or liability in connection with the Collateral or any licenses or agreements
relating thereto by reason of this Agreement or any payment received by Assignee
relating to the Collateral and Assignee shall not be required to perform any
covenant, duty or obligation of Assignor arising in connection with the
Collateral or any license or agreement related thereto or to take any other
action regarding the Collateral or any such licenses or agreement, except and
only to the extent that Assignee has acquired absolute ownership of the
Collateral upon an exercise of its remedies under Section 5 hereof.

            3. Representations and Warranties. Assignor represents and warrants
to Assignee that: (a) Assignor is the beneficial and record owner of the
Collateral, and no adverse claims have been made with respect to its title to or
the validity of the Collateral; (b) the Trademarks and the trademarks and
service marks covered by the Licenses are the only trademarks, service marks,
trademark and service mark registrations and applications therefor and trade
names in which Assignor has any or all right, title and interest; (c) none of
the Collateral is subject to any mortgage, pledge, lien, security interest,
lease, charge, encumbrance, settlement or consent, 


                                       3
<PAGE>

covenant not to sue, non-assertion assurance, release or license (by Assignor as
licensor), except as set forth on Exhibit 1; (d) Assignor has performed all acts
and has paid all renewal, maintenance and other fees and taxes required to
maintain each and every registration and application of the Collateral in full
force and effect; (e) no claims have been made that the use of any of the
Collateral violates the asserted rights of any third party; (f) to the best of
Assignor's knowledge, no third party is infringing upon any of the Collateral;
and (g) when this Agreement is filed in and recorded by the United States Patent
and Trademark Office (the "Trademark Office") and the Assignee has taken the
other actions contemplated by the Security Agreement and in this Agreement, this
Agreement will create a legal and valid perfected and continuing lien on and
security interest in the Collateral in favor of Assignee, enforceable against
Assignor and all third parties, subject to no other mortgage, lien, charge,
encumbrance, or security or other interest except as expressly permitted by the
Intercreditor Agreement.

            4. Covenants. Assignor will maintain and renew all items of
Collateral necessary for the conduct of its or any Guarantor's business and all
registrations of the Collateral necessary for the conduct of its or any
Guarantor's business and will defend the Collateral against the claims of all
persons. Assignor will maintain, and will cause each Guarantor or other person
that uses the Collateral to maintain, the same standards of quality for the
goods and services in connection with which the Trademarks and the trademarks
covered by the Licenses are used as Assignor or such other persons maintained
for such goods and services prior to entering into this Agreement. Assignee
shall have the right to enter upon Assignor's premises at all reasonable times
to monitor such quality standards. Assignor shall promptly notify Assignee if it
knows or has reason to know that any of the Collateral may become subject to any
adverse determination or development (including the institution of proceedings)
in any action or proceeding in the United States Patent and Trademark Office or
any court. In the event that any of the Collateral is infringed or diluted by a
third party, promptly after Assignor becomes aware of such infringement or
dilution, Assignor shall take all reasonable actions to stop such infringement
or dilution and protect its exclusive rights in such Collateral including, but
not limited to, the initiation of a suit for injunctive relief and to recover
damages. Without limiting the generality of the foregoing, Assignor shall not
permit the expiration, termination or abandonment of any Trademark, Future
Trademark or License used in or necessary for the conduct of its or any
Borrower's business without the prior written consent of Assignee. If, before
the Obligations have been satisfied in full, Assignor shall obtain rights to or
be licensed to use any new trademark, or become entitled to the benefit of any
trademark or service mark application or trademark or service mark registration
not identified on Exhibit 1 hereto, the provisions of Section 2 hereof shall
automatically apply


                                       4
<PAGE>

thereto and Assignor shall give Assignee prompt notice thereof in writing.

            5. Remedies Upon Default. Whenever any Event of Default shall occur
and be continuing, Assignee shall have all the rights and remedies granted to it
in such event by the Security Agreement and the other Financing Agreements,
which rights and remedies are specifically incorporated herein by reference and
made a part hereof. Assignee in such event may collect directly any payments due
to Assignor in respect of the Collateral and, subject to any limitations imposed
under any license agreements constituting part of the Collateral, may sell,
license, lease, assign, or otherwise dispose of the Collateral in the manner set
forth in the Security Agreement or the other Financing Agreements. Assignor
agrees that, in the event of any disposition of the Collateral upon any such
Event of Default, it will duly execute, acknowledge, and deliver all documents
necessary or advisable to record title to the Collateral in any transferee or
transferees thereof, including, without limitation, valid, recordable
assignments of the Trademarks, Future Trademarks, and Licenses. In the event
Assignor fails or refuses to execute and deliver such documents, Assignor hereby
irrevocably appoints Assignee as its attorney-in-fact, with power of
substitution, to execute, deliver, and record any such documents on Assignor's
behalf. Notwithstanding any provision hereof to the contrary, during the
continuance of an Event of Default, Assignor may sell, and permit Guarantors to
sell, merchandise or services bearing the Trademarks, Future Trademarks, and
trademarks covered by the Licenses in the ordinary course of their respective
business and in a manner consistent with its past practices, until it receives
written notice from Assignee of an intended sale or disposition of the
Collateral. The preceding sentence shall not limit any right or remedy granted
to Assignee with respect to Assignor's inventory under the Security Agreement or
any other agreement now or hereinafter in effect.

            6. Power of Attorney. Concurrently with the execution and delivery
hereof, Assignor shall execute and delivery to the Assignee, in the form of
Exhibit 2 hereto, five (5) originals of a Special Power of Attorney for the
implementation of the assignment, sale, license, lease or other disposition of
the Trademarks, Future Trademarks, and Licenses pursuant to Section 5. Assignor
hereby releases Assignee from any claims, causes of action and demands at any
time arising out of or with respect to any actions taken or omitted to be taken
by Assignee in accordance with Section 5 under the powers of attorney granted
therein, other than actions taken or omitted to be taken through the bad faith,
willful misconduct or gross negligence of Assignee, as determined by a final,
non-appealable order of a court of competent jurisdiction.

            7. Cumulative Remedies. The rights and remedies provided herein are
cumulative and not exclusive of any other 


                                       5
<PAGE>

rights or remedies provided by law. The security interest granted hereby is
granted in conjunction with the security interest granted to Assignee under the
Security Agreement. The rights and remedies of Assignee with respect to the
security interest granted hereby are in addition to those set forth in the
Security Agreement and the other Financing Agreements and those which are now or
hereafter available to Assignee as a matter of law or equity. The exercise by
Assignee of any one or more of the rights, powers or remedies provided for in
this Agreement, in the Security Agreement, in the other Financing Agreements or
now or hereafter existing at law or in equity shall not preclude the
simultaneous or later exercise by any person, including Assignee, of any or all
other rights, powers or remedies. The rights and remedies provided herein are
intended to be in addition to and not in substitution of the rights and remedies
provided by the Security Agreement.

            8. Amendments and Waivers. This Agreement may not be modified,
supplemented, or amended, or any of its provisions waived at the request of
Assignor, without the prior written consent of Assignee. Assignor hereby
authorizes Assignee to modify this Agreement by amending Exhibit 1 hereto to
include any Future Trademarks or additional licenses.

            9. Waiver of Rights. No course of dealing between the parties to
this Agreement or any failure or delay on the part of any such party in
exercising any rights or remedies hereunder shall operate as a waiver of any
rights and remedies of such party or any other party, and no single or partial
exercise of any rights or remedies by one party hereunder shall operate as a
waiver or preclude the exercise of any other rights and remedies of such party
or any other party. No waiver by Assignee of any breach or default by Assignor
shall be deemed a waiver of any other previous breach or default or of any
breach or default occurring thereafter.

            10. Assignment. The provisions of this Agreement shall be binding
upon and inure to the benefit of the respective successors and assigns of the
parties hereto; provided, however, that no interest herein or in or to the
Collateral may be assigned by Assignor without the prior written consent of
Assignee; and, provided, further, that the Assignee may assign the rights and
benefits hereof to any party acquiring any interest in the Obligations or any
part hereof.

            11. Further Acts. Assignor shall have the duty to prosecute
diligently any application for the Trademarks and Future Trademarks necessary
for the conduct of its or any Guarantor's business pending as of the date of
this Agreement or thereafter, until the Obligations shall have been paid in
full, and to make applications on material unregistered but registrable
trademarks necessary for the conduct of its or any Guarantor's business in any
location where Assignor does business and to preserve and maintain all rights in
the Trademarks and the other 


                                       6
<PAGE>

Collateral necessary for the conduct of its or any Guarantor's business. Any
expenses incurred in connection with such applications shall be borne by
Assignor. Assignor shall not abandon any right to file a trademark or service
mark application or registration for any trademark or service mark used in or
necessary for the conduct of its business, or abandon any such pending trademark
application or registration necessary for the conduct of its or any Guarantor's
business, without the consent of Assignee.

            12. Enforcement. Upon Assignor's failure to do so after Assignee's
demand, or upon an Event of Default, Assignee shall have the right but shall in
no way be obligated to bring suit in its own name to enforce the Trademarks,
Future Trademarks, Licenses, or the trademarks covered by the Licenses, and any
license under any of the foregoing, in which event Assignor shall at the request
of Assignee do any and all lawful acts and execute any and all proper documents
that may be reasonably requested by Assignee in aid of such enforcement
including, but not limited to, joining as a plaintiff in any such enforcement
action and Assignor shall promptly, upon demand, reimburse and indemnify
Assignee or its agents for all reasonable costs and expenses incurred by
Assignee in the exercise of its rights under this Section 12.

            13. Release and Re-Assignment. At such time as all of the
Obligations have been satisfied, and the Financing Agreements have been
terminated, other than upon enforcement of Assignee's remedies under the
Financing Agreements after an Event of Default, Assignee will execute and
deliver to Assignor all deeds, assignments and other instruments as may be
necessary or proper to release Assignor's lien in the Collateral and reassign to
Assignee any and all rights of Assignor therein which were granted to Assignor
hereunder, subject to any dispositions thereof which may have been made by
Assignee pursuant hereto.

            14. Severability. If any clause or provision of this Agreement shall
be held invalid or unenforceable, in whole or in part, in any jurisdiction, such
invalidity or unenforceability shall attach only to such clause or provision, or
part thereof, in such jurisdiction, and shall not in any manner affect any other
clause or provision in any other jurisdiction.

            15. Notices. All notices, requests and demands to or upon Assignor
or Assignee under this Agreement shall be given in the manner prescribed by
the Security Agreement.

            16. Governing Law. This Agreement shall be governed by and
construed, applied, and enforced in accordance with the federal laws of the
United States of America applicable to trademarks and the laws of the State of
New York, except that no doctrine of choice of law shall be used to apply the
laws of any other state or jurisdiction.



                                       7
<PAGE>

            17. Financing Agreement. This Agreement is one of the Financing
Agreements.


                                       8
<PAGE>

            IN WITNESS WHEREOF, the parties have entered into this Agreement as
of the date first above written.


                              ATLANTIC EXPRESS TRANSPORTATION
                              CORP., Assignor



                              By:  /s/ Domenic Gatto
                                 -------------------------------
                                   Name:   Domenic Gatto
                                   Title:  President



                              THE BANK OF NEW YORK, as
                              Trustee, Assignee



                              By:  /s/ Stephen J. Giurlando
                                 -------------------------------
                                   Name:   Stephen J. Giurlando
                                   Title:  Assistant Vice President
<PAGE>

STATE OF NEW YORK       )
                        ) ss:
COUNTY OF NEW YORK      )


            On the 4th day of February 1997 before me personally came
Domenic Gatto, to me known, who being by me duly sworn, did depose and say
that he is the President of ATLANTIC EXPRESS TRANSPORTATION CORP., the
corporation described in and which executed the foregoing instrument; and that
he signed his name thereto by order of the Board of Directors of said
corporation.


                              /s/ J. Weston Peterson
                              -------------------------------
                              Notary Public


STATE OF NEW YORK       )
                        ) ss:
COUNTY OF NEW YORK      )


            On the ____ day of February 1997 before me personally came
_______________, to me known, who being by me duly sworn, did depose and say
that he is a _______________ of THE BANK OF NEW YORK, the corporation described
in and which executed the foregoing instrument; and that he signed his name
thereto by order of the Board of Directors of said corporation.


                              _____________________________
                              Notary Public
<PAGE>

                                                                       EXHIBIT 1

                          LIST OF ASSIGNOR'S TRADEMARKS


Registered Trademarks         U.S.                    Date
and Service Marks             Registration No.        Registered
-----------------             ----------------        ----------

ATLANTIC EXPRESS              1,964,915               4/2/96

AE (Stylized Letters)         1,667,874               12/10/91



Applications                  U.S.
for Registration              Serial No.              Date Filed
----------------              ----------              ----------

AE ATLANTIC EXPRESS
  TRANSPORTATION GROUP        75-121,810              6/18/96
  and DESIGN


Trade Names
-----------

ATLANTIC EXPRESS
<PAGE>


STATE OF NEW YORK       )
                        )ss:
COUNTY OF NEW YORK      )


On the 4th day of February 1997 before me personally came Domenic Gatto, to me
known, who being be me duly sworn, did depose and say that he is the
President of ATLANTIC EXPRESS TRANSPORTATION CORP., the corporation
described in and which executed the foregoing instrument; and that he signed his
name thereto by order of the Board of Directors of said Corporation.


                              /s/ J. Weston Peterson
                              -------------------------------
                              Notary Public
<PAGE>

                                                                       EXHIBIT 2

                            SPECIAL POWER OF ATTORNEY

STATE OF NEW YORK       )
                        ) ss.
COUNTY OF NEW YORK      )


            KNOW ALL MEN BY THESE PRESENTS, THAT ATLANTIC EXPRESS TRANSPORTATION
CORP., a New York corporation with offices at 7 North Street, Staten Island, New
York 10302 (hereinafter called "Assignor"), hereby appoints and constitutes THE
BANK OF NEW YORK, a New York banking corporation with offices at 101 Barclay
Street -- 21W, New York, New York 10286 (hereinafter called "Assignee"), its
true and lawful attorney, with full power of substitution, and with full power
and authority to perform the following acts on behalf of Assignor:

            1. For the purpose of assigning, selling, licensing or otherwise
disposing of all right, title and interest of Assignor in and to any trademarks
and service marks, and all registrations, renewals, recordings and all pending
applications therefor, and all licenses therefor, and for the purpose of the
recording, registering and filing of, or accomplishing any other formality with
respect to, the foregoing, to execute and deliver any and all agreements,
documents, instruments of assignment or other papers necessary or advisable to
effect such purpose; and

            2. To execute any and all documents, statements, certificates or
other papers necessary or advisable in order to obtain the purposes described
above as Assignee may in its sole discretion determine.

            This power of attorney is made pursuant to a Collateral Assignment
of Trademarks (Security Agreement) dated the date hereof, between Assignor and
Assignee and takes effect solely for the purposes of Section 5 thereof and is
subject to the conditions thereof and may not be revoked until the payment in
full of all "Obligations" as defined in such Security Agreement.

Dated:  February 4, 1997


                              ATLANTIC EXPRESS TRANSPORTATION
                              CORP.

                              By: /s/ Domenic Gatto
                                 -------------------------------
                                    Name:  Domenic Gatto
                                    Title: President