<PAGE>

                                                                    EXHIBIT 10.4

                 THIRD AMENDMENT AND CONSENT TO CREDIT AGREEMENT
                 -----------------------------------------------

          THIRD AMENDMENT AND CONSENT TO CREDIT AGREEMENT (this "Amendment"),
dated as of April 17, 2001, among U.S.I. HOLDINGS CORPORATION, a Delaware
corporation (the "Borrower"), the various lenders from time to time party to the
Credit Agreement referred to below (the "Lenders"), CREDIT LYONNAIS CAYMAN
ISLAND BRANCH, as Administrative Agent (the "Administrative Agent") and THE
CHASE MANHATTAN BANK, as Syndication Agent (the "Syndication Agent"). All
capitalized terms used herein and not otherwise defined shall have the
respective meanings assigned to such terms in the Credit Agreement.

                                   W I T N E S S E T H:
                                   - - - - - - - - - -

          WHEREAS, the Borrower, the Lenders, the Administrative Agent and the
Syndication Agent are parties to a Credit Agreement, dated as of September 17,
1999 (as amended, modified or supplemented to the date hereof, the "Credit
Agreement");

          WHEREAS, the Borrower has requested that the Lenders agree to certain
amendments and consents in connection with the Credit Agreement as herein
provided; and

          WHEREAS, the Lenders have agreed to the amendments and consents in
connection with the Credit Agreement as herein provided, subject to the terms
and conditions set forth herein;

          NOW, THEREFORE, the parties hereto agree as follows:

          1. Notwithstanding anything to the contrary contained in the Credit
Agreement, the Borrower or any of its Subsidiaries may acquire all or
substantially all of the assets of the Zehnder/Innes book of business pursuant
to terms substantially consistent with those identified to the Lenders prior to
the Third Amendment Effective Date (as defined below) for aggregate
consideration of approximately 3.0 times the pro forma EBITDA of such entity
(which pro forma EBITDA at the date hereof is approximately $700,000) and
further provided that such acquisition shall be excluded from the application of
subparagraph (h) of the definition of Permitted Acquisition in Section 1.1 of
the Credit Agreement, and not counted toward the $10,000,000 limitation
contained therein.

          2. Notwithstanding anything to the contrary contained in the Credit
Agreement, the Borrower and its Subsidiaries may effect the mergers and
dissolutions contemplated by the Revised USI Proposed Legal Entity Consolidation
Memorandum dated September, 2000 (a copy of which has been delivered to each
Lender), substantially on the terms set forth in such Memorandum, and the
Administrative Agent is hereby authorized to take any and all action reasonably
requested by the Borrower to effect such mergers and dissolutions.

          3. On or prior to the tenth day following the Third Amendment
Effective Date, (a) USIS will grant to the Administrative Agent for the benefit
of the Lenders a first priority

<PAGE>

perfected security interest in 100% of the outstanding capital stock of
Henderson and Phillips, Inc., (b) USIS shall deliver to the Administrative Agent
a new stock certificate (and related stock power) representing 100% of the
outstanding capital stock of USI Midwest, Inc. and reflecting the name change of
Thomas E. Wood, Inc., to USI Midwest, Inc., (c) USI Benefit Group, Inc. shall
guaranty all Obligations by becoming party to the Subsidiary Guaranty in a
manner and pursuant to documentation in form and substance satisfactory to the
Administrative Agent, and (d) USI Benefit Group, Inc., will grant to the
Administrative Agent for the benefit of the Lenders a first priority perfected
security interest in 100% of the capital stock of USI Administrators, Inc., a
Delaware corporation.

          4. The Borrower hereby represents and warrants that (x) all
representations and warranties contained in Section 4 of the Credit Agreement
are true and correct in all material respects on and as of the Third Amendment
Effective Date after giving effect to this Amendment (unless such
representations and warranties relate to a specific earlier date, in which case
such representations and warranties shall be true and correct as of such earlier
date) and (y) there exists no Default or Event of Default on the Third Amendment
Effective Date, after giving effect to this Amendment.

          5. This Amendment is limited as specified and shall not constitute a
modification, acceptance or waiver of any provision of the Credit Agreement or
any other Loan Document except as expressly set forth herein.

          6. This Amendment may be executed in any number of counterparts and by
the different parties hereto on separate counterparts, each of which
counterparts when executed and delivered shall be an original, but all of which
shall together constitute one and the same instrument. A complete set of
counterparts shall be lodged with the Borrower and the Administrative Agent.

          7. THIS AMENDMENT AND THE RIGHTS AND OBLIGATIONS OF THE PARTIES
HEREUNDER SHALL BE CONSTRUED IN ACCORDANCE WITH AND GOVERNED BY THE LAW OF THE
STATE OF NEW YORK.

          8. This Amendment shall become effective as of the date hereof on the
date (the "Third Amendment Effective Date") when the Borrower and the Required
Lenders shall have signed a counterpart hereof (whether the same or different
counterparts) and shall have delivered (including by way of facsimile
transmission) the same to the Administrative Agent at its notice office.

          9. From and after the Third Amendment Effective Date, all references
in the Credit Agreement and each of the Loan Documents to the Credit Agreement
shall be deemed to be references to the Credit Agreement after giving effect to
this Amendment.

                                      * * *

                                        2

<PAGE>

          IN WITNESS WHEREOF, the parties hereto have caused a counterpart of
this Amendment to be duly executed and delivered as of the date first above
written.

                                             U.S.I. HOLDINGS CORPORATION


                                             By /s/ Bernard H. Mizel
                                               ---------------------------------
                                                Name:  Bernard H. Mizel
                                                Title: CEO


                                             CREDIT LYONNAIS CAYMAN ISLAND
                                                BRANCH


                                             By /s/ W. Michael George
                                               ---------------------------------
                                                Name:  W. Michael George
                                                Title: Authorized Signature


                                             THE CHASE MANHATTAN BANK


                                             By /s/ Elizabeth H. Schwobe
                                               ---------------------------------
                                                Name:  Elizabeth H. Schwobe
                                                Title: Managing Director


                                             FIRSTAR BANK, N.A.


                                             By
                                               ---------------------------------
                                                Name:
                                                Title:


                                             LASALLE NATIONAL BANK


                                             By /s/ George L. Kumis
                                               ---------------------------------
                                                Name:  George L. Kumis
                                                Title: