EXHIBIT 4(e)


                          ____________________________


                               GUARANTEE AGREEMENT
                                 BY AND BETWEEN
                            U.S.B. HOLDING CO., INC.

                                       AND

                       STATE STREET BANK AND TRUST COMPANY
                      OF CONNECTICUT, NATIONAL ASSOCIATION

                            DATED AS OF JULY 31, 2001

                         _____________________________

<PAGE>

                               GUARANTEE AGREEMENT

         This GUARANTEE AGREEMENT (the "Guarantee"), dated as of July 31, 2001,
is executed and delivered by U.S.B. Holding Co., Inc., a Delaware corporation
(the "Guarantor"), and State Street Bank and Trust Company of Connecticut,
National Association, a national banking association, organized under the laws
of the United States of America, as trustee (the "Guarantee Trustee"), for the
benefit of the Holders (as defined herein) from time to time of the Capital
Securities (as defined herein) of Union State Statutory Trust II, a Connecticut
statutory trust (the "Issuer").

         WHEREAS, pursuant to an Amended and Restated Declaration of Trust (the
"Declaration"), dated as of the date hereof among State Street Bank and Trust
Company of Connecticut, National Association, not in its individual capacity but
solely as institutional trustee, the administrators of the Issuer named therein,
the Guarantor, as sponsor, and the holders from time to time of undivided
beneficial interests in the assets of the Issuer, the Issuer is issuing on the
date hereof those undivided beneficial interests, having an aggregate
liquidation amount of $20,000,000, (the "Capital Securities'); and

         WHEREAS, as incentive for the Holders to purchase the Capital
Securities, the Guarantor desires irrevocably and unconditionally to agree, to
the extent set forth in this Guarantee, to pay to the Holders of Capital
Securities the Guarantee Payments (as defined herein) and to make certain other
payments on the terms and conditions set forth herein;

         NOW, THEREFORE, in consideration of the purchase by each Holder of the
Capital Securities, which purchase the Guarantor hereby agrees shall benefit the
Guarantor, the Guarantor executes and delivers this Guarantee for the benefit of
the Holders.

                                   ARTICLE I

                         DEFINITIONS AND INTERPRETATION

         SECTION 1.1. DEFINITIONS AND INTERPRETATION. In this Guarantee, unless
the context otherwise requires:

         (a) capitalized terms used in this Guarantee but not defined in the
preamble above have the respective meanings assigned to them in this Section
1.1;

         (b) a term defined anywhere in this Guarantee has the same meaning
throughout;

         (c) all references to "the Guarantee" or "this Guarantee" are to this
Guarantee as modified, supplemented or amended from time to time;

         (d) all references in this Guarantee to "Articles" or "Sections" are to
Articles or Sections of this Guarantee, unless otherwise specified;

         (e) terms defined in the Declaration as at the date of execution of
this Guarantee have the same meanings when used in this Guarantee, unless
otherwise defined in this Guarantee or unless the context otherwise requires;
and

         (f) a reference to the singular includes the plural and vice versa.

         "Affiliate" has the same meaning as given to that term in Rule 405 of
the Securities Act of 1933, as amended, or any successor rule thereunder.


<PAGE>

         "Beneficiaries" means any Person to whom the Issuer is or hereafter
becomes indebted or liable.

         "Capital Securities" has the meaning set forth in the recitals to this
Guarantee.

         "Common Securities" means the common securities issued by the Issuer to
the Guarantor pursuant to the Declaration.

         "Corporate Trust Office" means the office of the Guarantee Trustee at
which the corporate trust business of the Guarantee Trustee shall, at any
particular time, be principally administered, which office at the date of
execution of this Guarantee Agreement is located at 225 Asylum Street, Goodwin
Square, Hartford, Connecticut 06103.

         "Covered Person" means any Holder of Capital Securities.

         "Debentures" means the debt securities of the Guarantor designated the
Floating Rate Junior Subordinated Deferrable Interest Debentures due 2031 held
by the Institutional Trustee (as defined in the Declaration) of the Issuer.

         "Declaration Event of Default" means an "Event of Default" as defined
in the Declaration.

         "Event of Default" has the meaning set forth in Section 2.4(a).

         "Guarantee Payments" means the following payments or distributions,
without duplication, with respect to the Capital Securities, to the extent not
paid or made by the Issuer: (i) any accrued and unpaid Distributions (as defined
in the Declaration) which are required to be paid on such Capital Securities to
the extent the Issuer shall have funds available therefor, (ii) the Redemption
Price to the extent the Issuer has funds available therefor, with respect to any
Capital Securities called for redemption by the Issuer, (iii) the Special
Redemption Price to the extent the Issuer has funds available therefor, with
respect to Capital Securities redeemed upon the occurrence of a Special Event,
and (iv) upon a voluntary or involuntary liquidation, dissolution, winding-up or
termination of the Issuer (other than in connection with the distribution of
Debentures to the Holders of the Capital Securities in exchange therefor as
provided in the Declaration), the lesser of (a) the aggregate of the liquidation
amount and all accrued and unpaid Distributions on the Capital Securities to the
date of payment, to the extent the Issuer shall have funds available therefor,
and (b) the amount of assets of the Issuer remaining available for distribution
to Holders in liquidation of the Issuer (in either case, the "Liquidation
Distribution").

         "Guarantee Trustee" means State Street Bank and Trust Company of
Connecticut, National Association, until a Successor Guarantee Trustee has been
appointed and has accepted such appointment pursuant to the terms of this
Guarantee and thereafter means each such Successor Guarantee Trustee.

         "Guarantor" means U.S.B. Holding Co., Inc. and each of its successors
and assigns.

         "Holder" means any holder, as registered on the books and records of
the Issuer, of any Capital Securities; provided, however, that, in determining
whether the Holders of the requisite percentage of Capital Securities have given
any request, notice, consent or waiver hereunder, "Holder" shall not include the
Guarantor or any Affiliate of the Guarantor.

         "Indemnified Person" means the Guarantee Trustee, any Affiliate of the
Guarantee Trustee, or any officers, directors, shareholders, members, partners,
employees, representatives, nominees, custodians or agents of the Guarantee
Trustee.

                                       2
<PAGE>

         "Indenture" means the Indenture dated as of the date hereof between the
Guarantor and State Street Bank and Trust Company of Connecticut, National
Association, not in its individual capacity but solely as trustee, and any
indenture supplemental thereto pursuant to which the Debentures are to be issued
to the institutional trustee of the Issuer.

         "Issuer" has the meaning set forth in the opening paragraph to this
Guarantee.

         "Liquidation Distibution" has the meaning set forth in the definition
of "Guarantee Payments" herein.

         "Majority in liquidation amount of the Capital Securities" means
Holder(s) of outstanding Capital Securities, voting together as a class, but
separately from the holders of Common Securities, of more than 50% of the
aggregate liquidation amount (including the stated amount that would be paid on
redemption, liquidation or otherwise, plus accrued and unpaid Distributions to
the date upon which the voting percentages are determined) of all Capital
Securities then outstanding.

         "Obligations" means any costs, expenses or liabilities (but not
including liabilities related to taxes) of the Issuer other than obligations of
the Issuer to pay to holders of any Trust Securities the amounts due such
holders pursuant to the terms of the Trust Securities.

         "Officer's Certificate" means, with respect to any Person, a
certificate signed by one Authorized Officer of such Person. Any Officer's
Certificate delivered with respect to compliance with a condition or covenant
provided for in this Guarantee shall include:

                  (a) a statement that each officer signing the Officer's
         Certificate has read the covenant or condition and the definitions
         relating thereto;

                  (b) a brief statement of the nature and scope of the
         examination or investigation undertaken by each officer in rendering
         the Officer's Certificate;

                  (c) a statement that each such officer has made such
         examination or investigation as, in such officer's opinion, is
         necessary to enable such officer to express an informed opinion as to
         whether or not such covenant or condition has been complied with; and

                  (d) a statement as to whether, in the opinion of each such
         officer, such condition or covenant has been complied with.

         "Person" means a legal person, including any individual, corporation,
estate, partnership, joint venture, association, joint stock company, limited
liability company, trust, unincorporated association, or government or any
agency or political subdivision thereof, or any other entity of whatever nature.

         "Redemption Price" has the meaning set forth in the Indenture.

         "Responsible Officer" means, with respect to the Guarantee Trustee, any
officer within the Corporate Trust Office of the Guarantee Trustee including any
Vice President, Assistant Vice President, Secretary, Assistant Secretary or any
other officer of the Guarantee Trustee customarily performing functions similar
to those performed by any of the above designated officers and also, with
respect to a particular corporate trust matter, any other officer to whom such
matter is referred because of that officer's knowledge of and familiarity with
the particular subject.

         "Special Event" has the meaning set forth in the Indenture.

         "Special Redemption Price" has the meaning set forth in the Indenture.

                                        3

<PAGE>

         "Successor Guarantee Trustee" means a successor Guarantee Trustee
possessing the qualifications to act as Guarantee Trustee under Section 3.1.

         "Trust Securities" means the Common Securities and the Capital
Securities.

                                   ARTICLE II

                          POWERS, DUTIES AND RIGHTS OF
                                GUARANTEE TRUSTEE

         SECTION 2.1. POWERS AND DUTIES OF THE GUARANTEE TRUSTEE.

         (a) This Guarantee shall be held by the Guarantee Trustee for the
benefit of the Holders of the Capital Securities, and the Guarantee Trustee
shall not transfer this Guarantee to any Person except a Holder of Capital
Securities exercising his or her rights pursuant to Section 4.4(b) or to a
Successor Guarantee Trustee on acceptance by such Successor Guarantee Trustee of
its appointment to act as Successor Guarantee Trustee. The right, title and
interest of the Guarantee Trustee shall automatically vest in any Successor
Guarantee Trustee, and such vesting and cessation of title shall be effective
whether or not conveyancing documents have been executed and delivered pursuant
to the appointment of such Successor Guarantee Trustee.

         (b) If an Event of Default actually known to a Responsible Officer of
the Guarantee Trustee has occurred and is continuing, the Guarantee Trustee
shall enforce this Guarantee for the benefit of the Holders of the Capital
Securities.

         (c) The Guarantee Trustee, before the occurrence of any Event of
Default and after curing all Events of Default that may have occurred, shall
undertake to perform only such duties as are specifically set forth in this
Guarantee, and no implied covenants shall be read into this Guarantee against
the Guarantee Trustee. In case an Event of Default has occurred (that has not
been cured or waived pursuant to Section 2.4) and is actually known to a
Responsible Officer of the Guarantee Trustee, the Guarantee Trustee shall
exercise such of the rights and powers vested in it by this Guarantee, and use
the same degree of care and skill in its exercise thereof, as a prudent person
would exercise or use under the circumstances in the conduct of his or her own
affairs.

         (d) No provision of this Guarantee shall be construed to relieve the
Guarantee Trustee from liability for its own negligent action, its own negligent
failure to act, or its own willful misconduct, except that:

                  (i) prior to the occurrence of any Event of Default and after
          the curing or waiving of all such Events of Default that may have
          occurred:

                           (A) the duties and obligations of the Guarantee
                  Trustee shall be determined solely by the express provisions
                  of this Guarantee, and the Guarantee Trustee shall not be
                  liable except for the performance of such duties and
                  obligations as are specifically set forth in this Guarantee,
                  and no implied covenants or obligations shall be read into
                  this Guarantee against the Guarantee Trustee; and

                           (b) in the absence of bad faith on the part of the
                  Guarantee Trustee, the Guarantee Trustee may conclusively
                  rely, as to the truth of the statements and the correctness of
                  the opinions expressed therein, upon any certificates or
                  opinions furnished to the Guarantee Trustee and conforming to
                  the requirements of this Guarantee; but in the case of any
                  such certificates or opinions that by any provision hereof are
                  specifically

                                        4


<PAGE>

                  required to be furnished to the Guarantee Trustee, the
                  Guarantee Trustee shall be under a duty to examine the same to
                  determine whether or not they conform to the requirements of
                  this Guarantee;

                  (ii) the Guarantee Trustee shall not be liable for any error
         of judgment made in good faith by a Responsible Officer of the
         Guarantee Trustee, unless it shall be proved that such Responsible
         Officer of the Guarantee Trustee or the Guarantee Trustee was negligent
         ascertaining the pertinent facts upon which such judgment was made;

                  (iii) the Guarantee Trustee shall not be liable with respect
         to any action taken or omitted to be taken by it in good faith in
         accordance with the direction of the Holders of not less than a
         Majority in liquidation amount of the Capital Securities relating to
         the time, method and place of conducting any proceeding for any remedy
         available to the Guarantee Trustee, or exercising any trust or power
         conferred upon the Guarantee Trustee under this Guarantee; and

                  (iv) no provision of this Guarantee shall require the
         Guarantee Trustee to expend or risk its own funds or otherwise incur
         personal financial liability in the performance of any of its duties
         or in the exercise of any of its rights or powers, if the Guarantee
         Trustee shall have reasonable grounds for believing that the repayment
         of such funds is not reasonably assured to it under the terms of this
         Guarantee or security and indemnity, reasonably satisfactory to the
         Guarantee Trustee, against such risk or liability is not reasonably
         assured to it.

         SECTION 2.2. CERTAIN RIGHTS OF GUARANTEE TRUSTEE.

         (a)      Subject to the provisions of Section 2.1:

                  (i) The Guarantee Trustee may conclusively rely, and shall be
         fully protected in acting or refraining from acting upon, any
         resolution, certificate, statement, instrument, opinion, report,
         notice, request, direction, consent, order, bond, debenture, note,
         other evidence of indebtedness or other paper or document believed by
         it to be genuine and to have been signed, sent or presented by the
         proper party or parties.

                  (ii) Any direction or act of the Guarantor contemplated by
         this Guarantee shall be sufficiently evidenced by an Officer's
         Certificate.

                  (iii) Whenever, in the administration of this Guarantee, the
         Guarantee Trustee shall deem it desirable that a matter be proved or
         established before taking, suffering or omitting any action hereunder,
         the Guarantee Trustee (unless other evidence is herein specifically
         prescribed) may, in the absence of bad faith on its part, request and
         conclusively rely upon an Officer's Certificate of the Guarantor which,
         upon receipt of such request, shall be promptly delivered by the
         Guarantor.

                  (iv) The Guarantee Trustee shall have no duty to see to any
         recording, filing or registration of any instrument (or any
         re-recording, refiling or re-registration thereof).

                  (v) The Guarantee Trustee may consult with counsel of its
         selection, and the advice or opinion of such counsel with respect to
         legal matters shall be full and complete authorization and protection
         in respect of any action taken, suffered or omitted by it hereunder in
         good faith and in accordance with such advice or opinion. Such counsel
         may be counsel to the Guarantor or any of its Affiliates and may
         include any of its employees. The Guarantee Trustee shall have the
         right at any time to seek instructions concerning the administration of
         this Guarantee from any court of competent jurisdiction.

                                        5


<PAGE>

                  (vi) The Guarantee Trustee shall be under no obligation to
         exercise any of the rights or powers vested in it by this Guarantee at
         the request or direction of any Holder, unless such Holder shall have
         provided to the Guarantee Trustee such security and indemnity,
         reasonably satisfactory to the Guarantee Trustee, against the costs,
         expenses (including attorneys' fees and expenses and the expenses of
         the Guarantee Trustee's agents, nominees or custodians) and liabilities
         that might be incurred by it in complying with such request or
         direction, including such reasonable advances as may be requested by
         the Guarantee Trustee; provided, however, that nothing contained in
         this Section 2.2(a)(vi) shall relieve the Guarantee Trustee, upon the
         occurrence of an Event of Default, of its obligation to exercise the
         rights and powers vested in it by this Guarantee.

                  (vii) The Guarantee Trustee shall not be bound to make any
         investigation into the facts or matters stated in any resolution,
         certificate, statement, instrument, opinion, report notice, request,
         direction, consent, order, bond, debenture, note, other evidence of
         indebtedness or other paper or document, but the Guarantee Trustee, in
         its discretion, may make such further inquiry or investigation into
         such facts or matters as it may see fit.

                  (viii) The Guarantee Trustee may execute any of the trusts or
         powers hereunder or perform any duties hereunder either directly or by
         or through agents, nominees, custodians or attorneys, and the Guarantee
         Trustee shall not be responsible for any misconduct or negligence on
         the part of any agent or attorney appointed with due care by it
         hereunder.

                  (ix) Any action taken by the Guarantee Trustee or its agents
         hereunder shall bind the Holders of the Capital Securities, and the
         signature of the Guarantee Trustee or its agents alone shall be
         sufficient and effective to perform any such action. No third party
         shall be required to inquire as to the authority of the Guarantee
         Trustee to so act or as to its compliance with any of the terms and
         provisions of this Guarantee, both of which shall be conclusively
         evidenced by the Guarantee Trustee's or its agent's taking such action.

                  (x) Whenever in the administration of this Guarantee the
         Guarantee Trustee shall deem it desirable to receive instructions with
         respect to enforcing any remedy or right or taking any other action
         hereunder, the Guarantee Trustee (i) may request instructions from the
         Holders of a Majority in liquidation amount of the Capital Securities,
         (iii) may refrain from enforcing such remedy or right or taking such
         other action until such instructions are received, and (iii) shall be
         protected in conclusively relying on or acting in accordance with such
         instructions.

                  (xi) The Guarantee Trustee shall not be liable for any action
         taken, suffered, or omitted to be taken by it in good faith, without
         negligence, and reasonably believed by it to be authorized or within
         the discretion or rights or powers conferred upon it by this Guarantee.

         (b) No provision of this Guarantee shall be deemed to impose any duty
or obligation on the Guarantee Trustee to perform any act or acts or exercise
any right, power, duty or obligation conferred or imposed on it, in any
jurisdiction in which it shall be illegal or in which the Guarantee Trustee
shall be unqualified or incompetent in accordance with applicable law to perform
any such act or acts or to exercise any such right, power, duty or obligation.
No permissive power or authority available to the Guarantee Trustee shall be
construed to be a duty.

         SECTION 2.3. NOT RESPONSIBLE FOR RECITALS OR ISSUANCE OF GUARANTEE. The
recitals contained in this Guarantee shall be taken as the statements of the
Guarantor, and the Guarantee Trustee does not assume any responsibility for
their correctness. The Guarantee Trustee makes no representation as to the
validity or sufficiency of this Guarantee.

                                       6

<PAGE>

         SECTION 2.4. EVENTS OF DEFAULT; WAIVER.

         (a) An Event of Default under this Guarantee will occur upon the
failure of the Guarantor to perform any of its payment or other obligations
hereunder.

         (b) The Holders of a Majority in liquidation amount of the Capital
Securities may, voting or consenting as a class, on behalf of the Holders of all
of the Capital Securities, waive any past Event of Default and its consequences.
Upon such waiver, any such Event of Default shall cease to exist, and shall be
deemed to have been cured, for every purpose of this Guarantee, but no such
waiver shall extend to any subsequent or other default or Event of Default or
impair any right consequent thereon.

         SECTION 2.5. EVENTS OF DEFAULT; NOTICE.

         (a) The Guarantee Trustee shall, within 90 days after the occurrence of
an Event of Default, transmit by mail, first class postage prepaid, to the
Holders of the Capital Securities and the Guarantor, notices of all Events of
Default actually known to a Responsible Officer of the Guarantee Trustee, unless
such defaults have been cured before the giving of such notice, provided,
however, that the Guarantee Trustee shall be protected in withholding such
notice if and so long as a Responsible Officer of the Guarantee Trustee in good
faith determines that the withholding of such notice is in the interests of the
Holders of the Capital Securities.

         (b) The Guarantee Trustee shall not be deemed to have knowledge of any
Event of Default unless the Guarantee Trustee shall have received written notice
from the Guarantor or a Holder of the Capital Securities (except in the case of
a payment default), or a Responsible Officer of the Guarantee Trustee charged
with the administration of this Guarantee shall have obtained actual knowledge
thereof.

                                  ARTICLE III

                               GUARANTEE TRUSTEE

         SECTION 3.1. GUARANTEE TRUSTEE; ELIGIBILITY.

         (a)      There shall at all times be a Guarantee Trustee which shall:

                  (i)  not be an Affiliate of the Guarantor, and

                  (ii) be a corporation organized and doing business under the
         laws of the United States of America or any State or Territory thereof
         or of the District of Columbia, or Person authorized under such laws to
         exercise corporate trust powers, having a combined capital and surplus
         of at least 50 million U.S. dollars ($50,000,000), and subject to
         supervision or examination by Federal, State, Territorial or District
         of Columbia authority. If such corporation publishes reports of
         condition at least annually, pursuant to law or to the requirements of
         the supervising or examining authority referred to above, then, for the
         purposes of this Section 3.1(a)(ii), the combined capital and surplus
         of such corporation shall be deemed to be its combined capital and
         surplus as set forth in its most recent report of condition so
         published.

         (b) If at any time the Guarantee Trustee shall cease to be eligible to
so act under Section 3.1(a), the Guarantee Trustee shall immediately resign in
the manner and with the effect set out in Section 3.2(c).

                                       7

<PAGE>

         (c) If the Guarantee Trustee has or shall acquire any "conflicting
interest" within the meaning of Section 310(b) of the Trust Indenture Act, the
Guarantee Trustee shall either eliminate such interest or resign to the extent
and in the manner provided by, and subject to this Guarantee.

         SECTION 3.2. APPOINTMENT REMOVAL AND RESIGNATION OF GUARANTEE TRUSTEE.

         (a) Subject to Section 3.2(b), the Guarantee Trustee may be appointed
or removed without cause at any time by the Guarantor except during an Event of
Default.

         (b) The Guarantee Trustee shall not be removed in accordance with
Section 3.2(a) until a Successor Guarantee Trustee has been appointed and has
accepted such appointment by written instrument executed by such Successor
Guarantee Trustee and delivered to the Guarantor.

         (c) The Guarantee Trustee appointed to office shall hold office until a
Successor Guarantee Trustee shall have been appointed or until its removal or
resignation. The Guarantee Trustee may resign from office (without need for
prior or subsequent accounting) by an instrument in writing executed by the
Guarantee Trustee and delivered to the Guarantor, which resignation shall not
take effect until a Successor Guarantee Trustee has been appointed and has
accepted such appointment by an instrument in writing executed by such Successor
Guarantee Trustee and delivered to the Guarantor and the resigning Guarantee
Trustee.

         (d) If no Successor Guarantee Trustee shall have been appointed and
accepted appointment as provided in this Section 3.2 within 60 days after
delivery of an instrument of removal or resignation, the Guarantee Trustee
resigning or being removed may petition any court of competent jurisdiction for
appointment of a Successor Guarantee Trustee. Such court may thereupon, after
prescribing such notice, if any, as it may deem proper, appoint a Successor
Guarantee Trustee.

         (e) No Guarantee Trustee shall be liable for the acts or omissions to
act of any Successor Guarantee Trustee.

         (f) Upon termination of this Guarantee or removal or resignation of the
Guarantee Trustee pursuant to this Section 3.2, the Guarantor shall pay to the
Guarantee Trustee all amounts owing to the Guarantee Trustee under Sections 7.2
and 7.3 accrued to the date of such termination, removal or resignation.

                                   ARTICLE IV

                                    GUARANTEE

         SECTION 4.1. GUARANTEE.

         (a) The Guarantor irrevocably and unconditionally agrees to pay in full
to the Holders the Guarantee Payments (without duplication of amounts
theretofore paid by the Issuer), as and when due, regardless of any defense
(except the defense of payment by the Issuer), right of set-off or counterclaim
that the Issuer may have or assert. The Guarantor's obligation to make a
Guarantee Payment may be satisfied by direct payment of the required amounts by
the Guarantor to the Holders or by causing the Issuer to pay such amounts to the
Holders.

         (b) The Guarantor hereby also agrees to assume any and all Obligations
of the Issuer and in the event any such Obligation is not so assumed, subject to
the terms and conditions hereof, the Guarantor hereby irrevocably and
unconditionally guarantees to each Beneficiary the full payment, when and as
due, of any and all Obligations to such Beneficiaries. This Agreement is
intended to be for the benefit of, and

                                                        8



<PAGE>


to be enforceable by, all such Beneficiaries, whether or not such Beneficiaries
have received notice hereof.

         SECTION 4.2. WAIVER OF NOTICE AND DEMAND. The Guarantor hereby waives
notice of acceptance of this Guarantee and of any liability to which it applies
or may apply, presentment, demand for payment, any right to require a
proceeding first against the Issuer or any other Person before proceeding
against the Guarantor, protest, notice of nonpayment, notice of dishonor, notice
of redemption and all other notices and demands.

         SECTION 4.3. OBLIGATIONS NOT AFFECTED. The obligations, covenants,
agreements and duties of the Guarantor under this Guarantee shall in no way be
affected or impaired by reason of the happening from time to time of any of the
following:

         (a) the release or waiver, by operation of law or otherwise, of the
performance or observance by the Issuer of any express or implied agreement,
covenant, term or condition relating to the Capital Securities to be performed
or observed by the Issuer;

         (b) the extension of time for the payment by the Issuer of all or any
portion of the Distributions, Redemption Price, Special Redemption Price,
Liquidation Distribution or any other sums payable under the terms of the
Capital Securities or the extension of time for the performance of any other
obligation under, arising out of or in connection with, the Capital Securities
(other than an extension of time for payment of Distributions, Redemption
Price, Special Redemption Price, Liquidation Distribution or other sum payable
that results from the extension of any interest payment period on the Debentures
or any extension of the maturity date of the Debentures permitted by the
Indenture);

         (c) any failure, omission, delay or lack of diligence on the part of
the Holders to enforce, assert or exercise any right, privilege, power or
remedy conferred on the Holders pursuant to the terms of the Capital Securities,
or any action on the part of the Issuer granting indulgence or extension of any
kind;

         (d) the voluntary or involuntary liquidation, dissolution, sale of any
collateral, receivership, insolvency, bankruptcy, assignment for the benefit of
creditors, reorganization, arrangement, composition or readjustment of debt of,
or other similar proceedings affecting, the Issuer or any of the assets of the
Issuer;

         (e) any invalidity of, or defect or deficiency in, the Capital
Securities;

         (f) the settlement or compromise of any obligation guaranteed hereby or
hereby incurred; or

         (g) any other circumstance whatsoever that might otherwise constitute a
legal or equitable discharge or defense of a guarantor, it being the intent of
this Section 4.3 that the obligations of the Guarantor hereunder shall be
absolute and unconditional under any and all circumstances.

         There shall be no obligation of the Holders to give notice to, or
obtain consent of, the Guarantor with respect to the happening of any of the
foregoing.

         SECTION 4.4. RIGHTS OF HOLDERS.

         (a) The Holders of a Majority in liquidation amount of the Capital
Securities have the right to direct the time, method and place of conducting any
proceeding for any remedy available to the Guarantee Trustee in respect of this
Guarantee or to direct the exercise of any trust or power conferred upon the
Guarantee Trustee under this Guarantee; provided, however, that (subject to
Section 2.1) the Guarantee Trustee shall have the right to decline to follow
any such direction if the Guarantee Trustee

                                       9


<PAGE>




being advised by counsel determines that the action or proceeding so directed
may not lawfully be taken or if the Guarantee Trustee in good faith by its board
of directors or trustees, executive committees or a trust committee of directors
or trustees and/or Responsible Officers shall determine that the action or
proceedings so directed would involve the Guarantee Trustee in personal
liability.

         (b) Any Holder of Capital Securities may institute a legal proceeding
directly against the Guarantor to enforce the Guarantee Trustee's rights under
this Guarantee, without first instituting a legal proceeding against the Issuer,
the Guarantee Trustee or any other Person. The Guarantor waives any right or
remedy to require that any such action be brought first against the Issuer, the
Guarantee Trustee or any other Person before so proceeding directly against the
Guarantor.

         SECTION 4.5. GUARANTEE OF PAYMENT. This Guarantee creates a guarantee
of payment and not of collection.

         SECTION 4.6. SUBROGATION. The Guarantor shall be subrogated to all (if
any) rights of the Holders of Capital Securities against the Issuer in respect
of any amounts paid to such Holders by the Guarantor under this Guarantee;
provided, however, that the Guarantor shall not (except to the extent required
by mandatory provisions of law) be entitled to enforce or exercise any right
that it may acquire by way of subrogation or any indemnity, reimbursement or
other agreement, in all cases as a result of payment under this Guarantee, if,
after giving effect to any such payment, any amounts are due and unpaid under
this Guarantee. If any amount shall be paid to the Guarantor in violation of the
preceding sentence, the Guarantor agrees to hold such amount in trust for the
Holders and to pay over such amount to the Holders.

         SECTION 4.7. INDEPENDENT OBLIGATIONS. The Guarantor acknowledges that
its obligations hereunder are independent of the obligations of the Issuer with
respect to the Capital Securities and that the Guarantor shall be liable as
principal and as debtor hereunder to make Guarantee Payments pursuant to the
terms of this Guarantee notwithstanding the occurrence of any event referred to
in subsections (a) through (g), inclusive, of Section 4.3 hereof.

         SECTION 4.8. ENFORCEMENT BY A BENEFICIARY. A Beneficiary may enforce
the obligations of the Guarantor contained in Section 4.1(b) directly against
the Guarantor and the Guarantor waives any right or remedy to require that any
action be brought against the Issuer or any other person or entity before
proceeding against the Guarantor. The Guarantor shall be subrogated to all
rights (if any) of any Beneficiary against the Issuer in respect of any amounts
paid to the Beneficiaries by the Guarantor under this Agreement; provided,
however, that the Guarantor shall not (except to the extent required by
mandatory provisions of law) be entitled to enforce or exercise any rights that
it may acquire by way of subrogation or any indemnity, reimbursement or other
agreement, in all cases as a result of payment under this Agreement, if at the
time of any such payment, and after giving effect to such payment, any amounts
are due and unpaid under this Agreement.

                                    ARTICLE V

                    LIMITATION OF TRANSACTIONS; SUBORDINATION

         SECTION 5.1. LIMITATION OF TRANSACTIONS. So long as any Capital
Securities remain outstanding, if (a) there shall have occurred and be
continuing an Event of Default or a Declaration Event of Default or (b) the
Guarantor shall have selected an Extension Period as provided in the Declaration
and such period, or any extension thereof, shall be continuing, then the
Guarantor shall not and shall not permit any Affiliate to (x) declare or pay
any dividends or distributions on, or redeem, purchase, acquire, or make a
liquidation payment with respect to, any of the Guarantor's or such Affiliate's
capital stock (other than payments of dividends or distributions to the
Guarantor) or make any guarantee payments

                                       10

<PAGE>


with respect to the foregoing or (y) make any payment of principal of or
interest or premium, if any, on or repay, repurchase or redeem any debt
securities of the Guarantor or any Affiliate that rank pari passu in all
respects with or junior in interest to the Debentures (other than, with respect
to clauses (x) and (y) above, (i) repurchases, redemptions or other acquisitions
of shares of capital stock of the Guarantor in connection with any employment
contract, benefit plan or other similar arrangement with or for the benefit of
one or more employees, officers, directors or consultants, in connection with a
dividend reinvestment or stockholder stock purchase plan or in connection with
the issuance of capital stock of the Guarantor (or securities convertible into
or exercisable for such capital stock) as consideration in an acquisition
transaction entered into prior to the occurrence of the Event of Default,
Declaration Event of Default or Extension Period, as applicable, (ii) as a
result of any exchange or conversion of any class or series of the Guarantor's
capital stock (or any capital stock of a subsidiary of the Guarantor) for any
class or series of the Guarantor's capital stock or of any class or series of
the Guarantor's indebtedness for any class or series of the Guarantor's capital
stock, (iii) the purchase of fractional interests in shares of the Guarantor's
capital stock pursuant to the conversion or exchange provisions of such capital
stock or the security being converted or exchanged, (iv) any declaration of a
dividend in connection with any stockholder's rights plan, or the issuance of
rights, stock or other property under any stockholder's rights plan, or the
redemption or repurchase of rights pursuant thereto, (v) any dividend in the
form of stock, warrants, options or other rights where the dividend stock or the
stock issuable upon exercise of such warrants, options or other rights is the
same stock as that on which the dividend is being paid or ranks pari passu with
or junior to such stock and any cash payments in lieu of fractional shares
issued in connection therewith, or (vi) payments under this Guarantee).

         SECTION 5.2. RANKING. This Guarantee will constitute an unsecured
obligation of the Guarantor and will rank subordinate and junior in right of
payment to all present and future Senior Indebtedness (as defined in the
Indenture) of the Guarantor. By their acceptance thereof, each Holder of Capital
Securities agrees to the foregoing provisions of this Guarantee and the other
terms set forth herein.

         The right of the Guarantor to participate in any distribution of assets
of any of its subsidiaries upon any such subsidiary's liquidation or
reorganization or otherwise is subject to the prior claims of creditors of that
subsidiary, except to the extent the Guarantor may itself be recognized as a
creditor of that subsidiary. Accordingly, the Guarantor's obligations under this
Guarantee will be effectively subordinated to all existing and future
liabilities of the Guarantor's subsidiaries, and claimants should look only to
the assets of the Guarantor for payments thereunder. This Guarantee does not
limit the incurrence or issuance of other secured or unsecured debt of the
Guarantor, including Senior Indebtedness of the Guarantor, under any indenture
that the Guarantor may enter into in the future or otherwise.

                                   ARTICLE VI

                                   TERMINATION

         SECTION 6.1. TERMINATION. This Guarantee shall terminate as to the
Capital Securities (i) upon full payment of the Redemption Price or Special
Redemption Price of all Capital Securities, (ii) upon the distribution of the
Debentures to the Holders of all of the Capital Securities or (iii) upon full
payment of the amounts payable in accordance with the Declaration upon
dissolution of the Issuer. This Guarantee will continue to be effective or will
be reinstated, as the case may be, if at any time any Holder of Capital
Securities must restore payment of any sums paid under the Capital Securities or
under this Guarantee.

                                       11

<PAGE>




                                   ARTICLE VII

                                 INDEMNIFICATION

         SECTION 7.1. EXCULPATION.

         (a) No Indemnified Person shall be liable, responsible or accountable
in damages or otherwise to the Guarantor or any Covered Person for any loss,
damage or claim incurred by reason of any act or omission performed or omitted
by such Indemnified Person in good faith in accordance with this Guarantee and
in a manner that such Indemnified Person reasonably believed to be within the
scope of the authority conferred on such Indemnified Person by this Guarantee or
by law, except that an Indemnified Person shall be liable for any such loss,
damage or claim incurred by reason of such Indemnified Person's negligence or
willful misconduct with respect to such acts or omissions.

         (b) An Indemnified Person shall be fully protected in relying in good
faith upon the records of the Issuer or the Guarantor and upon such information,
opinions, reports or statements presented to the Issuer or the Guarantor by any
Person as to matters the Indemnified Person reasonably believes are within such
other Person's professional or expert competence and who, if selected by such
Indemnified Person, has been selected with reasonable care by such Indemnified
Person, including information, opinions, reports or statements as to the value
and amount of the assets, liabilities, profits, losses, or any other facts
pertinent to the existence and amount of assets from which Distributions to
Holders of Capital Securities might properly be paid.

         SECTION 7.2. INDEMNIFICATION

         (a) The Guarantor agrees to indemnify each Indemnified Person for, and
to hold each Indemnified Person harmless against, any and all loss, liability,
damage, claim or expense incurred without negligence or willful misconduct on
the part of the Indemnified Person, arising out of or in connection with the
acceptance or administration of the trust or trusts hereunder, including the
costs and expenses (including reasonable legal fees and expenses) of the
Indemnified Person defending itself against, or investigating, any claim or
liability in connection with the exercise or performance of any of the
Indemnified Person's powers or duties hereunder. The obligation to indemnify as
set forth in this Section 7.2 shall survive the resignation or removal of the
Guarantee Trustee and the termination of this Guarantee.

         (b) Promptly after receipt by an Indemnified Person under this Section
7.2 of notice of the commencement of any action, such Indemnified Person will,
if a claim in respect thereof is to be made against the Guarantor under this
Section 7.2, notify the Guarantor in writing of the commencement thereof; but
the failure so to notify the Guarantor (i) will not relieve the Guarantor from
liability under paragraph (a) above unless and to the extent that the Guarantor
did not otherwise learn of such action and such failure results in the
forfeiture by the Guarantor of substantial rights and defenses and (ii) will
not, in any event, relieve the Guarantor from any obligations to any Indemnified
Person other than the indemnification obligation provided in paragraph (a)
above. The Guarantor shall be entitled to appoint counsel of the Guarantor's
choice at the Guarantor's expense to represent the Indemnified Person in any
action for which indemnification is sought (in which case the Guarantor shall
not thereafter be responsible for the fees and expenses of any separate counsel
retained by the Indemnified Person or Persons except as set forth below);
provided, however, that such counsel shall be reasonably satisfactory to the
Indemnified Person. Notwithstanding the Guarantor's election to appoint counsel
to represent the Guarantor in an action, the Indemnified Person shall have the
right to employ separate counsel (including local counsel), and the Guarantor
shall bear the reasonable fees, costs and expenses of such separate counsel if
(1) the use of counsel chosen by the Guarantor to represent the Indemnified
Person would present such counsel with a conflict of interest, (ii) the actual
or potential defendants in, or targets of, any such action include both

                                       12


<PAGE>


the Indemnified Person and the Guarantor and the Indemnified Person shall have
reasonably concluded that there may be legal defenses available to it and/or
other Indemnified Person(s) which are different from or additional to those
available to the Guarantor, (iii) the Guarantor shall not have employed counsel
satisfactory to the Indemnified Person to represent the Indemnified Person
within a reasonable time after notice of the institution of such action or (iv)
the Guarantor shall authorize the Indemnified Person to employ separate counsel
at the expense of the Guarantor. The Guarantor will not, without the prior
written consent of the Indemnified Persons, settle or compromise or consent to
the entry of any judgment with respect to any pending or threatened claim,
action, suit or proceeding in respect of which indemnification or contribution
may be sought hereunder (whether or not the Indemnified Persons are actual or
potential parties to such claim or action) unless such settlement, compromise or
consent includes an unconditional release of each Indemnified Person from all
liability arising out of such claim, action, suit or proceeding.

         SECTION 7.3. COMPENSATION; REIMBURSEMENT OF EXPENSES. The Guarantor
agrees:

         (a) to pay to the Guarantee Trustee from time to time such compensation
for all services rendered by it hereunder as the parties shall agree to from
time to time (which compensation shall not be limited by any provision of law in
regard to the compensation of a trustee of an express trust); and

         (b) except as otherwise expressly provided herein, to reimburse the
Guarantee Trustee upon request for all reasonable expenses, disbursements and
advances incurred or made by it in accordance with any provision of this
Guarantee (including the reasonable compensation and the expenses
and disbursements of its agents and counsel), except any such expense,
disbursement or advance as may be attributable to its negligence or willful
misconduct.

         The provisions of this Section 7.3 shall survive the resignation or
removal of the Guarantee. Trustee and the termination of this Guarantee.

                                  ARTICLE VIII

                                  MISCELLANEOUS

         SECTION 8.1. SUCCESSORS AND ASSIGNS. All guarantees and agreements
contained in this Guarantee shall bind the successors, assigns, receivers,
trustees and representatives of the Guarantor and shall inure to the benefit of
the Holders of the Capital Securities then outstanding. Except in connection
with any merger or consolidation of the Guarantor with or into another entity or
any sale, transfer or lease of the Guarantor's assets to another entity, in each
case, to the extent permitted under the Indenture, the Guarantor may not assign
its rights or delegate its obligations under this Guarantee without the prior
approval of the Holders of at least a Majority in liquidation amount of the
Capital Securities.

         SECTION 8.2. AMENDMENTS. Except with respect to any changes that do not
adversely affect the rights of Holders of the Capital Securities in any material
respect (in which case no consent of Holders will be required), this Guarantee
may be amended only with the prior approval of the Holders of not less than a
Majority in liquidation amount of the Capital Securities. The provisions of the
Declaration with respect to amendments thereof apply to the giving of such
approval.

         SECTION 8.3. NOTICES. All notices provided for in this Guarantee shall
be in writing, duly signed by the party giving such notice, and shall be
delivered, telecopied or mailed by first class mail, as follows:

                                       13
<PAGE>




         (a) If given to the Guarantee Trustee, at the Guarantee Trustee's
mailing address set forth below (or such other address as the Guarantee Trustee
may give notice of to the Holders of the Capital Securities and the Guarantor):

         State Street Bank and Trust  Company of  Connecticut,
           National  Association
         225 Asylum  Street,  Goodwin  Square
         Hartford, Connecticut 06103
         Attention: Corporate Trust Department
         Telecopy: (860) 244-1889

         With a copy to:

         State  Street Bank and Trust  Company
         P.O.Box  778
         Boston,  Massachusetts  02102-0778
         Attention:  Paul D. Allen, Corporate Trust Department
         Telecopy: (617) 662-1462

         (b) If given to the Guarantor, at the Guarantor's mailing address set
forth below (or such other address as the Guarantor may give notice of to the
Holders of the Capital Securities and to the Guarantee Trustee):

         U.S.B.  Holding Co.,  Inc.
         100 Dutch Hill Road
         Orangeburg,  New York 10962
         Attention:  Steven T.  Sabatini/Mike Tedesco
         Telecopy: (845) 365-4695

         (c) If given to any Holder of the Capital Securities, at the address
set forth on the books and records of the Issuer.

         All such notices shall be deemed to have been given when received in
person, telecopied with receipt confirmed, or mailed by first class mail,
postage prepaid, except that if a notice or other document is refused delivery
or cannot be delivered because of a changed address of which no notice was
given, such notice or other document shall be deemed to have been delivered on
the date of such refusal or inability to deliver.

         SECTION 8.4. BENEFIT. This Guarantee is solely for the benefit of the
Holders of the Capital Securities and, subject to Section 2.1(a), is not
separately transferable from the Capital Securities.

         SECTION 8.5. GOVERNING LAW. THIS GUARANTEE SHALL BE GOVERNED BY, AND
CONSTRUED AND INTERPRETED IN ACCORDANCE WITH, THE LAW OF THE STATE OF NEW YORK,
WITHOUT REGARD TO CONFLICT OF LAWS PRINCIPLES THEREOF (OTHER THAN SECTION 5-1401
OF THE NEW YORK GENERAL OBLIGATIONS LAW).

         SECTION 8.6. COUNTERPARTS. This Guarantee may be executed in one or
more counterparts, each of which shall be an original, but all of which taken
together shall constitute one and the same instrument.

                     Signatures appear on the following page

                                       14


<PAGE>

THIS GUARANTEE is executed as of the day and year first above written.

                                U.S.B. HOLDING CO., INC.,
                                as Guarantor

                                By:
                                   -----------------------------------------
                                   Name:    Thomas E. Hales
                                   Title:   Chairman of the Board, President
                                           and Chief Executive Officer

                                STATE STREET BANK AND TRUST COMPANY
                                OF CONNECTICUT, NATIONAL
                                ASSOCTION, as Guarantee Trustee

                                By:      /s/ Paul D. Allen
                                   -----------------------------------------
                                         Name: Paul D. Allen
                                         Title: Vice President


                                       15