EXHIBIT 4(h)






                -------------------------------------------------




                               GUARANTEE AGREEMENT

                                 by and between

                            U.S.B. HOLDING CO., INC.

                                       and

                       STATE STREET BANK AND TRUST COMPANY
                      OF CONNECTICUT, NATIONAL ASSOCIATION

                            Dated as of June 26, 2002




                -------------------------------------------------


<PAGE>


                               GUARANTEE AGREEMENT
                               -------------------

         This GUARANTEE AGREEMENT (this "Guarantee"), dated as of June 26, 2002,
is executed and delivered by U.S.B. Holding Co., Inc., a Delaware corporation
(the "Guarantor"), and State Street Bank and Trust Company of Connecticut,
National Association, a national banking association, organized under the laws
of the United States of America, as trustee (the "Guarantee Trustee"), for the
benefit of the Holders (as defined herein) from time to time of the Capital
Securities (as defined herein) of USB Statutory Trust III, a Connecticut
statutory trust (the "Issuer").

         WHEREAS, pursuant to an Amended and Restated Declaration of Trust (the
"Declaration"), dated as of the date hereof among State Street Bank and Trust
Company of Connecticut, National Association, not in its individual capacity but
solely as institutional trustee, the administrators of the Issuer named therein,
the Guarantor, as sponsor, and the holders from time to time of undivided
beneficial interests in the assets of the Issuer, the Issuer is issuing on the
date hereof those undivided beneficial interests, having an aggregate
liquidation amount of $10,000,000 (the "Capital Securities"); and

         WHEREAS, as incentive for the Holders to purchase the Capital
Securities, the Guarantor desires irrevocably and unconditionally to agree, to
the extent set forth in this Guarantee, to pay to the Holders of Capital
Securities the Guarantee Payments (as defined herein) and to make certain other
payments on the terms and conditions set forth herein;

         NOW, THEREFORE, in consideration of the purchase by each Holder of the
Capital Securities, which purchase the Guarantor hereby agrees shall benefit the
Guarantor, the Guarantor executes and delivers this Guarantee for the benefit of
the Holders.

                                   ARTICLE I

                         DEFINITIONS AND INTERPRETATION

         SECTION 1.1. DEFINITIONS AND INTERPRETATION. In this Guarantee, unless
the context otherwise requires:


         (a) capitalized terms used in this Guarantee but not defined in the
preamble above have the respective meanings assigned to them in this Section
1.1;

         (b) a term defined anywhere in this Guarantee has the same meaning
throughout;

         (c) all references to "the Guarantee" or "this Guarantee" are to this
Guarantee as modified, supplemented or amended from time to time;

         (d) all references in this Guarantee to "Articles" or "Sections" are to
Articles or Sections of this Guarantee, unless otherwise specified;

         (e) terms defined in the Declaration as at the date of execution of
this Guarantee have the same meanings when used in this Guarantee, unless
otherwise defined in this Guarantee or unless the context otherwise requires;
and

         (f) a reference to the singular includes the plural and vice versa.

         "Affiliate" has the same meaning as given to that term in Rule 405 of
the Securities Act of 1933, as amended, or any successor rule thereunder.

<PAGE>


         "Beneficiaries" means any Person to whom the Issuer is or hereafter
becomes indebted or liable.

         "Capital Securities" has the meaning set forth in the recitals to this
Guarantee.

         "Common Securities" means the common securities issued by the Issuer to
the Guarantor pursuant to the Declaration.

         "Corporate Trust Office" means the office of the Guarantee Trustee at
which the corporate trust business of the Guarantee Trustee shall, at any
particular time, be principally administered, which office at the date of
execution of this Guarantee is located at 225 Asylum Street, Goodwin Square,
Hartford, Connecticut 06103.

         "Covered Person" means any Holder of Capital Securities.

         "Debentures" means the debt securities of the Guarantor designated the
Floating Rate Junior Subordinated Deferrable Interest Debentures due 2032 held
by the Institutional Trustee (as defined in the Declaration) of the Issuer.

         "Declaration Event of Default" means an "Event of Default" as defined
in the Declaration.

         "Event of Default" has the meaning set forth in Section 2.4(a).

         "Guarantee Payments" means the following payments or distributions,
without duplication, with respect to the Capital Securities, to the extent not
paid or made by the Issuer: (i) any accrued and unpaid Distributions (as defined
in the Declaration) which are required to be paid on such Capital Securities to
the extent the Issuer shall have funds available therefor, (ii) the Redemption
Price to the extent the Issuer has funds available therefor, with respect to any
Capital Securities called for redemption by the Issuer, (iii) the Special
Redemption Price to the extent the Issuer has funds available therefor, with
respect to Capital Securities redeemed upon the occurrence of a Special Event,
and (iv) upon a voluntary or involuntary liquidation, dissolution, winding-up or
termination of the Issuer (other than in connection with the distribution of
Debentures to the Holders of the Capital Securities in exchange therefor as
provided in the Declaration), the lesser of (a) the aggregate of the liquidation
amount and all accrued and unpaid Distributions on the Capital Securities to the
date of payment, to the extent the Issuer shall have funds available therefor,
and (b) the amount of assets of the Issuer remaining available for distribution
to Holders in liquidation of the Issuer (in either case, the "Liquidation
Distribution").

         "Guarantee Trustee" means State Street Bank and Trust Company of
Connecticut, National Association, until a Successor Guarantee Trustee has been
appointed and has accepted such appointment pursuant to the terms of this
Guarantee and thereafter means each such Successor Guarantee Trustee.

         "Guarantor" means U.S.B. Holding Co., Inc. and each of its successors
and assigns.

         "Holder" means any holder, as registered on the books and records of
the Issuer, of any Capital Securities; provided, however, that, in determining
whether the Holders of the requisite percentage of Capital Securities have given
any request, notice, consent or waiver hereunder, "Holder" shall not include the
Guarantor or any Affiliate of the Guarantor.

         "Indemnified Person" means the Guarantee Trustee, any Affiliate of the
Guarantee Trustee, or any officers, directors, shareholders, members, partners,
employees, representatives, nominees, custodians or agents of the Guarantee
Trustee.

                                       2

<PAGE>


         "Indenture" means the Indenture dated as of the date hereof between the
Guarantor and State Street Bank and Trust Company of Connecticut, National
Association, not in its individual capacity but solely as trustee, and any
indenture supplemental thereto pursuant to which the Debentures are to be issued
to the institutional trustee of the Issuer.

         "Issuer" has the meaning set forth in the opening paragraph to this
Guarantee.

         "Liquidation Distribution" has the meaning set forth in the definition
of "Guarantee Payments" herein.

         "Majority in liquidation amount of the Capital Securities" means
Holder(s) of outstanding Capital Securities, voting together as a class, but
separately from the holders of Common Securities, of more than 50% of the
aggregate liquidation amount (including the stated amount that would be paid on
redemption, liquidation or otherwise, plus accrued and unpaid Distributions to
the date upon which the voting percentages are determined) of all Capital
Securities then outstanding.

         "Obligations" means any costs, expenses or liabilities (but not
including liabilities related to taxes) of the Issuer other than obligations of
the Issuer to pay to holders of any Trust Securities the amounts due such
holders pursuant to the terms of the Trust Securities.

         "Officer's Certificate" means, with respect to any Person, a
certificate signed by one Authorized Officer of such Person. Any Officer's
Certificate delivered with respect to compliance with a condition or covenant
provided for in this Guarantee shall include:

              (a) a statement that the officer signing the Officer's Certificate
         has read the covenant or condition and the definitions relating
         thereto;

              (b) a brief statement of the nature and scope of the examination
         or investigation undertaken by the officer in rendering the Officer's
         Certificate;

              (c) a statement that the officer has made such examination or
         investigation as, in such officer's opinion, is necessary to enable
         such officer to express an informed opinion as to whether or not such
         covenant or condition has been complied with; and

              (d) a statement as to whether, in the opinion of the officer, such
         condition or covenant has been complied with.

         "Person" means a legal person, including any individual, corporation,
estate, partnership, joint venture, association, joint stock company, limited
liability company, trust, unincorporated association, or government or any
agency or political subdivision thereof, or any other entity of whatever nature.

         "Redemption Price" has the meaning set forth in the Indenture.

         "Responsible Officer" means, with respect to the Guarantee Trustee, any
officer within the Corporate Trust Office of the Guarantee Trustee including any
Vice President, Assistant Vice President, Secretary, Assistant Secretary or any
other officer of the Guarantee Trustee customarily performing functions similar
to those performed by any of the above designated officers and also, with
respect to a particular corporate trust matter, any other officer to whom such
matter is referred because of that officer's knowledge of and familiarity with
the particular subject.

         "Special Event" has the meaning set forth in the Indenture.

                                       3

<PAGE>


         "Special Redemption Price" has the meaning set forth in the Indenture.

         "Successor Guarantee Trustee" means a successor Guarantee Trustee
possessing the qualifications to act as Guarantee Trustee under Section 3.1.

         "Trust Securities" means the Common Securities and the Capital
Securities.

                                   ARTICLE II

                          POWERS, DUTIES AND RIGHTS OF
                                GUARANTEE TRUSTEE

         SECTION 2.1. POWERS AND DUTIES OF THE GUARANTEE TRUSTEE.

         (a) This Guarantee shall be held by the Guarantee Trustee for the
benefit of the Holders of the Capital Securities, and the Guarantee Trustee
shall not transfer this Guarantee to any Person except a Holder of Capital
Securities exercising his or her rights pursuant to Section 4.4(b) or to a
Successor Guarantee Trustee on acceptance by such Successor Guarantee Trustee of
its appointment to act as Successor Guarantee Trustee. The right, title and
interest of the Guarantee Trustee shall automatically vest in any Successor
Guarantee Trustee, and such vesting and cessation of title shall be effective
whether or not conveyancing documents have been executed and delivered pursuant
to the appointment of such Successor Guarantee Trustee.

         (b) If an Event of Default actually known to a Responsible Officer of
the Guarantee Trustee has occurred and is continuing, the Guarantee Trustee
shall enforce this Guarantee for the benefit of the Holders of the Capital
Securities.

         (c) The Guarantee Trustee, before the occurrence of any Event of
Default and after curing all Events of Default that may have occurred, shall
undertake to perform only such duties as are specifically set forth in this
Guarantee, and no implied covenants shall be read into this Guarantee against
the Guarantee Trustee. In case an Event of Default has occurred (that has not
been waived pursuant to Section 2.4) and is actually known to a Responsible
Officer of the Guarantee Trustee, the Guarantee Trustee shall exercise such of
the rights and powers vested in it by this Guarantee, and use the same degree of
care and skill in its exercise thereof, as a prudent person would exercise or
use under the circumstances in the conduct of his or her own affairs.

         (d) No provision of this Guarantee shall be construed to relieve the
Guarantee Trustee from liability for its own negligent action, its own negligent
failure to act, or its own willful misconduct, except that:

              (i) prior to the occurrence of any Event of Default and after the
         curing or waiving of all such Events of Default that may have occurred:

                  (A) the duties and obligations of the Guarantee Trustee shall
              be determined solely by the express provisions of this Guarantee,
              and the Guarantee Trustee shall not be liable except for the
              performance of such duties and obligations as are specifically set
              forth in this Guarantee, and no implied covenants or obligations
              shall be read into this Guarantee against the Guarantee Trustee;
              and

                  (B) in the absence of bad faith on the part of the Guarantee
              Trustee, the Guarantee Trustee may conclusively rely, as to the
              truth of the statements and the correctness of the opinions
              expressed therein, upon any certificates or opinions furnished

                                        4

<PAGE>


              to the Guarantee Trustee and conforming to the requirements of
              this Guarantee; but in the case of any such certificates or
              opinions that by any provision hereof are specifically required to
              be furnished to the Guarantee Trustee, the Guarantee Trustee shall
              be under a duty to examine the same to determine whether or not
              they conform to the requirements of this Guarantee;

                  (ii) the Guarantee Trustee shall not be liable for any error
              of judgment made in good faith by a Responsible Officer of the
              Guarantee Trustee, unless it shall be proved that such Responsible
              Officer of the Guarantee Trustee or the Guarantee Trustee was
              negligent in ascertaining the pertinent facts upon which such
              judgment was made;

                  (iii) the Guarantee Trustee shall not be liable with respect
              to any action taken or omitted to be taken by it in good faith in
              accordance with the written direction of the Holders of not less
              than a Majority in liquidation amount of the Capital Securities
              relating to the time, method and place of conducting any
              proceeding for any remedy available to the Guarantee Trustee, or
              relating to the exercise of any trust or power conferred upon the
              Guarantee Trustee under this Guarantee; and

                  (iv) no provision of this Guarantee shall require the
              Guarantee Trustee to expend or risk its own funds or otherwise
              incur personal financial liability in the performance of any of
              its duties or in the exercise of any of its rights or powers, if
              the Guarantee Trustee shall have reasonable grounds for believing
              that the repayment of such funds is not reasonably assured to it
              under the terms of this Guarantee or security and indemnity,
              reasonably satisfactory to the Guarantee Trustee, against such
              risk or liability is not reasonably assured to it.

         SECTION 2.2. CERTAIN RIGHTS OF GUARANTEE TRUSTEE.

         (a)  Subject to the provisions of Section 2.1:

              (i) The Guarantee Trustee may conclusively rely, and shall be
         fully protected in acting or refraining from acting upon, any
         resolution, certificate, statement, instrument, opinion, report,
         notice, request, direction, consent, order, bond, debenture, note,
         other evidence of indebtedness or other paper or document believed by
         it to be genuine and to have been signed, sent or presented by the
         proper party or parties.

              (ii) Any direction or act of the Guarantor contemplated by this
         Guarantee shall be sufficiently evidenced by an Officer's Certificate.

              (iii) Whenever, in the administration of this Guarantee, the
         Guarantee Trustee shall deem it desirable that a matter be proved or
         established before taking, suffering or omitting any action hereunder,
         the Guarantee Trustee (unless other evidence is herein specifically
         prescribed) may, in the absence of bad faith on its part, request and
         conclusively rely upon an Officer's Certificate of the Guarantor which,
         upon receipt of such request, shall be promptly delivered by the
         Guarantor.

              (iv) The Guarantee Trustee shall have no duty to see to any
         recording, filing or registration of any instrument (or any
         re-recording, refiling or re-registration thereof).

              (v) The Guarantee Trustee may consult with counsel of its
         selection, and the advice or opinion of such counsel with respect to
         legal matters shall be full and complete authorization and protection
         in respect of any action taken, suffered or omitted by it hereunder in
         good faith and

                                       5

<PAGE>


         in accordance with such advice or opinion. Such counsel may be counsel
         to the Guarantor or any of its Affiliates and may include any of its
         employees. The Guarantee Trustee shall have the right at any time to
         seek instructions concerning the administration of this Guarantee from
         any court of competent jurisdiction.

              (vi) The Guarantee Trustee shall be under no obligation to
         exercise any of the rights or powers vested in it by this Guarantee at
         the request or direction of any Holder, unless such Holder shall have
         provided to the Guarantee Trustee such security and indemnity,
         reasonably satisfactory to the Guarantee Trustee, against the costs,
         expenses (including attorneys' fees and expenses and the expenses of
         the Guarantee Trustee's agents, nominees or custodians) and liabilities
         that might be incurred by it in complying with such request or
         direction, including such reasonable advances as may be requested by
         the Guarantee Trustee; provided, however, that nothing contained in
         this Section 2.2(a)(vi) shall relieve the Guarantee Trustee, upon the
         occurrence of an Event of Default, of its obligation to exercise the
         rights and powers vested in it by this Guarantee.

              (vii) The Guarantee Trustee shall not be bound to make any
         investigation into the facts or matters stated in any resolution,
         certificate, statement, instrument, opinion, report, notice, request,
         direction, consent, order, bond, debenture, note, other evidence of
         indebtedness or other paper or document, but the Guarantee Trustee, in
         its discretion, may make such further inquiry or investigation into
         such facts or matters as it may see fit.

              (viii) The Guarantee Trustee may execute any of the trusts or
         powers hereunder or perform any duties hereunder either directly or by
         or through agents, nominees, custodians or attorneys, and the Guarantee
         Trustee shall not be responsible for any misconduct or negligence on
         the part of any agent or attorney appointed with due care by it
         hereunder.

              (ix) Any action taken by the Guarantee Trustee or its agents
         hereunder shall bind the Holders of the Capital Securities, and the
         signature of the Guarantee Trustee or its agents alone shall be
         sufficient and effective to perform any such action. No third party
         shall be required to inquire as to the authority of the Guarantee
         Trustee to so act or as to its compliance with any of the terms and
         provisions of this Guarantee, both of which shall be conclusively
         evidenced by the Guarantee Trustee's or its agent's taking such action.

              (x) Whenever in the administration of this Guarantee the Guarantee
         Trustee shall deem it desirable to receive instructions with respect to
         enforcing any remedy or right or taking any other action hereunder, the
         Guarantee Trustee (i) may request instructions from the Holders of a
         Majority in liquidation amount of the Capital Securities, (ii) may
         refrain from enforcing such remedy or right or taking such other action
         until such instructions are received, and (iii) shall be protected in
         conclusively relying on or acting in accordance with such instructions.

              (xi) The Guarantee Trustee shall not be liable for any action
         taken, suffered, or omitted to be taken by it in good faith, without
         negligence, and reasonably believed by it to be authorized or within
         the discretion or rights or powers conferred upon it by this Guarantee.

         (b) No provision of this Guarantee shall be deemed to impose any duty
or obligation on the Guarantee Trustee to perform any act or acts or exercise
any right, power, duty or obligation conferred or imposed on it, in any
jurisdiction in which it shall be illegal or in which the Guarantee Trustee
shall be unqualified or incompetent in accordance with applicable law to perform
any such act or acts or to exercise any such right, power, duty or obligation.
No permissive power or authority available to the Guarantee Trustee shall be
construed to be a duty.

                                       6

<PAGE>


         SECTION 2.3. NOT RESPONSIBLE FOR RECITALS OR ISSUANCE OF GUARANTEE. The
recitals contained in this Guarantee shall be taken as the statements of the
Guarantor, and the Guarantee Trustee does not assume any responsibility for
their correctness. The Guarantee Trustee makes no representation as to the
validity or sufficiency of this Guarantee.

         SECTION 2.4. EVENTS OF DEFAULT; WAIVER.

         (a) An Event of Default under this Guarantee will occur upon the
failure of the Guarantor to perform any of its payment or other obligations
hereunder.

         (b) The Holders of a Majority in liquidation amount of the Capital
Securities may, voting or consenting as a class, on behalf of the Holders of all
of the Capital Securities, waive any past Event of Default and its consequences.
Upon such waiver, any such Event of Default shall cease to exist, and shall be
deemed to have been cured, for every purpose of this Guarantee, but no such
waiver shall extend to any subsequent or other default or Event of Default or
impair any right consequent thereon.

         SECTION 2.5. EVENTS OF DEFAULT; NOTICE.

         (a) The Guarantee Trustee shall, within 90 days after the occurrence of
an Event of Default, transmit by mail, first class postage prepaid, to the
Holders of the Capital Securities and the Guarantor, notices of all Events of
Default actually known to a Responsible Officer of the Guarantee Trustee, unless
such defaults have been cured before the giving of such notice, provided,
however, that the Guarantee Trustee shall be protected in withholding such
notice if and so long as a Responsible Officer of the Guarantee Trustee in good
faith determines that the withholding of such notice is in the interests of the
Holders of the Capital Securities.

         (b) The Guarantee Trustee shall not be deemed to have knowledge of any
Event of Default unless the Guarantee Trustee shall have received written notice
from the Guarantor or a Holder of the Capital Securities (except in the case of
a payment default), or a Responsible Officer of the Guarantee Trustee charged
with the administration of this Guarantee shall have obtained actual knowledge
thereof.

                                  ARTICLE III

                                GUARANTEE TRUSTEE

         SECTION 3.1. GUARANTEE TRUSTEE; ELIGIBILITY.

         (a)  There shall at all times be a Guarantee Trustee which shall:

              (i) not be an Affiliate of the Guarantor, and

              (ii) be a corporation organized and doing business under the laws
         of the United States of America or any State or Territory thereof or of
         the District of Columbia, or Person authorized under such laws to
         exercise corporate trust powers, having a combined capital and surplus
         of at least 50 million U.S. dollars ($50,000,000), and subject to
         supervision or examination by Federal, State, Territorial or District
         of Columbia authority. If such corporation publishes reports of
         condition at least annually, pursuant to law or to the requirements of
         the supervising or examining authority referred to above, then, for the
         purposes of this Section 3.1(a)(ii), the combined capital and surplus
         of such corporation shall be deemed to be its combined capital and
         surplus as set forth in its most recent report of condition so
         published.

                                       7

<PAGE>


              (b) If at any time the Guarantee Trustee shall cease to be
         eligible to so act under Section 3.1(a), the Guarantee Trustee shall
         immediately resign in the manner and with the effect set out in Section
         3.2(c).

              (c) If the Guarantee Trustee has or shall acquire any "conflicting
         interest" within the meaning of Section 310(b) of the Trust Indenture
         Act, the Guarantee Trustee shall either eliminate such interest or
         resign to the extent and in the manner provided by, and subject to this
         Guarantee.

         SECTION 3.2. APPOINTMENT, REMOVAL AND RESIGNATION OF GUARANTEE TRUSTEE.

         (a) Subject to Section 3.2(b), the Guarantee Trustee may be appointed
or removed without cause at any time by the Guarantor except during an Event of
Default.

         (b) The Guarantee Trustee shall not be removed in accordance with
Section 3.2(a) until a Successor Guarantee Trustee has been appointed and has
accepted such appointment by written instrument executed by such Successor
Guarantee Trustee and delivered to the Guarantor.

         (c) The Guarantee Trustee appointed to office shall hold office until a
Successor Guarantee Trustee shall have been appointed or until its removal or
resignation. The Guarantee Trustee may resign from office (without need for
prior or subsequent accounting) by an instrument in writing executed by the
Guarantee Trustee and delivered to the Guarantor, which resignation shall not
take effect until a Successor Guarantee Trustee has been appointed and has
accepted such appointment by an instrument in writing executed by such Successor
Guarantee Trustee and delivered to the Guarantor and the resigning Guarantee
Trustee.

         (d) If no Successor Guarantee Trustee shall have been appointed and
accepted appointment as provided in this Section 3.2 within 60 days after
delivery of an instrument of removal or resignation, the Guarantee Trustee
resigning or being removed may petition any court of competent jurisdiction for
appointment of a Successor Guarantee Trustee. Such court may thereupon, after
prescribing such notice, if any, as it may deem proper, appoint a Successor
Guarantee Trustee.

         (e) No Guarantee Trustee shall be liable for the acts or omissions to
act of any Successor Guarantee Trustee.

         (f) Upon termination of this Guarantee or removal or resignation of the
Guarantee Trustee pursuant to this Section 3.2, the Guarantor shall pay to the
Guarantee Trustee all amounts owing to the Guarantee Trustee under Sections 7.2
and 7.3 accrued to the date of such termination, removal or resignation.

                                   ARTICLE IV

                                    GUARANTEE

         SECTION 4.1. GUARANTEE.

         (a) The Guarantor irrevocably and unconditionally agrees to pay in full
to the Holders the Guarantee Payments (without duplication of amounts
theretofore paid by the Issuer), as and when due, regardless of any defense
(except the defense of payment by the Issuer), right of set-off or counterclaim
that the Issuer may have or assert. The Guarantor's obligation to make a
Guarantee Payment may be satisfied by direct payment of the required amounts by
the Guarantor to the Holders or by causing the Issuer to pay such amounts to the
Holders.

                                       8

<PAGE>


         (b) The Guarantor hereby also agrees to assume any and all Obligations
of the Issuer and in the event any such Obligation is not so assumed, subject to
the terms and conditions hereof, the Guarantor hereby irrevocably and
unconditionally guarantees to each Beneficiary the full payment, when and as
due, of any and all Obligations to such Beneficiaries. This Guarantee is
intended to be for the benefit of, and to be enforceable by, all such
Beneficiaries, whether or not such Beneficiaries have received notice hereof.

         SECTION 4.2. WAIVER OF NOTICE AND DEMAND. The Guarantor hereby waives
notice of acceptance of this Guarantee and of any liability to which it applies
or may apply, presentment, demand for payment, any right to require a proceeding
first against the Issuer or any other Person before proceeding against the
Guarantor, protest, notice of nonpayment, notice of dishonor, notice of
redemption and all other notices and demands.

         SECTION 4.3. OBLIGATIONS NOT AFFECTED. The obligations, covenants,
agreements and duties of the Guarantor under this Guarantee shall in no way be
affected or impaired by reason of the happening from time to time of any of the
following:

         (a) the release or waiver, by operation of law or otherwise, of the
performance or observance by the Issuer of any express or implied agreement,
covenant, term or condition relating to the Capital Securities to be performed
or observed by the Issuer;

         (b) the extension of time for the payment by the Issuer of all or any
portion of the Distributions, Redemption Price, Special Redemption Price,
Liquidation Distribution or any other sums payable under the terms of the
Capital Securities or the extension of time for the performance of any other
obligation under, arising out of or in connection with, the Capital Securities
(other than an extension of time for payment of Distributions, Redemption Price,
Special Redemption Price, Liquidation Distribution or other sum payable that
results from the extension of any interest payment period on the Debentures or
any extension of the maturity date of the Debentures permitted by the
Indenture);

         (c) any failure, omission, delay or lack of diligence on the part of
the Holders to enforce, assert or exercise any right, privilege, power or remedy
conferred on the Holders pursuant to the terms of the Capital Securities, or any
action on the part of the Issuer granting indulgence or extension of any kind;

         (d) the voluntary or involuntary liquidation, dissolution, sale of any
collateral, receivership, insolvency, bankruptcy, assignment for the benefit of
creditors, reorganization, arrangement, composition or readjustment of debt of,
or other similar proceedings affecting, the Issuer or any of the assets of the
Issuer;

         (e) any invalidity of, or defect or deficiency in, the Capital
Securities;

         (f) the settlement or compromise of any obligation guaranteed hereby or
hereby incurred; or

         (g) any other circumstance whatsoever that might otherwise constitute a
legal or equitable discharge or defense of a guarantor, it being the intent of
this Section 4.3 that the obligations of the Guarantor hereunder shall be
absolute and unconditional under any and all circumstances.

         There shall be no obligation of the Holders to give notice to, or
obtain consent of, the Guarantor with respect to the happening of any of the
foregoing.

                                       9

<PAGE>


         SECTION 4.4. RIGHTS OF HOLDERS.

         (a) The Holders of a Majority in liquidation amount of the Capital
Securities have the right to direct the time, method and place of conducting any
proceeding for any remedy available to the Guarantee Trustee in respect of this
Guarantee or to direct the exercise of any trust or power conferred upon the
Guarantee Trustee under this Guarantee; provided, however, that (subject to
Section 2.1) the Guarantee Trustee shall have the right to decline to follow any
such direction if the Guarantee Trustee being advised by counsel determines that
the action or proceeding so directed may not lawfully be taken or if the
Guarantee Trustee in good faith by its board of directors or trustees, executive
committees or a trust committee of directors or trustees and/or Responsible
Officers shall determine that the action or proceedings so directed would
involve the Guarantee Trustee in personal liability.

         (b) Any Holder of Capital Securities may institute a legal proceeding
directly against the Guarantor to enforce the Guarantee Trustee's rights under
this Guarantee, without first instituting a legal proceeding against the Issuer,
the Guarantee Trustee or any other Person. The Guarantor waives any right or
remedy to require that any such action be brought first against the Issuer, the
Guarantee Trustee or any other Person before so proceeding directly against the
Guarantor.

         SECTION 4.5. GUARANTEE OF PAYMENT. This Guarantee creates a guarantee
of payment and not of collection.

         SECTION 4.6. SUBROGATION. The Guarantor shall be subrogated to all (if
any) rights of the Holders of Capital Securities against the Issuer in respect
of any amounts paid to such Holders by the Guarantor under this Guarantee;
provided, however, that the Guarantor shall not (except to the extent required
by mandatory provisions of law) be entitled to enforce or exercise any right
that it may acquire by way of subrogation or any indemnity, reimbursement or
other agreement, in all cases as a result of payment under this Guarantee, if,
after giving effect to any such payment, any amounts are due and unpaid under
this Guarantee. If any amount shall be paid to the Guarantor in violation of the
preceding sentence, the Guarantor agrees to hold such amount in trust for the
Holders and to pay over such amount to the Holders.

         SECTION 4.7. INDEPENDENT OBLIGATIONS. The Guarantor acknowledges that
its obligations hereunder are independent of the obligations of the Issuer with
respect to the Capital Securities and that the Guarantor shall be liable as
principal and as debtor hereunder to make Guarantee Payments pursuant to the
terms of this Guarantee notwithstanding the occurrence of any event referred to
in subsections (a) through (g), inclusive, of Section 4.3 hereof.

         SECTION 4.8. ENFORCEMENT BY A BENEFICIARY. A Beneficiary may enforce
the obligations of the Guarantor contained in Section 4.1(b) directly against
the Guarantor and the Guarantor waives any right or remedy to require that any
action be brought against the Issuer or any other person or entity before
proceeding against the Guarantor. The Guarantor shall be subrogated to all
rights (if any) of any Beneficiary against the Issuer in respect of any amounts
paid to the Beneficiaries by the Guarantor under this Guarantee; provided,
however, that the Guarantor shall not (except to the extent required by
mandatory provisions of law) be entitled to enforce or exercise any rights that
it may acquire by way of subrogation or any indemnity, reimbursement or other
agreement, in all cases as a result of payment under this Guarantee, if at the
time of any such payment, and after giving effect to such payment, any amounts
are due and unpaid under this Guarantee.

                                       10

<PAGE>


                                   ARTICLE V

                    LIMITATION OF TRANSACTIONS; SUBORDINATION

         SECTION 5.1. LIMITATION OF TRANSACTIONS. So long as any Capital
Securities remain outstanding, if (a) there shall have occurred and be
continuing an Event of Default or a Declaration Event of Default or (b) the
Guarantor shall have selected an Extension Period as provided in the Declaration
and such period, or any extension thereof, shall have commenced and be
continuing, then the Guarantor shall not and shall not permit any Affiliate to
(x) declare or pay any dividends or distributions on, or redeem, purchase,
acquire, or make a liquidation payment with respect to, any of the Guarantor's
or such Affiliate's capital stock (other than payments of dividends or
distributions to the Guarantor) or make any guarantee payments with respect to
the foregoing or (y) make any payment of principal of or interest or premium, if
any, on or repay, repurchase or redeem any debt securities of the Guarantor or
any Affiliate that rank pari passu in all respects with or junior in interest to
the Debentures (other than, with respect to clauses (x) and (y) above, (i)
repurchases, redemptions or other acquisitions of shares of capital stock of the
Guarantor in connection with any employment contract, benefit plan or other
similar arrangement with or for the benefit of one or more employees, officers,
directors or consultants, in connection with a dividend reinvestment or
stockholder stock purchase plan or in connection with the issuance of capital
stock of the Guarantor (or securities convertible into or exercisable for such
capital stock) as consideration in an acquisition transaction entered into prior
to the occurrence of the Event of Default, Declaration Event of Default or
Extension Period, as applicable, (ii) as a result of any exchange or conversion
of any class or series of the Guarantor's capital stock (or any capital stock of
a subsidiary of the Guarantor) for any class or series of the Guarantor's
capital stock or of any class or series of the Guarantor's indebtedness for any
class or series of the Guarantor's capital stock, (iii) the purchase of
fractional interests in shares of the Guarantor's capital stock pursuant to the
conversion or exchange provisions of such capital stock or the security being
converted or exchanged, (iv) any declaration of a dividend in connection with
any stockholders' rights plan, or the issuance of rights, stock or other
property under any stockholders' rights plan, or the redemption or repurchase of
rights pursuant thereto, (v) any dividend in the form of stock, warrants,
options or other rights where the dividend stock or the stock issuable upon
exercise of such warrants, options or other rights is the same stock as that on
which the dividend is being paid or ranks pari passu with or junior to such
stock and any cash payments in lieu of fractional shares issued in connection
therewith, or (vi) payments under this Guarantee).

         SECTION 5.2. RANKING. This Guarantee will constitute an unsecured
obligation of the Guarantor and will rank subordinate and junior in right of
payment to all present and future Senior Indebtedness (as defined in the
Indenture) of the Guarantor. By their acceptance thereof, each Holder of Capital
Securities agrees to the foregoing provisions of this Guarantee and the other
terms set forth herein.

         The right of the Guarantor to participate in any distribution of assets
of any of its subsidiaries upon any such subsidiary's liquidation or
reorganization or otherwise is subject to the prior claims of creditors of that
subsidiary, except to the extent the Guarantor may itself be recognized as a
creditor of that subsidiary. Accordingly, the Guarantor's obligations under this
Guarantee will be effectively subordinated to all existing and future
liabilities of the Guarantor's subsidiaries, and claimants should look only to
the assets of the Guarantor for payments hereunder. This Guarantee does not
limit the incurrence or issuance of other secured or unsecured debt of the
Guarantor, including Senior Indebtedness of the Guarantor, under any indenture
that the Guarantor may enter into in the future or otherwise.

                                       11

<PAGE>


                                   ARTICLE VI

                                   TERMINATION

         SECTION 6.1. TERMINATION. This Guarantee shall terminate as to the
Capital Securities (i) upon full payment of the Redemption Price or Special
Redemption Price of all Capital Securities then outstanding, (ii) upon the
distribution of all of the Debentures to the Holders of all of the Capital
Securities or (iii) upon full payment of the amounts payable in accordance with
the Declaration upon dissolution of the Issuer. This Guarantee will continue to
be effective or will be reinstated, as the case may be, if at any time any
Holder of Capital Securities must restore payment of any sums paid under the
Capital Securities or under this Guarantee.

                                  ARTICLE VII

                                 INDEMNIFICATION

         SECTION 7.1. EXCULPATION.

         (a) No Indemnified Person shall be liable, responsible or accountable
in damages or otherwise to the Guarantor or any Covered Person for any loss,
damage or claim incurred by reason of any act or omission performed or omitted
by such Indemnified Person in good faith in accordance with this Guarantee and
in a manner that such Indemnified Person reasonably believed to be within the
scope of the authority conferred on such Indemnified Person by this Guarantee or
by law, except that an Indemnified Person shall be liable for any such loss,
damage or claim incurred by reason of such Indemnified Person's negligence or
willful misconduct with respect to such acts or omissions.

         (b) An Indemnified Person shall be fully protected in relying in good
faith upon the records of the Issuer or the Guarantor and upon such information,
opinions, reports or statements presented to the Issuer or the Guarantor by any
Person as to matters the Indemnified Person reasonably believes are within such
other Person's professional or expert competence and who, if selected by such
Indemnified Person, has been selected with reasonable care by such Indemnified
Person, including information, opinions, reports or statements as to the value
and amount of the assets, liabilities, profits, losses, or any other facts
pertinent to the existence and amount of assets from which Distributions to
Holders of Capital Securities might properly be paid.

         SECTION 7.2. INDEMNIFICATION.

         (a) The Guarantor agrees to indemnify each Indemnified Person for, and
to hold each Indemnified Person harmless against, any and all loss, liability,
damage, claim or expense incurred without negligence or willful misconduct on
the part of the Indemnified Person, arising out of or in connection with the
acceptance or administration of the trust or trusts hereunder, including, but
not limited to, the costs and expenses (including reasonable legal fees and
expenses) of the Indemnified Person defending itself against, or investigating,
any claim or liability in connection with the exercise or performance of any of
the Indemnified Person's powers or duties hereunder. The obligation to indemnify
as set forth in this Section 7.2 shall survive the resignation or removal of the
Guarantee Trustee and the termination of this Guarantee.

         (b) Promptly after receipt by an Indemnified Person under this Section
7.2 of notice of the commencement of any action, such Indemnified Person will,
if a claim in respect thereof is to be made against the Guarantor under this
Section 7.2, notify the Guarantor in writing of the commencement thereof; but
the failure so to notify the Guarantor (i) will not relieve the Guarantor from
liability under

                                       12

<PAGE>

paragraph (a) above unless and to the extent that the Guarantor did not
otherwise learn of such action and such failure results in the forfeiture by the
Guarantor of substantial rights and defenses and (ii) will not, in any event,
relieve the Guarantor from any obligations to any Indemnified Person other than
the indemnification obligation provided in paragraph (a) above. The Guarantor
shall be entitled to appoint counsel of the Guarantor's choice at the
Guarantor's expense to represent the Indemnified Person in any action for which
indemnification is sought (in which case the Guarantor shall not thereafter be
responsible for the fees and expenses of any separate counsel retained by the
Indemnified Person or Persons except as set forth below); provided, however,
that such counsel shall be reasonably satisfactory to the Indemnified Person.
Notwithstanding the Guarantor's election to appoint counsel to represent the
Guarantor in an action, the Indemnified Person shall have the right to employ
separate counsel (including local counsel), and the Guarantor shall bear the
reasonable fees, costs and expenses of such separate counsel if (i) the use of
counsel chosen by the Guarantor to represent the Indemnified Person would
present such counsel with a conflict of interest, (ii) the actual or potential
defendants in, or targets of, any such action include both the Indemnified
Person and the Guarantor and the Indemnified Person shall have reasonably
concluded that there may be legal defenses available to it and/or other
Indemnified Person(s) which are different from or additional to those available
to the Guarantor, (iii) the Guarantor shall not have employed counsel
satisfactory to the Indemnified Person to represent the Indemnified Person
within a reasonable time after notice of the institution of such action or (iv)
the Guarantor shall authorize the Indemnified Person to employ separate counsel
at the expense of the Guarantor. The Guarantor will not, without the prior
written consent of the Indemnified Persons, settle or compromise or consent to
the entry of any judgment with respect to any pending or threatened claim,
action, suit or proceeding in respect of which indemnification or contribution
may be sought hereunder (whether or not the Indemnified Persons are actual or
potential parties to such claim or action) unless such settlement, compromise or
consent includes an unconditional release of each Indemnified Person from all
liability arising out of such claim, action, suit or proceeding.

         SECTION 7.3. COMPENSATION; REIMBURSEMENT OF EXPENSES. The Guarantor
agrees:

         (a) to pay to the Guarantee Trustee from time to time such compensation
for all services rendered by it hereunder as the parties shall agree to from
time to time (which compensation shall not be limited by any provision of law in
regard to the compensation of a trustee of an express trust); and

         (b) except as otherwise expressly provided herein, to reimburse the
Guarantee Trustee upon request for all reasonable expenses, disbursements and
advances incurred or made by it in accordance with any provision of this
Guarantee (including the reasonable compensation and the expenses and
disbursements of its agents and counsel), except any such expense, disbursement
or advance as may be attributable to its negligence or willful misconduct.

         The provisions of this Section 7.3 shall survive the resignation or
removal of the Guarantee Trustee and the termination of this Guarantee.

                                  ARTICLE VIII

                                  MISCELLANEOUS

         SECTION 8.1. SUCCESSORS AND ASSIGNS. All guarantees and agreements
contained in this Guarantee shall bind the successors, assigns, receivers,
trustees and representatives of the Guarantor and shall inure to the benefit of
the Holders of the Capital Securities then outstanding. Except in connection
with any merger or consolidation of the Guarantor with or into another entity or
any sale, transfer or lease of the Guarantor's assets to another entity, in each
case, to the extent permitted under the Indenture, the Guarantor may not assign

                                       13

<PAGE>

its rights or delegate its obligations under this Guarantee without the prior
approval of the Holders of at least a Majority in liquidation amount of the
Capital Securities.

        SECTION 8.2. AMENDMENTS. Except with respect to any changes that do not
adversely affect the rights of Holders of the Capital Securities in any material
respect (in which case no consent of Holders will be required), this Guarantee
may be amended only with the prior approval of the Holders of not less than a
Majority in liquidation amount of the Capital Securities. The provisions of the
Declaration with respect to amendments thereof apply to the giving of such
approval.

        SECTION 8.3. NOTICES. All notices provided for in this Guarantee shall
be in writing, duly signed by the party giving such notice, and shall be
delivered, telecopied or mailed by first class mail, as follows:

        (a) If given to the Guarantee Trustee, at the Guarantee Trustee's
mailing address set forth below (or such other address as the Guarantee Trustee
may give notice of to the Holders of the Capital Securities and the Guarantor):

        State Street Bank and Trust Company of Connecticut, National Association
        225 Asylum Street, Goodwin Square
        Hartford, Connecticut  06103
        Attention:  Corporate Trust Department
        Telecopy:  860-244-1889

        With a copy to:

        State Street Bank and Trust Company
        P.O. Box 778
        Boston, Massachusetts  02102-0778
        Attention:  Paul D. Allen, Corporate Trust Department
        Telecopy:  617-662-1462

        (b) If given to the Guarantor, at the Guarantor's mailing address set
forth below (or such other address as the Guarantor may give notice of to the
Holders of the Capital Securities and to the Guarantee Trustee):

        U.S.B. Holding Co., Inc.
        100 Dutch Hill Road
        Orangeburg, New York  10962
        Attention:  Steven T. Sabatini
        Telecopy:  845-365-4695

        (c) If given to any Holder of the Capital Securities, at the address
set forth on the books and records of the Issuer.

        All such notices shall be deemed to have been given when received in
person, telecopied with receipt confirmed, or mailed by first class mail,
postage prepaid, except that if a notice or other document is refused delivery
or cannot be delivered because of a changed address of which no notice was
given, such notice or other document shall be deemed to have been delivered on
the date of such refusal or inability to deliver.

        SECTION 8.4. BENEFIT. This Guarantee is solely for the benefit of the
Beneficiaries and, subject to Section 2.1(a), is not separately transferable
from the Capital Securities.

                                       14

<PAGE>


         SECTION 8.5. GOVERNING LAW. THIS GUARANTEE SHALL BE GOVERNED BY, AND
CONSTRUED AND INTERPRETED IN ACCORDANCE WITH, THE LAW OF THE STATE OF NEW YORK,
WITHOUT REGARD TO CONFLICT OF LAWS PRINCIPLES THEREOF (OTHER THAN SECTION 5-1401
OF THE NEW YORK GENERAL OBLIGATIONS LAW).

         SECTION 8.6. COUNTERPARTS. This Guarantee may be executed in one or
more counterparts, each of which shall be an original, but all of which taken
together shall constitute one and the same instrument.

         SECTION 8.7 SEPARABILITY. In case one or more of the provisions
contained in this Guarantee shall for any reason be held to be invalid, illegal
or unenforceable in any respect, such invalidity, illegality or unenforceability
shall not affect any other provisions of this Guarantee, but this Guarantee
shall be construed as if such invalid or illegal or unenforceable provision had
never been contained herein.



                     Signatures appear on the following page









                                       15
<PAGE>


         THIS GUARANTEE is executed as of the day and year first above written.



                               U.S.B. HOLDING CO., INC., as Guarantor

                               By:
                                  ----------------------------------------------
                                  Name:
                                  Title:



                               STATE STREET BANK AND TRUST COMPANY
                               OF CONNECTICUT, NATIONAL
                               ASSOCIATION, as Guarantee Trustee

                               By:
                                  ----------------------------------------------
                                  Name:
                                  Title:








                                       16