<PAGE>

                                                                   Exhibit 10.13

================================================================================

                               GUARANTEE AGREEMENT

                                     between

                              UNITED BANCORP, INC.,
                                  As Guarantor,

                                       and

                            WILMINGTON TRUST COMPANY,
                              As Guarantee Trustee

                          Dated as of November 17, 2005

                        UNITED BANCORP STATUTORY TRUST I

================================================================================

<PAGE>

                                TABLE OF CONTENTS

<TABLE>
<S>                                                                           <C>
ARTICLE I      INTERPRETATION AND DEFINITIONS..............................    2
SECTION 1.1    Interpretation..............................................    2
SECTION 1.2    Definitions.................................................    2

ARTICLE II     REPORTS.....................................................    6
SECTION 2.1    List of Holders.............................................    6
SECTION 2.2    Periodic Reports to the Guarantee Trustee...................    6
SECTION 2.3    Event of Default; Waiver....................................    6
SECTION 2.4    Event of Default; Notice....................................    6

ARTICLE III    POWERS, DUTIES AND RIGHTS OF THE GUARANTEE TRUSTEE..........    7
SECTION 3.1    Powers and Duties of the Guarantee Trustee..................    7
SECTION 3.2    Certain Rights of the Guarantee Trustee.....................    8
SECTION 3.3    Compensation................................................   10
SECTION 3.4    Indemnity...................................................   10
SECTION 3.5    Securities..................................................   11

ARTICLE IV     GUARANTEE TRUSTEE...........................................   11
SECTION 4.1    Guarantee Trustee; Eligibility..............................   11
SECTION 4.2    Appointment, Removal and Resignation of the Guarantee
               Trustee.....................................................   12

ARTICLE V      GUARANTEE...................................................   12
SECTION 5.1    Guarantee...................................................   12
SECTION 5.2    Waiver of Notice and Demand.................................   13
SECTION 5.3    Obligations Not Affected....................................   13
SECTION 5.4    Rights of Holders...........................................   14
SECTION 5.5    Guarantee of Payment........................................   14
SECTION 5.6    Subrogation.................................................   14
SECTION 5.7    Independent Obligations.....................................   14
SECTION 5.8    Enforcement.................................................   15

ARTICLE VI     COVENANTS AND SUBORDINATION.................................   15
SECTION 6.1    Dividends, Distributions and Payments.......................   15
SECTION 6.2    Subordination...............................................   16
SECTION 6.3    Pari Passu Guarantees.......................................   16

ARTICLE VII    TERMINATION.................................................   16
SECTION 7.1    Termination.................................................   16

ARTICLE VIII   MISCELLANEOUS...............................................   17
SECTION 8.1    Successors and Assigns......................................   17
SECTION 8.2    Amendments..................................................   17
SECTION 8.3    Notices.....................................................   17
SECTION 8.4    Benefit.....................................................   18
</TABLE>


                                        i

<PAGE>

<TABLE>
<S>                                                                           <C>
SECTION 8.5    Governing Law...............................................   18
SECTION 8.6    Submission to Jurisdiction..................................   18
SECTION 8.7    Counterparts................................................   19
</TABLE>


                                       ii

<PAGE>

     GUARANTEE AGREEMENT, dated as of November 17, 2005, executed and delivered
by UNITED BANCORP, INC., an Ohio corporation (the "Guarantor") having its
principal office at 201 South Fourth Street, P.O. Box 10, Martins Ferry, Ohio
43935-0010, and WILMINGTON TRUST COMPANY, a Delaware banking corporation, as
trustee (in such capacity, the "Guarantee Trustee"), for the benefit of the
Holders (as defined herein) from time to time of the Preferred Securities (as
defined herein) of United Bancorp Statutory Trust I, a Delaware statutory trust
(the "Issuer").

                                   WITNESSETH:

     WHEREAS, pursuant to an Amended and Restated Trust Agreement, dated as of
the date hereof (the "Trust Agreement"), among the Guarantor, as Depositor, the
Property Trustee, the Delaware Trustee and the Administrative Trustees named
therein and the holders from time to time of the Preferred Securities (as
hereinafter defined), the Issuer is issuing $4,000,000 aggregate Liquidation
Amount (as defined in the Trust Agreement) of its Preferred Securities
(Liquidation Amount $1,000 per preferred security) (the "Preferred Securities")
representing preferred undivided beneficial interests in the assets of the
Issuer and having the terms set forth in the Trust Agreement;

     WHEREAS, the Preferred Securities will be issued by the Issuer and the
proceeds thereof, together with the proceeds from the issuance of the Issuer's
Common Securities (as defined below), will be used to purchase the Notes (as
defined in the Trust Agreement) of the Guarantor; and

     WHEREAS, as incentive for the Holders to purchase Preferred Securities the
Guarantor desires irrevocably and unconditionally to agree, to the extent set
forth herein, to pay to the Holders of the Preferred Securities the Guarantee
Payments (as defined herein) and to make certain other payments on the terms and
conditions set forth herein.

     NOW, THEREFORE, in consideration of the purchase by each Holder of
Preferred Securities, which purchase the Guarantor hereby agrees shall benefit
the Guarantor, the Guarantor executes and delivers this Guarantee Agreement to
provide as follows for the benefit of the Holders from time to time of the
Preferred Securities:

<PAGE>

                                    ARTICLE I

                         INTERPRETATION AND DEFINITIONS

     SECTION 1.1 Interpretation.

     In this Guarantee Agreement, unless the context otherwise requires:

          (a) capitalized terms used in this Guarantee Agreement but not defined
     in the preamble hereto have the respective meanings assigned to them in
     Section 1.2;

          (b) the words "include", "includes" and "including" shall be deemed to
     be followed by the phrase "without limitation";

          (c) all references to "the Guarantee Agreement" or "this Guarantee
     Agreement" are to this Guarantee Agreement as modified, supplemented or
     amended from time to time;

          (d) all references in this Guarantee Agreement to Articles and
     Sections are to Articles and Sections of this Guarantee Agreement unless
     otherwise specified;

          (e) the words "hereby", "herein", "hereof" and "hereunder" and other
     words of similar import refer to this Guarantee Agreement as a whole and
     not to any particular Article, Section or other subdivision;

          (f) a reference to the singular includes the plural and vice versa;
     and

          (g) the masculine, feminine or neuter genders used herein shall
     include the masculine, feminine and neuter genders.

     SECTION 1.2 Definitions.

          As used in this Guarantee Agreement, the terms set forth below shall,
     unless the context otherwise requires, have the following meanings:

          "Affiliate" of any specified Person means any other Person directly or
     indirectly controlling or controlled by or under direct or indirect common
     control with such specified Person; provided, that the Issuer shall not be
     deemed to be an Affiliate of the Guarantor. For the purposes of this
     definition, "control" when used with respect to any specified Person means
     the power to direct the management and policies of such Person, directly or
     indirectly, whether through the ownership of voting securities, by contract
     or otherwise; and the terms "controlling" and "controlled" have meanings
     correlative to the foregoing.

          "Beneficiaries" means any Person to whom the Issuer is or hereafter
     becomes indebted or liable.


                                        2

<PAGE>

          "Board of Directors" means either the board of directors of the
     Guarantor or any duly authorized committee of that board.

          "Common Securities" means the securities representing common undivided
     beneficial interests in the assets of the Issuer.

          "Debt" means with respect to any Person, whether recourse is to all or
     a portion of the assets of such Person, whether currently existing or
     hereafter incurred, and whether or not contingent and without duplication,
     (i) every obligation of such Person for money borrowed; (ii) every
     obligation of such Person evidenced by bonds, debentures, notes or other
     similar instruments, including obligations incurred in connection with the
     acquisition of property, assets or businesses; (iii) every reimbursement
     obligation of such Person with respect to letters of credit, bankers'
     acceptances or similar facilities issued for the account of such Person;
     (iv) every obligation of such Person issued or assumed as the deferred
     purchase price of property or services (but excluding trade accounts
     payable arising in the ordinary course of business); (v) every capital
     lease obligation of such Person; (vi) all indebtedness of such Person,
     whether incurred on or prior to the date of this Guarantee Agreement or
     thereafter incurred, for claims in respect of derivative products,
     including interest rate, foreign exchange rate and commodity forward
     contracts, options, swaps and similar arrangements; (vii) every obligation
     of the type referred to in clauses (i) through (vi) of another Person and
     all dividends of another Person the payment of which, in either case, such
     Person has guaranteed or is responsible or liable for, directly or
     indirectly, as obligor or otherwise; and (viii) any renewals, extensions,
     refundings, amendments or modifications of any obligation of the type
     referred to in clauses (i) through (vii).

          "Event of Default" means a default by the Guarantor on any of its
     payment or other obligations under this Guarantee Agreement; provided, that
     except with respect to a default in payment of any Guarantee Payments, the
     Guarantor shall have received notice of default from the Guarantee Trustee
     and shall not have cured such default within thirty (30) days after receipt
     of such notice.

          "Guarantee Payments" means the following payments or distributions,
     without duplication, with respect to the Preferred Securities, to the
     extent not paid or made by or on behalf of the Issuer: (i) any accumulated
     and unpaid Distributions (as defined in the Trust Agreement) required to be
     paid on the Preferred Securities, to the extent the Issuer shall have funds
     on hand available therefor at such time, (ii) the Redemption Price with
     respect to any Preferred Securities to the extent the Issuer shall have
     funds on hand available therefor at such time, and (iii) upon a voluntary
     or involuntary termination, winding up or liquidation of the Issuer, unless
     Notes are distributed to the Holders, the lesser of (a) the aggregate of
     the Liquidation Amount of $1,000 per Preferred Security plus accumulated
     and unpaid Distributions on the Preferred Securities to the date of
     payment, to the extent that the Issuer shall have funds available therefor
     at such


                                        3

<PAGE>

     time and (b) the amount of assets of the Issuer remaining available for
     distribution to Holders in liquidation of the Issuer after satisfaction of
     liabilities to creditors of the Issuer in accordance with applicable law
     (in either case, the "Liquidation Distribution").

          "Guarantee Trustee" means Wilmington Trust Company in its capacity as
     trustee hereunder, until a Successor Guarantee Trustee, as defined below,
     has been appointed and has accepted such appointment pursuant to the terms
     of this Guarantee Agreement, and thereafter means each such Successor
     Guarantee Trustee.

          "Holder" means any holder, as registered on the books and records of
     the Issuer, of any Preferred Securities; provided, that, in determining
     whether the holders of the requisite percentage of Preferred Securities
     have given any request, notice, consent or waiver hereunder, "Holder" shall
     not include the Guarantor, the Guarantee Trustee or any Affiliate of the
     Guarantor or the Guarantee Trustee.

          "Indenture" means the Junior Subordinated Indenture, dated as of the
     date hereof, as supplemented and amended, between the Guarantor and
     Wilmington Trust Company, as trustee.

          "List of Holders" has the meaning specified in Section 2.1.

          "Majority in Liquidation Amount of the Preferred Securities" means a
     vote by the Holder(s), voting separately as a class, of more than fifty
     percent (50%) of the aggregate Liquidation Amount of all then outstanding
     Preferred Securities issued by the Issuer.

          "Obligations" means any costs, expenses or liabilities (but not
     including liabilities related to taxes) of the Issuer, other than
     obligations of the Issuer to pay to holders of any Trust Securities the
     amounts due such holders pursuant to the terms of the Trust Securities.

          "Officers' Certificate" means, with respect to any Person, a
     certificate signed by the Chief Executive Officer, Chief Financial Officer,
     President or a Vice President of such Person, and by the Treasurer, an
     Assistant Treasurer, the Secretary or an Assistant Secretary of such
     Person, and delivered to the Guarantee Trustee. Any Officers' Certificate
     delivered with respect to compliance with a condition or covenant provided
     for in this Guarantee Agreement (other than the certificate provided
     pursuant to Section 2.4) shall include:

          (a) a statement that each officer signing the Officers' Certificate
     has read the covenant or condition and the definitions relating thereto;

          (b) a brief statement of the nature and scope of the examination or
     investigation undertaken by each officer in rendering the Officers'
     Certificate;


                                        4

<PAGE>

          (c) a statement that each officer has made such examination or
     investigation as, in such officer's opinion, is necessary to enable such
     officer to express an informed opinion as to whether or not such covenant
     or condition has been complied with; and

          (d) a statement as to whether, in the opinion of each officer, such
     condition or covenant has been complied with.

          "Person" means a legal person, including any individual, corporation,
     estate, partnership, joint venture, association, joint stock company,
     limited liability company, trust, unincorporated association, government or
     any agency or political subdivision thereof or any other entity of whatever
     nature.

          "Responsible Officer" means, with respect to the Guarantee Trustee,
     any Senior Vice President, any Vice President, any Assistant Vice
     President, the Secretary, any Assistant Secretary, the Treasurer, any
     Assistant Treasurer, any Trust Officer or Assistant Trust Officer or any
     other officer in the Corporate Trust Office of the Guarantee Trustee with
     direct responsibility for the administration of this Guarantee Agreement
     and also means, with respect to a particular corporate trust matter, any
     other officer to whom such matter is referred because of that officer's
     knowledge of and familiarity with the particular subject.

          "Senior Debt" means the principal of and any premium and interest on
     (including interest accruing on or after the filing of any petition in
     bankruptcy or for reorganization relating to the Guarantor whether or not
     such claim for post-petition interest is allowed in such proceeding) all
     Debt of the Guarantor, whether incurred on or prior to the date of the
     Indenture or thereafter incurred, unless it is provided in the instrument
     creating or evidencing the same or pursuant to which the same is
     outstanding, that such obligations are not superior in right of payment to
     the Preferred Securities; provided, however, that if the Guarantor is
     subject to the regulation and supervision of an "appropriate Federal
     banking agency" within the meaning of 12 U.S.C. 1813(q), the Guarantor
     shall have received the approval of such appropriate Federal banking agency
     prior to issuing any such obligation if not otherwise generally approved;
     provided further, that Senior Debt shall not include any other debt
     securities, and guarantees in respect of such debt securities, issued to
     any trust other than the Issuer (or a trustee of such trust), partnership
     or other entity affiliated with the Guarantor that is a financing vehicle
     of the Guarantor (a "financing entity"), in connection with the issuance by
     such financing entity of equity securities or other securities that are
     treated as equity capital for regulatory capital purposes guaranteed by the
     Guarantor pursuant to an instrument that ranks pari passu with or junior in
     right of payment to this Guarantee Agreement.

          "Successor Guarantee Trustee" means a successor Guarantee Trustee
     possessing the qualifications to act as Guarantee Trustee under Section
     4.1.


                                        5

<PAGE>

          "Trust Indenture Act" means the Trust Indenture Act of 1939, as
     amended and as in effect on the date of this Guarantee Agreement.

Capitalized or otherwise defined terms used but not otherwise defined herein
shall have the meanings assigned to such terms in the Trust Agreement as in
effect on the date hereof.

                                   ARTICLE II

                                     REPORTS

     SECTION 2.1 List of Holders.

     The Guarantor shall furnish or cause to be furnished to the Guarantee
Trustee at such times as the Guarantee Trustee may request in writing, within
thirty (30) days after the receipt by the Guarantor of any such request, a list,
in such form as the Guarantee Trustee may reasonably require, of the names and
addresses of the Holders (the "List of Holders") as of a date not more than
fifteen (15) days prior to the time such list is furnished, in each case to the
extent such information is in the possession or control of the Guarantor and is
not identical to a previously supplied list of Holders or has not otherwise been
received by the Guarantee Trustee in its capacity as such. The Guarantee Trustee
may destroy any List of Holders previously given to it on receipt of a new List
of Holders.

     SECTION 2.2 Periodic Reports to the Guarantee Trustee.

     The Guarantor shall deliver to the Guarantee Trustee, within one hundred
and twenty (120) days after the end of each fiscal year of the Guarantor ending
after the date of this Guarantee Agreement, an Officers' Certificate covering
the preceding fiscal year, stating whether or not to the knowledge of the
signers thereof the Guarantor is in default in the performance or observance of
any of the terms or provisions or any of the conditions of this Guarantee
Agreement (without regard to any period of grace or requirement of notice
provided hereunder) and, if the Guarantor shall be in default thereof,
specifying all such defaults and the nature and status thereof of which they
have knowledge.

     SECTION 2.3 Event of Default; Waiver.

     The Holders of a Majority in Liquidation Amount of the Preferred Securities
may, on behalf of the Holders, waive any past Event of Default and its
consequences. Upon such waiver, any such Event of Default shall cease to exist,
and any Event of Default arising therefrom shall be deemed to have been cured,
for every purpose of this Guarantee Agreement, but no such waiver shall extend
to any subsequent or other default or Event of Default or impair any right
consequent therefrom.

     SECTION 2.4 Event of Default; Notice.


                                        6
<PAGE>

          (a) The Guarantee Trustee shall, within ninety (90) days after the
     occurrence of a default, transmit to the Holders notices of all defaults
     actually known to the Guarantee Trustee, unless such defaults have been
     cured or waived before the giving of such notice, provided, that, except in
     the case of a default in the payment of a Guarantee Payment, the Guarantee
     Trustee shall be protected in withholding such notice if and so long as the
     Board of Directors, the executive committee or a trust committee of
     directors and/or Responsible Officers of the Guarantee Trustee in good
     faith determine that the withholding of such notice is in the interests of
     the Holders. For the purpose of this Section 2.4, the term "default" means
     any event that is, or after notice or lapse of time or both would become,
     an Event of Default.

          (b) The Guarantee Trustee shall not be deemed to have knowledge of any
     default or Event of Default unless the Guarantee Trustee shall have
     received written notice, or a Responsible Officer charged with the
     administration of this Guarantee Agreement shall have received written
     notice, of such default or Event of Default from the Guarantor or a Holder.

                                   ARTICLE III

               POWERS, DUTIES AND RIGHTS OF THE GUARANTEE TRUSTEE

     SECTION 3.1 Powers and Duties of the Guarantee Trustee.

          (a) This Guarantee Agreement shall be held by the Guarantee Trustee
     for the benefit of the Holders, and the Guarantee Trustee shall not
     transfer this Guarantee Agreement to any Person except a Holder exercising
     its rights pursuant to Section 5.4(d) or to a Successor Guarantee Trustee
     upon acceptance by such Successor Guarantee Trustee of its appointment to
     act as Successor Guarantee Trustee. The right, title and interest of the
     Guarantee Trustee shall automatically vest in any Successor Guarantee
     Trustee, upon acceptance by such Successor Guarantee Trustee of its
     appointment hereunder, and such vesting and cessation of title shall be
     effective whether or not conveyancing documents have been executed and
     delivered pursuant to the appointment of such Successor Guarantee Trustee.

          (b) The rights, immunities, duties and responsibilities of the
     Guarantee Trustee shall be as provided by this Guarantee Agreement and
     there shall be no other duties or obligations, express or implied, of the
     Guarantee Trustee. Notwithstanding the foregoing, no provisions of this
     Guarantee Agreement shall require the Guarantee Trustee to expend or risk
     its own funds or otherwise incur any financial liability in the performance
     of any of its duties hereunder, or in the exercise of any of its rights or
     powers, if it shall have reasonable grounds for believing that repayment of
     such funds or adequate indemnity against such risk or liability is not
     reasonably assured to it. Whether or not herein expressly so provided,
     every provision of this Guarantee Agreement relating to the conduct or
     affecting the liability of or affording protection to the Guarantee Trustee
     shall be


                                       7

<PAGE>

     subject to the provisions of this Section 3.1. To the extent that, at law
     or in equity, the Guarantee Trustee has duties and liabilities relating to
     the Guarantor or the Holders, the Guarantee Trustee shall not be liable to
     any Holder for the Guarantee Trustee's good faith reliance on the
     provisions of this Guarantee Agreement. The provisions of this Guarantee
     Agreement, to the extent that they restrict the duties and liabilities of
     the Guarantee Trustee otherwise existing at law or in equity, are agreed by
     the Guarantor and the Holders to replace such other duties and liabilities
     of the Guarantee Trustee.

          (c) No provision of this Guarantee Agreement shall be construed to
     relieve the Guarantee Trustee from liability for its own negligent action,
     negligent failure to act or own willful misconduct, except that:

               (i) the Guarantee Trustee shall not be liable for any error of
          judgment made in good faith by a Responsible Officer of the Guarantee
          Trustee, unless it shall be proved that the Guarantee Trustee was
          negligent in ascertaining the pertinent facts upon which such judgment
          was made; and

               (ii) the Guarantee Trustee shall not be liable with respect to
          any action taken or omitted to be taken by it in good faith in
          accordance with the direction of the Holders of not less than a
          Majority in Liquidation Amount of the Preferred Securities relating to
          the time, method and place of conducting any proceeding for any remedy
          available to the Guarantee Trustee, or exercising any trust or power
          conferred upon the Guarantee Trustee under this Guarantee Agreement.

     SECTION 3.2 Certain Rights of the Guarantee Trustee.

          (a) Subject to the provisions of Section 3.1:

               (i) the Guarantee Trustee may conclusively rely and shall be
          fully protected in acting or refraining from acting in good faith and
          in accordance with the terms hereof upon any resolution, certificate,
          statement, instrument, opinion, report, notice, request, direction,
          consent, order, bond, debenture, note, other evidence of indebtedness
          or other paper or document reasonably believed by it to be genuine and
          to have been signed, sent or presented by the proper party or parties;

               (ii) any direction or act of the Guarantor contemplated by this
          Guarantee Agreement shall be sufficiently evidenced by an Officers'
          Certificate unless otherwise prescribed herein;

               (iii) the Guarantee Trustee may consult with counsel, and the
          advice of such counsel shall be full and complete authorization and
          protection in respect of any action taken, suffered or omitted to be
          taken by it hereunder in good faith and in reliance thereon and in
          accordance with such advice. Such counsel may be counsel to the
          Guarantee Trustee,


                                       8

<PAGE>

          the Guarantor or any of its Affiliates and may be one of its
          employees. The Guarantee Trustee shall have the right at any time to
          seek instructions concerning the administration of this Guarantee
          Agreement from any court of competent jurisdiction;

               (iv) the Guarantee Trustee shall be under no obligation to
          exercise any of the rights or powers vested in it by this Guarantee
          Agreement at the request or direction of any Holder, unless such
          Holder shall have provided to the Guarantee Trustee reasonable
          security or indemnity against the costs, expenses (including
          reasonable attorneys' fees and expenses) and liabilities that might be
          incurred by it in complying with such request or direction, including
          such reasonable advances as may be requested by the Guarantee Trustee;
          provided, that, nothing contained in this Section 3.2(a)(iv) shall be
          taken to relieve the Guarantee Trustee, upon the occurrence of an
          Event of Default, of its obligation to exercise the rights and powers
          vested in it by this Guarantee Agreement;

               (v) the Guarantee Trustee shall not be bound to make any
          investigation into the facts or matters stated in any resolution,
          certificate, statement, instrument, opinion, report, notice, request,
          direction, consent, order, bond, debenture, note, other evidence of
          indebtedness or other paper or document, but the Guarantee Trustee, in
          its discretion, may make such further inquiry or investigation into
          such facts or matters as it may see fit, and if the Guarantee Trustee
          shall determine to make such inquiry or investigation, it shall be
          entitled to examine the books, records and premises of the Guarantor,
          personally or by agent or attorney;

               (vi) the Guarantee Trustee may execute any of the trusts or
          powers hereunder or perform any duties hereunder either directly or by
          or through its agents, attorneys, custodians or nominees and the
          Guarantee Trustee shall not be responsible for any misconduct or
          negligence on the part of any such agent, attorney, custodian or
          nominee appointed with due care by it hereunder;

               (vii) whenever in the administration of this Guarantee Agreement
          the Guarantee Trustee shall deem it desirable to receive instructions
          with respect to enforcing any remedy or right hereunder, the Guarantee
          Trustee (A) may request instructions from the Holders of a Majority in
          Liquidation Amount of the Preferred Securities, (B) may refrain from
          enforcing such remedy or right or taking such other action until such
          instructions are received and (C) shall be protected in acting in
          accordance with such instructions;

               (viii) except as otherwise expressly provided by this Guarantee
          Agreement, the Guarantee Trustee shall not be under any obligation to
          take any action that is discretionary under the provisions of this
          Guarantee Agreement;


                                       9

<PAGE>

               (ix) whenever, in the administration of this Guarantee Agreement,
          the Guarantee Trustee shall deem it desirable that a matter be proved
          or established before taking, suffering or omitting to take any action
          hereunder, the Guarantee Trustee (unless other evidence is herein
          specifically prescribed) may, in the absence of bad faith on its part,
          request and rely upon an Officers' Certificate which, upon receipt of
          such request from the Guarantee Trustee, shall be promptly delivered
          by the Guarantor; and

               (x) the Guarantee Trustee shall have no duty to see to any
          recording, filing or registration of any instrument or other writing
          (or any rerecording, refilling or reregistration thereof).

          (b) No provision of this Guarantee Agreement shall be deemed to impose
     any duty or obligation on the Guarantee Trustee to perform any act or acts
     or exercise any right, power, duty or obligation conferred or imposed on it
     in any jurisdiction in which it shall be illegal, or in which the Guarantee
     Trustee shall be unqualified or incompetent in accordance with applicable
     law, to perform any such act or acts or to exercise any such right, power,
     duty or obligation. No permissive power or authority available to the
     Guarantee Trustee shall be construed to be a duty to act in accordance with
     such power and authority.

     SECTION 3.3 Compensation.

     The Guarantor agrees to pay to the Guarantee Trustee from time to time
reasonable compensation for all services rendered by it hereunder (which
compensation shall not be limited by any provisions of law in regard to the
compensation of a trustee of an express trust) and to reimburse the Guarantee
Trustee upon request for all reasonable expenses, disbursements and advances
(including the reasonable fees and expenses of its attorneys and agents)
incurred or made by the Guarantee Trustee in accordance with any provisions of
this Guarantee Agreement.

     SECTION 3.4 Indemnity.

     The Guarantor agrees to indemnify and hold harmless the Guarantee Trustee
(including in its individual capacity) and any of its Affiliates and any of
their officers, directors, shareholders, employees, representatives or agents
from and against any loss, damage, liability, tax (other than income, franchise
or other taxes imposed on amounts paid pursuant to Section 3.3), penalty,
expense or claim of any kind or nature whatsoever incurred without negligence,
bad faith or willful misconduct on its part, arising out of or in connection
with the acceptance or administration of this Guarantee Agreement, including the
costs and expenses of defending itself against any claim or liability in
connection with the exercise or performance of any of its rights, powers or
duties hereunder. The Guarantee Trustee will not claim or exact any lien or
charge on any Guarantee Payments as a result of any amount due to it under this
Guarantee Agreement. This indemnity shall survive the termination of this
Agreement or the resignation or removal of the Guarantee Trustee.


                                       10

<PAGE>

     In no event shall the Guarantee Trustee be liable for any indirect,
special, punitive or consequential loss or damage of any kind whatsoever,
including, but not limited to, lost profits, even if the Guarantee Trustee has
been advised of the likelihood of such loss or damage and regardless of the form
of action.

     In no event shall the Guarantee Trustee be liable for any failure or delay
in the performance of its obligations hereunder because of circumstances beyond
its control, including, but not limited to, acts of God, flood, war (declared or
undeclared), terrorism, fire, riot, embargo or government action, including any
laws, ordinances, regulations, governmental action or the like which delay,
restrict or prohibit the providing of the services contemplated by this
Guarantee Agreement.

     SECTION 3.5 Securities.

     The Guarantee Trustee or any other agent of the Guarantee Trustee, in its
individual or any other capacity, may become the owner or pledgee of Common or
Preferred Securities.

                                   ARTICLE IV

                                GUARANTEE TRUSTEE

     SECTION 4.1 Guarantee Trustee; Eligibility.

          (a) There shall at all times be a Guarantee Trustee which shall:

               (i) not be an Affiliate of the Guarantor; and

               (ii) be a corporation organized and doing business under the laws
          of the United States or of any State thereof, authorized to exercise
          corporate trust powers, having a combined capital and surplus of at
          least fifty million dollars ($50,000,000), subject to supervision or
          examination by Federal or State authority and having an office within
          the United States. If such corporation publishes reports of condition
          at least annually, pursuant to law or to the requirements of such
          supervising or examining authority, then, for the purposes of this
          Section 4.1, the combined capital and surplus of such corporation
          shall be deemed to be its combined capital and surplus as set forth in
          its most recent report of condition so published.

          (b) If at any time the Guarantee Trustee shall cease to be eligible to
     so act under Section 4.1(a), the Guarantee Trustee shall immediately resign
     in the manner and with the effect set out in Section 4.2(c).

          (c) If the Guarantee Trustee has or shall acquire any "conflicting
     interest" within the meaning of Section 310(b) of the Trust Indenture Act,
     the Guarantee Trustee shall either eliminate such interest or resign in the
     manner and with the effect set out in Section 4.2(c).


                                       11

<PAGE>

     SECTION 4.2 Appointment, Removal and Resignation of the Guarantee Trustee.

          (a) Subject to Section 4.2(b), the Guarantee Trustee may be appointed
     or removed without cause at any time by the Guarantor, except during an
     Event of Default.

          (b) The Guarantee Trustee shall not be removed until a Successor
     Guarantee Trustee has been appointed and has accepted such appointment by
     written instrument executed by such Successor Guarantee Trustee and
     delivered to the Guarantor.

          (c) The Guarantee Trustee appointed hereunder shall hold office until
     a Successor Guarantee Trustee shall have been appointed or until its
     removal or resignation. The Guarantee Trustee may resign from office
     (without need for prior or subsequent accounting) by an instrument in
     writing executed by the Guarantee Trustee and delivered to the Guarantor,
     which resignation shall not take effect until a Successor Guarantee Trustee
     has been appointed and has accepted such appointment by instrument in
     writing executed by such Successor Guarantee Trustee and delivered to the
     Guarantor and the resigning Guarantee Trustee.

          (d) If no Successor Guarantee Trustee shall have been appointed and
     accepted appointment as provided in this Section 4.2 within thirty (30)
     days after delivery to the Guarantor of an instrument of resignation, the
     resigning Guarantee Trustee may petition, at the expense of the Guarantor,
     any court of competent jurisdiction for appointment of a Successor
     Guarantee Trustee. Such court may thereupon, after prescribing such notice,
     if any, as it may deem proper, appoint a Successor Guarantee Trustee.

                                    ARTICLE V

                                    GUARANTEE

     SECTION 5.1 Guarantee.

          (a) The Guarantor irrevocably and unconditionally agrees to pay in
     full to the Holders the Guarantee Payments (without duplication of amounts
     theretofore paid by or on behalf of the Issuer), as and when due,
     regardless of any defense (except for the defense of payment by the
     Issuer), right of set-off or counterclaim which the Issuer may have or
     assert. The Guarantor's obligation to make a Guarantee Payment may be
     satisfied by direct payment of the required amounts by the Guarantor to the
     Holders or by causing the Issuer to pay such amounts to the Holders. The
     Guarantor shall give prompt written notice to the Guarantee Trustee in the
     event it makes any direct payment to the Holders hereunder.


                                       12

<PAGE>

          (b) The Guarantor hereby also agrees to assume any and all Obligations
     of the Issuer, and, in the event any such Obligation is not so assumed,
     subject to the terms and conditions hereof, the Guarantor hereby
     irrevocably and unconditionally guarantees to each Beneficiary the full
     payment, when and as due, of any and all Obligations to such Beneficiaries.
     This Guarantee is intended to be for the Beneficiaries who have received
     notice hereof.

     SECTION 5.2 Waiver of Notice and Demand.

     The Guarantor hereby waives notice of acceptance of the Guarantee Agreement
and of any liability to which it applies or may apply, presentment, demand for
payment, any right to require a proceeding first against the Guarantee Trustee,
Issuer or any other Person before proceeding against the Guarantor, protest,
notice of nonpayment, notice of dishonor, notice of redemption and all other
notices and demands.

     SECTION 5.3 Obligations Not Affected.

     The obligations, covenants, agreements and duties of the Guarantor under
this Guarantee Agreement shall in no way be affected or impaired by reason of
the happening from time to time of any of the following:

          (a) the release or waiver, by operation of law or otherwise, of the
     performance or observance by the Issuer of any express or implied
     agreement, covenant, term or condition relating to the Preferred Securities
     to be performed or observed by the Issuer;

          (b) the extension of time for the payment by the Issuer of all or any
     portion of the Distributions (other than an extension of time for payment
     of Distributions that results from the extension of any interest payment
     period on the Notes as provided in the Indenture), Redemption Price,
     Liquidation Distribution or any other sums payable under the terms of the
     Preferred Securities or the extension of time for the performance of any
     other obligation under, arising out of, or in connection with, the
     Preferred Securities;

          (c) any failure, omission, delay or lack of diligence on the part of
     the Holders to enforce, assert or exercise any right, privilege, power or
     remedy conferred on the Holders pursuant to the terms of the Preferred
     Securities, or any action on the part of the Issuer granting indulgence or
     extension of any kind;

          (d) the voluntary or involuntary liquidation, dissolution, sale of any
     collateral, receivership, insolvency, bankruptcy, assignment for the
     benefit of creditors, reorganization, arrangement, composition or
     readjustment of debt of, or other similar proceedings affecting, the Issuer
     or any of the assets of the Issuer;

          (e) any invalidity of, or defect or deficiency in, the Preferred
     Securities;


                                       13

<PAGE>

          (f) the settlement or compromise of any obligation guaranteed hereby
     or hereby incurred; or

          (g) any other circumstance whatsoever that might otherwise constitute
     a legal or equitable discharge or defense of a guarantor, it being the
     intent of this Section 5.3 that the obligations of the Guarantor hereunder
     shall be absolute and unconditional under any and all circumstances.

     There shall be no obligation of the Holders to give notice to, or obtain
the consent of, the Guarantor with respect to the happening of any of the
foregoing.

     SECTION 5.4 Rights of Holders.

     The Guarantor expressly acknowledges that: (a) this Guarantee Agreement
will be deposited with the Guarantee Trustee to be held for the benefit of the
Holders; (b) the Guarantee Trustee has the right to enforce this Guarantee
Agreement on behalf of the Holders; (c) the Holders of a Majority in Liquidation
Amount of the Preferred Securities have the right to direct the time, method and
place of conducting any proceeding for any remedy available to the Guarantee
Trustee in respect of this Guarantee Agreement or exercising any trust or power
conferred upon the Guarantee Trustee under this Guarantee Agreement; and (d) any
Holder may institute a legal proceeding directly against the Guarantor to
enforce its rights under this Guarantee Agreement, without first instituting a
legal proceeding against the Guarantee Trustee, the Issuer or any other Person.

     SECTION 5.5 Guarantee of Payment.

     This Guarantee Agreement creates a guarantee of payment and not of
collection. This Guarantee Agreement will not be discharged except by payment of
the Guarantee Payments in full (without duplication of amounts theretofore paid
by the Issuer) or upon distribution of Notes to Holders as provided in the Trust
Agreement.

     SECTION 5.6 Subrogation.

     The Guarantor shall be subrogated to all (if any) rights of the Holders
against the Issuer in respect of any amounts paid to the Holders by the
Guarantor under this Guarantee Agreement and shall have the right to waive
payment by the Issuer pursuant to Section 5.1; provided, that, the Guarantor
shall not (except to the extent required by mandatory provisions of law) be
entitled to enforce or exercise any rights it may acquire by way of subrogation
or any indemnity, reimbursement or other agreement, in all cases as a result of
payment under this Guarantee Agreement, if, at the time of any such payment, any
amounts are due and unpaid under this Guarantee Agreement. If any amount shall
be paid to the Guarantor in violation of the preceding sentence, the Guarantor
agrees to hold such amount in trust for the Holders and to pay over such amount
to the Holders.

     SECTION 5.7 Independent Obligations.


                                       14

<PAGE>

     The Guarantor acknowledges that its obligations hereunder are independent
of the obligations of the Issuer with respect to the Preferred Securities and
that the Guarantor shall be liable as principal and as debtor hereunder to make
Guarantee Payments pursuant to the terms of this Guarantee Agreement
notwithstanding the occurrence of any event referred to in subsections (a)
through (g), inclusive, of Section 5.3.

     SECTION 5.8 Enforcement.

     A Beneficiary may enforce the Obligations of the Guarantor contained in
Section 5.1(b) directly against the Guarantor, and the Guarantor waives any
right or remedy to require that any action be brought against the Issuer or any
other person or entity before proceeding against the Guarantor.

                                   ARTICLE VI

                           COVENANTS AND SUBORDINATION

     SECTION 6.1 Dividends, Distributions and Payments.

     So long as any Preferred Securities remain outstanding, if there shall have
occurred and be continuing an Event of Default or the Guarantor shall have
entered into an Extension Period as provided for in the Indenture and such
period, or any extension thereof, shall have commenced and be continuing, then
the Guarantor may not (a) declare or pay any dividends or distributions on, or
redeem, purchase, acquire or make liquidation payment with respect to, any of
the Guarantor's capital stock or (b) make any payment of principal of or any
interest or premium on or repay, repurchase or redeem any debt securities of the
Guarantor that rank pari passu in all respects with or junior in interest to the
Preferred Securities (other than (i) repurchases, redemptions or other
acquisitions of shares of capital stock of the Guarantor in connection with any
employment contract, benefit plan or other similar arrangement with or for the
benefit of any one or more employees, officers, directors or consultants, in
connection with a dividend reinvestment or stockholder stock purchase plan or in
connection with the issuance of capital stock of the Guarantor (or securities
convertible into or exercisable for such capital stock) as consideration in an
acquisition transaction entered into prior to the occurrence of such Event of
Default or the applicable Extension Period, (ii) as a result of an exchange or
conversion of any class or series of the Guarantor's capital stock (or any
capital stock of a subsidiary of the Guarantor) for any class or series of the
Guarantor's capital stock or any class of series of the Guarantor's indebtedness
for any class or series of the Guarantor's capital stock, (iii) the purchase of
fractional interests in shares of the Guarantor's capital stock pursuant to the
conversions or exchange provisions of such capital stock or the security being
converted or exchanged, (iv) any declaration of a dividend in connection with
any rights plan, the issuance of rights, stock or other property under any
rights plan or the redemption or repurchase of rights pursuant thereto, or (v)
any dividend in the form of stock, warrants, options or other rights where the
dividend stock or the stock issuable upon exercise of such warrants, options or
other rights is the same stock as that on which the dividend is being paid or
ranks pari passu with or junior to such stock).


                                       15
<PAGE>

     SECTION 6.2 Subordination.

     The obligations of the Guarantor under this Guarantee Agreement will
constitute unsecured obligations of the Guarantor and will rank subordinate and
junior in right of payment to all Senior Debt of the Guarantor.

     SECTION 6.3 Pari Passu Guarantees.

          (a) The obligations of the Guarantor under this Guarantee Agreement
     shall rank pari passu with the obligations of the Guarantor under any
     similar guarantee agreements issued by the Guarantor with respect to
     preferred securities (if any) similar to the Preferred Securities, issued
     by trusts other than the Issuer established or to be established by the
     Guarantor (if any), in each case similar to the Issuer.

          (b) The right of the Guarantor to participate in any distribution of
     assets of any of its subsidiaries upon any such subsidiary's liquidation or
     reorganization or otherwise is subject to the prior claims of creditors of
     that subsidiary, except to the extent the Guarantor may itself be
     recognized as a creditor of that subsidiary. Accordingly, the Guarantor's
     obligations under this Guarantee will be effectively subordinated to all
     existing and future liabilities of the Guarantor's subsidiaries, and
     claimants should look only to the assets of the Guarantor for payments
     thereunder. This Guarantee does not limit the incurrence or issuance of
     other secured or unsecured debt of the Guarantor, including Senior Debt of
     the Guarantor, under any indenture or agreement that the Guarantor may
     enter into in the future or otherwise.

                                   ARTICLE VII

                                   TERMINATION

     SECTION 7.1 Termination.

     This Guarantee Agreement shall terminate and be of no further force and
effect upon (a) full payment of the Redemption Price of all Preferred
Securities, (b) the distribution of Notes to the Holders in exchange for all of
the Preferred Securities or (c) full payment of the amounts payable in
accordance with the Trust Agreement upon liquidation of the Issuer.
Notwithstanding the foregoing, this Guarantee Agreement will continue to be
effective or will be reinstated, as the case may be, if at any time any Holder
must restore payment of any sums paid with respect to Preferred Securities or
this Guarantee Agreement. The obligations of the Guarantor under Sections 3.3
and 3.4 shall survive any such termination or the resignation and removal of the
Guarantee Trustee.

                                  ARTICLE VIII

                                  MISCELLANEOUS

     SECTION 8.1 Successors and Assigns.


                                       16

<PAGE>

     All guarantees and agreements contained in this Guarantee Agreement shall
bind the successors, assigns, receivers, trustees and representatives of the
Guarantor and shall inure to the benefit of the Holders of the Preferred
Securities then outstanding. Except in connection with a consolidation, merger
or sale involving the Guarantor that is permitted under Article VIII of the
Indenture and pursuant to which the successor or assignee agrees in writing to
perform the Guarantor's obligations hereunder, the Guarantor shall not assign
its rights or delegate its obligations hereunder without the prior approval of
the Holders of a Majority in Liquidation Amount of the Preferred Securities.

     SECTION 8.2 Amendments.

     Except with respect to any changes that do not adversely affect the rights
of the Holders in any material respect (in which case no consent of the Holders
will be required), this Guarantee Agreement may only be amended with the prior
approval of the Guarantor, the Guarantee Trustee and the Holders of not less
than a Majority in Liquidation Amount of the Preferred Securities. The
provisions of Article VI of the Trust Agreement concerning meetings or consents
of the Holders shall apply to the giving of such approval.

     SECTION 8.3 Notices.

     Any notice, request or other communication required or permitted to be
given hereunder shall be in writing, duly signed by the party giving such
notice, and delivered, telecopied or mailed by first class mail as follows:

          (a) if given to the Guarantor, to the address or facsimile number set
     forth below or such other address, facsimile number or to the attention of
     such other Person as the Guarantor may give notice to the Guarantee Trustee
     and the Holders:

               United Bancorp, Inc.
               201 South Fourth Street, P.O. Box 10
               Martins Ferry, Ohio 43935-0010
               Facsimile No.: (740) 633- 1448
               Attention: Chairman, President and Chief Executive Officer

          (b) if given to the Issuer, at the Issuer's address or facsimile
     number set forth below or such other address, facsimile number or to the
     attention of such other Person as the Issuer may give notice to the
     Guarantee Trustee and the Holders:

               United Bancorp Statutory Trust I
               c/o United Bancorp, Inc.
               201 South Fourth Street, P.O. Box 10
               Martins Ferry, Ohio 43935-0010
               Facsimile No.: (740) 633- 1448
               Attention: Administrative Trustee


                                       17

<PAGE>

          (c) if given to the Guarantee Trustee, at the address or facsimile
     number set forth below or such other address, facsimile number or to the
     attention of such other Person as the Guarantee Trustee may give notice to
     the Guarantor and the Holders:

               Wilmington Trust Company
               Rodney Square North, 1100 North Market Street
               Wilmington, Delaware 19890-0001
               Facsimile No.: (302) 636-4140
               Attention: Corporate Capital Markets

          (d) if given to any Holder, at the address set forth on the books and
     records of the Issuer.

     All notices hereunder shall be deemed to have been given when received in
person, telecopied with receipt confirmed, or mailed by first class mail,
postage prepaid, except that if a notice or other document is refused delivery
or cannot be delivered because of a changed address of which no notice was
given, such notice or other document shall be deemed to have been delivered on
the date of such refusal or inability to deliver.

     SECTION 8.4 Benefit.

     This Guarantee Agreement is solely for the benefit of the Holders and is
not separately transferable from the Preferred Securities.

     SECTION 8.5 Governing Law.

     THIS GUARANTEE AGREEMENT AND THE RIGHTS AND OBLIGATIONS OF EACH PARTY
HERETO, SHALL BE CONSTRUED AND ENFORCED IN ACCORDANCE WITH AND GOVERNED BY THE
LAWS OF THE STATE OF NEW YORK WITHOUT REFERENCE TO ITS CONFLICT OF LAWS
PROVISIONS (OTHER THAN SECTION 5-1401 OF THE GENERAL OBLIGATIONS LAW).

     SECTION 8.6 Submission to Jurisdiction.

     ANY LEGAL ACTION OR PROCEEDING BY OR AGAINST ANY PARTY HERETO OR WITH
RESPECT TO OR ARISING OUT OF THIS GUARANTEE AGREEMENT MAY BE BROUGHT IN OR
REMOVED TO THE COURTS OF THE STATE OF NEW YORK, IN AND FOR THE COUNTY OF NEW
YORK, OR OF THE UNITED STATES OF AMERICA FOR THE SOUTHERN DISTRICT OF NEW YORK
(IN EACH CASE SITTING IN THE BOROUGH OF MANHATTAN). BY EXECUTION AND DELIVERY OF
THIS GUARANTEE AGREEMENT, EACH PARTY ACCEPTS, FOR ITSELF AND IN RESPECT OF ITS
PROPERTY, GENERALLY AND UNCONDITIONALLY, THE JURISDICTION OF THE AFORESAID
COURTS (AND COURTS OF APPEALS THEREFROM) FOR LEGAL PROCEEDINGS ARISING OUT OF OR
IN CONNECTION WITH THIS GUARANTEE AGREEMENT.


                                       18

<PAGE>

     SECTION 8.7 Counterparts.

     This instrument may be executed in any number of counterparts, each of
which so executed shall be deemed to be an original, but all such counterparts
shall together constitute but one and the same instrument.

                      [THE NEXT PAGE IS THE SIGNATURE PAGE]


                                       19

<PAGE>

     IN WITNESS WHEREOF, the undersigned have executed this Guarantee Agreement
as of the date first above written.

                                        United Bancorp, Inc.


                                        By: /s/ James W. Everson
                                            ------------------------------------
                                            James W. Everson
                                            Chairman, President and
                                            Chief Executive Officer


                                        WILMINGTON TRUST COMPANY,
                                        not in its individual capacity, but
                                        solely as Guarantee Trustee


                                        By: /s/ Christopher J. Slaybaugh
                                            ------------------------------------
                                        Name: Christopher J. Slaybaugh
                                        Title: Senior Financial Services Officer

                                                                       GUARANTEE