<PAGE>
                                                                             174

                                  EXHIBIT 10.5

                               GUARANTEE AGREEMENT

                               EVANS BANCORP, INC.

                           Dated as of October 1, 2004

<PAGE>

                                                                             175

<TABLE>
<S>                                                                                                                  <C>
ARTICLE I     DEFINITIONS AND INTERPRETATION....................................................................      1

         Section 1.1.     Definitions and Interpretation........................................................      1

ARTICLE II     POWERS, DUTIES AND RIGHTS OF THE GUARANTEE TRUSTEE...............................................      4

         Section 2.1.     Powers and Duties of the Guarantee Trustee............................................      4

         Section 2.2.     Certain Rights of the Guarantee Trustee...............................................      5

         Section 2.3.     Not Responsible for Recitals or Issuance of Guarantee.................................      7

         Section 2.4.     Events of Default; Waiver.............................................................      7

         Section 2.5.     Events of Default; Notice.............................................................      8

ARTICLE III     THE GUARANTEE TRUSTEE...........................................................................      8

         Section 3.1.     The Guarantee Trustee; Eligibility....................................................      8

         Section 3.2.     Appointment, Removal and Resignation of the Guarantee Trustee.........................      9

ARTICLE IV     GUARANTEE........................................................................................     10

         Section 4.1.     Guarantee.............................................................................     10

         Section 4.2.     Waiver of Notice and Demand...........................................................     10

         Section 4.3.     Obligations Not Affected..............................................................     10

         Section 4.4.     Rights of Holders.....................................................................     11

         Section 4.5.     Guarantee of Payment..................................................................     11

         Section 4.6.     Subrogation...........................................................................     12

         Section 4.7.     Independent Obligations...............................................................     12

         Section 4.8.     Enforcement...........................................................................     12

ARTICLE V     LIMITATION OF TRANSACTIONS; SUBORDINATION.........................................................     12

         Section 5.1.     Limitation of Transactions............................................................     12

         Section 5.2.     Ranking...............................................................................     13

ARTICLE VI     TERMINATION......................................................................................     14

         Section 6.1.     Termination...........................................................................     14

ARTICLE VII     INDEMNIFICATION.................................................................................     14

         Section 7.1.     Exculpation...........................................................................     14

         Section 7.2.     Indemnification.......................................................................     14

         Section 7.3.     Compensation; Reimbursement of Expenses...............................................     15

ARTICLE VIII     MISCELLANEOUS..................................................................................     16

         Section 8.1.     Successors and Assigns................................................................     16

         Section 8.2.     Amendments............................................................................     16

         Section 8.3.     Notices...............................................................................     16

         Section 8.4.     Benefit...............................................................................     17

         Section 8.5.     Governing Law.........................................................................     17

         Section 8.6.     Counterparts..........................................................................     17
</TABLE>

<PAGE>

                                                                             176

                               GUARANTEE AGREEMENT

                  This GUARANTEE AGREEMENT (the "Guarantee"), dated as of
October 1, 2004, is executed and delivered by Evans Bancorp, Inc., a bank
holding company incorporated in New York (the "Guarantor"), and Wilmington Trust
Company, a Delaware banking corporation, as trustee (the "Guarantee Trustee"),
for the benefit of the Holders (as defined herein) from time to time of the
Capital Securities (as defined herein) of Evans Capital Trust I, a Delaware
statutory trust (the "Issuer").

                  WHEREAS, pursuant to an Amended and Restated Declaration of
Trust (the "Declaration"), dated as of October 1, 2004, among the trustees named
therein of the Issuer, Evans Bancorp, Inc., as sponsor, and the Holders from
time to time of undivided beneficial interests in the assets of the Issuer, the
Issuer is issuing on the date hereof securities, having an aggregate liquidation
amount of up to $11,000,000, designated the Capital Securities (the "Capital
Securities"); and

                  WHEREAS, as incentive for the Holders to purchase the Capital
Securities, the Guarantor desires irrevocably and unconditionally to agree, to
the extent set forth in this Guarantee, to pay to the Holders of Capital
Securities the Guarantee Payments (as defined herein) and to make certain other
payments on the terms and conditions set forth herein.

                  NOW, THEREFORE, in consideration of the purchase by each
Holder of the Capital Securities, which purchase the Guarantor hereby agrees
shall benefit the Guarantor, the Guarantor executes and delivers this Guarantee
for the benefit of the Holders.

                                    ARTICLE I

                         DEFINITIONS AND INTERPRETATION

                  Section 1.01. Definitions and Interpretation.

                  In this Guarantee, unless the context otherwise requires:

                  (a) capitalized terms used in this Guarantee but not defined
         in the preamble above have the respective meanings assigned to them in
         this Section 1.1;

                  (b) a term defined anywhere in this Guarantee has the same
         meaning throughout;

                  (c) all references to "the Guarantee" or "this Guarantee" are
         to this Guarantee as modified, supplemented or amended from time to
         time;

                  (d) all references in this Guarantee to Articles and Sections
         are to Articles and Sections of this Guarantee, unless otherwise
         specified;

                  (e) terms defined in the Declaration as of the date of
         execution of this Guarantee have the same meanings when used in this
         Guarantee, unless otherwise defined in this Guarantee or unless the
         context otherwise requires; and

                  (f) a reference to the singular includes the plural and vice
         versa.

                  "Beneficiaries" means any Person to whom the Issuer is or
hereafter becomes indebted or liable.

                  "Corporate Trust Office" means the office of the Guarantee
Trustee at which the corporate trust business of the Guarantee Trustee shall, at
any particular time, be principally administered, which office at the date of
execution of this Guarantee is located at Rodney Square North, 1100 North Market
Street, Wilmington, Delaware 19890-0001.

<PAGE>

                                                                             177

                  "Covered Person" means any Holder of Capital Securities.

                  "Debentures" means the junior subordinated debentures of Evans
Bancorp, Inc., designated the Floating Rate Junior Subordinated Debt Securities
due 2034, held by the Institutional Trustee (as defined in the Declaration) of
the Issuer.

                  "Event of Default" has the meaning set forth in Section 2.4.

                  "Guarantee Payments" means the following payments or
distributions, without duplication, with respect to the Capital Securities, to
the extent not paid or made by the Issuer: (i) any accrued and unpaid
Distributions (as defined in the Declaration) which are required to be paid on
such Capital Securities to the extent the Issuer has funds available in the
Property Account (as defined in the Declaration) therefor at such time, (ii) the
Redemption Price (as defined in the Indenture) to the extent the Issuer has
funds available in the Property Account therefor at such time, with respect to
the Capital Securities at maturity or any Capital Securities called for
redemption by the Issuer, (iii) the Special Redemption Price (as defined in the
Indenture) to the extent the Issuer has funds available in the Property Account
therefor at such time, with respect to Capital Securities called for redemption
upon the occurrence of a Special Event (as defined in the Indenture), and (iv)
upon a voluntary or involuntary liquidation, dissolution, winding-up or
termination of the Issuer (other than in connection with the distribution of
Debentures to the Holders of the Capital Securities in exchange therefor as
provided in the Declaration), the lesser of (a) the aggregate of the liquidation
amount and all accrued and unpaid Distributions on the Capital Securities to the
date of payment, to the extent the Issuer has funds available in the Property
Account therefor at such time, and (b) the amount of assets of the Issuer
remaining available for distribution to Holders in liquidation of the Issuer
after satisfaction of liabilities to creditors of the Issuer as required by
applicable law (in either case, the "Liquidation Distribution").

                  "Guarantee Trustee" means Wilmington Trust Company, until a
Successor Guarantee Trustee has been appointed and has accepted such appointment
pursuant to the terms of this Guarantee and thereafter means each such Successor
Guarantee Trustee.

                  "Holder" means any holder, as registered on the books and
records of the Issuer, of any Capital Securities; provided, however, that, in
determining whether the holders of the requisite percentage of Capital
Securities have given any request, notice, consent or waiver hereunder, "Holder"
shall not include the Guarantor or any Affiliate of the Guarantor.

                  "Indemnified Person" means the Guarantee Trustee (including in
its individual capacity), any Affiliate of the Guarantee Trustee, or any
officers, directors, shareholders, members, partners, employees,
representatives, nominees, custodians or agents of the Guarantee Trustee.

                  "Indenture" means the Indenture, dated as of October 1, 2004,
between the Guarantor and Wilmington Trust Company, not in its individual
capacity but solely as trustee, and any indenture supplemental thereto pursuant
to which the Debentures are to be issued to the Institutional Trustee of the
Issuer.

                  "Liquidation Distribution" has the meaning set forth in the
definition of "Guarantee Payments" herein.

                  "Majority in liquidation amount of the Capital Securities"
means Holder(s) of outstanding Capital Securities, voting together as a class,
but separately from the holders of Common Securities, of more than 50% of the
aggregate liquidation amount (including the stated amount that would be paid on
redemption, liquidation or otherwise, plus accrued and unpaid Distributions to,
but excluding, the date upon which the voting percentages are determined) of all
Capital Securities then outstanding.

                  "Obligations" means any costs, expenses or liabilities (but
not including liabilities related to taxes) of the Issuer, other than
obligations of the Issuer to pay to holders of any Trust Securities the amounts
due such holders pursuant to the terms of the Trust Securities.

<PAGE>

                                                                             178

                  "Officer's Certificate" means, with respect to any Person, a
certificate signed by one Authorized Officer of such Person. Any Officer's
Certificate delivered with respect to compliance with a condition or covenant
provided for in this Guarantee shall include:

                  (a) a statement that each officer signing the Officer's
         Certificate has read the covenant or condition and the definitions
         relating thereto;

                  (b) a brief statement of the nature and scope of the
         examination or investigation undertaken by each officer in rendering
         the Officer's Certificate;

                  (c) a statement that each such officer has made such
         examination or investigation as, in such officer's opinion, is
         necessary to enable such officer to express an informed opinion as to
         whether or not such covenant or condition has been complied with; and

                  (d) a statement as to whether, in the opinion of each such
         officer, such condition or covenant has been complied with.

                  "Person" means a legal person, including any individual,
corporation, estate, partnership, joint venture, association, joint stock
company, limited liability company, trust, unincorporated association, or
government or any agency or political subdivision thereof, or any other entity
of whatever nature.

                  "Responsible Officer" means, with respect to the Guarantee
Trustee, any officer within the Corporate Trust Office of the Guarantee Trustee
with direct responsibility for the administration of any matters relating to
this Guarantee, including any vice president, any assistant vice president, any
secretary, any assistant secretary, the treasurer, any assistant treasurer, any
trust officer or other officer of the Corporate Trust Office of the Guarantee
Trustee customarily performing functions similar to those performed by any of
the above designated officers and also means, with respect to a particular
corporate trust matter, any other officer to whom such matter is referred
because of that officer's knowledge of and familiarity with the particular
subject.

                  "Successor Guarantee Trustee" means a successor Guarantee
Trustee possessing the qualifications to act as Guarantee Trustee under Section
3.1.

                  "Trust Securities" means the Common Securities and the Capital
Securities.

                                   ARTICLE II

               POWERS, DUTIES AND RIGHTS OF THE GUARANTEE TRUSTEE

                  Section 2.01. Powers and Duties of the Guarantee Trustee.

                  (a) This Guarantee shall be held by the Guarantee Trustee for
         the benefit of the Holders of the Capital Securities, and the Guarantee
         Trustee shall not transfer this Guarantee to any Person except a Holder
         of Capital Securities exercising his or her rights pursuant to Section
         4.4(b) or to a Successor Guarantee Trustee on acceptance by such
         Successor Guarantee Trustee of its appointment to act as Successor
         Guarantee Trustee. The right, title and interest of the Guarantee
         Trustee shall automatically vest in any Successor Guarantee Trustee,
         and such vesting and cessation of title shall be effective whether or
         not conveyancing documents have been executed and delivered pursuant to
         the appointment of such Successor Guarantee Trustee.

                  (b) If an Event of Default actually known to a Responsible
         Officer of the Guarantee Trustee has occurred and is continuing, the
         Guarantee Trustee shall enforce this Guarantee for the benefit of the
         Holders of the Capital Securities.

<PAGE>

                                                                             179

                  (c) The Guarantee Trustee, before the occurrence of any Event
         of Default and after the curing or waiving of all Events of Default
         that may have occurred, shall undertake to perform only such duties as
         are specifically set forth in this Guarantee, and no implied covenants
         shall be read into this Guarantee against the Guarantee Trustee. In
         case an Event of Default has occurred (that has not been cured or
         waived pursuant to Section 2.4(b)) and is actually known to a
         Responsible Officer of the Guarantee Trustee, the Guarantee Trustee
         shall exercise such of the rights and powers vested in it by this
         Guarantee, and use the same degree of care and skill in its exercise
         thereof, as a prudent person would exercise or use under the
         circumstances in the conduct of his or her own affairs.

                  (d) No provision of this Guarantee shall be construed to
         relieve the Guarantee Trustee from liability for its own negligent
         action, its own negligent failure to act, or its own willful
         misconduct, except that:

                           (i) prior to the occurrence of any Event of Default
                  and after the curing or waiving of all Events of Default that
                  may have occurred:

                                    (A) the duties and obligations of the
                           Guarantee Trustee shall be determined solely by the
                           express provisions of this Guarantee, and the
                           Guarantee Trustee shall not be liable except for the
                           performance of such duties and obligations as are
                           specifically set forth in this Guarantee, and no
                           implied covenants or obligations shall be read into
                           this Guarantee against the Guarantee Trustee; and

                                    (B) in the absence of bad faith on the part
                           of the Guarantee Trustee, the Guarantee Trustee may
                           conclusively rely, as to the truth of the statements
                           and the correctness of the opinions expressed
                           therein, upon any certificates or opinions furnished
                           to the Guarantee Trustee and conforming to the
                           requirements of this Guarantee; but in the case of
                           any such certificates or opinions furnished to the
                           Guarantee Trustee, the Guarantee Trustee shall be
                           under a duty to examine the same to determine whether
                           or not on their face they conform to the requirements
                           of this Guarantee;

                           (ii) the Guarantee Trustee shall not be liable for
                  any error of judgment made in good faith by a Responsible
                  Officer of the Guarantee Trustee, unless it shall be proved
                  that such Responsible Officer of the Guarantee Trustee or the
                  Guarantee Trustee was negligent in ascertaining the pertinent
                  facts upon which such judgment was made;

                           (iii) the Guarantee Trustee shall not be liable with
                  respect to any action taken or omitted to be taken by it in
                  good faith in accordance with the written direction of the
                  Holders of not less than a Majority in liquidation amount of
                  the Capital Securities relating to the time, method and place
                  of conducting any proceeding for any remedy available to the
                  Guarantee Trustee, or exercising any trust or power conferred
                  upon the Guarantee Trustee under this Guarantee; and

                           (iv) no provision of this Guarantee shall require the
                  Guarantee Trustee to expend or risk its own funds or otherwise
                  incur personal financial liability in the performance of any
                  of its duties or in the exercise of any of its rights or
                  powers, if the Guarantee Trustee shall have reasonable grounds
                  for believing that the repayment of such funds is not
                  reasonably assured to it under the terms of this Guarantee, or
                  security and indemnity, reasonably satisfactory to the
                  Guarantee Trustee, against such risk or liability is not
                  reasonably assured to it.

                  Section 2.02. Certain Rights of the Guarantee Trustee.

                  (a) Subject to the provisions of Section 2.1:

                           (i) The Guarantee Trustee may conclusively rely, and
                  shall be fully protected in acting or refraining from acting
                  upon, any resolution, certificate, statement, instrument,
                  opinion, report, notice, request, direction, consent, order,
                  bond, debenture, note, other evidence of indebtedness or

<PAGE>

                                                                             180

                  other paper or document believed by it to be genuine and to
                  have been signed, sent or presented by the proper party or
                  parties.

                           (ii) Any direction or act of the Guarantor
                  contemplated by this Guarantee shall be sufficiently evidenced
                  by an Officer's Certificate.

                           (iii) Whenever, in the administration of this
                  Guarantee, the Guarantee Trustee shall deem it desirable that
                  a matter be proved or established before taking, suffering or
                  omitting any action hereunder, the Guarantee Trustee (unless
                  other evidence is herein specifically prescribed) may, in the
                  absence of bad faith on its part, request and conclusively
                  rely upon an Officer's Certificate of the Guarantor which,
                  upon receipt of such request, shall be promptly delivered by
                  the Guarantor.

                           (iv) The Guarantee Trustee shall have no duty to see
                  to any recording, filing or registration of any instrument or
                  other writing (or any rerecording, refiling or reregistration
                  thereof).

                           (v) The Guarantee Trustee may consult with counsel of
                  its selection, and the advice or opinion of such counsel with
                  respect to legal matters shall be full and complete
                  authorization and protection in respect of any action taken,
                  suffered or omitted by it hereunder in good faith and in
                  accordance with such advice or opinion. Such counsel may be
                  counsel to the Guarantor or any of its Affiliates and may
                  include any of its employees. The Guarantee Trustee shall have
                  the right at any time to seek instructions concerning the
                  administration of this Guarantee from any court of competent
                  jurisdiction.

                           (vi) The Guarantee Trustee shall be under no
                  obligation to exercise any of the rights or powers vested in
                  it by this Guarantee at the request or direction of any
                  Holder, unless such Holder shall have provided to the
                  Guarantee Trustee such security and indemnity, reasonably
                  satisfactory to the Guarantee Trustee, against the costs,
                  expenses (including attorneys' fees and expenses and the
                  expenses of the Guarantee Trustee's agents, nominees or
                  custodians) and liabilities that might be incurred by it in
                  complying with such request or direction, including such
                  reasonable advances as may be requested by the Guarantee
                  Trustee; provided, however, that nothing contained in this
                  Section 2.2(a)(vi) shall be taken to relieve the Guarantee
                  Trustee, upon the occurrence of an Event of Default, of its
                  obligation to exercise the rights and powers vested in it by
                  this Guarantee.

                           (vii) The Guarantee Trustee shall not be bound to
                  make any investigation into the facts or matters stated in any
                  resolution, certificate, statement, instrument, opinion,
                  report, notice, request, direction, consent, order, bond,
                  debenture, note, other evidence of indebtedness or other paper
                  or document, but the Guarantee Trustee, in its discretion, may
                  make such further inquiry or investigation into such facts or
                  matters as it may see fit.

                           (viii) The Guarantee Trustee may execute any of the
                  trusts or powers hereunder or perform any duties hereunder
                  either directly or by or through agents, nominees, custodians
                  or attorneys, and the Guarantee Trustee shall not be
                  responsible for any misconduct or negligence on the part of
                  any agent or attorney appointed with due care by it hereunder.

                           (ix) Any action taken by the Guarantee Trustee or its
                  agents hereunder shall bind the Holders of the Capital
                  Securities, and the signature of the Guarantee Trustee or its
                  agents alone shall be sufficient and effective to perform any
                  such action. No third party shall be required to inquire as to
                  the authority of the Guarantee Trustee to so act or as to its
                  compliance with any of the terms and provisions of this
                  Guarantee, both of which shall be conclusively evidenced by
                  the Guarantee Trustee's or its agent's taking such action.

                           (x) Whenever in the administration of this Guarantee
                  the Guarantee Trustee shall deem it desirable to receive
                  instructions with respect to enforcing any remedy or right or
                  taking any other action hereunder, the Guarantee Trustee (A)
                  may request instructions from the Holders of a

<PAGE>

                                                                             181

                  Majority in liquidation amount of the Capital Securities, (B)
                  may refrain from enforcing such remedy or right or taking such
                  other action until such instructions are received and (C)
                  shall be protected in conclusively relying on or acting in
                  accordance with such instructions.

                           (xi) The Guarantee Trustee shall not be liable for
                  any action taken, suffered, or omitted to be taken by it in
                  good faith and reasonably believed by it to be authorized or
                  within the discretion or rights or powers conferred upon it by
                  this Guarantee.

                  (b) No provision of this Guarantee shall be deemed to impose
         any duty or obligation on the Guarantee Trustee to perform any act or
         acts or exercise any right, power, duty or obligation conferred or
         imposed on it, in any jurisdiction in which it shall be illegal or in
         which the Guarantee Trustee shall be unqualified or incompetent in
         accordance with applicable law to perform any such act or acts or to
         exercise any such right, power, duty or obligation. No permissive power
         or authority available to the Guarantee Trustee shall be construed to
         be a duty.

                  Section 2.03. Not Responsible for Recitals or Issuance of
Guarantee.

                  The recitals contained in this Guarantee shall be taken as the
statements of the Guarantor, and the Guarantee Trustee does not assume any
responsibility for their correctness. The Guarantee Trustee makes no
representation as to the validity or sufficiency of this Guarantee.

                  Section 2.04. Events of Default; Waiver.

                  (a) An Event of Default under this Guarantee will occur upon
         the failure of the Guarantor to perform any of its payment or other
         obligations hereunder.

                  (b) The Holders of a Majority in liquidation amount of the
         Capital Securities may, voting or consenting as a class, on behalf of
         the Holders of all of the Capital Securities, waive any past Event of
         Default and its consequences. Upon such waiver, any such Event of
         Default shall cease to exist, and shall be deemed to have been cured,
         for every purpose of this Guarantee, but no such waiver shall extend to
         any subsequent or other default or Event of Default or impair any right
         consequent thereon.

                  Section 2.05. Events of Default; Notice.

                  (a) The Guarantee Trustee shall, within 90 days after the
         occurrence of an Event of Default, transmit by mail, first class
         postage prepaid, to the Holders of the Capital Securities, notices of
         all Events of Default actually known to a Responsible Officer of the
         Guarantee Trustee, unless such defaults have been cured before the
         giving of such notice, provided, however, that the Guarantee Trustee
         shall be protected in withholding such notice if and so long as a
         Responsible Officer of the Guarantee Trustee in good faith determines
         that the withholding of such notice is in the interests of the Holders
         of the Capital Securities.

                  (b) The Guarantee Trustee shall not be charged with knowledge
         of any Event of Default unless the Guarantee Trustee shall have
         received written notice thereof from the Guarantor or a Holder of the
         Capital Securities, or a Responsible Officer of the Guarantee Trustee
         charged with the administration of this Guarantee shall have actual
         knowledge thereof.

                                   ARTICLE III

                              THE GUARANTEE TRUSTEE

                  Section 3.01. The Guarantee Trustee; Eligibility.

                  (a) There shall at all times be a Guarantee Trustee which
shall:

<PAGE>

                                                                             182

                           (i) not be an Affiliate of the Guarantor; and

                           (ii) be a corporation or national association
                  organized and doing business under the laws of the United
                  States of America or any State or Territory thereof or of the
                  District of Columbia, or Person authorized under such laws to
                  exercise corporate trust powers, having a combined capital and
                  surplus of at least 50 million U.S. dollars ($50,000,000), and
                  subject to supervision or examination by Federal, State,
                  Territorial or District of Columbia authority. If such
                  corporation or national association publishes reports of
                  condition at least annually, pursuant to law or to the
                  requirements of the supervising or examining authority
                  referred to above, then, for the purposes of this Section
                  3.1(a)(ii), the combined capital and surplus of such
                  corporation or national association shall be deemed to be its
                  combined capital and surplus as set forth in its most recent
                  report of condition so published.

                  (b) If at any time the Guarantee Trustee shall cease to be
         eligible to so act under Section 3.1(a), the Guarantee Trustee shall
         immediately resign in the manner and with the effect set forth in
         Section 3.2(c).

                  (c) If the Guarantee Trustee has or shall acquire any
         "conflicting interest" within the meaning of Section 310(b) of the
         Trust Indenture Act, the Guarantee Trustee shall either eliminate such
         interest or resign to the extent and in the manner provided by, and
         subject to, this Guarantee.

                  Section 3.02. Appointment, Removal and Resignation of the
Guarantee Trustee.

                  (a) Subject to Section 3.2(b), the Guarantee Trustee may be
         appointed or removed without cause at any time by the Guarantor except
         during an Event of Default.

                  (b) The Guarantee Trustee shall not be removed in accordance
         with Section 3.2(a) until a Successor Guarantee Trustee has been
         appointed and has accepted such appointment by written instrument
         executed by such Successor Guarantee Trustee and delivered to the
         Guarantor.

                  (c) The Guarantee Trustee appointed to office shall hold
         office until a Successor Guarantee Trustee shall have been appointed or
         until its removal or resignation. The Guarantee Trustee may resign from
         office (without need for prior or subsequent accounting) by an
         instrument in writing executed by the Guarantee Trustee and delivered
         to the Guarantor, which resignation shall not take effect until a
         Successor Guarantee Trustee has been appointed and has accepted such
         appointment by an instrument in writing executed by such Successor
         Guarantee Trustee and delivered to the Guarantor and the resigning
         Guarantee Trustee.

                  (d) If no Successor Guarantee Trustee shall have been
         appointed and accepted appointment as provided in this Section 3.2
         within 60 days after delivery of an instrument of removal or
         resignation, the Guarantee Trustee resigning or being removed may
         petition any court of competent jurisdiction for appointment of a
         Successor Guarantee Trustee. Such court may thereupon, after
         prescribing such notice, if any, as it may deem proper, appoint a
         Successor Guarantee Trustee.

                  (e) No Guarantee Trustee shall be liable for the acts or
         omissions to act of any Successor Guarantee Trustee.

                  (f) Upon termination of this Guarantee or removal or
         resignation of the Guarantee Trustee pursuant to this Section 3.2, the
         Guarantor shall pay to the Guarantee Trustee all amounts owing to the
         Guarantee Trustee under Sections 7.2 and 7.3 accrued to the date of
         such termination, removal or resignation.

<PAGE>

                                                                             183

                                   ARTICLE IV

                                    GUARANTEE

                  Section 4.01. Guarantee.

                  (a) The Guarantor irrevocably and unconditionally agrees to
         pay in full to the Holders the Guarantee Payments (without duplication
         of amounts theretofore paid by the Issuer), as and when due, regardless
         of any defense (except as defense of payment by the Issuer), right of
         set-off or counterclaim that the Issuer may have or assert. The
         Guarantor's obligation to make a Guarantee Payment may be satisfied by
         direct payment of the required amounts by the Guarantor to the Holders
         or by causing the Issuer to pay such amounts to the Holders.

                  (b) The Guarantor hereby also agrees to assume any and all
         Obligations of the Issuer and in the event any such Obligation is not
         so assumed, subject to the terms and conditions hereof, the Guarantor
         hereby irrevocably and unconditionally guarantees to each Beneficiary
         the full payment, when and as due, of any and all Obligations to such
         Beneficiaries. This Guarantee is intended to be for the Beneficiaries
         who have received notice hereof.

                  Section 4.02. Waiver of Notice and Demand.

                  The Guarantor hereby waives notice of acceptance of this
Guarantee and of any liability to which it applies or may apply, presentment,
demand for payment, any right to require a proceeding first against the Issuer
or any other Person before proceeding against the Guarantor, protest, notice of
nonpayment, notice of dishonor, notice of redemption and all other notices and
demands.

                  Section 4.03. Obligations Not Affected.

                  The obligations, covenants, agreements and duties of the
Guarantor under this Guarantee shall in no way be affected or impaired by reason
of the happening from time to time of any of the following:

                  (a) the release or waiver, by operation of law or otherwise,
         of the performance or observance by the Issuer of any express or
         implied agreement, covenant, term or condition relating to the Capital
         Securities to be performed or observed by the Issuer;

                  (b) the extension of time for the payment by the Issuer of all
         or any portion of the Distributions, Redemption Price, Special
         Redemption Price, Liquidation Distribution or any other sums payable
         under the terms of the Capital Securities or the extension of time for
         the performance of any other obligation under, arising out of, or in
         connection with, the Capital Securities (other than an extension of
         time for the payment of the Distributions, Redemption Price, Special
         Redemption Price, Liquidation Distribution or other sums payable that
         results from the extension of any interest payment period on the
         Debentures or any extension of the maturity date of the Debentures
         permitted by the Indenture);

                  (c) any failure, omission, delay or lack of diligence on the
         part of the Holders to enforce, assert or exercise any right,
         privilege, power or remedy conferred on the Holders pursuant to the
         terms of the Capital Securities, or any action on the part of the
         Issuer granting indulgence or extension of any kind;

                  (d) the voluntary or involuntary liquidation, dissolution,
         sale of any collateral, receivership, insolvency, bankruptcy,
         assignment for the benefit of creditors, reorganization, arrangement,
         composition or readjustment of debt of, or other similar proceedings
         affecting, the Issuer or any of the assets of the Issuer;

                  (e) any invalidity of, or defect or deficiency in, the Capital
         Securities;

                  (f) the settlement or compromise of any obligation guaranteed
         hereby or hereby incurred; or

<PAGE>

                                                                             184

                  (g) any other circumstance whatsoever that might otherwise
         constitute a legal or equitable discharge or defense of a guarantor, it
         being the intent of this Section 4.3 that the obligations of the
         Guarantor hereunder shall be absolute and unconditional under any and
         all circumstances.

                  There shall be no obligation of the Holders to give notice to,
or obtain consent of, the Guarantor with respect to the happening of any of the
foregoing.

                  Section 4.04. Rights of Holders.

                  (a) The Holders of a Majority in liquidation amount of the
         Capital Securities have the right to direct the time, method and place
         of conducting any proceeding for any remedy available to the Guarantee
         Trustee in respect of this Guarantee or to direct the exercise of any
         trust or power conferred upon the Guarantee Trustee under this
         Guarantee; provided, however, that (subject to Sections 2.1 and 2.2)
         the Guarantee Trustee shall have the right to decline to follow any
         such direction if the Guarantee Trustee shall determine that the
         actions so directed would be unjustly prejudicial to the Holders not
         taking part in such direction or if the Guarantee Trustee being advised
         by counsel determines that the action or proceeding so directed may not
         lawfully be taken or if the Guarantee Trustee in good faith by its
         board of directors or trustees, executive committee or a trust
         committee of directors or trustees and/or Responsible Officers shall
         determine that the action or proceeding so directed would involve the
         Guarantee Trustee in personal liability.

                  (b) Any Holder of Capital Securities may institute a legal
         proceeding directly against the Guarantor to enforce the Guarantee
         Trustee's rights under this Guarantee, without first instituting a
         legal proceeding against the Issuer, the Guarantee Trustee or any other
         Person. The Guarantor waives any right or remedy to require that any
         such action be brought first against the Issuer, the Guarantee Trustee
         or any other Person before so proceeding directly against the
         Guarantor.

                  Section 4.05. Guarantee of Payment.

                  This Guarantee creates a guarantee of payment and not of
collection.

                  Section 4.06. Subrogation.

                  The Guarantor shall be subrogated to all (if any) rights of
the Holders of Capital Securities against the Issuer in respect of any amounts
paid to such Holders by the Guarantor under this Guarantee; provided, however,
that the Guarantor shall not (except to the extent required by applicable
provisions of law) be entitled to enforce or exercise any right that it may
acquire by way of subrogation or any indemnity, reimbursement or other
agreement, in all cases as a result of payment under this Guarantee, if, after
giving effect to any such payment, any amounts are due and unpaid under this
Guarantee. If any amount shall be paid to the Guarantor in violation of the
preceding sentence, the Guarantor agrees to hold such amount in trust for the
Holders and to pay over such amount to the Holders.

                  Section 4.07. Independent Obligations.

                  The Guarantor acknowledges that its obligations hereunder are
independent of the obligations of the Issuer with respect to the Capital
Securities and that the Guarantor shall be liable as principal and as debtor
hereunder to make Guarantee Payments pursuant to the terms of this Guarantee
notwithstanding the occurrence of any event referred to in subsections (a)
through (g), inclusive, of Section 4.3 hereof.

                  Section 4.08. Enforcement.

                  A Beneficiary may enforce the Obligations of the Guarantor
contained in Section 4.1(b) directly against the Guarantor, and the Guarantor
waives any right or remedy to require that any action be brought against the
Issuer or any other person or entity before proceeding against the Guarantor.

<PAGE>

                                                                             185

                  The Guarantor shall be subrogated to all rights (if any) of
any Beneficiary against the Issuer in respect of any amounts paid to the
Beneficiaries by the Guarantor under this Guarantee; provided, however, that the
Guarantor shall not (except to the extent required by applicable provisions of
law) be entitled to enforce or exercise any rights that it may acquire by way of
subrogation or any indemnity, reimbursement or other agreement, in all cases as
a result of payment under this Guarantee, if, after giving effect to such
payment, any amounts are due and unpaid under this Guarantee.

                                    ARTICLE V

                    LIMITATION OF TRANSACTIONS; SUBORDINATION

                  Section 5.01. Limitation of Transactions.

                  So long as any Capital Securities remain outstanding, if (a)
there shall have occurred and be continuing an Event of Default or (b) the
Guarantor shall have selected an Extension Period as provided in the Declaration
and such period, or any extension thereof, shall have commenced and be
continuing, then the Guarantor may not (x) declare or pay any dividends or
distributions on, or redeem, purchase, acquire, or make a liquidation payment
with respect to, any of the Guarantor's capital stock or (y) make any payment of
principal of or interest or premium, if any, on or repay, repurchase or redeem
any debt securities of the Guarantor that rank pari passu in all respects with
or junior in interest to the Debentures (other than (i) payments under this
Guarantee, (ii) repurchases, redemptions or other acquisitions of shares of
capital stock of the Guarantor (A) in connection with any employment contract,
benefit plan or other similar arrangement with or for the benefit of one or more
employees, officers, directors, or consultants, (B) in connection with a
dividend reinvestment or stockholder stock purchase plan or (C) in connection
with the issuance of capital stock of the Guarantor (or securities convertible
into or exercisable for such capital stock), as consideration in an acquisition
transaction entered into prior to the occurrence of the Event of Default or the
applicable Extension Period, (iii) as a result of any exchange or conversion of
any class or series of the Guarantor's capital stock (or any capital stock of a
subsidiary of the Guarantor) for any class or series of the Guarantor's capital
stock or of any class or series of the Guarantor's indebtedness for any class or
series of the Guarantor's capital stock, (iv) the purchase of fractional
interests in shares of the Guarantor's capital stock pursuant to the conversion
or exchange provisions of such capital stock or the security being converted or
exchanged, (v) any declaration of a dividend in connection with any
stockholder's rights plan, or the issuance of rights, stock or other property
under any stockholder's rights plan, or the redemption or repurchase of rights
pursuant thereto, or (vi) any dividend in the form of stock, warrants, options
or other rights where the dividend stock or the stock issuable upon exercise of
such warrants, options or other rights is the same stock as that on which the
dividend is being paid or ranks pari passu with or junior to such stock).

                  Section 5.02. Ranking.

                  This Guarantee will constitute an unsecured obligation of the
Guarantor and will rank subordinate and junior in right of payment to all
present and future Senior Indebtedness (as defined in the Indenture) of the
Guarantor. By their acceptance thereof, each Holder of Capital Securities agrees
to the foregoing provisions of this Guarantee and the other terms set forth
herein.

                  The right of the Guarantor to participate in any distribution
of assets of any of its subsidiaries upon any such subsidiary's liquidation or
reorganization or otherwise is subject to the prior claims of creditors of that
subsidiary, except to the extent the Guarantor may itself be recognized as a
creditor of that subsidiary. Accordingly, the Guarantor's obligations under this
Guarantee will be effectively subordinated to all existing and future
liabilities of the Guarantor's subsidiaries, and claimants should look only to
the assets of the Guarantor for payments thereunder. This Guarantee does not
limit the incurrence or issuance of other secured or unsecured debt of the
Guarantor, including Senior Indebtedness of the Guarantor, under any indenture
or agreement that the Guarantor may enter into in the future or otherwise.

<PAGE>

                                                                             186

                                   ARTICLE VI

                                   TERMINATION

                  Section 6.01. Termination.

                  This Guarantee shall terminate as to the Capital Securities
(i) upon full payment of the Redemption Price or the Special Redemption Price,
as the case may be, of all Capital Securities then outstanding, (ii) upon the
distribution of all of the Debentures to the Holders of all of the Capital
Securities or (iii) upon full payment of the amounts payable in accordance with
the Declaration upon dissolution of the Issuer. This Guarantee will continue to
be effective or will be reinstated, as the case may be, if at any time any
Holder of Capital Securities must restore payment of any sums paid under the
Capital Securities or under this Guarantee.

                                   ARTICLE VII

                                 INDEMNIFICATION

                  Section 7.01. Exculpation.

                  (a) No Indemnified Person shall be liable, responsible or
         accountable in damages or otherwise to the Guarantor or any Covered
         Person for any loss, damage or claim incurred by reason of any act or
         omission of such Indemnified Person in good faith in accordance with
         this Guarantee and in a manner that such Indemnified Person reasonably
         believed to be within the scope of the authority conferred on such
         Indemnified Person by this Guarantee or by law, except that an
         Indemnified Person shall be liable for any such loss, damage or claim
         incurred by reason of such Indemnified Person's negligence or willful
         misconduct with respect to such acts or omissions.

                  (b) An Indemnified Person shall be fully protected in relying
         in good faith upon the records of the Issuer or the Guarantor and upon
         such information, opinions, reports or statements presented to the
         Issuer or the Guarantor by any Person as to matters the Indemnified
         Person reasonably believes are within such other Person's professional
         or expert competence and who, if selected by such Indemnified Person,
         has been selected with reasonable care by such Indemnified Person,
         including information, opinions, reports or statements as to the value
         and amount of the assets, liabilities, profits, losses, or any other
         facts pertinent to the existence and amount of assets from which
         Distributions to Holders of Capital Securities might properly be paid.

                  Section 7.02. Indemnification.

                  (a) The Guarantor agrees to indemnify each Indemnified Person
         for, and to hold each Indemnified Person harmless against, any and all
         loss, liability, damage, claim or expense incurred without negligence
         or willful misconduct on the part of the Indemnified Person, arising
         out of or in connection with the acceptance or administration of the
         trust or trusts hereunder, including but not limited to the costs and
         expenses (including reasonable legal fees and expenses) of the
         Indemnified Person defending itself against, or investigating, any
         claim or liability in connection with the exercise or performance of
         any of the Indemnified Person's powers or duties hereunder. The
         obligation to indemnify as set forth in this Section 7.2 shall survive
         the resignation or removal of the Guarantee Trustee and the termination
         of this Guarantee.

                  (b) Promptly after receipt by an Indemnified Person under this
         Section 7.2 of notice of the commencement of any action, such
         Indemnified Person will, if a claim in respect thereof is to be made
         against the Guarantor under this Section 7.2, notify the Guarantor in
         writing of the commencement thereof; but the failure so to notify the
         Guarantor (i) will not relieve the Guarantor from liability under
         paragraph (a) above unless and to the extent that the Guarantor did not
         otherwise learn of such action and such failure results in the
         forfeiture by the Guarantor of substantial rights and defenses and (ii)
         will not, in any event, relieve the Guarantor from any obligations to
         any Indemnified Person other than the indemnification obligation
         provided in paragraph (a) above. The Guarantor shall be entitled to
         appoint counsel of the

<PAGE>

                                                                             187

         Guarantor's choice at the Guarantor's expense to represent the
         Indemnified Person in any action for which indemnification is sought
         (in which case the Guarantor shall not thereafter be responsible for
         the fees and expenses of any separate counsel retained by the
         Indemnified Person or Persons except as set forth below); provided,
         however, that such counsel shall be satisfactory to the Indemnified
         Person. Notwithstanding the Guarantor's election to appoint counsel to
         represent the Indemnified Person in any action, the Indemnified Person
         shall have the right to employ separate counsel (including local
         counsel), and the Guarantor shall bear the reasonable fees, costs and
         expenses of such separate counsel, if (i) the use of counsel chosen by
         the Guarantor to represent the Indemnified Person would present such
         counsel with a conflict of interest, (ii) the actual or potential
         defendants in, or targets of, any such action include both the
         Indemnified Person and the Guarantor and the Indemnified Person shall
         have reasonably concluded that there may be legal defenses available to
         it and/or other Indemnified Persons which are different from or
         additional to those available to the Guarantor, (iii) the Guarantor
         shall not have employed counsel satisfactory to the Indemnified Person
         to represent the Indemnified Person within a reasonable time after
         notice of the institution of such action or (iv) the Guarantor shall
         authorize the Indemnified Person to employ separate counsel at the
         expense of the Guarantor. The Guarantor will not, without the prior
         written consent of the Indemnified Persons, settle or compromise or
         consent to the entry of any judgment with respect to any pending or
         threatened claim, action, suit or proceeding in respect of which
         indemnification or contribution may be sought hereunder (whether or not
         the Indemnified Persons are actual or potential parties to such claim
         or action) unless such settlement, compromise or consent includes an
         unconditional release of each Indemnified Person from all liability
         arising out of such claim, action, suit or proceeding.

                  Section 7.03. Compensation; Reimbursement of Expenses.

                  The Guarantor agrees:

                  (a) to pay to the Guarantee Trustee from time to time such
compensation for all services rendered by it hereunder as the parties shall
agree to from time to time (which compensation shall not be limited by any
provision of law in regard to the compensation of a trustee of an express
trust); and

                  (b) except as otherwise expressly provided herein, to
reimburse the Guarantee Trustee upon request for all reasonable expenses,
disbursements and advances incurred or made by it in accordance with any
provision of this Guarantee (including the reasonable compensation and the
expenses and disbursements of its agents and counsel), except any such expense,
disbursement or advance as may be attributable to its negligence or willful
misconduct.

                  The provisions of this Section 7.3 shall survive the
resignation or removal of the Guarantee Trustee and the termination of this
Guarantee.

                                  ARTICLE VIII

                                  MISCELLANEOUS

                  Section 8.01. Successors and Assigns.

                  All guarantees and agreements contained in this Guarantee
shall bind the successors, assigns, receivers, trustees and representatives of
the Guarantor and shall inure to the benefit of the Holders of the Capital
Securities then outstanding. Except in connection with any merger or
consolidation of the Guarantor with or into another entity or any sale, transfer
or lease of the Guarantor's assets to another entity, in each case to the extent
permitted under the Indenture, the Guarantor may not assign its rights or
delegate its obligations under this Guarantee without the prior approval of the
Holders of not less than a Majority in liquidation amount of the Capital
Securities.

                  Section 8.02. Amendments.

<PAGE>

                                                                             188

                  Except with respect to any changes that do not adversely
affect the rights of Holders of the Capital Securities in any material respect
(in which case no consent of Holders will be required), this Guarantee may be
amended only with the prior approval of the Holders of not less than a Majority
in liquidation amount of the Capital Securities. The provisions of the
Declaration with respect to amendments thereof shall apply equally with respect
to amendments of the Guarantee.

                  Section 8.03. Notices.

                  All notices provided for in this Guarantee shall be in
writing, duly signed by the party giving such notice, and shall be delivered,
telecopied or mailed by first class mail, as follows:

                  (a) If given to the Guarantee Trustee, at the Guarantee
         Trustee's mailing address set forth below (or such other address as the
         Guarantee Trustee may give notice of to the Holders of the Capital
         Securities):

                           Wilmington Trust Company
                           Rodney Square North
                           1100 North Market Street
                           Wilmington, Delaware 19890-0001
                           Attention: Corporate Capital Markets
                           Telecopy:  302-636-4140
                           Telephone: 302-651-1000

                  (b) If given to the Guarantor, at the Guarantor's mailing
         address set forth below (or such other address as the Guarantor may
         give notice of to the Holders of the Capital Securities and to the
         Guarantee Trustee):

                           Evans Bancorp, Inc.
                           14-16 N. Main Street
                           P.O. Box 191
                           Angola, New York 14006-0191
                           Attention: Mark DeBacker
                           Telephone: (716) 549-6623

                  (c) If given to any Holder of the Capital Securities, at the
         address set forth on the books and records of the Issuer.

                  All such notices shall be deemed to have been given when
received in person, telecopied with receipt confirmed, or mailed by first class
mail, postage prepaid, except that if a notice or other document is refused
delivery or cannot be delivered because of a changed address of which no notice
was given, such notice or other document shall be deemed to have been delivered
on the date of such refusal or inability to deliver.

                  Section 8.04. Benefit.

                  This Guarantee is solely for the benefit of the Holders of the
Capital Securities and, subject to Section 2.1(a), is not separately
transferable from the Capital Securities.

                  Section 8.05. Governing Law.

                  THIS GUARANTEE SHALL BE GOVERNED BY, AND CONSTRUED IN
ACCORDANCE WITH, THE LAW OF THE STATE OF NEW YORK, WITHOUT REGARD TO CONFLICT OF
LAWS PRINCIPLES THEREOF.

                  Section 8.06. Counterparts.

<PAGE>

                                                                             189

                  This Guarantee may contain more than one counterpart of the
signature page and this Guarantee may be executed by the affixing of the
signature of the Guarantor and the Guarantee Trustee to any of such counterpart
signature pages. All of such counterpart signature pages shall be read as though
one, and they shall have the same force and effect as though all of the signers
had signed a single signature page.

<PAGE>

                                                                             190

                  THIS GUARANTEE is executed as of the day and year first above
written.

                                    EVANS BANCORP, INC.,
                                        as Guarantor

                                    By: /s/James Tilley
                                        ---------------
                                        Name: James Tilley
                                        Title: President and Chief Executive
                                               Officer

                                    WILMINGTON TRUST COMPANY, as Guarantee
                                    Trustee

                                    By: /s/Denise M. Geran
                                        ------------------
                                        Name: Denise M. Geran
                                        Title: Vice President