<PAGE>

                                                                    EXHIBIT 10.3




                              HUDSON VALLEY BANK

                     SUPPLEMENTAL RETIREMENT PLAN OF 1997
                     ------------------------------------

                        EFFECTIVE AS OF JULY 1, 1997



                                  ARTICLE ONE
                                  -----------

1.   PURPOSE OF THE PLAN
     -------------------

     The purpose of the Hudson Valley Bank Supplemental Retirement Plan
(hereinafter called the "Plan") is to provide the participating executives with
an added incentive in order to retain their services until retirement.

     The Plan is intended to be a non-qualified, unfunded Plan which is designed
to supplement all other benefits received, now or in the future, by the
participant.  The Plan may, at its option, purchase insurance on the
participant's life with the Bank being the beneficiary of these policies in
order to reimburse itself, in whole or in part, for its cost of such benefits.

                                  ARTICLE TWO
                                  -----------

2.   DEFINITIONS
     -----------

     The following words and phrases shall have the respective meaning set forth
below:

     A.  "Plan" shall mean the provisions of the Hudson Valley Bank Supplemental
Retirement Plan of 1997.

     B.  "Bank" shall mean the Hudson Valley Bank, its successors and assigns.

     C.  "Participants" shall be those present and future Senior Executives
selected by the Compensation Committee to participate in the Plan.
<PAGE>

     D.  "Beneficiary" shall mean the person(s) entitled to the benefits as a
beneficiary of a deceased participant.

     E.  "Effective Date" shall mean the 1st day of July, 1997.

     F.  "Base Annual Compensation" shall mean the regular basic salary of the
Participant (before any deductions for a 401(k) Plan or any other deferred
income) actually paid for the services rendered during any applicable calendar
year.  However, it shall exclude any bonuses, overtime, fringe or supplemental
compensation or any payments from prior periods of any amounts that have been
deferred at the request of the employee.

     However, if the Participant shall be considered "disabled" under the Plan,
the Base Annual Compensation shall be the regular basic salary at the time of
disability, increased by the annual percentage in the Cost of Living (for the
New York Metropolitan Area as determined by the Bureau of Labor Statistics)
limited to a maximum of 5% annually until the Normal Retirement Date.

     G.  "Normal Retirement Date" shall mean the first day of the month
coinciding with or next following the date which a Participant has attained Age
65 "Normal Retirement" and has completed at least ten (10) years of service.

     H.  "Interest Rate Offset" shall mean the yield on a 15 Year United States
Treasury Bond on the first day of the month, three (3) months prior to the
Participant's retirement.

     I.  "Lump Sum Equivalent" shall mean the value of the Participant's
Qualified Plan Account as of the date of retirement.

     J.  "Bank Pension Plan Benefit" shall mean the annual amount payable over a
fifteen (15) year period from the lump sum equivalent of any Qualified
Retirement Plan of the Bank at

                                                                               2
<PAGE>

the Participant's retirement assuming the interest rate offset. If the Bank
should terminate its Qualified Plan prior to the Participant's retirement, the
definition of lump sum shall be the lump sum equivalent at termination increased
annually by the interest rate offset on January 1st of each year.

     (An example of this definition is: $100,000 lump sum equivalent at
retirement - 7% interest rate offset - annual 15 year payment of $10,261).

     K.  "401(k) Matching Offset" shall mean the annual amount payable over a 15
year period from the maximum annual amount the Bank would have matched to the
executive's 401(k) contribution (irrespective of whether the executive
contributed the maximum amount) increased annually by the interest rate offset.

     (An example of this definition is: $50,000 lump sum equivalent at
retirement - 7% interest rate offset - annual 15 year payment of $5,130).

                                  ARTICLE THREE
                                  -------------

3.   ADMINISTRATION OF THE PLAN
     --------------------------

     The Compensation Committee of the Bank (hereinafter called the "Committee")
or other committee so designated by the Bank's Board of Directors shall
administer the Plan.  All questions of interpretation and application of the
Plan shall be determined by a majority of the committee and the determination of
such majority shall be final and binding on all persons.

                                                                               3
<PAGE>

                                  ARTICLE FOUR
                                  ------------

4.   POSTPONED RETIREMENT
     --------------------

     Each Participant shall be required to retire at such time as he or she
attains the Normal Retirement Age, provided said Normal Retirement is in
conformity with the State and/or Federal Laws applicable at that time.

     A.  Participant may remain in the employ of the Bank after he or she
attains the Normal Retirement Age for such periods or "at will" as may be agreed
to by the Board of Directors.

     No additional benefits shall accrue for any employment subsequent to Age
65, but upon retirement or separation from employment, the Participant shall
receive the Normal Supplemental Retirement Benefits he or she would have
received had the Participant retired at Age 65.

                                  ARTICLE FIVE
                                  ------------


5.   BENEFITS FORMULA
     ----------------

     A.  Supplemental Pension Benefit - Term - A Participant who retires from
         the Bank under Normal Retirement, or under Postponed Retirement, shall
         receive supplemental retirement benefits for a period of fifteen (15)
         years, payable on a monthly basis (180 months) beginning on the first
         day of the month following his Normal or Postponed Retirement Date .

     B.  Supplemental Pension Benefit - Amount - The Normal Supplemental Pension
         Benefit shall be equal to 60% of the average of the highest five years
         Base Annual Compensation during the last ten years of the Participant's
         employment reduced by the following :

         1.  The Lump Sum Equivalent of Qualified Plan Benefit;

                                                                               4
<PAGE>

         2.  The Lump Sum Equivalent of the 401(k) Matching benefit;

         3.  50% of the Executive's Primary Social Security Benefit; and

         4. The Lump Sum Equivalent of any other retirement type benefits which
have been provided and paid for by the Bank during the course of the
Participant's employment.

         An example of this provision is as follows:

                            Age                       Salary
                  63                                 $150,000
               ------------------------------------------------
                  64                                 $135,000
               ------------------------------------------------
                  62                                 $125,000
               ------------------------------------------------
                  61                                 $110,000
               ------------------------------------------------
                  60                                 $100,000
               ------------------------------------------------
                 Average                             $124,000
               ------------------------------------------------
                 60% of Average Salary               $ 74,400
               ------------------------------------------------
                 Minus Pension Plan                  $24,4000
                 Lump Sum Equivalent
               ------------------------------------------------
                 Minus 401(k) Match                  $  5,100
                 Lump Sum Equivalent
               ------------------------------------------------
                 Minus 50% Primary                   $  7,500
                 Social Security
               ------------------------------------------------
                 Equals 15 Yr.                       $ 37,400
                 Supplemental Pension
               ------------------------------------------------

     C. Transfer of Duties/Authorized Leave - Notwithstanding any provision to
the contrary, so long as the Participant shall continue to be in the employ of
the Bank or the Hudson Valley Holding Corp. or one or more of its subsidiaries,
his or her eligibility and vesting shall not be affected by any change of duties
or position. In addition, an authorized leave of absence shall not affect the
Participant's rights. However, nothing in this Plan or in any other agreement

                                                                               5
<PAGE>

shall confer upon any Participant any right to continue' in the employment of
the Bank, Holding Corp., or any such subsidiary.

                                  ARTICLE SIX
                                  -----------

6.   VOLUNTARY RESIGNATION, TERMINATION WITH AND WITHOUT CAUSE, PRIOR TO
                                                                --------
     RETIREMENT.
     -----------

     A.  Voluntary Resignation. In the event the Participant voluntarily
         ---------------------
resigns, before the Participant reaches Age 65, the Bank's obligations to make
any payments under this Agreement shall immediately terminate.

     B.  Termination with Cause. In the event that a Participant is terminated
         ----------------------
for cause including but not limited to wrongful application of funds, commission
of a crime or disregard of Board or Management policy or directives at any time
either before the Participant reaches Age 65 or after the Participant reaches
Age 65 if the Participant has postponed retirement in accordance with Article 4
herein, the Bank's obligations to make any payments under this Agreement shall
immediately terminate.  Additionally, if after receiving payments under this
plan, it comes to the attention of the Bank that while employed, the Participant
engaged in an act that would have caused Termination for Cause, the participant
will no longer be entitled to benefits under this Plan.

     The Compensations Committee's determination as to what constitutes "for
cause" shall be binding and final on all parties.

     C.  Termination without Cause.  In the event the Participant's employment
         -------------------------
is terminated without cause before the Participant shall have reached the Age of
65, the Bank's obligation to make any payments under this Agreement shall
immediately terminate, except the

                                                                               6
<PAGE>

Participant shall be entitled to receive a percentage of the annual benefits
based upon the following formula:


                                        # of Years        Non-Forfeiture
            Age at Termination          Of Service           Benefit
         ------------------------------------------------------------------
            60 but less than 61         10 or more              50%
         ------------------------------------------------------------------
            61 but less than 62         11 or more              60%
         ------------------------------------------------------------------
            62 but less than 63         12 or more              70%
         ------------------------------------------------------------------
            63 but less than 65         13 or more              80%
         ------------------------------------------------------------------
            64 but less than 65         14 or more              90%
         ------------------------------------------------------------------

                                  ARTICLE SEVEN
                                  -------------

7.   TOTAL DISABILITY
     ----------------

     A.  If, prior to the Participant's 65th birthday, Participant should become
disabled by "Total Disability" as defined herein, while employed by the Bank,
the Participant's rights and Bank's payment obligations as described in Article
Five hereof, shall remain unaffected due to such disability.  If such Total
Disability shall continue through the Participant's "Normal Retirement Date",
then the annual benefits shall be payable as if the Participant had fulfilled
all of the conditions in Paragraph 5 of this Agreement.

     B.  "Total Disability", for the purposes of this Agreement, shall be if the
Participant is deemed totally disabled under the Bank's Long Term Disability
Policy in effect at the time of the Participant's disability.

                                                                               7
<PAGE>

     C.  Under no circumstances shall this Article be construed to give the
Participant any additional disability benefits other than those which may be
provided by the Bank under separate disability plans or policies.

                                  ARTICLE EIGHT
                                  -------------


8.   DEATH BENEFITS
     --------------

     A. Death Prior to Age 60 - In the event of a Participant's death while
        ---------------------
employed by the Bank prior to Age 60 and prior to his or her Retirement Date,
the Bank agrees to pay to such beneficiary as Participant may have designated
pursuant to Article Nine or in the absence of any such designation, to the
Participant's surviving spouse, or if no surviving spouse, then to Participant's
estate, 60% of the Participant's Base Annual Compensation (see definitions) for
a period of fifteen (15) years (the benefit years) commencing upon the first day
of the month immediately following the Participant's death, with death benefits
in equal monthly installments subject to the provisions of Paragraph B below.

     B.  Death After Age 60 but Prior to Retirement - In the event of a
         ------------------------------------------
Participant's death after Age 60, but before receiving retirement benefits, the
amount paid to the designated beneficiary shall be the amount as if the
Participant had retired on the normal retirement date.

     C. Eligibility for Death Benefits Prior to Retirement - In order to be
        --------------------------------------------------
eligible for the pre-retirement death benefit under the Plan1 prior to
retirement, the Participant must qualify for life insurance at standard rates
from the insurer designated by the Bank. If the Participant does not qualify for
standard insurance, he or she can qualify for the Pre-Retirement Death Benefit
if the Participant agrees to reduce their compensation from the Bank for that
portion of the life insurance premium greater than the standard premium.

                                                                               8
<PAGE>

             An example of this paragraph is:

             If Standard Premium could be issued      =    $10,000

             Premium Offered by Insurer               =     12,500

             Executive's Annual Salary Reduction      =      2,500


     If the Participant was previously insured at standard rates and receives an
increase in Base Annual Compensation which necessitates additional insurance for
which Participant is uninsurable or cannot qualify for standard rates and the
Participant does not elect salary reduction as stated above, the Participant
will still be entitled to death benefits based upon the Participant's Base
Annual Compensation in effect at the time immediately prior to the increase in
Base Annual Compensation that triggered the need for increased insurance.

     D.  Death Benefit While Receiving Retirement Benefits - In the event of the
         -------------------------------------------------
Participant's death while receiving retirement payments under Articles Five and
Six, Death Benefits shall be payable to such beneficiary as Participant may have
designated pursuant to Article Nine or in the absence of any such designation,
to the Participant's surviving spouse, or if no surviving spouse, then to
Participant's estate, in the same manner as if payable to the Participant.  Any
amounts not fully paid by reason of any such payee's death while receiving
payments, if any, shall also be payable in the same manner and under the same
conditions as if payable to the Participant under Paragraph 8A above.

                                  ARTICLE NINE
                                  ------------

9.   DESIGNATION OF BENEFICIARY
     --------------------------

     To designate a beneficiary or beneficiaries to receive death benefits or
unpaid retirement benefits under this Agreement, the Bank shall provide each
Participant with beneficiary forms to

                                                                               9
<PAGE>

designate the name, address, relationship, and percentage benefit payable to the
beneficiary. If the Bank does not have an executed beneficiary designation of
Participant, the benefits shall be payable to the Participant's surviving spouse
or if the Participant has no surviving spouse, the estate of the Participant. If
the beneficiary of any benefits is the Surviving spouse of the Participant, said
surviving spouse shall have the right to designate a beneficiary to receive any
unpaid benefits due to the surviving spouse at their death. If the beneficiary
is not the surviving spouse, the Participant shall have the right to name
successor beneficiaries.

                                  ARTICLE TEN
                                  -----------

10.  RESTRICTIVE COVENANTS
     ---------------------

     A.  Employment and Solicitation.  The Participant agrees that during
         ---------------------------
employment or while receiving benefits hereunder, the Participant shall not
accept employment or consultancy with any bank, mortgage or brokerage company,
or financial institution, nor solicit as a customer, either directly or
indirectly, on behalf of any other bank, mortgage or brokerage company, or
financial institution, any person or entity that is domiciled or operating
within the Bank's primary market area as defined in its public C.R.A. statement
or anyone who is a current or prospective customer of the Bank, wherever
located.

     B.  Confidential Information.  During the term of employment or while
         ------------------------
receiving benefits hereunder, the Participant agrees to keep confidential all
information provided by the Bank except only such information as is already
known to the public, and including such information and material relating to any
customer, vendor, licensee, or other party transacting business with the Bank
and not to release, use, or disclose the same except with the prior written
permission of the Bank.

                                                                              10
<PAGE>

     Participant further agrees to consider all plans, strategies and techniques
with which the Participant has become familiar to be confidential and the
exclusive property of the Bank which will not be disclosed to anyone for any
reason whatsoever.  All records, files, reports, lists, customer lists, plans
documents, equipment, and the like relating to the business of the Bank, which
the Participant has used or come into contact with, shall remain the sole
property of the Bank.

     Participant agrees that on request of the Bank, and in any event upon
termination, Participant shall turn over to the Bank all documents, papers, or
other material in Participant's possession and under Participant's control which
may contain or be derived from confidential information, together with all
documents, notes, or other work product which is connected with or derived from
Participant's services to the Bank whether or not such material is at the date
hereof in Participant's possession.  Participant agrees that he or she shall
have no proprietary interest in any work product developed or used by him or her
arising out of employment with the Bank.

     C.  Injunctive Relief.  Participant acknowledges that the Bank will suffer
         ------------------
irreparable injury, not readily susceptible of valuation in monetary damages, if
the Participant breaches any of the obligations under paragraphs 10 A or 10 B
above.  Accordingly, the Participant agrees that the bank will be entitled to
injunctive relief against any breach or prospective breach by the Participant of
the Participant's obligation's under paragraphs 10 A or 10 B in any federal or
state court of competent jurisdiction sitting in the State of New York.  The
Participant hereby submits to the jurisdiction of such courts for the purposes
of any actions or proceeding instituted by the Bank to obtain such injunctive
relief, and agrees that process may be served on the Participant by

                                                                              11
<PAGE>

registered mail, addressed to the last address of the Participant known to the
Bank, or in any other manner authorized by law.

                                 ARTICLE ELEVEN
                                 --------------

11.  INSURANCE
     ---------

     It is the intention of the Bank to purchase a life insurance contract or
contracts on the Participant's life payable to the Bank as a means of
reimbursing itself, in whole or in part, for the cost of the benefits provided
in this Agreement.  The Participant agrees to cooperate with the Bank in
obtaining such insurance by giving necessary consents or be submitted to any
necessary physical examinations.  Nothing in this Agreement, however shall
create an obligation on the Bank's part to obtain life insurance or to set aside
any assets or funds to meet the obligations under this Agreement and the Bank
hereby reserves the absolute right in its sole discretion to terminate any
insurance contract it may obtain on the Participant's life, or to terminate in
whole or in part, any other funding program it may elect to undertake in
connection with this Agreement.

                                 ARTICLE TWELVE
                                 --------------

12.  REORGANIZATION
     --------------

     If the Bank is merged, consolidated into or with any other Corporation or
substantially all the assets are transferred to another Corporation, the
provisions of this Agreement are binding upon and inure to the benefit of the
corporation resulting from such merger, consolidation, or transfer.

     Notwithstanding any other provisions of this Agreement, in the event of a
merger, consolidation or sale of the Bank's assets where the Bank remains the
controlling party, such an

                                                                              12
<PAGE>

event shall have no effect on this Agreement and this Agreement shall remain in
full force and effect.

     Notwithstanding any other provisions of this Agreement, in the event of a
merger, consolidation or sale of substantially all of the Bank's assets where
the Bank is not the controlling party and the Participant would at the time of
said merger, consolidation or sale have ten (10) or more years of service and be
under Age 60, the Participant for purposes of this Agreement shall be deemed to
be Age 60.  Even though the Participant may be entitled to receive payments
prior to Age 65 if terminated under this paragraph, the provisions in regard to
postponed retirement under Article Four above shall only be effective after the
Participant's chronological age of 65.

     An example of this paragraph is:

     Executive Age 55 - 12 Year of Service - Bank is sold - Executive is
terminated without cause 2 years after sale - average 5-Year compensation at
that time is $100,000.

     Executive would be entitled to $35,000 less any Qualified Plan Offset 3
years hence, payable for 15 years: i.e.,

     $100,000     x     50%     x     70%     =     $35,000

                    (Pension)   (Age 62)      (Pension Before Offsets)

                                ARTICLE THIRTEEN
                                ----------------

13.  INCAPACITY OF RECIPIENT
      ----------------------

     In the event a Participant or Beneficiary is declared incompetent or the
provisions of the Participant or Beneficiary's Durable Power of Attorney has
been satisfied, then any benefits payable under the Plan to which such
Participant or Beneficiary shall be entitled may be paid to

                                                                              13
<PAGE>

such Guardian or Attorney-in-Fact or any other person legally charged with the
care of this person or estate. Except as provided above in this paragraph, if
the Compensation Committee in its sole discretion determines that a Participant
or Beneficiary is unable to manage his or her own financial affairs, the
Compensation Committee may direct the payments to any persons for the benefit of
the Participant or Beneficiary.

                                ARTICLE FOURTEEN
                                ----------------

14.  CLAIMS BY OTHERS
     ----------------

     At no time shall the Participant's estate, Participant's spouse, or any
other beneficiary the Participant may have designated under this Agreement, be
deemed to have any claims, rights, title, or any other interest in or to any
life insurance contract(s) the Bank may have obtained or any specific fund or
asset belonging to the Bank. Regarding any claim for unpaid benefits under this
Agreement, the Participant, Participant's spouse, or any other designated
beneficiary shall be an unsecured creditor of the Bank with no greater rights
than any other creditor having a general claim for unpaid compensation.

                                ARTICLE FIFTEEN
                                ---------------

15.  ENCUMBRANCES
     ------------

     It is expressly agreed all payments under this Agreement and any rights
thereto are nonassignable and nontransferable.  Neither the Participant, the
Participant's spouse, nor any other beneficiary shall have the right to commute,
sell, pledge, assign, transfer or otherwise convey the right to receive any
payments under this Agreement. Such payments shall not be subject to legal
process or levy of any kind.

                                                                              14
<PAGE>

                                ARTICLE SIXTEEN
                                ---------------

16.  RELATION TO OTHER BENEFITS
     --------------------------

     The benefits under this Agreement shall be independent of, and in addition
to, benefits payable under any other employment agreement that may exist from
time to time between the parties hereto, or any other compensation payable by
the Bank to the Participant whether as salary, or otherwise.  This Agreement
shall not be deemed to constitute a contract of employment between the parties,
nor shall any provision hereof restrict the absolute right of the Bank to
discharge the Participant at will or restrict the absolute right of the
Participant to terminate his/her employment at will.  However, it is intended
that this Agreement shall remain in effect until all benefits have been paid,
except in the event of earlier termination of the Agreement as provided under
any other termination provision of this Agreement.

                               ARTICLE SEVENTEEN
                               -----------------

17.  LUMP SUM PAYMENT OF BENEFITS
     ----------------------------

     The Corporation shall have the right to commute any benefits after they
become payable under the program but shall be limited to utilizing the Interest
Rate Offset.

         An example of this article is:

     1.  Annual Payment                 $ 37,500

     2.  Years Remaining                       5

     3.  Interest Rate Offset                  7%

     4.  Commuted Amount                $164,520

                                                                              15
<PAGE>

                                ARTICLE EIGHTEEN
                                ----------------

18.  AMENDMENTS
     ----------

     During the Participant's lifetime this Agreement may be terminated or
amended in any particular by the mutual written agreement of the Participant and
the Bank.

                                ARTICLE NINETEEN
                                ----------------

19.  BINDING EFFECT
     --------------

     This Agreement shall be binding upon the parties hereto, their heirs,
executors and administrators, conservators, attorney-in-fact and successors in
interest.

                                 ARTICLE TWENTY
                                 --------------

20.  GOVERNING LAW
     -------------

     This Plan shall be construed in accordance with the laws of the State of
New York, except to the extent (if any) preempted by federal law.

                               ARTICLE TWENTY-ONE
                               ------------------

21.  ARBITRATION
     -----------

     Unless otherwise provided in this Agreement, any controversy or claim
arising out of or relating to this contract, or the breach thereof, shall be
settled by the American Arbitration Association and the judgment upon the award
rendered by the arbitrators may be entered in any court having jurisdiction
thereof.

                                                                              16
<PAGE>

                               ARTICLE TWENTY-TWO
                               ------------------

22.  SEVERABILITY
     ------------

     It is agreed that the invalidity or unenforceability of any article,
section, provision of this Agreement shall not affect the validity or
enforceability of any one or more of the other articles, sections, paragraphs or
provisions.

                                              HUDSON VALLEY BANK



                                              BY: __________________________
                                              Chairman of the Board
                                              November 16, 1998



ATTEST:



_______________________________

Secretary


A:\RETIRE.WPD

                                                                              17