<PAGE>
                                  Exhibit 10.4

                   AMENDMENT TO SALARY CONTINUATION AGREEMENT

     This Amendment to Salary Continuation Agreement (the "Amendment") is made
this 27th day of December, 2005, by and among PeoplesBank, A Codorus Valley
Company, a Pennsylvania banking institution (the "Bank"), and a wholly owned
subsidiary of Codorus Valley Bancorp, Inc., a Pennsylvania business corporation
(the "Corporation") and Harry R. Swift, an adult individual (the "Executive").

                                   WITNESSETH

     WHEREAS, the Bank and the Executive entered into a certain Salary
Continuation Agreement effective the 1st day of October, 1998 (the "Salary
Continuation Agreement"), which is attached hereto;

     WHEREAS, as a result of action by the Board of Directors, the Bank and the
Executive desire to amend the Salary Continuation Agreement provisions regarding
the definition of "Change in Control" and "Change in Control Benefits," which
describes the payment of the Executive's benefit following a Change in Control,
and certain other provisions of the Salary Continuation Agreement;

     WHEREAS, in recognition of the valued services provided by the Executive in
the past to the Bank and the Corporation, the Bank desires to amend the Salary
Continuation Agreement, as an incentive for the Executive to continue to provide
such valued services in the future;

     NOW THEREFORE, in consideration of the mutual covenants and agreements set
forth herein, for good and valuable consideration, and intending to be legally
bound hereby, the Bank and the Executive agree as follows:

          1. The Amendment is incorporated into the Salary Continuation
     Agreement by the Bank, the Corporation and the Executive, in accordance
     with Article 7 of the Salary Continuation Agreement, entitled "Amendments
     and Termination."

          2. All terms set forth in the Amendment shall be defined and
     interpreted by the definitions, construction and intent of the Salary
     Continuation Agreement and shall have the same meaning as therein provided
     unless the context clearly requires a different meaning.

          3. Section 1.1.1 of the Salary Continuation Agreement is hereby
     amended by deleting the existing definition of "Change in Control" in its
     entirety and by adding a new definition of "Change in Control" as follows:

                    1.1.1 CHANGE OF CONTROL. For purposes of this Agreement, the
               term "Change of Control" shall mean: a Change in the


                                      -30-

<PAGE>

               Ownership of the Corporation or the Bank, (as defined below), a
               Change in the Effective Control of the Corporation or the Bank
               (as defined below), or a Change in the Ownership of a Substantial
               Portion of the Assets of the Corporation or the Bank, (as defined
               below).

               (a) Change in the Ownership of the Corporation or the Bank. A
          Change in the Ownership of the Corporation or the Bank occurs on the
          date that any one person, or more than one person acting as a group
          (as defined below), acquires ownership of stock of the Corporation or
          the Bank that, together with stock held by such person or group,
          constitutes more than 50 percent of the total fair market value or
          total voting power of the stock of the Corporation or the Bank.
          However, if any one person, or more than one person acting as a group,
          is considered to own more than 50 percent of the total fair market
          value or total voting power of the stock of the Corporation or the
          Bank, the acquisition of additional stock by the same person or
          persons is not considered to cause a Change in the Ownership of the
          Corporation or the Bank. An increase in the percentage of stock owned
          by any one person, or persons acting as a group, as a result of a
          transaction in which the Corporation or the Bank acquires its stock in
          exchange for property will be treated as an acquisition of stock for
          these purposes. A change in ownership of the Corporation or the Bank
          only occurs when there is a transfer or issuance of stock of the
          Corporation or the Bank and the stock remains outstanding after the
          transaction.

               (b) Change in Effective Control of the Corporation or the Bank. A
          Change in Effective Control of the Corporation or the Bank occurs only
          on the date that either:

                    (i) Any one person, or more than one person acting as a
               group (as defined below), acquires (or has acquired during the
               12-month period ending on the date of the most recent acquisition
               by such person or persons) ownership of stock of the Corporation
               or the Bank possessing 35 percent or more of the total voting
               power of the stock of the Corporation or the Bank; or

                    (ii) A majority of members of the Corporation's Board of
               Directors is replaced during any 12-month period by directors
               whose appointment or election is not endorsed by a majority of
               the members of the Corporation's Board of Directors prior to the
               date of the appointment or election.

                    If any one person, or more than one person acting as a
               group, is considered to effectively control the Corporation or
               the Bank, the acquisition of additional control of the
               Corporation or the Bank by the


                                      -31-

<PAGE>

               same person or persons is not considered to cause a Change in the
               Effective Control of the Corporation or the Bank.

               (c) Change in Ownership of a Substantial Portion of the
          Corporation's or the Bank's Assets. A Change in Ownership of a
          Substantial Portion of the Corporation's or the Bank's Assets occurs
          on the date that any one person, or more than one person acting as a
          group (as defined below), acquires (or has acquired during the
          12-month period ending on the date of the most recent acquisition by
          such person or persons) assets from the Corporation or the Bank that
          have a total gross fair market value equal to or more than 40 percent
          of the total gross fair market value of all of the assets of the
          Corporation or the Bank immediately prior to such acquisition or
          acquisitions. For this purpose, gross fair market value means the
          value of assets of the Corporation or the Bank, or the value of the
          assets being disposed of, determined without regard to any liabilities
          associated with such assets.

               There is no Change in Control under this Section 1.1.1 if there
          is a transfer of assets to an entity that is:

                    (i) A shareholder of the Corporation or the Bank
               (immediately before the asset transfer) in exchange for or with
               respect to its stock;

                    (ii) An entity, 50 percent or more of the total value or
               voting power of which is owned, directly or indirectly, by the
               Corporation or the Bank;

                    (iii) A person, or more than one person acting as a group,
               that owns, directly or indirectly, 50 percent or more of the
               total value or voting power of all the outstanding stock of the
               Corporation or the Bank; or

                    (iv) An entity, at least 50 percent of the total value or
               voting power of which is owned, directly or indirectly, by a
               person described in (i), (ii) or (iii) above.

               (d) For purposes of this Section 1.1.1, persons will not be
          considered to be acting as a group solely because they purchase or own
          stock or purchase assets of the Corporation or the Bank at the same
          time. However, persons will be considered to be acting as a group if
          they are owners of a corporation that enters into a merger,
          consolidation, purchase or acquisition of assets, or similar
          transaction, such shareholder is considered to be acting as a group
          with other shareholders in a corporation only to the extent of the
          ownership in that corporation prior to the transaction giving rise to
          the change and not with respect to the ownership interest in the other
          corporation.


                                      -32-

<PAGE>

          4. Section 1.1.2 of the Salary Continuation Agreement is hereby
     amended by deleting the existing definition of "Date of Change of Control"
     in its entirety and by adding a new definition of "Date of Change of
     Control" as follows:

               1.1.2 "Date of Change of Control" means the date on which a
          "Change of Control" event occurs under Section 1.1.1.

          5. Section 1.1.4 of the Salary Continuation Agreement is hereby
     amended by deleting the existing definition of "Disability" in its entirety
     and by adding a new definition of "Disability" as follows:

               1.1.4 "Disability" means, if the Executive is covered by a
          Company sponsored disability policy, total disability as defined in
          such policy without regard to any waiting period, provided that the
          definition of disability applied under such policy complies with the
          requirement of Section 1.409A-3(g)(4) of the Treasury regulations. If
          the Executive is not covered by such policy, Disability means that the
          Executive is unable to engage in any substantial gainful activity by
          reason of any medically determinable physical or mental impairment
          that can be expected to result in death or can be expected to last for
          a continuous period of not less than 12 months. As a condition to any
          benefit, the Company may require the Executive to submit to such
          physical or mental evaluations and tests as the Company's Board of
          Directors deems appropriate.

          6. Section 2.4 of the Salary Continuation Agreement is hereby amended
     to add a new subsection 2.4.4 "Excise Tax Matters" as follows:

               2.4.4 Excise Tax Matters.

               In the event that the amounts and benefits payable under this
          Section 2.4, when added to other amounts and benefits which may become
          payable to the Executive by the Corporation and/or Bank, are such that
          he becomes subject to the excise tax provisions of Section 4999 of the
          Internal Revenue Code of 1986, as amended (the "Code"), the
          Corporation and the Bank shall pay him such additional amount or
          amounts as will result in his retention (after the payment of all
          federal, state and local excise, employment, and income taxes on such
          payments and the value of such benefits) of a net amount equal to the
          net amount he would have retained had the initially calculated
          payments and benefits been subject only to income and employment
          taxation. For purposes of the preceding sentence, the Executive shall
          be deemed to be subject to the highest marginal federal, state and
          local tax rates. All calculations required to be made under this
          subparagraph shall be made by the Corporation's independent certified
          public accountants, subject to the right of Executive's representative
          to review the same. All such amounts required to be paid shall be paid
          at the time any withholding may be required under applicable law, and
          any additional amounts to which the


                                      -33-

<PAGE>

          Executive may be entitled shall be paid or reimbursed no later than
          fifteen (15) days following confirmation of such amount by the
          Corporation's accountants. In the event any amounts paid hereunder are
          subsequently determined to be in error because estimates were required
          or otherwise, the parties agree to reimburse each other to correct
          such error, as appropriate, and to pay interest thereon at the
          applicable federal rate (as determined under Code Section 1274A for
          the period of time such erroneous amount remained outstanding and
          unreimbursed). The parties recognize that the actual implementation of
          the provision of this subparagraph are complex and agree to deal with
          each other in good faith to resolve any questions or disagreements
          arising hereunder.

          7. Article 2 of the Salary Continuation Agreement is hereby amended by
     adding a new Section 2.5 to read as follows:

               2.5 Key Employee. Notwithstanding anything in this Article to the
          contrary, in the event Executive is determined to be a Key Employee,
          as that term is defined in Section 409A of the Code and the
          regulations promulgated thereunder, payments to the Executive under
          Section 2.1 or 2.2 of this Agreement shall begin not earlier than the
          first day of the seventh month following termination of employment.
          For purposes of the foregoing, the date upon which a determination is
          made as to the Key Employee status of the Executive, the
          Indemnification Date (as defined in Section 409A of the Code and the
          regulations promulgated thereunder) shall be December 31.

          8. Article 5 of the Salary Continuation Agreement is hereby amended by
     adding a new Section 5.3.4 to read as follows:

               5.3.4 Termination Following Change of Control. Notwithstanding
          the foregoing, it is the intention of the parties that the
          restrictions set forth in Sections 5.3.1(i). (ii), (iii) and (iv)
          shall not apply in the event Executive's employment terminates
          following a Change of Control, as defined in the Agreement.

          9. Article 8 of the Salary Continuation Agreement is hereby amended by
     adding a new Section 8.12 to read as follows:

               8.12 RABBI TRUST. The Corporation is establishing
          contemporaneously herewith a rabbi trust (the "Trust"), to which it is
          contributing an initial corpus of $100. In the event of a change of
          control as defined herein, the Corporation shall, in accordance with
          the terms of the Trust, contribute thereto the amount described in
          Section 1(e) thereof. Thereafter, amounts payable hereunder shall be
          paid first from the assets of such Trust and the income thereon. To
          the extent that the assets of the Trust and the income thereon are
          insufficient, the Corporation or any successor of the Corporation
          shall pay Executive the amount due hereunder.


                                      -34-

<PAGE>

          10. In all other respects, the Salary Continuation Agreement, as
     amended above, is hereby ratified and confirmed by the Bank, the
     Corporation and the Executive All other provisions of the Salary
     Continuation Agreement shall remain in full force and effect as amended
     hereby.

     IN WITNESS WHEREOF, the parties, each intending to be legally bound, have
executed the amendment as of the date, month and year first above written.

ATTEST:                                 PEOPLESBANK, A CODORUS VALLEY COMPANY


/s/ Barbara J. Myers                    By: /s/ Rodney L. Krebs
-------------------------------------       ------------------------------------
Secretary                                   Chairman of the Board


WITNESS:


/s/ Natalie L. Thompson                 /s/ Harry R. Swift
-------------------------------------   ----------------------------------------
                                        Harry R. Swift


                                      -35-