<PAGE>   1
                                                                Exhibit 10(o)(4)


                             THIRD AMENDMENT TO THE
                     PEOPLES HERITAGE FINANCIAL GROUP, INC.
                              THRIFT INCENTIVE PLAN

         The Peoples Heritage Financial Group, Inc. Thrift Incentive Plan (the
"Plan") was last amended and restated effective generally January 1, 1996, and
subsequently amended by a First Amendment effective as of the dates set forth
therein and a Second Amendment effective January 1, 1997. The terms used in this
Third Amendment shall have the meanings set forth in the Plan unless the context
indicates otherwise.

1.       Section 1.49 shall be amended by the addition of the following
Paragraph (h) at the end thereof:

                  "(h) Effective July 1, 1998, all Years of Service credited to
         an Employee under the CFX Corporation Employees' Savings & Profit
         Sharing Plan (the "CFX Plan") or the Concord Savings Bank 401(k) Plan
         (the "Concord Plan") as of the date on which CFX Corporation and its
         subsidiaries were acquired by the Company shall be recognized for
         participation and vesting purposes under this Plan."

2.       Section 2.4 shall be amended by the addition of the following Paragraph
(g) at the end thereof:

                  "(g) (i) Each Employee who was previously employed by CFX
         Corporation or any of its subsidiaries (collectively, "CFX"),
         immediately prior to the date on which CFX was acquired by the Company,
         shall be eligible to participate in the Plan as of the later of July 1,
         1998, or the first day of the Calendar Quarter coincident with or next
         following completion of a Year of Service, provided that either (A) a
         Participation Agreement has been filed with the Plan Administrator by
         the fifteenth day of the month immediately preceding such Calendar
         Quarter or (B) clause (ii) or (iii) applies to the Employee. For
         purposes of determining whether an Employee described in this Section
         has completed a Year of Service, his or her service credited under the
         CFX Plan or the Concord Plan shall be taken into account.

                           "(ii) Notwithstanding clause (i) to the contrary, if
         an Employee described in clause (i) is both employed by any former CFX
         subsidiary except Safety Fund National Bank on June 30, 1998, and a
         participant receiving elective deferrals under the CFX Plan or the
         Concord Plan on such date, then his or her deferral election in effect
         under the applicable plan on such date shall constitute his or her
         initial Participation Agreement under this Plan, provided that any
         terms of such deferral election that are not consistent with the
         provisions of this Plan shall be of no effect hereunder, and provided
         further that the Employee may file a new Participation Agreement in
         accordance with clause (i).

                  "(iii) Notwithstanding clause (i) to the contrary, if an
Employee described in clause (i) is both employed by Safety Fund National Bank
on May 22, 1998, and a participant receiving elective deferrals under the CFX
Plan on such date, then such Employee shall be eligible to participate in this
Plan as of May 22, 1998, and his or her deferral election in effect under the
CFX Plan on such date shall constitute his or her initial Participation
Agreement under


<PAGE>   2



this Plan, provided that any terms of such deferral election that are not
consistent with the provisions of this Plan shall be of no effect hereunder."

         IN WITNESS WHEREOF, to record the adoption of this Third Amendment as
of the effective date stated herein, Peoples Heritage Financial Group, Inc. has
caused this instrument to be executed by its duly authorized officer this 22nd
day of May, 1998.

                                          PEOPLES HERITAGE FINANCIAL GROUP, INC.



                                          By /s/ Carol Mitchell
                                          --------------------------------------
                                          Its Executive Vice President


                                       2