<Page>

                                                                    EXHIBIT 10.4

                                                                  EXECUTION COPY

                          PAYMENTS AND WAIVER AGREEMENT

     AGREEMENT, dated as of December 19, 2002 ("AGREEMENT DATE"), by and among
Seacoast Financial Services Corporation ("SEACOAST"), Bay State Bancorp, Inc.(
"BANCORP"), Bay State Federal Savings Bank (the "BANK"), a wholly-owned
subsidiary of Bancorp and Michael O. Gilles (the "EXECUTIVE").

                                   WITNESSETH

     WHEREAS, the Executive is senior vice president and chief financial officer
of Bancorp and Bank; and

     WHEREAS, each of Bancorp and the Bank have entered into a change in control
agreement as well as a variety of other agreements and arrangements with the
Executive under which the Executive is entitled to certain payments in the event
of a change in control of Bancorp or under certain other circumstances; and

     WHEREAS, Seacoast and Bancorp are prepared to enter into an Agreement and
Plan of Merger, dated as of the date hereof (the "MERGER AGREEMENT"), pursuant
to which Bancorp will merge with and into a subsidiary of Seacoast on the terms
and conditions set forth therein and, in connection therewith, outstanding
shares of Bancorp Common Stock will be converted into shares of Seacoast Common
Stock and/or cash in the manner set forth therein; and

     WHEREAS, as an inducement to Seacoast to enter into the Merger Agreement,
Bancorp and the Bank (collectively, the "EMPLOYERS") and the Executive desire to
enter into this Agreement among themselves and with Seacoast so as to set forth
their mutual understanding of various matters relating to the Executive; and

     NOW, THEREFORE, in consideration of the foregoing and the mutual covenants
set forth herein, the parties hereto agree as follows:

     1.  TERMINATION OF EMPLOYMENT. At the Effective Time, all employment
relationships between the Executive and Bancorp, the Bank or any affiliate of
Bancorp shall terminate. Without limiting the foregoing, the Employers and the
Executive agree that at the Effective Time:

            (i) the Executive shall cease to be senior vice-president and chief
     financial officer of Bancorp and the Bank;

            (ii) any other employment or consulting relationships between the
     Executive and either of Bancorp, the Bank or any other direct or indirect
     subsidiary of Bancorp shall terminate; and

            (iii) the Executive's membership on the boards of directors of any
     direct or indirect subsidiary of Bancorp shall terminate.

<Page>

     2.  TERMINATION OF VARIOUS EMPLOYMENT ARRANGEMENTS.

         (a) DEFINITION OF EMPLOYMENT ARRANGEMENT. The term "Employment
Arrangement" shall mean any plan, agreement or arrangement of Bancorp or the
Bank or any affiliate (i) to which the Executive is a party, or (ii) with
respect to which the Executive is a direct or indirect beneficiary, or (iii)
under or with respect to which the Executive may have any right to receive
compensation, payments or any other benefit.

         (b) TERMINATION OF EMPLOYMENT ARRANGEMENTS. Except as otherwise
provided in Section 3 or in Section 4, from and after the Effective Time, the
Executive shall not be entitled to receive any further payments or benefits
under any Employment Arrangements. Without limiting the generality of the
foregoing, the parties specifically agree that from and after the Effective
Time, each of the Employment Arrangements shall automatically terminate without
the necessity of any further action on the part of any party thereto, with the
result that any and all obligation of either Employer under the Employment
Arrangements shall be null and void and neither the Executive nor any heir,
successor or assignee shall have any continuing rights thereunder. In
furtherance of the limitations on rights and benefits described in this Section
2(a), from and after the Effective Time the following Employment Arrangements
will terminate:

             (i) The Three Year Change in Control Agreements between the
    Executive and Bancorp and Bank, dated as of March 2, 1998, each including,
    without limitation, any and all payments and benefits that by the terms of
    the Three Year Change in Control Agreements are to continue after
    termination or expiration of the Three Year Change in Control Agreement;

             (ii) The Management Supplemental Executive Retirement Plan
    maintained by the Bank; and

             (iii) Any other Employment Arrangement now in existence or
    hereinafter adopted that is not specifically listed as a "Permitted
    Arrangement" under Section 4.

     3.  PAYMENTS TO THE EXECUTIVE. In connection with the terminations provided
in Section 1 and in Section 2, Bancorp or Bank shall pay the following amounts
to the Executive, immediately prior to the Effective Time:

         (a) ACCRUED BASE SALARY AND VACATION. Immediately prior to the
Effective Time, Bancorp or Bank shall pay to the Executive a cash amount
equal to the sum of (i) the Executive's accrued but unpaid annual base salary
to the Effective Time and (ii) an amount in compensation for Executive's
accrued but unused vacation time, which amounts to $0 as of the Agreement
Date and will accrue until the Effective Time at the rate of 0.38 days per
week, less vacation days actually taken between the Agreement Date and the
Effective Time. The Executive represents to each of the other parties hereto
that his base salary has been paid to him in full through and including the
last regular payment date established by the Employers for the payment of
wages to its employees.

         (b) EMPLOYMENT ARRANGEMENT TERMINATION PAYMENT. Immediately prior to
the Effective Time, Bancorp or Bank shall pay to the Executive a cash amount
equal to the Termination Payment.

                                        2
<Page>

             (i) The Termination Payment shall be the lesser of (x) $801,000 and
    (y) the Permissible Amount.

             (ii) The Permissible Amount shall be equal to the maximum amount
    which may be paid to the Executive without causing any portion of such
    payment to be non-deductible to either Employer or Seacoast by reason of the
    operation of Section 280G of the Internal Revenue Code of 1986, as amended
    (the "CODE") (computed after deducting the value of all non-cash items, such
    as the acceleration of option vesting, that are treated as parachute
    payments under Section 280G of the Code).

     4.  CERTAIN SPECIFIED PERMITTED ARRANGEMENTS TO REMAIN IN EFFECT. Except as
expressly provided in this Agreement, the parties agree that the provisions of
Section 2(a) shall have no special effect on the rights and obligations of the
Executive and the Employers under the following plans and arrangements in the
form in effect on the Agreement Date and disclosed to Seacoast as of such date
(collectively, the "PERMITTED ARRANGEMENTS"):

         (a) Bancorp's 1998 Stock-Based Incentive Plan,

         (b) Bancorp's 401(a) Savings Plan,

         (c) Bancorp's Defined Benefit Pension Plan,

         (d) Bank's Employee Stock Ownership Plan, or

         (e) The Retirement Benefit Equalization Plan maintained by the Bank.

     5.  CONTINUATION OF CERTAIN OBLIGATIONS. Notwithstanding the provisions of
Section 2(b), the termination of the Employment Arrangements shall not relieve
the Executive of any obligation to keep business or other information related to
either Employer or to its customers or business confidential or to cooperate
with the Employers and their affiliates in certain litigation matters.

     6.  TERMINATION OF 1999 STOCK OPTION PLAN. The Executive understands and
agrees that as of the Agreement Date the Board of Directors of Bancorp has
amended the 1999 Stock Option Plan, and the Executive confirms that he has no
rights in connection with such Plan and that no options will be granted under
the Plan between the Agreement Date and the Effective Time.

     7.  NO OTHER RIGHTS TO BENEFITS IN CONNECTION WITH CHANGE IN CONTROL. For
the sake of clarity, the parties note that the following provisions (among
others) relate to the time period between the Agreement Date and the Expiration
Date (as such term is defined in Section 21).

         (a) The Executive understands that under various Employment
Arrangements a "change in control" may be deemed to have occurred before the
Effective Time and that the Executive may have various rights to obtain benefits
and payments before the Effective Time as a result of the occurrence or prospect
of a change in control, because of his expected termination of employment, or
otherwise.

         (b) The Executive agrees and confirms, for the benefit of each Employer
and for the benefit of Seacoast, that, notwithstanding the provisions of any
Employment Arrangement, he

                                        3
<Page>

will not be entitled to receive, will not take any action to obtain, and will
not accept, any payments or other benefits of any kind whatsoever between the
Agreement Date and the Expiration Date other than:

             (i) the benefits to which he is specifically entitled pursuant to
    Section 3 or Section 4 of this Agreement;

             (ii) regular incremental payments of his base salary at the rates
    in effect on the Agreement Date;

             (iii) payment of normal year-end bonus at a rate accrued for and
    paid in the ordinary course of business consistent with past practice and
    with Section 4.01(b)(7) of the Merger Agreement, to the extent voted by the
    Board of Directors of the Bank or Bancorp;

             (iv) reimbursement in the ordinary course of reasonable and
    necessary business expenses;

             (v) the continuation of medical benefits for the Executive, to the
    extent the Employers are able to provide such medical benefits, PROVIDED,
    HOWEVER, that the Executive shall be responsible for paying the full cost of
    such medical benefits; and

             (vi) benefits under any other employee benefit plan made available
    to employees of the Employers generally or under the terms of the Retirement
    Benefit Equalization Plan, pursuant to which the Executive has any accrued
    rights or is entitled to any benefits or payments during the time period
    specified above, PROVIDED, HOWEVER, that such benefits are made available to
    the Executive only on terms no more favorable to the Executive than to all
    of the employees of the Employers taken as a whole.

         (c) Each Employer agrees not to take any action that could result in
the Executive receiving, now or in the future, any benefit other than those
specifically permitted under Section 7(b).

     8.  NO AMENDMENTS TO EMPLOYMENT ARRANGEMENTS. Each Employer and the
Executive agree that from and after the Agreement Date they will not (a) modify
or amend, or permit any other person to modify or amend, any Employment
Arrangement or (b) adopt any new Employment Arrangement.

     9.  RELEASE. For, and in consideration of the commitments made herein by
Seacoast, the Executive, for himself and for his heirs and assigns, does hereby
release completely and forever discharge Seacoast and its subsidiaries,
affiliates, stockholders, attorneys, officers, directors, agents, employees,
successors and assigns, and any other party associated with Seacoast (the
"RELEASED PARTIES"), to the fullest extent permitted by applicable law, from any
and all claims, rights, demands, actions, liabilities, obligations, causes of
action of any and all kind, nature and character whatsoever, known or unknown,
in any way connected with his employment by Bancorp or any of its subsidiaries
(including in each case predecessors thereof), either as a director, officer or
employee, or termination of such employment. Notwithstanding the foregoing, the
Executive does not release Seacoast from any obligations of Seacoast to the
Executive under (i) the Permitted Arrangements, (ii) Section 7.15 of the Merger
Agreement and (iii) this Agreement.

                                        4
<Page>

     10. LIMITATION ON BENEFITS. The provisions of this Section 10 shall become
effective as of the Agreement Date.

         (a) It is the intention of the Executive, the Employers and Seacoast
that no payments by Seacoast or the Employers to or for the benefit of the
Executive shall be non-deductible to the Employers or Seacoast by reason of the
operation of Section 280G of the Code, relating to parachute payments. It is the
intention of the parties that the amount to be paid to the Executive under this
Agreement shall be not greater than the maximum amount which may be paid to the
Executive without causing any portion of such payment to be non-deductible to
the Employers or Seacoast by reason of the operation of Section 280G of the
Code.

         (b) The Employers agree not to pay (or agree to pay) to the Executive
any amount that could be in excess of the maximum amount which may be paid to
the Executive without causing any portion of such payment to be non-deductible
to the Employers or Seacoast by reason of the operation of Section 280G of the
Code.

         (c) If the Executive receives any payment in violation of Section 10(b)
or that is otherwise in excess of the maximum amount which may be paid to the
Executive without triggering an excise tax payment by Executive or causing any
portion of such payment to be non-deductible to the Employers or Seacoast by
reason of the operation of Section 280G of the Code (an "IMPERMISSIBLE
PAYMENT"), such Impermissible Payments shall be reduced to the maximum amount
which can be paid without triggering an excise tax payment by Executive and
which can be deducted by Seacoast or the Employers (as applicable). To the
extent that an Impermissible Payment exceeding such maximum deductible amount
has been made to or for the benefit of the Executive, the Executive agrees to
cure the issue by refunding such excess Impermissible Payments to Seacoast
(promptly upon written request of Seacoast) with interest thereon at the
applicable Federal Rate determined under Section 1274(d) of the Code, compounded
annually, or at such other rate as may be required in order that no such
Impermissible Payments shall trigger payment of an excise tax by Executive and
shall be non-deductible to the Employers by reason of the operation of said
Section 280G. To the extent that there is more than one method of reducing an
Impermissible Payment to bring it within the limitations of said Section 280G,
the Executive shall determine which method shall be followed, provided that if
the Executive fails to make such determination within forty-five (45) days after
Seacoast or the Employers have sent him written notice of the need for such
reduction, Seacoast may determine the method of such reduction in its sole
discretion.

         (d) If any dispute between Seacoast or the Employers and the Executive
as to any of the amounts to be determined under this Section 10 or the method of
calculating such amounts cannot be resolved by Seacoast and the Executive,
either party, after giving three days written notice to the other, may refer the
dispute to arbitration pursuant to Section 11. The determination of such
arbitrator as to the amounts to be determined under this Section 10 and the
method of calculating such amounts shall be final and binding on both (x)
Seacoast and the Employers and (y) the Executive. Seacoast shall bear the costs
of any such determination.

     11. RESOLUTION OF DISPUTES. Any dispute or controversy arising under or in
connection with this Agreement may, at either Seacoast's or the Executive's
option, be settled exclusively by arbitration in Boston, Massachusetts in
accordance with the rules of the American Arbitration

                                        5
<Page>

Association then in effect and at Seacoast's expense. Judgment may be entered on
the arbitrators award in any court having jurisdiction.

     12. REPRESENTATIONS AND WARRANTIES. The parties hereto represent and
warrant to each other that they have carefully read this Agreement and consulted
with respect thereto with their respective counsel, and that each of them fully
understands the content of this Agreement and its legal effect. Each party
hereto also represents and warrants that this Agreement is a legal, valid and
binding obligation of such party which is enforceable against such party in
accordance with its terms.

     13. SUCCESSORS AND ASSIGNS. This Agreement will inure to the benefit of,
and be binding upon, the Executive and his heirs and assigns, and upon Seacoast,
including any successor to Seacoast by merger or consolidation or any other
change in form or any other person or firm or corporation to which all or
substantially all of the assets and business of Seacoast may be sold or
otherwise transferred. This Agreement may not be assigned by any party hereto
without the consent of the other parties.

     14. NOTICES. Any communication to a party required or permitted under this
Agreement, including any notice, direction, designation, consent, instruction,
objection or waiver, shall be in writing and shall be deemed to have been given
at such time as it is delivered personally, or five (5) days after mailing if
mailed, postage prepaid, by registered or certified mail, return receipt
requested, addressed to such party at the address listed below or at such other
address as one such party may, by written notice, specify to the other party or
parties, as applicable:

     If to the Executive:

             Michael O. Gilles
             159 Robert Road
             Marlborough, Massachusetts 01752

     If to Seacoast:

             Seacoast Financial Services Corporation
             One Compass Place
             New Bedford, Massachusetts 02740
             Attention: President

     15. WITHHOLDING. Seacoast may withhold from any amounts payable under this
Agreement such federal, state, local or foreign taxes as shall be required to be
withheld pursuant to any applicable law or regulation.

     16. ENTIRE AGREEMENT; SEVERABILITY.

         (a) This Agreement contains the entire agreement of the parties
relating to the subject matter hereof and shall supersede in its entirety any
and all prior agreements or understandings, whether written or oral, between the
Employers and the Executive relating to the subject matter hereof, other than
those agreements expressly set forth in Section 5, Section 4, or Section 7(b) of
this Agreement. In reaching this Agreement, no party has relied upon any
representation or promise except those set forth herein.

                                        6
<Page>

         (b) Any term or provision of this Agreement which is invalid or
unenforceable in any jurisdiction shall, as to that jurisdiction, be ineffective
to the extent of such invalidity or unenforceability without rendering invalid
or unenforceable the remaining terms and provisions of this Agreement or
affecting the validity or enforceability of any of the terms or provisions of
this Agreement in any other jurisdiction. If any provision of this Agreement is
so broad as to be unenforceable, the provision shall be interpreted to be only
so broad as is enforceable. In all such cases, the parties shall use their
reasonable best efforts to substitute a valid, legal and enforceable provision
which, insofar as practicable, implements the original purposes and intents of
this Agreement.

     17. WAIVER. Failure to insist upon strict compliance with any of the terms,
covenants or conditions hereof shall not be deemed a waiver of such term,
covenant or condition. A waiver of any provision of this Agreement must be made
in writing, designated as a waiver and signed by the party against whom its
enforcement is sought. Any waiver or relinquishment of any right or power
hereunder at any one or more times shall not be deemed a waiver or
relinquishment of such right or power at any other time or times.

     18. COUNTERPARTS. This Agreement may be executed in two or more
counterparts, each of which shall be deemed an original, and all of which shall
constitute one and the same Agreement.

     19. GOVERNING LAW. This Agreement shall be governed by and construed and
enforced in accordance with the laws of The Commonwealth of Massachusetts
applicable to agreements made and to be performed entirely within such
jurisdiction.

     20. HEADINGS. The headings of sections in this Agreement are for
convenience of reference only and are not intended to qualify the meaning of any
section. Any reference to a section number shall refer to a section of this
Agreement, unless otherwise stated.

     21. EFFECTIVENESS. This Agreement shall become effective on the Agreement
Date and shall remain in effect until the Expiration Date. The Expiration Date
shall occur on the first to occur of (a) the date which is five years after the
Agreement Date and (b) such date as the Merger Agreement is terminated by the
parties thereto in accordance with its terms prior to consummation of the
transactions contemplated thereby.

     22. AMENDMENT. This Agreement may not be amended in any respect except by
means of a written agreement duly executed by the Executive and by an authorized
officer of each of Seacoast, the Bank and Bancorp.

                                        7
<Page>

     IN WITNESS WHEREOF, each of the undersigned has entered into this Agreement
as of the day and year first above written.


EXECUTIVE                               SEACOAST FINANCIAL SERVICES
                                        CORPORATION

/s/ Michael O. Gilles
-----------------------------
Michael O. Gilles                       By: /s/ Kevin G. Champagne
                                           -----------------------------------
                                           Name:  Kevin G. Champagne
                                           Title: President and Chief Executive
                                                  Officer


                                        BAY STATE BANCORP, INC.


                                        By: /s/ John F. Murphy
                                           -----------------------------------
                                           Name:  John F. Murphy
                                           Title: President and Chief Executive
                                                  Officer


                                        BAY STATE FEDERAL SAVINGS BANK


                                        By: /s/ John F. Murphy
                                           -----------------------------------
                                           Name:  John F. Murphy
                                           Title: Chairman and Chief Executive
                                                  Officer

                                        8