<PAGE>


                                                                    EXHIBIT 10.4

                      CHANGE IN CONTROL SEVERANCE AGREEMENT
                      -------------------------------------

     THIS CHANGE IN CONTROL SEVERANCE AGREEMENT ("Agreement") entered into this
1st day of January, 2002 ("Effective Date"), by and between Heritage Bank (the
"Bank") and Jack Bowdoin (the "Employee").

     WHEREAS, the Employee is currently employed by the Bank as Senior Vice
President and is experienced in all phases of the financial service industry and
the business of the Bank; and

     WHEREAS, the parties desire by this writing to set forth the rights and
responsibilities of the Bank and Employee if the Bank should undergo a change in
control (as defined hereinafter in the Agreement) after the Effective Date.

     NOW, THEREFORE, it is AGREED as follows:

     1.   Employment.
          ----------

     The Employee is employed in the capacity as a Senior Vice President of the
Bank. The Employee shall render such administrative and management services to
the Bank and CCF Holding Company ("Parent") as are currently rendered and as are
customarily performed by persons situated in a similar executive capacity. The
Employee's other duties shall be such as the President or the Board of Directors
for the Bank (the "Board of Directors" or "Board") may from time to time
reasonably direct, including normal duties as an officer of the Bank and the
Parent.

     2.   Term of Agreement.
          -----------------

     The term of this Agreement shall be for the period commencing on the
Effective Date and ending twelve (12) months thereafter ("Term"). The term of
this Agreement may be extended for up to an additional one year period beyond
the then effective expiration date upon a determination and resolution of the
Board of Directors that the performance of the Employee has met the requirements
and standards of the Board, and that the Term of such Agreement shall be
extended.

     3.   Termination of Employment in Connection with or Subsequent to a Change
          ----------------------------------------------------------------------
          in Control.
          ----------

     (a) Notwithstanding any provision herein to the contrary, in the event of
the involuntary termination of Employee's employment under this Agreement,
absent Just Cause, in connection with, or within twelve (12) months after, and
Change in Control of the Bank or Parent, Employee shall be paid an amount equal
to 100% of the taxable compensation paid to Employee by the Bank for the twelve
month period prior to the date of termination of employment (whether said
amounts were received or deferred by the Employee) and the costs associated with
maintaining coverage under the Bank's medical and dental insurance reimbursement
plans similar to that in effect on the date of termination of employment for a
period of one year thereafter. Said sum shall be paid, at the option of
Employee, either in one (1) lump sum within thirty (30) days of such termination
discounted to the present value of such payment using as the discount rate the
"prime rate" as published in the Wall Street Journal Eastern Edition as of the
date of such payment minus 100 basis points, or in periodic payments over the
next 12 months, and such payments shall be in lieu of any other future payments
which the Employee would be otherwise entitled to receive. Notwithstanding the
forgoing, all sums payable hereunder shall be reduced in such manner and to such
extent so that no such payments made hereunder when aggregated with all other
payments to be made to the Employee by the Bank or the Parent shall be deemed an
"excess parachute payment" in accordance with Section 280G of the Internal
Revenue Codes of 1986, as amended (the "Code") and be subject to the excise tax
provided at Section 4999 (a) of the Code. The term "Change in Control" shall
mean: (i) the execution of an agreement for the sale of all, or a material
portion, of the assets of the Bank or the Parent; (ii) the execution of an
agreement for a merger or recapitalization of the Bank or the Parent or any
merger or recapitalization whereby the Bank or the Parent is not the surviving
entity; (iii) a change in control of the Bank or the Parent, as otherwise
defined or determined by the Georgia Department of Banking and Finance or the
Federal Deposit Insurance Corporation or regulations promulgated by them; or
(iv) the acquisition, directly or indirectly, of the beneficial ownership
(within the meaning

<PAGE>


of that term as it is used in Section 13 (d) of the Securities Exchange Act of
1934 and the rules and regulations promulgated thereunder) of twenty-five
percent (25%) or more of the outstanding voting securities of the Bank or the
Parent by any person, trust, entity or group. The term "person" means an
individual other than the Employee, or a corporation, partnership, trust,
association, joint venture, pool, syndicate, sole proprietorship, unincorporated
organization or any other form of entity not specifically listed herein.

     (b) Notwithstanding any other provision of this Agreement to the contrary
except as provided at Sections 4 (b), 4 (c), 4 (d), 4 (e) and 5, Employee may
voluntarily terminate his employment under this Agreement within twenty-four
months following a Change in Control of the Bank or Parent, and Employee shall
thereupon be entitled to receive the payment and benefits described in Section 3
(a) of this Agreement, upon the occurrence, or within ninety (90) days
thereafter, of any of the following events, which have not been consented to in
advance by the Employee in writing: (i) if Employee would be required to move
his personal residence or perform his principal executive functions more than
thirty-five (35) miles from the Employee's primary office as of the signing of
this Agreement; (ii) if the Bank or Parent should fail to maintain the
Employee's base compensation in effect as of the date of the Change in Control
and existing employee benefits plans, including material fringe benefit, stock
option and retirement plans, except to the extent that such reduction in benefit
programs is part of an overall adjustment in benefits for all employees of the
Bank or Parent and does not disproportionately adversely impact the Employee;
(iii) if Employee would be assigned duties and responsibilities other than those
normally associated with his position as referenced at Section 1, herein; or
(iv) if Employee's responsibilities or authority have in any way been materially
diminished or reduced.

     4.   Other Changes in Employment Status.
          ----------------------------------

     (a) Except as provided for at Section 3, herein, the Board of Directors may
terminate the Employee's employment at any time with or without Just Cause
within its sole discretion. This Agreement shall not be deemed to give Employee
any right to be retained in the employment or service of the Bank, or to
interfere with the right of the Bank to terminate the employment of the Employee
at any time. The Employee shall have no right to receive compensation or other
benefits for any period after termination for Just Cause. Termination for "Just
Cause" shall include termination because of the Employee's personal dishonesty,
incompetence, willful misconduct, breach of fiduciary duty involving personal
profit, intentional failure to perform stated duties, willful violation of any
law, rule or regulation (other than traffic violations or similar offenses) or
final cease-and-desist order, or material breach of any provision of the
Agreement.

     (b) If the Employee is removed and/or permanently prohibited from
participating in the conduce of the Bank's affairs by an order issued under
Sections 8 (e) (4) or 8 (g) (1) of the Federal Deposit Insurance Act ("FDIA")
(12 U.S.C. 1818 (e) (4) and (g) (1) ), all obligations of the Bank under this
Agreement shall terminate, as of the effective date of the order, but the vested
rights of the parties shall not be affected.

     (c) If the Bank is in default (as defined in Section 3 (x) (1) of FDIA) all
obligations under this Agreement shall terminate as of the date of default, but
this paragraph shall not affect any vested rights of the contracting parties.

     (d) All obligations under this Agreement shall be terminated, except to the
extent determined that continuation of this Agreement is necessary for the
continued operation of the Bank: (i) by the Bank's primary regulator, or his or
her designee, at the time that the Federal Deposit Insurance Corporation
("FDIC") enters into an agreement to provide assistance to or on behalf of the
Bank under the authority contained in Section 13 (c) of FDIA; or (ii) by the
Bank's primary regulator, or his or her designee, at the time that they, or his
or her designee approves a supervisory merger to resolve problems related to
operation of the Bank or when the Bank is determined by the Georgia Department
of Banking and Finance to be in an unsafe or unsound condition. Any rights of
the parties that have already vested, however, shall not be affected by such
action.

     (e) Notwithstanding anything herein to the contrary, any payments made to
the Employee pursuant to the Agreement, or otherwise, shall be subject to and
conditioned upon compliance with 12 USC 1828 (k) and any regulations promulgated
thereunder.

<PAGE>

     5.   Suspension of Employment.
          ------------------------

     If the Employee is suspended and/or temporarily prohibited from
participating in the conduct of the Bank's affairs by a notice served under
Section 8 (e) (3) or (g) (1) of the FDIA (12 U.S.C. 1818 (e) (3) and (g) (1) ),
the Bank's obligations under the Agreement shall be suspended as of the date of
service, unless stayed by appropriate proceedings. If the charges in the notice
are dismissed, the Bank may in its discretion (i) pay the Employee all or part
of the compensation withheld while its contract obligations were suspended and
(ii) reinstate any of its obligations which were suspended.

     6.   Loyalty; Noncompetition.
          -----------------------

     (a) The Employee shall devote his full time and attention to the
performance of his employment under this Agreement. During the term of
Employee's employment under this Agreement, the Employee shall not engage in any
business or activity contrary to the business affairs or interests of the Bank
or Parent.

     (b) Nothing contained in this Section 6 shall be deemed to prevent or limit
the right of Employee to invest in the capital stock or other securities of any
business dissimilar from that of the Bank or Parent, or solely as a passive or
minority investor, in any business.

     (c) In consideration of entering into this Agreement and the sums payable
by the Bank under this Agreement, Employee agrees that for a period of not less
than one year from the date of termination of employment with the Bank whether
such termination is initiated by the Employee or the Bank, Employee shall not
engage in providing professional service or employment as an employee, director,
consultant, representative, or similar relationship to any financial services
enterprise (including but not limited to a savings and loan association, bank,
credit union, or insurance company) with offices or business activities located
in Fayette County in the State of Georgia. Breach of this provision not to
compete with the business of the Bank shall result in the forfeiture of all
compensation and benefits otherwise provided for herein, as well as the Bank
seeking such other legal remedies, including but not limited to seeking
injunctive relief and monetary damages for such contract breach. This limitation
on future activities shall not affect the payment of compensation payable in
accordance with this Agreement.

     7.   Successors and Assigns.
          ----------------------

     (a) This agreement shall inure to the benefit of and be binding upon any
corporate or other successor of the Bank which shall acquire, directly or
indirectly, by merger, consolidation, purchase or otherwise, all or
substantially all of the assets or stock of the Bank.

     (b) The Employee shall be precluded from assigning or delegating his rights
or duties hereunder without first obtaining the written consent of the Bank.

     8.   Amendments.
          ----------

     No amendments or additions to this Agreement shall be binding upon the
parties hereto unless made in writing and signed by both parties, except as
herein otherwise specifically provided.

<PAGE>

     9.   Applicable Law.
          --------------

     This agreement shall be governed by all respects whether as to validity,
construction, capacity, performance or otherwise, by the laws of the State of
Georgia, except to the extent that Federal law shall be deemed to apply.

     10.  Severability.
          ------------

     The provisions of this Agreement shall be deemed severable and the
invalidity or unenforceability of any provision shall not affect the validity or
enforceability of the other provisions hereof.

     11.  Arbitration.
          -----------

     Any controversy or claim arising out of or relating to this Agreement, or
the breach thereof, shall be settled by arbitration in accordance with the rules
then in effect of the district office of the American Arbitration Association
("AAA") nearest to the home office of the Bank, and judgment upon the award
rendered may be entered in any court having jurisdiction thereof, except to the
extent that the parties may otherwise reach a mutual settlement of such issue.
The Bank shall reimburse Employee for all reasonable costs and expenses,
including reasonable attorneys' fees, arising from such dispute, proceedings or
actions, following the delivery of the decision of the arbitrator finding in
favor of the Employee. Further, the settlement of the dispute to be approved by
the Board of the Bank or the Parent may include a provision for the
reimbursement by the Bank or Parent to the Employee for all reasonable costs and
expenses, including reasonable attorneys' fees, arising from such dispute,
proceedings or actions, or the Board of the Bank or the Parent may authorize
such reimbursement of such reasonable costs and expenses by separate action upon
a written action and determination of the Board following settlement of the
dispute.

     12.  Entire Agreement.
          ----------------

     This Agreement together with any understanding or modifications thereof as
agreed to in writing by the parties, shall constitute the entire agreement
between the parties hereto.

<PAGE>

     IN WITNESS WHEREOF, the parties have executed this Agreement on the day and
first hereinabove written.

                                                     Heritage Bank


ATTEST:                                              By: /s/ David B. Turner
                                                        ------------------------
/s/ Charles S. Tucker                                   David B. Turner
-------------------------
Charles S. Tucker
Secretary

WITNESS:

/s/ Dick Florin                                      /s/ John C. Bowdoin
-------------------------                            ---------------------------
Dick Florin                                          Jack Bowdoin, Employee