<PAGE>

                                                                    EXHIBIT 10.4

                        SUPPLEMENTAL RETIREMENT AGREEMENT


        THIS SUPPLEMENTAL RETIREMENT AGREEMENT is made as of January 1, 2006 by
and between ATLANTIC COAST FEDERAL (the "Bank"), its successors and assigns and
JON C. PARKER, SR. (the "Executive").

1.      DEFINITIONS. In this Agreement, the following words and phrases shall
        have the following meanings:

        (A)     ACCRUED BENEFIT PERCENTAGE shall mean, except as otherwise
                provided in this Agreement, 1.15% for each full calendar quarter
                of the Executive's employment with the Bank since January 1,
                2006, calculated through the last day of the calendar quarter in
                which the Executive (i) experiences a Separation from Service or
                (ii) attains the Normal Retirement Date, whichever shall first
                occur; PROVIDED, HOWEVER, that in no event shall the Accrued
                Benefit Percentage exceed 60%.

        (B)     ADMINISTRATOR shall mean the person or committee appointed by
                the Board of Directors of the Bank to administer this Agreement.
                If a committee is appointed by the Board of Directors, a
                majority of those persons shall constitute a quorum and the act
                of the majority of such of persons either at a meeting or by
                written consent, shall be the act of the Administrator. The
                administrator may adopt such rules and procedures, not
                inconsistent with this Agreement, as it deems necessary or
                appropriate in order to administer this Agreement.

        (C)     AVERAGE COMPENSATION shall mean the amount determined by
                dividing by three (3) the total monetary compensation earned by
                the Executive from the Bank and its affiliates and subsidiaries
                (or any successors thereto by merger or purchase) during the
                three annual periods in the ten year period prior to his
                Separation from Service that results in the largest total,
                including but not limited to salary, bonuses and incentive
                compensation (but excluding specifically stock-based
                compensation, such as restricted stock, stock options and stock
                appreciation rights). An annual period shall consist of any
                twelve (12) month consecutive period not including any portion
                of another twelve (12) month period.

        (D)     BENEFIT COMMENCEMENT DATE shall mean the first business day of
                the calendar month following the earliest of (i) the Executive's
                Normal Retirement Date; (ii) the Executive's Separation from
                Service; (iii) the Executive's death; (iv) the Executive's
                Disability; or (v) a Change in Control.

        (E)     CAUSE shall mean a Separation from Service due to the
                Executive's personal dishonesty, incompetence, willful
                misconduct, breach of fiduciary duty involving personal profit,
                intentional failure to perform stated duties, and willful
                violation of any law, rule, or regulation (other than traffic
                violations or similar offenses) or final cease-and-desist order.


                                       1
<PAGE>

        (F)     CHANGE IN CONTROL shall mean the following:

                (1)     "Change in Control" shall mean (i) a change in the
                ownership of the Bank or Atlantic Coast Federal Corporation (the
                "Company"), (ii) a change in the effective control of the Bank
                or Company, or (iii) a change in the ownership of a substantial
                portion of the assets of the Bank or Company, as described
                below. Notwithstanding anything herein to the contrary, the
                reorganization of Atlantic Coast Federal, MHC by way of a
                "second-step conversion" shall not be deemed a Change in
                Control.

                (2)     A change in ownership occurs on the date that any one
                person, or more than one person acting as a group (as defined in
                Proposed Treasury Regulations section 1.409A-3(g)(5)(v)(B)),
                acquires ownership of stock of the Bank or Company that,
                together with stock held by such person or group, constitutes
                more than 50% of the total fair market value or total voting
                power of the stock of such corporation.

                (3)     A change in the effective control of the Bank or Company
                occurs on the date that either (i) any one person, or more than
                one person acting as a group (as defined in Proposed Treasury
                Regulations section 1.409A-3(g)(5)(vi)(B)) acquires (or has
                acquired during the 12-month period ending on the date of the
                most recent acquisition by such person or persons) ownership of
                stock of the Bank or Company possessing 35% or more of the total
                voting power of the stock of the Bank or Company, or (ii) a
                majority of the members of the Bank's or Company's board of
                directors is replaced during any 12-month period by directors
                whose appointment or election is not endorsed by a majority of
                the members of the Bank's or Company's board of directors prior
                to the date of the appointment or election, provided that this
                sub-section "(ii)" is inapplicable where a majority shareholder
                of the Bank or Company is another corporation.

                (4)     A change in a substantial portion of the Bank's or
                Company's assets occurs on the date that any one person or more
                than one person acting as a group (as defined in Proposed
                Treasury Regulations section 1.409A-3(g)(5)(vii)(C)) acquires
                (or has acquired during the 12-month period ending on the date
                of the most recent acquisition by such person or persons) assets
                from the Bank or Company that have a total gross fair market
                value equal to or more than 40% of the total gross fair market
                value of (i) all of the assets of the Bank or Company, or (ii)
                the value of the assets being disposed of, either of which is
                determined without regard to any liabilities associated with
                such assets. For all purposes hereunder, the definition of
                Change in Control shall be construed to be consistent with the
                requirements of Proposed Treasury Regulations section
                1.409A-3(g)(5), except to the extent that such proposed
                regulations are superseded by subsequent guidance.

        (G)     DISABLED OR DISABILITY shall mean the Executive:

                (1)     is unable to engage in any substantial gainful activity
                by reason of any medically determinable physical or mental
                impairment which can be expected to result in death, or last for
                a continuous period of not less than 12 months;


                                       2
<PAGE>

                (2)     by reason of any medically determinable physical or
                mental impairment which can be expected to result in death, or
                last for a continuous period of not less than 12 months, is
                receiving income replacement benefits for a period of not less
                than three months under an accident and health plan covering
                employees of the Bank; or

                (3)     is determined to be totally disabled by the Social
                Security Administration.

        (H)     INVOLUNTARY TERMINATION shall mean Separation from Service
                without the Executive's express written consent, and shall
                include a material diminution of or interference with the
                Executive's duties, responsibilities and benefits as Executive
                Vice President of the Bank, including (without limitation) any
                of the following actions unless consented to in writing by the
                Executive: (i) a change in the principal workplace of the
                Executive to a location outside of a 30 mile radius from the
                Executive's principal workplace as of the date hereof; (ii) a
                material demotion of the Executive; (iii) a material reduction
                in the number or seniority of other personnel reporting to the
                Executive or a material reduction in the frequency with which,
                or on the nature of the matters with respect to which, such
                personnel are to report to the Executive, other than as part of
                an institution-wide reduction in staff; (iv) a material adverse
                change in the Executive's salary, perquisites, benefits,
                contingent benefits or vacation, other than as part of an
                overall program applied uniformly and with equitable effect to
                all members of the senior management of the Bank; and (v) a
                material permanent increase in the required hours of work or the
                workload of the Executive. The term "Involuntary Termination"
                does not include termination for Cause or termination of
                employment due to retirement, death, Disability or suspension or
                temporary or permanent prohibition from participation in the
                conduct of the Bank's affairs under Section 8 of the Federal
                Deposit Insurance Act.

        (I)     MONTHLY BENEFIT shall mean the Average Compensation multiplied
                by the Accrued Benefit Percentage and then divided by twelve
                (12), calculated at the Benefit Commencement Date.

        (J)     NORMAL RETIREMENT DATE shall mean the date the Executive attains
                age 55.

        (K)     SEPARATION FROM SERVICE shall mean the date of cessation of the
                employment relationship (other than an approved leave of
                absence) between the Executive and the Bank and its affiliates
                and subsidiaries (including any successor in interest, if
                applicable), and shall be construed to comply with Code Section
                409A and Proposed Treasury Regulations Section 1.409A-1(h).

        (L)     SPECIFIED EMPLOYEE shall mean a key employee of the Bank within
                the meaning of Code Section 416(i) without regard to paragraph 5
                thereof, determined in accordance with Code Section 409A and
                Proposed Treasury Regulations Section 1.409A-1(i).


                                       3
<PAGE>

2.      PAYMENT OF BENEFITS.

        (A)     NORMAL BENEFIT.

                If the Executive is living on the Benefit Commencement Date, the
                Bank shall pay the Monthly Benefit to him on such date and on
                the first business day of each calendar month thereafter for a
                total of 180 months (i.e., monthly payments for 15 years),
                regardless of whether the Executive has experienced a Separation
                from Service; provided however, that, if the Executive has
                experienced a Separation from Service, then, to the extent
                necessary to comply with Code Section 409A and the regulations
                thereunder, such payments shall not commence until the first day
                of the seventh month following the date of the Executive's
                Separation from Service if the Executive is a Specified Employee
                on his date of Separation from Service.

        (B)     DEATH BENEFIT.

                        (i)     DEATH DURING OR AFTER SERVICE. If the Executive
                                dies prior to the Normal Retirement Date, the
                                Bank shall pay to the beneficiary designated on
                                Exhibit A, using an Accrued Benefit Percentage
                                of 60%, the Monthly Benefit commencing on the
                                first business day of the month following what
                                would have been the Executive's Normal
                                Retirement Date and on the first business day of
                                each calendar month thereafter for a period of
                                180 months. The Average Compensation calculation
                                shall assume that the Executive's compensation
                                increased by 3% for each full calendar year that
                                occurs prior to what would have been his 55th
                                birthday.

                        (ii)    DEATH DURING BENEFIT PERIOD. If the Executive
                                dies on or after the Benefit Commencement Date,
                                the Bank shall continue to make the remaining
                                monthly payments due to the Executive to the
                                beneficiary designated by the Executive on
                                Exhibit A.

        (C)     DISABILITY BENEFIT.

                If the Executive becomes Disabled before the Normal Retirement
                Date, the Bank shall pay the Monthly Benefit to him, using an
                Accrued Benefit Percentage of not less than 60%, starting on the
                first business day of the calendar month following the date on
                which the Executive became Disabled and on the first business
                day of each calendar month thereafter for a total of 180 months
                (i.e., monthly payments for 15 years). If the Executive dies
                after becoming entitled to Disability benefits, the Bank shall
                continue to make the remaining monthly payments due to the
                Executive to the beneficiary designated by the Executive on
                Exhibit A.

        (D)     SEPARATION FROM SERVICE BENEFIT. In the event the Executive
                incurs a Separation from Service due to an Involuntary
                Termination before the Normal Retirement Date, the Bank shall
                pay the Monthly Benefit to the Executive, using an Accrued
                Benefit Percentage of not less than 60%, commencing on the first
                business day of the month following the Separation from Service


                                       4
<PAGE>

                and on the first business day of each calendar month thereafter
                for a total of 180 months; (i) provided, however, that in the
                event of Separation from Service due to Cause, except as may be
                prohibited by federal law, the Executive shall only be entitled
                to the Monthly Benefit calculated at the time of his Separation
                from Service with payment commencing on the first business day
                of the month following the Separation from Service and on the
                first business date of each calendar month thereafter for a
                total of 180 months; and (ii) provided, further, that, to the
                extent necessary to comply with Code Section 409A and the
                regulations thereunder, such payments shall not commence until
                the first day of the seventh month following the date of the
                Executive's Separation from Service if the Executive is a
                Specified Employee on his date of Separation from Service.

        (E)     CHANGE IN CONTROL BENEFIT. Subject to Section 5, if a Change in
                Control occurs before the Normal Retirement Date, then, within
                30 calendar days of a Change in Control, the Bank shall pay the
                Executive a lump sum equal to the present value of the Monthly
                Benefit that would otherwise be paid to the Executive hereunder,
                using an Accrued Benefit Percentage of not less than 60%,
                regardless of whether the Executive has experienced a Separation
                from Service; provided however, that, if the Executive has
                experienced a Separation from Service, then, to the extent
                necessary to comply with Code Section 409A and the regulations
                thereunder, such payments shall not be made until the first day
                of the seventh month following the date of the Executive's
                Separation from Service if the Executive is a Specified Employee
                on his date of Separation from Service.

3.      REQUIRED PROVISIONS.

        (a)     The Bank may terminate Executive's employment at any time, but
any termination by the Bank other than Separation from Service for Cause as
defined above shall not prejudice Executive's right to compensation or other
benefits under this Agreement. Executive shall have no right to receive
compensation or other benefits for any period after Separation from Service for
Cause.

        (b)     If Executive is suspended from office and/or temporarily
prohibited from participating in the conduct of the Bank's affairs by a notice
served under Section 8(e)(3) [12 USC ss.1818(e)(3)] or 8(g)(1) [12 USC
ss.1818(g)(1)] of the Federal Deposit Insurance Act (the "FDI Act"), the Bank's
obligations under this Agreement shall be suspended as of the date of service,
unless stayed by appropriate proceedings. If the charges in the notice are
dismissed, the Bank may in its discretion (i) pay Executive all or part of the
compensation withheld while its contract obligations were suspended and (ii)
reinstate (in whole or in part) any of its obligations which were suspended.

        (c)     If Executive is removed and/or permanently prohibited from
participating in the conduct of the Bank's affairs by an order issued under
Section 8(e)(4) [12 USC ss.1818(e)(4)] or 8(g)(1) [12 USC ss.1818(g)(1)] of the
FDI Act, all obligations of the Bank under this Agreement shall terminate as of
the effective date of the order, but vested rights of the contracting parties
shall not be affected.


                                       5
<PAGE>

        (d)     If the Bank is in default as defined in Section 3(x)(1) [12 USC
ss.1813(x)(1)] of the FDI Act, all obligations of the Bank under this Agreement
shall terminate as of the date of default, but this paragraph shall not affect
any vested rights of the contracting parties.

        (e)     All obligations under this Agreement shall be terminated, except
to the extent determined that continuation of this Agreement is necessary for
the continued operation of the Bank, (i) by the Director of the Office of Thrift
Supervision ("OTS") or his or her designee, at the time the FDIC enters into an
agreement to provide assistance to or on behalf of the Bank under the authority
contained in Section 13(c) [12 USC ss.1823(c)] of the FDI Act; or (ii) by the
Director or his or her designee at the time the Director or his or her designee
approves a supervisory merger to resolve problems related to operation of the
Bank or when the Bank is determined by the Director to be in an unsafe or
unsound condition. Any rights of the parties that have already vested, however,
shall not be affected by such action.

        (f)     Notwithstanding anything herein contained to the contrary, any
payments to Executive by the Company, whether pursuant to this Agreement or
otherwise, are subject to and conditioned upon their compliance with Section
18(k) of the FDI Act, 12 U.S.C. Section 1828(k), and the regulations promulgated
thereunder in 12 C.F.R. Part 359.

4.      CLAIMS. In the event a claim for benefits is wholly or partially denied
        under this Agreement, the Executive or any other person claiming
        benefits under this Agreement (a "Claimant") shall be given notice in
        writing within 30 calendar days after the Administrator's receipt of the
        claim. For good cause shown, the Administrator may extend this period
        for an additional 30 calendar days. Any denial must specifically set
        forth the reasons for the denial and any additional information
        necessary to rescind such denial. The Claimant shall have the right to
        seek a review of the denial by filing a written request with the
        Administrator within 60 calendar days of receipt of the denial. Such
        request may be supported by such documentation and evidence deemed
        relevant by the Claimant. Following receipt of this information, the
        Administrator shall make a final determination and notify the Claimant
        in writing within 60 calendar days of the Administrator's receipt of the
        request for review together with the specific reasons for the decision.

5.      GENERAL ASSETS AND FUNDING. The amounts payable under this Agreement are
        payable from the general assets of the Bank and no special fund or
        arrangement is intended to be established hereby nor shall the Bank be
        required to earmark, place in trust or otherwise segregate assets with
        respect to this Agreement or any benefits hereunder. The Administrator
        reserves the right to determine how the Bank will fund its obligation
        undertaken by this Agreement. Should the Administrator elect to purchase
        assets relating to this Agreement, in whole or in part, through the
        medium of life insurance or annuities, or both, the Bank shall be the
        owner and beneficiary of each such policy unless otherwise provided by
        this Agreement. Bank reserves the absolute right, in its sole
        discretion, to terminate such life insurance or annuities, as well as
        any other investment program, at any time, in whole or in part unless
        otherwise provided by this Agreement. Such termination shall in no way


                                       6
<PAGE>

        affect the Bank's obligation to pay the Executive the benefits as
        provided in this Agreement. At no time shall the Executive be deemed to
        have any right, title, or interest in or to any specific asset or assets
        of the Bank, including but not by way of restriction, any insurance or
        annuity contract and contracts or the proceeds therefrom.

6.      CERTAIN REDUCTIONS. Notwithstanding any other provision of this
        Agreement, if the value and amounts of benefits under this Agreement,
        together with any other amounts and the value of benefits received or to
        be received by the Executive in connection with a Change in Control
        would cause any amount to be nondeductible for federal income tax
        purposes by the Bank or the consolidated group of which the Bank is a
        member pursuant to Section 280G of the Code, then amounts and benefits
        under this Agreement shall be reduced (not less than zero) to the extent
        necessary so as to maximize amounts and the value of benefits to the
        Employee without causing any amount to become nondeductible by Bank
        pursuant to or by reason of such Section 280G. The Employee shall
        determine the allocation of such reduction among payments and benefits
        to the Employee.

7.      BENEFICIARY DESIGNATIONS. The Executive shall designate a beneficiary by
        filing with Bank a written designation of beneficiary on a form
        substantially similar to the form attached as Exhibit A. The Executive
        may revoke or modify the designation at any time by filing a new
        designation. However, designations will only be effective if signed by
        the Executive and accepted by the Bank during the Executive's lifetime.
        The Executive's beneficiary designation shall be deemed automatically
        revoked if the beneficiary predeceases the Executive, or if the
        Executive names a spouse as beneficiary and the marriage is subsequently
        dissolved. If the Executive dies without a valid beneficiary
        designation, all payments shall be made to the Executive's surviving
        spouse, if any, and if none, to the Executive's surviving children and
        the descendants of any deceased child by right of representation, and if
        no children or descendants survive, to the Executive's estate.

        If a benefit is payable to a minor, to a person declared incompetent, or
        to a person incapable of handling the disposition of his or her
        property, the Bank may pay such benefit to the guardian, legal
        representative or person having the care or custody of such minor,
        incompetent person or incapable person, or to a custodian selected by
        the Bank under the Georgia Uniform Transfers to Minors Act for the
        benefit of such minor. The Bank may require proof of incompetency,
        minority or guardianship as it may deem appropriate prior to
        distribution of the benefit. Such distribution shall completely
        discharge the Bank from all liability with respect to such benefit.

8.      AMENDMENT AND TERMINATION.

        (a)     AMENDMENT. The Bank may at any time amend the Agreement in whole
                or in part, provided, however, that no amendment shall decrease
                or restrict the amount accrued to the date of amendment.

        (b)     TERMINATION. The Bank may at any time partially or completely
                terminate the Agreement, if, in its judgment, the tax,
                accounting, or other effects of the continuance of the
                Agreement, or potential payments thereunder, would not be in the
                best interests of the Bank.

                                       7
<PAGE>

                (i)     PARTIAL TERMINATION. In the event of a partial
        termination, the Agreement shall continue to operate and be effective
        with regard to benefits accrued prior to the effective date of such
        partial termination, but no further benefits shall accrue after the date
        of such partial termination.

                (ii)    COMPLETE TERMINATION. Subject to the requirements of
        Code Section 409A, in the event of complete termination, the Agreement
        shall cease to operate and the Bank shall pay the Executive his Account
        as if he had terminated service as of the effective date of the complete
        termination. Such complete termination of the Agreement shall occur only
        under the following circumstances and conditions.

                        (A)     The Bank may terminate the Agreement within 12
                months of a corporate dissolution taxed under Code section 331,
                or with approval of a bankruptcy court pursuant to 11 U.S.C.
                ss.503(b)(1)(A), provided that the amounts accrued under the
                Agreement are included in the Executive's gross income in the
                latest of (i) the calendar year in which the Agreement
                terminates; (ii) the calendar year in which the amount is no
                longer subject to a substantial risk of forfeiture; or (iii) the
                first calendar year in which the payment is administratively
                practicable.

                        (B)     The Bank may terminate the Agreement within the
                30 days preceding a Change in Control (but not following a
                Change in Control), provided that the Agreement shall only be
                treated as terminated if all substantially similar arrangements
                sponsored by the Bank are terminated so that the Executive and
                all participants under substantially similar arrangements are
                required to receive all amounts of compensation deferred under
                the terminated arrangements within 12 months of the date of the
                termination of the arrangements.

                        (C)     The Bank may terminate the Agreement provided
                that (i) all arrangements sponsored by the Bank that would be
                aggregated with this Agreement under Proposed Treasury
                regulations section 1.409A-1(c) if any individual; covered by
                this Agreement was also covered by any of those other
                arrangements are also terminated; (ii) no payments other than
                payments that would be payable under the terms of the
                arrangement if the termination had not occurred are made within
                12 months of the termination of the arrangement; (iii) all
                payments are made within 24 months of the termination of the
                arrangements; and (iv) the Bank does not adopt a new arrangement
                that would be aggregated with any terminated arrangement under
                Proposed Treasury regulations section 1.409A-1(c) if the same
                individual participated in both arrangements, at any time within
                five years following the date of termination of the arrangement.

                        (D)     The Bank may terminate the Agreement pursuant to
                such other terms and conditions as the Internal Revenue Service
                may permit from time to time.


                                       8
<PAGE>

9.       MISCELLANEOUS.

        (A)     WITHHOLDING. To the extent amounts payable under this Agreement
                are determined by the Administrator, in good faith, to be
                subject to federal, state or local income tax, the Bank may
                withhold from each such payment an amount necessary to meet the
                Bank's obligation to withhold amounts under the applicable
                federal, state or local law.

        (B)     GOVERNING LAW. This Agreement shall be construed under the laws
                of the State of Georgia, except to the extent that federal law
                applies.

        (C)     FUTURE EMPLOYMENT. This Agreement shall not be construed as
                providing the Executive the right to be continued in the employ
                of the Bank or its affiliates or subsidiaries.

        (D)     NO PLEDGE OR ATTACHMENT. No benefit which is or may become
                payable under this Agreement shall be subject to any
                anticipation, alienation, sale, transfer, pledge, encumbrance or
                hypothecation or subject to any attachment, levy or similar
                process and any attempt to effect any such action shall be null
                and void.

        (E)     SUCCESSORS AND ASSIGNS. This Agreement and the obligations of
                the Bank herein shall be binding upon the successors and assigns
                of the Bank. This Agreement may not be assigned by the Bank
                without the prior written consent of the Executive or any other
                beneficiary receiving payments under this Agreement.

        (F)     PARTICIPATION IN PLANS. Nothing contained in this Agreement
                shall be construed to alter, abridge, or in any manner affect
                the rights and privileges of the Executive to participate in and
                be covered by any pension, profit sharing, group insurance,
                bonus, incentive, or other employee plans which the Bank or its
                affiliates or subsidiaries may now or hereafter have.

        (G)     NOTICES. Any notices under this Agreement shall be provided to
                the Executive at his last address on file with the Administrator
                and shall be provided to the Administrator in care of President,
                Atlantic Coast Federal, 505 Haines Avenue, Waycross, Georgia
                31501.

        (H)     HEADINGS. Headings of sections herein are inserted for
                convenience of reference. They are not to be considered in the
                construction of this Agreement.

        (I)     SAVINGS CLAUSE. If any provision of this Agreement shall be for
                any reason invalid or unenforceable, the remaining provisions
                shall be carried into effect.

        (J)     ENTIRE AGREEMENT. This Agreement constitutes the entire
                agreement between the Bank and the Executive as to the subject
                matter hereof. No rights are granted to the Executive be virtue
                of this Agreement other than as specifically set forth herein.


                                       9
<PAGE>

        (K)     SUICIDE. No benefits shall be payable if the Executive commits
                suicide within two (2) years after the date of this Agreement,
                or if the Executive has made any material misstatement of fact
                on any application for life insurance purchased by the Bank.

        (L)     TOP HAT AGREEMENT. For purposes of the Internal Revenue Code,
                the Bank intends this Agreement to be an unfunded, unsecured
                promise to pay on the part of the Bank. For purposes of ERISA,
                The Bank intends this Agreement to be an unfunded obligation
                solely for the benefit of the Executive for the purpose of
                qualifying this Agreement for the "top hat" exception under
                sections 201(2), 301(a)(3) and 401(a) of ERISA.

        The parties have caused this Agreement to be executed and delivered as
of the 8th day of November, 2006.


                                        ATLANTIC COAST FEDERAL


                                By:    /s/ Forrest W. Sweat, Jr.
                                       -----------------------------------
                                       Name:  Forrest W. Sweat, Jr.
                                       Title: Vice-Chairman/BOD


                                       EXECUTIVE


                                       /s/ Jon C. Parker, Sr.
                                       -----------------------------------
                                       Jon C. Parker, Sr.




                                       10