<PAGE>

                                                                   EXHIBIT 10.2

                               GUARANTEE AGREEMENT

                                     BETWEEN

                    COMMUNITY FINANCIAL HOLDING COMPANY, INC.
                                  AS GUARANTOR,

                                       AND

                            WILMINGTON TRUST COMPANY
                              AS GUARANTEE TRUSTEE,

                           DATED AS OF MARCH 26, 2004

<PAGE>

                              CFHC CAPITAL TRUST I

            CERTAIN SECTIONS OF THIS GUARANTEE AGREEMENT RELATING TO
                         SECTIONS 310 THROUGH 318 OF THE
                          TRUST INDENTURE ACT OF 1939:

<TABLE>
<CAPTION>
TRUST INDENTURE                                     GUARANTEE AGREEMENT
  ACT SECTION                                             SECTION
  -----------                                             -------
<S>                                                 <C>
Section 310   (a)(1).............................      4.1 (a)
              (a)(2).............................      4.1 (a)
              (a)(3).............................      Not Applicable
              (a)(4).............................      Not Applicable
              (b)................................      2.8, 4.1 (c)
Section 311   (a)................................      Not Applicable
              (b)................................      Not Applicable
Section 312   (a)................................      2.2 (a)
              (b)................................      2.2 (b)
              (c)................................      Not Applicable
Section 313   (a)................................      2.3
              (a)(4).............................      2.3
              (b)................................      2.3
              (c)................................      2.3
              (d)................................      2.3
Section 314   (a)................................      2.4
              (b)................................      2.4
              (c)(1).............................      2.5
              (c)(2).............................      2.5
              (c)(3).............................      2.5
              (e)................................      1.1, 2.5, 3.2
Section 315   (a)................................      3.1 (d)
              (b)................................      2.7
              (c)................................      3.1 (c)
              (d)................................      3.1 (d)
              (e)................................      Not Applicable
Section 316   (a)................................      1.1, 2.6, 5.4
              (a)(1)(A)..........................      5.4
              (a)(1)(B)..........................      5.4
              (a)(2).............................      Not Applicable
              (b)................................      5.3
              (c)................................      Not Applicable
Section 317   (a)(1).............................      Not Applicable
              (a)(2).............................      Not Applicable
              (b)................................      Not Applicable
Section 318   (a)................................      2.1
</TABLE>

Note: This reconciliation and tie shall not, for any purpose, be deemed to be a
part of the Guarantee Agreement.

                                      -i-

<PAGE>

                               TABLE OF CONTENTS

<TABLE>
<S>                                                                                        <C>
ARTICLE I.  DEFINITIONS................................................................     1
     SECTION 1.1.    Definitions.......................................................     1
ARTICLE II.  TRUST INDENTURE ACT.......................................................     4
     SECTION 2.1.    Trust Indenture Act; Application..................................     4
     SECTION 2.2.    List of Holders...................................................     4
     SECTION 2.3.    Reports by the Guarantee Trustee..................................     5
     SECTION 2.4.    Periodic Reports to the Guarantee Trustee.........................     5
     SECTION 2.5.    Evidence of Compliance with Conditions Precedent..................     5
     SECTION 2.6.    Events of Default; Waiver.........................................     5
     SECTION 2.7.    Event of Default; Notice..........................................     6
     SECTION 2.8.    Conflicting Interests.............................................     6
ARTICLE III.  POWERS, DUTIES AND RIGHTS OF THE GUARANTEE TRUSTEE.......................     6
     SECTION 3.1.    Powers and Duties of the Guarantee Trustee........................     6
     SECTION 3.2.    Certain Rights of Guarantee Trustee...............................     7
     SECTION 3.3.    Indemnity.........................................................     9
     SECTION 3.4.    Expenses..........................................................     9
ARTICLE IV.  GUARANTEE TRUSTEE.........................................................     9
     SECTION 4.1.    Guarantee Trustee; Eligibility....................................     9
     SECTION 4.2.    Appointment, Removal and Resignation of the Guarantee Trustee.....    10
ARTICLE V. GUARANTEE...................................................................    11
     SECTION 5.1.    Guarantee.........................................................    11
     SECTION 5.2.    Waiver of Notice and Demand.......................................    11
     SECTION 5.3.    Obligations Not Affected..........................................    11
     SECTION 5.4.    Rights of Holders.................................................    12
     SECTION 5.5.    Guarantee of Payment..............................................    12
     SECTION 5.6.    Subrogation.......................................................    12
     SECTION 5.7.    Independent Obligations...........................................    13
ARTICLE VI.  COVENANTS AND SUBORDINATION...............................................    13
     SECTION 6.1.    Subordination.....................................................    13
     SECTION 6.2.    Pari Passu Guarantees.............................................    13
ARTICLE VII.  TERMINATION..............................................................    13
     SECTION 7.1.    Termination.......................................................    13
ARTICLE VIII.  MISCELLANEOUS...........................................................    14
     SECTION 8.1.    Successors and Assigns............................................    14
     SECTION 8.2.    Amendments........................................................    14
     SECTION 8.3.    Notices...........................................................    14
     SECTION 8.4.    Benefit...........................................................    15
     SECTION 8.5.    Interpretation....................................................    15
     SECTION 8.6.    Governing Law.....................................................    16
     SECTION 8.7.    Counterparts......................................................    16
</TABLE>

                                      -ii-
<PAGE>

                               GUARANTEE AGREEMENT

            THIS GUARANTEE AGREEMENT, dated as of March 26, 2004, is executed
and delivered by COMMUNITY FINANCIAL HOLDING COMPANY, INC., a Georgia
corporation (the "Guarantor"), having its principal office at 2775 Buford
Highway, Duluth, Georgia 30096, and WILMINGTON TRUST COMPANY, a Delaware banking
corporation, as trustee (the "Guarantee Trustee"), for the benefit of the
Holders (as defined herein) from time to time of the Capital Securities (as
defined herein) of CFHC CAPITAL TRUST I, a Delaware statutory trust (the "Issuer
Trust").

            Pursuant to an Amended and Restated Trust Agreement (the "Trust
Agreement"), dated of even date herewith, among Community Financial Holding
Company, Inc., as Depositor, Wilmington Trust Company, as Property Trustee (the
"Property Trustee"), Wilmington Trust Company, as Delaware Trustee (the
"Delaware Trustee", and together with the Property Trustee, collectively, the
"Issuer Trustees"), the Administrators named therein and the Holders from time
to time of undivided beneficial interests in the assets of the Issuer Trust, the
Issuer Trust is issuing up to $5,500,000 aggregate Liquidation Amount (as
defined herein) of its Floating Rate Capital Securities (the "Capital
Securities"), representing preferred undivided beneficial interests in the
assets of the Issuer Trust and having the terms set forth in the Trust
Agreement;

            The Capital Securities will be issued by the Issuer Trust and the
proceeds thereof, together with the proceeds from the issuance of the Issuer
Trust's Common Securities (the "Common Securities"), representing common
undivided beneficial interests in the assets of the Issuer Trust of the
Guarantor, will be used to purchase the Junior Subordinated Debentures due March
31, 2034 (the "Junior Subordinated Debentures") of the Guarantor, which will be
deposited with Wilmington Trust Company, as Property Trustee under the Trust
Agreement, as trust assets; and

            As an inducement to the Holders to purchase the Capital Securities,
the Guarantor is willing to irrevocably and unconditionally agree, to the extent
set forth herein, to pay to the Holders of the Capital Securities the Guarantee
Payments (as defined herein) and to make certain other payments on the terms and
conditions set forth herein.

            NOW, THEREFORE, in consideration of the purchase of the Capital
Securities by each Holder, which purchase the Guarantor hereby acknowledges
shall benefit the Guarantor, the Guarantor, intending to be legally bound
hereby, executes and delivers this Guarantee Agreement for the benefit of the
Holders from time to time of the Capital Securities.

                             ARTICLE I. DEFINITIONS

      SECTION 1.1. Definitions.

            As used in this Guarantee Agreement, the terms set forth below shall
have the following meanings. Capitalized terms used but not otherwise defined
herein shall have the meanings assigned to such terms in the Trust Agreement.

<PAGE>

            "Affiliate" of any specified Person means any other Person directly
or indirectly controlling or controlled by or under direct or indirect common
control with such specified Person. For the purposes of this definition,
"control" when used with respect to any specified Person means the power to
direct the management and policies of such Person, directly or indirectly,
whether through the ownership of voting securities, by contract or otherwise;
and the terms "controlling" and "controlled" have meanings correlative to the
foregoing.

            "Event of Default" means (i) a default by the Guarantor in any of
its payment obligations under this Guarantee Agreement, or (ii) a default by the
Guarantor in any other obligation hereunder that remains unremedied for 30 days.

            "Guarantee Agreement" means this Guarantee Agreement, as modified,
amended or supplemented from time to time.

            "Guarantee Payments" means the following payments or distributions,
without duplication, with respect to the Capital Securities, to the extent not
paid or made by or on behalf of the Issuer Trust: (i) any accumulated and unpaid
Distributions (as defined in the Trust Agreement) required to be paid on the
Capital Securities, to the extent the Issuer Trust shall have funds on hand
available therefor at such time, (ii) the Redemption Price, with respect to the
Capital Securities called for redemption by the Issuer Trust to the extent that
the Issuer Trust shall have funds on hand available therefor at such time, and
(iii) upon a voluntary or involuntary dissolution, termination, winding-up or
liquidation of the Issuer Trust, unless the Junior Subordinated Debentures are
distributed to the Holders, the lesser of (a) the aggregate of the Liquidation
Amount of all outstanding Trust Securities and all accumulated and unpaid
Distributions to the date of payment to the extent the Issuer Trust shall have
funds on hand available to make such payment at such time and (b) the amount of
assets of the Issuer Trust remaining available for distribution to Holders on
liquidation of the Issuer Trust (in either case, the "Liquidation
Distribution").

            "Guarantee Trustee" means Wilmington Trust Company, until a
Successor Guarantee Trustee has been appointed and has accepted such appointment
pursuant to the terms of this Guarantee Agreement and thereafter means each such
Successor Guarantee Trustee.

            "Guarantor" shall have the meaning specified in the first paragraph
of this Guarantee Agreement.

            "Holder" means any holder, as registered on the books and records of
the Issuer Trust, of any Capital Securities; provided, however, that, in
determining whether the holders of the requisite percentage of Capital
Securities have given any request, notice, consent or waiver hereunder, "Holder"
shall not include the Guarantor, the Guarantee Trustee, or any Affiliate of the
Guarantor or the Guarantee Trustee.

            "Indenture" means the Junior Subordinated Indenture, dated of even
date herewith, between Community Financial Holding Company, Inc. and Wilmington
Trust Company, as trustee, as it may be modified, amended or supplemented from
time to time.

                                     - 2 -
<PAGE>

            "Issuer Trust" shall have the meaning specified in the first
paragraph of this Guarantee Agreement.

            "Like Amount" means (a) with respect to a redemption of Trust
Securities, Trust Securities having a Liquidation Amount equal to that portion
of the principal amount of Junior Subordinated Debentures to be
contemporaneously redeemed in accordance with the Indenture, allocated to the
Common Securities and to the Capital Securities based upon the relative
Liquidation Amounts of such classes and (b) with respect to a distribution of
Junior Subordinated Debentures to Holders of Trust Securities in connection with
a dissolution or liquidation of the Issuer Trust, Junior Subordinated Debentures
having a principal amount equal to the Liquidation Amount of the Trust
Securities of the Holder to whom such Junior Subordinated Debentures are
distributed.

            "Liquidation Amount" means the stated amount of $50,000 per Capital
Security and $1,000 per Common Security.

            "Majority in Liquidation Amount of the Capital Securities" means,
except as provided by the Trust Indenture Act, Capital Securities representing
more than 50% of the aggregate Liquidation Amount of all then outstanding
Capital Securities issued by the Issuer Trust.

            "Officers' Certificate" means a certificate signed by the Chairman
of the Board, Vice Chairman of the Board, Chief Executive Officer, President,
Executive Vice President or a Senior Vice President or Vice President, and by
the Treasurer, an Assistant Treasurer, the Chief Financial Officer, the
Secretary or an Assistant Secretary of the Guarantor, and delivered to the
Guarantee Trustee. Any Officers' Certificate delivered with respect to
compliance with a condition or covenant provided for in this Guarantee Agreement
shall include:

            (a)   a statement by each officer signing the Officers' Certificate
that such officer has read the covenant or condition and the definitions
relating thereto;

            (b)   a brief statement of the nature and scope of the examination
or investigation undertaken by such officer in rendering the Officers'
Certificate;

            (c)   a statement that such officer has made such examination or
investigation as, in such officer's opinion, is necessary to enable such officer
to express an informed opinion as to whether or not such covenant or condition
has been complied with; and

            (d)   a statement as to whether, in the opinion of such officer,
such condition or covenant has been complied with.

            "Person" means a legal person, including any individual,
corporation, estate, partnership, joint venture, association, joint stock
company, limited liability company, trust, unincorporated association, or
government or any agency or political subdivision thereof, or any other entity
of whatever nature.

                                     - 3 -
<PAGE>

            "Redemption Date" means, with respect to any Capital Security to be
redeemed, the date fixed for such redemption by or pursuant to the Trust
Agreement; provided that each Debenture Redemption Date and the stated maturity
of the Junior Subordinated Debentures shall be a Redemption Date for a Like
Amount of Capital Securities, including, but not limited to any date of
redemption pursuant to the occurrence of any Special Event.

            "Redemption Price" shall have the meaning specified in the Trust
Agreement.

            "Responsible Officer" means, when used with respect to the Guarantee
Trustee, any officer assigned to the Corporate Trust Office, including any
managing director, vice president, principal, assistant vice president,
assistant treasurer, assistant secretary or any other officer of the Guarantee
Trustee customarily performing functions similar to those performed by any of
the above designated officers and having direct responsibility for the
administration of this Guarantee Agreement, and also, with respect to a
particular matter, any other officer of the Guarantee Trustee to whom such
matter is referred because of such officer's knowledge of and familiarity with
the particular subject.

            "Senior Indebtedness" shall have the meaning specified in the
Indenture.

            "Successor Guarantee Trustee" means a successor Guarantee Trustee
possessing the qualifications to act as Guarantee Trustee under Article IV
hereof.

            "Trust Agreement" has the meaning specified in the recitals hereto.

            "Trust Indenture Act" means the Trust Indenture Act of 1939 (15
U.S.C. Sections 77aaa-77bbbb), as amended.

            "Trust Securities" means the Common Securities and the Capital
Securities.

                         ARTICLE II. TRUST INDENTURE ACT

      SECTION 2.1. Trust Indenture Act; Application.

            If any provision hereof limits, qualifies or conflicts with a
provision of the Trust Indenture Act that is required under such Act to be a
part of and govern this Guarantee Agreement, the provision of the Trust
Indenture Act shall control. If any provision of this Guarantee Agreement
modifies or excludes any provision of the Trust Indenture Act that may be so
modified or excluded, the latter provision shall be deemed to apply to this
Guarantee Agreement as so modified or excluded, as the case may be.

      SECTION 2.2. List of Holders.

            (a)   The Guarantor will furnish or cause to be furnished to the
Guarantee Trustee a list of Holders at the following times:

                                     - 4 -
<PAGE>

                  (i)   quarterly, not more than 15 days after the last day of
February, May, August and November, in each year, a list, in such form as the
Guarantee Trustee may reasonably require, of the names and addresses of the
Holders as of the last day of February, May, August and November, as applicable;
and

                  (ii)  at such other times as the Guarantee Trustee may request
in writing, within 30 days after the receipt by the Guarantor of any such
request, a list of similar form and content as of a date not more than 15 days
prior to the time such list is furnished.

            (b)   The Guarantee Trustee shall comply with the requirements of
Section 312(b) of the Trust Indenture Act.

      SECTION 2.3. Reports by the Guarantee Trustee.

      Not later than January 31 of each year, commencing January 31, 2005, the
Guarantee Trustee shall provide to the Holders such reports, if any, as are
required by Section 313 of the Trust Indenture Act in the form and in the manner
provided by Section 313 of the Trust Indenture Act. If this Guarantee Agreement
shall have been qualified under the Trust Indenture Act, the Guarantee Trustee
shall also comply with the requirements of Section 313(d) of the Trust Indenture
Act.

      SECTION 2.4. Periodic Reports to the Guarantee Trustee.

      The Guarantor shall provide to the Guarantee Trustee and the Holders such
documents, reports and information, if any, as required by Section 314 of the
Trust Indenture Act and the compliance certificate required by Section 314 of
the Trust Indenture Act, in the form, in the manner and at the times required by
Section 314 of the Trust Indenture Act, provided that such documents, reports
and information shall be required to be provided to the Securities and Exchange
Commission only if this Guarantee Agreement shall have been qualified under the
Trust Indenture Act.

      SECTION 2.5. Evidence of Compliance with Conditions Precedent.

      The Guarantor shall provide to the Guarantee Trustee such evidence of
compliance with such conditions precedent, if any, provided for in this
Guarantee Agreement that relate to any of the matters set forth in Section
314(c) of the Trust Indenture Act. Any certificate or opinion required to be
given by an officer pursuant to Section 314(c)(1) may be given in the form of an
Officers' Certificate.

      SECTION 2.6. Events of Default; Waiver.

      The Holders of a Majority in Liquidation Amount of the Capital Securities
may, by vote, on behalf of the Holders, waive any past Event of Default and its
consequences. Upon such waiver, any such Event of Default shall cease to exist,
and any Event of Default arising therefrom shall be deemed to have been cured,
for every purpose of this Guarantee Agreement, but no such waiver shall extend
to any subsequent or other default or Event of Default or impair any right
consequent therefrom.

                                     - 5 -
<PAGE>

      SECTION 2.7. Event of Default; Notice.

      (a)   The Guarantee Trustee shall, within 90 days after the occurrence of
an Event of Default, transmit by mail, first class postage prepaid, to the
Holders, notice of all Events of Default known to the Guarantee Trustee, unless
such Events of Default have been cured or waived before the giving of such
notice; provided that, except in the case of a default in the payment of a
Guarantee Payment, the Guarantee Trustee shall be protected in withholding such
notice if and so long as the Board of Directors, the executive committee or a
trust committee of directors and/or Responsible Officers of the Guarantee
Trustee in good faith determines that the withholding of such notice is in the
interests of the Holders.

      (b)   The Guarantee Trustee shall not be deemed to have knowledge of any
Event of Default unless a Responsible Officer charged with the administration of
this Guarantee Agreement shall have received written notice of such Event of
Default.

      SECTION 2.8. Conflicting Interests.

      The Trust Agreement shall be deemed to be specifically described in this
Guarantee Agreement for the purposes of clause (i) of the first proviso
contained in Section 310(b) of the Trust Indenture Act.

         ARTICLE III. POWERS, DUTIES AND RIGHTS OF THE GUARANTEE TRUSTEE

      SECTION 3.1. Powers and Duties of the Guarantee Trustee.

      (a)   This Guarantee Agreement shall be held by the Guarantee Trustee for
the benefit of the Holders, and the Guarantee Trustee shall not transfer this
Guarantee Agreement to any Person except a Holder exercising his or her rights
pursuant to Section 5.4(iv) or to a Successor Guarantee Trustee on acceptance by
such Successor Guarantee Trustee of its appointment to act as Successor
Guarantee Trustee hereunder. The right, title and interest of the Guarantee
Trustee, as such, hereunder shall automatically vest in any Successor Guarantee
Trustee upon acceptance by such Successor Guarantee Trustee of its appointment
hereunder, and such vesting and cessation of title shall be effective whether or
not conveyancing documents have been executed and delivered pursuant to the
appointment of such Successor Guarantee Trustee.

      (b)   If an Event of Default has occurred and is continuing, the Guarantee
Trustee shall enforce this Guarantee Agreement for the benefit of the Holders.

      (c)   The Guarantee Trustee, before the occurrence of any Event of Default
and after the cure or waiver of all Events of Default that may have occurred,
shall be obligated to perform only such duties as are specifically set forth in
this Guarantee Agreement (including pursuant to Section 2.1), and no implied
covenants shall be read into this Guarantee Agreement against the Guarantee
Trustee. If an Event of Default has occurred (that has not been cured or waived
pursuant to Section 2.6), the Guarantee Trustee shall exercise such of the
rights and powers vested in it by this

                                     - 6 -
<PAGE>

Guarantee Agreement, and use the same degree of care and skill in its exercise
thereof, as a prudent person would exercise or use under the circumstances in
the conduct of his or her own affairs.

      (d)   No provision of this Guarantee Agreement shall be construed to
relieve the Guarantee Trustee from liability for its own negligent action, its
own negligent failure to act or its own willful misconduct, except that:

            (i)   Prior to the occurrence of any Event of Default and after the
curing or waiving of all such Events of Default that may have occurred:

                  (A)   the duties and obligations of the Guarantee Trustee
shall be determined solely by the express provisions of this Guarantee Agreement
(including pursuant to Section 2.1), and the Guarantee Trustee shall not be
liable except for the performance of such duties and obligations as are
specifically set forth in this Guarantee Agreement (including pursuant to
Section 2.1); and

                  (B)   in the absence of bad faith on the part of the Guarantee
Trustee, the Guarantee Trustee may conclusively rely, as to the truth of the
statements and the correctness of the opinions expressed therein, upon any
certificates or opinions furnished to the Guarantee Trustee and conforming to
the requirements of this Guarantee Agreement; but in the case of any such
certificates or opinions that by any provision hereof or of the Trust Indenture
Act are specifically required to be furnished to the Guarantee Trustee, the
Guarantee Trustee shall be under a duty to examine the same to determine whether
or not they conform to the requirements of this Guarantee Agreement;

            (ii)  The Guarantee Trustee shall not be liable for any error of
judgment made in good faith by a Responsible Officer of the Guarantee Trustee,
unless it shall be proved that the Guarantee Trustee was negligent in
ascertaining the pertinent facts upon which such judgment was made;

            (iii) The Guarantee Trustee shall not be liable with respect to any
action taken or omitted to be taken by it in good faith in accordance with the
direction of the Holders of not less than a Majority in Liquidation Amount of
the Capital Securities relating to the time, method and place of conducting any
proceeding for any remedy available to the Guarantee Trustee, or exercising any
trust or power conferred upon the Guarantee Trustee under this Guarantee
Agreement; and

            (iv)  No provision of this Guarantee Agreement shall require the
Guarantee Trustee to expend or risk its own funds or otherwise incur personal
financial liability in the performance of any of its duties or in the exercise
of any of its rights or powers if the Guarantee Trustee shall have reasonable
grounds for believing that the repayment of such funds or liability is not
assured to it under the terms of this Guarantee Agreement or adequate indemnity
against such risk or liability is not reasonably assured to it.

      SECTION 3.2. Certain Rights of Guarantee Trustee.

      (a)   Subject to the provisions of Section 3.1:

                                     - 7 -
<PAGE>

            (i)   The Guarantee Trustee may conclusively rely and shall be fully
protected in acting or refraining from acting upon any resolution, certificate,
statement, instrument, opinion, report, notice, request, direction, consent,
order, bond, debenture, note, other evidence of indebtedness or other paper or
document reasonably believed by it to be genuine and to have been signed, sent
or presented by the proper party or parties.

            (ii)  Any direction or act of the Guarantor contemplated by this
Guarantee Agreement shall be sufficiently evidenced by an Officers' Certificate
unless otherwise prescribed herein.

            (iii) Whenever, in the administration of this Guarantee Agreement,
the Guarantee Trustee shall deem it desirable that a matter be proved or
established before taking, suffering or omitting to take any action hereunder,
the Guarantee Trustee (unless other evidence is herein specifically prescribed)
may, in the absence of bad faith on its part, request and conclusively rely upon
an Officers' Certificate which, upon receipt of such request from the Guarantee
Trustee, shall be promptly delivered by the Guarantor.

            (iv)  The Guarantee Trustee may consult with legal counsel, and the
advice or written opinion of such legal counsel with respect to legal matters
shall be full and complete authorization and protection in respect of any action
taken, suffered or omitted to be taken by it hereunder in good faith and in
accordance with such advice or opinion. Such legal counsel may be legal counsel
to the Guarantor or any of its Affiliates and may be one of its employees. The
Guarantee Trustee shall have the right at any time to seek instructions
concerning the administration of this Guarantee Agreement from any court of
competent jurisdiction.

            (v)   The Guarantee Trustee shall be under no obligation to exercise
any of the rights or powers vested in it by this Guarantee Agreement at the
request or direction of any Holder, unless such Holder shall have provided to
the Guarantee Trustee such security and indemnity as would satisfy a reasonable
person in the position of the Guarantee Trustee, against the costs, expenses
(including attorneys' fees and expenses) and liabilities that might be incurred
by it in complying with such request or direction, including such reasonable
advances as may be requested by the Guarantee Trustee.

            (vi)  The Guarantee Trustee shall not be bound to make any
investigation into the facts or matters stated in any resolution, certificate,
statement, instrument, opinion, report, notice, request, direction, consent,
order, bond, debenture, note, other evidence of indebtedness or other paper or
document, but the Guarantee Trustee, in its discretion, may make such further
inquiry or investigation into such facts or matters as it may see fit.

            (vii) The Guarantee Trustee may execute any of the trusts or powers
hereunder or perform any duties hereunder either directly or by or through its
agents or attorneys, and the Guarantee Trustee shall not be responsible for any
negligence or willful misconduct on the part of any such agent or attorney
appointed with due care by it hereunder.

            (viii) Whenever in the administration of this Guarantee Agreement
the Guarantee Trustee shall deem it desirable to receive instructions with
respect to enforcing any remedy or right

                                     - 8 -
<PAGE>

or taking any other action hereunder, the Guarantee Trustee (A) may request
instructions from the Holders, (B) may refrain from enforcing such remedy or
right or taking such other action until such instructions are received and (C)
shall be fully protected in acting in accordance with such instructions.

      (b)   No provision of this Guarantee Agreement shall be deemed to impose
any duty or obligation on the Guarantee Trustee to perform any act or acts or
exercise any right, power, duty or obligation conferred or imposed on it in any
jurisdiction in which it shall be illegal, or in which the Guarantee Trustee
shall be unqualified or incompetent in accordance with applicable law, to
perform any such act or acts or to exercise any such right, power, duty or
obligation. No permissive power or authority available to the Guarantee Trustee
shall be construed to be a duty to act in accordance with such power and
authority.

      SECTION 3.3. Indemnity.

      The Guarantor agrees to indemnify the Guarantee Trustee (which for
purposes of this Section 3.3 shall include its officers, directors, employees
and agents) for, and to hold it harmless against, any loss, liability or expense
incurred without negligence, willful misconduct or bad faith on the part of the
Guarantee Trustee, arising out of or in connection with the acceptance or
administration of this Guarantee Agreement, including the costs and expenses of
defending itself against any claim or liability in connection with the exercise
or performance of any of its powers or duties hereunder. The Guarantee Trustee
will not claim or exact any lien or charge on any Guarantee Payments as a result
of any amount due to it under this Guarantee Agreement. The provisions of this
Section 3.3 shall survive the termination of this Guarantee Agreement or the
resignation or removal of the Guarantee Trustee.

      SECTION 3.4. Expenses.

      The Guarantor shall from time to time reimburse the Guarantee Trustee for
its expenses and costs (including reasonable attorneys' or agents' fees)
incurred in connection with the performance of its duties hereunder.

                          ARTICLE IV. GUARANTEE TRUSTEE

      SECTION 4.1. Guarantee Trustee; Eligibility.

      (a)   There shall at all times be a Guarantee Trustee, which shall:

            (i)   not be an Affiliate of the Guarantor; and

            (ii)  be a Person that is eligible pursuant to the Trust Indenture
Act to act as such and has a combined capital and surplus of at least
$50,000,000, and shall be a corporation meeting the requirements of Section
310(a) of the Trust Indenture

                                     - 9 -
<PAGE>

Act. If such corporation publishes reports of condition at least annually,
pursuant to law or to the requirements of the supervising or examining
authority, then, for the purposes of this Section and to the extent permitted by
the Trust Indenture Act, the combined capital and surplus of such corporation
shall be deemed to be its combined capital and surplus as set forth in its most
recent report of condition so published.

      (b)   If at any time the Guarantee Trustee shall cease to be eligible to
so act under Section 4.1(a), the Guarantee Trustee shall immediately resign in
the manner and with the effect set out in Section 4.2.

      (c)   If the Guarantee Trustee has or shall acquire any "conflicting
interest" within the meaning of Section 310(b) of the Trust Indenture Act, the
Guarantee Trustee and Guarantor shall in all respects comply with the provisions
of Section 310(b) of the Trust Indenture Act.

      SECTION 4.2. Appointment, Removal and Resignation of the Guarantee
Trustee.

            (a)   No resignation or removal of the Guarantee Trustee and no
appointment of a Successor Guarantee Trustee pursuant to this Article shall
become effective until the acceptance of appointment by the Successor Guarantee
Trustee by written instrument executed by the Successor Guarantee Trustee and
delivered to the Holders and the Guarantee Trustee.

            (b)   Subject to the immediately preceding paragraph, a Guarantee
Trustee may resign at any time by giving written notice thereof to the Holders.
The Guarantee Trustee shall appoint a successor by requesting from at least
three Persons meeting the eligibility requirements such Persons' expenses and
charges to serve as the Guarantee Trustee, and selecting the Person who agrees
to the lowest expenses and charges. If the instrument of acceptance by the
Successor Guarantee Trustee shall not have been delivered to the Guarantee
Trustee within 30 days after the giving of such notice of resignation, the
Guarantee Trustee may petition, at the expense of the Guarantor, any court of
competent jurisdiction for the appointment of a Successor Guarantee Trustee.

            (c)   The Guarantee Trustee may be removed for cause at any time by
Act (within the meaning of Section 6.8 of the Trust Agreement) of the Holders of
at least a Majority in Liquidation Amount of the Capital Securities, delivered
to the Guarantee Trustee.

            (d)   If a resigning Guarantee Trustee shall fail to appoint a
successor, or if a Guarantee Trustee shall be removed or become incapable of
acting as Guarantee Trustee, or if any vacancy shall occur in the office of any
Guarantee Trustee for any cause, the Holders of the Capital Securities, by Act
of the Holders of record of not less than 25% in aggregate Liquidation Amount of
the Capital Securities then outstanding delivered to such Guarantee Trustee,
shall promptly appoint a successor Guarantee Trustee. If no Successor Guarantee
Trustee shall have been so appointed by the Holders of the Capital Securities
and such appointment accepted by the Successor Guarantee Trustee, any Holder, on
behalf of himself and all others similarly situated, may petition any court of
competent jurisdiction for the appointment of a Successor Guarantee Trustee.

                                    - 10 -
<PAGE>

                              ARTICLE V. GUARANTEE

      SECTION 5.1. Guarantee.

            The Guarantor irrevocably and unconditionally agrees to pay in full
to the Holders the Guarantee Payments (without duplication of amounts
theretofore paid by or on behalf of the Issuer Trust), as and when due,
regardless of any defense, right of set-off or counterclaim which the Issuer
Trust may have or assert, except the defense of payment. The Guarantor's
obligation to make a Guarantee Payment may be satisfied by direct payment of the
required amounts by the Guarantor to the Holders or by causing the Issuer Trust
to pay such amounts to the Holders. The Guarantor shall give prompt written
notice to the Guarantee Trustee in the event it makes any direct payment
hereunder.

      SECTION 5.2. Waiver of Notice and Demand.

            The Guarantor hereby waives notice of acceptance of the Guarantee
Agreement and of any liability to which it applies or may apply, presentment,
demand for payment, any right to require a proceeding first against the
Guarantee Trustee, the Issuer Trust or any other Person before proceeding
against the Guarantor, protest, notice of nonpayment, notice of dishonor, notice
of redemption and all other notices and demands.

      SECTION 5.3. Obligations Not Affected.

            The obligations, covenants, agreements and duties of the Guarantor
under this Guarantee Agreement shall in no way be affected or impaired by reason
of the happening from time to time of any of the following:

            (a)   the release or waiver, by operation of law or otherwise, of
the performance or observance by the Issuer Trust of any express or implied
agreement, covenant, term or condition relating to the Capital Securities to be
performed or observed by the Issuer Trust;

            (b)   the extension of time for the payment by the Issuer Trust of
all or any portion of the Distributions (other than an extension of time for
payment of Distributions that results from an election by the Depositor to defer
any interest payment on the Junior Subordinated Debentures as so provided in the
Indenture), Redemption Price, Liquidation Distribution or any other sums payable
under the terms of the Capital Securities or the extension of time for the
performance of any other obligation under, arising out of, or in connection
with, the Capital Securities;

            (c)   any failure, omission, delay or lack of diligence on the part
of the Holders to enforce, assert or exercise any right, privilege, power or
remedy conferred on the Holders pursuant to the terms of the Capital Securities,
or any action on the part of the Issuer Trust granting indulgence or extension
of any kind;

            (d)   the voluntary or involuntary liquidation, dissolution, sale of
any collateral, receivership, conservatorship, insolvency, bankruptcy,
assignment for the benefit of creditors,

                                    - 11 -
<PAGE>

reorganization, arrangement, composition or readjustment of debt of, or other
similar proceedings affecting, the Issuer Trust or any of the assets of the
Issuer Trust;

            (e)   any invalidity of, or defect or deficiency in, the Capital
Securities;

            (f)   the settlement or compromise of any obligation guaranteed
hereby or hereby incurred; or

            (g)   any other circumstance whatsoever that might otherwise
constitute a legal or equitable discharge or defense of a guarantor (other than
payment of the underlying obligation), it being the intent of this Section 5.3
that the obligations of the Guarantor hereunder shall be absolute and
unconditional under any and all circumstances.

            There shall be no obligation of the Holders to give notice to, or
obtain the consent of, the Guarantor with respect to the happening of any of the
foregoing.

      SECTION 5.4. Rights of Holders.

            The Guarantor expressly acknowledges that: (i) this Guarantee
Agreement will be deposited with the Guarantee Trustee to be held for the
benefit of the Holders; (ii) the Guarantee Trustee has the right to enforce this
Guarantee Agreement on behalf of the Holders; (iii) the Holders of a Majority in
Liquidation Amount of the Capital Securities have the right to direct the time,
method and place of conducting any proceeding for any remedy available to the
Guarantee Trustee in respect of this Guarantee Agreement or exercising any trust
or power conferred upon the Guarantee Trustee under this Guarantee Agreement;
and (iv) any Holder may institute a legal proceeding directly against the
Guarantor to enforce its rights under this Guarantee Agreement, without first
instituting a legal proceeding against the Guarantee Trustee, the Issuer Trust
or any other Person.

      SECTION 5.5. Guarantee of Payment.

            This Guarantee Agreement creates a guarantee of payment and not of
collection. This Guarantee Agreement will not be discharged except by payment of
the Guarantee Payments in full (without duplication of amounts theretofore paid
by the Issuer Trust) or upon the distribution of Junior Subordinated Debentures
to Holders as provided in the Trust Agreement.

      SECTION 5.6. Subrogation.

            The Guarantor shall be subrogated to all rights (if any) of the
Holders against the Issuer Trust in respect of any amounts paid to the Holders
by the Guarantor under this Guarantee Agreement; provided, however, that the
Guarantor shall not (except to the extent required by mandatory provisions of
law) be entitled to enforce or exercise any rights which it may acquire by way
of subrogation or any indemnity, reimbursement or other agreement, in all cases
as a result of payment under this Guarantee Agreement, if at the time of any
such payment, any amounts are due and unpaid under this Guarantee Agreement. If
any amount shall be paid to the Guarantor in

                                    - 12 -
<PAGE>

violation of the preceding sentence, the Guarantor agrees to hold such amount in
trust for the Holders and to pay over such amount to the Holders.

      SECTION 5.7. Independent Obligations.

            The Guarantor acknowledges that its obligations hereunder are
independent of the obligations of the Issuer Trust with respect to the Capital
Securities and that the Guarantor shall be liable as principal and as debtor
hereunder to make Guarantee Payments pursuant to the terms of this Guarantee
Agreement notwithstanding the occurrence of any event referred to in subsections
(a) through (g), inclusive, of Section 5.3 hereof.

                     ARTICLE VI. COVENANTS AND SUBORDINATION

      SECTION 6.1. Subordination.

            This Guarantee Agreement will constitute an unsecured obligation of
the Guarantor and will rank subordinate and junior in right of payment to all
Senior Indebtedness (as defined in the Indenture) of the Guarantor to the same
extent and in the same manner set forth in the Indenture with respect to the
Junior Subordinated Debentures, and the provisions of Article XIII of the
Indenture will apply, mutatis mutandis, to the obligations of the Guarantor
hereunder. The obligations of the Guarantor hereunder do not constitute Senior
Indebtedness of the Guarantor.

      SECTION 6.2. Pari Passu Guarantees.

            The obligations of the Guarantor under this Guarantee Agreement
shall rank pari passu with any similar guarantee agreements issued by the
Guarantor on behalf of the holders of Trust Securities issued by the Issuer
Trust and with any other security, guarantee or other obligation that is
expressly stated to rank pari passu with the obligations of the Guarantor under
this Guarantee Agreement.

                            ARTICLE VII. TERMINATION

      SECTION 7.1. Termination.

            This Guarantee Agreement shall terminate and be of no further force
and effect upon (i) full payment of the Redemption Price of all Outstanding
Capital Securities, (ii) the distribution of Junior Subordinated Debentures to
the Holders in exchange for all of the Outstanding Capital Securities or (iii)
full payment of the amounts payable in accordance with Article IX of the Trust
Agreement upon liquidation of the Issuer Trust. Notwithstanding the foregoing,
this Guarantee Agreement will continue to be effective or will be reinstated, as
the case may be, if at any time any Holder is required to restore payment of any
sums paid under the Capital Securities or this Guarantee Agreement.

                                    - 13 -
<PAGE>

                           ARTICLE VIII. MISCELLANEOUS

      SECTION 8.1. Successors and Assigns.

            All guarantees and agreements contained in this Guarantee Agreement
shall bind the successors, assigns, receivers, conservators, trustees and
representatives of the Guarantor and shall inure to the benefit of the Holders
of the Capital Securities then outstanding. Except in connection with a
consolidation, merger or sale involving the Guarantor that is permitted under
Article VIII of the Indenture and pursuant to which the assignee agrees in
writing to perform the Guarantor's obligations hereunder, the Guarantor shall
not assign its obligations hereunder, and any purported assignment that is not
in accordance with these provisions shall be void.

      SECTION 8.2. Amendments.

            Except with respect to any changes that do not materially adversely
affect the rights of the Holders (in which case no consent of the Holders will
be required), this Guarantee Agreement may only be amended with the prior
approval of the Holders of not less than a Majority in Liquidation Amount of the
Capital Securities. The provisions of Article VI of the Trust Agreement
concerning meetings of the Holders shall apply to the giving of such approval.

      SECTION 8.3. Notices.

            Any notice, request or other communication required or permitted to
be given hereunder shall be in writing, duly signed by the party giving such
notice, and delivered, telecopied (confirmed by delivery of the original) or
mailed by first class mail as follows:

            (a)   if given to the Guarantor, to the address or telecopy number
set forth below or such other address or telecopy number or to the attention of
such other Person as the Guarantor may give notice to the Holders:

                  Community Financial Holding Company, Inc.
                  2775 Buford Highway
                  Duluth, Georgia 30096
                  Facsimile No.: (770) 476-0334
                  Attention: Ms. Ann K. Marshall

            (b)   if given to the Issuer Trust, in care of the Guarantee
Trustee, at the Issuer Trust's (and the Guarantee Trustee's) address set forth
below or such other address or telecopy number or to the attention of such other
Person as the Guarantee Trustee on behalf of the Issuer Trust may give notice to
the Holders:

                  CFHC Capital Trust I
                  c/o Community Financial Holding Company, Inc.
                  2775 Buford Highway

                                    - 14 -
<PAGE>

                  Duluth, Georgia 30096
                  Facsimile No.: (770) 476-0334
                  Attention: Ms. Ann K. Marshall

            with a copy to:

                  Wilmington Trust Company
                  1100 North Market Street
                  Wilmington, Delaware 19890-0001
                  Facsimile No.: (302) 636-4140
                  Attention: Corporate Trust Administration

            (c)   if given to the Guarantee Trustee:

                  Wilmington Trust Company
                  1100 North Market Street
                  Wilmington, Delaware 19890-0001
                  Facsimile No.: (302) 636-4140
                  Attention: Corporate Trust Administration

            (d)   if given to any Holder of record, at the address set forth on
the books and records of the Issuer Trust.

            All notices hereunder shall be deemed to have been given when
received in person, telecopied with receipt confirmed, or mailed by first class
mail, postage prepaid, except that if a notice or other document is refused
delivery or cannot be delivered because of a changed address of which no notice
was given, such notice or other document shall be deemed to have been delivered
on the date of such refusal or inability to deliver.

      SECTION 8.4. Benefit.

            This Guarantee Agreement is solely for the benefit of the Holders
and is not separately transferable from the Capital Securities.

      SECTION 8.5. Interpretation.

            In this Guarantee Agreement, unless the context otherwise requires:

            (a)   capitalized terms used in this Guarantee Agreement but not
defined in the preamble hereto have the respective meanings assigned to them in
Section 1.1;

            (b)   a term defined anywhere in this Guarantee Agreement has the
same meaning throughout;

                                    - 15 -
<PAGE>

            (c)   all references to "the Guarantee Agreement" or "this Guarantee
Agreement" are to this Guarantee Agreement as modified, supplemented or amended
from time to time;

            (d)   all references in this Guarantee Agreement to Articles and
Sections are to Articles and Sections of this Guarantee Agreement unless
otherwise specified;

            (e)   a term defined in the Trust Indenture Act has the same meaning
when used in this Guarantee Agreement unless otherwise defined in this Guarantee
Agreement or unless the context otherwise requires;

            (f)   a reference to the singular includes the plural and vice
versa; and

            (g)   the masculine, feminine or neuter genders used herein shall
include the masculine, feminine and neuter genders.

      SECTION 8.6. Governing Law.

            THIS GUARANTEE AGREEMENT SHALL BE GOVERNED BY AND CONSTRUED AND
INTERPRETED IN ACCORDANCE WITH THE LAWS OF THE STATE OF NEW YORK WITHOUT REGARD
TO THE CONFLICT OF LAW PRINCIPLES THEREOF.

      SECTION 8.7. Counterparts.

            This instrument may be executed in any number of counterparts, each
of which so executed shall be deemed to be an original, but all such
counterparts shall together constitute but one and the same instrument.

                            [SIGNATURES ON NEXT PAGE]

                                    - 16 -
<PAGE>

            THIS GUARANTEE AGREEMENT is executed as of the day and year first
above written.

                                   COMMUNITY FINANCIAL HOLDING COMPANY
                                    as Guarantor

                                    By: /s/ Thomas J. Martin
                                        ----------------------------------------
                                    Name: Thomas J. Martin
                                    Title: President and Chief Executive Officer

                                    WILMINGTON TRUST COMPANY,
                                    as Guarantee Trustee, and not in its
                                    individual capacity

                                    By: /s/ Joann A. Rozell
                                        ----------------------------------------
                                    Name: Joann A. Rozell
                                    Title: Financial Services Officer

                                    - 17 -