<PAGE>   1
                                                                   EXHIBIT 10.4

                             SOUTHERN COMMUNITY BANK
                          SALARY CONTINUATION AGREEMENT

     THIS AGREEMENT is made this 23rd, day of February, 1999, by and between
SOUTHERN COMMUNITY BANCORP, a Florida corporation (the "Company") and Charlie W.
Brinkley, Jr. (the "Executive").

                                  INTRODUCTION

         To encourage the Executive to remain an employee of the Company, the
Company is willing to provide salary continuation benefits to the Executive. The
Company will pay the benefits from its general assets.

                                    AGREEMENT

         The Executive and the Company agree as follows:

                                    ARTICLE 1
                                   DEFINITIONS

         1.1 DEFINITIONS. Whenever used in this Agreement, the following words
and phrases shall have the meanings specified:

             1.1.1 "CHANGE OF CONTROL" means the transfer of shares of the
             Company's voting common stock such that one entity or one person
             acquires (or is deemed to acquire when applying Section 318 of the
             Code) more than 50 percent of the Company's outstanding voting
             common stock; provided, however, that the formation by the Company
             of a bank holding company which acquires all or substantially all
             of the shares of the Company's voting common stock shall not be
             deemed a Change of Control for purposes of this Agreement.

             1.1.2 "CODE" means the Internal Revenue Code of 1986, as amended.

             1.1.3 "CONSUMER PRICE INDEX" means an annual cost of living
             increase (but not decrease) of the benefit provided Executive in
             this Agreement, The cost of living increase shall be determined
             based upon the annual increase in the Consumer Price Index for all
             Urban Consumers - South, All Items (base year 1982-84 = 100) as
             published by the U.S. Department of Labor. If publication of the
             Consumer Price Index is discontinued, the parties shall accept
             comparable statistics on the cost of living for Orlando, Florida as
             such statistic are computed and published by a federal agency or by
             a recognized financial periodical selected by the parties.

                                        1


<PAGE>   2



             1.1.4 "DISABILITY" means, if the Executive is covered by a Company
             sponsored disability policy, total disability as defined in such
             policy without regard to any waiting period. If the Executive is
             not covered by such a policy, Disability means the Executive
             suffering a sickness, accident or injury which, in the judgment of
             a physician satisfactory to the Company, prevents the Executive
             from performing substantially all of the Executive's normal duties
             for the Company. As a condition to any benefits, the Company may
             require the Executive to submit to such physical or mental
             evaluations and tests as the Company's Board of Directors deems
             appropriate.

             1.1.5 "EARLY TERMINATION" means the Termination of Employment
             before Normal Retirement Age for reasons other than death,
             Disability, Termination for Cause or following a Change of Control.

             1.1.6 "EARLY TERMINATION DATE" means the month, day and year in
             which Early Termination occurs.

             1.1.7 "EFFECTIVE DATE" means January 1, 1999.

             1.1.8 "NORMAL RETIREMENT AGE" means the Executive's age 60.

             1.1.9 "NORMAL RETIREMENT DATE" means the later of the Normal
             Retirement Age or Termination of Employment.

             1.1.10 "PLAN YEAR" means a twelve-month period commencing on
             January 1st and ending on December 31st of each year. The initial
             Plan Year shall commence on the Effective Date of this Agreement.

             1.1.11 "TERMINATION FOR CAUSE" See Section 5.2.

             1.1.12 "TERMINATION OF EMPLOYMENT" means that the Executive ceases
             to be employed by the Company for any reason whatsoever other than
             by reason of a leave of absence which is approved by the Company.
             For purposes of this Agreement, if there is a dispute over the
             employment status of the Executive or the date of the Executive's
             Termination of Employment, the Company shall have the sole and
             absolute right to decide the dispute.

                                    ARTICLE 2
                                LIFETIME BENEFITS

       2.1 NORMAL RETIREMENT BENEFIT. Upon Termination of Employment on or
after the Normal Retirement Age for reasons other than death, the Company shall
pay to the Executive the benefit described in this Section 2.1 in lieu of any
other benefit under this Agreement.

                                        2


<PAGE>   3



             2.1.1 AMOUNT OF BENEFIT. The annual benefit under this Section 2.1
             is $ 50,507 (fifty thousand five hundred seven dollars) in the
             first Plan Year, increased by five percent (5.0%) each successive
             Plan Year until Termination of Employment

             2.1.2 PAYMENT OF BENEFIT. The Company shall pay the annual benefit
             to the Executive in 12 equal monthly installments payable on the
             first day of each month commencing with the month following the
             Executive's Normal Retirement Date and continuing for 239
             additional months.

             2.1.3 BENEFIT INCREASES. Commencing on the first anniversary of the
             first benefit payment, and continuing on each subsequent
             anniversary, the annual benefit will be increased by the Consumer
             Price Index.

         2.2 VOLUNTARY EARLY TERMINATION BENEFIT. Upon voluntary Early
Termination, the Company shall pay to the Executive the benefit described in
this Section 2.2 in lieu of any other benefit under this Agreement.

             2.2.1 AMOUNT OF BENEFIT. The benefit under this Section 2.2 is the
             Voluntary Early Termination Annual Benefit amount set forth in
             Schedule A for the Plan Year ending immediately prior to the Early
             Termination Date. (The Vesting Schedule for Voluntary Early
             Termination is 10% after the first Plan Year and an additional 10%
             for each additional Plan Year until vesting is 100% at the end of
             the tenth Plan Year.)

             2.2.2 PAYMENT OF BENEFIT. The Company shall pay the annual benefit
             to the Executive in 12 equal monthly installments payable on the
             first day of each month commencing with the month following the
             Normal Retirement Age and continuing for 239 additional months.

             2.2.3 BENEFIT INCREASES. Benefit payments will be increased as
             provided in Section 2.1.3.

         2.3 INVOLUNTARY EARLY TERMINATION BENEFIT. Upon involuntary Early
Termination, the Company shall pay to the Executive the benefit described in
this Section 2.3 in lieu of any other benefit under this Agreement.

             2.3.1 AMOUNT OF BENEFIT. The benefit under this Section 2.3 is the
             Involuntary Early Termination Annual Benefit amount set forth in
             Schedule A for the Plan Year ending immediately prior to the Early
             Termination Date. (The Vesting Schedule for Involuntary Early
             Termination is 50% during the first five Plan Years and an
             additional 10% for each additional Plan Year thereafter until
             vesting is 100% at the end of the tenth Plan Year.)

             2.3.2 PAYMENT OF BENEFIT. The Company shall pay the annual benefit
             to the Executive in 12 equal monthly installments payable on the
             first day of each month

                                        3


<PAGE>   4



             commencing with the month following the Normal Retirement Age and
             continuing for 239 additional months.

             2.3.3 BENEFIT INCREASES. Benefit payments will be increased as
             provided in Section 2.1.3.

         2.4 DISABILITY BENEFIT. If the Executive terminates employment due to
Disability prior to Normal Retirement Age, the Company shall pay to the
Executive the benefit described in this Section 2.4 in lieu of any other benefit
under this Agreement.

             2.4.1 AMOUNT OF BENEFIT. The benefit under this Section 2.4 is the
             Disability Annual Benefit amount set forth in Schedule A for the
             Plan Year ending immediately prior to the date in which the
             Termination of Employment occurs. (The Disability Annual Benefit
             amount is the twelve month total of monthly payments determined by
             calculating a fixed annuity which is payable in 180 equal monthly
             installments, crediting interest on the unpaid balance of the
             Accrual Balance at an annual rate of 7.5 %, compounded monthly.)

             2.4.2 PAYMENT OF BENEFIT. The Company shall pay the annual benefit
             amount to the Executive in 12 equal monthly installments payable on
             the first day of each month commencing with the month following the
             Termination of Employment and continuing for 239 additional months.

             2.4.3 BENEFIT INCREASES. Benefit payments may be increased as
             provided in Section 2.1.3.

         2.5 CHANGE OF CONTROL BENEFIT. Upon a Termination of Employment
following a Change of Control, the Company shall pay to the Executive the
benefit described in this Section 2.5 in lieu of any other benefit under this
Agreement.

             2.5.1 AMOUNT OF BENEFIT. The annual benefit under this Section 2.5
             is the Change of Control Annual Benefit set forth in Schedule A for
             the Plan Year in which Termination of Employment occurs. (The
             Change of Control Annual Benefit is the Normal Retirement Benefit
             amount described in Section 2.1.1 as if the date of the Termination
             of Employment was the Executive's Normal Retirement Date.)

             2.5.2 PAYMENT OF BENEFIT. The Company shall pay the annual benefit
             amount to the Executive in 12 equal monthly installments payable on
             the first day of each month commencing with the month following the
             Normal Retirement Date and continuing for 239 additional months.

             2.5.3 BENEFIT INCREASES. Benefit payments will be increased as
             provided in Section 2.1.3.


                                        4


<PAGE>   5



                                    ARTICLE 3
                                 DEATH BENEFITS

         3.1 DEATH DURING ACTIVE SERVICE. If the Executive dies while in the
active service of the Company, the Company shall pay to the Executive's
beneficiary the benefit described in this Section 3.1. This benefit shall be
paid in lieu of the Lifetime Benefits of Article 2.

             3.1.1 AMOUNT OF BENEFIT. The annual benefit under this Section 3.1
             is the Normal Retirement Benefit amount described in Section 2.1.1.

             3.1.2 PAYMENT OF BENEFIT. The Company shall pay the annual benefit
             to the beneficiary in 12 equal monthly installments payable on the
             first day of each month commencing with the month following the
             Executive's death and continuing for 239 additional months.

         3.2 DEATH DURING BENEFIT PERIOD. If the Executive dies after the
benefit payments have commenced under this Agreement but before receiving all
such payments, the Company shall pay the remaining benefits to the Executive's
beneficiary at the same time and in the same amounts they would have been paid
to the Executive had the Executive survived.

         3.3 DEATH AFTER TERMINATION OF EMPLOYMENT BUT BEFORE BENEFIT PAYMENTS
COMMENCE. If the Executive is entitled to benefit payments under this Agreement,
but dies prior to the commencement of said benefit payments, the Company shall
pay the benefit payments to the Executive's beneficiary that the Executive was
entitled to prior to death except that the benefit payments shall commence on
the first day of the month following the date of the Executive's death.

                                    ARTICLE 4
                                  BENEFICIARIES

         4.1 BENEFICIARY DESIGNATIONS. The Executive shall designate a
beneficiary by filing a written designation with the Company. The Executive may
revoke at any time by filing a new designation. However, designations will only
be effective if signed by the Executive and accepted by the Company during the
Executive's lifetime. The Executive's beneficiary designation shall be deemed
automatically revoked if the beneficiary predeceases the Executive, or if the
Executive names a spouse as beneficiary and the marriage is subsequently
dissolved. If the Executive dies without a valid beneficiary designation, all
payments shall be made to the Executive's estate.

         4.2 FACILITY OF PAYMENT. If a benefit is payable to a minor, to a
person declared incapacitated, or to a person incapable of handling the
disposition of his or her property, the Company may pay such benefit to the
guardian, legal representative or person having the care or custody of such
minor, incapacitated person or incapable person. The Company may require proof
of incapacity, minority or guardianship as it may deem appropriate prior to
distribution of the benefit. Such distribution shall completely discharge the
Company from all liability with respect to such benefit.


                                        5


<PAGE>   6



                                    ARTICLE 5
                               GENERAL LIMITATIONS

         Notwithstanding any provision of this Agreement to the contrary, the
Company shall not pay any benefit under this Agreement:

         5.1 EXCESS PARACHUTE PAYMENT. TO the extent the benefit would create an
excise tax under the excess parachute rules of Section 280G of the Code.

         5.2 TERMINATION FOR CAUSE. If the Company terminates the Executive's
employment for:

             5.2.1 Gross negligence or gross neglect of duties;

             5.2.2 Commission of a felony or of a gross misdemeanor involving
             moral turpitude; or

             5.2.3 Fraud, disloyalty, dishonesty or willful violation of any law
             or significant Company policy committed in connection with the
             Executive's employment and resulting in an adverse effect on the
             Company.

         5.3 SUICIDE OR MISSTATEMENT. No benefits shall be payable if the
Executive commits suicide within two years after the date of this Agreement, or
if the Executive has made any material misstatement of fact on any application
for life insurance purchased by the Company.

                                    ARTICLE 6
                          CLAIMS AND REVIEW PROCEDURES

         6.1 CLAIMS PROCEDURE. The Company shall notify any person or entity
that makes a claim against the Agreement (the "Claimant") in writing, within
ninety (90) days of Claimant's written application for benefits, of his or her
eligibility or noneligibility for benefits under the Agreement. If the Company
determines that the Claimant is not eligible for benefits or full benefits, the
notice shall set forth (1) the specific reasons for such denial, (2) a specific
reference to the provisions of the Agreement on which the denial is based, (3 )
a description of any additional information or material necessary for the
Claimant to perfect his or her claim, and a description of why it is needed, and
(4) an explanation of the Agreement's claims review procedure and other
appropriate information as to the steps to be taken if the Claimant wishes to
have the claim reviewed. If the Company determines that there are special
circumstances requiring additional time to make a decision, the Company shall
notify the Claimant of the special circumstances and the date by which a
decision is expected to be made, and may extend the time for up to an additional
ninety-day period.

         6.2 REVIEW PROCEDURE. If the Claimant is determined by the Company not
to be eligible for benefits, or if the Claimant believes that he or she is
entitled to greater or different benefits, the Claimant shall have the
opportunity to have such claim reviewed by the Company by filing a petition for
review with the Company within sixty (60) days after receipt of the notice
issued by the Company. Said petition shall state the specific reasons which the
Claimant believes entitle him or her to benefits or to greater or different
benefits. Within sixty (60) days after receipt by the

                                        6


<PAGE>   7



Company of the petition, the Company shall afford the Claimant (and counsel, if
any) an opportunity to present his or her position to the Company orally or in
writing, and the Claimant (or counsel) shall have the right to review the
pertinent documents. The Company shall notify the Claimant of its decision 'in
writing within the sixty-day period, stating specifically the basis of its
decision, written in a manner calculated to be understood by the Claimant and
the specific provisions of the Agreement on which the decision is based. If,
because of the need for a hearing, the sixty-day period is not sufficient, the
decision may be deferred for up to another sixty-day period at the election of
the Company, but notice of this deferral shall be given to the Claimant.

                                    ARTICLE 7
                           AMENDMENTS AND TERMINATION

         This Agreement may be amended or terminated only by a written agreement
signed by the Company and the Executive.

                                    ARTICLE 8
                                  MISCELLANEOUS

         8.1 BINDING EFFECT. This Agreement shall bind the Executive and the
Company, and their beneficiaries, survivors, executors, successors,
administrators and transferees.

         8.2 NO GUARANTEE OF EMPLOYMENT. THIS Agreement is not an employment
policy or contract. It does not give the Executive the right to remain an
employee of the Company, nor does it interfere with the Company's right to
discharge the Executive. It also does not require the Executive to remain an
employee nor interfere with the Executive's right to terminate employment at any
time.

         8.3 NON-TRANSFERABILITY. Benefits under this Agreement cannot be sold,
transferred, assigned, pledged, attached or encumbered in any manner.

         8.4 REORGANIZATION. The Company shall not merge or consolidate into or
with another company, or reorganize, or sell substantially all of its assets to
another company, firm, or person unless such succeeding or continuing company,
firm, or person agrees to assume and discharge the obligations of the Company
under this Agreement. Upon the occurrence of such event, the term "Company" as
used in this Agreement shall be deemed to refer to the successor or survivor
company.

         8.5 TAX WITHHOLDING. The Company shall withhold any taxes that are
required to be withheld from the benefits provided under this Agreement.

         8.6 APPLICABLE LAW. The Agreement and all rights hereunder shall be
governed by the laws of the State of Florida, except to the extent preempted by
the laws of the United States of America.

                                        7


<PAGE>   8



         8.7 UNFUNDED ARRANGEMENT. The Executive and beneficiary are general
unsecured creditors of the Company for the payment of benefits under this
Agreement. The benefits represent the mere promise by the Company to pay such
benefits. The rights to benefits are not subject in any manner to anticipation,
alienation, sale, transfer, assignment, pledge, encumbrance, attachment, or
garnishment by creditors. Any insurance on the Executive's life is a general
asset of the Company to which the Executive and beneficiary have no preferred or
secured claim.

         8.8 ENTIRE AGREEMENT. This Agreement constitutes the entire agreement
between the Company and the Executive as to the subject matter hereof. No rights
are granted to the Executive by virtue of this Agreement other than those
specifically set forth herein.

         8.9 ADMINISTRATION. The Company shall have powers which are necessary
to administer this Agreement, including but not limited to:

             8.9.1 Interpreting the provisions of the Agreement;

             8.9.2 Establishing and revising the method of accounting for the
             Agreement;

             8.9.3 Maintaining a record of benefit payments; and

             8.9.4 Establishing rules and prescribing any forms necessary or
             desirable to administer the Agreement.

         8.10 NAMED FIDUCIARY. For purposes of the Employee Retirement Income
Security Act of 1974, if applicable, the Company shall be the named fiduciary
and plan administrator under the Agreement. The named fiduciary may delegate to
others certain aspects of the management and operation responsibilities of the
plan including the employment of advisors and the delegation of ministerial
duties to qualified individuals.

         IN WITNESS WHEREOF, the Executive and a duly authorized Company officer
have signed this Agreement.

EXECUTIVE:                                   COMPANY:

                                             SOUTHERN COMMUNITY BANCORP

/s/ CHARLIE W. BRINKLEY, JR.                 By: /s/ STEPHEN R. JEUCK
-----------------------------                   ------------------------------
CHARLIE W. BRINKLEY, JR.                         Stephen R. Jeuck
                                                 Secretary and CFO

                                        8


<PAGE>   9



                             BENEFICIARY DESIGNATION

                             SOUTHERN COMMUNITY BANK
                          SALARY CONTINUATION AGREEMENT

                            CHARLIE W. BRINKLEY, JR.

I designate the following as beneficiary of any death benefits under this Salary
Continuation Agreement:

Primary:
        ------------------------------------------------------------------------

--------------------------------------------------------------------------------


Contingent:
             -------------------------------------------------------------------

--------------------------------------------------------------------------------


NOTE:    TO NAME A TRUST AS BENEFICIARY, PLEASE PROVIDE THE NAME OF THE
         TRUSTEE(S) AND THE EXACT NAME AND DATE OF THE TRUST AGREEMENT.

I understand that I may change these beneficiary designations by filing a new
written designation with the Company. I further understand that the designations
will be automatically revoked if the beneficiary predeceases me, or, if I have
named my spouse as beneficiary and our marriage is subsequently dissolved.

Signature ____________________________________

Date ________________________________________

Accepted by the Company this 29th day of March, 1999.

By __________________________________________

Title _________________________________________

                                        9


<PAGE>   10


                             SOUTHERN COMMUNITY BANK
                            CHARLIE W. BRINKLEY, JR.

                      SALARY CONTINUATION PLAN - SCHEDULE A
<TABLE>
<CAPTION>

                                                                          INVOLUNTARY
                                 EARLY                                       EARLY              CHANGE OF
                                 TERM.                  VOLUNTARY EARLY    TERMINATION         CONTROL ANNUAL     DISABILITY ANNUAL
PLAN    BENEFIT     ACCRUAL     VESTING       VESTED      TERMINATION     ANNUAL BENEFIT      BENEFIT PAYABLE      BENEFIT PAYABLE
YEAR     LEVEL      BALANCE    SCHEDULE      BENEFIT     ANNUAL BENEFIT   PAYABLE AT 60            AT 60             IMMEDIATELY
----   --------     -------    ---------    --------     --------------   -------------            -----             -----------
<S>      <C>         <C>          <C>          <C>          <C>             <C>                 <C>                    <C>
1        50,507      18,343       10.00%       5,051        5,051           25,253              100,000                1,773
2        53,032      41,510       20.00%      10,606       10,606           26,516              100,000                4,013
3        55,684      70,454       30.00%      16,705       16,705           27,842              100,000                6,811
4        58,468     106,293       40.00%      23,387       23,387           29,234              100,000               10,275
5        61,392     150,340       50.00%      30,696       30,696           30,696              100,000               14,534
6        64,461     204,134       60.00%      38,677       38,677           38,677              100,000               19,734
7        67,684     269,478       70.00%      47,379       47,379           47,379              100,000               26,051
8        71,068     348,477       80.00%      56,855       56,855           56,855              100,000               33,688
9        74,622     443,595       90.00%      67,160       67,160           67,160              100,000               42,883
10       78,353     557,705      100.00%      78,353       78,353           78,353              100,000               53,914
11       82,271     631,051      100.00%      82,271       82,271           82,271              100,000               61,004
12       86,384     714,043      100.00%      86,384       86,384           86,384              100,000               69,027
13       90,703     807,950      100.00%      90,703       90,703           90,703              100,000               78,106
14       95,238     914,207      100.00%      95,238       95,238           95,238              100,000               88,378
15      100,000   1,034,439      100.00%     100,000      100,000          100,000              100,000              100,000


</TABLE>




                       BANK COMPENSATION STRATEGIES GROUP


                                       10