<PAGE>   1
 
                                                                   EXHIBIT 10.24
 
                        ALABAMA NATIONAL BANCORPORATION
 
                           DEFERRED COMPENSATION PLAN
               FOR DIRECTORS WHO ARE NOT EMPLOYEES OF THE COMPANY
 
I. ELIGIBILITY AND PURPOSE.
 
     Each member of the Board of Directors (the "Board) of Alabama National
BanCorporation (the "Company") who is not an employee of the Company or its
subsidiaries shall be eligible to participate in the Alabama National
BanCorporation Deferred Compensation Plan for Directors Who Are Not Employees of
the Company (the "Plan"). Any member of the Board who elects to participate in
the Plan ("Director") shall thereby defer the receipt of all or any portion of
the annual retainer, meeting and committee fees payable by the Company to such
Director for serving as a member of the Board or one or more of its committees
(the "Deferrable Compensation").
 
II. DEFERRAL OF COMPENSATION.
 
     A Director may elect to defer all or any portion of the Deferrable
Compensation by executing a form prescribed by the Company and delivering such
election form to the Company prior to the first day of the calendar year for
which the election is to be effective or at such other time and subject to such
other conditions as the Company shall determine, provided that any such election
shall be applicable only to Deferrable Compensation with respect to which the
Director, at the time of election, has no current right to receive. In the
calendar year that a Director first becomes eligible to participate in the Plan,
such Director may elect to defer all or any portion of the Deferrable
Compensation, provided that the election form is delivered to the Company within
thirty (30) days after the Director first becomes eligible to participate in the
Plan for such year. An election made in this manner will be applicable only to
Deferrable Compensation earned after the effective date of the election. The
amount of Deferrable Compensation deferred shall be paid or distributed to the
Director in accordance with the provisions of Section 5 or Section 6 hereof.
 
III. DEFERRED COMPENSATION ACCOUNT.
 
     The Company shall establish a deferred compensation account (the "Account")
for the Director. As of the date payments of Deferrable Compensation otherwise
would be made to the Director, the Company shall credit to the Account, in cash
or stock equivalents, or a combination thereof, as hereinafter provided, that
amount of the Deferrable Compensation which the Director has elected to defer.
 
IV.  CASH OR STOCK ELECTION.
 
     1. As of the date payments of Deferrable Compensation otherwise would be
made to the Director, the amount due the Director shall be credited to the
Account either as a cash allotment or as a stock allotment, or a portion to
each, as the Director shall elect.
 
     2. If a cash allotment is elected in whole or in part, the Account shall be
credited with the dollar amount of the allotment. Interest (at the rate
described below) on the Average Daily Balance (computed as described below)
shall be credited to the Account as of the last day of each calendar month
before and after the termination of the Director's service and after the
Director's death or disability until the total balance in the Account has been
paid out in accordance with the provisions of Section 5 or Section 6 hereof. The
interest rate for each calendar month shall be the 30-Day London Interbank
Offered Rate (LIBOR) plus 75 basis points for the last business day of the
immediately preceding calendar month as reported on the Bloomberg financial news
system. The "Average Daily Balance" shall be the quotient obtained by dividing
the sum of the closing balance in the Account at the end of each calendar day in
a calendar month by the number of days in such calendar month.
 
                                        1
<PAGE>   2
 
     3. (i) If a stock allotment is elected in whole or in part, the Account
shall be credited with a stock equivalent that shall be equal to the number of
full and fractional shares of the Company's Common Stock, par value $1.00 per
share (the "Common Stock"), that could be purchased with the dollar amount of
the allotment using the Average Closing Price (as defined below) of the Common
Stock for the twenty (20) trading days ending on the day preceding the date the
Account is so credited. The "Average Closing Price" of the Common Stock means
the average of the daily closing prices for a share of the Common Stock for the
applicable twenty (20) trading days on the Composite Tape for the New York Stock
Exchange D Listed Stocks, or, if the Common Stock is not listed on such
Exchange, on the principal United States securities exchange registered under
the Securities Exchange Act of 1934, as amended (the "Exchange Act"), on which
the Common Stock is listed, or, if the Common Stock is not listed on any such
Exchange, the average of the daily closing bid quotations with respect to a
share of the Common Stock for such twenty (20) trading days on the National
Association of Securities Dealers, Inc., Automated Quotations Systems or any
system then in use, or, if no such quotations are available, the fair market
value of a share of the Common Stock as determined by a majority of the Board.
 
     (ii) The Account also shall be credited as of the payment date for each
dividend on the Common Stock with additional stock equivalents computed as
follows: The dividend paid, either in cash or property (other than Common
Stock), upon a share of Common Stock to a shareholder of record shall be
multiplied by the number of stock equivalents in the Account and the product
thereof shall be divided by the Average Closing Price of the Common Stock for
the twenty (20) trading days ending on the day preceding the dividend payment
date. In the case of dividends payable in property, the amount paid shall be
based on the fair market value of the property at the time of distribution of
the dividend, as determined by a majority of the Board.
 
     (iii) In the event of any change in the Common Stock, upon which the stock
equivalency hereunder is based, by reason of a merger, consolidation,
reorganization, recapitalization, stock dividend, stock split, combination or
exchange of shares, or any other change in corporate structure, the number of
shares credited to the Account shall be adjusted in such manner as a majority of
the Board shall determine to be fair under the circumstances.
 
V.  DISTRIBUTION.
 
     1. Except as otherwise provided in the Plan, at the Director's election,
the balance in the account shall be paid out to the Director commencing on the
date which the Director has specified on his or her election form.
 
     Except as otherwise provided in the Plan, the balance in the Account shall
be paid either in a lump sum or, at the Director's election, in monthly,
quarterly, semiannual or annual installments, but such installments shall be
payable over a period of years not to exceed ten (10) years (the "Payout
Period"). In order to be effective, an election to change the method and/or
timing of distribution with respect to the Account must be in a form prescribed
by the Company and received by the Company at least six months prior to such
Director's retirement as Director of the Company and prior to the first day of
the calendar year in which payments (i) are to begin pursuant to such election
and (ii) would have begun absent such election. The amount of each installment
shall be determined as of the first day of the period in which payment is to be
made by dividing the then balance in the Account by the then remaining number of
payment dates in the Payout Period. The lump sum or first periodic installment
shall be paid by the Company as promptly as is convenient, but not more than
sixty (60) days following the date specified by the Director.
 
     2. Notwithstanding the provisions of Section 5(a), in the event the
Director ceases to hold office as a member of the Board, other than after a
Change in Control (as defined in Section 6(a) below) or due to such Director's
retirement from the Board, prior to distribution of the entire balance in the
Director's Account, the balance in the Account shall be payable in a lump sum.
 
     3. In the event of the death of a Director prior to distribution of the
entire balance in the Director's Account, the balance in the Account shall be
payable in a lump sum to
 
                                        2
<PAGE>   3
 
          (a) the surviving beneficiary (or surviving beneficiaries in such
     proportions as) the Director may have designated by notice in writing to
     the Company unrevoked by a later notice in writing to the Company or, in
     the absence of an unrevoked notice,
 
          (b) the beneficiary (or beneficiaries in such proportions as) the
     Director may have designated by will or, if no beneficiary is designated,
 
          (c) the legal representative of the Director's estate.
 
     In the event a Director becomes disabled, the payment commencement date
and/or payment schedule with respect to a balance in a Director's Account may be
accelerated by the Board (or its designee) in its sole discretion.
 
     4. The provisions of the Plan shall apply to and be binding upon the
beneficiaries, distributees and personal representatives and any other
successors in interest of the Director.
 
     5. Distribution of the cash in the Account shall be made in cash.
Distribution of stock equivalents in the Account shall be made in whole shares
of the Company's Common Stock; fractional shares shall be paid in cash in an
amount equal to the number of fractional shares multiplied by the Average
Closing Price of the Common stock for the twenty (20) trading days ending on the
day preceding the date of distribution.
 
     6. The Company shall deduct from all distributions hereunder any taxes
required to be withheld by the Federal or any State or local government.
 
VI. ACCELERATION OF DISTRIBUTION.
 
     1. "Change in Control" means: (i) acquisition by any person (within the
meaning of Section 13(d) of the Securities Exchange Act of 1934 (the "Exchange
Act")) of beneficial ownership (within the meaning of Rule 13d-3 under the
Exchange Act) of 20% or more of the Common Stock then outstanding; or (ii) the
consummation of (A) any consolidation or merger of the Company in which the
Company is not the continuing or surviving corporation or pursuant to which
shares of the Common Stock are converted into cash, securities or other
property, other than a merger of the Company in which the holders of Common
Stock immediately prior to the merger have the same proportionate ownership of
common stock of the surviving corporation immediately after the merger as they
had in Common Stock immediately prior to the merger, or (B) any sale, lease,
exchange or other transfer (in one transaction or a series of related
transactions) of all, or substantially all, of the assets of the Company,
including, without limitation, any sale, lease, exchange or other transfer (in
one transaction or a series of related transactions) of all or substantially
all, of the assets of the Company.
 
     2. Notwithstanding any other provisions of the Plan, if a Change in Control
occurs and at any time after or in connection with the occurrence of such Change
in Control either of the following events occurs:
 
          (i) the Directors ceases to hold office as a member of the Board;
 
          (ii) the Plan is terminated; or
 
          (iii) The Company's capital structure is change materially;
 
then the balance in the Account shall be payable in a lump sum to the Director
as soon as practicable after January 1 of the following calendar year unless
such Director completes a new election form prior to the end of the current
calendar year, determining the method and timing of election, provided, that, no
such election shall cause a distribution to occur earlier than the calendar year
following such election. If payment is payable in a lump sum, such payment shall
be made by the Company as promptly as practicable, but not more than thirty (30)
days following the date on which the right to such payment arose.
 
     3. Distribution shall be in accordance with Sections 5(b), 5(c) and 5(e),
above, except that distribution of stock equivalents in the Account shall be
made in cash in an amount equal to the number of stock equivalents to be
distributed multiplied by the greater of (i) the Average Closing Price of the
Common Stock for the twenty (20) trading days ending on the day preceding the
date on which the right to such distribution
 
                                        3
<PAGE>   4
 
arose; (ii) the Average Closing Price of the Common Stock for the twenty (20)
trading days ending on the day preceding the date of the Change in Control; or
(iii) the highest price per share of Common Stock in the transaction or series
of transactions constituting the Change in Control.
 
     4. The Company shall promptly reimburse the Director for all legal fees and
expenses reasonably incurred in successfully seeking to obtain or enforce any
right or benefit provided under this Section 6.
 
     5. This Section 6 may not be amended or modified after the occurrence of a
Change in Control.
 
VII. MISCELLANEOUS
 
     1. Except as provided in 6(b) above, the election to defer Deferrable
Compensation, including, but not limited to, the allocation of the amount
deferred between the cash allotment or the stock allotment portion of the
Account, or a combination thereof, shall be irrevocable as to amounts earned
following the time when the election is made and shall remain irrevocable until
a new election form reflecting a change or revocation with respect to amounts
earned in a subsequent time period is delivered to the Company not later than
ten (10) days preceding the first day of the calendar month to which such change
or revocation is applicable.
 
     2. Neither the Director nor any other person shall have any interest in any
fund or in any specific asset of the Company by reason of amounts credited to
the Account of a Director hereunder, nor the right to exercise any of the rights
or privileges of a shareholder with respect to any stock equivalents credited to
the Account, nor the right to receive any distribution under the Plan except as
and to the extent expressly provided for in the Plan. Distributions hereunder
shall be made from the general funds of the Company, and the rights of the
Director shall be those of an unsecured general creditor of the Company.
 
     3. The interest of the Director under the Plan shall not be assignable by
the Director or the Director's beneficiary or legal representative, either by
voluntary assignment or by operation of law, and any assignment of such
interest, whether voluntary or by operation of law, shall be ineffective to
transfer the Director's interest; provided, however, that (i) the Director may
designate a beneficiary to receive any benefit payable under the Plan upon
death, and (ii) the legal representative of the Director's estate may assign the
Director's interest under the Plan to the persons entitled to any benefit
payable under the Plan upon the Director's death.
 
     4. Except as provided in Section 6 hereof, the Company may amend, modify,
terminate or discontinue the Plan at any time; provided, however, that no such
action shall reduce the amounts credited to the Account of the Director
immediately prior to such action, nor change the time, method or manner of
distribution of such amount, including, without limitation, distribution in
accordance with Section 6 hereof.
 
     5. Nothing contained herein shall impose any obligation on the Company to
continue the tenure of the Director beyond the term for which such Director may
have been elected or shall prevent the removal of such Director.
 
     6. This Plan shall be interpreted by and all questions arising in
connection therewith shall be determined by a majority of the Board, whose
interpretation or determination, where made in good faith, shall be conclusive
and binding.
 
     7. If any amounts deferred pursuant to the Plan are found in a
"determination" (within the meaning of Section 1313(a) of the Internal Revenue
Code of 1986, as amended) to have been includible in gross income by a Director
prior to payment of such amounts from his Director's Account, such amounts shall
be immediately paid to such director, notwithstanding his elections pursuant to
Section 2 hereof.
 
                                        4