<PAGE>   1
 
                                                                   EXHIBIT 10.22
 
                                                                [EXECUTION COPY]
 
                                   AGREEMENT
 
     THIS AGREEMENT (this "Agreement") made as of the 30th day of April, 1996 by
and between JAMES A. TAYLOR ("Taylor"), and ALABAMA NATIONAL BANCORPORATION, a
Delaware corporation (the "Company").
 
                                    RECITALS
 
     WHEREAS, pursuant to that certain Employment Agreement dated November 18,
1994 (the "Employment Agreement"), the Company employed Taylor as its Chairman
of the Board and Chief Executive Officer, subject to the terms and conditions
thereof;
 
     WHEREAS, on December 29, 1995, the Company engaged in a "Change in
Control", as that term is defined in the Employment Agreement, and Taylor has
now provided a "Notice of Termination" for "Good Reason" to the Company;
 
     WHEREAS, Taylor and the Company have agreed on a proposed settlement of the
Employment Agreement as set forth herein, and Taylor and the Company wish to
settle all other potential claims and disputes that may exist and to avoid any
potential litigation; and
 
          WHEREAS, the parties desire to evidence their full and complete
     agreement by this writing;
 
                                   AGREEMENT
 
     NOW, THEREFORE, in consideration of the premises and for and in the further
consideration of the mutual promises of the parties contained herein, the
parties hereby agree as follows:
 
          1. Termination of Employment.  Effective April 30, 1996, as requested
     by Taylor pursuant to his Notice of Termination, Taylor is hereby
     terminated as an employee and officer of the Company and any of its
     subsidiaries. In addition, simultaneous with the execution of this
     Agreement, Taylor shall provide to the Company and each of its subsidiaries
     an acceptable letter of resignation whereby Taylor immediately resigns his
     position as a director of the Company and all of its subsidiaries.
 
          2. Termination of Employment Agreement and Compensation
     Therefor.  Upon execution of this Agreement, the Employment Agreement shall
     immediately terminate and be of no further force and effect, and, other
     than as set forth herein, the Company shall have no further duties or
     obligations to Taylor thereunder. In consideration of Taylor's agreement to
     terminate the Employment Agreement, the Company shall pay to Taylor in
     readily available funds an amount equal to Three Hundred Eighty Thousand
     Three Hundred Sixteen Dollars ($380,316), less any withholding and other
     applicable taxes, and, furthermore, the Company agrees to the following
     conditions:
 
             (a) For the period from the date hereof through the date that
        Taylor attains the age of 65 (the "Continuation Period"), the Company
        shall at its expense continue on behalf of Taylor and his dependents and
        beneficiaries the life insurance, disability, medical, dental and
        hospitalization benefits as specifically set forth on Exhibit A attached
        hereto and made a part hereof; provided, however, notwithstanding
        anything to the contrary contained in this Section 2(a), the amounts
        paid by the Company and/or its subsidiaries (except for the medical
        reimbursement plan currently maintained by First National Bank of
        Ashland which has an annual cap of $14,000) on an annual basis for the
        coverage and benefits (including deductibles and costs) provided in this
        Section 2(a) during the Continuation Period shall not exceed the amounts
        paid by the Company and/or the subsidiaries for the 12-month period
        ended March 31, 1996 for such coverage and benefits
 
                                        1
<PAGE>   2
 
        (including deductibles and costs) and (ii) the Company or Taylor may
        replace any of the coverages set forth on Exhibit A, as long as the
        replacement coverage is comparable, if the Company replaces a coverage,
        or the cost to the Company is comparable, if Taylor replaces a coverage.
        The Company's obligations hereunder with respect to the foregoing
        benefits shall be limited to the extent that Taylor obtains any such
        benefits pursuant to a subsequent employer's benefit plans, in which
        case the Company may reduce the coverage of any benefits it is required
        to provide Taylor hereunder as long as the aggregate coverages and
        benefits of the combined benefit plan is no less favorable to Taylor
        than the coverages and benefits required to be provided hereunder.
 
             (b) The restrictions on any outstanding incentive awards (including
        stock options) granted to Taylor under the Company's 1994 Stock Option
        Plan or under any other incentive plan or arrangement shall lapse and
        such incentive award shall become 100% vested, all stock options and
        stock appreciation rights granted to Taylor shall become immediately
        exercisable and shall become 100% vested, and all stock options granted
        to Taylor shall become 100% vested.
 
             (c) Taylor shall not be required to mitigate the amount of any
        payment provided for in this Agreement by seeking other employment or
        otherwise and no such payment shall be offset or reduced by the amount
        of any compensation or benefits provided to Taylor in any subsequent
        employment except as provided for in Section 2(a).
 
             (d) In the event that any payment or benefit (within the meaning of
        Section 280 G(b)(2) of the Internal Revenue Code of 1986, as amended
        (the "Code")) to Taylor or for his benefit paid or payable to
        distributed or distributable pursuant to the terms of this Agreement or
        otherwise in connection with, or arising out of, his employment with the
        Company or a change in ownership or effective control of the Company or
        of a substantial portion of its assets (a "Payment" or "Payments"),
        would be subject to the excise tax imposed by Section 4999 of the Code
        or any interest or penalties are incurred by Taylor with respect to such
        excise tax (such excise tax, together with any such interest and
        penalties, are hereinafter collectively referred to as the "Excise
        Tax"), then Taylor will be entitled to receive an additional payment (a
        "Gross-Up Payment") in an amount such that after payment by Taylor of
        all taxes (including any interest or penalties, other than interest and
        penalties imposed by reason of Taylor's failure to file timely a tax
        return or pay taxes shown due on his return, imposed with respect to
        such taxes and the Excise Tax), including any Excise Tax imposed upon
        the Gross-Up Payment, Taylor retains an amount of the Gross-Up Payment
        equal to the Excise Tax imposed upon the Payments.
 
             (e) The severance pay and benefits provided for in this Section 2
        shall be in lieu of any other severance or termination pay to which
        Taylor may be entitled under any Company severance or termination plan,
        program, practice or arrangement.
 
          3. Consulting Agreement.  In connection with and as a condition to
     this Agreement, Taylor and the Company shall enter into a Consulting
     Agreement in the form of Exhibit B attached hereto and made a part hereof
     (the "Consulting Agreement"), pursuant to which, among other things, Taylor
     shall agree to act as a consultant to the Company for a period of two years
     after the date hereof. In consideration thereof, the Company shall pay to
     Taylor those amounts and in those increments set forth in the Consulting
     Agreement.
 
          4. Demand Registration Rights.
 
             (a) Subject to the provisions of this Section 4(a), Taylor may
        request registration for sale under the Securities Act of 1933 (the
        "Act") of all or part of the common stock of the Company (the "Common
        Stock") then held by him. Any such request shall specify the number of
        shares proposed to be registered and sold and the name of the managing
        underwriter of the proposed offering (who must be acceptable to the
        Company in its reasonable discretion).
 
             (b) Exceptions.  The Company shall not be required to effect a
        demand registration under the Act pursuant to Section 4(a) above if (i)
        the aggregate market value of the shares of Common Stock proposed to be
        registered does not equal or exceed $1,000,000; (ii) within twelve
        months prior
 
                                        4
<PAGE>   3
 
        to any such request for registration, a registration of securities of
        the Company has been effected in which Taylor had the right to
        participate to this Section 4 or Section 5 hereof; (iii) the Company
        received such request for registration within 180 days preceding the
        anticipated effective date of a proposed underwritten public offering of
        securities of the Company approved by the Company's Board of Directors
        prior to the Company's receipt of such request; or (iv) the Board of
        Directors of the Company reasonably determines in good faith that
        effecting such a demand registration at such time would have a material
        adverse effect upon a proposed sale of all (or substantially all) of the
        assets of the Company, or a merger, reorganization, recapitalization or
        similar transaction materially affecting the capital structure or equity
        ownership of the Company which is actively being negotiated with another
        party whose identity is disclosed to Taylor; provided, however, that the
        Company may only delay a demand registration pursuant to this Section
        4(a)(iv) for a period not exceeding 6 months (or until such earlier time
        as such transaction is consummated or no longer proposed). The Company
        shall promptly notify in writing Taylor of any decision not to effect
        any such request for registration pursuant to this Section 4(b), which
        notice shall set forth in reasonable detail the reason for such decision
        and shall include an undertaking by the Company promptly to notify
        Taylor as soon as a demand registration may be effected.
 
             (c) Reduction.  If the managing underwriters advise the Company and
        Taylor in writing that in their opinion the number of shares of Common
        Stock held by Taylor which he requested to be included in such
        registration exceeds the number which can be sold in such offering, then
        the amount of such shares that may be included in such registration
        shall be reduced to the number of shares that the managing underwriters
        determine is marketable.
 
             (d) Withdrawal.  Taylor may withdraw at any time before a
        registration statement filed pursuant to this section is declared
        effective, in which event the Company may withdraw such registration
        statement. If the Company withdraws a registration statement under this
        Section 4(d) in respect of a registration for which the Company would
        otherwise be required to pay some expenses under Section 6(c), (d) and
        (e) hereof, then Taylor shall be liable to the Company for all expenses
        of such registration specified in Section 6(c), (d) and (e).
 
          5. Piggyback Registration Rights.
 
             (a) Rights.  Subject to the provisions of this Section 5, if the
        Company proposes to make a registered public offering of any of its
        securities under the Act (whether to be sold by it or by one or more
        third parties), other than an offering pursuant to a demand registration
        under Section 4 hereof or an offering registered on Form S-8, Form S-4,
        or comparable forms, the Company shall, not less than 45 days prior to
        the proposed filing date of the registration form, give written notice
        of the proposed registration to Taylor, and at the written request of
        Taylor delivered to the Company within 15 days after the receipt of such
        notice, shall include in such registration and offering, and in any
        underwriting of such offering, all shares of Common Stock as may have
        been designated in Taylor's request.
 
             (b) Primary Offering Reduction.  If a registration in which Taylor
        has the right to participate pursuant to this Section 5 is an
        underwritten primary registration on behalf of the Company, and the
        managing underwriters advise the Company in writing that in their
        opinion the number of securities requested to be included in such
        registration exceeds the number which can be sold in such offering, the
        Company shall include in such registration (i) first, the securities of
        the Company proposed to be sold, and (ii) second, the Common Stock owned
        by Taylor.
 
             (c) Secondary Offering Reduction.  If a registration in which
        Taylor has the right to participate pursuant to this Section 5 is an
        underwritten secondary registration, and the managing underwriters
        advise the Company in writing that in their opinion the number of shares
        requested to be included in such registration exceeds the number of
        shares which can be sold in such offering, then the Company shall
        include in such offering the number of shares of Common Stock owned and
        proposed to be sold by the Company and by any other participants
        (including Taylor) proposing (and entitled) to sell shares pursuant to
        such registration which the underwriters advise the
 
                                        5
<PAGE>   4
 
        Company can be sold in the offering, in proportion to the number of
        shares of Common Stock so requested by each of them to be included.
 
          6. Other Registration Issues.
 
             (a) The Company shall have no obligation to file a registration
        statement pursuant to Section 4 hereof, or to include shares of Common
        Stock owned by Taylor in a registration statement pursuant to Section 5
        hereof, unless and until Taylor has furnished the Company with all
        information and statements about or pertaining to Taylor in such
        reasonable detail as is reasonably deemed by the Company to be necessary
        or appropriate with respect to the preparation of the registration
        statement. Whenever Taylor has requested that any shares of Common Stock
        be registered pursuant to Sections 4 or 5 hereof, subject to the
        provisions of those Sections, the Company shall, as expeditiously as
        reasonably possible:
 
                (i) prepare and file with the Securities and Exchange Commission
           (the "SEC") a registration statement with respect to such shares and
           use its best efforts to cause such registration statement to become
           effective as soon as reasonably practicable thereafter (provided that
           before filing a registration statement or prospectus or any
           amendments or supplements thereto, the Company shall furnish counsel
           for Taylor with copies of all such documents proposed to be filed);
 
                (ii) prepare and file with the SEC such amendments and
           supplements to such registration statement and prospectus used in
           connection therewith as may be necessary to keep such registration
           statement effective for a period of not less than nine months or
           until the underwriters have completed the distributions described in
           such registration statement, whichever occurs first;
 
                (iii) furnish to Taylor such number of copies of such
           registration statement, each amendment and supplement thereto, the
           prospectus included in such registration statement (including each
           preliminary prospectus), and such other documents as Taylor may
           reasonably request;
 
                (iv) use its best efforts to register or qualify such shares
           under such other securities or Blue Sky Laws of such jurisdictions as
           Taylor reasonably requests (and to maintain such registrations and
           qualifications effective for a period of nine months or until the
           underwriters have completed the distribution of such shares,
           whichever occurs first), and to do any and all other acts and things
           which may be necessary or advisable to enable Taylor or underwriters
           to consummate the disposition in such jurisdictions of such shares;
           provided, however, that the Company will not be required to (A)
           qualify generally to do business in any jurisdiction where it would
           not be required but for this Section 6(a)(iv), or (B) subject itself
           to taxation in any such jurisdiction; provided, further, that,
           notwithstanding anything to the contrary in this Agreement with
           respect to the bearing of expenses, if any such jurisdiction shall
           require that expenses incurred in connection with the qualification
           of such shares in that jurisdiction be borne in part or full by
           Taylor, then Taylor shall pay such expenses to the extent required by
           such jurisdiction;
 
                (v) cause all such shares to be listed on securities exchanges,
           if any, on which similar securities issued by the Company are then
           listed;
 
                (vi) provide a transfer agent and registrar for all such shares
           not later than the effective date of such registration statements;
 
                (vii) enter into such customary agreements (including an
           underwriting agreement in customary form) and take all such other
           actions as Taylor and the underwriters reasonably request (and
           subject to approval by the Company's counsel) in order to expedite or
           facilitate the disposition of such shares; and
 
                                        6
<PAGE>   5
 
                (viii) make available for inspection by Taylor, by any
           underwriter participating in any distribution pursuant to such
           registration statement, and by any attorney, accountant or other
           agent retained by Taylor or such underwriter, or by any such
           underwriter, all financial and other records, pertinent corporate
           documents and properties (other than confidential intellectual
           property) of the Company; provided, however, that the Company can
           condition delivery of any information, records or corporate documents
           upon the receipt from Taylor and the underwriter and their counsel,
           accountants, advisors and agents, of a confidentiality agreement in
           form and substance acceptable to the Company and its counsel in the
           exercise of their exclusive discretion.
 
             (b) Holdback Agreement.  In the event that the Company effects an
        underwritten public offering of any of the Company's equity securities,
        Taylor agrees, if requested by the managing underwriters, not to effect
        any sale or distribution, including any sale pursuant to Rule 144 under
        the Act, of any equity securities (except as part of such underwritten
        offering) during the 180-day period commencing with the effective date
        of the registration statement for such offering.
 
             (c) Stockholder Expenses.  If, pursuant to Section 4 or 5 hereof,
        shares of Common Stock owned by Taylor are included in a registration
        statement, then Taylor shall pay all transfer taxes, if any, relating to
        the sale of his shares, the fees and expenses of his own counsel, and
        his pro rata portion of any underwriting discounts, fees or commission
        or the equivalent thereof.
 
             (d) The Company's Expenses.  Except for the fees and expenses
        specified in Section 6(c) hereof and except as provided in this Section
        6(d), the Company shall pay all expenses incident to the registration
        and to the Company's performance of or compliance with Sections 4, 5 and
        6 of this Agreement, including, without limitation, all registration and
        filing fees, fees and expenses of compliance with securities or Blue Sky
        Laws, underwriting discounts, fees and commissions (other than Taylor's
        pro rata portion of any underwriting discounts or commission or the
        equivalent thereof), printing expenses, messenger and delivery expenses,
        and fees and expenses of counsel for the Company and all independent
        certified public accountants and other persons retained by the Company.
        If the Company shall previously have paid, pursuant to this Section
        6(d), the expenses of a registration pursuant to this Agreement, then
        Taylor shall pay all expenses described in this Section 6(d) (but not
        expenses described in Section 6(e) hereof).
 
             (e) Other.  With respect to any registration pursuant to Section 4
        or 5 hereof, the Company shall pay its internal expenses (including,
        without limitation, all salaries and expenses of its officers and
        employees performing legal or accounting duties) and the expenses and
        fees for listing the securities to be registered on exchanges on which
        similar securities issued by the Company are then listed.
 
             (f) Indemnity.  In the event that any shares of Common Stock owned
        by Taylor are offered or sold by means of a registration statement
        pursuant to Section 4 or 5 hereof, the Company agrees to indemnify and
        hold harmless Taylor and each person, if any, who controls or may
        control Taylor within the meaning of the Act (Taylor and any such other
        persons being hereinafter referred to individually as an "Indemnified
        Person" and collectively as "Indemnified Persons") from and against all
        demands, claims, actions or causes of action, assessments, losses,
        damages, liabilities, costs and expenses, including, without limitation,
        interest, penalties and reasonable attorneys fees and disbursements,
        asserted against, resulting to, imposed upon or incurred by such
        Indemnified Person, jointly or severally, directly or indirectly
        (hereinafter referred to in this Section 6(f) in the singular as a
        "claim" and in the plural as "claims"), based upon, arising out of or
        resulting from any untrue statement or alleged untrue statement of a
        material fact contained in the registration statement, any preliminary
        or final prospectus contained herein, or any amendment or supplement
        thereto, or any document incident to registration or qualification of
        any such shares, or any omission or alleged omission to state therein a
        material fact necessary to make the statements made therein, in light of
        the circumstances under which they were made, not misleading, or any
        violation by the Company of the Act or any state securities or Blue Sky
        Laws, except insofar as such claim is based
 
                                        7
<PAGE>   6
 
        upon, arises out of or results from information developed or certified
        by Taylor for use in connection with the registration statement or
        arises out of or results from the omission of information known to
        Taylor prior to the violation or alleged violation; provided, that the
        maximum amount of liability in respect of such indemnification shall be
        limited, in the case of the Company, to an amount equal to the net
        proceeds actually received by the Company from the sale of such shares
        effected pursuant to such registration. Taylor agrees to indemnify and
        hold harmless the Company, its officers and directors, and each person,
        if any, who controls or may control the Company within the meaning of
        the Act (the Company, its officers and directors, and any such persons
        also being hereinafter referred to individually in this context as an
        "Indemnified Person" and collectively as "Indemnified Persons") from and
        against all claims based upon, arising out of, or resulting from any
        untrue statement of a material fact contained in the registration
        statement, or any omission to state therein a material fact necessary in
        order to make the statement made therein, in the light of the
        circumstances under which they were made, not misleading, to the extent
        that such claim is based upon, arises out of, or results from
        information developed or certified by Taylor for use in connection with
        the registration statement or arises out of, or results from an omission
        of information known to Taylor prior to the violation or alleged
        violation; provided, that the maximum amount of liability in respect of
        such indemnification shall be limited, in the case of Taylor, to an
        amount equal to the net proceeds actually received by Taylor from the
        sale of such shares effected pursuant to such registration. The
        indemnification set forth herein shall be in addition to any liability
        the Company or Taylor may otherwise have to the Indemnified Persons.
        Promptly after actually receiving definitive notice of any claim in
        respect of which an Indemnified Person may seek indemnification under
        this Section 6(f), such Indemnified Person shall submit written notice
        thereof to either the Company or Taylor, as the case may be (sometimes
        being hereinafter referred to as the "Indemnifying Person"). The
        omission of the Indemnified Person to so notify the Indemnifying Person
        of any such claim shall not relieve the Indemnifying Person from any
        liability it may have hereunder except to the extent that (i) such
        liability was caused or increased by such omission, or (ii) the ability
        of the Indemnifying Person to reduce such liability was materially
        adversely affected by such omission. In addition, the omission of the
        Indemnified Person to notify the Indemnifying Person of any such claim
        shall not relieve the Indemnifying Person from any liability it may have
        otherwise have hereunder. The Indemnifying Person shall have the right
        to undertake, by counsel or representatives of its own choosing, the
        defense, compromise or settlement (without admitting liability of the
        Indemnified Person) of any such claim asserted, such defense, compromise
        or settlement to be undertaken at the expense and risk of the
        Indemnifying Person, and the Indemnified Person shall have the right to
        engage separate counsel, at its own expense, whom counsel for the
        Indemnifying Person shall keep informed and consult with in a reasonable
        manner. In the event the Indemnifying Person shall elect not to
        undertake such defense by its own representatives, the Indemnifying
        Person shall give prompt written notice of such election to the
        Indemnified Person, and the Indemnified Person shall undertake the
        defense, compromise and settlement (without admitting liability of the
        Indemnified Person) thereof on behalf of and for the account and risk of
        the Indemnifying Person by counsel or other representatives designed by
        the Indemnified Person. In the event that any claim shall arise out of a
        transaction or cover any period or periods wherein the Company and
        Taylor shall each be liable hereunder for part of the liability or
        obligation arising therefrom, then the parties shall, each choosing its
        own counsel and bearing its own expenses, defend such claim, and no
        settlement or compromise of such claim may be made without the joint
        consent or approval of the Company and Taylor. Notwithstanding the
        foregoing, no Indemnifying Person shall be obligated hereunder with
        respect to amounts paid in settlement of any claim if such settlement is
        effected without the consent of such Indemnifying Person (which consent
        shall not be unreasonably withheld).
 
          7. Release of All Claims by Taylor.  As part of the inducement to the
     Company to enter into this Agreement and to consummate the transactions
     provided for herein, and as a material part of the consideration for this
     Agreement, Taylor does for himself and for his heirs, executors,
     administrators and assigns, hereby release, acquit and forever discharge
     Alabama National BanCorporation and any of its subsidiaries; the law firm
     of Maynard, Cooper and Gale, P.C., its shareholders and its predecessor
     firms
 
                                        8
<PAGE>   7
 
     and shareholders; all of the Company's and its subsidiaries' directors,
     officers, shareholders, agents, servants, their heirs, executors,
     administrators and assigns; and all other related persons, firms,
     corporations, associations or partnerships of and from any and all claims,
     actions, causes of action, demands, rights, damages, costs, expenses,
     compensation and liability of whatsoever kind or nature, including, but not
     limited to, claims arising out of the Employment Agreement and Taylor's
     past association with the Company and its subsidiaries. Such full and
     complete release specifically includes, but is not limited to, any and all
     claims which might or could have been asserted by Taylor in connection with
     his employment by the Company and his termination of employment as provided
     for herein. It is the intention of Taylor by this release as of this date
     to give a full, final and complete release of any and all claims Taylor now
     has or might ever have against any released parties arising out of, or in
     any way directly or indirectly connected with, any present or past business
     relationship between Taylor and any released party, except for the right of
     Taylor to enforce the terms and conditions of this Agreement.
 
          8. Release of All Claims by the Company.  Other than any outstanding
     loans from the Company or its subsidiaries to Taylor or James A. Taylor,
     Jr., the Company, for itself and for its successors and assigns, releases,
     acquits and forever discharges Taylor and Taylor's attorneys, James A.
     Taylor, Jr., Esq. and the law firm of Rothgerber, Appel, Powers & Johnson,
     its partners and its predecessor firms and partners, from any and all
     claims, actions, causes of action, demands, rights, damages, costs,
     expenses, compensation and liability of whatsoever kind or nature, whether
     accrued or unaccrued, asserted or unasserted, known or unknown, including,
     but not limited to, claims arising out of the Employment Agreement and
     Taylor's past association with the Company and its subsidiaries. It is the
     intention of the Company to give a full, final and complete release of any
     and all claims that they now have or might ever have against Taylor arising
     out of, or in any way in connection with, any present or past business
     relationship between the Company and any of the released parties, except
     for the right of the releasers to enforce the terms and conditions of this
     Agreement.
 
          9. Lack of Admission.  This Agreement is not to be construed as an
     admission of liability on the part of the party or parties released in this
     Agreement, and each released party denies liability and intends merely to
     avoid litigation and buy peace.
 
          10. Representations and Warranties of Taylor.  As a material part of
     the inducement to the Company to enter into this Agreement and as a part of
     the consideration with respect thereto, Taylor represents and warrants to
     the Company and to each releasee named herein, the following:
 
             (a) Taylor is now, and has at all times during the negotiations
        which resulted in this Agreement been, represented by independent legal
        counsel of his choosing;
 
             (b) Taylor is now, and has at all times during the negotiations
        which resulted in this Agreement, consulted with such independent legal
        counsel of his choosing;
 
             (c) the terms and conditions of this Agreement are transactions
        which Taylor has determined are in his best interest based upon the
        exercise of independent business judgment by Taylor after consultation
        with such independent attorneys, accountants or other financial
        consultants or advisors as Taylor deems necessary to form the basis for
        such business judgment;
 
             (d) Taylor covenants and warrants that he has not heretofore
        assigned or transferred to any other person or entity any legal or
        equitable ownership in or claims, actions or causes of action, to be
        released by Taylor by the terms of this Agreement; and
 
             (e) the parties have been involved in contentious negotiations with
        respect to the matters addressed by this Agreement and various and
        sundry claims have been asserted by the parties against each other in
        connection therewith. Taylor is represented by James A. Taylor, Jr.,
        Esq. and William P. Johnson, Esq., upon whom Taylor has relied as to the
        execution and consummation of this Agreement. None of the parties hereto
        in an adversary posture repose any trust or confidence in the adversary
        parties. Taylor hereby waives, forgives and relinquishes any obligations
        arising from any duties owed by the Company or any of its officers,
        directors or shareholders to Taylor which obligations are not expressly
        set forth herein as a part of this Agreement.
 
                                        9
<PAGE>   8
 
          The warranties and representations of Taylor shall survive the
     consummation of the transactions provided for in this Agreement.
 
          11. Confidentiality.  The parties agree that any and all financial
     terms of this settlement, including without limitation any amounts paid in
     this Agreement, shall remain confidential and shall not be disclosed to any
     party other than the parties to this Agreement, absent a court order or
     applicable law or regulation (including the securities laws and
     regulations) to the contrary. The parties, however, agree that each party
     may disclose the financial terms of this settlement and any amounts paid
     pursuant to this Agreement to their bookkeeper(s), accountant(s) and the
     person(s) preparing their state and federal income tax returns, as well as
     to any governmental official or agency that may audit such party's books or
     records. In the event of such disclosure, the party to whom such
     information is disclosed shall be made aware of this confidentiality
     provision and this Agreement. The parties acknowledge that this
     confidentiality provision agreement became effective as of the date of this
     Agreement.
 
          12. Additional Documents.  Taylor agrees for himself and for his
     heirs, executors, administrators and assigns to execute, upon request of
     the Company, all such other further, or different documents which shall be
     reasonably necessary in the opinion of counsel for the Company to carry out
     the provisions of this Agreement.
 
          13. Entire Agreement.  This document contains the entire agreement
     existing between the parties and all prior or contemporaneous agreements
     are merged herein. Taylor expressly agrees that there is no money or other
     benefit or compensation due him other than as expressly stated herein.
     Taylor expressly agrees that there are no oral promises, representations or
     inducements relied upon by him as a basis for executing this Agreement.
 
          14. Fees and Expenses.  The Company shall pay all legal fees and
     related expenses (included without limitation the costs of experts,
     accountants and counsel) incurred by Taylor as a result of Taylor
     successfully seeking to obtain or enforce any right or benefit provided for
     by this Agreement.
 
          15. No Third Party Beneficiaries.  Nothing in this Agreement, express
     or implied, is intended to confer upon any person, other than the parties
     hereto and their heirs, successors and assigns, any rights or remedies
     under or by reason of this Agreement.
 
          16. Counterparts.  This Agreement may be executed in several
     counterparts, each of which so executed shall be deemed to be an original,
     and such counterparts shall together constitute and be one and the same
     instrument.
 
          17. Addresses for Notices, Etc.  All notices, requests, demands,
     consents and other communications provided for hereunder and under the
     related documents shall be in writing and shall be deemed to have been duly
     given when delivered by hand, by facsimile transmission (confirmed in
     writing) or by registered or certified mail, postage prepaid, to such party
     at its address set forth below or such other address as such party may
     specify by notice to the parties hereto:
 
<TABLE>
        <S>                    <C>
        If to Taylor to:       James A. Taylor
                               7 Turnberry Place
                               Shoal Creek, Alabama 35242
 
        Copy to:               James A. Taylor, Jr.
                               1035 Sims Avenue
                               Birmingham, Alabama 35213
 
        If to the Company to:  Alabama National BanCorporation
                               1927 First Avenue North
                               Birmingham, Alabama 35203
                               Attention: Chief Executive Officer
</TABLE>
 
                                       10
<PAGE>   9
        Copy to:               Mark L. Drew
                               Maynard, Cooper & Gale, P.C.
                               1901 Sixth Avenue North
                               2400 AmSouth/Harbert Plaza
                               Birmingham, Alabama 35203
 
          18. Applicable Law.  This Agreement shall be construed and the legal
     relations between the parties determined in accordance with Title 9 of the
     U.S. Code and the laws of the State of Alabama, without regard to
     principles of conflicts of law and is intended to take effect as an
     instrument under seal.
 
          19. Arbitration.  Taylor and the Company acknowledge and agree that
     this Agreement shall be performed in substantial interstate commerce. Any
     controversy or claim arising out of or relating to this Agreement, or the
     breach thereof, shall be resolved by binding arbitration in the City of
     Birmingham, State of Alabama, in accordance with Title 9 of the U.S. Code
     and the Commercial Arbitration Rules of the American Arbitration
     Association (the "AAA"). Whenever an arbitration is required hereunder, the
     arbitrator shall be selected in accordance with the Commercial Arbitration
     Rules of the AAA. The AAA shall designate a panel of ten (10) potential
     arbitrators knowledgeable in the subject matter of the dispute. Each of
     Taylor and the Company shall designate, within thirty (30) days of the
     receipt of the list of potential arbitrators, one of the potential
     arbitrators to serve, and the two arbitrators so designated shall select a
     third arbitrator from the eight remaining potential arbitrators. The panel
     of three (3) arbitrators shall determine the resolution of the dispute.
 
     IN WITNESS WHEREOF, the undersigned have hereunto set their hands and seals
as of the date first above written.
 
<TABLE>
<S>                                              <C>
/s/ JAMES A. TAYLOR, JR.                         /s/ JAMES A. TAYLOR JR.
---------------------------------------------    ---------------------------------------------
Witness                                          James A. Taylor
 
/s/ MARTHA W. TAYLOR
---------------------------------------------
Witness
 
ATTEST:                                          ALABAMA NATIONAL BANCORPORATION
 
By: /s/ FRANK W. WHITEHEAD                       By: /s/ JOHN H. HOLCOMB, III
---------------------------------------------    -----------------------------------------
  Its: Executive Vice President                    Its: President
</TABLE>
 
                                       11