<PAGE>   1
                                                                    EXHIBIT 10.2




                                 BANK OF GRANITE

                     SUPPLEMENTAL EXECUTIVE RETIREMENT PLAN



This Supplemental Executive Retirement Plan ("Plan") is adopted effective
December 12, 1994. The Plan is established and maintained by the Bank of Granite
for the purpose of providing benefits for certain of its salaried employees who
participate in the Bank of Granite Profit Sharing Plan ("Qualified Plan") and
whose benefits under such Qualified Plan are limited by the Internal Revenue
Code.

Accordingly, Employer hereby adopts the Plan pursuant to the terms and
provisions set forth below.

                                    ARTICLE 1
                                   DEFINITIONS

Whenever used herein the following terms shall have the meanings hereinafter set
forth. Words in the masculine gender shall include the feminine and the singular
shall include the plural, and vice versa, unless qualified by the context. Any
headings used herein are included for ease of reference only, and are not to be
construed so as to alter the terms hereof.

1.1.     "BENEFICIARY" shall mean the person or persons Participant has
designated in writing to Employer to receive benefits under the Plan in the
event of the Participant's death. If the Participant has not specifically
designated any Beneficiary for purposes of the Plan, then the beneficiary chosen
by the Participant for purposes of the Qualified Plan shall become Beneficiary
for purposes of the Plan. In the event no Beneficiary is named for purposes of
either the Plan or Qualified Plan, then the Beneficiary shall become the
Participant's estate.

1.2.     "BOARD" shall mean the Board of Directors of The Bank of Granite.

1.3.     "CODE" shall mean the Internal Revenue code of 1986, as amended.

1.4.     "EMPLOYER" shall mean the Bank of Granite, or to the extent provided in
Section 7.9, any successor corporation or other entity resulting from a merger
or consolidation into or with Employer or a transfer or sale of substantially
all of the assets of Employer.

1.5.     "PARTICIPANT" shall mean a salaried employee of Employer who is a
participant in the Qualified Plan (or any successor or replacement retirement
plan) and to whom or with respect to whom a benefit is payable under the Plan.


Exhibit 10.2                         Page 1
Bank of Granite Corporation, Form 10-K, December 31, 2000, Page 168


<PAGE>   2




1.6.     "PLAN" shall mean the Bank of Granite Supplemental Executive Retirement
Plan.

1.7.     "QUALIFIED PLAN" shall mean the Bank of Granite Profit Sharing Plan,
and each predecessor, successor or replacement tax-qualified cash or deferred
arrangement sponsored by Employer.

1.8.     "QUALIFIED PLAN DISCRETIONARY CONTRIBUTIONS" shall mean the total of
all discretionary contributions, if any, made by Employer during a calendar year
to the Qualified Plan for the Benefit of all employees of the Employer.

1.9.     "SUPPLEMENTAL EMPLOYER DISCRETIONARY CONTRIBUTION" shall mean the
discretionary contribution made by the Employer to the Plan on behalf of the
Participant on account of Code limitations applicable to the Participant under
the Qualified Plan.

                                   ARTICLE II
                                   ELIGIBILITY

2.1.     ELIGIBILITY. Any Employee eligible to participate in the Qualified Plan
shall be eligible to participate in the Plan. Participation shall be initially
be determined by the Board and shall continue thereafter in accordance with the
terms of the Plan.

                                   ARTICLE III
                            CONTRIBUTIONS TO THE PLAN

3.1.     PURPOSE OF CONTRIBUTIONS. Because of Code limitations on Employer
contribution amounts to the Qualified Plan, Employer contributions to any
Participant's DCA under the Plan are intended to provide additional retirement
income to a Participant which may be limited because of such Code limitations.

3.2.     DETERMINATION OF CONTRIBUTIONS. Contributions to the Plan on behalf of
any Participant during any Plan Year shall initially consist of the following
contribution. The determination of any contribution under this Section 3.2 shall
be made during or as soon as reasonably possible following the end of any Plan
Year. The Board shall have complete discretion in its determination of any such
contribution, notwithstanding the completion of annual or quarterly
administration of the Qualified Plan.

         (A)      SUPPLEMENTAL EMPLOYER DISCRETIONARY CONTRIBUTION. The
         Supplemental Employer Discretionary to be made on behalf of any
         Participant during a Plan Year shall be an amount equal to the
         difference between (i) and (ii) below:

                  (i)      The portion of the Qualified Plan Discretionary
                  Contribution which would have been allocated to the
                  Participant's Qualified Plan accounts,


Exhibit 10.2                         Page 2
Bank of Granite Corporation, Form 10-K, December 31, 2000, Page 169


<PAGE>   3


         pursuant to the terms of the Qualified Plan, without giving effect to
         any limitations imposed by Section 415 of the Code, as amended, on the
         Qualified Plan,

         LESS

         (ii)     The amount of the Qualified Plan Discretionary Contribution
         actually allocated to the Participant's Qualified Plan accounts.

3.3.     DISCRETIONARY CONTRIBUTIONS. Notwithstanding the provisions of Section
3.2, the Board may make additional contributions to the Plan on behalf of any
Participant. The amounts of any such additional contributions shall be
determined by the Board in its sole discretion.

3.4.     TIMING OF CONTRIBUTIONS. Contributions to the Plan pursuant to Sections
3.2 or 3.3 of the Plan shall be determined and made by the Board on an annual or
more frequent basis.

                                   ARTICLE IV
                         DEFERRED COMPENSATION ACCOUNTS

4.1.     DEFERRED COMPENSATION ACCOUNTS. For each Participant, Employer shall
create a Deferred Compensation Account ("DCA"). On an annual or more frequent
basis, current contributions to the Plan on behalf of each Participant shall be
allocated to each Participant's DCA, along with any interest or other earnings
on prior contributions which have been allocated to each Participant's DCA.

4.2.     DCA INVESTMENTS. Contributions to each Participant's DCA shall be
invested in one or more interest-bearing accounts determined by the Board and
established and controlled by Employer. Such accounts shall be credited with
interest and other earnings in accordance with normal practice. The type of such
accounts shall be in the sole discretion of the Board. The establishment and
control of such accounts shall be in the sole discretion of Employer.

                                    ARTICLE V
                                  DISTRIBUTIONS

5.1.     DISTRIBUTION UPON TERMINATION OF EMPLOYMENT AT OR AFTER NORMAL
RETIREMENT AGE. All amounts credited to a Participant's DCA, including any
earnings and losses on such DCA, shall normally be distributed to or with
respect to a Participant only upon termination of the Participant's employment
with the Employer at age 65 or later. Should employment be terminated prior to
age 65 (and the Participant does not qualify for Early Retirement), any
distribution will be delayed until the former Participant attains age 65.


Exhibit 10.2                         Page 3
Bank of Granite Corporation, Form 10-K, December 31, 2000, Page 170


<PAGE>   4




Distribution shall commence on the first business day of the third month
following termination of employment (or if termination occurs prior to age 65,
the first business day of the third month following age 65). At the election of
the Participant, distribution may be taken in accordance with either of the
following two methods. The choice of method of distribution is to be made by the
Participant prior to or concurrent with termination of employment with Employer.

         (A)      LUMP SUM DISTRIBUTION. An amount equal to the balance in the
         Participant's DCA on the date of termination of employment shall be
         paid to the Participant in one lump sum distribution on the first
         business day of the third month following the Participant's termination
         of employment.

         (B)      PERIODIC DISTRIBUTION. An amount equal to the balance in the
         Participant's DCA on the date of termination (plus any earnings on such
         DCA during the payment period) shall be payable in up to 10 annual
         installments. The first payment shall commence on the first business
         day of the third month following the date of termination of employment,
         and each successive payment shall occur annually in succeeding years on
         the first business day of such month. Prior to termination of
         employment, the Participant shall elect the number of annual
         installments to be made.

         To determine the amount of each installment payment, a fraction shall
         be applied to the former Participant's DCA balance on each payment
         date. The numerator shall consist of one (1) and the denominator shall
         consist of the total number of installment payments remaining
         (including the current payment). During the installment payment period,
         interest and other earnings shall continue to be allocated to the
         former Participant's DCA.

5.2.     DISTRIBUTION UPON EARLY RETIREMENT. Should a Participant attain age 50
and have 7 years of service with Employer, the Participant will qualify for an
Early Retirement distribution. Upon termination of employment after meeting such
requirements, distribution will be made, at the Participant's election prior to
or concurrent with termination of employment, in accordance with the provisions
of Section 5.1(a) or (b) of the Plan.

5.3.     DISTRIBUTION UPON DISABILITY. Should a Participant become permanently
disabled, the Participant will qualify for a Disability Distribution. Upon
termination of employment due to permanent disability, distribution will be
made, at the Participant's election prior to or concurrent with termination of
employment, in accordance with the provisions of Section 5.1(a) or (b) of the
Plan. For purposes of this Section 5.3, the determination of permanent
disability shall be in the sole discretion of the Board.


Exhibit 10.2                         Page 4
Bank of Granite Corporation, Form 10-K, December 31, 2000, Page 171



<PAGE>   5



5.4.     DEATH OF PARTICIPANT. If a Participant should die before distribution
of the full amount of his or her DCA, distribution of such DCA will be made in
the following manner.

         (a)      DEATH BEFORE TERMINATION OF EMPLOYMENT. Should a Participant
         die before termination of employment, distribution of the Participant's
         DCA will be made to the Participant's Beneficiary. Such distribution,
         the amount of which shall consist of the balance of the Participant's
         DCA at the date of death, will be made in one distribution on the first
         business day of the third month following the date of death.

         (b)      DEATH AFTER TERMINATION OF EMPLOYMENT. Should a Participant
         die after termination of employment, and before the full distribution
         of the Participant's DCA, distribution will continue to be made to the
         Participant's Beneficiary in the same manner as it was previously being
         made to the Participant.

                                   ARTICLE VI
                           ADMINISTRATION OF THE PLAN

6.1.     ADMINISTRATION BY EMPLOYER. Employer shall be responsible for the
general operation and administration of the Plan and for carrying out the
provisions thereof. The Board or Employer may engage the services of outside
counsel, accountants, financial advisors and other such professional to assist
it in its administrative duties.

6.2.     GENERAL POWERS OF ADMINISTRATION. Employer is hereby designated as a
fiduciary under the Plan. Employer, as fiduciary, shall have authority to
control, interpret and manage the operation and administration of the Plan. Any
decision by Employer or the Board denying a claim by a Participant or a
Beneficiary for benefits under the Plan shall be stated in writing and shall be
delivered or mailed to the Participant or Beneficiary. Such statement shall set
forth the specific reasons for the denial, written to the best of the Employer's
ability in a manner that may be understood without legal counsel. In addition,
Employer shall afford a reasonable opportunity to the Participant or Beneficiary
for a full and fair review of the decision denying such claim.

Notwithstanding the above provisions of Section 6.2, to the extent that the
Employee Retirement Income Security Act ("ERISA") may require specific
procedures to be followed in the event of a denial of a claim, such provisions
of ERISA will be followed.


Exhibit 10.2                         Page 5
Bank of Granite Corporation, Form 10-K, December 31, 2000, Page 172


<PAGE>   6


                                   ARTICLE VII
                            AMENDMENT OF TERMINATION

7.1.     AMENDMENT OR TERMINATION. Although Employer intends the Plan to be
permanent, it reserves the right, in its sole discretion, to amend or terminate
the Plan. Any such amendment or termination shall be made pursuant to a
resolution of the Board.

7.2.     EFFECT OF AMENDMENT OR TERMINATION. No amendment or termination of the
Plan shall directly or indirectly reduce the balance of any Participant's DCA
below that existing as of the date of the Board resolution amending or
terminating the Plan. Should the Plan be terminated, no additional Employer
contributions will be made to the Plan, and each Participant's DCA balance on
the date of Plan termination will be distributed to the Participant (or
Participant's Beneficiary) in one lump sum on the first business day of the
third month following the date of Plan termination.

                                  ARTICLE VIII
                               GENERAL PROVISIONS

8.1.     PARTICIPANT'S RIGHTS UNSECURED. The Plan at all times shall be entirely
unfunded and no provision shall at any time be made with respect to segregation
any assets of Employer for payment of any distributions pursuant the Plan. The
right of a Participant or any Beneficiary to receive a distribution hereunder
shall be an uninsured claim against the general assets of Employer, and neither
the Participant nor any Beneficiary shall have any rights in or against any
specific assets of Employer. All amounts credited to Participant's DCAs shall
constitute general assets of Employer and may be disposed of by Employer at such
time and for such purposes as it may deem appropriate.

8.2.     INDEPENDENCE OF OTHER BENEFIT PLANS. Participation in the Plan shall in
no way restrict or otherwise impact a Participant's participation in any other
retirement benefit plan sponsored by Employer.

8.3.     NO GUARANTEE OF BENEFITS. Nothing contained in the Plan shall
constitute a guaranty by Employer or any other person or entity that the assets
of Employer will be sufficient to pay any benefit hereunder.

8.4.     NO ENLARGEMENT OF EMPLOYEE RIGHTS. No Participant shall have any right
to receive a distribution of any contribution made under the Plan except in
accordance with the terms of the Plan. Establishment of the Plan shall not be
construed to give any Participant the right to be retained in the service of
Employer.


Exhibit 10.2                         Page 6
Bank of Granite Corporation, Form 10-K, December 31, 2000, Page 173



<PAGE>   7


8.5.     SPENDTHRIFT PROVISION. No interest of any person or entity in, or right
to receive a distribution under, the Plan shall be subject in any manner to
sale, transfer, assignment, pledge, attachment, garnishment, or other alienation
or encumbrance of any kind; nor may such interest or right to receive a
distribution be taken, either voluntarily or involuntarily for the satisfaction
of the debts of, or other obligations or claims against, such person or entity,
including claims for alimony, support, separate maintenance and claims in
bankruptcy proceedings.

8.6.     APPLICABLE LAW. The Plan shall be construed and administered under the
laws of the State of North Carolina.

8.7.     SEVERABILITY. In the event that any of the provisions of the Plan are
held to be inoperative or invalid by any court of competent jurisdiction, then:
(i) insofar as is reasonable, effect will be given to the intent manifested in
the provision held invalid or inoperative, and (ii) the validity and
enforceability of the remaining provisions of the Plan will not be affected
thereby.

8.8.     INCAPACITY OF RECIPIENT. If any person entitled to a distribution under
the Plan is deemed by Employer to be incapable (physically or mentally) of
personally receiving and giving a valid receipt for any payment pursuant to the
Plan, then, unless and until claim therefore shall have been made by a duly
appointed guardian or other legal representative of such person, Employer may
provide for such payment or any part thereof to be made to any other person or
institution then contributing toward or providing for the care and maintenance
of such person. Any such payment shall be a payment for the account of such
person and a complete discharge of any liability of Employer and the Plan with
respect to such payment.

8.9.     CORPORATE SUCCESSORS. The Plan shall not be automatically terminated by
a transfer or sale of assets of Employer or by the merger or consolidation of
Employer into or with any other corporation or other entity. Should such sale,
merger or consolidation occur, the Plan will be continued unless the transferee,
purchaser or successor entity specifically declines in writing to continue the
Plan. In the event that the Plan is not continued by the transferee, purchaser
or successor entity, then the Plan shall terminate subject to the provisions of
Section 7.2 of the Plan.

8.10.    UNCLAIMED BENEFITS. Each Participant shall keep Employer informed of
his or her current address and the current address of his or her Beneficiary.
Employer shall not be obligated to search for the whereabouts of any person. If
the location of a Participant is not made known to Employer within a one (1)
year period after the date on which payment of the Participant's DCA is first to
be made, then payment may be made by the Employer to the Participant's
Beneficiary instead. If, within one (1) additional year after such initial one
(1) year period, Employer is unable to locate any designated Beneficiary of the
Participant, then Employer shall have no further obligation to pay any benefit
under the Plan to such Participant or designated Beneficiary and any such
benefit shall be irrevocably forfeited.


Exhibit 10.2                         Page 7
Bank of Granite Corporation, Form 10-K, December 31, 2000, Page 174


<PAGE>   8



8.11.    LIMITATIONS ON LIABILITY. Notwithstanding any of the preceding
provisions of the Plan, neither Employer nor any individual acting as employee
or agent of Employer shall be liable to any Participant, former Participant or
other person for any claim, loss, liability or expense incurred in connection
with the Plan.

8.12.    FORFEITURE OF BENEFITS. Notwithstanding any other provision of the
Plan, should a Participant engage in theft, fraud, embezzlement or willful
misconduct causing significant property damage to Employer, then any benefits
payable to such Participant under the Plan will automatically be forfeited. The
determination of theft, embezzlement or willful misconduct will be made by the
Board in good faith, but such determination does not require an actual criminal
indictment or conviction prior to or after such decision. In any determination
of forfeiture pursuant to this Section 8.12, the Participant will be given the
opportunity to refute any such decision by the Board, but the Board's decision
on the matter will be considered final and binding on the Participant and all
other parties.



                                    Adopted by the Board of Directors of the
                                    Bank of Granite on December 12, 1994.



Exhibit 10.2                         Page 8
Bank of Granite Corporation, Form 10-K, December 31, 2000, Page 175