<PAGE>   1

                                                                     Exhibit 4.1


                           BANK OF GRANITE CORPORATION
                        2001 INCENTIVE STOCK OPTION PLAN


                                    ARTICLE 1

                            PURPOSE AND TERM OF PLAN

         1.1 Purpose. The purposes of the Plan are to aid Bank of Granite
Corporation ("Granite") and its subsidiaries (Granite and its subsidiaries being
referred to as the "Company") in attracting and retaining Key Employees and
motivating such Key Employees to put forth maximum efforts toward the continued
growth, profitability and success of the Company by providing incentives to such
Key Employees through the ownership of options to purchase Common Stock of
Granite. Toward this objective, the Committee (as defined in Section 2.8 hereof)
may grant incentive stock options to Key Employees of the Company on the terms
and subject to the conditions set forth in the Plan.

         1.2 Term. The Plan shall become effective as of January 8, 2001,
subject to its approval by Granite's shareholders at the 2001 Annual Meeting of
shareholders. No options granted hereunder shall be exercisable or payable
before approval of the Plan has been obtained from Granite's shareholders.
Options shall not be granted pursuant to the Plan after January 8, 2011.

                                    ARTICLE 2

                                   DEFINITIONS

         2.1 Board. "Board" means the Board of Directors of Granite.

         2.2 Cause. "Cause" means (a) the willful and continued failure by a Key
Employee to substantially perform his or her duties with his or her employer
after written warnings identifying the lack of substantial performance are
delivered to the Key Employee by his or her employer to specifically identify
the manner in which the employer believes that the Key Employee has not
substantially performed his or her duties, or (b) the willful engaging by an
Employee in illegal or dishonest conduct which is materially and demonstrably
injurious to Granite or a Subsidiary.

         2.3 CEO. "CEO" means the Chief Executive Officer of Granite.

         2.4 Change In Control. "Change in Control" means any of the following:

         2.4.1    A tender offer or exchange offer is made whereby the effect of
                  such offer would be to take over and control the affairs of
                  Granite and such offer is consummated for the ownership of
                  securities of Granite representing 50% or more of the combined
                  voting power of Granite's then-outstanding voting securities;

         2.4.2    Granite is merged or consolidated with another corporation
                  and, as a result of such merger or consolidation, outstanding
                  securities representing less than 50% of the voting power of
                  the surviving or resulting corporation shall then be owned in
                  the aggregate by the former shareholders of Granite other than
                  affiliates (within the meaning of the Exchange Act) of any
                  party to such merger or consolidation;

         2.4.3    Granite transfers all or substantially all of its assets to
                  another corporation or entity that is not a wholly owned
                  Subsidiary of Granite;


                             Bank of Granite Corporation, Form S-8, Page 8 of 23

<PAGE>   2


         2.4.4    Any "person" (as such term is used in Sections 3(a)(9) and
                  13(d)(3) of the Exchange Act) is or becomes the beneficial
                  owner, directly or indirectly, of securities of Granite
                  representing 50% or more of the combined voting power of
                  Granite's then-outstanding securities; or

         2.4.5    As the result of a tender offer, merger, consolidation, sale
                  of assets, contested election, or any combination of such
                  transactions, the persons who were members of the Board
                  immediately before the transaction cease to constitute at
                  least a majority thereof.

         2.5 Change In Control Price. "Change In Control Price" means the
highest closing price per share paid for the purchase of Common Stock during the
ninety (90) day period ending on the date the Change In Control occurs on the
primary securities exchange on which the Common Stock is then traded, or the
merger or tender price if higher.

         2.6 Change In Ownership. "Change In Ownership" means a Change In
Control which results directly or indirectly in the Common Stock ceasing to be
actively traded on the primary securities exchange or quotation system on which
the Common Stock was traded immediately before such Change in Control.

         2.7 Code. "Code" means the Internal Revenue Code of 1986, as amended
from time to time, including regulations thereunder and successor provisions and
regulations thereto.

         2.8 Committee. "Committee" means the Compensation Committee of the
Board, or such other Board committee as may be designed by the Board to
administer the Plan; provided that the Committee shall consist of two or more
directors, each of whom is both a "non-employee director" within the meaning of
Rule 16b-3 under the Exchange Act and an "outside director" within the meaning
of the definition of such term as contained in Treasury Regulation Section
1.162-27(e)(3), or any successor definition adopted.

         2.9 Common Stock. "Common Stock" means Common Stock, $1.00 par value
per share, of Granite which may be newly issued shares, shares issued and
outstanding, treasury stock or shares owned by a Subsidiary.

         2.10 Company. "Company" means Granite and its Subsidiaries.

         2.11 Covered Employee. "Covered Employee" means an Employee who is a
"Covered Employee" within the meaning of Section 162(m) of the Code.

         2.12 Disability. "Disability" means a disability under the terms of any
long-term disability plan maintained by the Company.

         2.13 Effective Date. "Effective Date" means the date an Option Award is
determined to be effective by the Committee upon its grant of such Option Award.

         2.14 Employee. "Employee" means an employee of Granite or its
Subsidiaries.

         2.15 Exchange Act. "Exchange Act" means the Securities Exchange Act of
1934, as amended from time to time, including rules thereunder and successor
provisions and rules thereto.

         2.16 Granite. "Granite" means Bank of Granite Corporation.

         2.17 Key Employee. "Key Employee" means an Employee who holds a
position of responsibility in a managerial, administrative, or professional
capacity.


                             Bank of Granite Corporation, Form S-8, Page 9 of 23

<PAGE>   3


         2.18 NASDAQ. "NASDAQ" means the National Association of Dealers
Automated Quotation System.

         2.19 Option Award. "Option Award" means incentive stock options granted
under the Plan to a Participant by the Committee pursuant to such terms,
conditions, restrictions and/or limitations, if any, as the Committee may
establish by the Option Notice or otherwise.

         2.20 Option Notice. "Option Notice" means a written notice from the
Company to a Participant that establishes the terms, conditions, restrictions
and/or limitations applicable to an Option Award in addition to those
established by this Plan and by the Committee's exercise of its administrative
powers.

         2.21 Participant. "Participant" means any Key Employee to whom an
Option Award has been granted by the Committee under the Plan.

         2.22 Plan. "Plan" means the Bank of Granite Corporation 2001 Stock
Option Plan.

         2.23 Subsidiary. "Subsidiary" means a corporation or other business
entity in which Granite directly or indirectly has an ownership interest of 80
percent or more.


                                    ARTICLE 3

                       ELIGIBILITY AND PLAN ADMINISTRATION

         3.1 In General. Only Key Employees are eligible to participate in the
Plan. The Committee shall select, from time to time, Key Employees who, in the
opinion of the Committee, can further the Plan's purposes. Once a Key Employee
is so selected, the Committee shall determine the Option Awards to be made to
the Participant and shall establish in the related Option Notice the terms,
conditions, restrictions and/or limitations, if any, applicable to the Option
Awards in addition to those set forth in this Plan and the administrative rules
and regulations issued by the Committee.

         3.2 Responsibility. The Committee shall have total and exclusive
responsibility to control, operate, manage and administer the Plan in accordance
with its terms.

         3.3 Authority of the Committee. The Committee shall have all the
authority that may be necessary or helpful to enable it to discharge its
responsibilities with respect to the Plan. Without limiting the generality of
the preceding sentence, the Committee shall have the exclusive right to: (a)
interpret the Plan; (b) determine eligibility for the participation in the Plan;
(c) decide all questions concerning the amount of Option Awards payable under
the Plan; (d) construe any ambiguous provision of the Plan; (e) correct any
default; (f) supply any omission; (g) reconcile any inconsistency; (h) issue
administrative guidelines as an aid to administer the Plan and make changes in
such guidelines as it from time to time deems proper; (i) make regulations for
carrying out the Plan and make changes in such regulations as it from time to
time deems proper; (j) to the extent permitted under the Plan, grant waivers of
Plan terms, conditions, restrictions and limitations; (k) accelerate the
vesting, exercise, or payment of an Option Award when such action or actions
would be in the best interest of the Company; (l) subject to Section 6.2, grant
Option Awards in replacement of Option Awards previously granted under this
Plan; and (m) take any and all other action it deems necessary or advisable for
the proper operation or administration of the Plan.

         3.4 Discretionary Authority. The Committee shall have full
discretionary authority in all matters related to the discharge of its
responsibilities and the exercise of its authority under the Plan including,
without limitation, its construction of the terms of the Plan and its
determination of eligibility for participation in Option Awards under the Plan.
It is the intent of the Plan that the decisions of the Committee and its action
with respect to the Plan shall be final, binding and conclusive upon all persons
having or claiming to have any right or interest in or under the Plan.


                            Bank of Granite Corporation, Form S-8, Page 10 of 23

<PAGE>   4


         3.5 Section 162(m) of the Code. With regard to all Covered Employees,
the Plan shall, for all purposes, be interpreted and construed in accordance
with Section 162(m) of the Code.

         3.6 Action by the Committee. The Committee may act only by a majority
of its members. Any determination of the Committee may be made, without a
meeting, by a writing or writings signed by all of the members of the Committee.
In addition, the Committee may authorize only one or more of its number to
execute and deliver documents on behalf of the Committee.

         3.7 Delegation of Authority. The Committee may delegate some or all of
its authority under the Plan to any person or persons provided that any such
delegation be in writing; provided, however, that only the Committee may select
and grant Option Awards to Participants who are subject to Section 16 of the
Exchange Act or are Covered Employees.


                                    ARTICLE 4

                             FORMS OF OPTION AWARDS

         4.1 In General. All Option Awards shall be subject to the terms,
conditions, restrictions and limitations of the Plan. The Committee may, in its
sole judgment, subject an Option Award to such other terms, conditions,
restrictions and/or limitations (including, but not limited to, the time and
conditions of exercise and restrictions on transferability and vesting),
provided they are not inconsistent with the terms of the Plan. Option Awards
under the Plan need not be uniform. Any combination of Option Awards may be
granted at one time and on more than one occasion to the same Employee.


                                    ARTICLE 5

                             SHARES SUBJECT TO PLAN

         5.1 Available Shares. The maximum number of shares of Common Stock
available for grant of Option Awards under the Plan during its term is 200,000.
(Such amount shall be subject to adjustment as provided in Section 5.2.) Any
shares of Common Stock related to Option Awards which, without the issuance of
such shares, are settled in cash in lieu of Common Stock, shall not be available
again for grant under the Plan. Any shares of Common Stock related to Option
Awards which terminate by expiration, forfeiture or cancellation shall be
available again for grant under the Plan.

         5.2 Adjustment to Shares.

         (a) In General. The provisions of this Subsection 5.2(a) are subject to
the limitation contained in Subsection 5.2(b). If there is any change in the
number of outstanding shares of Common Stock through the declaration of stock
dividends, stock splits or the like, the number of shares available for Option
Awards, the shares subject to any Option Award and the option prices or exercise
prices of Option Awards shall be automatically adjusted. If there is any change
in the number of outstanding shares of Common Stock through any change in the
capital account of Granite, or through a merger, consolidation, separation
(including a spin-off or other distribution of stock or property),
reorganization (whether or not such reorganization comes within the meaning of
such term in Section 368(a) of the Code) or partial or complete liquidation, the
Committee shall make appropriate adjustments in the maximum number of shares of
Common Stock which may be issued under the Plan and to any Participant and any
adjustments and/or modifications to outstanding Option Awards as it, in its sole
discretion, deems appropriate. In event of any other change in the capital
structure or in the Common Stock, the Committee shall also be authorized to make
such appropriate adjustments in the maximum number of shares of Common Stock
available for issuance under the Plan and to any Participant and any adjustments
and/or modifications to outstanding Option Awards as it, in its sole discretion,
deems appropriate. The maximum


                            Bank of Granite Corporation, Form S-8, Page 11 of 23

<PAGE>   5


number of shares available for issuance under the Plan shall be automatically
adjusted to the extent necessary to reflect any dividend equivalents paid in the
form of Common Stock; and

         (b) Covered Employees. In no event shall the Option Award of any
Participant who is a Covered Employee be adjusted pursuant to Subsection 5.2(a)
to the extent it would cause such Option Award to fail to qualify as
"Performance-Based Compensation" under Section 162(m) of the Code.

                                    ARTICLE 6

                                  STOCK OPTIONS

         6.1 In General. Option Awards are granted to Employees in the form of
stock options which shall be incentive stock options within the meaning of
Section 422 of the Code. All Option Awards under the Plan issued to Covered
Employees shall qualify as "Performance-Based Compensation" under Section 162(m)
of the Code.

         6.2 Terms and Conditions of Stock Options. An option shall be
exercisable in whole or in such installments and at such times as may be
determined by the Committee. The price at which Common Stock may be purchased
upon exercise of a stock option shall be not less than 100% of the fair-market
value of the Common Stock, as determined by the Committee, on the Effective Date
of the option's grant. Moreover, all options shall expire not later than 10
years from the Effective Date of the option's grant. Stock options shall not be
repriced, i.e., there shall be no grant of a stock option(s) to a Participant in
exchange for a Participant's agreement to cancellation of a higher-priced stock
option that was previously granted to such Participant. Stock Option Awards are
intended to comply with Section 422 of the Code.

         6.3 Additional Terms and Conditions. The Committee may, by way of the
Option Notice or otherwise, establish such other terms, conditions, restrictions
and/or limitations, if any, for any Option Award, provided they are not
inconsistent with the Plan.

         6.4 Exercise. Upon exercise, the option price of a stock option granted
hereunder must be paid in cash. Subject to Section 10.8, stock options awarded
under the Plan may be exercised by way of the Company's broker-assisted exercise
program, provided such program is available at the time of the option's
exercise.

         6.5 Maximum Option Award Payable. Notwithstanding any provision
contained in the Plan to the contrary, the maximum number of shares for which
stock options may be granted under the Plan to any one Participant during any
calendar year is 5,000 shares of Common Stock.


                                    ARTICLE 7

                  TERMINATION OF EMPLOYMENT AND NONCOMPETITION

         7.1 Termination of Employment. Options granted under the Plan that
remain outstanding on the date of a Participant's termination of employment with
the Company shall cease to be exercisable and shall terminate and be forfeited
on the date of such Participant's termination, except as otherwise provided in
this Section 7.1 and Articles 8 and 9. To the extent permitted under Rule 16b-3,
on the date of a Participant's termination of employment with the Company
because of death or Disability, all of such Participant's unexercised stock
options granted under this Plan shall remain exercisable, to the extent such
options were exercisable as of the time of termination (including any
exercisability resulting from such termination), for an additional period of one
year after the date of such Participant's termination (but in no event beyond
the original expiration date of such options). If a Participant's employment
with the Company terminates for any other reason, other than for Cause, all of
such Participant's unexercised stock options granted under this Plan shall
remain exercisable for a period of 90 days after the Participant's termination
(but not beyond the original expiration date of such options) to the same extent
as they were exercisable as of the time of Participant's termination (including
any


                            Bank of Granite Corporation, Form S-8, Page 12 of 23

<PAGE>   6


exercisability resulting from such termination). If the Participant's employment
is terminated for Cause, the option held by such Participant shall cease to be
exercisable, shall terminate and be forfeited on the date of Participant's
termination of employment.

         7.2 Noncompetition. Unless the Option Notice specifies otherwise, a
Participant shall forfeit all unexercised Option Awards, including, but not by
way of limitation, Option Awards earned but not yet paid, all unpaid dividends
and dividend equivalents and all interest, if any, accrued on the foregoing if,
(i) in the opinion of the Committee, the Participant, without the prior written
consent of Granite, engages directly or indirectly in any manner or capacity as
principal, agent, partner, officer, director, stockholder, employee, or
otherwise, in any business or activity competitive with the business conducted
by Granite or any Subsidiary; (ii) at any time divulges to any person or any
entity other than the Company any trade secrets, methods, processes or the
proprietary or confidential information of the Company; or (iii) the Participant
performs any act or engages in any activity which in the opinion of the CEO is
inimical to the best interests of the Company. For purposes of this Section 7.2,
a Participant shall not be deemed a stockholder if the Participant's record and
beneficial ownership amount to not more than 5% of the outstanding capital stock
of any company subject to the periodic and other reporting requirements of the
Exchange Act.


                                    ARTICLE 8

                               CHANGE IN OWNERSHIP

         8.1 Background. Notwithstanding any provision contained in the Plan,
including, but not limited to, Section 3.3, the provisions of this Article 8
shall control over any contrary provision. Upon a Change In Ownership: (i) the
terms of this Article 8 shall immediately become operative, without further
action or consent by any person or entity; (ii) all terms, conditions,
restrictions and limitations in effect on any unexercised and/or deferred Option
Award, or any other outstanding Option Award, shall immediately lapse as of the
date of such event; (iii) no other terms, conditions, restrictions and/or
limitations shall be imposed upon any Option Awards on or after such date, and
in no circumstance shall an Option Award be forfeited on or after such date; and
(iv) all unexercised Option Awards or any other outstanding Option Awards shall
automatically become one hundred percent (100%) vested immediately.

         8.2 Valuation of Option Awards. Upon a Change In Ownership, all
outstanding stock options (including incentive stock options) shall be valued
and cashed out on the basis of the Change In Control Price.

         8.3 Payment of Option Awards. Upon a Change In Ownership, any
Participant, whether or not he or she is still employed by the Company, shall be
paid, in a single lump sum cash payment, as soon as practicable but in no event
later than 90 days after the Change In Ownership, the aggregate value (using the
Change In Control Price) of all of his or her stock options (including incentive
stock options) awarded under this Plan; provided that any options having an
exercise price greater than or equal to the Change in Control Price shall have a
value of zero, shall be cancelled, and the owner thereof shall not be entitled
to any payment.

         8.4 Miscellaneous. Upon a Change In Ownership, (i) the provisions of
Sections 7.1, 7.2 and 10.2 hereof shall become null and void and of no further
force and effect; and (ii) no action, including, but not by way of limitation,
the amendment, suspension, or termination of the Plan, shall be taken which
would affect the rights of any Participant or the operation of the Plan with
respect to any Option Award to which the Participant may have become entitled
hereunder on or prior to the date of such action or as a result of such Change
In Ownership.


                            Bank of Granite Corporation, Form S-8, Page 13 of 23


<PAGE>   7



                                    ARTICLE 9

                                CHANGE IN CONTROL

         9.1 Background. Notwithstanding any provision contained in the Plan,
including, but not limited to, Section 3.3, the provisions of this Article 9
shall control over any contrary provision. A Participant shall be eligible for
the treatment afforded by this Article 9 if his or her employment terminates
within two years following a Change in Control, unless the termination is for
Cause.

         9.2 Vesting and Lapse of Restrictions. If a Participant is eligible for
treatment under this Article 9, (i) all of the terms, conditions, restrictions
and limitations in effect on any of his or her unexercised and/or deferred
Option Awards shall immediately lapse as of the date of his or her termination
of employment, (ii) no other terms, conditions, restrictions and/or limitations
shall be imposed upon any of his or her Option Awards on or after such date, and
in no event shall any of his or her Option Awards be forfeited on or after such
date; and (iii) all of his or her unexercised Option Awards shall automatically
become one hundred percent (100%) vested immediately upon his or her termination
of employment.

         9.3 Valuation of Option Awards. If a Participant is eligible for
treatment under this Article 9, his or her Option Award shall be valued and
cashed out in accordance with the provisions of Section 8.2.

         9.4 Payment of Option Awards. If a Participant is eligible for
treatment under this Article 9, he or she shall be paid, in a single lump sum
cash payment, as soon as practicable but in no event later than 90 days after
the date of his or her termination of employment, the aggregate value (using the
Change in Control Price) of all of his or her stock options (including incentive
stock options) granted under the Plan; provided that any options having an
exercise price greater than or equal to the Change in Control Price shall have a
value of zero, shall be cancelled, and the owner thereof shall not be entitled
to any payment.

         9.5 Miscellaneous. Upon a Change In Control, (i) the provisions of
Sections 7.1, 7.2 and 10.2 hereof shall become null and void and of no force and
effect insofar as they apply to a Participant who has been terminated under the
conditions described in Section 9.1 above, and (ii) no action, including, but
not by way of limitation, the amendment, suspension or termination of the Plan,
shall be taken which would affect the rights of any Participant or the operation
of the Plan with respect to any Option Award to which the Participant may have
become entitled hereunder on or prior to the date of the Change In Control or to
which he or she may become entitled as a result of such Change In Control.

         9.6 Legal Fees. Granite shall pay all legal fees and related expenses
incurred by a Participant in seeking to obtain or enforce any payment, benefit
or right he or she may be entitled to under the Plan after a Change In Control;
provided, however, the Participant shall be required to repay any such amounts
to Granite to the extent a court of competent jurisdiction issues a final and
non-appealable order setting forth the determination that the position taken by
the Participant was frivolous or advanced in bad faith.


                                   ARTICLE 10

                                  MISCELLANEOUS

         10.1 Nonassignability. No Option Awards shall be subject in any manner
to alienation, anticipation, sale, transfer (except by will or the laws of
descent and distribution or pursuant to a qualified domestic relations order as
defined by the Code), assignment, pledge, or encumbrance. During the lifetime of
an optionee, options granted pursuant to the Plan shall be exercisable only by
the optionee personally or the optionee's legal representative.


                            Bank of Granite Corporation, Form S-8, Page 14 of 23

<PAGE>   8


         10.2 Amendments to Option Awards. The Committee may at any time
unilaterally amend any unexercised Option Award, including, but not by way of
limitation, Option Awards earned but not yet paid, to the extent it deems
appropriate; provided, however, that any such amendment which, in the opinion of
the Committee, is adverse to the Participant shall require the Participant's
consent.

         10.3 Regulatory Approvals and Listings. Notwithstanding anything
contained in this Plan to the contrary, the Company shall have no obligation to
issue or deliver certificates of Common Stock evidencing Stock Option Awards or
any other Option Award resulting in the payment of Common Stock prior to (i) the
obtaining of any approval from any governmental agent which the Company shall,
in its sole discretion, determine to be necessary or advisable, (ii) the
admission of such shares to listing on the stock exchange on which the Common
Stock may be listed and (iii) the completion of any registration or other
qualification of said shares under any state or Federal law or ruling of any
governmental body which the Company shall, in its sole discretion, determine to
be necessary or advisable.

         10.4 No Right to Continued Employment or Grants. Participation in the
Plan shall not give any Employee any right to remain in the employ of Granite or
any Subsidiary. Granite, or, in the case of employment with a Subsidiary, the
Subsidiary, reserves the right to terminate any Employee at any time. Further,
the adoption of this Plan shall not be deemed to give any Employee or any other
individual any right to be selected as a Participant or to be granted an Option
Award.

         10.5 Amendment/Termination. The Committee may suspend or terminate the
Plan at any time with or without prior notice. In addition, the Committee may,
from time to time and with or without prior notice, amend the Plan in any
manner, but may not without shareholder approval adopt any amendment which would
require the vote of the shareholders of Granite pursuant to Section 16 of the
Exchange Act or Section 162(m) of the Code, but only insofar as such amendment
affects Covered Employees.

         10.6 Governing Law. The Plan shall be governed by and construed in
accordance with the laws of the State of North Carolina, except as superseded by
applicable Federal Law.

         10.7 No Right, Title, or Interest in Company Assets. No Participant
shall have any rights as a shareholder as a result of participation in the Plan
until the date of issuance of a stock certificate in his or her name. To the
extent any person acquires a right to receive payments from the Company under
the Plan, such rights shall be no greater than the rights of an unsecured
creditor of the Company and the Participant shall not have any rights in or
against any specific assets of the Company. All of the Option Awards granted
under the Plan shall be unfunded.

         10.8 Section 16 of the Exchange Act. In order to avoid any Exchange Act
violations, the Committee may, from time to time, impose additional restrictions
upon an Option Award.

         10.9 No Guarantee of Tax Consequences. No person connected with the
Plan in any capacity, including, but not limited to, Granite and its
Subsidiaries and their directors, officers, agents and employees makes any
representation, commitment, or guarantee that any tax treatment, including, but
not limited to, Federal, state and local income, estate and gift tax treatment,
will be applicable with respect to amounts deferred under the Plan, or paid to
or for the benefit of a Participant under the Plan, or that such tax treatment
will apply to or be available to a Participant on account of participation in
the Plan.

         10.10 Other Benefits. No Option Award granted under the Plan shall be
considered compensation for purposes of computing benefits under any retirement
plan of the Company nor affect any benefits or compensation under any other
benefit or compensation plan of the Company now or subsequently in effect.


                            Bank of Granite Corporation, Form S-8, Page 15 of 23