<PAGE>
 
                                                                   EXHIBIT 10.31

                      NON EMPLOYEE DIRECTORS STOCK PLAN OF
                        GRANITE BROADCASTING CORPORATION
                                 APRIL 29, 1997


     1.   PURPOSE.  The purpose of this Non-Employee Directors' Stock Plan (the
          -------                                                              
"Plan") of Granite Broadcasting Corporation (the "Company"), is to advance the
interests of the Company and its stockholders by providing a means to attract
and retain highly qualified persons to serve as non-employee directors of the
Company and to enable such persons to acquire or increase a proprietary interest
in the Company, thereby promoting a closer identity of interests between such
persons and the Company's stockholders.

     2.   DEFINITIONS.  In addition to terms defined elsewhere in the Plan, the
          -----------                                                          
following are defined terms under the Plan:

          (a) "Code" means the Internal Revenue Code of 1986, as amended from
               ----                                                          
time to time.  References to any provision of the Code shall be deemed to
include regulations thereunder and successor provisions and regulations thereto.

          (b) "Disability" means a permanent physical or mental incapacity
               ----------                                                 
which, in the reasonable determination of the Board, renders the Participant
unable to perform his duties as a director of the Company.

          (c) "Fair Market Value" of a Share on a given date shall mean the
               -----------------                                           
closing price reported on the Nasdaq National Market or the principal securities
exchange on which the Common Stock (Nonvoting) may then be traded, as the case
may be, or, if there is no such sale on the relevant date, then on the last
previous day on which a sale was reported.

          (d) "Participant" means a person who, as a non-employee director of
               -----------                                                   
the Company, has been granted Shares under the Plan.

          (e) "Share" means a share of Common Stock (Nonvoting), $.01 par value,
               -----                                                            
of the Company and such other securities as may be substituted for such Share or
such other securities pursuant to Section 8.

     3.   SHARES AVAILABLE UNDER THE PLAN.  Subject to adjustment as provided in
          -------------------------------                                       
Section 8, as of any date, the total number of Shares issuable under the Plan
shall be 100,000.  Such Shares may be authorized but unissued Shares, treasury
Shares, or Shares acquired in the market for the account of the Participant.
<PAGE>
 
     4.   ADMINISTRATION OF THE PLAN.  The Plan will be administered by the
          --------------------------                                       
Board of Directors of the Company (the "Board").

     5.   ELIGIBILITY.  Only directors of the Company who are not employees of
          -----------                                                         
the Company or any subsidiary of the Company shall participate in the Plan.

     6.   GRANT OF SHARES.  On April 29, 1997 and on the date of the second
          ---------------                                                  
quarterly meeting of the Board of each calendar year during the term of the
Plan, each Participant shall receive a number of Shares equal to $20,000 divided
by the Fair Market Value per Share on the date of grant.

     7.   DEFERRAL OF SHARES.  Each director of the Company may elect to defer
          ------------------                                                  
the payment of Shares by submitting an election form to the Board, in accordance
with this Section 7.

          (a) ELECTIONS.  Each director who elects to defer the payment of
              ---------                                                   
Shares for a given calendar year must file an irrevocable written election with
the Secretary of the Company no later than December 31 of the year preceding
such calendar year; provided, that, any newly elected or appointed director may
                    --------  ----                                             
file an election for any year not later than 30 days after the date such person
first became a director, and a director may file an election for the year in
which the Plan became effective not later than 30 days after the date of
effectiveness of the Plan.  An election by a director shall be deemed to be
continuing and therefore applicable to subsequent Plan years unless the director
revokes or changes such election by filing a new election form by the due date
for such form specified in this Section 7(a).  The election must specify the
following:

               (i) A percentage or number of Shares to be deferred under the
Plan; and

          (ii) The date on which the commencement of payments of Shares should
begin, which date shall not be later than 10 years from the date the Shares
originally were payable;

provided, however, that, notwithstanding an election pursuant to this Section
--------  -------  ----                                                      
7(a), all Shares of a Participant for which payment has not otherwise occurred,
shall be paid upon death, Disability or termination of directorship of the
Participant.

          (b) DEFERRAL OF SHARES.  The Company will establish a deferral account
              ------------------                                                
for each Participant who elects to defer Shares under this Section 7.  At any
date Shares are payable to a Participant who has elected to defer Shares, the
Company will credit such Participant's deferral account with a the number of
Shares so deferred.

                                      -2-
<PAGE>
 
          (c) CREDITING OF DIVIDEND EQUIVALENTS.  Whenever dividends are paid or
              ---------------------------------                                 
distributions made with respect to Shares, a Participant to whom Shares are then
credited in a deferral account shall be entitled, on the dividend payment date,
as dividend equivalents, to an amount equal in value to the amount of the
dividend paid or property distributed on a single Share multiplied by the number
of Shares credited to his or her deferral account as of the record date for such
dividend or distribution.  Such dividend equivalents shall be credited to the
Participant's deferral account by payment to such account of a number of Shares
determined by dividing the aggregate value of such dividend equivalents by the
Fair Market Value of a Share at the payment date of the dividend or
distribution.

          (d) SETTLEMENT OF DEFERRED SHARES.  The Company will settle the
              -----------------------------                              
Participant's deferral account by delivering to the Participant (or his or her
beneficiary) a number of Shares equal to the number of whole Shares then
credited to his or her deferral account (or a specified portion in the event of
any partial settlement), together with cash in lieu of any fractional Share
remaining at a time that less than one whole Share is credited to such deferral
account.  Such settlement shall be made at the time or times specified in the
Participant's election filed in accordance with Section 7(a); provided, however,
                                                              --------  ------- 
that a Participant may further defer settlement of Shares if counsel to the
Company determines that such further deferral likely would be effective under
applicable federal income tax laws and regulations.

          (e) NONFORFEITABILITY.  The interest of each Participant in any Shares
              -----------------                                                 
(and any deferral account relating thereto) at all times will be nonforfeitable.

     8.   ADJUSTMENT PROVISIONS.  In the event any dividend or other
          ---------------------                                     
distribution (whether in the form of cash, Shares or other property),
recapitalization, forward or reverse split, reorganization, merger,
consolidation, spin-off, combination, repurchase, exchange of Shares or other
securities of the Company, extraordinary dividend (whether in the form of cash,
Shares, or other property), liquidation, dissolution, or other similar corporate
transaction or event affects the Shares such that an adjustment is appropriate
in order to prevent dilution or enlargement of each Participant's rights under
the Plan, then an adjustment shall be made, in a manner that is proportionate to
the change to the Shares and otherwise equitable, in (i) the number and kind of
Shares remaining reserved and available for issuance under Section 3, and (ii)
the number and kind of Shares to be issued upon settlement of deferred Shares
under Section 7.  In addition, the Board is authorized to make such adjustments
in recognition  of unusual or non-recurring events (including, without
limitation, events described in the preceding sentence) affecting the Company or
any subsidiary or the financial statements of the Company or any subsidiary, or
in response to changes in applicable laws,

                                      -3-
<PAGE>
 
regulations or accounting principles.  The foregoing notwithstanding, no
adjustment may be made hereunder except as will be necessary to maintain the
proportionate interest of the Participant under the Plan and to preserve,
without exceeding, the value of outstanding deferred Shares.

     9.   CHANGES TO THE PLAN.  The Board may amend, alter, suspend,
          -------------------                                       
discontinue, or terminate the Plan or authority to grant Shares under the Plan
without the consent of stockholders or Participants, except that any amendment
or alteration will be subject to the approval of the Company's stockholders at
or before the next annual meeting of stockholders for which the record date is
after the date of such Board action if such stockholder approval is required by
any federal or state law or regulation or the rules of any stock exchange or
automated quotation system as then in effect, and the Board may otherwise
determine to submit other such amendments or alterations to stockholders for
approval; provided, however, that, without the consent of an affected
          --------  -------  ----                                    
Participant, no such action may materially impair the rights of such Participant
with respect to any previously granted Shares.

     10.  GENERAL PROVISIONS.
          ------------------ 

          (a) AGREEMENTS.  Any right or obligation under the Plan may be
              ----------                                                
evidenced by agreements or other documents executed by the Company and the
Participant incorporating the terms and conditions set forth in the Plan,
together with such other terms and conditions not inconsistent with the Plan, as
the Board may from time to time approve.

          (b) COMPLIANCE WITH LAWS AND OBLIGATIONS.  The Company will not be
              ------------------------------------                          
obligated to issue or deliver Shares in a transaction subject to the
registration requirements of the Securities Act of 1933, as amended, or any
other federal or state securities law, any requirement under any listing
agreement between the Company and any stock exchange or automated quotation
system, or any other law, regulation, or contractual obligation of the Company,
until the Company is satisfied that such laws, regulations, and other
obligations of the Company have been complied with in full.  Certificates
representing Shares issued under the Plan will be subject to such stop-transfer
orders and other restrictions as may be applicable under such laws, regulations,
and other obligations of the Company, including any requirement that a legend or
legends be placed thereon.

          (c) LIMITATIONS ON TRANSFERABILITY.  Deferred Shares under the Plan
              ------------------------------                                 
will not be transferable by a Participant except by will or the laws of descent
and distribution or to a beneficiary in the event of the Participant's death.
Deferred Shares may not be pledged, mortgaged, hypothecated or otherwise
encumbered, and shall not be subject to the claims of creditors.

                                      -4-
<PAGE>
 
          (d) NO RIGHT TO CONTINUE AS A DIRECTOR.  Nothing contained in the Plan
              ----------------------------------                                
or any agreement hereunder will confer upon any Participant any right to
continue to serve as a director of the Company.

          (e) NO STOCKHOLDER RIGHTS CONFERRED.  Nothing contained in the Plan or
              -------------------------------                                   
any agreement hereunder will confer upon any Participant (or any person or
entity claiming rights by or through a Participant) any rights of a stockholder
of the Company unless and until Shares are in fact issued to such Participant
(or person).

          (f) NONEXCLUSIVITY OF THE PLAN.  Neither the adoption of the Plan by
              --------------------------                                      
the Board nor its submission to the stockholders of the Company for approval
shall be construed as creating any limitations on the power of the Board to
adopt such other compensatory arrangements for directors as it may deem
desirable.

          (g) GOVERNING LAW.  The validity, construction, and effect of the Plan
              -------------                                                     
and any agreement hereunder will be determined in accordance with the laws of
the State of New York, without giving effect to principles of conflicts of laws,
and applicable federal law.

     11.  STOCKHOLDER APPROVAL, EFFECTIVE DATE, AND PLAN TERMINATION.  The Plan
          ----------------------------------------------------------           
will be effective as of the date of its adoption by the Board, subject to
stockholder approval if necessary or appropriate, and, unless earlier terminated
by action of the Board, shall terminate at such time as no Shares remain
available for issuance under the Plan and the Company and Participants have no
further rights or obligations under the Plan.

                                      -5-