<PAGE>
 
                                                                    EXHIBIT 10.4

                      PAB BANKSHARES, INC. 1994 DIRECTORS
                         DEFERRED STOCK PURCHASE PLAN

                                        
1.   PURPOSE OF THE PLAN.
     ------------------- 

     This Directors Deferred Stock Purchase Plan (the "Plan") for PAB
Bankshares, Inc. ("Company") is intended to encourage Directors of the Company
to develop a proprietary interest in the success of the Company, and to retract
and retain such Directors of the Company.  Unless otherwise indicated by the
context herein, reference to the "Company" includes the Company and its
subsidiaries.

2.   THE PLAN.
     -------- 

     As provided herein, each Director of the company may elect to receive
Company common stock in lieu of the cash compensation otherwise payable to such
Director for his service as a member of the Board of Directors or any committee
thereof.  The Company shares issuable to any electing Director shall be issued
to such Director on each January 15 (or next business day thereafter) (the
"Deferred Payment Date") following the end of the calendar year the director
election is in effect.

3.   ELIGIBILITY.
     ----------- 
     All Directors of the Company, including Officers and Directors who are
employees, shall be eligible to participate in this Plan.

4.   STOCK SUBJECT TO PLAN.
     --------------------- 

     The stock subject to the Plan shall be shares of the Company's common stock
which are authorized but unissued or reacquired by the Company, or any
authorized but unissued or reacquired shares to which such shares of common
stock are exchanged, reclassified or
<PAGE>
 
converted.  The aggregate number of shares for which all Directors may elect to
receive Company common stock in lieu of cash under the Plan shall not exceed
25,000 as may be adjusted for stock splits, share dividends, reclassification of
shares, merger or otherwise.

5.   DETERMINATION OF COMPANY SHARES DISTRIBUTABLE TO ELECTING DIRECTOR.
     ------------------------------------------------------------------ 

     The number of Company shares issuable to any electing Director on each
Deferred Payment Date is a number resulting when the amount of the electing
Director's fees for the previous year, subject to his or her stock purchase
election, is divided by one hundred percent of the fair market value of PAB
common stock as of January 1 of the preceding year.  No fractional shares shall
be issued under the Plan.  The value of any fractional shares shall be paid by
the Company in cash to the Director on the Deferred Payment Date.  The fair
market value of the PAB common stock as of January 1 of each year shall be that
amount determined by resolution of the Board of Directors of the Company at the
first regularly scheduled meeting of the Directors of the Company.

6.   MANNER OF ELECTION.
     ------------------ 

     Any Director of the Company may elect to be a participant in the Plan by
delivering his election to so participate in writing to the Company.  The
election shall state:  (i) that the Director is electing to be a participant in
the Plan; (ii) that the election is applicable to all fees and other
compensation payable to the Director after the date of the election by virtue of
his position as a Director with the Company (in addition to regular board fees,
such compensation includes committee fees and such bonuses the Board of
Directors of the Company and its subsidiaries may grant the Directors from time
to time); and (iii) and the dollar amount or percentage of such fees which will
be payable in (1) Company common stock; and (2) cash 

                                       2
<PAGE>
 
payable on the Deferred Payment Date. Such election shall be continuing until:
(i) modified in writing by the Director to the company; (ii) revoked in writing
by the Director to the Company; (iii) upon the resignation or removal of the
Director from the Board of Directors of the Company and any subsidiary in which
such Director serves in similar capacity; and (iv) upon the death of the
Director.

7.   LEGEND ON CERTIFICATE ISSUED.
     ---------------------------- 

     Each share certificate evidencing the acquisition of Company shares
pursuant to the provisions of the Plan shall contain a legend indicating that
pursuant to Rule 144 promulgated under the Securities Act of 1933, as amended,
the shares evidenced by the certificate may not be sold for a period of at least
two years from the date of the issuance of the share certificate; and,
thereafter, may only be sold with the opinion of legal counsel of the Company
that the shares may be sold or disposed of in a transaction exempt from
registration under the Securities Act of 1933 or any applicable State Securities
Act.

8.   AMENDMENT OR TERMINATION OF THE PLAN.
     ------------------------------------ 

     The Board of Directors may, without further stockholder approval, amend or
discontinue this Plan at anytime, provided that without the approval of the
stockholders, no amendment may increase the maximum number of Company shares
which the electing Directors may purchase pursuant to the Plan.

9.   TERM OF THE PLAN.
     ---------------- 

     This Plan shall become effective on the earlier of the date of its adoption
by the Board of Directors of the Company or its approval by the stockholders of
the Company.  Unless otherwise terminated as provided herein, the Plan shall
terminate on the earlier of (i) January 15, 1999; or, 

                                       3
<PAGE>
 
(ii) the date that all Company shares reserved herein for Directors electing to
participate in the Plan have been issued.

10.  SHAREHOLDER APPROVAL.
     -------------------- 

     This Plan shall be submitted to the stockholders of the Company for their
approval within twelve months of its adoption by the Board of Directors.

11.  NOTICES.
     ------- 

     All notices or other communications hereunder must be in writing and will
be deemed given on the date delivered if delivered in person or on the third
business day after mailed by depositing the same, postage prepaid, in a post
office addressed to the Company at its principal office and to any other person
at their last known address furnished to the Company.

                                       4