<PAGE>

                              GALEY & LORD, INC.

                          1996 RESTRICTED STOCK PLAN
                           (As Currently in Effect)

          1.  Purpose.  The purpose of the Galey & Lord, Inc., 1996 Restricted
              -------
Stock Plan (the "Plan") is to induce certain non-employee directors to continue
to serve as directors of Galey & Lord, Inc. (the "Corporation"), to encourage
ownership of shares in the Corporation by such non-employee directors and to
provide additional incentive for such directors to promote the success of the
Corporation's business.

          2.  Effective Date of the Plan.  The Plan became effective on November
              --------------------------
12, 1996, by action of the Board of Directors (the "Board") subject to approval
by the stockholders of the Corporation.

          3.  Stock Subject to Plan.  200,000 of the authorized but unissued
              ---------------------
shares of the common stock, $.01 par value, of the Corporation (the "Common
Stock") are hereby reserved for issuance under the Plan; provided, however, that
                                                         --------  -------
the number of shares so reserved may from time to time be reduced to the extent
that a corresponding number of issued and outstanding shares of the Common Stock
are purchased by the Corporation and set aside for issuance under the Plan.  If
any shares of the Common Stock issued under the Plan ("Restricted Shares") are
reacquired by the Corporation as provided in Section 9, such shares shall again
be available for the purposes of the Plan.

          4.  Committee.  The committee (the "Committee") shall consist of two
              ---------
or more members of the Board.  The Chief Executive Officer of the Corporation
shall also be a member of the Committee, ex-officio, whether or not he or she is
otherwise a member of the

                                       1
<PAGE>

Committee. The Committee shall be appointed annually by the Board, which may at
any time and from time to time remove any members of the Committee, with or
without cause, appoint additional members to the Committee and fill vacancies,
however caused, in the Committee. In the event that no Committee shall have been
appointed, the Board shall act as the Committee. A majority of the members of
the Committee shall constitute a quorum. All determinations of the Committee
shall be made by a majority of its members present at a meeting duly called and
held. Any decision or determination of the Committee reduced to writing and
signed by all of the members of the Committee shall be fully effective as if it
had been made at a meeting duly called and held.

          5.  Administration.  Subject to the provisions of the Plan, the
              --------------
Committee shall have authority to interpret the Plan and to prescribe, amend and
rescind rules and regulations relating to it.  Any determination by the
Committee in carrying out, administering or construing the Plan shall be final
and binding for all purposes and upon all interested persons and their heirs,
successors and personal representatives.

          6.  Eligibility.  An allocation of Restricted Shares may only be made
              -----------
to persons who are directors of the Corporation and who are not common law
employees of the Corporation.

          7.  Grants of Restricted Shares.  A. Annually on the third Trading Day
              ---------------------------
after the date of the Annual Meeting of stockholders of the Corporation, through
and including the Annual Meeting of stockholders of the Corporation held in
2005, and on the third Trading Day after the date on which each person described
below first becomes a director (if such date is a date other than the Annual
Meeting of Stockholders of the Corporation), each person who (a) is

                                       2
<PAGE>

serving as a director of the Corporation on the date of grant, (b) is not a
common law employee of the Corporation and (c) has filed an election with the
Corporation (at such time and in such manner as may be prescribed by the
Corporation) to receive an allocation of Restricted Shares for the current year
in lieu of an option under the Amended and Restated 1989 Stock Option Plan of
Galey & Lord, Inc. (or any other stock option plan subsequently adopted by the
Corporation), shall automatically be allocated a number Restricted Shares
derived by dividing eighteen thousand (18,000) by the Fair Market Value per
Share of the Common Stock on the date of grant (rounded to the next highest
share), subject to availability under the Plan.

          B.   On the third Trading Day after each January 1, through and
including January 1, 2006, and at the time each person described below first
becomes a director, each person who (a) is serving as a director of the
Corporation on such date, (b) is not a common law employee of the Corporation on
such date and (c) filed an election with the Corporation on or prior to the
preceding December 1, or at the time a person first became a director (at such
time and in such manner as may be prescribed by the Corporation) to receive all
of his or her annual director's fee for the subsequent calendar year (or, for
the remainder of the current calendar year, in the case of a person who first
becomes a director during such calendar year) in the form of a combination of
cash and Restricted Shares in lieu of (i) cash or (ii) a grant of options under
the Amended and Restated 1989 Stock Option Plan of Galey & Lord, Inc. (or any
other stock option plan subsequently adopted by the Corporation), shall be paid
for such subsequent calendar year (or, for the remainder of the current calendar
year, in the case of a person who first becomes a director during such calendar
year) in the form of a combination of ten thousand dollars ($10,000) of cash and
an allocation, subject to availability under the Plan, of the number of
Restricted Shares derived by dividing fifteen thousand (15,000) by the Fair
Market Value per

                                       3
<PAGE>

Share of the Common Stock on the third Trading Day after January 1 of such
subsequent calendar year (or the third Trading Day after the date on which such
person first becomes a director, as the case may be), rounded to the next
highest share. Notwithstanding the foregoing provisions of this Section 7B, the
amount of the annual directors' fee to be paid (whether in the form of cash,
stock options or a combination of cash and Restricted Shares) with respect to a
calendar year to a director who first becomes a director during such calendar
year shall be reduced to an amount calculated by multiplying the amount to which
a director otherwise would be payable pursuant to the foregoing provisions of
this Section 7B (whether in the form of cash, stock options or a combination of
cash and Restricted Shares) by a fraction, the numerator of which is the number
of calendar days remaining in such calendar year measured from the date on which
such person first becomes a director and the denominator of which is three
hundred sixty-five (365).

          8.  Purchase Price.  Each person who shall be allocated Restricted
              --------------
Shares hereunder shall purchase the same from the Corporation at and for a
purchase price of $.01 a share.  Failure by a Participant to purchase and pay
for all of the Restricted Shares allocated to him or her within thirty days
after he or she shall have been given written notice of such allocation shall
result in a cancellation of such allocation and he or she shall no longer have
the right to purchase the same hereunder.

          9.  Restricted Shares.  A.  Except as otherwise provided in this
              -----------------
Section, the Restricted Shares allocated to a Participant may not be sold,
assigned, transferred or otherwise disposed of, and may not be pledged or
hypothecated.  The Stock Restrictions shall terminate on the third

                                       4
<PAGE>

anniversary of the date of his or her Restricted Stock Agreement (as such term
is defined in paragraph E. of this Section 9).

          B.  In addition, if the Participant to whom Restricted Shares have
been allocated as of any Allocation Date leaves the service of the Corporation
and its subsidiaries by reason of his or her Discharge for Cause prior to the
termination of the Stock Restrictions with respect to the Restricted Shares
allocated to such Participant as of such Allocation Date, he or she shall be
obligated to redeliver such Restricted Shares to the Corporation immediately and
the Corporation shall pay to him or her, in redemption of such shares, an amount
equal to the price paid by the Participant for such Restricted Shares.

          C.  If the Participant to whom Restricted Shares have been allocated
as of any Allocation Date leaves the service of the Corporation and its
subsidiaries for any reason other than his or her Discharge for Cause prior to
the termination of the Stock Restrictions with respect to the Restricted Shares
allocated to such Participant as of such Allocation Date, such Participant (or,
in the event of his or her death, the executors or administrators of his or her
estate) shall retain all of his or her rights with respect to such Restricted
Shares; provided, however, that such Restricted Shares shall remain subject to
        --------  -------
the terms of the Plan and of such Participant's Restricted Stock Agreement.

          D.  Upon issuance of the certificate or certificates for the
Restricted Shares in the name of a Participant, the Participant shall thereupon
be a stockholder with respect to all the Restricted Shares represented by such
certificate or certificates and shall have the rights of a stockholder with
respect to such Restricted

                                       5
<PAGE>

Shares, including the right to vote such Restricted Shares and to receive all
dividends and other distributions paid with respect to such Restricted Shares.

          E.  Each Participant receiving Restricted Shares shall (a) agree that
such Restricted Shares shall be subject to, and shall be held by him or her in
accordance with all of the applicable terms and provisions of, the Plan, (b)
represent and warrant to the Corporation that he or she is acquiring such
Restricted Shares for investment for his or her own account (unless there is
then current a prospectus relating to the Restricted Shares under Section 10(a)
of the Securities Act of 1933, as amended) and, in any event, that he or she
will not sell or otherwise dispose of said shares except in compliance with the
Securities Act of 1933, as amended, and (c) agree that the Corporation may place
on the certificates representing the Restricted Shares or new or additional or
different shares or securities distributed with respect to the Restricted Shares
such legend or legends as the Corporation may deem appropriate and that the
Corporation may place a stop transfer order with respect to such Restricted
Shares with the Transfer Agent(s) for the Common Stock.  The foregoing
agreement, representation and warranty shall be contained in an agreement in
writing ("Restricted Stock Agreement") which shall be delivered by the
Participant to the Corporation.  The Committee shall adopt, from time to time,
such rules with respect to the return of executed Restricted Stock Agreements as
it deems appropriate and failure by a Participant to comply with such rules
shall terminate the allocation of such Restricted Shares to such Participant.

          10.  Adjustment of Number of Shares.  A.  In the event that a dividend
               ------------------------------
shall be declared upon the Common Stock payable in shares of the Common Stock,
the number of shares of the Common Stock then subject to any Restricted Stock
Agreement and the number of shares

                                       6
<PAGE>

of the Common Stock reserved for issuance in accordance with the provisions of
the Plan but not yet issued shall be adjusted by adding to each such share the
number of shares which would be distributable thereon if such shares had been
outstanding on the date fixed for determining the stockholders entitled to
receive such stock dividend. In the event that the outstanding shares of the
Common Stock shall be changed into or exchanged for a different number or kind
of shares of stock or other securities of the Corporation or of another
corporation, whether through reorganization, recapitalization, stock split-up,
combination of shares, sale of assets, merger or consolidation in which the
Corporation is the surviving corporation, then, there shall be substituted for
each share of the Common Stock then subject to a Restricted Stock Agreement and
for each share of the Common Stock reserved for issuance in accordance with the
provisions of the Plan but not yet issued, the number and kind of shares of
stock or other securities into which each outstanding share of the Common Stock
shall be so changed or for which each such share shall be exchanged.

          B.  In the event that there shall be any change, other than as
specified in this Section 10, in the number or kind of outstanding shares of the
Common Stock, or of any stock or other securities into which the Common Stock
shall have been changed, or for which it shall have been exchanged, then, if the
Committee shall, in its sole discretion, determine that such change equitably
requires an adjustment in the number or kind of shares then subject to a
Restricted Stock Agreement and the number or kind of shares reserved for
issuance in accordance with the provisions of the Plan but not yet issued, such
adjustment shall be made by the Committee and shall be effective and binding for
all purposes of the Plan and of each Restricted Stock Agreement entered into in
accordance with the provisions of the Plan.

                                       7
<PAGE>

          C.   No adjustment or substitution provided for in this Section 10
shall require the Corporation to deliver a fractional share under the Plan or
any Restricted Stock Agreement.

          11.  Withholding and Waivers.  In the event of the death of a
               -----------------------
Participant, an additional condition to the Corporation's obligation to release
Restricted Shares to the executors or administrators of such Participant's
estate and to release the Stock Restrictions provided hereunder on any
Restricted Shares owned by such Participant as provided in Section 9 shall be
the delivery to the Corporation of such tax waivers, letters testamentary and
other documents as the Committee may reasonably determine.

     12.  Definitions.  For the purposes hereof:
          -----------

          A.   The term "Allocation Date" shall mean the date as of which a
          Participant shall have received a grant of Restricted Shares.

          B.   The term "Discharge for Cause" shall mean the termination of a
          Participant's service as a director of the Corporation by reason of
          (i) the commission by such Participant of any act or omission that
          would constitute a crime under federal, state or equivalent foreign
          law, (ii) the commission by such Participant of any act of moral
          turpitude, (iii) fraud, dishonesty or other acts or omissions that
          result in a breach of any fiduciary or other material duty to the
          Corporation or (iv) continued alcohol or other substance abuse that
          renders such Participant incapable of performing his or her material
          duties to the satisfaction of the Corporation.

          C.   The term "Fair Market Value per Share " shall mean the last
          reported sale price for Common Stock (regular way) or, in case no such
          reported sale takes

                                       8
<PAGE>

          place on such Trading Day, the average of the closing bid and asked
          prices (regular way) for the Common Stock for such Trading Day, in
          either case on the principal national securities exchange on which the
          Common Stock is listed or admitted to trading, or if the Common Stock
          is not listed or admitted to trading on any national securities
          exchange, but is traded in the over-the-counter market, the closing
          sale price of the Common Stock or, if no sale is publicly reported,
          the average of the closing bid and asked quotations for the Common
          Stock, as reported by the National Association of Securities Dealers
          Automated Quotation System ("Nasdaq") or any comparable system or, if
          the Common Stock is not listed on Nasdaq or a comparable system, the
          closing sale price of the Common Stock, or, if no sale is publicly
          reported, the average of the closing bid and asked prices, as
          furnished by two members of the National Association of Securities
          Dealers, Inc. who make a market in the Common Stock selected from time
          to time by the Corporation for that purpose. In addition, for purposes
          of this definition and as otherwise used in the Plan, "Trading Day"
          shall mean, if the Common Stock is listed on any national securities
          exchange, a business day during which such exchange was open for
          trading and at least one trade of Common Stock was effected on such
          exchange on such business day, or, if the Common Stock is not listed
          on any national securities exchange but is traded in the over-the-
          counter market, a business day during which the over-the-counter
          market was open for trading and at least one "eligible dealer" quoted
          both a bid and asked price for the Common Stock. An "eligible dealer"
          for any day shall include any broker-dealer

                                       9
<PAGE>

          who quoted both a bid and asked price for such day, but shall not
          include any broker-dealer who quoted only a bid or only an asked price
          for such day.

          D.   The term "Stock Restrictions" shall mean the restrictions on the
          ability of a Participant to transfer Restricted Shares issued to such
          Participant hereunder referred to in Section 9 and embodied in a
          Restricted Stock Agreement between the Corporation and such
          Participant.

          13.  Expenses of Administration.  All costs and expenses incurred in
               --------------------------
the operation and administration of the Plan shall be borne by the Corporation.

          14.  No Employment Right.  Neither the existence of the Plan nor the
               -------------------
grant of any Restricted Shares hereunder shall require the Corporation or any
subsidiary to continue any Participant as a director of the Corporation or any
subsidiary.

          15.  Amendment of Plan.  The Board may, at any time and from time to
               -----------------
time, by a resolution appropriately adopted, make such modifications of the Plan
as it shall deem advisable.  No amendment of the Plan may, without the consent
of the Participants to whom any Restricted Shares shall theretofore have been
allocated, adversely affect the rights or obligations of such Participants with
respect to such Restricted Shares.  The Committee may, in its discretion, cause
the restrictions imposed in accordance with the provisions of Section 9 hereof
with respect to any Restricted Shares to terminate, in whole or in part, prior
to the time when they would otherwise terminate.

                                       10
<PAGE>

          16.  Expiration and Termination of the Plan.  The Plan shall terminate
               --------------------------------------
on November 11, 2006 or at such earlier time as the Board may determine;
provided, however, that such termination shall not, without the consent of the
--------  -------
Participants to whom any Restricted Shares shall theretofore have been
allocated, adversely affect the rights or obligations of such Participants with
respect to such Restricted Shares.

          17.  Governing Law.  The Plan shall be governed by the laws of the
               -------------
State of New York.

                                       11