<PAGE>
 
                         NOTICE OF GUARANTEED DELIVERY
 
                                 IN RESPECT OF
 
                            ALL OF THE OUTSTANDING
 
                         12 1/4% SENIOR NOTES DUE 2002
 
                                      OF
 
                              POLYMER GROUP, INC.
 
     PURSUANT TO THE OFFER TO PURCHASE AND CONSENT SOLICITATION STATEMENT
                              DATED JUNE 5, 1997
 
             The Depositary for the Offer and the Solicitation is:
 
                       HARRIS TRUST COMPANY OF NEW YORK
 
       By Mail:              By Overnight Courier:             By Hand:
  Wall Street Station        77 Water Street, 4th           Receive Window
     P.O. Box 1023                   Floor               77 Water Street, 5th
  New York, NY 10268-         New York, NY 10005                 Floor
         1023                     Attention:              New York, NY 10005
      Attention:             Reorganization Dept.             Attention:
 Reorganization Dept.                                    Reorganization Dept.
 
                          By Facsimile Transmission:
                       (for Eligible Institutions Only)
                            (212) 701-7636 or 7637
 
                   For Information Telephone (call collect):
                                (212) 701-7624
 
  DELIVERY OF THIS NOTICE OF GUARANTEED DELIVERY TO AN ADDRESS, OR
TRANSMISSION VIA FACSIMILE TO A NUMBER, OTHER THAN AS SET FORTH ABOVE WILL NOT
CONSTITUTE VALID DELIVERY.
 
  As set forth in the Offer to Purchase and Consent Solicitation Statement
dated June 5, 1997 (as it may be supplemented and amended from time to time,
the "Statement") of Polymer Group, Inc. (the "Company") under Item 7,
"Procedures for Tendering Notes and Delivery Consents," and in the
Instructions of the Consent and Letter of Transmittal (the "Consent and Letter
of Transmittal"), this form, or one substantially equivalent hereto, or an
Agent's Message relating to the guaranteed delivery procedures, must be used
to accept the Company's offer (the "Offer") to purchase for cash any and all
of its outstanding 12 1/4% Senior Notes due 2002 (the "Notes"), if, after the
Consent Expiration Date (as defined in the Statement), time will not permit
the Consent and Letter of Transmittal, certificates representing such Notes
and other required documents to reach the Depositary, or the procedures for
book-entry transfer cannot be completed, on or prior to the Tender Offer
Expiration Date (as defined in the Statement).
 
  In conjunction with the Offer, the Company is also soliciting (the
"Solicitation") consents (the "Consents") to certain proposed amendments (the
"Proposed Amendments") to the Indenture dated as of June 24, 1994, between the
Company and First Union National Bank of South Carolina ("First Union"), as
amended by the First Supplemental Indenture dated as of March 15, 1995 between
the Company and First Union, the Second Supplemental Indenture dated as of
September 14, 1995 among the Company, First Union and Harris Trust and Savings
Bank, as trustee (the "Trustee"), the Third Supplemental Indenture dated as of
April 9, 1996 between the Company and the Trustee, and the Fourth Supplemental
Indenture dated as of August 14, 1996 between the Company and the Trustee (as
so amended, the "Indenture"), pursuant to which the Notes were issued. This
Notice of Guaranteed Delivery may not be used to tender Notes or deliver
Consents on or prior to the Consent Expiration Date. This form must be
delivered by an Eligible Institution (as defined herein) by mail or hand
delivery or transmitted via facsimile to the Depositary as set forth above.
All capitalized terms used herein but not defined herein shall have the
meanings ascribed to them in the Statement.
 
  This form is not to be used to guarantee signatures. If a signature on the
Consent and Letter of Transmittal is required to be guaranteed by a Medallion
Signature Guarantor under the instructions thereto, such signature guarantee
must appear in the applicable space provided in the Consent and Letter of
Transmittal.
<PAGE>
 
Ladies and Gentlemen:
 
  The undersigned hereby tender(s) to the Company, upon the terms and subject
to the conditions set forth in the Statement and the Consent and Letter of
Transmittal (receipt of which is hereby acknowledged), the principal amount of
the Notes specified below pursuant to the guaranteed delivery procedures set
forth in the Statement under Item 7, "Procedures for Tendering Notes and
Delivering Consents -- Guaranteed Delivery." The undersigned hereby authorizes
the Depositary to deliver this Notice of Guaranteed Delivery to the Company
and the Trustee with respect to the Notes tendered pursuant to the Offer.
 
  The undersigned understands that Holders who desire to tender their Notes
pursuant to the Offer and receive the Total Consideration are required to
provide Consents to the Proposed Amendments with respect to the full principal
amount of the Notes so tendered on or prior to the Consent Expiration Date.
 
  The undersigned understands that tenders of Notes will be accepted only in
principal amounts equal to $1,000 or integral multiples thereof. The
undersigned also understands that tenders of Notes may be withdrawn at any
time prior to the Tender Offer Expiration Date. This Notice of Guaranteed
Delivery may only be utilized after the Consent Expiration Date and on or
prior to the Tender Offer Expiration Date.
 
  The undersigned understands that payment for Notes purchased will be made
only after timely receipt by the Depositary of (i) such Notes, or a Book-Entry
Confirmation, and (ii) a Consent and Letter of Transmittal (or a manually
signed facsimile thereof), including by means of an Agent's Message, of the
transfer of such Notes into the Depositary's account at a Book-Entry Transfer
Facility, with respect to such Notes, properly completed and duly executed,
with any signature guarantees and any other documents required by the Consent
and Letter of Transmittal within two New York Stock Exchange, Inc. trading
days after the execution hereof. The undersigned also understands that under
no circumstances will interest be paid by the Company by reason of any delay
in making payment to the undersigned and that the Tender Offer Consideration
for Notes tendered pursuant to the guaranteed delivery procedures will be the
same as that for Notes delivered to the Depositary after the Consent
Expiration Date and on or prior to the Tender Offer Expiration Date, even if
the Notes to be delivered pursuant to the guaranteed delivery procedures are
not so delivered to the Depositary, and therefore payment by the Depositary on
account of such Notes is not made, until after the Payment Date. THE
UNDERSIGNED IS AWARE THAT, ON OR PRIOR TO THE CONSENT EXPIRATION DATE, TENDERS
OF NOTES AND THE RELATED CONSENTS CANNOT BE DELIVERED USING THE GUARANTEED
DELIVERY PROCESS AND THAT USE OF THE GUARANTEED DELIVERY PROCESS COULD RESULT
IN A TENDER OF NOTES AND THE RELATED CONSENT BEING DEFECTIVE.
 
  All authority conferred or agreed to be conferred by this Notice of
Guaranteed Delivery shall not be affected by, and shall survive, the death or
incapacity of the undersigned, and every obligation of the undersigned under
this Notice of Guaranteed Delivery shall be binding upon the heirs, executors,
administrators, trustees in bankruptcy, personal and legal representatives,
successors and assigns of the undersigned.
 
  All capitalized terms used herein but not defined herein shall have the
meanings ascribed to them in the Statement.
 
                                       2
<PAGE>
 
                           PLEASE SIGN AND COMPLETE
 
 
 
 Signature(s) of Registered                Date: _____________________________
 Holder(s) or Authorized
 Signatory: ________________________
 
                                           Address: __________________________
 
 
 -----------------------------------       -----------------------------------
 
 
 -----------------------------------       Area Code and Telephone No. _______
 
 
 Name(s) of Registered Holder(s): __       If Notes will be delivered by
                                           book-entry transfer, check trust
                                           company below:
 
 -----------------------------------
 
 
 -----------------------------------       [_] The Depositary Trust Company
 
 
 Principal Amount of Notes                 DepositoryAccount No. _____________
 Tendered: _________________________
 
 
 
 -----------------------------------
  HOLDERS WHO DESIRE TO TENDER THEIR NOTES PURSUANT TO THE OFFER AND RECEIVE
THE TOTAL CONSIDERATION ARE REQUIRED TO CONSENT TO THE PROPOSED AMENDMENTS
WITH RESPECT TO SUCH NOTES ON OR PRIOR TO THE CONSENT EXPIRATION DATE.
 
 Certificate No.(s) of Notes (if
 available) ________________________
 
 
 
   This Notice of Guaranteed Delivery must be signed by the Holder(s) exactly
 as their name(s) appear(s) on certificate(s) for Notes or on a security
 position listing as the owner of Notes, or by person(s) authorized to become
 Holder(s) by endorsements and documents transmitted with this Notice of
 Guaranteed Delivery. If signature is by a trustee, executor, administrator,
 guardian, attorney-in-fact, officer or other person acting in a fiduciary or
 representative capacity, such person must provide the following information.
 
                     Please print name(s) and address(es)
 
 Name(s): ____________________________________________________________________
 
 -----------------------------------------------------------------------------
 
 Capacity: ___________________________________________________________________
 
 Address(es): ________________________________________________________________
 
 -----------------------------------------------------------------------------
 
 -----------------------------------------------------------------------------
 
 
  DO NOT SEND NOTES WITH THIS FORM. NOTES SHOULD BE SENT TO THE DEPOSITARY
TOGETHER WITH A PROPERLY COMPLETED AND DULY EXECUTED CONSENT AND LETTER OF
TRANSMITTAL.
 
                                       3
<PAGE>
 
 
                                   GUARANTEE
 
                   (NOT TO BE USED FOR SIGNATURE GUARANTEE)
 
   The undersigned, a member of the Securities Transfer Agents Medallion
 Program, the Stock Exchange Medallion Program or the New York Stock
 Exchange, Inc. Medallion Signature Program (each, an "Eligible
 Institution"), hereby represents that the tender of Notes hereby complies
 with Rule 14e-4 promulgated under the Securities Exchange Act of 1934, as
 amended, and guarantees that the Notes tendered hereby are in proper form
 for transfer (pursuant to the procedures set forth in the Statement under
 Item 7, "Procedures for Tendering Notes and Delivering Consents-- Guaranteed
 Delivery"), and that the Depositary will receive (a) such Notes, or a Book-
 Entry Confirmation of the transfer of such Notes into the Depositary's
 account at a Book-Entry Transfer Facility and (b) a properly completed and
 duly executed Consent and Letter of Transmittal (or facsimile thereof) with
 any required signature guarantees and any other documents required by the
 Consent and Letter of Transmittal, or a properly transmitted Agent's
 Message, within two New York Stock Exchange, Inc. trading days after the
 date of execution hereof.
 
   The Eligible Institution that completes this form must communicate the
 guarantee to the Depositary and must deliver the Consent and Letter of
 Transmittal and Notes to the Depositary within the time period shown herein.
 Failure to do so could result in a financial loss to such Eligible
 Institution.
 
 Name of Firm: _______________________________________________________________
 
 Authorized Signature: _______________________________________________________
 
 Title: ______________________________________________________________________
 
 Address: ____________________________________________________________________
 
 -----------------------------------------------------------------------------
                                                     (Zip Code)
 
 Area Code and Telephone Number: _____________________________________________
 
 Dated: _____________________ , 1997
 
 
                                       4