<PAGE>   1

                                                                    EXHIBIT 10.5

                                              March 26, 2001

WPS Receivables Corporation
507 West 10th Street
Post Office Box 71
West Point, Georgia  31833
Attn: Nelson Griffith

WestPoint Stevens, Inc.
507 West 10th Street
Post Office Box 71
West Point, Georgia  31833
Attn: Nelson Griffith

                             Re: Conditional Waiver

Ladies and Gentlemen:

         Reference is made to (i) the Asset Interest Transfer Agreement, dated
as of December 18, 1998 (as amended, restated, supplemented or otherwise
modified, the "Asset Transfer Agreement"), among WPS Receivables Corporation
("Transferor"), WestPoint Stevens, Inc. ("WestPoint"), as initial Servicer, Blue
Ridge Asset Funding Corporation ("Blue Ridge") and Wachovia Bank, N.A., as
administrative agent (in such capacity, the "Administrator") and (ii) the Second
Amended and Restated Credit Agreement, dated as of June 9, 1998 (as amended,
restated, supplemented or otherwise modified, the "Credit Agreement") among
WestPoint, as borrower, certain subsidiaries of WestPoint, the financial
institutions from time to time party thereto (the "CA Lenders") and Bank of
America, N.A. (f/k/a NationsBank, N.A.), as agent for the CA Lenders (in such
capacity, the "CA Agent"). Capitalized terms used herein and not defined herein
shall have the respective meanings assigned thereto in the Asset Transfer
Agreement.

         Transferor is a wholly-owned Subsidiary of WestPoint. As a condition to
the effectiveness of the Fifth Amendment Agreement, dated as of March 26, 2001,
which amends the Credit Agreement, the CA Lenders are requiring WestPoint to
pledge all of its right, title and interest in and to the capital stock of
Transferor to the CA Agent, for the benefit of the CA Lenders, pursuant to a
pledge agreement to be entered into between WestPoint and the CA Lender (the
"Pledge Agreement") as additional security for WestPoint's obligations under the
CA Credit Agreement (the "Proposed Pledge").

         Pursuant to the terms of Section 6.01(u) of the Asset Transfer
Agreement and Section 5.1(w) of the Sale Agreement (collectively, the "Relevant
Provisions"), WestPoint represents and warrants that it owns 100% of the issued
and outstanding capital stock of Transferor, free and clear of any lien or
security interest. WestPoint and Transferor have requested that the
Administrator and the Transferees waive compliance by WestPoint with the
Relevant Provisions for the purpose of allowing WestPoint to consummate the
Proposed Pledge.

<PAGE>   2

         Subject to the immediately succeeding sentence, upon delivery to the
Administrator of an executed counterpart of this waiver letter from WestPoint
and Transferor, the Administrator and each Transferee hereby waive compliance by
WestPoint with the Relevant Provisions, but solely to the extent that such
provisions require WestPoint to represent and warrant that there are no liens or
security interests on the capital stock of Transferor owned by WestPoint and
subject to the condition that WestPoint shall not permit any lien or security
interest, other than in favor of the CA Agent, for the benefit of the CA
Lenders, to exist upon any of the capital stock of Transferor. In the event that
WestPoint or Seller fails to deliver to the Administrator the following
documents, in each case, in form and substance satisfactory to the Administrator
in sole discretion, on or prior to the date hereof, the waiver and the
agreements of the Administrator and the Transferees specified in the immediately
preceding sentence shall be null and void, have no force and effect and the
Relevant Provisions shall be reinstated and shall continue in full force and
effect as currently provided in the Asset Transfer Agreement and the Sale
Agreement:

         (1)      in the event that any Person, other than the CA Lenders, has
the right, directly or indirectly, to direct or instruct the CA Agent to
exercise any rights, remedies, powers, privileges or options with respect to the
common stock of Transferor pursuant to the terms of the Pledge Agreement, such
Person shall enter into an agreement setting forth provisions that are
substantially the same as the provisions specified in the agreement to be
executed on the date hereof by and between the CA Lender and the Administrator,
a form of which is attached as Exhibit A hereto;

         (2)      a copy of the Pledge Agreement, duly executed and delivered by
the parties thereto; and

         (3)      an agreement, substantially in the form of Exhibit A attached
hereto, duly executed and delivered by the CA Agent.

         WestPoint and Transferor agree that the waiver set forth in the
preceding paragraph shall be limited to the precise meaning of the words as
written therein and shall not be deemed (i) to be a consent to any waiver or
modification of any other term, provision or condition of the Transaction
Documents, as amended, modified, supplemented or amended and restated from time
to time, or (ii) to prejudice any right or remedy that the Administrator or any
Transferee may now have or may in the future have under or in connection with
the Transaction Documents, as amended, modified, supplemented or amended and
restated from time to time.

         Except as expressly set forth herein, the waiver described in the
second preceding paragraph shall not modify, alter, affect, release or prejudice
in any way any of WestPoint's or Transferor's obligations under the Transaction
Documents, as amended, modified, supplemented or amended and restated from time
to time. This waiver letter shall not constitute or operate as a waiver of any
other terms or provisions of, or rights or remedies of, the Administrator or any
Transferee under the Transaction Documents, as amended, modified, supplemented
or amended and restated from time to time, and shall not be construed as
establishing a course of conduct on the part of the Administrator or any
Transferee on which WestPoint and/or Transferor may rely at any time in the
future. Each of WestPoint and Transferor expressly waives any right to assert
any claim to such effect at any time.


                                       2
<PAGE>   3

                  This waiver letter (i) shall be governed by and construed in
accordance with the internal laws of the State of New York, (ii) may be executed
in any number of counterparts, each of which shall be deemed to be an original
but all of which, taken together, shall constitute one and the same instrument,
(iii) may not be amended, modified, supplemented or amended and restated or the
any of the terms or provisions hereof waived, except by a written instrument
executed by all of the parties hereto, and (iv) applies to, and inures to the
benefit of, and binds all of the parties hereto and their permitted successors
and assigns.


                     [Rest of Page Intentionally Left Blank]


                                       3
<PAGE>   4

                  If the foregoing correctly sets forth the agreement among
WestPoint, Transferor, the Administrator and the Transferees with respect to the
limited waiver of the Relevant Provisions, please execute two counterparts of
this waiver letter and deliver them to the Administrator.

                                          Very truly yours,


                                          WACHOVIA BANK, N.A., as
                                          Administrator


                                          By:/s/ Frances W. Josephic
                                             -----------------------------------
                                             Name: Frances W. Josephic
                                             Title: Vice President


                                          BLUE RIDGE ASSET FUNDING
                                          CORPORATION


                                          By: WACHOVIA BANK N.A.,
                                              as Attorney-in- Fact



                                          By: /s/ Kenny Karpowicz
                                              ----------------------------------
                                              Name: Kenny Karpowicz
                                              Title: Vice President


                                       4
<PAGE>   5

Consented and Agreed to as of
the date hereof:


WPS RECEIVABLES CORPORATION

By: /s/ Nelson Griffith
    ----------------------------------------
Name: Nelson Griffith
Title: President


WESTPOINT STEVENS, INC.

By: /s/ Nelson Griffith
    ----------------------------------------
Name: Nelson Griffith
Title: Senior Vice President & Controller


                                       5