<PAGE>

                                                                   EXHIBIT 10.24

                                    CONSENT


     THIS CONSENT (this "Consent"), dated as of the 31st day of March, 2000,
                         -------
between PILLOWTEX CORPORATION, a Texas corporation (the "Borrower"), and BANK OF
                                                         --------
AMERICA, N.A. (formerly known as NationsBank, N.A.) ("Lender").
                                                      ------


                                 BACKGROUND
                                 ----------

     A.  The Borrower executed that certain Promissory Note, dated May 4,1999,
in the original principal amount of $20,000,000 and increased by amendment to
$35,000,000 (said Promissory Note, as amended, the "Promissory Note"; terms
                                                    ---------------
defined in the Promissory Note and not otherwise defined herein shall be used
herein as defined in the Promissory Note).

     B.  Article IV of the Promissory Note incorporates by reference Articles 5
(except for Section 5.8), 6 and 7 of the Credit Agreement and all definitions
thereto, and provides that no amendment of the Credit Agreement after May 4,
1999 shall be effective with respect to such Articles as they are incorporated
into the Promissory Note without the written consent of the Lender.

     C.  The Borrower is entering into a Waiver and Seventh Amendment to Amended
and Restated Credit Agreement, dated to be effective as of March 31, 2000, with
the lenders party thereto and Bank of America, N.A., as administrative agent
(the "Seventh Amendment"), which, among other things, amends certain provisions
      -----------------
of Articles 6 and 7 of the Credit Agreement.

     D.  The Borrower has requested that the Lender execute this Consent.

     NOW, THEREFORE, in consideration of the covenants, conditions and
agreements hereafter set forth, and for other good and valuable consideration,
the receipt and adequacy of which are all hereby acknowledged, the parties
hereto covenant and agree as follows:

     1.  CONSENT.  The Lender hereby (a) consents to the Seventh Amendment and
         -------
(b) agrees that the amendment of certain provisions of Articles 6 and 7 of the
Credit Agreement as provided in the Seventh Amendment shall be effective as of
March 31, 2000 with respect to the Promissory Note such that those provisions of
the Credit Agreement, as amended by the Seventh Amendment, are incorporated into
the Promissory Note.

     2.  REPRESENTATIONS AND WARRANTIES TRUE; NO EVENT OF DEFAULT.  By its
         --------------------------------------------------------
execution and delivery hereof, the borrower represents and warrants that, as of
the date hereof and after giving effect to the Consent set forth in the
foregoing Section 1:
     (a) the representations and warranties contained in the Promissory Note are
true and correct on and as of the date hereof as made on and as of such date;
and
<PAGE>

     (b) no event has occurred and is continuing which constitutes an Event of
Default.

     3.  CONDITIONS OF EFFECTIVENESS.  This Consent shall be effective as of
         ---------------------------
March 31, 2000, subject to the following:

     (a) the representations and warranties set forth in Section 2 of this
Consent shall be true and correct;

     (b) the Lender shall have received counterparts of this Consent executed by
the Borrower and acknowledged by each Guarantor; and

     (c) the Lender shall have received in form and substance satisfactory to
the Lender, such other documents, certificates and instruments as the Lender
shall require.

     4.  GUARANTOR'S ACKNOWLEDGMENT. By signing below, each Guarantor (i)
         --------------------------
acknowledges, consents and agrees to the execution, delivery and performance by
the Borrower of this Consent, (ii) acknowledges and agrees that its obligations
in respect of its Guaranty Agreement are not released, diminished, waived,
modified, impaired or affected in any manner by this Consent, or any of the
provisions contemplated herein, (iii) ratifies and confirms its obligations
under its Guaranty Agreement, and (iv) acknowledges and agrees that it has no
claim or offsets against, or defenses or counterclaims to, its Guaranty
Agreement as a result of this Consent.

     5.  REFERENCE TO THE PROMISSORY NOTE.
         --------------------------------

     (a) Upon the effectiveness of this Consent, each reference in the
Promissory Note to "this Note", "hereunder", or words of like import shall mean
and be a reference to the Promissory Note, as affected by this Consent.

     (b) The Promissory Note, as affected by this Consent, shall remain in full
force and effect and is hereby ratified and confirmed.

     6.  COSTS, EXPENSES AND TAXES.  The Borrower shall be obligated to pay the
         -------------------------
costs and expenses of the Lender in connection with the preparation,
reproduction, execution and delivery of this Consent and the other instruments
and documents to be delivered hereunder.

     7.  EXECUTION IN COUNTERPARTS; EXECUTION VIA FACSIMILE.  This Consent may
         --------------------------------------------------
be executed in any number of counterparts and by different parties hereto in
separate counterparts, each of which when so executed and delivered shall be
deemed to be an original and all of which when taken together shall constitute
but one and the same instrument.  This Consent may be validly executed and
delivered by facsimile or other electronic transmission.

                                      -2-
<PAGE>

     8.  GOVERNING LAW; BINDING EFFECT.  This Consent shall be governed by and
         -----------------------------
construed in accordance with the laws of the State of Texas, and shall be
binding upon the Borrower and each Lender and their respective successors and
assigns.

     9.  HEADINGS.  Section headings in this Consent are included herein for
         --------
convenience of reference only and shall not constitute a part of this Consent
for any other purpose.

     10.  ENTIRE AGREEMENT.  THE PROMISSORY NOTE, AS AMENDED BY THIS CONSENT,
          ----------------
REPRESENT THE FINAL AGREEMENT BETWEEN THE PARTIES AS TO THE SUBJECT MATTER
THEREIN AND HEREIN AND MAY NOT BE CONTRADICTED BY EVIDENCE OF PRIOR,
CONTEMPORANEOUS, OR SUBSEQUENT ORAL AGREEMENTS BETWEEN THE PARTIES. THERE ARE NO
UNWRITTEN ORAL AGREEMENTS BETWEEN THE PARTIES.


===============================================================================
                         REMAINDER OF PAGE LEFT INTENTIONALLY BLANK
===============================================================================

                                      -3-
<PAGE>

     IN WITNESS WHEREOF, the parties hereto have executed this Consent as of the
date first above written.

                              PILLOWTEX CORPORATION

                              Jaime Vasquez
                              Vice President and Treasurer

                                      -4-
<PAGE>

BANK OF AMERICA, N.A. (formerly known as NationsBank, N.A.)



ACKNOWLEDGED AND AGREED:

PILLOWTEX, INC.
PTEX HOLDING COMPANY
PILLOWTEX MANAGEMENT SERVICES COMPANY
BEACON MANUFACTURING COMPANY
MANETTA HOME FASHIONS, INC.
TENNESSEE WOOLEN MILLS
FIELDCREST CANNON, INC.
CRESTFIELD COTTON COMPANY
ENCEE, INC.
FCC CANADA, INC.
FIELDCREST CANNON FINANCING, INC.
FIELDCREST CANNON LICENSING, INC.
FIELDCREST CANNON INTERNATIONAL, INC.
FIELDCREST CANNON SF, INC. (formerly know as Fieldcrest Cannon Sure Fit, Inc.)
FIELDCREST CANNON TRANSPORTATION, INC,
ST. MARYS, INC.
AMOSKEAG COMPANY
AMOSKEAG MANAGEMENT CORPORATION
DOWNEAST SECURITIES CORPORATION
BANGOR INVESTMENT COMPANY
MOORE'S FALLS CORPORATION
THE LESHNER CORPORATION
LESHNER OF CALIFORNIA, INC.
OPELIKA INDUSTRIES, INC.


Jaime Vasquez
Vice President and Treasurer

                                      -5-