<PAGE>   1
                                FOURTH AMENDMENT
                                       TO
                              AMENDED AND RESTATED
                           LOAN AND SECURITY AGREEMENT
                                   AND CONSENT


               THIS FOURTH AMENDMENT TO AMENDED AND RESTATED LOAN AND SECURITY
AGREEMENT AND CONSENT (this "Amendment"), dated as of November 16, 1999, is
entered into between CONGRESS FINANCIAL CORPORATION (WESTERN), a California
corporation ("Lender"), on the one hand, and WHEREHOUSE ENTERTAINMENT, INC., a
Delaware corporation, WHEREHOUSE.COM, INC., a California corporation, WHEREHOUSE
SUBSIDIARY I CO., INC., a Delaware corporation, WHEREHOUSE SUBSIDIARY II CO.,
INC., a California corporation, and WHEREHOUSE SUBSIDIARY III CO., INC., a
Delaware corporation (collectively, "Borrower"), on the other.

                                    RECITALS

               A. Borrower and Lender have previously entered into an Amended
and Restated Loan and Security Agreement dated as of October 26, 1998, as
amended by a First Amendment dated as of November 30, 1998, by a Second
Amendment dated as of May 14, 1999, and as further amended by a Third Amendment
dated as of August 31, 1999 (the "Loan Agreement"), pursuant to which Lender has
made certain loans and financial accommodations available to Borrower. Terms
used herein without definition shall have the meanings ascribed to them in the
Loan Agreement.

               B. Borrower and Lender wish to amend the Loan Agreement on the
terms and conditions set forth in this Amendment. Borrower is entering into this
Amendment with the understanding and agreement that, except as specifically
provided herein, none of Lender's rights or remedies as set forth in the Loan
Agreement is being waived or modified by the terms of this Amendment.

               NOW, THEREFORE, in consideration of the foregoing and the mutual
covenants herein contained, and for other good and valuable consideration, the
receipt and sufficiency of which are hereby acknowledged, the parties hereby
agree as follows:

               1. Amendment to Covenant Regarding Investments. Clause (f) of
Section 9.10 of the Loan Agreement is amended to read in its entirety as
follows:

                        "(f) Investments or Distributions, if the aggregate
                amount of cash consideration or real and other personal property
                consideration relating to Investments made pursuant to this
                Section 9.10 does not (1) exceed Twenty Million Dollars
                i($20,000,000) (the "Investment Basket"), provided that Borrower
                has at the time of such Investment or Distribution or entering
                into a commitment therefor, after giving effect to such
                Investment or Distribution, Excess Availability of at least
                Twenty Million Dollars ($20,000,000) under the Tranche


<PAGE>   2

                        A Line, or (2) exceed Fifty Million Dollars
                ($50,000,000) (the "Investment Cap"), provided that, Borrower
                has at the time of such Investment or Distribution or entering
                into a commitment therefor, after giving effect to such
                Investment or Distribution, Excess Availability of at least
                Thirty Million Dollars ($30,000,000) under the Tranche A Line;
                provided that in the case of each of clause (1) or (2), at the
                time of such Investment or Distribution, after giving effect to
                such Investment or Distribution, there is no Event of Default
                and no event that with notice or passage of time or both would
                be an Event of Default; provided further that, with respect to
                an Investment hereunder, such Investment is an Investment in a
                Person with a line of business similar to Borrower's business;
                provided further that, the Investment Basket shall automatically
                be increased by the amount of any proceeds from the sale of
                equity securities of Borrower (not to exceed the Investment
                Cap);"

               2. Amendment to Number of Interest Periods. Subclause (iv) of
clause (b) of Section 3.1 of the Loan Agreement is amended to read in its
entirety as follows:

               "(iv) no more than six (6) Interest Periods may be in effect at
any one time,"

               3. Amendment Regarding Sale of Wherehouse Stock. Clause (b) of
Section 9.7 of the Loan Agreement is amended by adding the parenthetical phrase
"(other than stock of Wherehouse Entertainment, Inc., provided that the
consideration is in cash or the amount of stock issued is less than 15% of the
outstanding common stock of Wherehouse Entertainment, Inc. at the time of such
issuance)" after the word "stock" at the end of the fourth line of Section 9.7.

               4. Consent and Waiver. Subject to the terms and conditions set
forth herein, and in reliance on the representations and warranties of Borrower
herein contained, Lender hereby consents to any future merger, consolidation or
sale of assets of Wherehouse.com, Inc. and waives compliance by Borrower with
the terms of Section 9.7 with regard to any such merger, consolidation or sale
of assets, provided that Lender receives a pledge agreement between Lender and
Borrower, in form reasonably satisfactory to Lender relating to any securities
received by Borrower in connection with such future merger, consolidation or
sale of assets of Wherehouse.com, Inc. (which pledge agreement may grant a right
of first refusal to the issuer of such securities), provided further that, the
net book value of the assets of Wherehouse.com, Inc. transferred in one or more
such mergers, consolidations or sales of assets is less than $1,000,000 in the
aggregate. Furthermore, upon any such merger, consolidation or sale of assets of
Wherehouse.com, Inc., Lender hereby waives compliance by Wherehouse.com, Inc. of
the terms of Section 9.1.

               5. Effectiveness of this Amendment. Lender must have received the
following items, in form and content acceptable to Lender, before this Amendment
is effective and before Lender is required to extend any credit to Borrower as
provided for by this Amendment:

               (a) Amendment. This Amendment fully executed in a sufficient
number of counterparts for distribution to Lender and Borrower;



<PAGE>   3

               (b) Authorizations. Evidence that the execution, delivery and
performance by Borrower and each guarantor or subordinating creditor of this
Amendment and any instrument or agreement required under this Amendment have
been duly authorized;

               (c) Representations and Warranties. The representations and
warranties set forth in the Loan Agreement must be true and correct;

               (d) Consents. Counterparts of the Consent appended hereto (the
"Consent") executed on behalf of each of Wherehouse Holding I Co., Inc., a
Delaware corporation and Wherehouse Holding II Co., Inc., a Delaware corporation
("Guarantors", and together with Borrower, each a "Loan Party" and collectively
the "Loan Parties");

               (e) Fee. Lender shall have received an amendment fee in the
amount of $15,500.

               6. Representations and Warranties. Borrower represents and
warrants as follows:

               (a) Authority. Each Loan Party has the requisite corporate power
and authority to execute and deliver this Amendment or the Consent, as
applicable, and to perform its obligations hereunder and under the Financing
Agreements (as amended or modified hereby) to which it is a party. The
execution, delivery and performance by Borrower of this Amendment and by each
other Loan Party of the Consent, and the performance by each Loan Party of each
Financing Agreement (as amended or modified hereby) to which it is a party have
been duly approved by all necessary corporate action of such Loan Party and no
other corporate proceedings on the part of such Loan Party are necessary to
consummate such transactions.

               (b) Enforceability. This Amendment has been duly executed and
delivered by Borrower. The Consent has been duly executed and delivered by each
Guarantor. This Amendment and each Financing Agreement (as amended or modified
hereby) is the legal, valid and binding obligation of each Loan Party hereto or
thereto, enforceable against such Loan Party in accordance with its terms, and
is in full force and effect.

               (c) Representations and Warranties. The representations and
warranties contained in each Financing Agreement (other than any such
representations or warranties that, by their terms, are specifically made as of
a date other than the date hereof) are correct on and as of the date hereof as
though made on and as of the date hereof.

               (d) No Default. No event has occurred and is continuing that
constitutes an Event of Default.

               7. Choice of Law. The validity of this Amendment, its
construction, interpretation and enforcement, the rights of the parties
hereunder, shall be determined under, governed by, and construed in accordance
with the internal laws of the State of California governing contracts only to be
performed in that State.



<PAGE>   4

               8. Counterparts. This Amendment may be executed in any number of
counterparts and by different parties and separate counterparts, each of which
when so executed and delivered, shall be deemed an original, and all of which,
when taken together, shall constitute one and the same instrument. Delivery of
an executed counterpart of a signature page to this Amendment or the Consent by
telefacsimile shall be effective as delivery of a manually executed counterpart
of this Amendment or the Consent.

               9. Reference to and Effect on the Financing Agreements.

               (a) Upon and after the effectiveness of this Amendment, each
reference in the Loan Agreement to "this Agreement", "hereunder", "hereof" or
words of like import referring to the Loan Agreement, and each reference in the
other Financing Agreements to "the Loan Agreement", "thereof" or words of like
import referring to the Loan Agreement, shall mean and be a reference to the
Loan Agreement as modified and amended hereby.

               (b) Except as specifically amended above, the Loan Agreement and
all other Financing Agreements, are and shall continue to be in full force and
effect and are hereby in all respects ratified and confirmed and shall
constitute the legal, valid, binding and enforceable obligations of Borrower to
Lender.

               (c) The execution, delivery and effectiveness of this Amendment
shall not, except as expressly provided herein, operate as a waiver of any
right, power or remedy of Lender under any of the Financing Agreements, nor
constitute a waiver of any provision of any of the Financing Agreements.

               IN WITNESS WHEREOF, the parties have entered into this Amendment
as of the date first above written.

                                      CONGRESS FINANCIAL CORPORATION (WESTERN)


                                      By: _________________________________
                                      Name: D.B. Laughton
                                      Title: SVP


                                      WHEREHOUSE ENTERTAINMENT, INC.


                                      By: _________________________________
                                      Name:  Charles Fuertsch
                                      Title: Vice President, Treasurer




<PAGE>   5

                                      WHEREHOUSE.COM, INC.


                                      By: _________________________________
                                      Name: Charles Fuertsch
                                      Title: Vice President, Treasurer


                                      WHEREHOUSE SUBSIDIARY I CO., INC.


                                      By: _________________________________
                                      Name: Charles Fuertsch
                                      Title: Vice President, Treasurer


                                      WHEREHOUSE SUBSIDIARY II CO., INC.


                                      By: _________________________________
                                      Name: Charles Fuertsch
                                      Title: Vice President, Treasurer


                                      WHEREHOUSE SUBSIDIARY III CO., INC.


                                      By: _________________________________
                                      Name: Charles Fuertsch
                                      Title: Vice President, Treasurer


<PAGE>   6

                                     CONSENT



                          Dated as of November 16, 1999



               The undersigned, as Guarantors under their respective Guarantee,
each dated as of October 26, 1998 (as such terms are defined in and under the
Loan Agreement referred to in the foregoing Amendment), each hereby consents and
agrees to said Amendment and hereby confirms and agrees that its respective
Guarantee is, and shall continue to be in, in full force and effect and is
hereby ratified and confirmed in all respects except that, upon the
effectiveness of, and on and after the date of said Amendment, each reference in
each such Guarantor's Guarantee to the "Loan Agreement", "thereunder", "thereof"
or words of like import referring to the Loan Agreement, shall mean and be a
reference to the Loan Agreement as amended or modified by the said Amendment.



                                     WHEREHOUSE HOLDING I CO., INC.


                                     By: _________________________________
                                     Name: Charles Fuertsch
                                     Title: Vice President, Treasurer



                                     WHEREHOUSE HOLDING II CO., INC.


                                     By: _________________________________
                                     Name: Charles Fuertsch
                                     Title: Vice President, Treasurer