<PAGE>

                                                                   EXHIBIT 10.20

                   FOURTH AMENDMENT TO BUSINESS LOAN AGREEMENT

                This Fourth Amendment to Business Loan Agreement (the
"Amendment") is made as of January 14, 2003, between Bank of America, N.
A.("Bank"), and Pacific Sunwear of California, Inc., a California corporation
(the "Borrower").

                                    RECITALS

        A. Borrower and Bank entered into that certain Business Loan Agreement
dated as of April 3, 2001, as previously amended (the "Agreement").

        B. Borrower and Bank desire to further amend certain terms and
provisions of the Agreement.

                                    AGREEMENT

        1. Definitions. Capitalized terms used but not defined in this Amendment
shall have the meaning given to them in the Agreement.

        2. Amendments. The Agreement is hereby amended as follows:

                2.1 In Section 9.7 (f) of the Agreement, the words "Five Million
                Dollars ($5,000,000.00)" are hereby deleted in its entirety and
                replaced with "Ten Million Dollars ($10,000,000.00)".

        3. Representations and Warranties. Borrower hereby represents and
warrants to Bank that: (i) no default specified in the Agreement and no event
which with notice or lapse of time or both would become such a default has
occurred and is continuing and has not been previously waived, (ii) the
representations and warranties of Borrower pursuant to the Agreement are true on
and as of the date hereof as if made on and as of said date, (iii) the making
and performance by Borrower of this Amendment have been duly authorized by all
necessary action, and (iv) no consent, approval, authorization, permit or
license is required in connection with the making or performance of the
Agreement as amended hereby.

        4. Conditions. This Amendment will be effective when the Bank receives
the following items, in form and content acceptable to the Bank:

                4.1 This Amendment duly executed by all parties hereto.

                4.2 Payment of all out-of-pocket expenses, including attorneys'
        fees, incurred by the Bank in connection with the preparation of this
        Amendment.



                                       1
<PAGE>

        5. Effect of Amendment. Except as provided in this Amendment, the
Agreement shall remain in full force and effect and shall be performed by the
parties hereto according to its terms and provisions.


        IN WITNESS WHEREOF, this Amendment has been executed by the parties
hereto as of the date first above written.


                                        BANK OF AMERICA, N.A.


                                        By: /s/ C.K. Goodfellow
                                           -------------------------------------
                                        Name: Cynthia K. Goodfellow
                                             -----------------------------------
                                        Title: Vice President
                                           -------------------------------------


                                        PACIFIC SUNWEAR OF CALIFORNIA, INC.


                                        By: /s/ Carl W. Womack
                                           -------------------------------------
                                        Name: Carl W. Womack
                                             -----------------------------------
                                        Title: Chief Financial Officer
                                           -------------------------------------


                                        By: /s/ Greg H. Weaver
                                           -------------------------------------
                                        Name: Greg H. Weaver
                                             -----------------------------------
                                        Title: Chief Executive Officer
                                           -------------------------------------




                                       2