<PAGE>   1
                                                                    EXHIBIT 10.4

                                FIRST AMENDMENT
                      AMENDED AND RESTATED LOAN AGREEMENT


     THIS FIRST AMENDMENT is made and entered into as of September 05, 1996 by
and between QUIKSILVER, INC., a Delaware Corporation ("Borrower"), with an
office at 1740 Monrovia, Costa Mesa, California 92627 and UNION BANK, A DIVISION
OF UNION BANK OF CALIFORNIA, INC., N.A. ("Bank"), with an office at 500 S. Main
Street, 2nd Floor, Orange, California 92868.

     WHEREAS, Bank and Borrower entered into an Amended and Restated Loan
Agreement dated April 30, 1996 ("Loan Agreement"); and

     WHEREAS, the Bank and Borrower desire to enter into this First Amendment to
the Loan Agreement in order to reflect the current terms of the agreement
between the parties.

     NOW, THEREFORE, in consideration of the foregoing recitals and of the
mutual covenants set forth below and other good and valuable consideration, the
receipt and adequacy of which is hereby acknowledged, the parties hereto agree
as follows:

     The following section is amended:

     Section 1.1.1.1, line nine of the Loan Agreement is hereby amended by
substituting the number "180" for the number "120".

     Except as amended hereby, the Agreement shall remain unaltered and in full
force and effect.  This letter shall not be a waiver of any existing default or
breach of a covenant unless specified herein.

WITNESS the due execution hereof in the date set forth hereinabove.

"Borrower"                              "Bank"

QUIKSILVER, INC.                         UNION BANK, a division of Union Bank of
                                         California, Inc., N.A.


By; /s/ RANDALL E. HERREL, SR.           By: /s/ [ILLEGIBLE SIGNATURE]
---------------------------------        ----------------------------------
        Randall E. Herrel, Sr.               
Title:  President                        Title: Vice President


By: /s/ ROBERT E. MCKNIGHT, JR.          By: /s/ [ILLEGIBLE SIGNATURE]
---------------------------------        ----------------------------------
        Robert E. McKnight, Jr.               
Title:  Chief Executive Officer          Title:  Vice President
<PAGE>   2

                                SECOND AMENDMENT
                     TO AMENDED AND RESTATED LOAN AGREEMENT


THIS SECOND AMENDMENT TO AMENDED AND RESTATED LOAN AGREEMENT (this "Second
Amendment") dated as of October 22, 1996, is made and entered into by and
between QUIKSILVER, INC., a Delaware corporation ("Borrower"), and UNION BANK
OF CALIFORNIA, N.A.

                                   RECITALS:

A.    Borrower and Bank are parties to that certain Amended and Restated Loan
Agreement dated April 30, 1996 (the "Agreement"), pursuant to which Bank
agreed to extend credit to Borrower and amendment thereto dated September 5,
1996.

B.    Borrower and Bank desire to amend the Agreement subject to the terms and
conditions of this Second Amendment.

                                   AGREEMENT:

In consideration of the above recitals and of the mutual covenants and
conditions contained herein, Borrower and Bank agree as follows:

1.    DEFINED TERMS.  Initially capitalized terms used herein which are not
otherwise defined shall have the meanings assigned thereto in the Agreement.

2.    AMENDMENTS TO THE AGREEMENT.

      (a)   Section 5.7 of the Agreement is hereby amended in its entirety as
follows:

      5.7   RETIREMENT OF STOCK.  Without consent of Bank, between
September 13, 1996 and September 30, 1997, Borrower will not acquire, redeem,
purchase, or retire more than Five Million Dollars ($5,000,000) of its capital
stock for value whether now or hereafter outstanding; or grant or issue any
equity or security or warrant, right, or option pertaining thereto except as
set forth in Section 3.13 and as approved by Bank in writing.  From October 1,
1997, without consent of Bank, Borrower will not acquire, redeem, purchase, or
retire any share of its capital stock for value whether now or hereafter
outstanding; or grant or issue any equity or security or warrant, right, or
option pertaining thereto except as set forth in Section 3.13 and as approved
by Bank in writing.

3.    EFFECTIVENESS OF THE SECOND AMENDMENT.  This Second Amendment shall
become effective as of the date hereof when, and only when, Bank shall have
received the following, in form and substance satisfactory to Bank:

      (a)    The counterpart of this Second Amendment, duly executed by
Borrower;


                                      -1-
<PAGE>   3
4.    RATIFICATION.  Except as specifically amended hereinabove, the Agreement
shall remain in full force and effect and is hereby ratified and confirmed.

5.    REPRESENTATIONS AND WARRANTIES.  Borrower represents and warrants as
follows:

      (a)    Each of the representations and warranties contained in the
Agreement, as may be amended hereby, is hereby reaffirmed as of the date
hereof, each as if set forth herein;

      (b)    The execution, delivery and performance of the Second Amendment
and any other instruments or documents in connection herewith are within
Borrower's power, have been duly authorized, are legal, valid and binding
obligations of Borrower, and are not in conflict with the terms of any charter,
bylaw, or other organization papers of Borrower or with any law, indenture,
agreement or undertaking to which Borrower is a party or by which Borrower is
bound or affected;

      (c)    No event has occurred and is continuing or would result from this
Second Amendment which constitutes or would constitute an Event of Default
under the Agreement.

6.    GOVERNING LAW.  This Second Amendment and all other instruments or
documents in connection herewith shall be governed by and construed according
to the laws of the State of California.

7.    COUNTERPARTS.  This Second Amendment may be executed in two or more
counterparts, each of which shall be deemed an original and all of which
together shall constitute one and the same instrument.

WITNESS the due execution hereof as of the date first above written.


QUIKSILVER, INC.                                 UNION BANK OF CALIFORNIA, N.A.

By: /s/ RANDALL L. HERREL, SR.                   By: /s/ RITA DAILEY
-------------------------------                  -------------------------------
        Randall L. Herrel, Sr.                           Rita Dailey
        President                                        Vice President


By: /s/ ROBERT B. MCKNIGHT, JR.
-------------------------------
        Robert B. McKnight, Jr.
        Chief Executive Officer





                                      -2-
<PAGE>   4

                                THIRD AMENDMENT
                     TO AMENDED AND RESTATED LOAN AGREEMENT


THIS THIRD AMENDMENT TO AMENDED AND RESTATED LOAN AGREEMENT (this "Third
Amendment") dated as of November 29, 1996, is made and entered into by and
between QUIKSILVER, INC., a Delaware corporation ("Borrower"), and UNION BANK
OF CALIFORNIA, N.A.

                                   RECITALS:

A.    Borrower and Bank are parties to that certain Amended and Restated Loan
Agreement dated April 30, 1996 (the "Agreement"), pursuant to which Bank
agreed to extend credit to Borrower and amendments thereto dated September 5,
1996 and October 22, 1996.

B.    Borrower and Bank desire to amend the Agreement subject to the terms and
conditions of this Third Amendment.

                                   AGREEMENT:

In consideration of the above recitals and of the mutual covenants and
conditions contained herein, Borrower and Bank agree as follows:

1.    DEFINED TERMS.  Initially capitalized terms used herein which are not
otherwise defined shall have the meanings assigned thereto in the Agreement.

2.    AMENDMENTS TO THE AGREEMENT.

      (a)    Section 1.1.1.1, line nine of the Loan Agreement is hereby amended
by substituting the number "240" for the number "180".

3.    EFFECTIVENESS OF THE THIRD AMENDMENT.  This Third Amendment shall become
effective as of the date hereof when, and only when, Bank shall have received
the following, in form and substance satisfactory to Bank:

      (a)    The Third Amendment, duly executed by Borrower;

4.    RATIFICATION.  Except as specifically amended hereinabove, the Agreement
shall remain in full force and effect and is hereby ratified and confirmed.

5.    REPRESENTATIONS AND WARRANTIES.  Borrower represents and warrants as
follows:

      (a)    Each of the representations and warranties contained in the
Agreement, as may be amended hereby, is hereby reaffirmed as of the date
hereof, each as if set forth herein;



                                      -1-
<PAGE>   5
      (b)    The execution, delivery and performance of the Third Amendment and
any other instruments or documents in connection herewith are within Borrower's
power, have been duly authorized, are legal, valid and binding obligations of
Borrower, and are not in conflict with the terms of any charter, bylaw, or
other organization papers of Borrower or with any law, indenture, agreement or
undertaking to which Borrower is a party or by which Borrower is bound or
affected;

      (c)    No event has occurred and is continuing or would result from this
Third Amendment which constitutes or would constitute an Event of Default under
the Agreement.

6.    GOVERNING LAW.  This Third Amendment and all other instruments or
documents in connection herewith shall be governed by and construed according
to the laws of the State of California.

7.    COUNTERPARTS.  This Third Amendment may be executed in two or more
counterparts, each of which shall be deemed an original and all of which
together shall constitute one and the same instrument.

WITNESS the due execution hereof as of the date first above written.


QUIKSILVER, INC.                                 UNION BANK OF CALIFORNIA, N.A.

By: /s/ ROBERT B. MCKNIGHT, JR.                  By: /s/ RITA DAILEY
--------------------------------                 ------------------------------
        Robert B. McKnight, Jr.                          Rita Dailey 
        Chief Executive Officer                          Vice President


By: /s/ STEVEN L. BRINK                          By: /s/ ROBERT E. MCGOWN
--------------------------------                 ------------------------------
        Steven L. Brink                                  Robert E. McGown 
        Vice President                                   Senior Vice President



                                      -2-