<PAGE>
 
                                                                    EXHIBIT 10.4

     AMENDMENT No. 3 dated as of December 3, 1996 (this "Amendment"), to the
Credit Agreement dated as of October 16, 1995, as amended by Amendment No. 1
thereto ("Amendment No. 1") dated as of February 28, 1996, and by Amendment No.
2 thereto ("Amendment No. 2") dated as of April 15, 1996 (as so amended and as
the same may be further amended, restated, supplemented, waived or otherwise
modified from time to time, the "Credit Agreement"), among RBX GROUP, INC., a
Delaware corporation ("RBX Group"), RBX CORPORATION, a Delaware corporation and
a wholly owned subsidiary of RBX Group (the "Borrower"), the several banks and
other financial institutions party to the Credit Agreement (the "Lenders"), and
THE CHASE MANHATTAN BANK (as successor in interest to Chemical Bank), a New York
banking corporation, as agent for the Lenders (in such capacity, the "Agent").

     A.   The Borrower has requested that Subsection 8(a), Subsection 8(b),
Subsection 8(c), Subsection 8(e) and Section 8.10 of the Credit Agreement be
amended in accordance with the terms and subject to the conditions set forth
herein.

     B.   The Borrower and the Required Lenders have agreed to the foregoing
amendments, in each case on the terms and subject to the conditions set forth
herein.

     C.   Capitalized terms used but not defined herein shall have the meanings
assigned to them in the Credit Agreement as amended hereby.

     Accordingly, in consideration of the mutual agreements herein contained and
other good and valuable consideration, the sufficiency and receipt of which are
hereby acknowledged, the parties hereto agree as follows:

     SECTION 1.  Amendments to Credit Agreement.

     (a)  Amendments to Section 1.1

          (I)  the definition of the term "Consolidated EBIT" shall be deleted
and the following shall be inserted in lieu thereof:

               "Consolidated EBIT": of the Borrower and its consolidated
          Subsidiaries, for any period. Consolidated Net Income for such period
          (a) plus, without duplication and to the extent reflected as a charge
          in the statement of such Consolidated Net Income for such period, the
          sum of (I) Consolidated Interest Expense and any other interest
          expense, (ii) total income tax expense (whether paid, accrued or
          deferred), (iii) non-cash provisions for reserves for discontinued
          operations, (iv) any non-cash, extraordinary losses (including,
          whether or not otherwise includable as a separate item in the
          statement of such Consolidated Net Income for such period, losses on
          the Disposition of assets outside of the ordinary course of business)
          and (v) charges not in excess of $30,000,000, relating to Financial
          Accounting Standards No. 121 and special charges not in excess of
          $8,000,000, each as reflected on the Borrower's consolidated financial
          statements for the period ended December 31, 1996, and (b) minus, to
          the extent reflected as a credit in the statement of such Consolidated
          Net Income for such period, any non-cash, extraordinary gains
          (including, whether or not otherwise includable as a separate item in
          the statement of such Consolidated Net Income for such period, gains
          on the Disposition of assets outside of the ordinary course of
          business)."

          (ii)  the definition of "Consolidated Net Worth" shall be amended (x)
     deleting the word "or" in clause (ii) thereof and inserting a "," in lieu
     thereof and (y) by adding the following provision at the end of such
     definition:


<PAGE>

                                                                               2

 
          "or (iv) charges not in excess of $30,000,000, relating to Financial
          Accounting Standards No. 121 and special charges not in excess of
          $8,000,000, each as reflected on the Borrower's consolidated financial
          statements for the period ended December 31, 1996."

     (b)  Amendment to Subsection 8.1(a). Subsection 8.1(a) of the Credit
Agreement shall be amended by deleting the ratio set forth therein opposite each
date set forth below and inserting in place thereof the ratio set forth opposite
such date below:

<TABLE>
<CAPTION>
 
                                         Consolidated
                                           Interest
          Date                          Expense Ratio
          ----                          --------------
          <S>                           <C> 
 
          December 31, 1996             1.20 to 1.00
    
          March 31, 1997                1.25 to 1.00
          June 30, 1997                 1.30 to 1.00
          September 30, 1997            1.35 to 1.00
          December 31, 1997             1.40 to 1.00

          March 31, 1998                1.40 to 1.00
          June 30, 1998                 1.45 to 1.00    
          September 30, 1998            1.60 to 1.00
          December 31, 1998             1.65 to 1.00

          March 31, 1999                1.80 to 1.00
          June 30, 1999                 1.85 to 1.00 
          September 30, 1999            1.90 to 1.00   
          December 31, 1999             1.95 to 1.00

          March 31, 2000                2.00 to 1.00
          June 30, 2000                 2.10 to 1.00
          September 30, 2000            2.15 to 1.00
          December 31, 2000             2.20 to 1.00

          March 31, 2001                2.30 to 1.00
          June 30, 2001                 2.35 to 1.00
          September 30, 2001            2.40 to 1.00
          December 31, 2001             2.50 to 1.00

          March 31, 2002                2.60 to 1.00
          June 30, 2002                 2.65 to 1.00  
          September 30, 2002            2.75 to 1.00  
          December 31, 2002             2.80 to 1.00

          March 31, 2003                3.00 to 1.00 
          June 30, 2003                 3.10 to 1.00
          September 30, 2003            3.20 to 1.00

</TABLE>
     (c) Amendment to Subsection 8.1(b).  Subsection 8.1(b) of the Credit
Agreement is hereby amended and restated in its entirety as follows:
<PAGE>
 
                                                                               3

          "(b) Maintenance of Consolidated EBITDA. Permit for any period of four
consecutive fiscal quarters ending on any fiscal quarter set forth below the
Consolidated EBITDA for the Borrower to be less than the amount set forth below
opposite the applicable fiscal quarter or fiscal year:

<TABLE>
<CAPTION>                   
                                       Consolidated             
          Fiscal Year Ending              EBITDA                  
          ------------------           ------------             
          <S>                          <C> 
          December 31, 1996            $22,000,000             
                                                               
          March 31, 1997                23,000,000             
          June 30, 1997                 24,000,000             
          September 30, 1997            25,000,000             
          December 31, 1997             26,000,000             
                                                               
          March 31, 1998                27,000,000             
          June 30, 1998                 28,500,000             
          September 30, 1998            31,000,000             
          December 31, 1998             33,000,000             
                                                               
          March 31, 1999                34,500,000             
          June 30, 1999                 35,500,000             
          September 30, 1999            36,000,000             
          December 31, 1999             36,500,000             
                                                               
          March 31, 2000                36,500,000             
          June 30, 2000                 37,000,000             
          September 30, 2000            37,500,000             
          December 31, 2000             38,000,000             
                                                               
          March 31, 2001                38,000,000             
          June 30, 2001                 38,500,000             
          September 30, 2001            38,500,000             
          December 31, 2001             39,000,000             
                                                               
          March 31, 200                 39,500,000             
          June 30, 2002                 40,000,000             
          September 30, 2002            40,500,000             
          December 31, 2002             40,500,000             
                                                               
          March 31, 2003                41,000,000  
          June 30, 2003                 41,000,000  
          September 30, 2003            41,500,000   
</TABLE>

     (d) Amendment to Subsection 8.1(c). Subsection 8.1(c) of the Credit
Agreement is hereby amended by deleting the ratio set forth therein opposite
each date set forth below and inserting in place thereof the ratio set forth
opposite such date below:

<TABLE>
<CAPTION>                   
          Fiscal Quarter Ending         Leverage Ratio
          ---------------------         --------------             
          <S>                           <C> 
          December 31, 1996             8.40 to 1.00

          March 31, 1997                8.20 to 1.00
          June 30, 1997                 8.00 to 1.00
</TABLE>

<PAGE>
 
                                                                               4

<TABLE>
<CAPTION> 
                         Fiscal Quarter Ending    Leverage Ratio
                         ---------------------    ---------------     
                         <S>                      <C>
                         September 30, 1997       7.40 to 1.00
                         December 31, 1997        7.15 to 1.00

                         March 31, 1998           7.00 to 1.00
                         June 30, 1998            6.60 to 1.00
                         September 30, 1998       5.80 to 1.00
                         December 31, 1998        5.40 to 1.00

                         March 31, 1999           5.25 to 1.00
                         June 30, 1999            5.00 to 1.00
                         September 30, 1999       4.75 to 1.00
                         December 31, 1999        4.60 to 1.00

                         March 31, 2000           4.60 to 1.00
                         June 30, 2000            4.45 to 1.00
                         September 30, 2000       4.10 to 1.00
                         December 31, 2000        4.00 to 1.00

                         March 31, 2001           4.00 to 1.00
                         June 30, 2001            3.85 to 1.00
                         September 30, 2001       3.60 to 1.00
                         December 31, 2001        3.40 to 1.00

                         March 31, 2002           3.40 to 1.00
                         June 30, 2002            3.20 to 1.00
                         September 30, 2002       2.90 to 1.00
                         December 31, 2002        2.80 to 1.00

                         March 31, 2003           2.80 to 1.00
                         June 30, 2003            2.70 to 1.00
                         September 30, 2003       2.50 to 1.00
</TABLE>
     (e)  Amendment to Subsection 8.1(e). Subsection 8.1(e) of the Credit
Agreement is hereby amended by deleting the ratio set forth therein opposite
each date set forth below and inserting in place thereof the ratio set forth
opposite such date below:
<TABLE>
<CAPTION>

                         
                         Fiscal Quarter Ending    Leverage Ratio
                         ---------------------    --------------
                         <S>                      <C>
                         December 31, 1996        3.85 to 1.00

                         March 31, 1997           3.85 to 1.00
                         June 30, 1997            3.85 to 1.00
                         September 30, 1997       3.40 to 1.00
                         December 31, 1997        3.35 to 1.00

                         March 31, 1998           3.35 to 1.00
                         June 30, 1998            3.10 to 1.00
                         September 30, 1998       2.65 to 1.00
                         December 31, 1998        2.50 to 1.00
</TABLE>
<PAGE>
 
                                                                               5

<TABLE>                                          
                        Fiscal Quarter Ending    Leverage Ratio
                        ---------------------    --------------
                        <S>                      <C>
                        March 31, 1999           2.35 to 1.00
                        June 30, 1999            2.20 to 1.00
                        September 30, 1999       2.00 to 1.00
                        December 31, 1999        1.85 to 1.00
                        March 31, 2000           1.85 to 1.00
                        June 30, 2000            1.75 to 1.00
                        September 30, 2000       1.45 to 1.00
                        December 31, 2000        1.40 to 1.00
                        
                        March 31, 2001           1.40 to 1.00
                        June 30, 2001            1.40 to 1.00
                        September 30, 2001       1.40 to 1.00
                        December 31, 2001        1.40 to 1.00
        
                        March 31, 2002           1.40 to 1.00
                        June 30, 2002            1.40 to 1.00
                        September 30, 2002       1.40 to 1.00
                        December 31, 2002        1.40 to 1.00
        
                        March 31, 2003           1.40 to 1.00
                        June 30, 2003            1.40 to 1.00
                        September 30, 2003       1.40 to 1.00
</TABLE> 


     (f) Amendment to Section 8.10.  Section 8.10 of the Credit Agreement is
hereby amended by deleting all references therein to "$10,000,000" and inserting
in lieu thereof "$5,000,000".

     SECTION 2.  Fees.  In consideration of the agreements of the Required
Lenders contained herein, the Borrower agrees to pay to each Lender, through the
Agent, an amendment fee (the "Amendment Fee") to any Lender that shall execute
and deliver this Amendment to the Agent on or prior to December 18, 1996, an
amount equal to 1/5 of 1% of the sum of (I) the outstanding Term Loans of such
Lender and (ii) the Revolving Credit Commitment (whether used or unused) of such
Lender.  The Amendment Fee shall be payable in immediately available funds on
the Effective Date (as defined below).  Once paid, the Amendment Fee shall not
be refundable under any circumstances.

     SECTION 3.  Representations and Warranties.  To induce the other parties
hereto to enter into this Amendment, the Borrower represents and warrants to
each of the Lenders and the Agent, that, after giving effect to this Amendment,
(a) the representations and warranties set forth in Section 5 of the Credit
Agreement are true and correct in all material respects on and as of the date
hereof with the same effect as though made on and as of the date hereof, except
to the extent such representations and warranties expressly relate to an earlier
date, and (b) no Default or Event of Default has occurred and is continuing.

     SECTION 4.  Conditions to Effectiveness.  This Amendment shall become
effective on the date (the "Effective Date") that the Agent shall have received
counterparts of this Amendment that, when taken together, bear the signatures of
RBX Group, the Borrower and the Required Lenders.

     SECTION 5.  Effect of Amendment.  Except as expressly set forth herein,
this Amendment shall not by implication or otherwise limit, impair, constitute a
waiver of, or otherwise affect the rights and remedies of the Lenders, the
Agent, RBX Group or the Borrower under the Credit Agreement, Amendment No. 1,
Amendment No. 2 or any other Loan Document, and shall not alter, modify, amend
or in any way affect any of the terms, conditions, obligations, covenants or
agreements contained in the Credit Agreement, Amendment No. 1, Amendment No. 2
or
<PAGE>
         
                                                                               6

any other Loan Document, all of which are ratified and affirmed in all respects
and shall continue in full force and effect.  Nothing herein shall be deemed to
entitle RBX Group or the Borrower to a consent to, or a waiver, amendment,
modification or other change of, any of the terms, conditions, obligations,
covenants or agreements contained in the Credit Agreement, Amendment No. 1,
Amendment No. 2 or any other Loan Document in similar or different
circumstances.

     SECTION 6.  Counterparts.  This Amendment may be executed in any number of
counterparts and by different parties hereto in separate counterparts, each of
which when so executed and delivered shall be deemed an original, but all such
counterparts together shall constitute but one and the same instrument.
Delivery of any executed counterpart of a signature page of this Amendment by
facsimile transmission shall be as effective as delivery of a manually executed
counterpart hereof.

     SECTION 7.  APPLICABLE LAW.  THIS AMENDMENT SHALL BE GOVERNED BY, AND
CONSTRUED IN ACCORDANCE WITH, THE LAWS OF THE STATE OF NEW YORK.

     SECTION 8.  Headings.  The headings of this Amendment are for purposes of
reference only and shall not limit or otherwise affect the meaning hereof.


     IN WITNESS WHEREOF, the parties hereto have caused this Amendment to be
duly executed by their duly authorized officers, all as of the date and year
first above written.



                                          RBX GROUP, INC.,

                                          by /s/ Thomas F. Lemker
                                          ----------------------------------
                                          Name:  Thomas F. Lemker
                                          Title: VP, CFO & Treasurer


                                          RBX CORPORATION,


                                          by /s/ Thomas F. Lemker
                                          ----------------------------------
                                          Name:  Thomas F. Lemker
                                          Title:  VP, CFO & Treasurer


                                          THE CHASE MANHATTAN BANK,
                                          as Agent and as a Lender,


                                          by /s/ William J. Caggiano
                                          ----------------------------------
                                          Name:  William J. Caggiano
                                          Title:  Managing Director
<PAGE>
 
                                                                               7

                                CHASE MANHATTAN BANK DELAWARE, as an Issuing
                                Lender,


                                by /s/ Michael P. Handago
                                   -----------------------------------------
                                   Name: Michael P. Handago
                                   Title: Vice President


                                ABN-AMRO BANK N.V.,
                                San Francisco International Branch,


                                by -----------------------------------------
                                   Name:
                                   Title:


                                by -----------------------------------------
                                   Name:
                                   Title:


                                THE BANK OF NOVA SCOTIA,


                                by /s/ James York
                                   -----------------------------------------
                                   Name: James York
                                   Title: Office Head


                                CREDITANSTALT BANKVEREIN,


                                by /s/ Gregory T. Mathis
                                   -----------------------------------------
                                   Name: Gregory T. Mathis
                                   Title: Vice President


                                by /s/ Fiona McKone
                                   -----------------------------------------
                                   Name: Fiona McKone
                                   Title: Senior Associate
<PAGE>
 
                                                                               8

                                CRESTAR BANK,



                                by /s/ Stuart J. Turille
                                   --------------------------------------------
                                   Name: Stuart J. Turille
                                   Title: Executive Vice President


                                FIRST NATIONAL BANK OF BOSTON,



                                by
                                   --------------------------------------------
                                   Name:
                                   Title:


                                FLEET BANK OF MASSACHUSETTS, N.A.,



                                by /s/ James C. Silva
                                   --------------------------------------------
                                   Name: James C. Silva
                                   Title: AVP


                                MIDLAND BANK PLC, New York Branch,



                                by /s/ Gina Sidorsky
                                   --------------------------------------------
                                   Name: Gina Sidorsky
                                   Title: Director


                                NATIONSBANK, N.A.,



                                by /s/ Hugh S. Miles, III
                                   --------------------------------------------
                                   Name: Hugh S. Miles, III
                                   Title: Executive Vice President
<PAGE>
 
                                                                               9

                                THE PROVIDENT BANK,



                                by /s/ Alan R. Henning
                                   -------------------------------------------- 
                                   Name: Alan R. Henning
                                   Title: Vice President


                                VAN KAMPEN AMERICAN CAPITAL
                                PRIME RATE INCOME TRUST,



                                by /s/ Jeffrey W. Maillet
                                   -------------------------------------------- 
                                   Name: Jeffrey W. Maillet
                                   Title: Senior Vice President & Director