<PAGE>

                                                                  EXHIBIT 10.4

                                                                EXECUTION COPY


                    INTERCREDITOR AGREEMENT dated as of December 11, 1997,
               among RBX CORPORATION, a Delaware corporation (the "Borrower"),
               each Subsidiary of the Borrower listed on Schedule I hereto (each
               such Subsidiary, individually, a "Subsidiary Guarantor" and,
               collectively, the "Subsidiary Guarantors"; the Borrower and the
               Subsidiary Guarantors are referred to collectively herein as the
               "Grantors"), THE CHASE MANHATTAN BANK, a New York banking
               corporation ("Chase"), as agent (in such capacity, the "Agent")
               for the Senior Secured Parties (as defined below), and STATE
               STREET BANK AND TRUST COMPANY, as trustee (in such capacity, the
               "Trustee") for the Noteholders (as defined below) pursuant to the
               Senior Secured Note Indenture (as defined below).

        Reference is made to (a) the Credit Agreement dated as of December 11,
1997 (as amended, supplemented or otherwise modified from time to time, the
"Credit Agreement"), among the Borrower, the several lenders from time to time
parties thereto (the "Lenders") and the Agent, (b) the Guarantee Agreement
dated as of December 11, 1997 (as amended, supplemented or otherwise modified
from time to time, the "Guarantee Agreement"), among the Subsidiary Guarantors
and the Agent, and (c) the Security Agreement dated as of December 11, 1997,
among the Borrower, the Subsidiary Guarantors party thereto and the Agent (as
amended, supplemented or otherwise modified from time to time, the "Senior
Security Agreement", and together with this Agreement and any other documents
given after the date hereof for purposes of providing collateral security or
credit support for the loans made pursuant to the Credit Agreement, the
"Senior Security Documents").  The Lenders, the Agent, the beneficiaries of
each indemnification obligation undertaken by any Grantor under the Credit
Agreement, the Senior Security Documents and the Guarantee Agreement, and the
successors and assigns of each of the foregoing, are referred to herein as the
"Senior Secured Parties".

        Reference is also made to (a) the Indenture dated as of December 11,
1997 (the "Senior Secured Note Indenture"), between the Borrower and the
Trustee for the holders (collectively, the "Noteholders") of the Borrower's
12% Senior Secured Notes due 2003 (the "Notes"), (b) the Company Security
Agreement dated as of December 11, 1997, between the Borrower and the Trustee
for the benefit of the Junior Secured Parties (as defined below) (as amended,
supplemented or otherwise modified from time to time, the "Company Security
Agreement") and (c) the Subsidiaries' Security Agreement dated as of December
11, 1997, among the Subsidiary Guarantors party thereto and the Trustee for
the benefit of the Junior Secured Parties (as amended, supplemented or
otherwise modified from time to time, the "Subisidiaries' Security Agreement;
the Company Security Agreement and the Subsidiaries' Security Agreement are
referred to collectively herein as the "Junior Security Agreements" and,
together with this Agreement, as the "Junior Security Documents").  The
Noteholders, the Trustee, the beneficiaries of each indemnification obligation
undertaken by any Grantor under the Notes, the Senior Secured Note Indenture,
the Junior Security Agreements and the successors and assigns of each of and
the foregoing, are referred to herein as the "Junior Secured Parties".  The
Senior Secured Parties and the Junior Secured Parties are referred to herein
as the "Secured Parties".

        Pursuant to the Senior Security Agreement, the Grantors have granted
to the Agent for the ratable benefit of the Senior Secured Parties
first-priority security interests (the "Senior Security Interest") in the
Collateral (as defined in the Senior Security Agreement as in effect on the
date hereof) to secure the payment and performance of their monetary and other
obligations under the
<PAGE>

                                                                               2


Credit Agreement, the Guarantee Agreement and the Senior Security Documents
(the "Senior Obligations").  Pursuant to the Junior Security Agreements, the
Grantors have granted to the Trustee for the ratable benefit of the Junior
Secured Parties, among other collateral, second-priority security interests
(the "Junior Security Interest") in the Collateral to secure the payment and
performance of their monetary and other obligations with respect to the Notes,
the Senior Secured Note Indenture, the Junior Security Documents and any other
documents providing collateral security or credit support for the obligations
under the Indenture and the Notes (the "Junior Obligations").

        Pursuant to the Lockbox Assignment Agreement dated as of December 11,
1997 (as amended, supplemented or otherwise modified from time to time, the
"Lockbox Assignment Agreement"), among the Borrower, the Agent, for the
ratable benefit of the Lenders, the Trustee, for the benefit of the
Noteholders and Crestar Bank (the "Lockbox Bank", which term shall include any
successor bank acting in such capacity), the Lockbox Bank shall maintain, for
the benefit of the Agent, the Trustee and the Secured Parties, the Lockbox
Accounts (as defined below).  The Senior Obligations will be secured by a
first-priority security interest in the Lockbox Accounts and the Junior
Obligations will be secured by a second-priority security interest in the
Lockbox Accounts.  The term "Lockbox Accounts" means each lockbox and related
demand deposit account that is the subject of the Lockbox Assignment
Agreement.

        To induce each of the Lenders to extend credit to the Borrower
pursuant to the Credit Agreement and to permit the incurrence and existence of
the Junior Security Interest, each of the Grantors and the Trustee has agreed
to enter into this Agreement to provide for certain rights and obligations
with respect to the Collateral.  In extending credit to the Borrower and in
entering into this Agreement, each of the Senior Secured Parties is relying on
the undertakings of each of the Grantors and the Trustee as set forth herein.

        Capitalized terms used but not otherwise defined herein shall have the
meanings assigned to them in the Credit Agreement.

        Accordingly, in consideration of the mutual agreements herein
contained and other good and valuable consideration, the sufficiency and
receipt of which are hereby acknowledged, the Grantors, the Agent, on behalf
of itself and each Senior Secured Party, and the Trustee, on behalf of itself
and each Junior Secured Party, hereby agree as follows:

        SECTION 1.  Perfection and Priority of Security Interests.  (a)  Any
and all security interests, assignments, pledges and other liens, charges or
encumbrances now existing or hereafter created or arising in favor of the
Junior Secured Parties with respect to the Collateral are expressly junior in
priority, operation and effect to any and all security interests, assignments,
pledges and other secured claims, liens, charges or encumbrances now existing
or hereafter created or arising in favor of the Agent for the benefit of the
Senior Secured Parties with respect to the Collateral, notwithstanding
anything to the contrary contained in any agreement or filing to which the
Trustee or any Junior Secured Party may now or hereafter be a party, and
irrespective of the time, order or method of attachment or perfection of any
financing statements or other security interests, assignments, pledges and
other liens, charges or encumbrances or any defect or deficiency or alleged
defect or deficiency in any of the foregoing.

        (b)  The Trustee, on behalf of itself and the other Junior Secured
Parties, acknowledges that the Senior Obligations represent indebtedness that
is revolving in nature and that the amount thereof that may be outstanding at
any time or from time to time may be increased or
<PAGE>

                                                                               3

reduced and subsequently reborrowed, and that the terms of the Senior
Obligations may be modified, extended or amended from time to time, and the
aggregate amount of the Senior Obligations may be increased, replaced or
refinanced, all without notice to or consent by the Junior Secured Parties and
without affecting the provisions hereof; provided that no such modification,
extension or amendment shall be effective for purposes of this Agreement if it
increases the principal amount of the loans outstanding under the Credit
Agreement to an amount in excess of $25,000,000.  The lien priorities provided
in this Section 1 shall not be altered or otherwise affected by any amendment,
modification, supplement, extension, increase, replacement, renewal,
restatement or refinancing of either the Senior Obligations or the Junior
Obligations, or any portion thereof, nor by any action that the Senior Secured
Parties or the Junior Secured Parties may take or fail to take in respect of
the Collateral.

        (c)  For purposes of perfecting the Junior Security Interest in the
Collateral and the proceeds thereof, the Grantors and the Agent hereby
acknowledge that the Company will file such UCC-1 financing statements
covering the Collateral as it shall deem appropriate, naming the Borrower (or
one or more Subsidiary Guarantors) as debtor, and the Trustee, on behalf of
the Junior Secured Parties, as secured party.  Each of the Trustee and the
Junior Secured Parties agrees that all UCC-1 Financing Statements filed by or
on behalf of Junior Secured Parties shall be in form satisfactory to the Agent
and shall contain the following notation:  "The interest of the Secured Party
in the collateral described herein is junior and subordinate to the interests
of The Chase Manhattan Bank and its successors and assigns, as agent, and the
Lenders party to that certain Credit Agreement dated as of December 11, 1997,
as amended from time to time, with RBX Corporation, pursuant to that certain
Intercreditor Agreement dated as of December 11, 1997, as amended from time to
time."

        (d)  It is acknowledged that (i) pursuant to the Lockbox Assignment
Agreement, the Grantors have granted to the Trustee for the benefit of the
Junior Secured Parties a second-priority security interest in the Lockbox
Accounts and (ii) the Lockbox Accounts are maintained with the Lockbox Bank
under the dominion and control of the Agent in accordance with the terms of
the Lockbox Assignment Agreement, which grants to the Agent for the benefit of
the Senior Secured Parties a first-priority security interest in the Lockbox
Accounts.  Notwithstanding anything to the contrary in this Agreement, under
no circumstances shall the Agent have, or be deemed to have, any fiduciary
duty or any other duty (other than as expressly set forth in paragraph (e)
below) to any Junior Secured Party or the Trustee with respect to actions it
takes or is permitted to take hereunder with respect to the Lockbox Accounts.

        (e)  The Trustee, on behalf of itself and the other Junior Secured
Parties, acknowledges and agrees with the Senior Secured Parties that the
arrangements described in paragraphs (a), (b), (c) and (d) above are solely
for the purpose of providing the Junior Secured Parties with a perfected
Junior Security Interest in the Collateral and shall in no way be construed as
imposing any duties or other obligations on the Agent other than to transfer
to the Junior Secured Parties (or as a court of competent jurisdiction may
otherwise direct) the proceeds, if any, that remain following a sale, transfer
or other disposition of the Collateral by the Agent on behalf of the Senior
Secured Parties and the payment and satisfaction in full of any and all Senior
Obligations or, if the Agent shall still be in possession of all or any part
of the Collateral after such payment and satisfaction in full, the Collateral
or such part thereof remaining, without representation or warranty on the part
of the Agent or the Senior Secured Parties.  In furtherance of the foregoing,
the Junior Secured Parties acknowledge and agree with the Senior Secured
Parties that, at all times until the Senior Obligations shall have been paid
and satisfied in full, the Agent shall be entitled to (i) maintain control
over the Lockbox Accounts in accordance with the Lockbox Assignment Agreement
and (ii) sell, transfer or otherwise dispose of or deal with the Collateral as
provided in the Senior Security Documents without regard to the security
<PAGE>

                                                                               4

interest of the Junior Secured Parties therein, or any rights to which the
Junior Secured Parties would otherwise be entitled as a result of such
security interest, the only obligation of the Agent to the Junior Secured
Parties in respect thereof being to promptly deliver to the Trustee (unless
otherwise directed in writing by the Trustee or by a court of competent
jurisdiction) any proceeds remaining from such sale, transfer or other
disposition of the Collateral after such payment and satisfaction in full of
the Senior Obligations or, if the Agent shall still be in possession of all or
any part of the Collateral after such payment and satisfaction in full, the
Collateral or such part thereof remaining, without representation or warranty
on the part of the Agent or the Senior Secured Parties, provided that nothing
contained in this sentence shall be construed to give rise to, nor shall the
Trustee or the Junior Secured Parties have, any claims whatsoever against the
Agent or any Senior Secured Party on account of any act or omission to act in
connection with the exercise of any right or remedy of the Agent with respect
to the Collateral (other than with respect to any claims that may arise as a
result of the failure of the Agent, after the payment and satisfaction in full
of the Senior Obligations, to deliver any remaining Collateral or proceeds to
the Trustee or the Junior Secured Parties).  The Trustee, on behalf of itself
and the other Junior Secured Parties, agrees that, without the prior written
consent of the Agent, it shall not, and shall not attempt to, exercise any
rights of enforcement with respect to (A) the Senior Security Interest in the
Collateral or (B) the Junior Security Interest in the Collateral, whether
pursuant to the Junior Security Documents or otherwise, until the payment and
satisfaction in full of the Senior Obligations.

        (f)  The Trustee, on behalf of itself and the other Junior Secured
Parties, agrees that in the event a proceeding under Title 11 of the U.S. Code
(the "Bankruptcy Code") or any other Federal, state or foreign bankruptcy,
insolvency, receivership or similar law shall be commenced by or against any
Grantor, the Trustee or such Junior Secured Party shall not file any objection
or in any other manner attempt to prevent the court conducting such proceeding
from (and shall be deemed to have consented to such proceeding) (i)
authorizing such Grantor under Section 364 of the Bankruptcy Code (or the
analogous provision of such other law) to incur secured indebtedness and
priority administration indebtedness to the Senior Secured Parties (or any of
them) for loans that may be made by the Senior Secured Parties (or any of
them) to such Grantor after the commencement and/or termination of such
proceedings (but only to the extent that the principal amount of the
commitments to provide such post-petition secured indebtedness, when
aggregated with the principal amount of the Loans outstanding under the Credit
Agreement immediately prior to the commencement of such proceeding, would not
exceed $25,000,000), and to secure the repayment and performance of such
post-petition indebtedness (and interest thereon and costs and expenses in
connection therewith) by granting to the Senior Secured Parties (or any of
them) a first-priority security interest in and a first-priority lien on all
the Collateral, (ii) pursuant to Section 363 of the Bankruptcy Code (or the
analogous provision of such other law), granting the Senior Secured Parties
(or any of them) a first-priority security interest in and a first-priority
lien on all the Collateral, (iii) authorizing such Grantor under the
Bankruptcy Code to grant the Senior Secured Parties (or any of them) adequate
protection, whether in the form of cash payments, accrued interest or
otherwise, in connection with the use of cash collateral or in connection with
the transactions contemplated by clauses (i) and (ii) hereof.

        SECTION 2.  No Interference; Right to Instruct Agent; Payment Over;
Reinstatement; Permitted Actions.  (a)  The Trustee, on behalf of itself and
the other Junior Secured Parties, agrees that (i) it will not take or cause to
be taken any action, the purpose or effect of which is, or could be, to make
any Junior Security Interest pari passu with, or to give any Junior Secured
Party any preference or priority relative to, the Senior Security Interest or
the Senior Secured Parties with respect to the Collateral or any part thereof,
(ii) it will not interfere, hinder or delay, in any manner, whether by
judicial proceedings or otherwise, any sale, transfer or other disposition of
the
<PAGE>

                                                                               5

Collateral by the Agent or any other Senior Secured Party, (iii) it has no
right to (A) direct the Agent or any other Senior Secured Party to exercise
any right, remedy or power with respect to the Collateral or (B) consent to
the exercise by the Agent or any other Senior Secured Party of any right,
remedy or power with respect to the Collateral, (iv) it will not institute any
suit or assert in any suit, bankruptcy, insolvency or other proceeding any
claim against the Agent or any other Senior Secured Party seeking damages from
or other relief by way of specific performance, instructions or otherwise,
with respect to, and neither the Agent nor any other Senior Secured Party
shall be liable for, any action taken or omitted to be taken by the Agent or
the Senior Secured Parties with respect to the Collateral, (v) it will not
commence judicial or nonjudicial foreclosure proceedings with respect to, seek
to have a trustee, receiver, liquidator or similar official appointed for or
over, attempt any action to take possession of, exercise any right, remedy or
power with respect to, or otherwise take any action to enforce its interest in
or realize upon, the Collateral (other than filing a proof of claim) until all
the Senior Obligations shall have been paid and satisfied in full, (vi) it
will not seek, and hereby waives any right, to have the Collateral or any part
thereof marshaled upon any foreclosure or other disposition of the Collateral,
(vii) until the date which occurs three months following the peaceful
possession by the Trustee or its agent(s), as owners or pursuant to any
judicial or nonjudicial proceeding, of any of the Mortgaged Properties (any
such property, a "Trustee Owned Property", and all such properties, the
"Trustee Owned Properties"), the Trustee will grant to the Agent, the Senior
Secured Parties and their designees access to the Trustee Owned Properties for
the purposes of inspecting or auditing the Collateral located therein or
otherwise dealing with such Collateral in any manner permitted by the Senior
Security Documents or applicable law; and (viii) it will not attempt, directly
or indirectly, whether by judicial proceedings or otherwise, to challenge the
enforceability of any provision of this Agreement.

        (b)  The Trustee, on behalf of itself and the other Junior Secured
Parties, hereby agrees that, in the event of a sale, transfer or other
disposition of Collateral in accordance with the provisions of the Indenture
or pursuant to any judicial or nonjudicial foreclosure proceedings with
respect to the Collateral undertaken by or on behalf of the Agent or any other
Senior Secured Party, any security interest or lien of the Junior Secured
Parties in such Collateral shall terminate and be released in accordance with
the provisions of Section 314(d) of the Trust Indenture Act of 1939, as
amended, if applicable, or, if not applicable, automatically and without
further action to the extent that the lien of the Agent for the benefit of the
Senior Secured Parties in such Collateral is released.  The Trustee further
agrees to execute all necessary instruments and documents to evidence such
termination and release.  Notwithstanding the foregoing, the Trustee shall not
be deemed to release or terminate the security interest or lien of the Junior
Secured Parties to the extent that any proceeds of the sale, transfer or
disposition of Collateral are in excess of the amount necessary to pay in full
all the Senior Obligations.

        (c)  The Trustee, on behalf of itself and the other Junior Secured
Parties, hereby agrees that if it shall obtain possession of any of the
Collateral (other than cash proceeds deposited with the Trustee to pay
principal or interest in respect of the Notes at a time when the Agent has not
informed the Trustee of the occurrence of a Default or an Event of Default
under the Credit Agreement), or shall realize any payment in respect of the
Collateral, in either case prior to the time when the Senior Obligations have
been paid in full, then it shall hold such Collateral or payment in trust for
the Senior Secured Parties and transfer such Collateral or payment, as the
case may be, to the Agent.  If, at any time, all or part of any payment with
respect to the Senior Obligations previously made is rescinded for any reason
whatsoever, the Trustee and the Junior Secured Parties shall promptly turn any
Collateral then held by any of them over to the Agent, and the provisions set
forth in this Agreement shall be reinstated as if such payment had not been
made, until the payment and satisfaction in full of the Senior Obligations.
<PAGE>

                                                                               6

        SECTION 3.  Status of Agreement.  This Agreement shall constitute a
Loan Document for all purposes under the Credit Agreement.

        SECTION 4.  GOVERNING LAW.  THIS AGREEMENT SHALL BE GOVERNED BY, AND
CONSTRUED IN ACCORDANCE WITH, THE LAWS OF THE STATE OF NEW YORK.

        SECTION 5.  Further Assurances.  Each of the parties hereto agrees to
execute such further documentation and take such further action as shall be
reasonably necessary to fully effectuate the provisions of this Agreement.

        SECTION 6.  Changes, Waiver, etc.  Neither this Agreement nor any
provision hereof may be waived, discharged or terminated orally, but only by a
statement or instrument in writing signed by the party or parties against
which enforcement of the waiver, discharge or termination shall be sought.

        SECTION 7.  Successors and Assigns.  This Agreement shall be binding
upon and inure to the benefit of the parties hereto, each Noteholder, Junior
Secured Party, Lender, Senior Secured Party and their respective successors
and assigns.  Each Noteholder, by accepting a Note and the benefits of the
Senior Secured Note Indenture, shall be deemed to have agreed to be bound by
the provisions of this Agreement.

        SECTION 8.  Severability.  In case any one or more of the provisions
contained in this Agreement should be invalid, illegal or unenforceable in any
respect, the validity, legality and enforceability of the remaining provisions
contained herein shall not in any way be affected or impaired thereby.

        SECTION 9.  Counterparts.  This Agreement may be executed in more than
one counterpart, each of which shall constitute an original but all of which,
when taken together, shall constitute but one instrument.  Delivery of an
executed signature page to this Agreement by facsimile transmission shall be
as effective as delivery of a manually executed counterpart hereof.

        SECTION 10.  Headings.  The sections headings of this Agreement are
for purposes of reference only and shall not limit or otherwise affect the
meaning or construction of any provisions hereof.
<PAGE>

                                                                               7

        IN WITNESS WHEREOF, the Agent, the Trustee for the benefit of the
Noteholders, the Borrower and the Subsidiary Guarantors have caused this
Agreement to be duly executed and delivered by their respective duly
authorized officers, all as of the day and year first above written.


                                      RBX CORPORATION,

                                      by
                                          /s/ Frank Roland
                                          ------------------------------------
                                          Name: Frank Roland
                                          Title: President and Chief Executive
                                                 Officer

                                      THE SUBSIDIARY GUARANTORS LISTED ON
                                      SCHEDULE I HERETO,

                                      by
                                          /s/ Frank Roland
                                          ------------------------------------
                                          Name: Frank Roland
                                          Title: President and Chief Executive
                                                 Officer


                                      THE CHASE MANHATTAN BANK, as Agent,

                                      by
                                          /s/ William Caggiano
                                          ------------------------------------
                                          Name: William Caggiano
                                          Title: Managing Director


                                      STATE STREET BANK AND TRUST COMPANY, as
                                      Trustee,

                                      by
                                          /s/ Michael M. Hopkins
                                          ------------------------------------
                                          Name: Michael A. Hopkins
                                          Title: Vice President
<PAGE>

                                                                    Schedule I
                                                                    ----------



                                  GUARANTORS



Rubatex Corporation
Groendyk Manufacturing Company, Inc.
Hoover-Hanes Rubber Custom Mixing, Corp.
Midwest Rubber Custom Mixing Corp.
Oletex Inc.
Universal Polymer & Rubber Inc.
Universal Rubber Corporation
Waltex Corporation