<PAGE>   1
                                                                       Exhibit 4

                  THIRD AMENDMENT TO REVOLVING CREDIT AGREEMENT

         This Third Amendment to Revolving Credit Agreement (this "Amendment")
is entered into at Columbus, Ohio, by and among The Huntington National Bank,
The Bank of New York and Bank One, N.A., as lenders (the "Banks"); The
Huntington National Bank as agent (the "Agent"); and R.G. Barry Corporation, as
borrower (the "Borrower"), as of the 27th day of October, 2000, in order to
amend the Revolving Credit Agreement entered into by and among the Banks and the
Borrower as of the 28th day of February, 1996, which has been amended by an
Amendment to Revolving Credit Agreement dated as of March 17, 2000, and by a
Second Amendment to Revolving Credit Agreement dated as of June 30, 2000 (as so
amended and modified, the "Credit Agreement").

                                    RECITALS:

         A. The Borrower has advised the Agent and the Banks that the delivery
of the financial statements of the Borrower for the fiscal period ending as of
September 30, 2000, will evidence that the Borrower is in default of Section
9.17, "EBITDA," of the Credit Agreement (the "Identified Default"); and

         B. The Borrower has requested that the Agent and the Banks waive the
Identified Default and modify certain other provisions of the Credit Agreement,
and the Agent and the Banks are willing to do so, subject to the terms and
execution of this Amendment.

         NOW, THEREFORE, in consideration of the mutual covenants, agreements
and promises contained herein, the receipt and sufficiency of which are hereby
acknowledged, and intending to be legally bound, the parties hereto for
themselves and their successors and assigns do hereby agree, represent and
warrant as follows:

         1. Section 1.1, "Basic Commitment Terms," of the Credit Agreement is
hereby amended to recite in its entirety as follows:

                  1.1 Basic Commitment Terms. The Borrower has applied to the
                  Banks for revolving credit loans up to an aggregate principal
                  amount of $30,000,000, the proceeds of which are to be used by
                  the Borrower for general corporate purposes, including,
                  without limitation, seasonal financing of inventory and
                  accounts receivable. Each of the Banks is willing to make such
                  loans to the Borrower upon the terms and subject to the
                  conditions hereinafter set forth up to a maximum aggregate
                  principal amount not in excess of the amount set forth
                  opposite its name below and otherwise in accordance with the
                  pro rata requirements of Section 4 hereof (said amount being
                  hereinafter called the "Commitment" of such Bank and
                  collectively called the "Commitments"):

                  BANK                      ADDRESS                  COMMITMENT
                  ----                      -------                  ----------

                  The Bank of New York      One Wall Street
                                            New York,
                                            New York, 10286          $10,000,000

                  The Huntington            41 South High Street
                  National Bank             Columbus, OH 43287       $10,000,000

                  Bank One, N.A.            100 East Broad Street
                                            Columbus, Ohio 43215     $10,000,000

         2. Notwithstanding anything to the contrary contained in any Note
evidencing the Loan, the principal amount advanced by each Bank pursuant to the
Note held by such Bank shall not exceed the amount of such Bank's Commitment.

                                       14
<PAGE>   2
         3. Section 1.2, "Commitment Limitations," of the Credit Agreement is
hereby amended to recite in its entirety as follows:

                  1.2 Commitment Limitations. Notwithstanding the foregoing,
                  during the following periods in each year occurring during the
                  term of this Agreement, the aggregate Commitments of the Banks
                  (each Bank's individual Commitment being one third (1/3) of
                  the aggregate) shall be in an amount equal to the lesser of
                  the following amounts or the amount to which the Commitments
                  have been reduced pursuant to Section 4.6 hereof.

                  Period                                      Commitment
                  ------                                      ----------

                  From 01/01 through 01/31                    $ 3,000,000
                  From 02/01 through 03/31                    $15,000,000
                  From 04/01 through 11/29                    $30,000,000
                  From 11/30 through 12/31                    $27,000,000

         4. A new Section 2.15, "Locations of Business," is hereby added to the
Credit Agreement, to read in its entirety as follows:

                  2.15 Locations of Business. Schedule 8.13 attached to the
                  Third Amendment sets forth each place of business in the
                  United States (including without limitation any location at
                  which personal property of such entity is stored or located)
                  for the Borrower.

         5. Section 3.36 of the Credit Agreement is hereby amended to recite in
its entirety as follows:

                  3.36 "Note" or "Notes" means the Revolving Credit Notes as
                  defined in Section 4.2.

         6. A new Section 3.48 is hereby added to the Credit Agreement, to read
in its entirety as follows:

                  3.48. "Loan Document" means each document, instrument and
                  agreement executed in connection with the Credit Agreement.

         7. Section 4.2, "Evidence of Loans Made Under Revolving Credit," of the
Credit Agreement is hereby amended to recite in its entirety as follows:

                  4.2 Evidence of Loans Made Under Revolving Credit. All Loans
                  made by each Bank pursuant to such Bank's Commitment shall be
                  evidenced by a promissory note, substantially in the form
                  attached as Exhibit "A" to that certain Amendment to Revolving
                  Credit Agreement between the Borrower, the Agent and the Banks
                  dated as of March 17, 2000 (hereinafter collectively called
                  the "Revolving Credit Notes"), payable to the order of such
                  Bank, duly executed on behalf of the Borrower, dated the date
                  of this Agreement. Each Bank is hereby authorized by the
                  Borrower to note on the schedule attached to any Revolving
                  Credit Note held by such Bank the date, amount and type of
                  each Loan made to the Borrower, the duration of the related
                  Interest Period if applicable, the amount of each payment or
                  prepayment of principal thereon, and the other information
                  provided for on such schedule, which schedule shall constitute
                  prima facie evidence of the information so noted; provided,
                  that failure of such Bank to make any such notation shall not
                  relieve the Borrower of its obligation to repay the
                  outstanding principal amount of any Loan or Loans made to it,
                  all accrued interest thereon and all other

                                       15
<PAGE>   3
                  amounts payable in accordance with the terms of any Revolving
                  Credit Note or this Agreement. Interest on the Loans evidenced
                  by such Revolving Credit Notes shall be payable at the rates
                  specified in Sections 3.2 and 3.21 hereof and on each Interest
                  Payment Date, as hereinafter defined. The terms and conditions
                  of this Agreement are incorporated in the Revolving Credit
                  Notes by reference as though the same were written therein.

         8. Section 8.12, "Restriction on Consolidated Asset," of the Credit
Agreement is hereby amended to recite in its entirety as follows:

                  8.12 Restriction on Consolidated Asset. Maintain seventy-five
                  percent (75%) of consolidated tangible assets, excluding
                  intercompany assets which are eliminated when consolidated in
                  accordance with GAAP, under the ownership of and in the name
                  of the Borrower. In the event the Borrower fails to maintain
                  seventy-five percent (75%) of its consolidated tangible assets
                  under its ownership or in its own name, then the Borrower,
                  within a reasonable time thereafter, shall make all
                  significant Subsidiaries, as determined in a commercially
                  reasonable manner by the Banks, obligors on all indebtedness
                  owing to the Banks under this Agreement, either by assumption
                  of such indebtedness as a co-maker with Borrower or the
                  guaranty of such indebtedness, as the Borrower may elect.

         9. The prefatory paragraph of SECTION 8, "AFFIRMATIVE COVENANTS," of
the Credit Agreement is hereby amended to recite in its entirety as follows:

                  For as long as the Banks are obligated to lend hereunder and
                  until payment in full of the Notes and interest thereon, the
                  Borrower covenants that it will and will cause each
                  Subsidiary, except in the case of Sections 8.1, 8.2, 8.9,
                  8.10, 8.13 and 8.14 hereof and unless the Banks shall
                  otherwise consent in writing, to:

         10. A new section 8.13, "Collateral," is hereby added to the Credit
Agreement, to read in its entirety as follows:

                  8.13 Collateral. Grant to the Agent, for itself and the
                  ratable benefit of the Banks, a security interest in all of
                  the personal property of the Borrower pursuant to a security
                  agreement substantially in the form of Exhibit 8.13 attached
                  to that certain Third Amendment to Revolving Credit Agreement
                  dated as of October 27, 2000, between and among the Borrower,
                  the Agent and the Banks (the "Third Amendment"); provided,
                  however that such security interests shall not attach and
                  shall not become enforceable until the occurrence of an Event
                  of Default. The principal amount of Debt (as that term is
                  defined in the Metropolitan Agreement in effect as of the date
                  of the Third Amendment) secured by such security interests
                  shall not exceed 15% of Consolidated Net Tangible Assets (as
                  that term is defined in the Metropolitan Agreement in effect
                  as of the date of the Third Amendment). The Borrower shall
                  execute and deliver to the Agent such financing statements as
                  the Agent may deem necessary to perfect the security interests
                  in the aforementioned personal property, which financing
                  statements the Agent and the Banks agree not to file until the
                  occurrence of an Event of Default.

         11. A new section 8.14, "Resting of Loan," is hereby added to the
Credit Agreement, to read in its entirety as follows:

                                       16
<PAGE>   4
                  8.14 Resting of Loan. So long as the Commitments are
                  outstanding, the Borrower shall reduce the principal balance
                  of the Loan to zero and maintain the same at zero for a period
                  of not less than 60 consecutive days during each Resting
                  Period. "Resting Period" means a period beginning on December
                  1 and ending on the immediately following March 30.

         12. Section 9.4, "Maintenance of Consolidated Tangible Net Worth," of
the Credit Agreement is hereby amended to recite in its entirety as follows:

                  9.4 Maintenance of Consolidated Tangible Net Worth. Permit
                  Consolidated Tangible Net Worth to be less than (a)
                  $52,000,000 as of September 30, 2000, and (b) $54,750,000 as
                  of the last day of fiscal year 2000, and as of the end of each
                  fiscal quarter thereafter.

         13. Section 9.8.6 of the Credit Agreement is hereby amended to recite
in its entirety as follows:

                  9.8.6 other Liens on property of the Borrower securing Current
                  Debt or Funded Debt of the Borrower to the Banks, and each of
                  them, under this Agreement, provided that at the time of
                  incurrence of any such Lien and after giving effect to the
                  Debt secured thereby, (A) the aggregate principal amount of
                  all such Debt secured by Liens permitted by this Section 9.8.6
                  shall not exceed 15% of Consolidated Tangible Net Worth, and
                  (B) the Borrower shall be in compliance with Section 9.2
                  hereof.

         14. A new Subsection 9.8.7 is hereby added to the Credit Agreement, to
read in its entirety as follows:

                  9.8.7 Liens in favor of the Agent, for the ratable benefit of
                  the Banks, to secure the Loans and any other amounts due to
                  the Agent and the Banks, or any of them, under this Agreement
                  or under the Loan Documents.

         15. Section 12.9, "Amendments and Modifications," of the Credit
Agreement is hereby deleted in its entirety.

         16. The Agent and the Banks hereby waive the Identified Default through
and including September 30, 2000.

         17. The Borrower represents and warrants that, except for the
Identified Default, no Event of Default has occurred and is continuing, nor will
any occur immediately after the execution and delivery of this Amendment by the
performance or observance of any provision hereof.

         18. Each reference to the Credit Agreement, whether by use of the
phrase "Credit Agreement," "Agreement," the prefix "herein" or any other term,
and whether contained in the Credit Agreement itself, in this Amendment, in any
document executed concurrently herewith or in any loan documents executed
hereafter, shall be construed as a reference to the Credit Agreement as
previously amended and as amended by this Amendment. This Amendment shall be
limited precisely as provided for herein, and shall not be deemed to be a waiver
of, amendment of, consent to or modification of any of the rights of the Agent,
the Banks, or any of them, under, any other term or provisions of the Credit
Agreement, any Loan Document, or other agreement executed in connection
therewith, or of any term or provision of any other instrument referred to
therein or herein or of any transaction or future action on the part of the
Borrower which would require the consent of the Agent or the Banks, including,
without limitation, waivers of Events of Default which may exist after giving
effect hereto.

                                       17
<PAGE>   5
         19. Except as previously amended and as modified herein, the Credit
Agreement and the Loan Documents shall remain as written originally and in full
force and effect in all respects, and nothing herein shall affect, modify, limit
or impair any of the rights and powers which the Banks may have thereunder.

         20. The Borrower agrees to perform and observe all the covenants,
agreements, stipulations and conditions to be performed on its part under the
Credit Agreement, the Loan Documents, and all other related agreements, as
amended by this Amendment.

         21. The Borrower hereby represents and warrants to the Agent and the
Banks that (a) the Borrower has legal power and authority to execute and deliver
the within Amendment; (b) the respective officer executing the within Amendment
on behalf of the Borrower has been duly authorized to execute and deliver the
same and bind the Borrower with respect to the provisions provided for herein;
(c) the execution by the Borrower and the performance and observance by the
Borrower of the provisions hereof do not violate or conflict with the articles
of incorporation, regulations or by-laws of the Borrower or any law applicable
to the Borrower or result in the breach of any provision of or constitute a
default under any agreement, instrument or document binding upon or enforceable
against the Borrower; and (d) this Amendment and the other Loan Documents
executed and delivered in connection herewith, constitute valid and legally
binding obligations upon the Borrower, subject to applicable bankruptcy,
insolvency, reorganization or other similar laws affecting creditors' rights
generally, to general equitable principles and to applicable doctrines of
commercial reasonableness.

         22. This Amendment shall become effective only upon its execution by
the Borrower, the Banks and the Agent; the payment by the Borrower to the Agent,
for the ratable benefit of the Banks, of an amendment and waiver fee of
$52,500.00; the execution and delivery to the Agent by the Borrower of a
Security Agreement substantially in the form attached hereto as Exhibit 8.13;
the execution and delivery to the Agent by the Borrower of such UCC financing
statements as deemed necessary by the Agent; the delivery to the Agent of such
certificates and other evidence as the Agent may require with respect to (i) the
authority of the Borrower to grant a security interest in its personal property
to the Agent as contemplated herein, and (ii) the incumbency of the officers
executing documents in favor of the Agent and the Banks. Execution of this
Amendment by the parties hereto may be in any number of counterparts, but all of
such counterparts when taken together shall constitute one and the same
document. The facsimile or other electronically transmitted copy of this
Amendment shall be treated the same as an originally executed copy hereof.

         23. The capitalized terms used herein shall have the same meanings as
the capitalized terms used in the Credit Agreement, as amended hereby.

             THE REMAINDER OF THIS PAGE IS INTENTIONALLY LEFT BLANK.

                                       18
<PAGE>   6
         IN WITNESS WHEREOF, the Borrower, the Banks and the Agent have hereunto
set their hands as of the 27th day of October, 2000.


ATTEST:                               R. G. BARRY CORPORATION

/s/ Travis Wahl                       By: /s/ Michael Krasnoff
---------------------------               --------------------------------------
                                          Michael S. Krasnoff, Vice President


                                      THE HUNTINGTON NATIONAL BANK,
                                      Individually and as Agent

                                      By: /s/ John M. Luehmann
                                          --------------------------------------
                                          John M. Luehmann, Assistant Vice
                                          President


                                      THE BANK OF NEW YORK

                                      By: /s/ William M. Barnum, Jr.
                                          --------------------------------------
                                          William M. Barnum, Jr., Vice President


                                      BANK ONE, N.A.

                                      By: /s/ Jeffrey C. Nicholson
                                          --------------------------------------
                                          Jeffrey C. Nicholson, Vice President

                                       19