<PAGE>

                                                                   EXHIBIT 10.56

                                                            March 31, 2005

THE CIT GROUP/COMMERCIAL SERVICES, INC.
1211 Avenue of the Americas
New York, New York 10036

                    Re: AMENDED AND RESTATED PLEDGE AGREEMENT

Gentlemen:

Reference is hereby made to that certain Pledge Agreement dated as of March 29,
2004 (the "Original Pledge Agreement") made by the undersigned, R.G. BARRY
CORPORATION (the "Client") in favor of you. Reference is hereby also made to
that certain Factoring Agreement (as amended, together with all related
agreements, documents and instruments, the "Factoring Agreement") between you
and the Client dated as of March 29, 2004, and that certain Financing Agreement
dated as of the date hereof (as amended, supplemented, restated or otherwise
from time to time modified, the "Financing Agreement"). This Pledge Agreement
amends and restates, in its entirety, the Original Pledge Agreement. As security
for the full and indefeasible payment and performance when due of all now
existing and future Obligations, as defined in each of the Factoring Agreement
and the Financing Agreement, the undersigned hereby reaffirms the assignment and
transfer made under the Original Pledge Agreement and hereby further pledges and
collaterally assigns, transfers, delivers and sets over to you all of its right,
title and interest in and to the securities listed on the attached schedule,
issued as indicated on said schedule.

This pledge includes all right, title and interest in and to, and a continuing
lien upon and security interest in, all of said securities together with any and
all rights, coupons, warrants or rights to subscribe, options, dividends,
liquidating dividends, splits, dividends paid in stock, dividends paid in
securities, new or reclassified securities, or any other property which the
undersigned is or may hereafter become entitled to receive on account of such
securities, any and all increments, substitutions, additions or replacements
thereof, and any and all proceeds thereof (all collectively hereinafter referred
to as "Collateral").

This pledge is executed as an inducement to you to make loans or advances to the
Client or issue Guaranties to suppliers of the Client on behalf of the Client,
or otherwise to extend credit or financial accommodations to the Client or to
enter into or continue a financing arrangement with the Client, and is executed
in consideration of your doing or having done any of the foregoing. The
undersigned agrees that any of the foregoing shall be done or extended by you in
your sole discretion and shall be deemed to have been done or extended by you in
consideration of and in reliance upon the execution of this pledge, but that
nothing herein shall obligate you to do any of the foregoing.

                                       1
<PAGE>

Any of the following shall constitute an Event of Default under this Pledge
Agreement:

      l.    The occurrence and continuance of an Event of Default under either
            the Financing Agreement or the Factoring Agreement or

      2.    If any warranty, representation or statement contained in this
            Pledge Agreement is materially incorrect when made.

In the event of the happening of any Event of Default as defined herein, then on
ten (10) days prior notice to the undersigned; without the curing of such
default within such time, you may, without demand of performance, advertisement
or notice of intention to sell, or of the time or place of sale, and without
notice to redeem, or other notice or demand whatsoever to or upon the
undersigned (all and each of which demands, advertisements and/or notices are
hereby expressly waived), forthwith or at any time or times thereafter, transfer
to and/or register in your name, or the name of your nominee, any or all of the
Collateral and/or collect, receive, appropriate and realize upon said
Collateral. In addition, and also without any of the aforesaid demands,
advertisements and/or notices, in the event of the happening of any Event of
Default as defined herein, you may sell, assign, transfer and deliver the whole
or any part of the Collateral then held by you under this agreement or subject
to this agreement in one or more parcels, at public or private sale or sales, at
any Exchange Broker's Board, at your office or elsewhere, on such terms and
conditions, and at such prices as you may deem advisable, for cash, upon credit,
or for future delivery, with the right on your part to become the purchaser
thereof at any such sale or sales, free and clear of any right or equity of
redemption (which right or equity is hereby expressly waived and released). Any
notice of sale, disposition, or other intended action by you required by
applicable law and sent to the undersigned at least five (5) days prior to such
action shall constitute reasonable notice to the undersigned. Prior to
exercising your rights contained herein you may in your discretion forward the
various coupons coming due on any bonds covered hereby directly to the
undersigned for collection.

Net proceeds of any such disposition as aforesaid, after deducting all costs,
including reasonable attorneys' fees and expenses of every kind incurred
therein, shall be applied to the payment in whole or in part, in such order as
you may elect, of any of the Obligations, whether then due or not due. You agree
to pay over and return any remaining balance to the undersigned or to any
person, entitled thereto, upon proper demand being made therefor, and if there
be any deficiency, the Client shall continue to be fully liable for same.

Further, you are hereby expressly granted the right and irrevocable proxy, in
the event of the happening of any Event of Default as defined herein, and on ten
(10) days prior notice to the undersigned, without the curing of such Event of
Default within such time, to transfer to yourself or to your nominee any or all
of the Collateral or to register same in your name on the books of the company
or entity issuing same; to receive cash dividends, coupons and income thereon
and to hold the same as additional collateral security hereunder, or to apply it
against the Obligations secured hereby and to exercise any voting rights with
respect to said Collateral for any purposes as you in your discretion deem
advisable, and to otherwise exercise as to such Collateral, all

                                       2
<PAGE>

rights, powers and remedies as the owner thereof.

The undersigned hereby warrants and represents that the Collateral is owned by
the undersigned absolutely, and is free and clear of all liens and encumbrances
other than liens in favor of you and Permitted Encumbrances (as such term is
defined in the Financing Agreement); that there are no restrictions upon the
pledge or transfer of any of the Collateral; that the undersigned has the full
right to pledge and transfer the same in accordance with the terms and
conditions of the Pledge Agreement, free of all encumbrances and without the
consent of any other person, firm, entity or corporation and without the need to
notify the issuing company and/or obtain their consent to the pledge; and that
said Collateral is not subject to any assessment. The undersigned agrees to
defend its title to the Collateral at its own cost and expense, and to pay,
satisfy and discharge any and all assessment, liens or charges now or hereafter
placed upon the Collateral, other than Permitted Encumbrances.

In the event that it becomes necessary to comply with any Federal or State law
or regulation or to make or file any registration thereunder in order for you to
exercise any of your rights hereunder, the undersigned expressly agrees to do or
will cause to be done all acts and prepare and execute all documents necessary
to affect such compliance or registration, and to bear all reasonable costs in
connection therewith. The undersigned agrees to indemnify and to hold you
harmless from and against any claim or liability caused by any untrue statement
of material fact, or omission to state a material fact (as required in any
registration or prospectus); or caused by a failure to register or comply with
any such law or regulation.

The undersigned affirms and certifies that any Obligations of the Client secured
by this pledge were not, and will not be, incurred for the purpose of providing
or obtaining any credit for purchasing or trading in margin securities or other
marketable securities.

The undersigned hereby agrees at your request to execute all necessary stock
powers in blank, to have the signatures on said powers guaranteed, to execute a
letter or other form confirming that the Collateral is not being pledged to you
for the purpose of providing or obtaining any credit for purchasing or trading
in registered equity securities or other marketable securities, and to execute
any further documents or papers whatsoever in order to carry out the intend and
purpose of this Pledge Agreement.

The pledge provided for herein shall be in addition to, and shall not be deemed
to effect, modify or limit any other rights, collateral, agreements or security
which you may now or hereafter hold whether granted or given to you by the
undersigned or by any other person, firm or corporation.

It is understood and agreed that the rights and remedies herein enumerated are
not intended to be exhaustive but are in addition to any other rights or
remedies at law or in equity. You shall have the absolute right in your sole
discretion to determine the order in which your rights and remedies are to be
exercised, and your exercise of any right or remedy shall not preclude the
exercise of any other rights or remedies or be deemed to be a waiver thereof. No
act of forbearance, or agreement to forebear the enforcement of, or extension of
the date of a maturity

                                       3
<PAGE>

of any Obligation, shall in any way constitute a release of, or a waiver or
relinquishment of any of your rights or remedies.

The undersigned expressly agrees that it shall not exercise any right of
subrogation, reimbursement, indemnity or recourse to or with respect to any of
the assets or properties of the Client until such time as all of the Obligations
to you shall have been paid and satisfied in full.

This Pledge Agreement is to be governed by the laws of the State of New York,
and shall be binding on the heirs, administrators, executors, successors and
assigns of the undersigned, and shall inure to the benefit of your successors
and assigns.

                  [REMAINDER OF PAGE INTENTIONALLY LEFT BLANK]

                            [SIGNATURE PAGE FOLLOWS]

                                       4
<PAGE>

      IN WITNESS WHEREOF, the undersigned have executed and delivered this
Amended and Restated Pledge Agreement effective as of the date above set forth.

                                    Very truly yours,

                                    R.G. BARRY CORPORATION

                                    By: /s/ Thomas Von Lehman
                                        ----------------------------------------
                                    Name: Thomas Von Lehman
                                    Title: President and Chief Executive Officer

                                       5
<PAGE>

                          SCHEDULE TO PLEDGE AGREEMENT

<TABLE>
<CAPTION>

        NAME OF                     # OF               # OF                     CLASS
SUBSIDIARY CORPORATION           SHARES HELD        SHARES OWNED               OF SHARES            CERTIFICATE #
-------------------------        -----------        ------------        --------------------        -------------
<S>                              <C>                <C>                 <C>                         <C>
The Dearfoams Company                100                100%            Common, No Par Value              1

R.G. Barry International, Inc.        500                100%            Common, Par Value               No Cert.
                                                                               $1.00

RGB Technology, Inc.               1,000                100%            Common, Par Value                  9
                                                                               $1.00
</TABLE>

                                       6