<PAGE>

                                                                    EXHIBIT 10.1

--------------------------------------------------------------------------------

                               FIRST AMENDMENT TO

                       MASTER RECAPITALIZATION AGREEMENT

                                  by and among

                        IMPERIAL CREDIT INDUSTRIES, INC.

                                      and

                    EACH OF THE INVESTORS REFERRED TO HEREIN

                           Dated as of June 27, 2001


--------------------------------------------------------------------------------
<PAGE>

              FIRST AMENDMENT TO MASTER RECAPITALIZATION AGREEMENT

     This First Amendment to Master Recapitalization Agreement, dated as of June
27, 2001 (this "Amendment"), is entered into by and among Imperial Credit
Industries, Inc., a California corporation (the "Company"), and the respective
Senior Secured Debt Purchasers and Signatory Debtholders named on the signature
pages hereof on the basis of the following facts:

      A.  The Company, the Senior Secured Debt Purchasers and the Signatory
Debtholders (collectively, the "Investors") are parties to that certain Master
Recapitalization Agreement, dated as of March 29, 2001 (the "Recapitalization
Agreement").  All initially-capitalized terms not otherwise defined herein shall
have the meanings set forth in the Recapitalization Agreement unless the context
clearly indicates otherwise.

      B.  The Company and the Investors desire to amend certain terms and
conditions of the Recapitalization Agreement and related exhibits thereto as
provided in this Amendment.

      C.  In accordance with Section 6.3 of the Recapitalization Agreement, with
respect to the Senior Secured Debt Purchasers, this Amendment shall be
enforceable against all Senior Secured Debt Purchasers upon the approval of
holders of a majority in aggregate principal amount of the Senior Secured Debt.

      NOW, THEREFORE, in consideration of the mutual covenants and agreements
set forth herein and for other good and valuable consideration, the receipt and
adequacy of which is hereby acknowledged, the parties hereto agree to amend the
Recapitalization Agreement as follows:


                                   AGREEMENT
                                   ---------

      1.  The following definitions are hereby added to Section 1.1 of the
                                                        -----------
Recapitalization Agreement:

      "Convertible Subordinated Indenture" means the indenture relating to the
Secured Convertible Subordinated Debt, which shall be substantially in the form
attached as Exhibit D hereto, as amended, supplemented or otherwise modified
from time to time with the consent of the Senior Secured Debt Purchasers.

      "Reverse Stock Split" means the proposed reverse stock split of the
Company's Common Stock that is to be presented for approval by the Company's
shareholders at the Company's 2001 Annual Meeting of Shareholders.

      "Reverse Stock Split Ratio" means a factor of four, three or two, relating
to the number of shares of outstanding Common Stock, as determined by the Board
of Directors, that will be combined into one share of the Company's Common Stock
in the Reverse Stock Split.

      "Secured Convertible Subordinated Debt" means the Secured Convertible
Subordinated Notes that are to be purchased by the Secured Convertible
Subordinated Debt Purchasers pursuant to Section 2.3, which shall be in the form
specified in and issued pursuant to the Convertible Subordinated Indenture.

                                       2
<PAGE>

      "Secured Convertible Subordinated Debt Placement Closing" has the meaning
set forth in Section 2.3(c).

      "Secured Convertible Subordinated Debt Placement Closing Date" has the
meaning set forth in Section 2.3(c).

      "Secured Convertible Subordinated Debt Placement" means the private
placement of Secured Convertible Subordinated Debt provided for in Section 2.3.

      "Secured Convertible Subordinated Debt Purchasers" means the Investors who
will become parties to this Agreement after the initial effective date hereof,
will be identified as such in Exhibit A hereto and will agree to purchase
Secured Convertible Subordinated Debt in the Secured Convertible Subordinated
Debt Placement, all as provided in Section 2.3.

      2.  The following definitions in Section 1.1 of the Recapitalization
                                       -----------
Agreement are hereby deleted in their entirety and replaced with the
corresponding definitions above as follows:

      "Convertible Subordinated Debt" is deleted and all references to
Convertible Subordinated Debt in the Recapitalization Agreement and related
exhibits thereto shall be replaced with the term "Secured Convertible
Subordinated Debt."

      "Convertible Subordinated Debt Placement Closing" is deleted and all
references to Convertible Subordinated Debt Placement Closing in the
Recapitalization Agreement and related exhibits thereto shall be replaced with
the term "Secured Convertible Subordinated Debt Placement Closing."

      "Convertible Subordinated Debt Placement Closing Date" is deleted and all
references to Convertible Subordinated Debt Placement Closing Date in the
Recapitalization Agreement and related exhibits thereto shall be replaced with
the term "Secured Convertible Subordinated Debt Placement Closing Date."

      "Convertible Subordinated Debt Placement" is deleted and all references to
Convertible Subordinated Debt Placement in the Recapitalization Agreement and
related exhibits thereto shall be replaced with the term "Secured Convertible
Subordinated Debt Placement."

      "Convertible Subordinated Debt Purchasers" is deleted and all references
to Convertible Subordinated Debt Purchasers in the Recapitalization Agreement
and related exhibits thereto shall be replaced with the term "Secured
Convertible Subordinated Debt Purchasers."

      3.  Section 2.2(b)(ii) of the Recapitalization Agreement is hereby amended
          ------------------
by adding the following sentence immediately prior to the end thereof:

                                       3
<PAGE>

      "The amounts of Common Stock reflected in this Section 2.2(b) do not take
into account the effect of the Reverse Stock Split.  Notwithstanding anything
herein to the contrary, if the Reverse Stock Split is approved by the Company's
shareholders in accordance with applicable law and effected by the Company's
Board of Directors prior to the consummation of the Debt Exchange, the numbers
of shares of Common Stock issuable to exchanging holders of Old Notes and
underlying the Debt Exchange Warrants issued pursuant to this Section 2.2(b)
shall be reduced correspondingly by the Reverse Stock Split Ratio."

      4.  The first sentence of Section 2.2(g) of the Recapitalization Agreement
                                --------------
is hereby amended to insert "(i)" before the first sentence thereof and to amend
the two existing sentences thereof to read in their entirety as follows:

      "(i) Upon completion of the Debt Exchange, but in no event later than July
1, 2001, the Company shall issue an aggregate of 5.04 million shares of Common
Stock to the Signatory Debtholders. If no additional Exchange Notes are required
to be issued by the Company to any holders of Unexchanged Notes as provided in
Section 2.4(c), the Company shall issue an aggregate of 2 million additional
shares of Common Stock to the Signatory Debtholders on the earlier of (A) the
date all Exchange Notes issued to holders of Senior Secured Debt in exchange for
such notes are exchanged for Secured Convertible Subordinated Debt pursuant to
Section 2.4(b) or (B) April 15, 2002."

      5.  Section 2.2(g) of the Recapitalization Agreement is hereby further
          --------------
amended by adding a new paragraph (ii) thereto, which new paragraph shall read
in its entirety as follows:

      "(ii)  The amounts of Common Stock reflected in this Section 2.2(g) do not
take into account the effect of the Reverse Stock Split.  Notwithstanding
anything herein to the contrary, if the Reverse Stock Split is approved by the
Company's shareholders in accordance with applicable law and effected by the
Company's Board of Directors, the numbers of shares of Common Stock issuable to
the Signatory Debtholders pursuant to this Section 2.2(g), if issued prior to
the Reverse Stock Split being effected, will be reduced correspondingly by the
Reverse Stock Split Ratio.  If the nonconvertible promissory notes referred to
in Section 2.2(g)(i) are required to be issued prior to the Reverse Stock Split
being effected, the notes shall be issued in an aggregate principal amount equal
to the product of (A) the number of shares not so issuable and (B) $3.00,
increased correspondingly by the Reverse Stock Split Ratio, but subject to a
maximum aggregate principal amount of such notes of $1.872 million."

      6.  Section 2.4(a) of the Recapitalization Agreement is hereby amended by
          --------------
adding the following immediately prior to the end thereof:

      "The amounts of Common Stock reflected in this Section 2.4(a) do not take
into account the effect of the Reverse Stock Split.  Notwithstanding anything
herein to the contrary, if the Reverse Stock Split is approved by the Company's
shareholders in accordance with applicable law and effected by the Company's
Board of Directors prior to the exchange of Senior Secured Debt contemplated in
this Section 2.4(a), the numbers of shares of Common Stock issuable to the
Senior Secured Debt Purchasers and underlying the Debt Exchange Warrants issued
pursuant to this Section 2.4(a) will be reduced correspondingly by the Reverse
Stock Split Ratio."

                                       4
<PAGE>

      7.  The third sentence of Section 3.1(a) of the Recapitalization Agreement
                                --------------
is hereby amended to read in its entirety as follows:

      "Immediately prior to each of the Senior Secured Debt Closing, the Debt
Exchange Closing and the Convertible Subordinated Debt Placement Closing, the
Company's outstanding Capital Stock will be as set forth in the preceding
sentence, except for (i) any increases in outstanding Common Stock as a result
of the transactions provided for herein or of the exercise of options or
warrants referred to in the penultimate sentence of this Section 3.1 and (ii)
any decreases in outstanding Common Stock as a result of the Reverse Stock
Split."

      8.  Section 5.9 of the Recapitalization Agreement is hereby amended by
          -----------
adding the following immediately prior to the end thereof:

      "The amounts of Common Stock reflected in this Section 5.9 do not take
into account the effect of the Reverse Stock Split. Notwithstanding anything
herein to the contrary, if the Reverse Stock Split is approved by the Company's
shareholders in accordance with applicable law and effected by the Company's
Board of Directors, the numbers of shares of Common Stock issuable upon exercise
of the stock options granted pursuant to this Section 5.9 will be reduced
correspondingly by the Reverse Stock Split Ratio."

      9. The Collateral Agency and Security Agreement attached as Exhibit B-2
                                                                  -----------
to the Recapitalization Agreement is amended in its entirety and replaced by the
"Amended and Restated Collateral Agency and Security Agreement" attached as
Exhibit B-2 hereto. All references to the Collateral Agency and Security
Agreement in the Recapitalization Agreement and related exhibits thereto shall
be replaced with the term "Amended and Restated Collateral Agency and Security
Agreement. "

      10.  In accordance with Section 7(a) of the Form of Debt Exchange Warrant
Agreement attached as Exhibit C-2 to the Recapitalization Agreement, if the Debt
                      -----------
Exchange Warrants shall be issued after the Reverse Stock Split, the Exercise
Price referred to therein shall be adjusted to the price (calculated to the
nearest 1,000th of one cent) determined by multiplying the amount of such
Exercise Price immediately prior to the Reverse Stock Split by a fraction, the
numerator of which shall be the number of shares of Common Stock purchasable
upon exercise of the Debt Exchange Warrant immediately prior to the Reverse
Stock Split and the denominator of which shall be the number of shares of Common
Stock purchasable upon exercise of the Debt Exchange Warrant after the
adjustment.

       11.  With respect to the Registration Rights Agreement attached to the
Recapitalization Agreement as Exhibit E-1, the amounts of Common Stock
                              -----------
registrable pursuant thereto and reflected therein shall be reduced
correspondingly by the Reverse Stock Split Ratio.

       12.  This Amendment shall be deemed to be an amendment to the
Recapitalization Agreement, and the Recapitalization Agreement, as amended
hereby, is hereby ratified, approved and confirmed in each and every respect.
All references to the Recapitalization Agreement in

                                       5
<PAGE>

any other document, instrument, agreement or writing shall hereafter be deemed
to refer to the Recapitalization Agreement as amended hereby.

      13. Except as expressly amended herein, the Recapitalization Agreement and
related exhibits thereto, and all documents, instruments and agreements relating
thereto or executed in connection therewith shall remain in full force and
effect as currently written.

      14.  This Amendment may be executed in any number of counterparts, all of
which together shall constitute one agreement.

                                       6
<PAGE>

     IN WITNESS WHEREOF, the parties hereto have caused this Amendment to be
executed by their respective officers thereunto duly authorized as of the day
and year first above written.

                                    IMPERIAL CREDIT INDUSTRIES, INC.


                                    By:  /s/ H. Wayne Snavely
                                       -----------------------------
                                    Name: H. Wayne Snavely
                                    Title: Chief Executive Officer
<PAGE>

                              SENIOR SECURED DEBT PURCHASERS (HOLDERS OF
                              MAJORITY IN AGGREGATE PRINCIPAL AMOUNT):

                              AFFINITY BANK HOLDINGS, INC.

                              By: /s/ Paul Schedler
                                 ------------------------

                                  Name:  Paul Schedler
                                         ----------------
                                  Title: Vice President
                                         ----------------

                                 /s/ Andris A. Baltins
                              ---------------------------
                              Andris A. Baltins
<PAGE>

                              SIGNATORY DEBTHOLDERS:

                              IMPERIAL HOLDINGS GROUP, LLC

                              By: /s/ Michael Riley
                                 -----------------------

                                 Name:  Michael Riley
                                        ----------------
                                 Title:
                                        ----------------