<PAGE>
 
                                                                 EXHIBIT 10.3(2)

                          [LETTERHEAD OF METLIFE/R/]

August 15, 1996

Chart House, Inc.
115 South Acacia Avenue
Solana Beach, California 92075
Attn:  William R. Kuntz, Jr.
       Vice President, Chief Financial Officer and General Counsel


       Re:  Waiver of Specified Events of Default; Amendment of Note Agreements 
            -------------------------------------------------------------------


Ladies and Gentlemen:

     Reference is hereby made to those certain Note Purchase and Guarantee
Agreements dated as of December 30, 1993 (as amended to date, the "Note
Agreements" and with capitalized terms not otherwise defined herein used with
the meanings given such terms in the Note Agreements) by and among Chart House,
Inc. (the "Company"), the Guarantors and Metropolitan Life Insurance Company
("MetLife").

     There exist certain Events of Default under the Note Agreements consisting
of the breach by the Company of certain financial covenants.  The Company has
requested MetLife, as holder of the entire outstanding principal amount of the
Notes, to waive such Events of Default and MetLife has agreed to do so on the
terms and subject to the conditions set forth below.

     1.  Waiver.  On the terms and subject to the conditions set forth herein,
         ------                                                               
effective as of the Effective Date (as defined in Paragraph 6 below), MetLife
hereby waives the Events of Default existing under the Note Agreements by virtue
of the failure of the Company to have been in compliance with the financial
covenants set forth in (S)7.7 and (S)7.31 thereof as of July 1, 1996.

     2.  Additional Financial Information.   Pursuant to subsection (g) of
         --------------------------------                                 
(S)7.9 of the Note Agreements, MetLife hereby requests, and the Company hereby
agrees to provide to MetLife, the following information:

          (a) No later than September 15, 1996, an updated forecast, by month,
              consisting of balance sheet, profit and loss and cash flow of the
              Parent and its consolidated Subsidiaries for each of September,
              October, November and December 1996; and

          (b) Within 30 days following the end of each fiscal quarter of the
              Parent, consolidated statements of income, balance sheets and cash
              flow statements of the Parent and its consolidated Subsidiaries,
              accompanied in each case by the Officers' Certificates referred to
              in subsections (c) and (d) of (S)7.9 of the Note Agreements.

                                       1
<PAGE>
 
     3.  Restriction on Capital Expenditures.  To reflect the agreement of the
         -----------------------------------                                  
Company and the Parent to limit the dollar amount and nature of capital
expenditures of the Parent and its Subsidiaries during calendar year 1996,
effective as of the Effective Date, (S)7.8 of the Note Agreements is hereby
amended to insert the following proviso immediately preceding the period in the
last line thereof:

     "; provided, however, that notwithstanding anything contained herein,
     capital expenditures of the Parent and its Subsidiaries during the 1996
     calendar year shall not exceed $11,000,000 in the aggregate and in no event
     shall at any time following August 15, 1996 funds of the Parent or any of
     its Subsidiaries, including, without limitation, proceeds of the Revolving
     Credit Loans, be utilized for capital expenditures other than expenditures
     for the purpose of the maintenance of existing properties and assets and
     completion of pending minor remodeling projects for which contractual
     commitments were made prior to August 14, 1996".

     4.  Approval of Disposition of Assets.  To reflect the agreement of MetLife
         ---------------------------------                                      
and the Company with respect to the sale of the assets of the recently closed
Long Beach Chart House, effective as of the Effective Date, MetLife hereby
approves the sale thereof at not less than fair market value.

     5.  Reaffirmation of Obligations; No Additional Waiver or Amendment.  The
         ---------------------------------------------------------------      
Company and the Parent and the Guarantors, by executing this letter in the space
provided below, reaffirm all of their respective obligations under the Notes,
the Note Agreements and the other Loan Documents and the Guarantees, as
applicable, and all documents, instruments and agreements executed in connection
therewith or otherwise relating thereto and acknowledge and agree that such
obligations are not altered, modified or affected in any manner or to any extent
except to the extent expressly provided herein.  The Company, the Parent and the
Guarantors further acknowledge that the waiver set forth in Paragraph 1 above is
the sole waiver made by MetLife with respect to the existence of Defaults or
Events of Default (or possible occurrence or existence of additional Defaults or
Events of Default) under the Note Agreements and the other Loan Documents and
the Guarantees, regardless of whether MetLife is aware of, or would upon
investigation have discovered, the existence of such additional Defaults or
Events of Default.

     6.  Effective Date.  This letter agreement, including, without limitation,
         --------------                                                        
the waiver of MetLife set forth in Paragraph 1 above, shall be effective on the
date (the "Effective Date") upon which there shall have been delivered to
MetLife each of the following:

          (a) a copy of this letter agreement, duly executed by all signatories
              set forth below;

          (b) a waiver fee in the amount of $25,000; and

          (c) a duly executed copy of a letter agreement from the Banks
              substantially in the form of Annex 1 hereto.

          If the Effective Date shall not have occurred on or before August 15,
1996, then the agreements of MetLife set forth herein shall, at the option of
MetLife as evidenced by written notice to such effect delivered by MetLife to
the Company, terminate and be of no further force or effect.

          7.  Authorization; Representation and Warranties.  The Company, the
              --------------------------------------------                   
Parent and the Guarantors hereby represent and warrant to MetLife as follows:

                                       2
<PAGE>
 
          (a) The execution, delivery and performance of this letter agreement
    by the officers of each of said parties executing the same has been duly
    authorized by all necessary corporate action (each of said parties hereby
    committing to deliver to MetLife no later than 10 days following the
    Effective Date corporate resolutions and incumbency certificates evidencing
    the accuracy of such representation and warranty); and

          (b) Upon the Effective Date and after giving effect to the waiver set
    forth in Paragraph 1 above, all representations and warranties of the
    Company and the Guarantors contained in the Note Agreements are accurate and
    complete in all material respects on and as of the Effective Date.

    8.  Counterparts.  This letter agreement may be executed in counterparts,
        ------------                                                         
all of which taken together shall constitute one and the same instrument.
 
 
                                         Very truly yours,
 
 
                                         METROPOLITAN LIFE INSURANCE COMPANY
 
                                         By: /s/ JACQUELINE D. JENKINS
                                            -----------------------------------
                                                 Jacqueline D. Jenkins
 
ACKNOWLEDGED AND AGREED TO
this 15th day of August, 1996
 
CHART HOUSE, INC.
 
 
 
By: /s/ WILLIAM R. KUNTZ, JR.
   ----------------------------------
Name:   William R. Kuntz, Jr.
     --------------------------------
Title:  Executive Vice President
      -------------------------------
 
 
CHART HOUSE ENTERPRISES, INC.
 
 
 
By: /s/ WILLIAM R. KUNTZ, JR.
   ----------------------------------
Name:   William R. Kuntz, Jr.
     --------------------------------
Title:  Executive Vice President
      ------------------------------- 
 

                                       3
<PAGE>
 
BIG WAVE, INC. (formerly known as Islands Restaurants, Inc.)


By: /s/ WILLIAM R. KUNTZ, JR.
   ----------------------------------
Name:   William R. Kuntz, Jr.
     --------------------------------
Title:  Vice President
      -------------------------------

                                       4