<PAGE>
 
                                                                 EXHIBIT 10.3(5)

as of June 27, 1997


Chart House, Inc.
115 South Acacia Avenue
Solana Beach, CA  92075

Attention:  James Wendler
            Vice President and Chief Financial Officer

Gentlemen:

Reference is hereby made to (x) the Note Purchase and Guarantee Agreement, dated
as of December 30, 1993 (as amended, the "6.69% Note Agreement"), among Chart
House, Inc. (the "Company"), the Guarantors and Metropolitan Life Insurance
Company ("MetLife"), pursuant to which the Company issued its 6.69% Senior
Secured Notes due 2001 (as amended, the "6.69% Notes"), and (y) the Amended and
Restated Note Purchase and Guarantee Agreement, dated as of December 30, 1993
(as amended, the "10.40% Note Agreement" and, collectively with the 6.69% Note
Agreement, the "Note Agreements"), among the Company, the Guarantors and
MetLife, pursuant to which there is outstanding the Company's 10.40% Senior
Secured Notes due 2000 (as amended, the "10.40% Notes" and, collectively with
the 6.69% Notes, the "Notes"). Capitalized terms not otherwise defined herein
shall have the meanings ascribed thereto in the Note Agreements as amended
hereby.

As holder of the Notes and party to the Note Agreements, MetLife hereby consents
and agrees that, effective as of the Effective Date (as hereinbelow defined) and
subject to the first Credit Event (as defined in the Bank Agreement) having
occurred:

     a) the due dates for the $3,000,000 mandatory principal prepayments with
respect to the 10.40% Notes payable on July 24, 1998 and July 24, 1999 pursuant
to (S)4.2 of the 10.40% Note Agreement shall in each case be modified (and
(S)4.2 of the 10.40% Note Agreement shall be deemed amended mutatis mutandis to
                                                            ------- --------  
reflect such modification) to be June 30, 1997, and the Company's failure to 
make such mandatory principal prepayments with respect to the 10.40% Notes, 
together with accrued interest thereon, on June 30, 1997 shall constitute an 
Event of Default under (S)(S)5.1(a) and/or (b) of the 10.40% Note Agreement 
(after the applicable

                                       1
<PAGE>
 
grace periods therein have expired) and under (S)5.1(f) of the 6.69% Note 
Agreement;

     b) the increase in the interest rate applicable to (i) the 6.69% Notes to 
8.69% per annum and (ii) the 10.40% Notes to 12.40% per annum, in each case from
and after June 30, 1997 as provided in the letter agreement, dated November 12, 
1996, among MetLife, the Company and the Guarantors (the "Letter Agreement") 
shall be, and hereby is, rescinded;

     c) the amendment to the Interest Payment Dates applicable to the Notes 
effected by the Letter Agreement to be the second day of each month, commencing 
with July 2, 1997, shall be, and hereby is, rescinded, so that the Interest 
Payment Dates applicable to (i) the 6.69% Notes shall be January 2 and July 2 
of each year, and (ii) the 10.40% Notes shall be January 24, April 24, July 24 
and October 24 of each year;

     d) (S)7 of the Note Agreements shall be amended to read in its entirety as 
set forth in Exhibit A hereto;

     e) (S)5.1 of the Note Agreements shall be amended as follows:
                
          i) subsection (c) of (S)5.1 shall be amended to read as follows:
         
          "(c) if the Company or any Guarantor shall fail to maintain its
     corporate existence or shall default in the observance or performance of
     any covenant or agreement contained in (S)7.1, (S)(S)7.4-7.6, inclusive, or
     (S)(S)7.8-7.26, inclusive;";

          ii) subsection (d) of (S)5.1 shall be amended to read as follows:
          
          "(d) if the Company or any Guarantor shall fail to observe or perform
     any term or provision contained in (S)(S)7.2, 7.3 or 7.7 and such failure
     shall continue for ten (10) days or the Company or any Guarantor shall fail
     to observe or perform any other term or provision contained in the Loan
     Documents (other than as specified in subsections (a), (b), (c), (h) and
     (i) of this (S)5.1) and such failure shall continue for thirty (30) days
     after the earlier of (1) written notice of such failure has been given to
     the Company by any holder of a Note and (2) the date the Company or the
     applicable Guarantor has knowledge of such failure;";

          iii) subsection (f) or (S)5.1 shall be amended to read as follows: 
 
          "(f) if the Company, any Guarantor or any or their

                                       2
<PAGE>
 
Subsidiaries shall default in any payment of principal of or interest on any 
Indebtedness (other than the Notes) in an aggregate amount for all such Persons 
in excess of $1,000,000 or any other event shall occur, the effect of which is 
to permit such Indebtedness to be declared or otherwise to become due prior to 
its stated maturity;";

     iv)  subsection (g) of (S)5.1 shall be amended to read as follows:

      "(g) if one or more judgments or decrees shall be entered against the
Company, any Guarantor or any of their Subsidiaries and such judgments or
decrees (other than decrees involving amounts in the aggregate for all such
Persons of less that $1,000,000) shall not have been vacated, discharged,
stayed, satisfied or bonded pending appeal within thirty (30) days from the
entry thereof or in any event later that five (5) days prior to the date of any
proposed sale thereunder;";

      v) subsection (h) of (S)5.1 shall be amended by deleting the references
to "(S)7.20" therein and substituting therefor "(S)7.13";

      vi) subsection  (i) of (S)5.1 shall be amended by adding the phrase "or 
shall attempt to rescind or revoke the Guarantee" immediately prior to the 
semicolon at the end thereof;

      vii) subsection (j) of (S)5.1 shall be amended to read as follows:

      "(j) if (1) any Person shall engage in any "prohibited transaction" (as
defined in Section 406 of ERISA or Section 4975 of the Code) involving any Plan,
(2) any "accumulated funding deficiency" (as defined in Section 302 of ERISA),
whether or not waived, shall exist with respect to any Plan, (3) a Reportable
Event shall occur with respect to, or proceedings shall commence to have a
trustee appointed, or a trustee shall be appointed, to administer or to
terminate any Single Employer Plan, which Reportable Event or institution of
proceedings is, in the reasonable opinion of the Required Holders, likely to
result in the termination of such Plan for purposes of Title IV of ERISA, and,
in the case of a Reportable Event, the continuance of such Reportable Event
unremedied for ten days after notice of such Reportable Event pursuant to
Section 4043(a), (c) or (d) of ERISA is given or the continuance of such
proceedings for ten days after commencement thereof, as the case may be, (4) any
Single Employer Plan shall terminate for purposes of Title IV of ERISA, (5) any
withdrawal liability to a Multiemployer Plan shall be incurred by the Company or
(6) any other event or
                                       3
<PAGE>
 
     condition shall occur or exist; and in each case in clauses (1) through (6)
     above, such event or condition, together with all other such events or
     conditions, if any, is likely to subject the Company, any Guarantor or any
     of their Subsidiaries to any tax, penalty or other liabilities in the
     aggregate material in relation to the business, operations, property or
     financial or other condition of the Company or any Guarantor;";

          viii) subsection (k) of (S)5.1 shall be amended to read as follows:

          "(k) if (1) the Company, any Guarantor or any of their Subsidiaries
     shall commence any case, proceeding or other action (i) under any existing
     or future law of any jurisdiction, domestic or foreign, relating to
     bankruptcy, insolvency, reorganization or relief of debtors, seeking to
     have an order for relief entered with respect to it, or seeking to
     adjudicate it a bankrupt or insolvent, or seeking reorganization,
     arrangement, adjustment, winding-up, liquidation, dissolution, composition
     or other relief with respect to it or its debts, or (ii) seeking
     appointment of a receiver, trustee, custodian or other similar official for
     it or for all or any substantial part of its assets, or the Company, any
     Guarantor or any of their Subsidiaries shall make a general assignment for
     the benefit of its creditors; or (2) there shall be commenced against the
     Company, any Guarantor or any of their Subsidiaries any case, proceeding or
     other action of a nature referred to in clause (1) above which (i) results
     in the entry of an order for relief or any such adjudication or
     appointment, or (ii) remains undismissed, undischarged or unbonded for a
     period of thirty (30) days; or (3) there shall be commenced against the
     Company, any Guarantor or any of their Subsidiaries any case, proceeding or
     other action seeking issuance of a warrant of attachment, execution,
     distraint or similar process against all or substantially all of its assets
     which results in the entry of an order for any such relief which shall not
     have been vacated, discharged, stayed, satisfied or bonded pending appeal
     within sixty (60) days from the entry thereof; or (4) the Company, any
     Guarantor or any of their Subsidiaries shall take any action in furtherance
     of, or indicating its consent to, approval of, or acquiescence in (other
     than in connection with a final settlement), any of the acts set forth in
     clauses (1), (2) or (3) above; or (5) the Company, any Guarantor or any of
     their Subsidiaries shall generally not, or shall be unable to, or shall
     admit in writing its inability to, pay its debts as they become due;";
     
          ix) (S)5.1 shall be amended by adding new subsections (1), (m), (n)
     and (o) to read as follows:

                                       4
<PAGE>
 
          "(1) if the Company shall voluntarily suspend the transaction of
     business for more than five days in any calendar year commencing in fiscal
     year 1997;

          (m) if there shall occur an "Event of Default" under (and as that term
     is defined in) the Bank Agreement;

          (n) if the Parent shall cease to own one hundred percent (100%) of the
     outstanding capital stock of the Company or there shall occur a Change of
     Control of the Parent; or

          (o) if the independent public accountants of the Parent at June 27,
     1997 shall resign and a replacement firm of independent public accountants
     reasonably acceptable to the Required Holders shall not have been appointed
     (and accepted such appointment) within sixty (60) days following such
     resignation.";

     f) (S)10 of the Note Agreements shall be amended as follows:

          i) the following defined terms shall be added in appropriate
alphabetical order (and, where applicable, shall replace any identical term
already defined in said (S)10):

              "BankBoston" shall mean BankBoston, N.A., formerly known as The 
               ----------
     First National Bank of Boston.

              "Bank/Metropolitan Obligations" shall mean, collectively, the Bank
               -----------------------------
     Obligations and the Metropolitan Obligations.

              "Bank Obligations" shall have the meaning given such term in each 
               ----------------
     of the Security Documents.

              "Capital Expenditures" shall mean, for any period, the aggregate 
               --------------------  
     of all expenditures by the Parent and its Subsidiaries for the acquisition
     of fixed or capital assets or additions to equipment (including
     replacements, capitalized repairs and improvements during such period)
     which should be capitalized under GAAP on a consolidated balance sheet of
     the Parent and its Subsidiaries. For the purpose of this definition, the
     purchase price of equipment which is purchased simultaneously with the
     trade-in of existing equipment owned by the Company or any of its
     Subsidiaries or with insurance proceeds shall be included in Capital
     Expenditures only to the extent of the gross amount of such purchase price
     less the credit granted by the seller of such equipment for such equipment
     being traded in at such time, or the amount of such proceeds, as the case
     may be.

                                       5
<PAGE>
 
     "Capitalized Leases" shall mean leases under which the Parent or any of its
      ------------------
Subsidiaries is the lessee or obligor, the future rental payment obligations 
under which are required to be capitalized on the balance sheet of the lessee or
obligor in accordance with GAAP.

     "Change of Control" shall mean if any Person other than either Chart House 
      -----------------
Investors, L.L.C. or Metropolitan shall own more than twenty percent (20%) of 
the outstanding capital stock of the Parent.

     "Collateral" shall mean the real and personal property (tangible and 
      ----------
intangible) and fixtures which are the subject of the Company Security 
Documents.

     "Commonly Controlled Entity" of a Person shall mean a Person, whether or 
      --------------------------
not incorporated, which is under common control with such Person within the
meaning of Section 414(c) of the Code .


     "Consolidated Net Worth" shall mean the excess of the consolidated total 
      ----------------------
assets of the Parent and its Subsidiaries, determined in accordance with GAAP, 
over Consolidated Total Liabilities, excluding in any event all non-cash gains 
or losses arising in connection with any Other Permitted Asset Sale.

     "Consolidated Total Liabilities" shall mean all liabilities of the Parent
      ------------------------------
and its subsidiaries determined on consolidated basis required to be shown on a
balance sheet of the Parent in accordance with GAAP.

     "Contractual Obligation" as to any Person shall mean any provision of any 
      ----------------------
security issued by such Person or of any agreement, instrument or undertaking to
which such Person is a party or by which it or any of its property is bound.

     "EBITDA" shall mean for any period the sum of (a) net income (or net loss) 
      ------
for such period, plus (b) all amounts treated as expenses for interest, 
amortization, depreciation, taxes (to the extent included in the determination 
of net income (or net loss)), and other non-cash charges for such period, plus 
(c) to the extent not otherwise included in subclause (b) above and only for 
purposes of determining compliance with (S)(S)7.23 and 7.24 through June 30, 
1998, cash restructuring charges (not to exceed $900,000), and minus (d) 
non-cash credits for such period.

     "GAAP" shall mean generally accepted accounting principles in the United 
      ----
States of America in effect from time to time.

                                       6
<PAGE>
 
      "Governmental Authority" shall mean any nation or government, any state or
       ----------------------
other political subdivision thereof, or any entity exercising executive, 
legislative, judicial, regulatory or administrative functions of or pertaining 
to government.

      "Hazardous Materials" shall mean:
       -------------------

          (a) "hazardous substances", "hazardous wastes", "hazardous materials" 
or "toxic substances", as defined in any of the Hazardous Materials Laws;

          (b) any pollutant or contaminant, or hazardous, dangerous or toxic 
chemical, material, waste or substance which Hazardous Materials Laws prohibit, 
limit or otherwise regulate as to use, exposure, release, generation, 
manufacture, sale, transport, handling, storage, treatment, reuse, presence, 
disposal or recycling;

          (c) petroleum, crude oil or any fraction of petroleum or crude oil;

          (d) any radioactive material, including any source, special nuclear or
by-product material, as defined at 42 U.S.C. (S)2011 et seq., and amendments 
thereto and reauthorizations thereof;                ------   

          (e) asbestos-containing materials in any form or condition; and

          (f) polychlorinated biphenyls.

      "Hazardous Materials Claims" shall mean any enforcement, cleanup, removal 
       --------------------------
or other governmental or regulatory action or order with respect to the Property
pursuant to any Hazardous Materials Laws, and/or any claim asserted in writing 
by any third party relating to damage, contribution, cost recovery compensation,
loss or injury resulting from any Hazardous Materials.

      "Hazardous Materials Laws" shall mean any applicable federal, state or 
      -------------------------
local laws, ordinances or regulations relating to Hazardous Materials.

      "Indebtedness" of any Person shall mean all items of indebtedness which, 
       ------------
in accordance with GAAP and practices, would be included in determining 
liabilities as shown on the liability side of a balance sheet of such Person as 
of the date as of which Indebtedness is to be determined including, without 
limitation, all obligations for money borrowed and

                                       7
<PAGE>
 
capitalized lease obligations, and shall also include all indebtedness and 
liabilities of others assumed or guaranteed by such Person or in respect of 
which such Person is secondarily or contingently liable (other than by 
endorsement of instruments in the course of collection) whether by reason of any
agreement to acquire such indebtedness or to supply or advance sums or 
otherwise.

        "Interest Expense" shall mean for any period, interest accrued or paid
         ----------------
to the extent not accrued in a prior period by the Parent and its subsidiaries
for such period (including, without limitation, capitalized interest with
respect to Indebtedness, facility fees, commitment fees and similar fees
incurred in connection with the borrowing of money).

        "Lien" shall mean any security interest, mortgage, pledge, lien, claim
         ----
on property, charge or encumbrance (including any conditional sale or other
title retention agreement), any lease in the nature thereof, and the filing of
or agreement to give any financing statement under the Uniform Commercial Code
of any jurisdiction.

        "Loan Documents" shall mean this Agreement, the Notes, the Security 
         --------------
documents and each other document, instrument or agreement executed by the 
Company or any Guarantor or otherwise delivered in connection herewith or 
therewith, as any of the same may be amended, extended or replaced form time to
time with the consent of the holders of the Notes.

        "Metropolitan Obligations" shall have the meaning given such term in 
         ------------------------
each of the Security Documents. 

        "Mortgages" shall mean those real property mortgages described more 
         --------- 
particularly on Exhibit O attached to the Bank Agreement (as in effect on June 
27, 1997) and any other mortgages or deeds of trust that may subsequently be 
executed and delivered by the Company to the Security Agent, as any of the same
may be amended, extended or replaced from time to time with the consent of the 
holders of the Notes.

        "Multiemployer Plan" as to any Person shall mean a Plan of such Person 
         ------------------
which is a multiemployer plan as defined in Section 4001(a)(3) of ERISA.

        "Other Permitted Asset Sales" shall mean sales of those assets described
         ---------------------------  
on Exhibit P attached to the Bank Agreement (as in effect on June 27, 1997).

        "Other Permitted Debt" shall mean Indebtedness described on Exhibit Q 
         --------------------
attached to the Bank agreement (as in


                                       8




 





<PAGE>
 
effect on June 27, 1997).

     "Other Permitted Guaranties" shall mean those guaranties described on
      -------------------------- 
Exhibit R attached to the Bank Agreement (as in effect on June 27, 1997).

     "Other Permitted Investments" shall mean Investments described on Exhibit
      --------------------------- 
S attached to the Bank Agreement (as in effect on June 27, 1997).

     "Other Permitted Secured Debt" shall mean Indebtedness described as such
      ----------------------------
on Exhibit Q attached to the Bank Agreement (as in effect on June 27, 1997).

     "Plan" shall mean as to any Person, any pension plan that is covered by 
      ----
Title IV of ERISA and in respect of which such Person or a Commonly Controlled 
Entity of such Person is an "employer" as defined in Section 3(5) of ERISA.

     "Property" shall mean, collectively and severally, any and all real 
      --------
property, including all improvements and fixtures thereon, owned or occupied by 
the Company or any of its Subsidiaries.

     "Reportable Event" shall mean a reportable event as defined in Title IV of 
      ----------------
ERISA, except actions of general applicability by the Secretary of Labor under 
Section 110 of ERISA.

     "Required Principal Payments of Debt" shall mean for any period all
      -----------------------------------
regularly scheduled payments on account of Indebtedness for borrowed money and 
capitalized lease obligations during such period, provided that "regularly 
scheduled payments" shall not include (i) the aggregate $9,000,000 of principal 
payments due on the Metropolitan Obligations on July 24, 1996, January 2, 1997 
and July 24, 1997, regardless of when paid, or (ii) otherwise regularly 
scheduled payments due during such period which are prepaid during a previous 
period.

     "Requirements of Law" shall mean as to any Person the certificate of
      -------------------
incorporation and bylaws or other organizational or governing documents of such
Person, and any law, treaty, rule or regulation, or a final and binding
determination of an arbitrator or a determination of a court or other
Governmental Authority, in each case applicable to or binding upon such Person
or any of its property or to which such Person or any of its property is
subject.

     "Secured Parties" shall have the meaning given such term in each of the
      --------------- 
Security Documents.

                                       9
<PAGE>
 
     "Security Agent" shall mean Sumitomo and any successor to Sumitomo as the 
      --------------
Security Agent under the Security Documents.

     "Single Employer Plan" shall mean as to any Person any Plan of such Person 
      --------------------
which is not a Multiemployer Plan.

     "Subsidiary" shall mean with respect to any Person, any corporation more 
      ----------
than fifty percent (50%) of the stock of which having by the terms thereof 
ordinary voting power to elect the board of directors, managers or trustees of 
such corporation shall, at the time as of which any determination is being made,
be owned by such Person, either directly or through Subsidiaries of such Person,
either directly or through Subsidiaries of such Person (irrespective of whether
or not at the time stock of any other class or classes of such corporation shall
have or might have voting power by reason of the happening of any contingency).

     "Sumitomo" shall mean The Sumitomo Bank of California.
      --------

     "Total Funded Debt" shall mean for any Person all Indebtedness for borrowed
      -----------------
money, capitalized lease obligations and contingent liabilities with respect to 
issued and outstanding letters of credit.";
      
     ii) the definition of "Bank Agreement" shall be amended to read as 
follows:

     "Bank Agreement" shall mean the Second Amended and Restated Credit 
      --------------
Agreement, dated as of June 27, 1997, as amended from time to time with the 
consent of the Required Holders.";
     
     iii) the definition of "Bank Notes" shall be amended to read as follows:

     "Bank Notes" shall mean the Replacement Notes (as defined in the Bank 
      ----------
Agreement) issued or to be issued by the Company to the Banks under, and 
pursuant to the provisions of, the Bank Agreement.";

     iv) the definition of "Banks" shall be amended to read as follows:

     "Banks" shall mean BankBoston, Sumitomo and such other financial
institutions that may become parties to the Bank Agreement pursuant to the terms
thereof.";

     v) the definition of "Change of Control Date" shall be amended by (i) 
adding the phrase "or Chart House Investors, L.L.C."

                                      10
<PAGE>
 
after the word "Metropolitan", and (ii) deleting the reference to "(S)7.24" and 
substituting therefor "(S)7.18";

     vi) the definition of "generally accepted accounting principles" and 
references to such term shall be deleted and the definition of "GAAP" and 
references to such term shall be substituted therefor, respectively; and

     vii) the definitions of "Consolidated Net Earnings Available for Fixed 
Charges", "Consolidated Net Earnings Available for Interest Charges", 
"Consolidated Stockholders Equity", "Consolidated Tangible Net Worth", "Fixed 
Charges", "FNBB", "Interest Charges", "Loans", "Net Proceeds", "Paradise", 
"Revolving Credit Loans" and "Sanwa" shall be deleted;

     g) (S)10 of the 10.40% Note Agreement shall be amended by adding the 
following definitions in appropriate alphabetical order:

          "Other Metropolitan Agreement" shall mean the 6.69% Agreement, as 
           ----------------------------
     amended from time to time.

          "Other Metropolitan Loan Documents" shall mean the "Loan Documents",
           --------------------------------- 
     as defined in the Other Metropolitan Agreement.

          "Other Metropolitan Notes" shall mean the 6.69% Notes, as amended from
           ------------------------
     time to time.";

     h) (S)10 of the 6.69% Note Agreement shall be amended by adding the 
following definitions in appropriate alphabetical order:

          "Other Metropolitan Agreement" shall mean the 1990 Agreement, as 
           ----------------------------       
     amended from time to time.

          "Other Metropolitan Loan Documents" shall mean the 1990 Loan 
           ---------------------------------
     Documents.

          "Other Metropolitan Notes" shall mean the 1990 Notes, as amended from 
           ------------------------
     time to time.";

     i) (S)6.2 of the Note Agreements shall be amended, mutatis mutandis, to 
                                                        ------- --------
incorporate therein the terms and provisions of paragraphs 1-10 and 12-16 of the
Amended and Restated Guaranties (as defined in the Bank Agreement and as such 
Guaranties exist on the date hereof), and the terms and provisions of said 
paragraphs shall be deemed to be a part of said (S)6.2 as though set forth in
full therein, except that, for this purpose, (i) references in said paragraphs
to "Obligations" shall mean the respective Metropolitan Obligations, (ii)
references in said paragraphs to "Agent", "Banks" or "Bank" shall mean the
holders of the 6.69% Notes and the 10.40% Notes, respectively, and (iii)
references in said paragraphs to "Credit Agreement" shall mean the 6.69% Note
                                      11
<PAGE>
 
Agreement and the 10.40% Note Agreement, respectively; and

      j) (S)4.5 of the 6.69% Note Agreement and (S)4.6 of the 10.40% Note 
Agreement  (and references in the respective Note Agreements to said Sections) 
shall be deleted.

By their execution hereof, the Company and each of the Guarantors affirms and 
agrees that (a) the execution and delivery by such Persons of, and the 
performance of their respective obligations under, this consent and agreement 
and the Bank Agreement shall not in any way amend, impair, invalidate or 
otherwise affect any of such Person's obligations or the rights, remedies and 
powers of MetLife under the Loan Documents, including, without limitation, the 
Security Documents, as the same are amended hereby and (in the case of the 
Security Documents) by the Bank Agreement, (b) all Loan Documents remain in 
full force and effect, (c) the term "Obligations" as used in the Security 
Documents (as amended by the Bank Agreement) includes the Metropolitan 
Obligations, and (d) they will deliver or cause to be delivered to 
MetLife, concurrently with the delivery thereof to the Security Agent, the 
Additional Mortgage Documents with respect to the Additional Florida 
Properties (as such terms are defined in the Bank Agreement), all of which 
shall be satisfactory to MetLife.

By their execution hereof, each of the Company, the Parent and Big Wave, 
severally and independently as to itself only, represents and warrants that as 
of the date hereof and at and as of the Effective Date:

      (a)   Except to the extent such were by their terms made solely as of a 
prior date, the representations and warranties of such party contained in the 
Loan Documents are accurate and complete in all material respects.

      (b)  The execution and delivery by such party of this consent and 
agreement and the performance by such party of its obligations hereunder are 
within the corporate power of such party, have been (or as of the Effective Date
will be) duly authorized by all necessary corporate action and do not and will 
not (1) contravene any provision of such party's charter, other incorporation 
papers, by-laws or any stock provisions, or any amendment thereof, (2) 
conflict with, or result in a breach of any material term, condition or 
provision of, or constitute a default under, or result in the creation of any 
mortgage, lien, pledge, charge, security interest or other encumbrance upon any 
of the property of any of such parties pursuant to, any agreement, deed of 
trust, indenture, mortgage or other instrument agreement, deed of trust, 
indenture, mortgage or other instrument to which such party is a party or by 
which any of its properties are bound, (3) violate or contravene any provision
of any law, regulation, order, ruling or interpretation thereunder or any 

                                      12
<PAGE>
 
decree, order or judgment of any court or governmental or regulatory authority, 
bureau, agency or official, (4) require any waiver, consent or approval of any 
Person other than such as have been obtained and copies of which have been 
provided to MetLife, or (5) require any approval, consent, order, authorization
or license by, or giving of notice to, or taking any other action with respect 
to, any governmental or regulatory authority or agency under any provision of 
law, except those actions which have been taken or will be taken prior to the 
Effective Date.

     (c) This consent and agreement constitutes the legal, valid and binding 
obligation of such party enforceable against such party in accordance with its 
terms.

     (d) No Default or Event of Default has occurred or is continuing or will 
occur as a result of (1) the execution and delivery of this consent and 
agreement, or (2) the consummation of the transactions contemplated hereby.

     (e) There has been no material adverse change in the financial condition or
prospects of such party since December 30, 1996.

This consent and agreement shall become effective upon the date (the "Effective 
Date") upon which each of the following shall have occurred:

     (a) the Company shall have paid to MetLife, in the manner specified in 
(S)4.8 of the 10.40% Note Agreement, the mandatory principal prepayments with 
respect to the 10.40% Notes referred to in paragraph a) hereof, together with 
accrued interest thereon;

     (b) the Company shall have paid to MetLife the amount of $50,000 as an 
amendment fee;

     (c) there shall have been delivered to MetLife a duly executed copy of the 
Bank Agreement, together with all Exhibits thereto, which shall be satisfactory 
in form and substance to MetLife;

     (d) there shall have been delivered to MetLife the Additional Mortgage
Documents for the Dobbs Ferry Property (as defined in the Bank Agreement), all
of which shall be satisfactory to MetLife;

     (e) there shall have been delivered to MetLife evidence satisfactory to 
MetLife that all conditions precedent to the first Credit Event set forth in 
Paragraph 7(a) of the Bank Agreement shall have been satisfied and that the Bank
Agreement is in full force and effect;

                                      13

<PAGE>
 
     (f) there shall have been delivered to MetLife copies of the items referred
to in clauses (v), (x) (certified in favor of MetLife), (xii), (xiii), (xvi), 
(xvii) (as to the Lien of the Security Documents with respect to the 
Metropolitan Obligations), (xviii) and (xix) of Paragraph 7(a) (1) of the Bank 
Agreement;

     (g) there shall have been delivered to MetLife a copy of this consent and 
agreement, duly executed by the Company and the Guarantors;

     (h) there shall have been delivered to MetLife a counterpart of this 
consent and agreement, acknowledged and agreed to by the Agent, the Security 
Agent and the Banks; and

     (i) there shall have been delivered to MetLife from or with respect to each
of Big Wave, the Company and the Parent, (i) copies (certified in favor of 
MetLife) of corporate resolutions authorizing the execution, delivery and 
performance of (A) this consent and agreement, (B) the Loan Documents as amended
hereby and (in the case of the Security Documents) by the Bank Agreement and (C)
the Bank Agreement, (ii) incumbency certificates with respect to this consent 
and agreement, and (iii) good standing certificates from the Secretary of State 
of California and of Delaware, in each case satisfactory to MetLife.

In the event the Effective Date does not occur on June 30, 1997, then this 
consent and agreement shall, at the election of MetLife, as evidenced by written
notice of such election delivered by MetLife to the Company, terminate and be of
no further force or effect.

The representations, warranties, covenants and agreements of the Company, the 
Parent and Big Wave set forth herein shall survive the Effective Date.

This consent and agreement shall be governed by and construed in accordance with
the laws of the State of New York.

                                      14
<PAGE>
 
This consent and agreement may be executed in counterparts and such counterparts
shall, when taken together, constitute one and the same agreement.

Very truly yours,

METROPOLITAN LIFE INSURANCE COMPANY

By: /s/ JACQUELINE D. JENKINS
    -------------------------
    Jacqueline D. Jenkins
    Vice President


ACKNOWLEDGED AND AGREED TO
this 27th day of June, 1997.

CHART HOUSE, INC.

By: /s/ JAMES C. WENDLER
    ------------------------------
Name:  James C. Wendler
      
Title: Vice President, Treasurer &
       Assistant Secretary


CHART HOUSE ENTERPRISES, INC.

By:  /s/ JAMES C. WENDLER
     --------------------------
     James C. Wendler
     Vice President - Finance &
     Chief Financial Officer


BIG WAVE, INC.

By: /s/ JAMES C. WENDLER
    ------------------------------
Name:  James C. Wendler

Title: Vice President, Treasurer &
       Assistant Secretary


BANKBOSTON, N.A.
as Agent

By: /s/ THOMAS F. FARLEY, JR.
    -------------------------
Name:  Thomas F. Farley, Jr.

Title: Director

                                      15

<PAGE>
 
THE SUMITOMO BANK OF CALIFORNIA
as Security Agent

By: /s/ MATTHEW VAN STEENHUYSE
    --------------------------
Name:  Matthew Van Steenhuyse
      
Title: Vice President
     

BANKBOSTON, N.A.

By: /s/ THOMAS F. FARLEY, JR.
    -------------------------
Name:  Thomas F. Farley, Jr.
    
Title: Director


THE SUMITOMO BANK OF CALIFORNIA


By: /s/ MATTHEW VAN STEENHUYSE
    --------------------------
Name:  Matthew Van Steenhuyse
     
Title: Vice President
     

                                      16