<PAGE>

                                                                     Exhibit 4.3

                          SUBSCRIPTION AGENT AGREEMENT

     This Subscription Agent Agreement (the "Agreement") is made as of May 10,
2001 between Chart House Enterprises, Inc. (the "Company") and EquiServe Trust
Company, N.A. as subscription agent (the "Agent").  All terms not defined herein
shall have the meaning given in the prospectus (the "Prospectus") included in
the Registration Statement on Form S-2 filed by the Company with the Securities
and Exchange Commission on March 27, 2001, as amended by any amendment filed
with respect thereto (the "Registration Statement").

     WHEREAS, the Company intends to issue (the "Rights Offering") to the
holders of record of its outstanding common stock, $.01 par value per share
("Common Stock"), as of a certain record date (the "Record Date"), rights
("Rights") to purchase one share of Series A Preferred Stock, $1.00 par value
per share, of the Company (the "Series A Preferred Shares") for a specified
number of shares of Common Stock held; and

     WHEREAS, the Company desires the Agent to act on the Company's behalf, and
the Agent is willing to so act, in connection with the issuance  of the Rights,
collection of funds from Rights holders exercising Rights, and issuance and
delivery of Series A Preferred Shares upon the exercise of the Rights, all upon
the terms and conditions set forth herein.

     NOW, THEREFORE, in consideration of the foregoing and of the mutual
agreements set forth herein, the parties agree as follows:

1.  Appointment.  The Company hereby appoints the Agent to act as subscription
agent in connection with the Rights Offering, including the over printing of
subscription certificates evidencing the Rights in the form designated by the
Company (the "Subscription Certificates"), in accordance with the terms set
forth in this Agreement and the Agent hereby accepts such appointment.

2.  Form and Execution of Subscription Certificates.

     (a) Each Subscription Certificate shall be irrevocable and transferable.
The Agent shall, in its capacity as Transfer Agent of the Company, maintain a
register of Subscription Certificates and the holders of record thereof (each of
whom shall be deemed a "Stockholder" hereunder for purposes of determining the
rights of holders of Subscription Certificates).  Each Subscription Certificate
shall, subject to the provisions thereof, entitle each holder of record of
Common Stock on the Record Date (a "Record Date Stockholder") in whose name it
is recorded to the following:

          (1) The right to acquire prior to the Expiration Date (as defined
below) a number of Series A Preferred Shares equal to one Series A Preferred
Share for every one Right (the "Basic Subscription Privilege"); and

          (2) The right to subscribe for additional Series A Preferred Shares,
subject to the availability of such shares and to the allotment of such shares
as may be
<PAGE>

available among Record Date Stockholders or transferees who exercise
Over-Subscription Rights on the basis specified in the Prospectus; provided,
however, that such Record Date Stockholder or transferee has exercised all of
his or her Rights pursuant to the Basic Subscription Privilege (the "Over-
Subscription Privilege").

3.  Rights and Issuance of Subscription Certificates.

     (a)  Each Subscription Certificate shall evidence the Rights of the
Stockholder therein named to purchase Series A Preferred Shares upon the terms
and conditions therein and herein set forth.

     (b)  Upon the written direction of the Company, signed by any of its duly
authorized officers, as of the Record Date, the Agent shall, from a list of the
Company's Record Date Stockholders to be prepared by the Agent in its capacity
as Agent of the Company, prepare and record Subscription Certificates in the
names of the Record Date Stockholders, setting forth the number of Rights to
subscribe for the Company's Series A Preferred Shares calculated on the basis of
one Right for a specified number of shares of Common Stock recorded on the books
in the name of each such Record Date Stockholder. The number of Rights that are
issued to Record Date Stockholders or transferees will be rounded up to the
nearest whole number, by the Agent, as fractional Rights will not be issued.
Each Subscription Certificate shall be dated as of the Record Date.

4.  Mailing of Subscription Documents.  The Agent and Company agree that the
mailing of any and all documents in connection with the Rights Offering shall be
according to those terms agreed by the parties at a later date.

5.   Exercise.

     (a) Stockholders may acquire Series A Preferred Shares pursuant to the
Basic Subscription Privilege and the Over-Subscription Privilege by delivery to
the Agent as specified in the Prospectus of (i) the Subscription Certificate
with respect thereto, duly executed by such Stockholder in accordance with and
as provided by the terms and conditions of the Subscription Certificate,
together with (ii) the Subscription Price for each Series A Preferred Share
subscribed for by exercise of such Rights, in U.S. dollars by check or bank
draft (cashier's check) drawn upon a United States bank or a postal, telegraphic
or express money order payable to the order of the Agent.

     (b) Rights may be exercised at any time after the date of issuance of the
Subscription Certificates with respect thereto but no later than 5:00 P.M. New
York time on such date as the Company shall designate to the Agent in writing
(the "Expiration Date").  Once a holder of Rights has exercised his or her
Rights, such exercise may not be revoked or rescinded.  For the purpose of
determining the time of the exercise of any Rights, delivery of any material to
the Agent shall be deemed to occur when such materials are received at the
Corporate Actions Division of the Agent specified in the Prospectus.

                                       2
<PAGE>

     (c) Notwithstanding the provisions of Section 4(a) and 4(b) regarding
delivery of an executed Subscription Certificate to the Agent prior to 5:00 P.M.
New York time on the Expiration Date, if prior to such time the Agent receives
payment in full for the Series A Preferred Shares purchased by the Stockholder
pursuant to his or her Basic Subscription Privilege and Over-Subscription
Privilege and a Notice of Guaranteed Delivery by facsimile (telecopy) or
otherwise from a member firm of a registered national securities exchange or a
member of the National Association of Securities Dealers, Inc., or a commercial
bank or trust company having a correspondent in the United States guaranteeing
delivery of a properly completed and executed Subscription Certificate, then
such exercise of Rights pursuant to the Basic Subscription Privilege and Over-
Subscription Privilege shall be regarded as timely, subject, however, to receipt
of the duly executed Subscription Certificate within three Business Days (as
defined below) after the Expiration Date.  For the purposes of the Prospectus
and this Agreement, "Business Day" shall mean any day on which trading is
conducted on the New York Stock Exchange.

     (d) As soon as practicable after the Expiration Date (the "Confirmation
Date") the Agent shall send to each exercising Stockholder (or, if shares of
Common Stock on the Record Date are held by Cede & Co. or any other depository
or nominee, to Cede & Co. or such other depository or nominee) a confirmation
showing the number of Series A Preferred Shares acquired pursuant to the Basic
Subscription Privilege, and, if applicable, the Over-Subscription Privilege, the
per share and total purchase price for such shares, and any excess to be
refunded by the Company to such Stockholder in the form of a check and stub,
along with a letter explaining the allocation of Series A Preferred Shares
pursuant to the Over-Subscription Privilege.

     (e) Any excess payment to be refunded by the Company to a Stockholder will
be mailed by the Agent within ten Business Days after the Confirmation Date.
The Agent will not issue or deliver certificates for shares subscribed for until
payment in full therefore has been received, including collection of checks.

6.  Validity and Form of Subscriptions.  All questions concerning the
timeliness, validity, form and eligibility of any exercise of Rights pursuant to
the Basic Subscription and Over-Subscription Privilege will be determined by the
Company, whose determination will be final and binding.  The Agent shall examine
the Subscription Certificates it receives to ascertain whether they appear to
have been completed and executed in accordance with the Prospectus and the
instructions included therein.  In the event that the Agent determines that the
Subscription Certificate does not appear to have been properly completed or
executed, or where the Subscription Certificates do not appear to be in proper
form for subscription, or any other irregularity in connection with the
subscription appears to exist, the Agent will follow its regular procedures to
attempt to cause such irregularity to be corrected.  The Agent is not authorized
to waive any irregularity in connection with the subscription, unless it has
received from the Company notification, duly dated and signed by an authorized
officer of the Company, indicating that any irregularity in the Subscription
Certificate has been cured or waived and that such Subscription Certificate has
been accepted by the Company.  Subscription Certificates containing
irregularities received by the Agent that are not properly waived

                                       3
<PAGE>

or cured, shall be returned by the Agent to the holder who submitted such
Subscription Certificate along with the funds accompanying such Subscription
Certificate. Such return shall be made by either first class mail under a
blanket surety bond or insurance protecting the Agent and the Company from
losses or liabilities arising out of the non-receipt or nondelivery of
Subscription Certificates or by registered mail insured separately for the value
of such Subscription Certificates, and shall include a letter of notice to be
furnished by the Agent explaining the reasons for the return of the Subscription
Certificates and other documents.

7.  Over-Subscription.  If, after allocation of Series A Preferred Shares to
Record Date Stockholders or transferees, there remain unexercised Rights, then
the Agent shall allot the shares issuable upon exercise of such unexercised
Rights (the "Remaining Shares") to Stockholders who have exercised all their
Rights pursuant to the Basic Subscription Privilege and who wish to acquire more
Series A Preferred Shares pursuant to the Over-Subscription Privilege.  Shares
subscribed for pursuant to the Over-Subscription Privilege will be allocated in
the amounts of such over-subscriptions.  If the number of shares for which the
Over-Subscription Privilege has been exercised is greater than the Remaining
Shares, the Agent shall allocate the Remaining Shares pro rata to Stockholders
exercising the Over-Subscription Privilege in proportion to the number of Series
A Preferred Shares the Stockholder purchased pursuant to the Basic Subscription
Privilege.  The percentage of Remaining Shares each over-subscribing Record Date
Stockholder or transferee may acquire shall be rounded up or down to the nearest
whole number.  The Agent shall advise the Company immediately upon the
completion of the allocation set forth above as to the total number of shares
subscribed and distributable.  A Record Date Stockholder or transferee who is
not allocated the full amount of shares that the holder subscribes for pursuant
to the Over-Subscription Privilege will receive a refund of the Subscription
Price paid by such holder for shares that are not allocated to and purchased by
such holder.  Such refund will be made by a check mailed by the Subscription
Agent.

8.  Standby Purchase.  The Agent understands that the Series A Preferred Shares
not sold by the Company through the exercise of rights in the Basic Subscription
or by means of the Over-Subscription Privilege will be purchased by the Standby
Purchaser pursuant to the terms of a standby purchase agreement between the
Company and the Standby Purchaser.  As soon as practicable after the Expiration
Date (but not later than 9:00 a.m., New York time, on the day following the
Expiration Date), the Agent shall advise the Company and the Standby Purchaser
in writing as to the number of Series A Preferred Shares unsubscribed for (and,
in the case of the exercise of Rights pursuant to the guaranteed delivery
procedures set forth in the Prospectus, such communication shall be confirmed to
the Company and the Standby Purchaser by 5:30 P.M., New York time, three New
York Stock Exchange trading days after the Expiration Date).

9.  Delivery of Certificates.  The Agent will deliver (i) certificates
representing those Series A Preferred Shares purchased pursuant to exercise of
the Basic Subscription Privilege as soon as practicable after the Expiration
Date and (ii) certificates representing those Series A Preferred Shares
purchased pursuant to exercise of the Over-Subscription Privilege as soon as
practicable after the Expiration Date.

                                       4
<PAGE>

10.  Holding Proceeds of Rights Offering.

     (a) All proceeds received by the Agent from Stockholders in respect of the
exercise of Rights shall be held by the Agent, on behalf of the Company, in a
segregated account, in the name of and for the benefit of the Company (the
"Account").  No interest shall accrue to stockholders on funds held in the
Account pending disbursement in the manner described in Section 4(e) above.

     (b) The Agent shall deliver all proceeds received in respect of the
exercise of Rights to the Company as promptly as practicable, but in no event
later than five business days after the Confirmation Date.

11.  Reports.  The Subscription Agent shall deliver daily to the Company a
written report showing the following:  (i) the number of Rights exercised
pursuant to the Basic Subscription Privilege on such day, and the aggregate
number of Rights exercised pursuant to the Basic Subscription Privilege through
such date; (ii) the amount of funds received on such day in payment of the
Subscription Price pursuant to the Basic Subscription Privilege, and the
aggregate amount of funds on deposit or invested for the account of the Company
from payment of the Subscription Price pursuant to the Basic Subscription
Privilege through such date; (iii) the number of Series A Preferred Shares
subscribed for on such day pursuant to the Over-Subscription Privilege, and the
aggregate number of Series A Preferred Shares subscribed for through such date
pursuant to the Over-Subscription Privilege; (iv) the aggregate amount of funds
received on such day in payment of the Subscription Price pursuant to the Over-
Subscription Privilege, and the aggregate amount of funds on deposit for the
account of the Company from payment of the Subscription Price through the Over-
Subscription Privilege through such date; and (v) the aggregate number of Series
A Preferred Shares that would be allocated to the Standby Purchaser if no
further subscriptions were to be received.

12.  Loss or Mutilation.  If any Subscription Certificate is lost, stolen,
mutilated or destroyed, the Agent may, on such terms which will indemnify and
protect the Company and the Agent as the Agent may in its discretion impose
(which shall, in the case of a mutilated Subscription Certificate include the
surrender and cancellation thereof), issue a new Subscription Certificate of
like denomination in substitution for the Subscription Certificate so lost,
stolen, mutilated or destroyed.

13.  Compensation for Services.  The Company agrees to pay to the Agent
compensation for its services as such in accordance with its Fee Schedule to act
as Agent attached hereto as Exhibit A.  The Company further agrees that it will
reimburse the Agent for its reasonable out-of-pocket expenses incurred in the
performance of its duties as such.

14.  Instructions and Indemnification.  The Agent undertakes the duties and
obligations imposed by this Agreement upon the following terms and conditions:

     (a) The Agent shall be entitled to rely upon any instructions or directions
furnished to it by an appropriate officer of the Company, whether in conformity
with the

                                       5
<PAGE>

provisions of this Agreement or constituting a modification hereof or a
supplement hereto.  Without limiting the generality of the foregoing or any
other provision of this Agreement, the Agent, in connection with its duties
hereunder, shall not be under any duty or obligation to inquire into the
validity or invalidity or authority or lack thereof of any instruction or
direction from an officer of the Company which conforms to the applicable
requirements of this Agreement and which the Agent reasonably believes to be
genuine and shall not be liable for any delays, errors or loss of data occurring
by reason of circumstances beyond the Agent's control.

     (b) The Company will indemnify the Agent and its employees and agents
against, and hold it harmless from, all liability and expense which may arise
out of or in connection with the services described in this Agreement or the
instructions or directions furnished to the Agent relating to this Agreement by
an appropriate officer of the Company, except for any liability or expense which
shall arise out of the gross negligence, bad faith or willful misconduct of the
Agent or such employees or agents.

15.  Changes in Subscription Certificate.  The Agent may, without the consent or
concurrence of the Stockholders in whose names Subscription Certificates are
registered, by supplemental agreement or otherwise, concur with the Company in
making any changes or corrections in a Subscription Certificate that it shall
have been advised by counsel (who may be counsel for the Company) is appropriate
to cure any ambiguity or to correct any defective or inconsistent provision or
clerical omission or mistake or manifest error therein or herein contained, and
which shall not be inconsistent with the provision of the Subscription
Certificate except insofar as any such change may confer additional rights upon
the Stockholders.

16.  Assignment, Delegation.

     (a) Except as provided in Section 15(c) below, neither this Agreement nor
any rights or obligations hereunder may be assigned or delegated by either party
without the written consent of the other party.

     (b) This Agreement shall inure to the benefit of and be binding upon the
parties and their respective permitted successors and assigns.  Nothing in this
Agreement is intended or shall be construed to confer upon any other person any
right, remedy or claim or to impose upon any other person any duty, liability or
obligation.

     (c) The Agent may, without further consent on the part of the Company, (i)
subcontract for the performance hereof with EquiServe Limited Partnership or
(ii) subcontract with other subcontractors for systems, processing, and
telephone and mailing services as may be required from time to time; provided,
however, that the Agent shall be as fully responsible to the Company for the
acts and omissions of any subcontractor as it is for its own acts and omissions.

17.  Governing Law.  The validity, interpretation and performance of this
Agreement shall be governed by the law of the Commonwealth of Massachusetts.

                                       6
<PAGE>

18.  Third Party Beneficiaries.  This Agreement does not constitute an agreement
for a partnership or joint venture between the Agent and the Company.  Neither
party shall make any commitments with third parties that are binding on the
other party without the other party's prior written consent.

19.  Force Majeure.  In the event either party is unable to perform its
obligations under the terms of this Agreement because of acts of God, strikes,
equipment or transmission failure or damage reasonably beyond its control, or
other cause reasonably beyond its control, such party shall not be liable for
damages to the other for any damages resulting from such failure to perform or
otherwise from such causes.  Performance under this Agreement shall resume when
the affected party or parties are able to perform substantially that party's
duties.

20.  Consequential Damages.  Neither party to this Agreement shall be liable to
the other party for any consequential, indirect, special or incidental damages
under any provisions of this Agreement or for any consequential, indirect,
special or incidential damages arising out of any act or failure to act
hereunder even if that party has been advised of or has foreseen the possibility
of such damages.

21.  Severability.  If any provision of this Agreement shall be held invalid,
unlawful, or unenforceable, the validity, legality, and enforceability of the
remaining provisions shall not in any way be affected or impaired.

22.  Counterparts.  This Agreement may be executed in one or more counterparts,
each of which shall be deemed an original and all of which together shall be
considered one and the same agreement.

23.  Captions.  The captions and descriptive headings herein are for the
convenience of the parties only.  They do not in any way modify, amplify or
alter the provisions hereof.

24.  Facsimile Signatures.  Any facsimile signature of any party hereto shall
constitute a legal, valid and binding execution hereof by such party.

25.  Confidentiality.  The Agent and the Company agree that all books, records,
information and data pertaining to the business of the other party which are
exchanged or received pursuant to the negotiation or the carrying out of this
Agreement including the fees for services set forth in the attached Exhibit A
shall remain confidential, and shall not be voluntarily disclosed to any other
person, except as may be required by law; provided, however, that this Agreement
and the Exhibits hereto may be filed with the Securities and Exchange Commission
as an exhibit to the Registration Statement.

26.  Term.  This Agreement shall remain in effect until terminated on September
30, 2001 (the "Termination Date") or, prior to the Termination Date, upon 30
days' written notice by either party to the other.  Upon termination of the
Agreement, the Agent shall retain all canceled Certificates and related
documentation as required by applicable law.

                                       7
<PAGE>

27.  Merger of Agreement.  This Agreement constitutes the entire agreement
between the parties hereto and supersedes any prior agreement with respect to
the subject matter hereof whether oral or written.

                                       8
<PAGE>

     IN WITNESS WHEREOF, the parties hereto have caused this Agreement to be
executed by their respective officers, hereunto duly authorized, as of the day
and year first above written.


EQUISERVE TRUST COMPANY, N.A.        CHART HOUSE ENTERPRISES, INC.



/s/ Laurie Trickett                  /s/ Kenneth R. Posner
-----------------------------        -----------------------------------
Signature                            Signature

Director Corporate Actions           President
-----------------------------        -----------------------------------
Title                                Title

5/10/01                              5/10/01
-----------------------------        -----------------------------------
Date                                 Date


                                       9