<PAGE>   1
                                                                 Exhibit 10.41


                            EXCHANGE RIGHTS AGREEMENT

               This Exchange Rights Agreement (this "Agreement") is made as of
March 11, 1997 among Starwood Lodging Trust, a real estate investment trust
organized under the laws of the State of Maryland (the "Trust"), Starwood
Lodging Corporation, a Maryland corporation (the "Corporation"), SLT Realty
Limited Partnership, a Delaware limited partnership (the "Realty Partnership"),
SLC Operating Limited Partnership, a Delaware limited partnership (the
"Operating Partnership"), and The Hermitage, L.P., a Tennessee limited
partnership (the "Hermitage Partner"). Unless otherwise indicated, capitalized
terms used herein are used herein as defined in Section 10.

               WHEREAS, pursuant to an Agreement of Purchase and Sale dated as
of January 16, 1997 (the "Purchase Agreement") among the Realty Partnership,
Operating Partnership and Hermitage Partner, (i) the Realty Partnership and
Operating Partnership are acquiring from the Hermitage Partner a hotel, and (ii)
a portion of the consideration set forth in Section 2.02 of the Purchase
Agreement may, at the election of the Hermitage Partner, be paid in Units of the
Realty Partnership (as such Units are defined in the Limited Partnership
Agreement of the Realty Partnership ("Realty Units")) and Units of the Operating
Partnership (as such Units are defined in the Limited Partnership Agreement of
the Operating Partnership ("Operating Units")); and

               WHEREAS, pursuant to the Purchase Agreement the parties hereto
are entering into this Agreement to provide for the rights of the Hermitage
Partner to tender Realty Units and Operating Units in exchange for either Paired
Shares (as defined herein), cash or a combination of Paired Shares and cash, on
the terms and conditions set forth herein;

               NOW, THEREFORE, in consideration of the premises and the mutual
covenants set forth herein, the parties hereto agree as follows:

               SECTION 1. RIGHT TO TENDER STARWOOD UNITS. (a) Upon the terms and
subject to the conditions of this Agreement, each holder of Starwood Units (as
defined below) shall have the right to tender to the Trust outstanding Realty
Units and the right to tender to the Corporation outstanding Operating Units.
Notwithstanding anything to the contrary contained in this Agreement (i) no
Realty Unit may be tendered to the Trust unless simultaneously therewith the
tendering holder also tenders to the Corporation an Operating Unit and no
Operating Unit may be tendered to the Corporation unless simultaneously
therewith the tendering holder also tenders to the Trust a Realty Unit (a Realty
Unit tendered for exchange and the Operating Unit


<PAGE>   2
simultaneously tendered for exchange being hereinafter collectively referred to
as a "Starwood Unit") and (ii) any attempted tender of a Realty Unit or an
Operating Unit which is not accompanied by a simultaneous tender of an Operating
Unit or Realty Unit, respectively, shall be void and of no effect; it being
understood that a simultaneous tender of unequal numbers of Realty Units and
Operating Units shall be valid under this sentence to the extent of the lesser
of the number of Realty Units or Operating Units, as the case may be, included
in such tender.

               (b) Notwithstanding any other provision of this Agreement, no
Paired Shares or cash shall be issued or paid in respect of any tender of
Starwood Units (i) if the right to tender Starwood Units and receive Paired
Shares or cash would result in the Trust not satisfying the REIT Requirements in
any respect or would result in any person or entity Beneficially Owning Trust
Shares exceeding the Ownership Limit, (ii) prior to the expiration or
termination of the waiting period applicable to such exchange and issuance, if
any, under the Hart-Scott-Rodino Antitrust Improvements Act of 1976, as it may
be amended from time to time, or (iii) prior to the receipt of all governmental
and regulatory approvals which are required to be obtained prior to such tender
and issuance or payment, including, without limitation, any required approvals
of the gaming authorities of the State of Nevada and of Clark County, Nevada
(the "Gaming Approvals"). Prior to the receipt of Gaming Approvals, such holder
shall, as a condition to any tender of Starwood Units which would (if the Paired
Share Option (as defined below) were to be elected in respect of such tender)
cause such holder to beneficially own, in the aggregate, Paired Shares
representing more than 4.9% of the then issued and outstanding Paired Shares,
give not less than 90 days' written notice to the Trust and the Corporation (at
the offices provided pursuant to Section 9) of its intent to tender Starwood
Units. In the event that the ability to receive Paired Shares or cash would
result in the Trust not satisfying the REIT Requirements in any respect or would
result in any person or entity Beneficially Owning Trust Shares exceeding the
Ownership Limit, and as a result thereof no Paired Shares or cash may be issued
or paid in respect of any tender of Starwood Units pursuant to Section 1(b)(i)
above, the parties hereto shall use their respective best efforts to restructure
the terms and provisions of this Agreement (and, if necessary, the Partnership
Agreements), or to agree to terms and provisions in addition to such terms and
provisions, so as to provide to each such party the same substantive rights (or
substantive rights as close thereto as is reasonably practicable) as those
provided by this Agreement, the Partnership Agreements and the Registration
Rights Agreement.


               (c) The rights to exchange Starwood Units pursuant to


                                      -2-
<PAGE>   3
this Agreement constitute a continuous offer and may not be withdrawn, amended
or modified by the Trust or the Corporation without the prior written consent of
each holder of outstanding Starwood Units adversely affected by such withdrawal,
amendment or modification; provided that any withdrawal, amendment or
modification that does not adversely affect any holder of outstanding Starwood
Units may be effected without the consent of such holder.

               SECTION 2. ACCEPTANCE OF TENDER; ELECTION OF METHOD OF PAYMENT
FOR TENDERED STARWOOD UNITS. (a) Upon the terms and subject to the conditions of
this Agreement, the Trust and the Corporation shall accept Starwood Units
validly tendered in proper form and meeting all of the requirements of this
Agreement. In order for Starwood Units to be validly tendered pursuant to this
Agreement, the registered holder thereof shall deliver to the Trust and the
Corporation, at the address provided pursuant to Section 9, (i) a completed and
duly executed Letter of Transmittal in the form attached hereto as Exhibit A
(the "Letter of Transmittal") and any other documents required by the Letter of
Transmittal and (ii) a calculation, to the best knowledge of such registered
holder after due inquiry (together with such supporting documentation as the
Trust may reasonably request), of the maximum number of Paired Shares that may
be issued to such registered holder without causing either (x) the Trust to not
satisfy the REIT Requirements in any respect or (y) any person or entity to
Beneficially Own Trust Shares exceeding the Ownership Limit. The Trust and the
Corporation shall make all determinations as to the validity and form of any
tender of Starwood Units in accordance with the provisions of this Agreement and
upon rejection of a tender shall give the tendering holder written notice of
such rejection, which shall include the reasons therefor.

               (b) Unless otherwise determined by agreement of the Trust and the
Corporation, tenders of Starwood Units pursuant to this Agreement shall be
irrevocable and shall not be subject to withdrawal or modification; provided
that if the Trust and the Corporation make the Paired Share Election, as defined
below, with respect to a tender, then within 5 days after such Election the
tendering holder may elect to revoke such tender so long as (i) no public
disclosure of such tender has been made prior to such revocation and (ii) such
tendering holder reimburses the Trust and the Corporation for all reasonable
costs and expenses incurred in connection with such tender.

               (c) Within 15 days after the valid tender pursuant to this
Agreement of Starwood Units, the Trust and the Corporation shall make an
election to pay for such Starwood Units by delivering either (i) Paired Shares
(the "Paired Share Election"), (ii) cash (the "Cash Election") or (iii) a


                                       -3-
<PAGE>   4
combination of Paired Shares and cash (the "Combined Election"). Such election
shall be made pursuant to an agreement as to such election between the Trust and
the Corporation. If the Trust and the Corporation do not so agree within such
15-day period, they shall be deemed to have made the Cash Election.

               SECTION 3. PAIRED SHARE ELECTION. (a) If with respect to any
tender of Starwood Units pursuant to this Agreement, the Trust and the
Corporation make the Paired Share Election, then, except as provided in Section
2(b), within fifteen days after the expiration of the 5-day period referred to
in Section 2(b), the Trust and the Corporation shall deliver to the tendering
holder one Paired Share for each Starwood Unit validly tendered pursuant to the
provisions of this Agreement.

               (b) No fractional Paired Shares or scrip representing fractional
Paired Shares shall be issued upon exchange of Starwood Units pursuant to this
Agreement. If more than one Letter of Transmittal shall be delivered at one time
by the same holder, the number of full Paired Shares which shall be issuable
upon exchange of the Starwood Units tendered thereby shall be computed on the
basis of the aggregate number of Starwood Units so tendered. Instead of any
fractional Paired Shares which would otherwise be issuable upon exchange of any
Starwood Units, the Trust and the Corporation shall pay a cash adjustment in
respect of such fraction in an amount equal to the same fraction of the Paired
Share Closing Price on the last Business Day preceding the date of exchange.

               (c) If a holder exchanges Starwood Units pursuant to this
Agreement, the Trust and the Corporation shall pay any documentary, stamp or
similar issue or transfer tax due on any issue of Paired Shares upon such
exchange. Such holder, however, shall (i) pay to the Trust and the Corporation
the amount of any additional documentary, stamp or similar issue or transfer tax
which is due (or shall establish to the satisfaction of the Trust and the
Corporation the payment thereof) as a result of Paired Shares being issued in a
name other than the name of such holder and (ii) be responsible for all income
or other taxes as a result of such exchange.

               SECTION 4. CASH ELECTION. (a) If with respect to any tender of
Starwood Units pursuant to this Agreement, the Trust and the Corporation make or
are deemed to have made the Cash Election, then within 20 days after such tender
the Trust and the Corporation shall pay to the tendering holder an aggregate
amount of cash (the "Aggregate Cash Payment") equal to the product of (i) the
number of Paired Shares which would have been delivered to such holder if the
Trust and the Corporation had made the Paired Share Election with respect to
such tender and (ii) the average Paired Share Closing Price for the ten trading
day period


                                       -4-
<PAGE>   5
ending one day prior to the date of such tender.

               (b) In connection with any Aggregate Cash Payment pursuant to
Section 4(a) or any cash payment pursuant to Section 5(a)(ii), the Trust shall
pay 95% of such Aggregate Cash Payment or such cash payment and the Corporation
shall pay 5% of such Aggregate Cash Payment or such cash payment (such
percentages being herein called the "Issuance Percentages"); provided that the
Trust and the Corporation may from time to time change the Issuance Percentages
based on their determination of the relative fair values of the Trust Shares and
the Corporation Shares.

               SECTION 5. COMBINED ELECTION. (a) If with respect to any tender
of Units pursuant to this Agreement, the Trust and the Corporation shall make
the Combined Election, then, except as provided in Section 2(b), within 15 days
after the expiration of the 5-day period referred to in Section 2(b), the Trust
and the Corporation shall (i) notify the tendering holder of the number of such
tendered Units which will be exchanged for cash (the "Cash Units") and the
number of such tendered Units which will be exchanged for Paired Shares (the
"Paired Share Units"), (ii) pay to the tendering holder, in respect of each Cash
Unit validly tendered pursuant to the provisions of this Agreement, an amount of
cash (with each of the Trust and the Corporation paying its then respective
Issuance Percentage of such amount of cash) equal to the average Paired Share
Closing Price for the ten trading day period ending one day prior to the date of
such tender and (iii) deliver to the tendering holder one Paired Share for each
Paired Share Unit validly tendered pursuant to the provisions of this Agreement.

               (b) The provisions of Sections 3(b) and 3(c) of this Agreement
shall apply to the issuance of Paired Shares pursuant to Section 5(a).

               SECTION 6. REPRESENTATIONS OF TENDERING HOLDER. Each tender of
Starwood Units shall constitute a representation and warranty by the tendering
holder of each of the representations and warranties set forth in the form of
Letter of Transmittal. Without limiting the generality of the foregoing, unless,
at the time of a tender for exchange of Starwood Units pursuant to this
Agreement, a registration statement relating to the delivery of any Paired
Shares upon such tender is effective under the Securities Act of 1933, as
amended (the "Securities Act"), such tender shall constitute a representation
and warranty by the tendering holder to the Trust and the Corporation that such
tendering holder (i) is an "accredited investor" within the meaning of Rule 501
under the Securities Act, (ii) has sufficient knowledge and experience in
financial and business matters and in investing in entities similar to the
Partnerships, the Trust and the Corporation so as to be able to evaluate the
risks and merits


                                      -5-
<PAGE>   6
of its investment in the Partnerships, the Trust and the Corporation and it is
able financially to bear the risks thereof, (iii) has had an opportunity to
discuss the business, management and financial affairs of the Trust, the
Corporation and the Partnerships with the management of the Trust, the
Corporation and the Partnerships, and (iv) understands that the Paired Shares
have not been registered under the Securities Act by reason of their issuance in
a transaction exempt from the registration requirements of the Securities Act
pursuant to Section 4(2) thereof or Rule 505 or 506 promulgated under the
Securities Act and such Paired Shares must be held indefinitely unless a
subsequent disposition thereof is registered under the Securities Act and
applicable state securities laws or is exempt from such registration.

               SECTION 7. STATUS OF TENDERING HOLDER. Until the holder of
Starwood Units tendered pursuant to this Agreement becomes a holder of record of
the Paired Shares issued in exchange therefor (in the case of a Paired Share
Election or a Combined Election) or until such holder has received cash in
exchange therefor (in the case of a Cash Election or a Combined Election), such
holder shall continue to hold and own such Starwood Units for all purposes of
the Realty Partnership Agreement and the Operating Partnership Agreement. In the
case of a Paired Share Election or a Combined Election, no such holder shall
have any rights as a shareholder of the Trust or a stockholder of the
Corporation in respect of such Paired Shares until such holder becomes a holder
of record of such Paired Shares.

               SECTION 8. RESERVATION OF SHARES; CLOSING OF TRANSFER BOOKS. (a)
The Trust shall reserve and shall at all times have reserved out of its
authorized but unissued Trust Shares, solely for the purpose of effecting the
exchange of Realty Units pursuant to this Agreement, enough Trust Shares to
permit the exchange of the then outstanding Realty Units. The Corporation shall
reserve and shall at all times have reserved out of its authorized but unissued
Corporation Shares, solely for the purpose of effecting the exchange of
Operating Units pursuant to this Agreement, enough Corporation Shares to permit
the exchange of the then outstanding Operating Units. All Paired Shares which
may be issued upon exchange of Starwood Units shall be validly issued, fully
paid and nonassessable and free from all taxes, liens and charges with respect
to the issuance thereof other than income taxes resulting from such exchange.

               (b) The Trust shall not close its transfer books so as to prevent
the timely issuance of Trust Shares pursuant to this Agreement. The Corporation
shall not close its transfer books so as to prevent the timely issuance of
Corporation Shares pursuant to this Agreement.



                                      -6-
<PAGE>   7
               SECTION 9. NOTICES. All notices and requests given pursuant to
this Agreement shall be in writing and shall be made by hand-delivery,
first-class mail (registered or certified, return receipt requested), confirmed
facsimile or overnight air courier guaranteeing next Business Day delivery to
the relevant address specified below. Except as otherwise provided in this
Agreement, the date of each such notice and request shall be deemed to be, and
the date on which each such notice and request shall be deemed given shall be:
at the time delivered, if personally delivered or mailed; when receipt is
acknowledged, if sent by facsimile; and the next Business Day after timely
delivery to the courier, if sent by overnight air courier guaranteeing next
Business Day delivery. All notices and requests shall be given at the following
address (or to such other address as such party may have specified by notice
given to the other parties pursuant to this provision):

        (a) If to the Trust or the Realty Partnership, to:

               Starwood Lodging Trust
               2231 E. Camelback Road, Suite 410
               Phoenix, AZ 85016
               Attention:  Chief Financial Officer
               Telecopy No.:  (602) 852-0984
               Telephone No.:  (602) 852-3900


        with a copy to:

               Sidley & Austin
               555 West 5th Street
               Los Angeles, California  90013
               Attention:  Sherwin L. Samuels
               Telecopy No.:  (213) 896-6600
               Telephone No.:(213) 896-6000

        (b) If to the Corporation or the Operating Partnership, to:

               Starwood Lodging Corporation
               2231 E. Camelback Road, Suite 400
               Phoenix, AZ 85016
               Attention:  General Counsel
               Telecopy No.:  (602) 852-0686
               Telephone No.:  (602) 852-3900


        with a copy to:

               Sidley & Austin
               555 West 5th Street



                                      -7-
<PAGE>   8
               Los Angeles, California  90013
               Attention:  Sherwin L. Samuels
               Telecopy No.:  (213) 896-6600
               Telephone No.:(213) 896-6000


        (c) If to Hermitage Partner:

               The Hermitage, L.P.
               c/o Hermitage of Nashville, Inc.
               1407 Union Avenue, Suite 400
               Memphis, TN 38014
               Attention:  Pace Cooper, President
               Telecopy No.:  (901) 274-9169
               Telephone No.:  (901) 725-9631


        with a copy to:

               Harkavy, Shainberg, Kosten & Kaplan
               530 Oak Court Drive, Suite 350
               Memphis, TN  38117
               Attention:  Raymond M. Shainberg
               Telecopy No.:  (901) 763-3340
               Telephone No.:  (901) 761-1263

               SECTION 10.  DEFINITIONS.  For purposes of this
Agreement:

               "Beneficially Owning" means owning Trust Shares directly,
        indirectly or constructively by a person or entity through the
        application of Section 318(a) of the Code, as modified by Section
        856(d)(5) of the Code, or Section 544 of the Code, as modified by
        Section 856(h) of the Code. The term "Beneficially Own" shall have a
        correlative meaning.

               "Business Day" means any day other than Saturday, Sunday and any
        day on which banks are not open to do business in Phoenix, Arizona.

               "Code" means the Internal Revenue Code of 1986, and the rules and
        regulations promulgated thereunder, as amended from time to time.

               "Corporation Shares" means the shares of common stock, par value
        $.01 per share, of the Corporation.

               "Declaration of Trust" means the Declaration of Trust of the
        Trust dated August 25, 1969, as amended and restated as of June 6, 1988,
        and as further amended on February 1, 1995 and as amended from time to
        time after the date of this Agreement.



                                       -8-
<PAGE>   9
              "Disinterested Members" when used with respect to the Trust has
        the meaning set forth in the Code of Regulations of the Trust and, when
        used with respect to the Corporation, has the meaning set forth in the
        By-Laws of the Corporation, in each case as amended from time to time.

               "Operating Partnership" means the Limited Partnership
        Agreement of the Operating Partnership.

               "Ownership Limit" when used with respect to Trust Shares, has the
        meaning set forth in the Declaration of Trust and, when used with
        respect to the Corporation Shares, has the meaning set forth in the
        Restated Articles, in each case as amended from time to time.

               "Paired Share" means a Corporation Share and a Trust Share which
        are paired pursuant to the Pairing Agreement.

               "Paired Share Closing Price" shall mean, with respect to a
        particular date, the last reported sales price regular way on such date
        or, in case no such reported sale takes place on such date, the average
        of the reported closing bid and asked prices regular way on such date,
        in either case on the New York Stock Exchange, or if the Paired Shares
        are not then listed or admitted to trading on such Exchange, on the
        principal national securities exchange on which the Paired Shares are
        then listed or admitted to trading or, if not then listed or admitted to
        trading on any national securities exchange, the closing sale price on
        such date of the Paired Shares or, in case no reported sale takes place
        on such date then, the average of the closing bid and asked prices on
        such date, on NASDAQ or any comparable system. If the Paired Shares are
        not then quoted on NASDAQ or any comparable system, the Board of
        Trustees of the Trust and the Board of Directors of the Corporation
        shall in good faith determine the Paired Share Closing Price.

               "Pairing Agreement" means the Pairing Agreement dated June 25,
        1980 between the Trust and the Corporation, as it may be amended from
        time to time.

               "Partnerships" mean the Realty Partnership Agreement
        and the Operating Partnership Agreement.

               "Realty Partnership" means the Limited Partnership
        Agreement of the Realty Partnership.

               "Registration Rights Agreement" means the Registration Rights
        Agreement dated as of the date first written above between the Trust,
        the Corporation and the Hermitage Partner.




                                      -9-
<PAGE>   10
              "REIT Requirements" shall mean the requirements for the Trust to
         (i) qualify as a REIT under the Code and the rules and regulations
         promulgated thereunder, (ii) avoid any federal income or excise tax
         liability, (iii) retain its status as grandfathered pursuant to Section
         136(c)(3) of the Deficit Reduction Act of 1984 and (iv) retain the
         benefits of that certain private letter ruling issued by the Internal
         Revenue Service to the Trust dated as of January 4, 1980.

              "Restated Articles" means the Restated Articles of Incorporation
         of the Corporation, as amended from time to time after the date of this
         Agreement.

              "Trust Shares" means the shares of beneficial interest, $.01 par
         value, of the Trust.

               SECTION 11. DETERMINATIONS AND INTERPRETATION. All agreements
between the Trust and the Corporation provided for in this Agreement shall be
made on behalf of the Trust and the Corporation by their respective
Disinterested Members, including, without limitation, any agreement between the
Trust and the Corporation as to the election of the Paired Share Election, the
Cash Election or the Combined Election with respect to a tender of Starwood
Units pursuant to Section 2(c), any agreement to permit the revocation,
withdrawal or modification of a tender of Starwood Units pursuant to Section
1(c) and any adjustment of the Issuance Percentages pursuant to Section 4(b).
All interpretations of the terms of this Agreement shall be resolved on behalf
of the Trust and the Corporation by their respective Disinterested Members.

               SECTION 12. PARTIAL INVALIDITY. In case any one or more of the
provisions contained herein shall, for any reason, be held to be invalid,
illegal or unenforceable in any respect, such invalidity, illegality or
unenforceability shall not affect any other provisions of this Agreement, but
this Agreement shall be construed as if such invalid, illegal or unenforceable
provision or provisions had never been contained herein unless the deletion of
such provision or provisions would result in such a material change as to cause
completion of the transactions contemplated hereby to be unreasonable.

               SECTION 13. SUCCESSORS AND ASSIGNS. This Agreement shall be
binding upon and inure to the benefit of the parties hereto and their respective
permitted successors or assigns. In addition, and whether or not any express
assignment shall have been made, the provisions of this Agreement which are for
the benefit of the parties hereto other than the Corporation, the Trust, the
Realty Partnership and the Operating Partnership, shall also be for the benefit
of and enforceable by any


                                      -10-
<PAGE>   11
subsequent holder of any Units.

               SECTION 14. EXECUTION IN COUNTERPARTS. This Agreement may be
executed in one or more counterparts, each of which shall be considered an
original counterpart, and shall become a binding agreement when the Trust, the
Corporation, the Realty Partnership, the Operating Partnership and the Hermitage
Partner shall have each executed a counterpart of this Agreement.

               SECTION 15. TITLES AND HEADINGS. Titles and headings to Articles
and Sections herein are inserted for convenience of reference only and are not
intended to be a part of or to affect the meaning or interpretation of this
Agreement.

               SECTION 16. EXHIBITS. The Exhibits referred to in this Agreement
shall be construed with, and as an integral part of, this Agreement to the same
extent as if the same had been set forth verbatim herein.

               SECTION 17. ENTIRE AGREEMENT; AMENDMENTS AND WAIVERS. This
Agreement, including the Exhibits, contains the entire understanding of the
parties hereto with regard to the subject matter contained herein. In addition
to amendments and modifications permitted by Section 1(c), the parties hereto,
by mutual agreement in writing, may amend, modify and supplement this Agreement;
provided that any such amendment, modification or supplement shall be approved
by a majority of the Disinterested Members of each of the Trust and the
Corporation. The failure of any party hereto to enforce at any time any
provision of this Agreement shall not be construed to be a waiver of such
provision, nor in any way to affect the validity of this Agreement or any part
hereof or the right of such party thereafter to enforce each and every such
provision. No waiver of any breach of this Agreement shall be held to constitute
a waiver of any other or subsequent breach.

               SECTION 18. GOVERNING LAW. Except to the extent that Maryland law
is mandatorily applicable to the rights and obligations of the shareholders of
the Trust and the stockholders of the Corporation, this Agreement, and the
application or interpretation thereof, shall be governed exclusively by its
terms and by the internal laws of the State of Delaware, without regard to
principles of conflicts of laws as applied in the State of Delaware or any other
jurisdiction which, if applied, would result in the application of any laws
other than the internal laws of the State of Delaware.

               SECTION 19. STARWOOD LODGING TRUST. The parties hereto understand
and agree that the name "Starwood Lodging Trust" is a designation of the Trust
and its Trustees (as Trustees but not personally) under the Declaration of
Trust, and


                                      -11-
<PAGE>   12
all persons dealing with the Trust shall look solely to the Trust's assets for
the enforcement of any claims against the Trust, and that the Trustees,
officers, agents and security holders of the Trust assume no personal liability
for obligations entered into on behalf of the Trust, and their respective
individual assets shall not be subject to the claims of any person relating to
such obligations.

               SECTION 20. SUBMISSION TO JURISDICTION. Each of the parties
hereto irrevocably submits and consents to the jurisdiction of the United States
District Court for the Southern District of New York and the United States
District Court for the District of Maryland in connection with any action or
proceeding arising out of or relating to this Agreement, and irrevocably waives
any immunity from jurisdiction thereof and any claim of improper venue, forum
non conveniens or any similar basis to which it might otherwise be entitled in
any such action or proceeding.

               IN WITNESS WHEREOF, this Agreement has been duly executed and
delivered by the parties hereto or by their duly authorized officers, all as of
the date first above written.


                             STARWOOD LODGING TRUST,
                             a Maryland real estate investment
                               trust


                             By: /s/ Steven R. Goldman
                                ----------------------------
                                Name:  Steven R. Goldman
                                Title: Senior Vice President



                             STARWOOD LODGING CORPORATION,
                              a Maryland corporation


                             By: /s/ Joseph Long
                                ----------------------------
                                Name:  Joseph Long
                                Title: Vice President



                             SLT REALTY LIMITED PARTNERSHIP,

                             By: STARWOOD LODGING TRUST,
                                 general partner



                                      -12-
<PAGE>   13
                             By: /s/ Steven R. Goldman
                                --------------------------------
                                 Name: Steven R. Goldman
                                 Title: Senior Vice President


                             SLC OPERATING LIMITED PARTNERSHIP

                             By: STARWOOD LODGING CORPORATION,
                                 general partner


                                 By: /s/ Joseph Long
                                   --------------------------------
                                     Name: Joseph Long
                                     Title: Vice President




                              THE HERMITAGE, L.P.

                              By:    Hermitage of Nashville, Inc.,
                                     its General Partner

                                  By: /s/ Pace Cooper
                                     --------------------------------

                                       Pace Cooper
                                       President



                                      -13-
<PAGE>   14
                                                                    EXHIBIT A TO
                                                       Exchange Rights Agreement

                              LETTER OF TRANSMITTAL

                                 To Tender Units


                    Pursuant to the Exchange Rights Agreement
                            Dated as of March , 1997


TO:  Starwood Lodging Trust
        2231 E. Camelback Road, Suite 410
        Phoenix, AZ 85016

        Starwood Lodging Corporation
        2231 E. Camelback Road, Suite 400
        Phoenix, AZ 85016


                              DESCRIPTION OF UNITS


--------------------------------------------------------------------------------
        NAME(S) AND ADDRESS (ES)    UNITS TENDERED (ATTACH
        OF REGISTERED OWNERS        ADDITIONAL LIST IF
                                    NECESSARY)





Realty
Units:



Operating
Units:



                     --------------------------------------
                                 Total


<PAGE>   15
                     NOTE: SIGNATURES MUST BE PROVIDED BELOW
               PLEASE READ THE ACCOMPANYING INSTRUCTIONS CAREFULLY


Ladies and Gentlemen:

               The undersigned hereby tenders to Starwood Lodging Trust (the
"Trust") the above-described Realty Units (as defined in the Exchange Rights
Agreement dated as of March   , 1997 (the "Exchange Rights Agreement")) and
hereby tenders to Starwood Lodging Corporation (the "Corporation") the
above-described Operating Units (as defined in the Exchange Rights Agreement)
in accordance with the terms and conditions of the Exchange Rights Agreement
and this Letter of Transmittal (which together constitute the "Offer"), receipt
of which is hereby acknowledged. All terms used herein but not defined herein
are used as defined in the Exchange Rights Agreement.

               Subject to, and effective upon the issuance of Paired Shares
and/or the payment of cash, as the case may be, for the Starwood Units tendered
hereby, the undersigned hereby assigns and transfers (i) to the Trust all right,
title and interest in and to all the Realty Units that are being tendered hereby
and irrevocably constitutes and appoints the Trust (the "Realty Unit Agent"),
with full power of substitution (such power of attorney being deemed to be an
irrevocable power coupled with an interest), to (a) transfer such Realty Units
on the books of the Realty Partnership and (b) receive all rights, privileges
and benefits, and any and all obligations and liabilities appertaining thereto
and otherwise exercise all rights of beneficial ownership of such Realty Units,
all in accordance with the terms of the Offer and (ii) to the Corporation all
right, title and interest in and to all the Operating Units that are being
tendered hereby and irrevocably constitutes and appoints the Corporation (the
"Operating Unit Agent" and, together with the Realty Unit Agent, the "Agents"),
with full power of substitution (such power of attorney being deemed to be an
irrevocable power coupled with an interest), to (a) transfer such Operating
Units on the books of the Operating Partnership and (b) receive all rights,
privileges and benefits, and any and all obligations and liabilities
appertaining thereto and otherwise exercise all rights of beneficial ownership
of such Operating Units, all in accordance with the terms of the Offer.

               The undersigned hereby represents and warrants (i) to the Trust
that the undersigned has full power and authority to tender, sell, assign and
transfer the tendered Realty Units and that upon payment therefor, the Trust
will acquire unencumbered title thereto, free and clear of all liens,
restrictions, charges and encumbrances and the same will not be subject to any
adverse claim, (ii) to the Corporation that the undersigned has full power and
authority to tender, sell, assign and transfer the



                                      -15-
<PAGE>   16
tendered Operating Units and that upon payment therefor, the Corporation will
acquire unencumbered title thereto, free and clear of all liens, restrictions,
charges and encumbrances and the same will not be subject to any adverse claim,
(iii) to the Trust and the Corporation that the tender complies with each and
every provision of Section 1 of the Exchange Rights Agreement, and (iv) attached
hereto is a calculation, to the best knowledge of the undersigned after due
inquiry (together with such supporting documentation as the Trust may reasonably
request) of the maximum number of Paired Shares that may be issued to the
undersigned without causing either (x) the Trust to not satisfy the REIT
Requirements in any respect or (y) any person or entity to Beneficially Own
Trust Shares exceeding the Ownership Limit. The undersigned will, upon request,
execute any additional documents deemed by the Trust or the Corporation to be
reasonably necessary or desirable to complete the sale, assignment and transfer
of the tendered Starwood Units.

               Unless a registration statement relating to any Paired Shares to
be delivered to the undersigned is effective under the Securities Act of 1933,
as amended (the "Securities Act"), the undersigned hereby represents and
warrants to the Trust and the Corporation that the undersigned (A) is an
"accredited investor" within the meaning of Rule 501 under the Securities Act,
or (B) has sufficient knowledge and experience in financial and business matters
and in investing in entities similar to the Partnerships, the Trust and the
Corporation so as to be able to evaluate the risks and merits of its investment
in the Partnerships, the Trust and the Corporation and it is able financially to
bear the risks thereof, and in either case, (i) has had an opportunity to
discuss the business, management and financial affairs of the Trust, the
Corporation and the Partnerships with the management of the Trust, the
Corporation and the Partnerships, and (ii) understands that any such Paired
Shares have not been registered under the Securities Act by reason of their
issuance in a transaction exempt from the registration requirements of the
Securities Act pursuant to Section 4(2) thereof or Rule 505 or 506 promulgated
under the Securities Act and any such Paired Shares must be held indefinitely
unless a subsequent disposition thereof is registered under the Securities Act
and applicable state securities laws or is exempt from such registration. If not
sold pursuant to an effective registration statement, any such Paired Shares
will bear an appropriate legend indicating that such Paired Shares have not been
registered under the Securities Act and resale of such Paired Shares is
restricted under applicable securities laws.

               All authority conferred or agreed to be conferred in this Letter
of Transmittal shall not be affected by, and shall survive, the death or
incapacity of the undersigned, and any obligation of the undersigned hereunder
shall be binding upon the



                                      -16-
<PAGE>   17
successors, assigns, heirs, executors, administrators and legal representatives
of the undersigned.

               The undersigned understands that, except as provided in Section
2(b) of the Exchange Rights Agreement, a tender of Starwood Units pursuant to
the Exchange Rights Agreement is irrevocable and constitutes a binding agreement
between the undersigned and the Trust and the Corporation upon the terms and
subject to the conditions of the Exchange Rights Agreement.

               Unless otherwise indicated under "Special Delivery Instructions",
please mail any Paired Shares issuable upon exchange of the Starwood Units
tendered hereby (or, if the Cash Election or the Combined Election is made, the
cash payment payable pursuant thereto) to the address(es) of the registered
holder(s) appearing under "Description of Units." In the event that the Special
Delivery Instructions are completed, please issue the Paired Shares (or, if the
Cash Election or the Combined Election is made, the cash payment payable
pursuant thereto) in the name of the registered holder(s) and transmit the same
to the person or persons so indicated.

               The Trust, the Corporation and the undersigned agree that they
will cooperate with each other and will make, execute, acknowledge, deliver,
record and file, or cause to be made, executed, acknowledged, delivered,
recorded and filed, at such times and places as the other may reasonably deem
necessary, all other and further documents and instruments, and will take all
other and further actions, as the other may reasonably request from time to time
in order to effectuate the purposes and provisions of the tender made pursuant
to this Letter of Transmittal.



                                      -17-
<PAGE>   18
                          SPECIAL DELIVERY INSTRUCTIONS
                           (SEE INSTRUCTIONS 4 AND 5)


               To be completed ONLY if Paired Shares or the cash payment are to
be sent to someone other than the undersigned or to the undersigned at an
address other than that above.

Mail certificate(s) for Paired Shares or cash payment to:



Name_____________________________________________________________
               (please print)

Address__________________________________________________________

_________________________________________________________________
               (include Zip Code)

_________________________________________________________________

_________________________________________________________________
                 (Tax Identification or Social Security Number)

                                    SIGN HERE

                      Complete Substitute Form W-9 included

_________________________________________________________________

_________________________________________________________________
                        (Signature(s) of holder of Units)

(Must be signed by registered holder(s) as name(s) appear(s) on books and
records of the Partnership. If signature is by trustees, executors,
administrators, guardians, attorneys-in-fact, officers of corporations or others
acting in a fiduciary or representative capacity, please set forth full title
and see instruction 4.

Dated____________________________________________________________

Name(s)__________________________________________________________
                      (please print)
Capacity
(Full Title)_____________________________________________________

Address__________________________________________________________
                      (include Zip Code)



                                      -18-
<PAGE>   19
Area Code and Tel. No.___________________________________________

Tax Identification or
Social Security No.______________________________________________
                              (Complete Substitute Form W-9)


                            Guarantee of Signature(s)
                               (See Instruction 1)

Authorized
Signature________________________________________________________

Name of
Firm_____________________________________________________________

Dated____________________________________________________________


                                  INSTRUCTIONS

                 Forming Part of the Terms and Conditions of the
                            Exchange Rights Agreement

               1. GUARANTEE OF SIGNATURE. No signature guarantee on this Letter
of Transmittal is required unless the registered holder of the Starwood Units
has completed the box entitled "Special Delivery Instructions". In such case all
signatures on this Letter of Transmittal must be guaranteed by a member firm of
any registered national securities exchange in the United States or of the
National Association of Securities Dealers, Inc. or by a commercial bank or
trust company (not a savings bank or a savings and loan association) having an
office, branch or agency in the United States.

               2. DELIVERY OF LETTER OF TRANSMITTAL. This Letter of Transmittal
is to be completed by the holder of Starwood Units. A properly completed and
duly executed Letter of Transmittal and any other documents required by this
Letter of Transmittal must be received by the Agents.

               No alternative, conditional or contingent tenders will be
accepted, except as permitted pursuant to the Exchange Rights Agreement.

               3. INADEQUATE SPACE. If the space provided herein is inadequate,
the Units tendered and/or other information required should be listed on a
separate schedule attached hereto.

               4. SIGNATURES ON LETTER OF TRANSMITTAL. The signature must
correspond with the name as shown on the books and


                                      -19-
<PAGE>   20
records of the Partnerships without any change whatsoever.

               If any of the Starwood Units tendered hereby are owned of record
by two or more joint owners, all such owners must sign the Letter of
Transmittal.

               If any tendered Starwood Units are registered in different names,
it will be necessary to complete, sign and submit as many separate Letters of
Transmittal as there are different registrations.

               If this Letter of Transmittal is signed by trustees, executors,
administrators, guardians, attorneys-in-fact, officers of corporations or others
acting in a fiduciary or representative capacity, each person should so indicate
when signing, and proper evidence satisfactory to the Agents of their authority
so to act must be submitted.

               5. SPECIAL DELIVERY INSTRUCTIONS. If a certificate for Paired
Shares or the cash payment is to be sent to someone other than the signer of
this Letter of Transmittal or to an address other than that shown above, the
appropriate boxes on this Letter of Transmittal should be completed.

               6. WAIVER OF CONDITIONS. Each of the Trust and the Corporation
reserves the right to waive in its sole discretion any of the specified
conditions of the Offer in the case of the Starwood Units tendered; provided
that any such waiver shall not adversely affect any holder of outstanding
Starwood Units without the consent of such holder.

               7. BACK-UP WITHHOLDING. Under the Federal income tax law, a
person surrendering Starwood Units must provide the Agents with his correct
taxpayer identification number ("TIN") on Substitute Form W-9 below unless an
exemption applies. If the correct TIN is not provided, a $50 penalty may be
imposed by the Internal Revenue Service and payments made in exchange for the
surrendered Starwood Units may be subject to back-up withholding of that rate
provided by the Federal income tax law (such rate being at the date of the
Exchange Rights Agreement, 31%).

               The TIN that must be provided is that of the registered holder of
the Starwood Units. The TIN for an individual is his social security number.

               8. REQUESTS FOR ASSISTANCE OR ADDITIONAL COPIES. Questions and
requests for assistance or additional copies of the Exchange Rights Agreement
and the Letter of Transmittal may be directed to the Agents at the address set
forth above.


                                      -20-
<PAGE>   21
                            IMPORTANT TAX INFORMATION

               Under Federal income tax laws, a holder whose tendered Starwood
Units are accepted for payment is required by law to provide the Agents (as
payers) with his correct taxpayer identification number on Substitute Form W-9
below. If such holder is an individual, the taxpayer identification number is
his social security number. If the Agents are not provided with the correct
taxpayer identification number, the holder may be subject to a $50 penalty
imposed by the Internal Revenue Service. In addition, payments that are made to
such holder with respect to Starwood Units purchased pursuant to the Offer may
be subject to back-up withholding.

               If back-up withholding applies, the Agents are required to
withhold, at that rate provided by the Federal income tax law (such rate being
at the date of the Exchange Rights Agreement 31%), of any such payments made to
the holder of Starwood Units. Paired Shares otherwise deliverable hereunder may,
at the expense (and with all risk of loss for the account) of the undersigned,
be sold to pay such amounts. Back-up withholding is not an additional tax.
Rather, the tax liability of persons subject to back-up withholding will be
reduced by the amount of tax withheld. If withholding results in an overpayment
of taxes, a refund may be obtained.

PURPOSE OF SUBSTITUTE FORM W-9

               To prevent back-up withholding on payments that are made to a
holder of Starwood Units purchased pursuant to the Offer, the holder is required
to notify the Agents of his correct taxpayer identification number by completing
the form below certifying that the taxpayer identification number provided on
Substitute Form W-9 is correct.

WHAT NUMBER TO GIVE THE AGENT

               The holder is required to give the Agents the social security
number or employer identification number of the record owner of the Starwood
Units.



                                      -21-
<PAGE>   22
PAYER'S NAME: Starwood Lodging Trust
              Starwood Lodging Corporation


<TABLE>
<S>                     <C>                                                       <C>
-------------------------------------------------------------------------------------------------
Substitute              Part 1 - Please provide your TIN in the box at            Social Security
Form W-9                right and certify by signing and dating below             Number/Employer
                                                                                  Identification
                                                                                  Number
-------------------------------------------------------------------------------------------------
Department of the        Certification - Under the penalties of perjury,
Treasury/Internal       (i) I certify that the information provided on this
Revenue Service         form is true, correct and complete and (ii) I am
                        not subject to backup withholding because:  (a) I
                        am exempt from backup Service withholding, or (b) I
                        have not been notified by the Internal Revenue Service
                        (IRS) that I am subject to backup withholding as a
                        result of a failure to report all interest or dividends,
                        or (c) the IRS has notified me that I am no longer
                        subject to backup
                        withholding.
-------------------------------------------------------------------------------------------------

                        Signature _______________________________________Date ______________

-------------------------------------------------------------------------------------------------
</TABLE>



NOTE:          FAILURE TO COMPLETE AND RETURN THIS FORM MAY RESULT IN BACK-UP
               WITHHOLDING AT THAT RATE PROVIDED BY THE FEDERAL INCOME TAX LAW
               (SUCH RATE BEING AT THE DATE OF THE EXCHANGE RIGHTS AGREEMENT
               31%) OF ANY PAYMENTS MADE TO YOU PURSUANT TO THE OFFER.


                                      -22-