<PAGE>   1
                                                                   Exhibit 10.3

                       THIRD AMENDMENT TO CREDIT AGREEMENT


            THIRD AMENDMENT TO CREDIT AGREEMENT (this "Amendment"), dated as of
June 15, 1998, among STARWOOD HOTELS & RESORTS, a Maryland real estate
investment trust ("Starwood REIT"), SLT REALTY LIMITED PARTNERSHIP, a Delaware
limited partnership ("SLT RLP"), STARWOOD HOTELS & RESORTS WORLDWIDE, INC., a
Maryland corporation (the "Corporation"), ITT CORPORATION, a Nevada corporation
("ITT" and, together with Starwood REIT, SLT RLP and the Corporation, the
"Borrowers") the lenders from time to time party to the Credit Agreement
referred to below (the "Lenders"), BANKERS TRUST COMPANY and THE CHASE MANHATTAN
BANK, as Administrative Agents (in such capacity, the "Administrative Agents)
and LEHMAN COMMERCIAL PAPER INC. and BANK OF MONTREAL, as Syndication Agents (in
such capacity, the "Syndication Agents"). Unless otherwise defined herein, all
capitalized terms used herein shall have the respective meanings provided such
terms in the Credit Agreement referred to below.

                              W I T N E S S E T H:

            WHEREAS, the Borrowers, the Lenders, the Administrative Agents and
the Syndication Agents are parties to a certain Credit Agreement, dated as of
February 23, 1998 (as amended, modified or supplemented to the date hereof, the
"Credit Agreement"); and

            WHEREAS, in August 1994, Madison Square Garden, L.P. ("MSG"), a
partnership among subsidiaries of ITT and Cablevision Systems Corporation
("Cablevision"), was formed to acquire the business previously operated by
Madison Square Garden Corporation;

            WHEREAS, ITT, through ITT MSG Inc. ("ITT MSG"), owns an
approximately 7.81% limited partnership interest in MSG;

            WHEREAS, ITT MSG under a certain Partnership Interest Transfer
Agreement had two "put" options - one to require Cablevision to purchase (or
cause MSG to redeem) one half of ITT's continuing limited partnership interest
in MSG for $94 million on June 17, 1998 and one to require Cablevision to
purchase (or cause MSG to redeem) the entire remaining interest on June 17, 1999
for an additional $94 million or thereabouts. The first put option was exercised
in March 1998. In addition, on June 17, 2000, Cablevision has the right to
purchase (or cause MSG to redeem) ITT MSG's remaining interest in MSG at a price
determined by an investment banking firm to be the fair market value, subject to
a "floor" price equal to the proportionate "put" price;

            WHEREAS, in order to comply with National Basketball Association and
National Hockey League regulations that prohibit the pledge of any direct or
indirect interest in ITT MSG, ITT MSG intends to sell its remaining limited
partnership interest in MSG to the Corporation;
<PAGE>   2
            WHEREAS, the purchase price will be paid in the form of an unsecured
promissory note or notes and any proceeds received by ITT MSG from Cablevision
will be loaned to the Corporation in exchange for other notes (all of the
foregoing notes, collectively, the "Note") in each case executed by the
Corporation in the aggregate principal amount of $188 million. The Note will
constitute additional intercompany indebtedness that will be subordinated to the
obligations under the Credit Agreement pursuant to a Subordination Agreement to
be executed by ITT MSG and the Collateral Agent in substantially the same form
attached to the Credit Agreement as Exhibit L (the "ITT MSG Subordination
Agreement");

            WHEREAS, all amounts received by ITT MSG or the Corporation from the
transactions described in the third preceding recital shall be applied in
accordance with the requirements of Section 4.02(e) of the Credit Agreement;

            WHEREAS, following the sale by ITT MSG to the Corporation, the
Borrowers have agreed that ITT Sheraton will pledge 100% of the stock in ITT MSG
to the Lenders and ITT MSG will be added as a guarantor pursuant to the
Guaranty;

            WHEREAS, the Credit Agreement may prohibit some of the transactions
described in the foregoing recitals (collectively, the "MSG Transactions") and,
accordingly, the Borrowers wish to secure the Lenders' consent and approval of
the MSG Transactions;

            WHEREAS, in April 1998 SLT RLP sold eight suites hotels to Felcor
(the "Felcor Sale") and received approximately $245,000,000 in proceeds,
$225,000,000 of which are presently being held in escrow in anticipation of a
like kind exchange under Section 1031 of the Internal Revenue Code which is to
occur within 180 days of the Felcor Sale;

            WHEREAS, at the time of the Felcor Sale the Borrowers delivered a
certificate (the "Reinvestment Notice") to the Administrative Agents pursuant to
Sections 9.02(viii) and 4.02(e) advising the Lenders of the intent to reinvest
the proceeds of such sale in accordance with the terms of the Credit Agreement;

            WHEREAS, in order to effect the like kind exchange pursuant to
Section 1031 of the Internal Revenue Code (the "Exchange Transaction"), SLT RLP
intends to purchase from a subsidiary of the Corporation either of the assets
known as The Phoenician, Scottsdale, Arizona or the St. Regis Hotel, New York,
New York (the "Exchange Asset") for the fair market value of either such
Exchange Asset;

            WHEREAS, upon receipt of the remaining approximately $225,000,000 of
proceeds from the Felcor Sale together with the balance of the consideration for
the Exchange Asset, if any (which amount shall be drawn by Starwood REIT from a
Revolving Loan), the Corporation has agreed to cause all such proceeds received
in connection with the Exchange Transaction to be applied to reduce the
outstanding amounts under the Revolving Loans;


                                     - 2 -
<PAGE>   3
            WHEREAS, the Borrowers wish to request a one time waiver from the
restrictions set forth in Section 9.02 (including Section 9.02(xi)) of the
Credit Agreement in order to permit the Exchange Transaction;

            WHEREAS, the parties hereto wish to amend the Credit Agreement as
herein provided to evidence their agreement regarding the MSG Transactions and
the Exchange Transaction;

            NOW, THEREFORE, it is agreed:

      I.    Waivers, Amendments and Agreements with Respect to the Credit
Agreement

            A. Notwithstanding anything to the contrary contained in the Credit
Agreement, the Lenders hereby consent to the MSG Transactions; provided that;
simultaneous with the closing of ITT MSG's transfer of all or any portion of the
limited partnership interest in MSG to the Corporation, (i) ITT Sheraton shall
confirm to the Administrative Agents' reasonable satisfaction that 100% of the
capital stock of ITT MSG is pledged to the Collateral Agent pursuant to the
Pledge and Security Agreement and shall deliver to the Collateral Agent the
certificates for such stock of ITT MSG (together with stock powers executed in
blank), (ii) ITT MSG shall become a Guarantor and shall execute and deliver an
instrument in the form of Annex 1 to the Guaranty (in accordance with the
requirements of Section 22(f) of the Guaranty) and, if at any time ITT MSG
acquires assets of the type required to be pledged to the Lenders pursuant to
the Pledge and Security Agreement, ITT MSG shall become a party to the Pledge
and Security Agreement as a Pledgor in accordance with the provisions of Section
24 of the Pledge and Security Agreement, and (iii) ITT MSG shall execute and
deliver to the Collateral Agent the ITT MSG Subordination Agreement, together
with an Acknowledgment in the form of Exhibit A to Exhibit L to the Credit
Agreement executed and delivered by the Corporation.

            B. The Corporation agrees that all Net Proceeds received by the
Corporation or ITT MSG from the MSG Transactions shall be applied in accordance
with Section 4.02(e) of the Credit Agreement, subject to the right of the Parent
Companies to deliver a reinvestment notice as contemplated by the first proviso
to said Section 4.02(e) (which the Parent Companies advise the Lenders that they
intend to deliver). In the event that a reinvestment notice is delivered, the
Corporate Borrowers hereby jointly and severally agree that, on the date of the
receipt by the Parent Companies or any of their Subsidiaries of any Net Proceeds
from the MSG Transactions, same shall be applied to reduce outstanding Revolving
Loans, subject to the rights of the various Revolving Loan Borrowers to reborrow
same in accordance with the terms and conditions contained in the Credit
Agreement.

            C. The Lenders agree, in connection with the Exchange Transaction
only, to waive the requirements of Section 9.02 of the Credit Agreement and to
permit the Borrowers to cause the Exchange Transaction to occur within 180 days
of the date of the Felcor Sale. Such Exchange Transaction (i) shall not violate
the provisions of the Credit Agreement and shall not be deemed to be a transfer
of Assets by the Corporation or its Subsidiaries to Starwood REIT or 


                                     - 3 -
<PAGE>   4
its Subsidiaries for purposes of Subsection 9.02(xi) of the Credit Agreement,
and (ii) shall not be deemed to constitute a reinvestment of the proceeds from
the Felcor Sale pursuant to Sections 9.02 (viii) and 4.02(e) of the Credit
Agreement. The Corporation agrees that all cash proceeds received by the
Corporation or its Subsidiaries in connection with the Exchange Transaction
shall be applied to reduce outstanding Revolving Loans (but not the Revolving
Loan Commitments) under the Credit Agreement, subject to the right of the
Borrowers to re-borrow Revolving Loans, including, without limitation, to effect
a reinvestment of the proceeds from the Felcor Sale as contemplated by the
Reinvestment Notice. Because the Exchange Transaction does not constitute a
reinvestment of the proceeds from the Felcor Sale pursuant to Section 9.02(viii)
and 4.02(e) of the Credit Agreement, to the extent that the Net Proceeds
received from the Felcor Sale are not reinvested in accordance with the first
proviso to Section 4.02(e) of the Credit Agreement within the time period set
forth therein (which shall run from the date of the Felcor Sale), such Net
Proceeds shall continue to be subject to the requirements of the second proviso
to Section 4.02(e) of the Credit Agreement.

      II.   Miscellaneous Provisions

            A. This Amendment is limited as specified and shall not constitute a
modification, acceptance or waiver of any other provision of the Credit
Agreement or any other Credit Document.

            D. This Amendment may be executed in any number of counterparts and
by the different parties hereto on separate counterparts, each of which
counterparts when executed and delivered shall be an original, but all of which
shall together constitute one and the same instrument. A complete set of
counterparts shall be lodged with the Borrowers and the Paying Agent.

            E. THIS AMENDMENT AND THE RIGHTS AND OBLIGATIONS OF THE PARTIES
HEREUNDER SHALL BE CONSTRUED IN ACCORDANCE WITH AND GOVERNED BY THE LAWS OF THE
STATE OF NEW YORK.

            F. This Amendment shall become effective on the date (the "Amendment
Effective Date") when each of the Borrowers and the Required Lenders shall have
signed a counterpart hereof (whether the same or different counterparts) and
shall have delivered (including by way of facsimile transmission) the same to
the Paying Agent at its Notice Office.

            G. From and after the Amendment Effective Date, all references in
the Credit Agreement and each of the other Credit Documents to the Credit
Agreement shall be deemed to be references to the Credit Agreement as modified
hereby.

                                     *  *  *


                                     - 4 -
<PAGE>   5
      IN WITNESS WHEREOF, the parties hereto have caused their duly authorized
officers to execute and deliver this Amendment as of the date first above
written.

                                    STARWOOD HOTELS & RESORTS
                                     a Maryland real estate investment trust


                                    By:_______________________________
                                       Name:
                                       Title:


                                    STARWOOD HOTELS & RESORTS
                                     WORLDWIDE, INC., a Maryland corporation



                                    By:_______________________________
                                       Name:
                                       Title:

                                    SLT REALTY LIMITED PARTNERSHIP,
                                     a Delaware limited partnership

                                    By: Starwood Hotels & Resorts, a Maryland
                                         real estate investment trust, its 
                                          general partner



                                    By:_______________________________
                                       Name:
                                       Title:


                                    ITT CORPORATION, a Nevada corporation



                                    By:_______________________________
                                       Name:
                                       Title:


                                     - 5 -
<PAGE>   6
                                    BANKERS TRUST COMPANY, Individually and as 
                                     Administrative Agent and as Paying Agent



                                    By:_______________________________
                                       Name:
                                       Title:


                                    THE CHASE MANHATTAN BANK,
                                     Individually and as Administrative Agent



                                    By:_______________________________
                                       Name:
                                       Title:


                                    LEHMAN COMMERCIAL PAPER, INC.,
                                     Individually and as Syndication Agent



                                    By:_______________________________
                                       Name:
                                       Title:


                                    BANK OF MONTREAL, CHICAGO BRANCH
                                     Individually and as Syndication Agent



                                    By:_______________________________
                                       Name:
                                       Title:


                                     - 6 -
<PAGE>   7
                                    ARAB BANKING CORPORATION (B.S.C.)



                                    By:_______________________________
                                       Name:
                                       Title:


                                    BANCA POPOLARE DI MILANO


                                    By:_______________________________
                                       Name:
                                       Title:


                                    By:_______________________________
                                       Name:
                                       Title:


                                    BANKBOSTON, N.A.



                                    By:_______________________________
                                       Name:
                                       Title:


                                    BANK LEUMI USA



                                    By:_______________________________
                                       Name:
                                       Title:


                                     - 7 -
<PAGE>   8
                                    THE BANK OF TOKYO-MITSUBISHI,
                                     LIMITED, NEW YORK BRANCH



                                    By:_______________________________
                                       Name:
                                       Title:


                                    BANK POLSKA KASA OPIEKI S.A.
                                     PEKAO S.A. GROUP, NEW YORK BRANCH



                                    By:_______________________________
                                       Name:
                                       Title:



                                    BANQUE PARIBAS



                                    By:_______________________________
                                       Name:
                                       Title:


                                    By:_______________________________
                                       Name:
                                       Title:


                                    BANQUE WORMS CAPITAL CORP.



                                    By:_______________________________
                                       Name:
                                       Title:


                                     - 8 -
<PAGE>   9
                                    BEAR STEARNS INVESTMENT
                                     PRODUCTS INC.



                                    By:_______________________________
                                       Name:
                                       Title:


                                    BARCLAYS BANK PLC



                                    By:_______________________________
                                       Name:
                                       Title:


                                    CHANG HWA COMMERCIAL BANK, LTD.,
                                     NEW YORK BRANCH



                                    By:_______________________________
                                       Name:
                                       Title:


                                    CHIAO TUNG BANK CO., LTD.
                                     NEW YORK AGENCY



                                    By:_______________________________
                                       Name:
                                       Title:


                                    CIBC INC.


                                    By:_______________________________


                                     - 9 -
<PAGE>   10
                                       Name:
                                       Title:


                                    COMPAGNIE FINANCIERE DE CIC ET DE
                                     L'UNION EUROPEENNE



                                    By:_______________________________
                                       Name:
                                       Title:


                                    By:_______________________________
                                       Name:
                                       Title:


                                    CREDIT LYONNAIS NEW YORK BRANCH



                                    By:_______________________________
                                       Name:
                                       Title:


                                    CREDIT SUISSE FIRST BOSTON



                                    By:_______________________________
                                       Name:
                                       Title:


                                    By:_______________________________
                                       Name:
                                       Title:


                                    CREDITO ITALIANO


                                     - 10 -
<PAGE>   11
                                    By:_______________________________
                                       Name:
                                       Title:


                                    By:_______________________________
                                       Name:
                                       Title:


                                    DEUTSCHE BANK AG NEW YORK
                                     AND/OR CAYMAN ISLANDS BRANCH



                                    By:_______________________________
                                       Name:
                                       Title:


                                    By:_______________________________
                                       Name:
                                       Title:


                                    ERSTE BANK DER OESTERREICHISCHEN
                                     SPARKASSEN AG



                                    By:_______________________________
                                       Name:
                                       Title:


                                    By:_______________________________
                                       Name:
                                       Title:


                                    FIRST COMMERCIAL BANK



                                    By:_______________________________


                                     - 11 -
<PAGE>   12
                                       Name:
                                       Title:



                                    FIRST SECURITY BANK, N.A.



                                    By:_______________________________
                                       Name:
                                       Title:


                                    FLEET BANK, N.A.


                                    By:_______________________________
                                       Name:
                                       Title:


                                    GENERAL ELECTRIC CAPITAL
                                     CORPORATION



                                    By:_______________________________
                                       Name:
                                       Title:


                                    GOLDMAN SACHS CREDIT PARTNERS L.P.



                                    By:_______________________________
                                       Name:
                                       Title:


                                    GULF INTERNATIONAL BANK B.S.C.


                                     - 12 -
<PAGE>   13
                                    By:_______________________________
                                       Name:
                                       Title:


                                    HUA NAN COMMERCIAL BANK, LTD.
                                     NEW YORK AGENCY



                                    By:_______________________________
                                       Name:
                                       Title:


                                    THE INDUSTRIAL BANK OF JAPAN,
                                     LIMITED NEW YORK BRANCH



                                    By:_______________________________
                                       Name:
                                       Title:


                                    ISTITUTO BANCARIO DI TORINO SpA



                                    By:_______________________________
                                       Name:
                                       Title:


                                    LAND BANK OF TAWAIN, LOS ANGELES
                                     BRANCH


                                    By:_______________________________
                                       Name:
                                       Title:


                                    THE LONG TERM CREDIT BANK OF


                                     - 13 -
<PAGE>   14
                                     JAPAN, LTD.


                                    By:_______________________________
                                       Name:
                                       Title:


                                    MITSUBISHI TRUST & BANKING
                                     CORPORATION


                                    By:_______________________________
                                       Name:
                                       Title:


                                    NATIONSBANK, N.A.



                                    By:_______________________________
                                       Name:
                                       Title:


                                    THE ROYAL BANK OF SCOTLAND, PLC



                                    By:_______________________________
                                       Name:
                                       Title:


                                    SOCIETE GENERALE, SOUTHWEST AGENCY



                                    By:_______________________________
                                       Name:
                                       Title:


                                    SOUTHERN PACIFIC BANK


                                     - 14 -
<PAGE>   15
                                    By:_______________________________
                                       Name:
                                       Title:




                                    THE SUMITOMO BANK, LIMITED,
                                     NEW YORK BRANCH



                                    By:_______________________________
                                       Name:
                                       Title:


                                    WACHOVIA BANK, N.A.



                                    By:_______________________________
                                       Name:
                                       Title:


                                    WESTDEUTSCHE LANDESBANK
                                     GIROZENTRALE



                                    By:_______________________________
                                       Name:
                                       Title:


                                    By:_______________________________
                                       Name:
                                       Title:


                                     - 15 -
<PAGE>   16
                                    VAN KAMPEN AMERICAN CAPITAL PRIME
                                     RATE INCOME TRUST

                                    By:_____________________________________

                                    VAN KAMPEN CLO I, LIMITED

                                    By: Van Kampen American Capital Management
                                        Inc., as collateral manager


                                        By:__________________________________


                                    THE TORONTO DOMINION BANK

                                    By:_____________________________________


                                     - 16 -