<PAGE>   1

                                                                    EXHIBIT 10.4


                               STARWOOD LETTERHEAD




PERSONAL & CONFIDENTIAL
-----------------------

                                  May 24, 1999


Mr. Ronald C. Brown
6026 E. Cholla Lane
Scottsdale, AZ  85253

Dear Ron:

We are very pleased to extend this offer to you as you transfer from the
Company's Phoenix, AZ office to the Corporate Headquarters in White Plains, NY
as Executive Vice President & Chief Financial Officer for Starwood Hotels &
Resorts Worldwide, Inc. ("Starwood"). The following will outline the specifics
of our offer to you:

BASE SALARY:
Your initial base salary, expressed in semi-monthly terms, will be $17,708.33
(on an annualized basis equivalent to $425,000), effective June 1, 1999 and will
be subject to the appropriate withholdings for FICA, state and federal taxes,
and Medicare.

ANNUAL INCENTIVE:
You will be eligible to receive a performance award based upon the achievement
of specified performance criteria, which will be defined in accordance with the
Company's 1999 plan.

EMPLOYEE BENEFITS:
As an employee of Starwood, you will be entitled to the same health and welfare
benefits as the other executives of the Company. In the event that changes are
made to any of the above benefit plans, compensation and incentive programs, or
standard operating procedures, the changes will apply to you as they do other
executives of the Company.

OPTIONS:
The Company will recommend to the Compensation Committee of the Board of
Directors (the "Committee"), a grant of options to purchase from the Company
shares of common stock of the Company and Class B Shares of Starwood Hotels &
Resorts (together, "Units") for you in accordance with the Company's 1995 Long
Term Incentive Plan (the "LTIP"). Subject to the approval of the Committee, the
options may be granted and vested in accordance with the provisions of the LTIP
and be exercisable at the fair market value of the Units on the date of the
grant or as otherwise determined by the Committee.

RELOCATION EXPENSES:
The Company will pay the reasonable, out-of-pocket costs of relocating your
household furnishings and your family from Scottsdale, AZ to the
Fairfield/Westchester County area



<PAGE>   2


Mr. Ronald C. Brown
May 24, 1999
Page 2



according to the provisions of Starwood's Relocation Program. Upon your move to
the Fairfield/Westmont County area, the Company will make a second mortgage home
loan available to you in the amount of $600,000 which would be due in five years
or upon termination of employment for any reason. The loan would be non-interest
bearing and will be secured by a second mortgage on your home.

TERMINATION/SEVERANCE:
The Company reserves the right to terminate your employment with or without
cause at any time. In the event of an involuntary termination without cause, you
shall receive, as your sole right, exclusive remedy and liquidated damages, a
one time termination payment equal to twelve (12) months base salary. The
Company will also continue to provide medical benefits coverage during the
12-month period subsequent to the termination of your employment. No severance
shall be due in the event that you are terminated for gross misconduct or in the
event that you leave the full-time employ of the Company voluntarily.

RESOLUTION OF DISPUTES:
In the event of any employment-related disputes with respect to your employment
by the Company, you and the Company agree that the same shall be resolved
through binding arbitration in the jurisdiction of the Company's headquarters
and in accordance with the rules and procedures from time to time of the
American Arbitration Association.

Very truly yours,

STARWOOD HOTELS & RESORTS WORLDWIDE, INC.



Susan R. Bolger
Executive Vice President, Human Resources


ACCEPTED AND AGREED TO:



/s/ Ronald C. Brown                                    May 24, 1999
-------------------------------                      --------------------
Ronald C. Brown                                      Date


cc:  Barry Sternlicht
     Personnel File