<PAGE>   1
                                                                   Exhibit 10.4

                    AMENDED AND RESTATED EMPLOYMENT AGREEMENT

     AGREEMENT effective January 1, 1997, by and between RFS Managers, Inc., a
Tennessee corporation (the "Company"), and Robert M. Solmson (the "Executive").


                              W I T N E S S E T H:

     WHEREAS, the Company provides management services to RFS Hotel Investors,
Inc. (the "Parent") pursuant to a Management Services Agreement dated December
30, 1994 (the "Management Agreement"); and

     WHEREAS, the Company desires to employ the Executive to serve as the Chief
Executive Officer of the Company; and

     WHEREAS, the Company and the Executive have previously entered into an
employment agreement dated January 1, 1996 (the "Original Agreement"); and

     WHEREAS, the parties desire to amend and restate the Original Agreement in
its entirety effective as of January 1, 1997 as set forth herein.

     NOW, THEREFORE, in consideration of the premises and mutual obligations
hereinafter set forth the parties agree as follows:

     1. EMPLOYMENT. The Company shall employ the Executive, and the Executive
agrees to be so employed, in the capacity of Chief Executive Officer of the
Company to serve for the Term hereof, subject to earlier termination as
hereinafter provided.

     2. TERM. The term of the Executive's employment hereunder shall be three
years and shall commence on January 1, 1997 and shall be extended automatically,
for so long as the Executive remains employed by the Company hereunder, each
January 1 beginning January 1, 1998 for an additional twelve-month period (such
period, as it may be extended from time to time, being herein referred to as the
"Term"), unless terminated earlier in accordance with the terms of this
Agreement, to the effect that on each January 1, the remaining term of this
Agreement and the Executive's employment hereunder shall be three years.

     3. SERVICES. The Executive shall devote such amount of his time and
attention to the Company's affairs as are necessary to perform his duties to the
Company and to allow the Company to perform its duties specified in the
Management Agreement. Pursuant to the Management Agreement, the Executive shall
have authority and responsibility with respect to the day to day operations and
management of the Parent and RFS Partnership, L.P. (the "Partnership"), for
which the Parent currently serves as sole general partner, as well as
implementation of the long range growth strategy of the Parent and the
Partnership, consistent with direction from the Parent's Board of Directors (the
"Board").




<PAGE>   2



     4. COMPENSATION. (a) During the Term, the Company shall pay the Executive
for his services an annual base salary of $225,000 (the "Base Salary"), to be
paid in semi-monthly payments of $9,375.00, such Base Salary being subject to
any increases approved by the Compensation Committee of the Board (the
"Compensation Committee").

        (b) In addition to the Base Salary described in Section 4(a) above, the
Executive shall be entitled to a cash bonus ("Base Salary Bonus") for 1996
payable on or before April 1, 1997 determined as follows:

               (i) If fully diluted net income per share of common stock of the
               Parent, for the year ending December 31, 1996, as reported in the
               Parent's audited financial statements for the year ending
               December 31, 1996, as adjusted as described in the following
               sentence ("1996 Net Income Per Share") is at least $1.35, the
               Executive shall be entitled to receive a cash bonus equal to ten
               percent (10%) of the Base Salary. For purposes of determining the
               cash bonus under this Section 4(b), fully diluted Net Income per
               Share of common stock of the Parent shall be exclusive of any
               gain or loss on the sale of property, any expenses relating to
               the transactions between the Company and Doubletree Corporation
               and its affiliates and any expenses which the Compensation
               Committee deems appropriate to exclude from the calculation of
               fully diluted net income per share for purposes of determining
               the cash bonus; and

               (ii) For each $.01 increase in Net Income Per Share for 1996 in
               excess of $1.35, computed by rounding to the closest cent, the
               Executive shall be entitled to receive an additional bonus equal
               to three percent (3%) of the Base Salary; and

               (iii) Notwithstanding the provisions of Section 4(b)(ii) above,
               the maximum cash bonus payable to the Executive pursuant to this
               Section 4(b) shall be fifty percent (50%) of the Base Salary.

        (c) In addition to the Base Salary Bonus, the Executive may be entitled
to receive other incentive compensation, including but not limited to,
additional grants of stock options or shares of stock of the Parent, which
awards shall be made (if at all) in consideration of and as an incentive for
services performed solely for the Company, in accordance with rules and criteria
established by the Compensation Committee. Such criteria may include, but not be
limited to, the growth in the Parent's Net Income Per Share and/or other
performance goals.

     5. BENEFITS. The Company agrees to provide the Executive with the following
benefits:

        (a) Vacation. The Executive shall be entitled each calendar year to a
vacation, during which time his compensation shall be paid in full. The time
allotted for such vacation


                                        2

<PAGE>   3



shall be three (3) weeks.

        (b) Employee Benefits. This Agreement shall not be in lieu of any
rights, benefits and privileges to which the Executive may be entitled as a
management level employee of the Company, including but not limited to any
retirement, pension, profit-sharing, insurance, hospital or other plans which
may now be in effect or which may hereafter be adopted. The Executive shall have
the same rights and privileges to participate in such plans and benefits as any
other management level employee during the Term.

     6. EXPENSES. The Company recognizes that the Executive will have to incur
certain out-of-pocket expenses, including but not limited to travel expenses,
related to his services and the Company's and the Parent's business and the
Company agrees to reimburse the Executive for all reasonable expenses
necessarily incurred by him in the performance of his duties upon presentation
of a voucher or documentation indicating the amount and business purposes of any
such expenses.

     7. TERMINATION IN CASE OF DEATH OR DISABILITY. In the event of the
Executive's death or a complete physical or mental inability, confirmed by a
licensed physician, to perform the services described in Section 3 above that
continues for a period of one hundred twenty (120) consecutive days) ("Permanent
Disability"), the Company may elect to terminate this Agreement, subject to
continuation of the payments described in Section 10.

     8. DEFINITIONS. For purposes of this Agreement, the following terms shall
have the following definitions:

        (a) "Voluntary Termination" means, subject to the provisions of Section
11 hereof, the Executive's voluntary termination of his employment hereunder,
which may be effected by the Executive giving the Board not less than 90 days'
prior written notice of the Executive's desire to terminate his employment or
the Executive's failure to provide substantially all the services described in
Section 3 hereof for a period greater than four consecutive weeks by reason of
the Executive's voluntary refusal to perform such services. Notwithstanding the
foregoing, if the Executive gives notice of Voluntary Termination and, prior to
the expiration of the 90-day notice period, the Executive voluntarily refuses or
fails to provide substantially all the services described in Section 3 hereof
for a period greater than two consecutive weeks, the Voluntary Termination shall
be deemed to be effective as of the date on which the Executive so ceases to
carry out his duties. For purposes of this Section 8, voluntary refusal to
perform services shall not include taking vacation otherwise permitted in
accordance with Section 5(a) hereof, the Executive's failure to perform services
on account of his illness or the illness of a member of his immediate family,
provided such illness is adequately substantiated at the reasonable request of
the Company, or any other absence from service with the written consent of the
Board.




                                        3

<PAGE>   4



        (b) "Termination Without Cause" means the termination of the Executive's
employment by the Company for any reason other than Voluntary Termination or
Termination With Cause.

        (c) "Termination With Cause" means the termination of the Executive's
employment by act of the Board for any of the following reasons:

               (i) the Executive's conviction for a felony;

               (ii) the Executive's theft, embezzlement, misappropriation of or
               intentional and malicious infliction of damage to the Company's
               or the Parent's property or business opportunity;

               (iii) the Executive's intentional and material breach of the
               noncompetition covenant in Section 11 hereof;

               (iv) the Executive's continuous neglect of his duties hereunder
               or his continuous failure or refusal to follow any reasonable,
               unambiguous duly adopted written direction of the Board or any
               duly constituted committee thereof that is not inconsistent with
               the description of the Executive's duties set forth in Section 3
               above; and

               (v) the Executive's abuse of alcohol, drugs or other substances,
               or his engaging in other deviant personal activities in a manner
               that, in the reasonable judgment of the Board, adversely affects
               the reputation, goodwill or business position of the Company.

        (d) "Involuntary Termination" means conduct on the part of the Company
that constitutes continuous and material interference by the Company with the
Executive's performance of his duties as set forth in Section 3 hereof or the
intentional or material breach by the Company of this Agreement.

     9. VOLUNTARY TERMINATION; TERMINATION WITH CAUSE. If (i) the Executive
shall cease being an employee of the Company on account of a Voluntary
Termination or (ii) there shall be a Termination With Cause, the Executive shall
not be entitled to any compensation after the effective date of such Voluntary
Termination or Termination With Cause (except Base Salary and vacation accrued
but unpaid on the effective date of such event). In the event of a Voluntary
Termination or Termination With Cause, the Executive shall continue to be
subject to the noncompetition covenant contained in Section 11 hereof for the
remainder of the Term.

     10. DEATH OR DISABILITY; TERMINATION WITHOUT CAUSE; OR INVOLUNTARY
TERMINATION. Following (i) the death of the Executive, (ii) Permanent Disability
of the Executive, (iii) an Involuntary Termination, or (iv) a Termination
Without Cause, the Company shall continue to pay the Executive or his heirs,
devisees, executors, legatees or personal representatives, as



                                        4

<PAGE>   5



appropriate, the semi-monthly payments of the Base Salary then in effect for
three years from the date of the termination of the Executive's employment.

     11. CHANGE OF CONTROL COMPENSATION.

        (a) Compensation. In the event of the Company's termination of the
Executive's employment or the Executive's resignation for Good Reason (as
defined below) after a Change of Control (as defined below), the Company shall,
on the date of such termination or resignation, pay the Executive, in addition
to any Base Salary earned but not paid through the date of termination or
resignation for Good Reason, a cash amount equal to three (3) times the Base
Salary for the fiscal year in which Change of Control occurs (the "Termination
Payment"). In addition, the Company shall cause the Executive's insurance
benefits, as in effect immediately prior to the Change of Control, to remain in
effect for at least one year following the date of the termination of
Executive's employment by the Company or the Executive's resignation for Good
Reason.

        (b) A "Change of Control", for purposes of this Agreement, shall be
deemed to have occurred if, at any time during the Term, any of the following
events occurs:

               (i) any "person", as that term is used in Section 13(d) and
          Section 14(d)(2) of the Securities Exchange Act of 1934, as amended
          (the "Exchange Act"), becomes, is discovered to be, or files a report
          on Schedule 13D or 14D-1 (or any successor schedule, form or report)
          disclosing that such person is, a beneficial owner (as defined in Rule
          13d-3 under the Exchange Act or any successor rule or regulation),
          directly or indirectly, of securities of the Company representing 50%
          or more of the combined voting power of the Company's then outstanding
          securities entitled to vote generally in the election of directors;

               (ii) individuals who, as of the Effective Date, constitute the
          Board of Directors of the Company cease for any reason to constitute
          at least a majority of the Board of Directors of the Company, unless
          any such change is approved by the vote of at least 80% of the members
          of the Board of Directors of the Company in office immediately prior
          to such cessation;

               (iii) the Company is merged, consolidated or reorganized into or
          with another corporation or other legal person, or securities of the
          Company are exchanged for securities of another corporation or other
          legal person, and immediately after such merger, consolidation,
          reorganization or exchange less than a majority of the combined voting
          power of the then-outstanding securities of such corporation or person
          immediately after such transaction are held, directly or indirectly,
          in the aggregate by the holders of securities entitled to vote
          generally in the election of directors of the Company immediately
          prior to such transaction;


                                        5

<PAGE>   6



               (iv) the Company in any transaction or series of related
          transactions, sells all or substantially all of its assets to any
          other corporation or other legal person and less than a majority of
          the combined voting power of the then-outstanding securities of such
          corporation or person immediately after such sale or sales are held,
          directly or indirectly, in the aggregate by the holders of securities
          entitled to vote generally in the election of directors of the Company
          immediately prior to such sale;

               (v) the Company and its affiliates shall sell or transfer of (in
          a single transaction or series of related transactions) to a
          non-affiliate business operations or assets that generated at least
          two-thirds of the consolidated revenues (determined on the basis of
          the Company's four most recently completed fiscal quarters for which
          reports have been filed under the Exchange Act) of the Company and its
          subsidiaries immediately prior thereto;

               (vi) the Company files a report or proxy statement with the
          Securities and Exchange Commission pursuant to the Exchange Act
          disclosing in response to Form 8-K (or any successor, form or report
          or item therein) that a change in control of the Company has occurred;
          or

               (vii) any other transaction or series of related transactions
          occur that have substantially the effect of the transactions specified
          in any of the preceding clauses in this sentence.

        (c) Certain Transactions. Notwithstanding the provisions of Section
11(b)(i) or 11(b)(vi) hereof, unless otherwise determined in a specific case by
majority vote of the Board of Directors of the Company, a Change in Control
shall not be deemed to have occurred for purposes of this Agreement solely
because (i) the Company, (ii) an entity in which the Company directly or
indirectly beneficially owns 50% or more of the voting securities or (iii) any
Company-sponsored employee stock ownership plan, or any other employee benefit
plan of the Company, either files or becomes obligated to file a report or a
proxy statement under or in response to Schedule 13D, Schedule 14D-1, Form 8-K
or Schedule 14A (or any successor schedule, form or report or item thereon)
under the Exchange Act, disclosing beneficial ownership by it of shares of stock
of the Company, or because the Company reports that a Change in Control of the
Company has or any have occurred or will or may occur in the future by reason of
such beneficial ownership.

        (d) Good Reason. "Good Reason," for purposes of this Agreement, shall be
deemed to mean any of the following:

               (i) a change in the Executive's status, position or
          responsibilities (including reporting responsibilities) which, in the
          Executive's reasonable judgment, does not represent a promotion from
          the Executive's status, position or responsibilities as in effect
          immediately prior to a Change in Control; the



                                        6

<PAGE>   7



          assignment to the Executive of any duties or responsibilities which,
          in the Executive's reasonable judgment, are inconsistent with such
          status, position or responsibilities; or any removal of the Executive
          from or failure to reappoint or reelect the Executive to any of such
          positions, except in connection with a Termination with Cause as
          defined in Section 8(c), as a result of the Executive's death or
          Permanent Disability, or by Voluntary Termination;

               (ii) a reduction in the Executive's Base Salary and Bonus as in
          effect on the date hereof or as the same may be increased from time to
          time;

               (iii) the relocation of the Company's or the Parent's principal
          executive offices to a location outside a thirty-mile radius of
          Memphis, Tennessee or the Company's or the Parent's requiring the
          Executive to be based at any place other than a location within a
          thirty-mile radius of Memphis, Tennessee, except for reasonably
          required travel on the Company's or the Parent's business which is not
          materially greater than such travel requirements prior to the Change
          in Control;

               (iv) the failure by the Company or the Parent to continue to
          provide the Executive with compensation and benefits provided for
          under this agreement or benefits substantially similar to those
          provided to the Executive under any of the employee benefit plans in
          which the Executive is or becomes a participant, or the taking of any
          action by the Company or the Parent which would directly or indirectly
          materially reduce any of such benefits or deprive the Executive of any
          material fringe benefit enjoyed by the Executive at the time of the
          Change in Control;

               (v) any material breach by the Company of any provision of this
          Agreement; and

               (vi) the failure of the Company to obtain a satisfactory
          agreement from any successor or assign of the Company to assume and
          agree to perform this Agreement.

        (e) Tax Matters. In the event the Executive's receipt of the Termination
Payment, together with any other payment or compensation which the Executive may
be entitled to receive from the Company or the Parent or any of their affiliates
as a result of the Change in Control, would, in the reasonable opinion of the
Executive's tax advisor, cause the Executive to incur tax liability under
Section 4999 of the Internal Revenue Code of 1986, as amended (the "Code"), or
any successor or similar provision, the Executive may elect to receive either:
(i) the Termination Payment or (ii) the maximum portion of the Termination
Payment which the Executive may receive, which taken together with any other
payment or compensation which the Executive may be entitled to receive from the
Company or the Parent or any of their affiliates as a result of the Change in



                                        7

<PAGE>   8



Control, would not cause the Executive to incur tax liability under Section 4999
of the Code.

     12. NONCOMPETITION. During the Term and for a period of two (2) years
thereafter, the Executive shall not, other than through the Parent or affiliates
of the Parent, own more than a 10% interest in any hotel property (other than
hotels owned by the Parent and the Partnership), as partner, shareholder or
otherwise, or directly or indirectly, for his own account or for the account of
others, either as an officer, director, shareholder, owner, partner, promoter,
employee, consultant, advisor, agent, manager, or in any other capacity engage
in the acquisition, development, operation or management of any hotel property
located within 20 miles of any hotel property owned by the Parent or the
Partnership at the time of termination of employment. The foregoing sentence
shall not restrict the Executive from owning up to 10% of the outstanding
securities of any entity, including any entity whose securities are traded in
public securities markets.

     The Executive agrees that damages at law for violation of the restrictive
covenant contained herein would not be an adequate or proper remedy to the
Company, and that should the Executive violate or threaten to violate any of the
provisions of such covenant, the Company, its successors or assigns, shall be
entitled to obtain a temporary or permanent injunction against the Executive in
any court having jurisdiction over the person and the subject matter,
prohibiting any further violation of any such covenants. The injunctive relief
provided herein shall be in addition to any award of damages, compensatory,
exemplary or otherwise, payable by reason of such violation.

     Furthermore, the Executive acknowledges that this Agreement has been
negotiated at arms' length by the parties, neither being under any compulsion to
enter into this Agreement, and that the foregoing restrictive covenant does not
in any respect inhibit his ability to earn a livelihood in his chosen profession
without violating the restrictive covenant contained herein. The Company by
these presents has attempted to limit the Executive's right to compete only to
the extent necessary to protect the Company from unfair competition. The Company
recognizes, however, that reasonable people may differ in making such a
determination. Consequently, the Company agrees that if the scope or
enforceability of the restricted covenant contained herein is in any way
disputed at any time, a court or other trier of fact may modify and enforce the
covenant to the extent that it believes to be reasonable under the circumstances
existing at the time.


     13. NOTICES. All notices or deliveries authorized or required pursuant to
this Agreement shall be deemed to have been given when in writing and personally
delivered or when deposited in the U.S. mail, certified, return receipt
requested, postage prepaid, addressed to the parties at the following addresses
or to such other addresses as either may designate in writing to the other
party:



                                        8

<PAGE>   9



         To the Company:                 RFS Managers, Inc.
                                         850 Ridge Lake Boulevard
                                         Suite 220
                                         Memphis, TN 38120

         To the Executive:               Robert M. Solmson
                                         850 Ridge Lake Boulevard, Suite 220
                                         Memphis, TN 38120

     14. ENTIRE AGREEMENT. This Agreement contains the entire understanding
between the parties hereto with respect to the subject matter hereof and shall
not be modified in any manner except by instrument in writing signed, by or on
behalf of, the parties hereto; provided, however, that any amendment or
termination of the covenant of noncompetition in Section 11 must be approved by
a majority of the Directors of the Parent other than the Executive, if the
Executive is then a director of the Parent. This Agreement shall be binding upon
and inure to the benefit of the heirs, successors and assigns of the parties
hereto.

     15. ARBITRATION. Any claim or controversy arising out of, or relating to,
this Agreement or its breach, shall be settled by arbitration in accordance with
the governing rules of the American Arbitration Association. Judgment upon the
award rendered may be entered in any court of competent jurisdiction.

     16. APPLICABLE LAW. This Agreement shall be governed and construed in
accordance with the laws of the State of Tennessee.

     17. ASSIGNMENT. The Executive acknowledges that his services are unique and
personal. Accordingly, the Executive may not assign his rights or delegate his
duties or obligations under this Agreement, except with respect to certain
rights to receive payments as described in Section 10. The Company's rights and
obligations under this Agreement shall inure to the benefit of and shall be
binding upon the Company's successors and assigns.

     18. HEADINGS. Headings in this Agreement are for convenience only and shall
not be used to interpret or construe its provisions.

     19. ORIGINAL AGREEMENT. This Agreement amends and restates the Original
Agreement in its entirety as of the effective date of this Agreement.



                                        9

<PAGE>   10


     IN WITNESS WHEREOF, the parties have executed this Agreement effective as
of the date first above written.


                                            RFS MANAGERS, INC.


                                            By: /s/ Minor W. Perkins
                                                --------------------
                                            Name: ____________________
                                            Title: President


                                            EXECUTIVE:

                                            /s/ Robert M. Solmson
                                            ---------------------
                                            Robert M. Solmson












                                       10