<PAGE>   1
                                                                   EXHIBIT 10.17


                   AMENDMENT NO. 5 TO LOAN AGREEMENT AND NOTES


         AMENDMENT NO. 5 TO LOAN AGREEMENT (this "FIFTH AMENDMENT"), made and
executed this 8th day of February, 2001, by and between:

         OMEGA WORLDWIDE, INC., a Maryland corporation (the "BORROWER");

         The Banks that have executed the signature pages hereto (individually,
a "BANK" and collectively, the "BANKS"); and

         FLEET NATIONAL BANK (formerly Fleet Bank, N.A.), a national banking
association, as Agent for the Banks (in such capacity, together with its
successors in such capacity, the "AGENT").

                             PRELIMINARY STATEMENTS

         (A) The Borrower has entered into a certain Loan Agreement dated
November 20, 1998, as amended by (i) Amendment No. 1 to Loan Agreement dated
October 22, 1999, effective as of August 18, 1999, (ii) Amendment No. 2 and
Waiver to Loan Agreement dated January 10, 2000, (iii) Amendment No. 3 and
Waiver to Loan Agreement dated May 12, 2000, effective as of March 17, 2000, and
(iv) Amendment No. 4 to Loan Agreement dated July 7, 2000, effective as of June
29, 2000 (as so amended, hereinafter referred to as the "LOAN AGREEMENT") with
the Agent and the Banks; and

         (B) The Borrower has requested that the Banks and the Agent amend
certain provisions of the Loan Agreement, and the Banks and the Agent are
willing to do so, all on the terms and conditions hereinafter set forth;

         NOW, THEREFORE, in consideration of the agreements and provisions
contained herein, the parties hereto hereby agree as follows:

         1. DEFINITIONS. Capitalized terms used but not otherwise defined herein
shall have the meanings ascribed to such terms in the Loan Agreement.

         2. CERTAIN AMENDMENTS TO THE LOAN AGREEMENT AND NOTES.

                  The parties hereto confirm that, as of the date of the Fifth
Amendment, the aggregate principal amount of the Term Loans outstanding is
$2,850,000 (the "PRINCIPAL BALANCE"). Notwithstanding anything to the contrary
contained in Section 2.6 of the Loan Agreement or the Notes, the parties agree
that the Borrower shall pay to the Agent for the account of each Bank, the
Principal Balance in three (3) installments, on the dates and in the amounts set
forth below, provided, that, the last such installment on June 30, 2001 shall be
in an amount sufficient to repay in full the outstanding principal amount of the
Term Loans on such date:


<PAGE>   2
<TABLE>
<CAPTION>
---------------------------- ---------------------------- --------------------------- ----------------------------------
      Dates on Which             Amount of Principal         Amount of Principal         Amount of Principal Balance
 Installments of Principal     Balance Payable on Each     Balance Payable to Fleet      Payable to Harris Trust and
          are Due                     Such Date                 National Bank                   Savings Bank
---------------------------- ---------------------------- --------------------------- ----------------------------------
<S>                          <C>                          <C>                         <C>
February 8, 2001                     $1,000,000                    $600,000                       $400,000

---------------------------- ---------------------------- --------------------------- ----------------------------------
March 31, 2001                         $500,000                    $300,000                       $200,000

---------------------------- ---------------------------- --------------------------- ----------------------------------
June 30, 2001                        $1,350,000                    $810,000                       $540,000

---------------------------- ---------------------------- --------------------------- ----------------------------------
</TABLE>
The Loan Agreement and the Notes are each hereby deemed modified and amended to
give effect to the foregoing.

         3. REPRESENTATIONS AND WARRANTIES. In order to induce the Banks and the
Agent to enter into this Fifth Amendment, each of the Loan Parties hereby
represents and warrants to the Banks and the Agent, as to itself with respect to
the Loan Documents to which it is a party, that:

                  3.1 No Default. After giving effect to this Fifth Amendment,
no Default or Event of Default shall have occurred or be continuing.

                  3.2 Existing Representations and Warranties. As of the date
hereof and after giving effect to this Fifth Amendment, each and every one of
the representations and warranties set forth in the Loan Documents are true,
accurate and complete in all respects and with the same effect as though made on
the date hereof, and each is hereby incorporated herein in full by reference as
if restated herein in its entirety, except for changes in the ordinary course of
business which are not prohibited by the Loan Agreement (as amended hereby) and
which do not, either singly or in the aggregate, have a Material Adverse Effect.

                  3.3 Authority; Enforceability. (i) The execution, delivery and
performance by each Loan Party of this Fifth Amendment are within its
organizational powers and have been duly authorized by all necessary action
(corporate or otherwise) on the part of each Loan Party, (ii) this Fifth
Amendment is the legal, valid and binding obligation of each Loan Party,
enforceable against each Loan Party in accordance with its terms, and (iii) this
Fifth Amendment and the execution, delivery and performance by each Loan Party
thereof does not: (A) contravene the terms of any Loan Party's organization
documents, (B) conflict with or result in any breach or contravention of, or the
creation of any Lien under, any document evidencing any contractual obligation
to which any Loan Party is a party or any order, injunction, writ or decree to
which any Loan Party or its property is subject, or (C) violate any requirement
of law.

         4. REFERENCE TO AND EFFECT UPON THE LOAN AGREEMENT.

                  4.1 Effect. Except as specifically set forth herein, the Loan
Agreement and the other Loan Documents shall remain in full force and effect in
accordance with their terms and are hereby ratified and confirmed.


                                       2
<PAGE>   3

                  4.2 No Waiver; References. The execution, delivery and
effectiveness of this Fifth Amendment shall not operate as a waiver of any
right, power or remedy of the Agent or any Bank under the Loan Agreement, nor
constitute a waiver of any provision of the Loan Agreement, except as
specifically set forth herein. Upon the effectiveness of this Fifth Amendment,
each reference in:

                           (i) the Loan Agreement to "this Agreement",
"hereunder", "hereof", "herein" or words of similar import shall mean and be a
reference to the Loan Agreement as amended hereby;

                           (ii) the other Loan Documents to the "Loan Agreement"
shall mean and be a reference to the Loan Agreement as amended hereby;

                           (iii) the Loan Documents to the "Loan Documents"
shall be deemed to include this Fifth Amendment; and

                           (iv) the "Notes" shall be deemed to refer to the
Notes as amended hereby.

         5. MISCELLANEOUS.

                  5.1 Expenses. The Loan Parties agree to pay the Agent upon
demand for all reasonable expenses, including reasonable attorneys' fees and
expenses of the Agent, incurred by the Agent in connection with the preparation,
negotiation and execution of this Fifth Amendment.

                  5.2. Law. THIS FIFTH AMENDMENT SHALL BE CONSTRUED IN
ACCORDANCE WITH AND GOVERNED BY THE INTERNAL LAWS OF THE STATE OF NEW YORK.

                  5.3 Successors. This Fifth Amendment shall be binding upon the
Loan Parties, the Banks and the Agent and their respective successors and
assigns, and shall inure to the benefit of the Loan Parties, the Banks and the
Agent and the successors and assigns of the Banks and the Agent.

                  5.4 Execution in Counterparts. This Fifth Amendment may be
executed in any number of counterparts and by different parties hereto in
separate counterparts, each of which when so executed and delivered shall be
deemed to be an original and all of which taken together shall constitute one
and the same instrument.





                           [Signature Page to Follow]





                                       3
<PAGE>   4


         IN WITNESS WHEREOF, the parties hereto have caused this Fifth Amendment
to be executed and delivered by their respective officers thereunto duly
authorized as of the date first written above.

                                    OMEGA WORLDWIDE, INC.


                                    By  /s/ John Storey
                                      ------------------------------------------
                                        Treasurer                          Title

                                    FLEET NATIONAL BANK, as Agent
                                    and as a Bank


                                    By  /s/ Christian Covello
                                      ------------------------------------------
                                        Vice President                     Title

                                    HARRIS TRUST AND SAVINGS BANK


                                    By  /s/ Edward P. McGuire
                                      ------------------------------------------
                                        Vice President                     Title

Agreed to and Accepted:

OMEGA HEALTHCARE INVESTORS, INC.


By  /s/ Susan A. Kovach
  -----------------------------------------
    Vice President                   Title















                                       4