<PAGE>   1
                                                                    EXHIBIT 10.4

                      SECOND AMENDMENT TO CREDIT AGREEMENT


         THIS SECOND AMENDMENT TO CREDIT AGREEMENT (this "Amendment"), dated as
of March 16, 2000, is entered into by and among EXTENDICARE HEALTH SERVICES,
INC., a Delaware corporation (the "Borrower"), each of the Persons identified as
a "Guarantor" on the signature pages hereto, each of the Persons identified as a
"Lender" on the signature pages hereto and BANK OF AMERICA, N.A., formerly
NationsBank, N.A. (in such capacity, the "Agent").

                                    RECITALS

         A.    The Borrower, the Guarantors, the Lenders and the Agent, are
party to that certain Credit Agreement dated as of November 26, 1997 (as
previously amended prior to the date hereof, the "Credit Agreement"). Unless
otherwise defined herein or the context otherwise requires, capitalized terms
used in this Amendment, including its preamble and recitals, have the meanings
provided in the Credit Agreement.

         B.    The Credit Parties have requested that the Required Lenders amend
the Credit Agreement.

         C.    The Required Lenders have agreed to amend the Credit Agreement as
set forth herein.

         NOW, THEREFORE, in consideration of the agreements herein contained,
the parties hereto hereby agree as follows:

         1.    Amendments.

         (a)   The following definitions appearing in Section 1.1 of the Credit
Agreement are hereby amended and restated in their entireties to read as
follows:

               "Adjusted Senior Funded Indebtedness" means, at any time, the sum
         of (i) total Funded Indebtedness (other than Subordinated Indebtedness)
         of the Consolidated Parties on a consolidated basis plus (ii) Adjusted
         Consolidated Rental Expense plus (iii) future cash payments requested
         or required by a Fiscal Intermediary to be made by the Consolidated
         Parties to such Fiscal Intermediary related to (a) amounts in dispute
         between the Consolidated Parties and such Fiscal Intermediary, (b)
         amounts on appeal with the PRRB or (c) amounts in the process of being
         appealed to the PRRB.

               "Adjusted Total Funded Indebtedness" means, at any time, the sum
         of (i) total Funded Indebtedness (including without limitation
         Subordinated Indebtedness) of the Consolidated Parties on a
         consolidated basis plus (ii) Adjusted Consolidated Rental Expense plus
         (iii) future cash payments requested or required by a Fiscal
         Intermediary to
<PAGE>   2

         be made by the Consolidated Parties to such Fiscal Intermediary related
         to (a) amounts in dispute between the Consolidated Parties and such
         Fiscal Intermediary, (b) amounts on appeal with the PRRB or (c) amounts
         in the process of being appealed to the PRRB.

               "Consolidated EBITDA" means, for any period, the sum of (i)
         Consolidated Net Income for such period, plus (ii) an amount which, in
         the determination of Consolidated Net Income for such period, has been
         deducted for (A) Consolidated Interest Expense, (B) total federal,
         state, local and foreign income, value added and similar taxes, (C)
         depreciation and amortization expense, (D) provisions (not to exceed an
         aggregate amount of $30,000,000) recorded as deductions from revenue
         relating to issues identified in cost reports filed for cost reporting
         periods through December 31, 1998 (including the 1995 base year cost
         report upon which subsequent prospective rates are determined) which
         are related to amounts (1) in dispute between the Consolidated Parties
         and a Fiscal Intermediary, (2) on appeal to the PRRB or (3) in the
         process of being appealed to the PRRB, (E) provisions expensed during
         such period for estimated future payments of general and professional
         liability litigation claims, provided that such amounts do not exceed
         an aggregate amount of (I) $55,000,000 during fiscal year 2000 and (II)
         an amount set forth in the annual budget of the Consolidated Parties
         delivered pursuant to Section 7.1(d) and approved by the Required
         Lenders for each fiscal year thereafter, and (F) all other non-cash
         charges, all as determined in accordance with GAAP; provided, however,
         (x) Consolidated EBITDA shall not include revenues related to any
         recoveries by the Consolidated Parties from Fiscal Intermediaries of
         any amounts described in subclause (D) of clause (ii) above and (y) to
         the extent not otherwise deducted from Consolidated EBITDA,
         Consolidated EBITDA for any period shall be reduced by cash payments
         made by the Consolidated Parties of general and professional liability
         litigation claims during such period.

         (b)   The following new definitions are hereby added to Section 1.1 of
the Existing Credit Agreement in the appropriate alphabetical order to read as
follows:

               "Fiscal Intermediary" means (i) any contractor of the U.S.
         Department of Health and Human Services Health Care Financing
         Administration responsible for administration of the Federal Medicare
         Program or (ii) any state agency or other contractor responsible for
         administration of the Federal Medicaid Program.

               "PRRB" means the Provider Reimbursement Review Board established
         by the U.S. Congress and which is responsible for resolving Medicare
         payment disputes involving health care providers.

         (c)   Amendment to Section 2.1. The last sentence of Section 2.1(a) of
the Existing Credit Agreement is hereby amended and restated in its entirety to
read as follows:

Notwithstanding anything to the contrary contained herein, the Borrower shall
maintain at least $25,000,000 of undrawn availability under the Revolving
Committed Amount which may only be accessed with the approval of the Required
Lenders.

                                       2
<PAGE>   3

         (d)   Amendment to Section 7.1. Clause (l) of the Existing Credit
Agreement is hereby reidentified as clause (n), and new clauses (l) and (m) are
hereby added to Section 7.1 of the Existing Credit Agreement to read as follows:

               (l)  Reimbursement Requests. Immediately upon receipt by a
         Consolidated Party of a notice, request or demand from a Fiscal
         Intermediary for recoupment, offset, payment or repayment by any of the
         Consolidated Parties to such Fiscal Intermediary, the Borrower shall
         notify the Agent and deliver a copy of such request to the Agent.

               (m)  Monthly Financial Statements. Upon the request of any
         Lender, the Borrower shall deliver to such Lender within 20 days of
         such request, internally prepared financial statements for the fiscal
         month most recently ended.

         2.    Effective Date. This Amendment shall be and become effective when
all of the following conditions shall have been satisfied:

               (a)  The Agent shall have received executed counterparts (or
         other evidence of execution, including facsimile signatures,
         satisfactory to the Agent) of this Amendment, which collectively shall
         have been duly executed on behalf of each of the Credit Parties and the
         Required Lenders; and

               (b)  The Agent shall have received, for the account of each
         Lender approving this Amendment on or before 5:00 P.M., EST, March 16,
         2000, an amendment fee equal to 25 basis points on such Lender's
         Commitments.

         3.    Construction. This Amendment is a Credit Document executed
pursuant to the Credit Agreement and shall (unless otherwise expressly indicated
therein) be construed, administered and applied in accordance with the terms and
provisions of the Credit Agreement.

         4.    References. At such time as this Amendment shall become effective
pursuant to the terms of paragraph 2 above, all references in the Credit
Documents to the "Credit Agreement" shall be deemed to refer to the Credit
Agreement as amended by this Amendment.

         5.    Representations and Warranties. Each Credit Party hereby
represents and warrants that (i) each Credit Party that is party to this
Amendment: (a) has the requisite corporate power and authority to execute,
deliver and perform this Amendment, as applicable and (b) is duly authorized to,
and has been authorized by all necessary corporate action, to execute, deliver
and perform this Amendment, (ii) the representations and warranties contained in
Section 6 of the Credit Agreement are true and correct in all material respects
on and as of the date hereof upon giving effect to this Amendment as though made
on and as of such date (except for those which expressly relate to an earlier
date) and (iii) no Default or Event of Default exists under the Credit Agreement
on and as of the date hereof upon giving effect to this Amendment.

                                       3
<PAGE>   4

         6.    Acknowledgment. The Guarantors acknowledge and consent to all of
the terms and conditions of this Amendment and agree that this Amendment does
not operate to reduce or discharge the Guarantors' obligations under the Credit
Agreement (as amended by this Amendment) or the other Credit Documents. The
Guarantors further acknowledge and agree that the Guarantors have no claims,
counterclaims, offsets, or defenses to the Credit Documents and the performance
of the Guarantors' obligations thereunder or if the Guarantors did have any such
claims, counterclaims, offsets or defenses to the Credit Documents or any
transaction related to the Credit Documents, the same are hereby waived,
relinquished and released in consideration of the Lenders' execution and
delivery of this Amendment.

         7.    Counterparts. This Amendment may be executed by the parties
hereto in several counterparts, each of which shall be deemed to be an original
and all of which shall constitute together but one and the same agreement.

         8.    Binding Effect. This Amendment, the Credit Agreement and the
other Credit Documents embody the entire agreement between the parties and
supersede all prior agreements and understandings, if any, relating to the
subject matter hereof. These Credit Documents represent the final agreement
between the parties and may not be contradicted by evidence of prior,
contemporaneous or subsequent oral agreements of the parties. Except as
expressly modified and amended in this Amendment, all the terms, provisions and
conditions of the Credit Documents shall remain unchanged and shall continue in
full force and effect.

         9.    GOVERNING LAW. THIS AMENDMENT AND THE RIGHTS AND OBLIGATIONS OF
THE PARTIES HEREUNDER SHALL BE GOVERNED BY AND CONSTRUED AND INTERPRETED IN
ACCORDANCE WITH THE LAWS OF THE STATE OF NEW YORK.

                                       4
<PAGE>   5
         IN WITNESS WHEREOF, each of the parties hereto has caused a counterpart
of this Amendment to be duly executed and delivered as of the date first above
written.

BORROWER:                               EXTENDICARE HEALTH SERVICES, INC.,


                                        By:_________________________________
                                        Name: Mark W. Durishan
                                        Title: Vice President-Finance

GUARANTORS:                             EXTENDICARE HOLDINGS, INC.
                                        EXTENDICARE HEALTH FACILITY
                                         HOLDINGS, INC.
                                        EXTENDICARE HEALTH FACILITIES, INC.
                                        COVENTRY CARE, INC.
                                        NORTHERN HEALTH FACILITIES, INC.
                                        EXTENDICARE HOMES, INC.
                                        EXTENDICARE HEALTH NETWORK, INC.
                                        THE PROGRESSIVE STEP CORPORATION
                                        EXTENDICARE OF INDIANA, INC.
                                        UNITED REHABILITATION SERVICES, INC.
                                        EDGEWOOD NURSING CENTER, INC.
                                        ELDER CREST, INC.
                                        HAVEN CREST, INC.
                                        MEADOW CREST, INC.
                                        OAK HILL HOME OF REST AND CARE, INC.
                                        EXTENDICARE GREAT TRAIL, INC.
                                        FIR LANE TERRACE CONVALESCENT
                                         CENTER, INC.
                                        UNITED PROFESSIONAL SERVICES, INC.
                                        ADULT SERVICES UNLIMITED, INC.
                                        ARBORS EAST, INC.
                                        ARBORS AT TOLEDO, INC.
                                        HEALTH POCONOS, INC.
                                        MARSHALL PROPERTIES, INC.

                                        By:_________________________________
                                        Name: Mark W. Durishan
                                        Title: Vice President-Finance

                             [Signatures continue.]
                                      S-1
<PAGE>   6

                               INDIANA HEALTH AND REHABILITATION
                                CENTER PARTNERSHIP

                               By:   Extendicare Homes, Inc.,
                                      as General Partner


                                      By:_______________________________________
                                      Name: Mark W. Durishan
                                      Title: Vice President-Finance

                               By:   Extendicare of Indiana, Inc.,
                                      as General Partner


                                     By:________________________________________
                                     Name: Mark W. Durishan
                                     Title: Vice President-Finance

                               CONCORDIA MANOR, LLC
                               FIRST COAST HEALTH AND
                                REHABILITATION CENTER, LLC
                               JACKSON HEIGHTS REHABILITATION
                                CENTER, LLC
                               TREASURE ISLE CARE CENTER, LLC

                               By:   Extendicare Homes, Inc., as sole member


                                     By:________________________________________
                                     Name: Mark W. Durishan
                                     Title: Vice President-Finance


                               KAUFMAN STREET, WV, LLC
                               NEW CASTLE CARE, LLC

                               By:   Fir Lane Terrace Convalescent Center, Inc.,
                                     as sole member


                                     By:________________________________________
                                     Name: Mark W. Durishan
                                     Title: Vice President-Finance


                             [Signatures continue.]

                                      S-2
<PAGE>   7
                               ALPINE HEALTH AND REHABILITATION
                                CENTER, LLC
                               COLONIAL CARE, LLC
                               GREENBRIAR CARE, LLC
                               GREENBROOK CARE, LLC
                               HERITAGE CARE, LLC
                               LADY LAKE CARE, LLC
                               NEW HORIZON CARE, LLC
                               NORTH REHABILITATION CARE, LLC
                               PALM COURT CARE, LLC
                               RICHEY MANOR, LLC
                               ROCKLEDGE CARE, LLC
                               SOUTH HERITAGE HEALTH AND
                               REHABILITATION CENTER, LLC
                               THE OAKS RESIDENTIAL AND
                                REHABILITATION CENTER, LLC
                               WINTER HAVEN HEALTH AND
                                REHABILITATION CENTER, LLC

                               By:   Extendicare Health Facilities, Inc.,
                                     as sole member


                                     By:________________________________________
                                     Name: Mark W. Durishan
                                     Title: Vice President-Finance


                             [Signatures continue.]

                                      S-3
<PAGE>   8
                               ARBORS AT BAYONET POINT,
                               LLC ARBORS AT TAMPA, LLC
                               JACKSONVILLE CARE, LLC
                               KISSIMMEE CARE, LLC ORANGE
                               PARK CARE, LLC PORT
                               CHARLOTTE CARE, LLC SAFETY
                               HARBOR CARE, LLC SARASOTA
                               CARE, LLC SEMINOLE CARE,
                               LLC WINTER HAVEN CARE, LLC
                               BLANCHESTER CARE, LLC
                               CANTON CARE, LLC COLUMBUS
                               HEALTH CARE LLC DAYTON
                               CARE, LLC DELAWARE CARE,
                               LLC FAIRLAWN CARE, LLC
                               GALLIPOLIS CARE, LLC
                               HILLIARD CARE, LLC LONDON
                               CARE, LLC MARIETTA CARE,
                               LLC TOLEDO CARE, LLC
                               WATERVILLE CARE, LLC WEST
                               JEFFERSON CARE, LLC

                               By:   Northern Health Facilities, Inc.,
                                     as sole member


                                     By:________________________________________
                                     Name: Mark W. Durishan
                                     Title: Vice President-Finance

                          [Lenders' signatures follow.]

                                      S-4
<PAGE>   9


LENDERS:                       BANK OF AMERICA, N.A.,
                               formerly NationsBank, N.A.,
                               individually in its capacity as a
                               Lender and in its capacity as Agent

                               By:_________________________________
                               Name:_______________________________
                               Title:______________________________


                               ROYAL BANK OF CANADA

                               By:_________________________________
                               Name:_______________________________
                               Title:______________________________


                               FIRSTAR BANK MILWAUKEE, N.A.

                               By:_________________________________
                               Name:_______________________________
                               Title:______________________________


                               CREDIT LYONNAIS NEW YORK BRANCH

                               By:_________________________________
                               Name:_______________________________
                               Title:______________________________


                               LEHMAN COMMERCIAL PAPER I


                               By:_________________________________
                               Name:_______________________________
                               Title:______________________________


                               DLJ CAPITAL FUNDING, INC.


                               By:_________________________________
                               Name:_______________________________
                               Title:______________________________

                                      S-5
<PAGE>   10
                               THE BANK OF NOVA SCOTIA

                               By:_________________________________
                               Name:_______________________________
                               Title:______________________________


                               KEY CORPORATE CAPITAL INC.

                               By:_________________________________
                               Name:_______________________________
                               Title:______________________________


                               LASALLE BANK NATIONAL ASSOCIATION

                               By:_________________________________
                               Name:_______________________________
                               Title:______________________________


                               TORONTO DOMINION (TEXAS), INC.

                               By:_________________________________
                               Name:_______________________________
                               Title:______________________________


                               BANK ONE, N.A.

                               By:_________________________________
                               Name:_______________________________
                               Title:______________________________


                               BANK ONE DAYTON

                               By:_________________________________
                               Name:_______________________________
                               Title:______________________________

                                      S-6
<PAGE>   11
                               BANK OF MONTREAL

                               By:_________________________________
                               Name:_______________________________
                               Title:______________________________


                               THE FUJI BANK, LTD., CHICAGO BRANCH

                               By:_________________________________
                               Name:_______________________________
                               Title:______________________________


                               U.S. BANK NATIONAL ASSOCIATION

                               By:_________________________________
                               Name:_______________________________
                               Title:______________________________


                               BANK OF TOKYO-MITSUBISHI
                                TRUST COMPANY

                               By:_________________________________
                               Name:_______________________________
                               Title:______________________________

                               COMERICA BANK

                               By:_________________________________
                               Name:_______________________________
                               Title:______________________________


                               AMSOUTH BANK

                               By:_________________________________
                               Name:_______________________________
                               Title:______________________________

                                      S-7
<PAGE>   12

                               VAN KAMPEN PRIME
                                RATE INCOME TRUST

                               By:_________________________________
                               Name:_______________________________
                               Title:______________________________

                               VAN KAMPEN SENIOR
                                INCOME TRUST

                               By:_________________________________
                               Name:_______________________________
                               Title:______________________________


                               THE PRUDENTIAL INSURANCE
                                COMPANY OF AMERICA

                               By:_________________________________
                               Name:_______________________________
                               Title:______________________________


                               WILLIAM E SIMON SONS

                               By:_________________________________
                               Name:_______________________________
                               Title:______________________________


                               BANKBOSTON, N.A.

                               By:_________________________________
                               Name:_______________________________
                               Title:______________________________


                               INDOSUEZ CAPITAL FUNDING III,
                               LIMITED

                               By: Indosuez Capital Luxembourg SA,
                                   as Collateral Manager

                                     By:_________________________________
                                     Name:_______________________________
                                     Title:______________________________

                                      S-8
<PAGE>   13


                               MARINER LDC


                               By:_________________________________
                               Name:_______________________________
                               Title:______________________________


                               PARIBAS CAPITAL FUNDING LLC

                               By:_________________________________
                               Name:_______________________________
                               Title:______________________________


                               CRESCENT/MACH I PARTNERS, L.P.

                               By: TCW Asset Management Company,
                                   its Investment Manager

                               By:_________________________________
                               Name:_______________________________
                               Title:______________________________


                               KZH CRESCENT LLC

                               By:_________________________________
                               Name:_______________________________
                               Title:______________________________


                               KZH CRESCENT-2 LLC

                               By:_________________________________
                               Name:_______________________________
                               Title:______________________________

                                      S-9
<PAGE>   14


                               ROYALTON COMPANY

                               By: Pacific Investment Management Company,
                                   as its Investment Advisor

                               By:_________________________________
                               Name:_______________________________
                               Title:______________________________


                               CAPTIVA III FINANCE LTD.


                               By:_________________________________
                               Name:_______________________________
                               Title:______________________________


                               JACKSON NATIONAL LIFE INSURANCE
                                COMPANY

                               By: PPM America, Inc., as attorney-in-fact,
                                   on behalf of Jackson National Life
                                   Insurance Company

                               By:_________________________________
                               Name:_______________________________
                               Title:______________________________


                               ARCHIMEDES FUNDING LLC

                               By:_________________________________
                               Name:_______________________________
                               Title:______________________________

                                      S-10
<PAGE>   15


                       SENIOR DEBT PORTFOLIO

                       By: By Boston Management and Research, as
                           Investment Advisor

                       By:_________________________________
                       Name:_______________________________
                       Title:______________________________


                       EATON VANCE INSTITUTIONAL SENIOR
                       LOAN FUND

                       By: Eaton Vance Management, as
                           Investment Advisor


                       By:_________________________________
                       Name:_______________________________
                       Title:______________________________


                       CYPRESSTREE INVESTMENT PARTNERS I, LTD.
                       By: CypressTree Investment Management Company, Inc., as
                       Portfolio Manager

                       By:_________________________________
                       Name:_______________________________
                       Title:______________________________


                       JOHN HANCOCK MUTUAL LIFE

                       By:_________________________________
                       Name:_______________________________
                       Title:______________________________


                       JOHN HANCOCK VARIABLE LIFE

                       By:_________________________________
                       Name:_______________________________
                       Title:______________________________

                                      S-11
<PAGE>   16

                               NATIONAL CITY BANK

                               By:_________________________________
                               Name:_______________________________
                               Title:______________________________


                               CITY NATIONAL BANK

                               By:_________________________________
                               Name:_______________________________
                               Title:______________________________

                                      S-12