<PAGE>   1

                                                                    EXHIBIT 10.4

                         GUARANTY OF PAYMENT AGREEMENT

         THIS GUARANTY OF PAYMENT AGREEMENT (this "Agreement") is made this
28th day of September, 1998, by ALTERNATIVE LIVING SERVICES, INC., a Delaware
corporation, (the "Guarantor") for the benefit of BANK UNITED ("Agent") for
itself and for such additional lenders (collectively with Agent, the "Lenders")
who may from time to time  agree to act as lenders in the Credit Facility (as
hereinafter defined).

                                    RECITALS

         A.      The Borrower (as defined in the Financing Agreement) has
obtained from the Agent a revolving credit facility (such credit facility being
hereinafter referred to as the "Credit Facility" or the "Loan") in the maximum
principal sum of up to $45,000,000 or such greater amount as the Lenders may
from time to time commit to lend pursuant to an Agency Agreement which may be
executed in the future, not to exceed $100,000,000.  Advances or readvances
under the Credit Facility are to be made pursuant to, and secured by, the
provisions of that certain Financing and Security Agreement dated the same date
as this Agreement by and between the Agent and the Borrower (as amended,
restated or substituted from time to time, the "Financing Agreement").

         B.      The Loan will be evidenced by one or more Promissory Notes (as
amended, restated, renewed or resubstituted from time to time, whether one or
more, collectively the "Note") payable to the Agent on behalf of the respective
Lenders.  The Note will be secured by, among other things, certain Deeds of
Trust (as defined in the Financing Agreement) from a Borrower in favor of the
Lenders covering the Borrower's interest in the Land and the Improvements for
the applicable Facility (as defined hereinafter) and such other real and
personal property as shall be therein more particularly set forth
(collectively, the "Property").  The Credit Facility is evidenced, secured and
guaranteed by certain Financing Documents (as defined in the Financing
Agreement).

         C.      The Guarantor has requested the Lender to enter into the
Financing Agreement with the Borrower and to make the Loan to the Borrower
pursuant thereto.

         D.      The Lender has required, as a condition to the making of the
Loan, that the Guarantor execute and deliver this Agreement to the Lender.

         E.      All defined terms used in this Agreement and not defined
herein shall have the meaning given to such terms in the Financing Agreement.

         NOW, THEREFORE,  in order to induce the Lender to make the Loan to the
Borrower, the Guarantor covenants and agrees with the Lender as follows:
<PAGE>   2


                                   ARTICLE 1
                                  THE GUARANTY

         Section 1.1      Guaranty.

                 The Guarantor hereby unconditionally and irrevocably
guarantees to the Lenders:

                                  (a)      the due and punctual payment in full
(and not merely the collectibility) of the principal of the Note and the
interest thereon, in each case when due and payable, whether on any installment
payment date or at the stated or accelerated maturity, all according to the
terms of the Note and the other Financing Documents;
                                  (b)      the due and punctual payment in full
(and not merely the collectibility) of all Obligations and other sums and
charges which may at any time be due and payable in accordance with, or secured
by, the Note or any of the other Financing Documents;

                                  (c)      the due and punctual performance of
all of the other terms, covenants and conditions contained in the Financing
Documents; and

                                  (d)      all indebtedness, obligations and
liabilities of any kind and nature of the Borrower to the Lenders, whether now
existing or hereafter created or arising, direct or indirect, matured or
unmatured, and whether absolute or contingent, joint, several or joint and
several, and howsoever owned, held or acquired.

         Section 1.2      Guaranty Unconditional.

         The obligations and liabilities of the Guarantor under this Agreement
shall be absolute and unconditional, irrespective of the genuineness, validity,
priority, regularity or enforceability of the Note or any of the Financing
Documents or any other circumstance which might otherwise constitute a legal or
equitable discharge of a surety or guarantor.  The Guarantor expressly agrees
that the Lenders may, in their sole and absolute discretion, without notice to
or further assent of the Guarantor and without in any way releasing, affecting
or in any way impairing the obligations and liabilities of the Guarantor
hereunder:

                                  (a)      waive compliance with, or any
defaults under, or grant any other indulgences under or with respect to any of
the Financing Documents;

                                  (b)      modify, amend, change or terminate
any provisions of any of the Financing Documents;

                                  (c)      grant extensions or renewals of or
with respect to the Note or any of the other Financing Documents;

                                  (d)      effect any release, subordination,
compromise or settlement in connection with the Note or any of the other
Financing Documents;

                                      2
<PAGE>   3

                                  (e)      agree to the substitution, exchange,
release or other disposition of the Collateral or any part thereof, or any
other collateral for the Credit Facility or to the subordination of any lien or
security interest therein;

                                  (f)      make advances for the purpose of
performing any term, provision or covenant contained in the Note or any of the
other Financing Documents with respect to which the Borrower shall then be in
default;

                                  (g)       make future advances to the
Borrower pursuant to the Financing Agreement or any of the other Financing
Documents;

                                  (h)      assign, pledge, hypothecate or
otherwise transfer the Note, any of the other Financing Documents or this
Agreement or any interest therein;

                                  (i)      deal in all respects with the
Borrower as if this Agreement were not in effect; and

                                  (j)      effect any release, compromise or
settlement with any of the Guarantor or any other guarantor.

         Section 1.3      Guaranty Primary.

           The obligations and liabilities of the Guarantor under this
Agreement shall be primary, direct and immediate, shall not be subject to any
counterclaim, recoupment, set off, reduction or defense based upon any claim
that the Guarantor may have against the Borrower, the Lenders and/or any other
guarantor and shall not be conditional or contingent upon pursuit or
enforcement by the Lenders of any remedies they may have against the Borrower
with respect to the Note or any of the other Financing Documents, whether
pursuant to the terms thereof or by operation of law.  Without limiting the
generality of the foregoing and except to the extent any applicable notice
and/or opportunity to cure is required under the Financing Documents, the
Lenders shall not be required to make any demand upon the Borrower or to sell
the Collateral or otherwise pursue, enforce or exhaust their remedies against
the Borrower or the Collateral either before, concurrently with or after
pursuing or enforcing their rights and remedies hereunder.  Any one or more
successive or concurrent actions or proceedings may be brought against the
Guarantor under this Agreement, either in the same action, if any, brought
against the Borrower or in separate actions or proceedings, as often as the
Lenders may deem expedient or advisable.  Without limiting the foregoing, it is
specifically understood that any modification, limitation or discharge of any
of the liabilities or obligations of the Borrower or any other obligor under
any of the Financing Documents, arising out of, or by virtue of, any
bankruptcy, arrangement, reorganization or similar proceeding for relief of
debtors under federal or state law initiated by or against the Borrower or the
Guarantor or any obligor under any of the Financing Documents shall not modify,
limit, lessen, reduce, impair, discharge, or otherwise affect the liability of
the Guarantor hereunder in any manner whatsoever, and this Agreement shall
remain and continue in full force and effect.  It is the intent and purpose of
this Agreement that the Guarantor shall and does hereby waive all rights and
benefits which might accrue to any other guarantor by reason of any such
proceeding, and the Guarantor





                                       3
<PAGE>   4

agrees that it shall be liable for the full amount of the obligations and
liabilities under this Agreement, regardless of, and irrespective to, any
modification, limitation or discharge of the liability of the Borrower, any
other guarantor or any obligor under any of the Financing Documents, that may
result from any such proceedings.

         Section 1.4      Certain Waivers by the Guarantor.

         Except with respect to any applicable notice and/or opportunity to
cure required to be given under the Financing Documents, the Guarantor hereby
unconditionally, irrevocably and expressly waives:

                                  (a)      presentment and demand for payment
of the principal of or interest on the Note and protest of non-payment;

                                  (b)      notice of acceptance of this
Agreement and of presentment, demand and protest thereof;

                                  (c)      notice of any default hereunder or
under the Note or any of the other Financing Documents and notice of all
indulgences;

                                  (d)      notice of any increase in the amount
of any portion of or all of the indebtedness guaranteed by this Agreement;

                                  (e)      demand for observance, performance
or enforcement of any of the terms or provisions of this Agreement, the Note or
any of the other Financing Documents;

                                  (f)      all errors and omissions in
connection with the Lenders' administration of all indebtedness guaranteed by
this Agreement, except errors and omissions resulting from acts of bad faith or
gross negligence;

                                  (g)      any right or claim of right to cause
a marshalling of the assets of the Borrower;

                                  (h)      any act or omission of the Lenders
(except acts or omissions in bad faith or constituting gross negligence) which
changes the scope of the Guarantor's risk hereunder; and

                                  (i)      all other notices and demands
otherwise required by law which the Guarantor may lawfully waive.

         Section 1.5      Reimbursement for Expenses.

         In the event the Lenders shall commence any action or proceeding for
the enforcement of this Agreement, then the Guarantor will reimburse the
Lenders, promptly upon demand, for any and all expenses incurred by the Lenders
in connection with such action or proceeding including, without limitation,
reasonable, actual attorneys' fees together with interest thereon at the
Post-Default Rate.





                                       4
<PAGE>   5

         Section 1.6      Events of Default.

         The occurrence of any one or more of the following events shall
constitute an "Event of Default" under the provisions of this Agreement
(individually, an "Event of Default" and collectively, the "Events of
Default"):

                                  (a)      The failure of the Guarantor to pay
and/or perform any of the Obligations within five (5) calendar days of the date
as and when due and payable and/or perform any Obligations when required in
accordance with the provisions of this Agreement.

                                  (b)      Any representation or warranty made
in this Agreement or in any report, statement, schedule, certificate, opinion
(including any opinion of counsel for the Guarantor), financial statement or
other document furnished in connection with this Agreement, shall prove to have
been false or misleading when made (or, if applicable, when reaffirmed) in any
material respect.

                                  (c)      The failure of the Guarantor to
comply with Section 3.2 and 3.3 hereof.

                                  (d)      The failure of the Guarantor to
perform, observe or comply with any non- monetary covenant, condition or
agreement of Guarantor contained in this Agreement other than as set forth in
this Section, which default shall remain unremedied for more than thirty (30)
days after written notice thereof to the Guarantor by the Agent, unless the
nature of the failure is such that (a) it cannot be cured within the thirty
(30) day period, and (b) the Guarantor institutes corrective action within the
thirty (30) day period and (c) the Guarantor diligently pursues such action and
completes the cure within ninety (90) days.

                                  (e)      A default shall occur under any of
the other Financing Documents and such default is not cured within any
applicable grace period provided therein.

                                  (f)      The Guarantor shall (i) apply for or
consent to the appointment of a receiver, trustee or liquidator of itself or
any of its property, (ii) admit in writing its inability to pay its debts as
they mature, (iii) make a general assignment for the benefit of creditors, (iv)
be adjudicated a bankrupt or insolvent, (v) file a voluntary petition in
bankruptcy or a petition or an answer seeking or consenting to reorganization
or an arrangement with creditors or to take advantage of any bankruptcy,
reorganization, insolvency, readjustment of debt, dissolution or liquidation
law or statute, or an answer admitting the material allegations of a petition
filed against it in any proceeding under any such law, or take corporate action
for the purposes of effecting any of the foregoing, or (vi) by any act indicate
its consent to, approval of or acquiescence in any such proceeding or the
appointment of any receiver of or trustee for any of its property, or suffer
any such receivership, trusteeship or proceeding to continue undischarged for a
period of sixty (60) days, or (vii) by any act indicate its consent to,
approval of or acquiescence in any order, judgment or decree by any court of
competent jurisdiction or any Governmental Authority





                                       5