<PAGE>   1
                                                                   Exhibit 10.81

THIS SECURITY HAS NOT BEEN REGISTERED UNDER THE SECURITIES ACT OF 1933, AS
AMENDED (THE "ACT"), OR APPLICABLE STATE SECURITIES LAWS AND MAY NOT BE
TRANSFERRED OR OTHERWISE DISPOSED OF UNLESS IT HAS BEEN REGISTERED UNDER THE ACT
AND SUCH LAWS OR (1) REGISTRATION UNDER APPLICABLE STATE SECURITIES LAWS IS NOT
REQUIRED AND (2) AN OPINION OF COUNSEL SATISFACTORY TO THE COMPANY IS FURNISHED
TO THE COMPANY TO THE EFFECT THAT REGISTRATION UNDER THE ACT IS NOT REQUIRED.

                                   B TERM NOTE


$5,000,000                                                     November 14, 1996


         FOR VALUE RECEIVED, NEXTHEALTH, INC., a Delaware corporation (the
"Borrower"), hereby promises to pay to the order of AP LOM LLC (the "Lender"),
in lawful money of the United States of America in immediately available funds,
at Lender's office located at 1301 Avenue of the Americas, 38th Floor, New York,
New York 10019, or at such other office as may be designated, from time to time,
by the holder hereof in writing, on the Maturity Date the principal sum of FIVE
MILLION DOLLARS ($5,000,000), or, if less, the unpaid principal amount of the B
Term Loans made by the Lender pursuant to the Credit Agreement. Unless otherwise
defined herein, capitalized terms used herein shall have the meanings ascribed
to such terms in the Credit Agreement referred to below.

         The Borrower promises also to pay interest on the unpaid principal
amount hereof in like money at said office from the date hereof until paid at
the rates and at the times provided in Section 2.4 of the Credit Agreement.

         This Note is the B Term Note referred to in the Credit Agreement, dated
as of November 14, 1996 (as amended, modified or supplemented from time to time,
the "Credit Agreement"), between the Borrower and the Lender and is entitled to
the benefits thereof and of the other Loan Documents. This Note is secured by
the Mortgage, the Security Agreement and the Pledge Agreement and is entitled to
the benefits of the Guaranty. This Note is subject to voluntary prepayment and
mandatory repayment, in whole or in part, prior to the Maturity Date as provided
in the Credit Agreement.

         Lender shall, and is hereby authorized by Borrower to, endorse on the
Schedule annexed to this Note, or otherwise record in Lender's internal records,
an appropriate notation evidencing the date and amount of all principal
disbursed by the Lender hereunder, as well as the date and amount of each
repayment and prepayment hereunder; provided, however, that the failure of
Lender to make such a notation or any error in such notation shall not affect
the obligation of the Borrower under this Note. The Lender shall provide to the
Borrower on request from time to time copies of the Schedule annexed to this
Note showing all endorsements thereto at the time of each such request.


         In case an Event of Default shall occur and be continuing, the
principal of and accrued interest on this Note may become or be declared to be
due and payable in the manner and with the effect provided in the Credit
Agreement.


<PAGE>   2
         The Borrower hereby waives presentment, demand, protest, notice of
dishonor, notice of nonpayment, notice of protest and diligence in collection,
and assents to the terms hereof and to any extension or postponement of the time
for payment or any other indulgence in connection with this Note. The Borrower
consents that the property securing this Note, or any part of such security, may
be released, exchanged, added to or substituted for by the Lender, without in
any way modifying, altering, releasing, affecting or limiting its liability
hereunder or the liens of the Mortgage, Security Agreement and Pledge Agreement,
and further agrees that the Lender shall not be required to first institute any
suit, or to exhaust any of its remedies against the Borrower or any other person
or party liable or to become liable hereunder, in order to enforce payment of
this Note.

         This Note shall be construed in accordance with and governed by the
laws of the State of New York.



                                       NEXTHEALTH, INC.

(Seal)
                                       By: /s/ William T O'Donnell, Jr
                                          -----------------------------
                                       Name: William T O'Donnell, Jr
                                       Title: President


                                       -2-