<PAGE>   1

                                                                   EXHIBIT 10.77

                 First Amendment, dated December 3, 1999 to the
                   Fourth Amended and Restated Loan Agreement
            by and among the Registrant, certain of its subsidiaries
                         and Comerica Bank - California



<PAGE>   2

                        FIRST AMENDMENT TO FOURTH AMENDED
                           AND RESTATED LOAN AGREEMENT


        THIS FIRST AMENDMENT TO FOURTH AMENDED AND RESTATED LOAN AGREEMENT
("Amendment") is made and entered into as of November __, 1999 by and among: (a)
The Sports Club Company, Inc., a Delaware corporation; The Spectrum Club
Company, Inc., a California corporation; Pontius Realty, Inc., a California
corporation; The SportsMed Company, Inc., a California corporation; LA/Irvine
Sports Clubs, Ltd., a California limited partnership; Talla New York, Inc., a
New York corporation; SCC Sports Club, Inc., a Texas corporation; Canoga Agoura
Spectrum Club, Inc., a California corporation; Irvine Sports Club, Inc., a
California corporation; Green Valley Spectrum Club, Inc., a Nevada corporation;
Sports Club, Inc. of California, a California corporation; Spectrum Liquidating
Corp., a California corporation f/k/a Spectrum Club/Anaheim Hills, Inc.; TVE,
Inc., a California corporation; Spectrum Club Anaheim, a California Corporation;
SF Sports Club, Inc., a Delaware corporation; Washington D.C. Sports Club, Inc.,
a Delaware corporation; HFA Services, Inc., a California corporation; and NY
Sports Club, Inc., a Delaware corporation (collectively, the "Borrowers"); (b)
Comerica Bank-California ("Comerica"), as the Bank; and (c) Comerica
Bank-California, as agent (in such capacity, the "Agent") for the banks (the
"Banks") that are parties to the Loan Agreement (as defined below), and is made
with reference to the following:

        A. Comerica, the Agent and the Borrowers have entered into that certain
Fourth Amended and Restated Loan Agreement, dated as of April 1, 1999. (Such
Loan Agreement, as so amended and as the same may hereafter be amended,
modified, extended and/or restated, is hereinafter referred to as the "Loan
Agreement".) Pursuant to the Loan Agreement, the Bank has made certain Loans to
the Borrowers and has committed to make additional Loans in the future upon the
satisfaction of certain conditions.

        B. The Borrowers, the Agent and the Bank wish to amend the Loan
Agreement as more particularly set forth below.

        C. The Borrowers have also requested the consent of the Agent and the
Bank set forth in Section 2 of this Amendment and the waiver set forth in
Section 7 of this Amendment.

        NOW, THEREFORE, in consideration of the premises and the agreements,
conditions and covenants contained herein, the parties hereby agree as follows:

                                 Defined Terms.

                "Effective Date" shall mean the date on which each of the
conditions precedent set forth in Section 6 of this Amendment is satisfied.

                All capitalized terms used in this Amendment and not otherwise
defined herein shall have the meanings assigned to them in the Loan Agreement.
Consent of the Agent and the Bank to the Sale of Certain Assets. Effective as of
the Effective Date, the Agent and the Bank hereby consent to the sale by the
Borrowers party thereto, pursuant to the Stock Purchase Agreement, dated as of
September 16, 1999, by and among Racquetball and Fitness Clubs, Inc., a Texas
corporation, The Spectrum Club Company, Inc., a California



<PAGE>   3

corporation, Canoga/Agoura Spectrum Club, Inc., a California corporation,
Spectrum Club Anaheim, a California corporation, El Segundo-TDC, Ltd., a
California limited partnership, TVE, Inc., a California corporation, and The
Sports Club Company, Inc., a Delaware corporation (the "Stock Purchase
Agreement"), of the Assets (but not of the Excluded Assets), in each case as
defined in the Asset Purchase Agreement. Notwithstanding the generality of the
foregoing sentence, the consent of the Agent and the Bank set forth therein do
not extend to any asset that is located at The Sports Club/Irvine or The Sports
Club/Las Vegas (or is reasonably related to the operation of one or more of such
facilities, but is not also reasonably related to the operation of any other
facility).

Amendments to Loan Agreement. Effective as of the Effective Date, the
Loan Agreement is hereby amended in the following respects:

Reduction of Loan Commitment.

The definition of "Commitment" in Section 1.1 of the Loan Agreement is hereby
amended to read in full as follows:

                "`Commitment' means, collectively, the lending commitment
        hereunder of the Banks, as such commitment may be reduced or offset
        under this Agreement. The percentage obligations of each Bank with
        respect to the Commitment are as follows:

<TABLE>
<CAPTION>
                  Bank                                 Amount               Percentage
                  ----                                 ------               ----------
<S>                                                  <C>                    <C>
         Comerica Bank - California                  $15,000,000                100%"
</TABLE>

Section 2.1(a) of the Loan Agreement is hereby amended to read in full as
follows:

                "(a) Subject to the terms and conditions set forth in this
        Agreement, at any time and from time to time prior to the Maturity Date,
        each Bank shall, pro rata according to that Bank's percentage of the
        then Commitment, make Loans to Borrowers in such amounts as Borrowers
        may request that do not exceed in the aggregate at any one time
        outstanding the amount of the Commitment; provided that, Banks shall not
        be obligated to make a Loan if, after giving effect to such Loan, the
        Total Outstanding would exceed $15,000,000. Except as may otherwise be
        payable on an earlier date as provided in Section 3.1, all Obligations
        of Borrowers hereunder shall be due and payable on the Maturity Date.
        Subject to the limitations set forth herein and in Section 3.1(e),
        Borrowers may borrow, repay and reborrow under the Commitment without
        premium or penalty."

Section 2.6(a) of the Loan Agreement is hereby amended to read in full as
follows:

                "(a) Subject to the terms and conditions hereof, at any time and
        from time to time from the Closing Date through the Banking Day
        immediately preceding May 31, 2001 or other applicable Maturity Date,
        the Issuing Bank shall issue such Standby Letters of Credit as a
        Responsible Official of a Borrower on behalf of Borrowers may request by
        a Request for Standby Letter of Credit; provided that, upon giving
        effect to such Standby Letter of Credit, (i) Total Outstanding shall not
        exceed $15,000,000 and (ii) Outstanding Standby Letters of Credit shall
        not exceed the Maximum Standby Letter of Credit Amount. Unless


<PAGE>   4

        the Requisite Banks otherwise consent in writing, the term of any
        Standby Letter of Credit shall not exceed the Maturity Date. If on the
        Maturity Date, there exist any Outstanding Standby Letters of Credit,
        Borrowers shall provide to Agent a standby letter of credit issued by a
        bank satisfactory to the Requisite Banks, in form and substance
        satisfactory to the Requisite Banks, in favor of Banks in a face amount
        equal to the Outstanding Standby Letters of Credit on that date, or
        shall make other provisions satisfactory to the Requisite Banks for the
        collateralization or settlement of such Outstanding Standby Letters of
        Credit. No Standby Letter of Credit shall be issued except in the
        ordinary course of business of Borrowers or their Subsidiaries. Unless
        otherwise agreed to by the Requisite Banks, the face amount of any
        Standby Letter of Credit shall not be less than $250,000."

Section 2.6(e) of the Loan Agreement is hereby amended to read in full as
follows:

                "(e) At all times prior to the Maturity Date, if Borrowers fail
        to make any payment required by Section 2.6(d), Agent may, but is not
        required to, without notice to or the consent of Borrowers, make Loans
        under the Commitment in an aggregate amount equal to the amount paid by
        the Issuing Bank on the relevant Standby Letter of Credit, whether or
        not the same would cause the Commitment to exceed $15,000,000, and, for
        this purpose, the conditions precedent set forth in Article 8 and the
        amount limitations set forth in Section 2.1(d) shall not apply. The
        proceeds of such Loans shall be retained by the Issuing Bank to
        reimburse it for the payment made by it under the Standby Letter of
        Credit."


Irvine Environmental Resolution.

The definition of "Irvine Environmental Resolution" in Section 1.1 of the Loan
Agreement is deleted in its entirety.

The words "; and" in the final line of Section 8.2(g) of the Loan Agreement are
replaced with "." and Section 8.2(h) is deleted in its entirety.

Section 8.4 of the Loan Agreement is deleted in its entirety.

Amendments to Certain Financial Covenants.

The definition of "Debt Service Coverage Ratio" in Section 1.1 of the Loan
Agreement is amended to read in full as follows:

        "`Debt Service Coverage Ratio' means, with respect to each applicable
fiscal period, the ratio of (i) EBITDA plus or minus, without double counting,
all non-recurring items of income or expense, plus Start-Up Revenue for that
fiscal period, to (ii) interest expense on all indebtedness, including
capitalized interest, plus the current portion of long term debt of Borrowers
for that fiscal period, minus interest income for that fiscal period, determined
in accordance with generally accepted accounting principles, consistently
applied. The Debt Service Coverage Ratio shall be determined as of the end of
each fiscal quarter of the Borrowers and their Subsidiaries, computed on a
rolling four quarter basis."


<PAGE>   5

The following new definition is hereby added to Section 1.1 of the Loan
Agreement in proper alphabetical order:

        "`Start-Up Revenue' means, for any fiscal period, for Borrowers, their
Subsidiaries, and Sports Connection-ES/MB, to the extent of Borrowers' interest
therein, all revenues received in respect of a New Club Development prior to the
opening of the related new Club and treated as a deferred liability in
accordance of generally accepted accounting principles, consistently applied."

                (a) Section 6.8 of the Loan Agreement is amended by deleting the
number "$20,200,000" in the ninth line thereof and substituting therefor the
number "$24,500,000."

                (b) Section 6.12 of the Loan Agreement is amended by adding the
words "beginning in the fiscal year commencing January 1, 2000" after the words
"in any fiscal year" in the third line thereof and by deleting the words "five
percent (5%)" in the third line thereof and substituting therefor the words "six
percent (6%)".

Section 6.15 of the Loan Agreement is hereby deleted and replaced in its
entirety with the following:

                "6.15 Debt Service Coverage Ratio. For Borrowers and their
        Subsidiaries, permit the Debt Service Coverage Ratio to be less than (a)
        1.35:1.00 for any fiscal quarter to and including the fiscal quarter
        ending September 30, 2000; and (b) 1.75:1.00 for any fiscal quarter
        ending thereafter, with such ratio to be calculated at the end of each
        such fiscal quarter, on a rolling four quarter basis."

Release of Canoga Agoura Spectrum Club, Inc. as Collateral.

The following definitions in Section 1.1 of the Loan Agreement are hereby
amended to read in full as follows:

                "`Deeds of Trust' means, collectively, the Deed of Trust,
        Security Agreement and Fixture Filing (With Assignment of Rents and
        Leases) executed by each of Irvine Sports Club, Inc. and Green Valley
        Spectrum Club, Inc. in favor of Agent for the ratable benefit of Banks,
        in the forms of those attached hereto as Exhibits B-1 and B-3, as the
        same may from time to time hereafter be supplemented, modified, amended,
        restated or extended."

                "`Security Agreements (All Assets)' means, collectively, the
        Security Agreement (All Assets) executed by each of Irvine Sports Club,
        Inc. and Green Valley Spectrum Club, Inc. in favor of Agent, for the
        ratable benefit of Banks, in the forms of those attached hereto as
        Exhibits H-1 and H-3, as the same may from time to time hereafter be
        supplemented, modified, amended, restated or extended."

The definitions of "The Spectrum Club/Agoura Hills" and "The Spectrum
Club/Canoga Park" in Section 1.1 of the Loan Agreement are deleted in their
entirety.


<PAGE>   6

Sections 4.5(b), 5.11, 6.16, 6.20 and 11.11 are amended to delete any and all
occurrences of the terms "Canoga Agoura Spectrum Club, Inc.," "The Spectrum
Club/Agoura Hills" and "The Spectrum Club/Canoga Park."

Exhibits B-2: Form of Deed of Trust (Canoga Agoura) and H-2: Form of Security
Agreement (All Assets) (Canoga Agoura) are deleted in their entirety.

Establishment of Accounts. Section 5.11 of the Loan Agreement is amended by
deleting the words "ninety (90) days following the execution of this Agreement"
in the first and second lines thereof and substituting therefor the words
"December 31, 1999."

General Amendment. Effective as of the Effective Date, the Loan Agreement and
all other Loan Documents are hereby amended to the further extent required to
give effect to the terms and conditions of the amendments to the Loan Agreement
effected pursuant to Section 3 above. In furtherance of the foregoing, the legal
descriptions of Property 1 and Property 2 are deleted as of the Effective Date
from Exhibit "A" to the Environmental Indemnity.

Full Force and Effect. Effective as of the Effective Date, each of the Loan
Documents is hereby amended such that all references to the Loan Agreement
contained in any such documents shall be deemed to be references to the Loan
Agreement, as amended by this Amendment. Except as amended hereby, the Loan
Agreement and the other Loan Documents shall remain unaltered and in full force
and effect.

Conditions Precedent. The satisfaction of the following shall be conditions
precedent for the benefit of the Agent and the Bank to the effectiveness of this
Amendment:

                1.2 Sale of Agoura Hills and Canoga Park Spectrum Clubs. The
sale of The Spectrum Clubs/Agoura Hills and The Spectrum Club/Canoga Park to
Racquetball and Fitness Clubs, Inc. pursuant to the Stock Purchase Agreement
shall have been consummated.

                1.3 Restructuring Fee. The Borrowers shall have paid to the
Agent a restructuring fee in the amount of Ten Thousand Dollars ($10,000).

Schedules 4.4, 4.10 and 4.17. The Borrower shall have delivered to the Agent a
revised version of Schedules 4.4, 4.10 and 4.17 to the Loan Agreement, updated
to the Effective Date.

Amended and Restated Revolving Loan Note. The Agent shall have received an
Amended and Restated Revolving Loan Note in the form of Exhibit "A" to this
Amendment duly executed by each of the Borrowers.

Reaffirmation of Intercreditor Agreement. The Agent shall have received a
Reaffirmation of Intercreditor Agreement in the form of Exhibit "B" to this
Amendment duly executed by each of the parties to the same.

Solvency Certificates. The Agent shall have received Solvency Certificates in
the form of Exhibit "C" to this Amendment executed by the chief financial
officers of Irvine Sports Club, Inc. and Green Valley Spectrum Club, Inc.

Amended and Restated Contribution Agreement. The Agent shall have received an
Amended and Restated Contribution Agreement in the form of Exhibit "D" to this
Amendment duly executed by the chief financial officers of Irvine Sports Club,
Inc. and Green Valley Spectrum Club, Inc.

                1.4 Corporate Documents. The Agent shall have received (a) a
certificate of an officer of each Borrower to the effect that such Borrower is
in compliance in all material respects with all material requirements of
applicable law; (b) a certificate of the chief financial officer of each
Borrower stating that, after giving effect to the modifications to the Loan
Agreement effected herein, as of the Effective Date no Default or Event of
Default shall have occurred and be continuing on the Effective Date; and (c)
such additional approvals, documents


<PAGE>   7

and other information, in form and substance satisfactory to the Agent, as the
Agent may reasonably request.

Bank Expenses. All legal fees, costs and other expenses which the Agent and the
Bank have incurred in connection with this Amendment as of the Effective Date
but which have not previously been reimbursed by the Borrowers shall have been
so reimbursed by the Borrowers.

Waiver of Debt Service Coverage Ratio Default. Effective as of the Effective
Date, the Agent and the Bank hereby waive the requirement that the Borrowers
comply with Section 6.15(a) of the Loan Agreement with respect to the fiscal
quarter ended September 30, 1999. This waiver is strictly limited as provided
above and shall not extend to any fiscal quarter other than that ended September
30, 1999 or to any other matter or transaction, other than the express waiver
reflected in the immediately preceding sentence. Except for the matter waived in
that sentence or as otherwise amended by this Amendment, all terms and
conditions set forth in the Loan Agreement and the other Loan Documents are
unchanged and remain in full force and effect. Each of the Agent and the Bank
reserve all of its powers, rights, remedies, claims, causes of action, defenses
and privileges under or in respect of the Loan Agreement and the other Loan
Documents. Each of the Borrowers acknowledges that neither the consent reflected
in this Section 7 nor the Agent's or the Bank's present awareness of the
existence of any Default or Event of Default: (a) imposes any obligation on the
Agent or any Bank to defer the enforcement of its powers, rights, remedies,
claims, causes of action, defenses or privileges under the Loan Agreement or any
of the Loan Documents, such enforcement action to be taken in the sole
discretion of the Agent and/or the Banks, as the case may be, when it or they
determine that is appropriate to do so; or (b) shall affect or diminish any
Borrower's obligation to comply with any other term or provision of the Loan
Agreement or any of the Loan Documents.

         2. Representations and Warranties. Each Borrower hereby represents and
warrants to the Agent and the Bank that each representation and warranty made by
it in Article IV of the Loan Agreement and each representation and warranty made
by it in each other Loan Document is true and correct on and as of the Effective
Date as though made as of the Effective Date, except to the extent such
representations and warranties relate solely to an earlier date.

Counterparts. This Amendment may be executed in multiple counterparts, each of
which shall constitute an original and all of which, taken together, shall
constitute but one and the same instrument.

Governing Law. This Amendment shall be governed by, and construed in accordance
with, the laws of the State of California.



        [Remainder of page intentionally left blank; signatures follow]


<PAGE>   8

        IN WITNESS WHEREOF, the parties hereto have executed this Amendment by
their respective duly authorized officers as of the date first above written.

BORROWERS:

THE SPORTS CLUB COMPANY, INC.               THE SPECTRUM CLUB COMPANY,
A Delaware corporation                      INC., a California corporation


By: /s/ Timothy O'Brien                     By: /s/ Timothy O'Brien
    --------------------------------------      --------------------------------
    Timothy O'Brien                             Timothy O'Brien
    Its:     Chief Financial Officer            Its:     Chief Financial Officer



<PAGE>   9

<TABLE>
<S>                                              <C>
PONTIUS REALTY, INC.                             LA/IRVINE SPORTS CLUBS, LTD.
a California corporation                         a California limited partnership

                                                 By: Sports Club, Inc. of California, general partner
By:  /s/ Timothy O'Brien
     --------------------------------------
     Timothy O'Brien
     Its:     Chief Financial Officer            By: /s/ Timothy O'Brien
                                                     ----------------------------------------
                                                     Timothy O'Brien
                                                     Its:     Chief Financial Officer

SPORTS CLUB, NC. OF CALIFORNIA,                  TALLA NEW YORK, INC.,
A California corporation                         a New York corporation


By:  /s/ Timothy O'Brien                         By: /s/ Timothy O'Brien
     --------------------------------------          ----------------------------------------
     Timothy O'Brien                                 Timothy O'Brien
     Its:     Chief Financial Officer                Its:     Chief Financial Officer


TVE, INC.,                                       SPECTRUM CLUB ANAHEIM,
a California corporation                         a California Corporation


By:  /s/ Timothy O'Brien                         By: /s/ Timothy O'Brien
     --------------------------------------          ----------------------------------------
     Timothy O'Brien                                 Timothy O'Brien
     Its:     Chief Financial Officer                Its:     Chief Financial Officer


IRVINE SPORTS CLUB, INC.,                        GREEN VALLEY SPECTRUM CLUB,
a California corporation                         INC., a Nevada corporation


By:  /s/ Timothy O'Brien                         By: /s/ Timothy O'Brien
     --------------------------------------          ----------------------------------------
     Timothy O'Brien                                 Timothy O'Brien
     Its:     Chief Financial Officer                Its:     Chief Financial Officer


THE SPORTSMED COMPANY, INC.
a California corporation


By:  /s/ Timothy O'Brien
     --------------------------------------
     Timothy O'Brien
     Its:     Chief Financial Officer
</TABLE>



<PAGE>   10

CANOGA AGOURA SPECTRUM CLUB,
INC., a California corporation


By: /s/ Timothy O'Brien
    ---------------------------------------
    Timothy O'Brien
    Its:     Chief Financial Officer


SCC SPORTS CLUB, INC.,
a Texas corporation


By:       /s/ Timothy O'Brien
    ---------------------------------------
          Timothy O'Brien
          Its:     Chief Financial Officer


SPECTRUM LIQUIDATING CORPORATION,
a California corporation,


By:       /s/ Timothy O'Brien
    ---------------------------------------
          Timothy O'Brien
          Its:     Chief Financial Officer


HFA SERVICES, INC.
a California corporation,


By:       /s/ Timothy O'Brien
    ---------------------------------------
          Timothy O'Brien
          Its:     Chief Financial Officer


NY SPORTS CLUB, INC.
a Delaware corporation


By:       /s/ Timothy O'Brien
    ---------------------------------------
          Timothy O'Brien
          Its:     Chief Financial Officer


<PAGE>   11


SF SPORTS CLUB, INC.,
a Delaware corporation


By: /s/ Timothy O'Brien
    ---------------------------------------
    Timothy O'Brien
    Its:     Chief Financial Officer


WASHINGTON D.C. SPORTS CLUB, INC.,
a Delaware corporation,


By: /s/ Timothy O'Brien
    ---------------------------------------
    Timothy O'Brien
    Its:     Chief Financial Officer


AGENT:

COMERICA BANK-CALIFORNIA
a California banking corporation, as Agent


By: /s/ Joseph Yurosek
    ---------------------------------------
    Joseph Yurosek
    Vice President


COMERICA BANK-CALIFORNIA,
a California banking corporation


By: /s/ Joseph Yurosek
    ---------------------------------------
    Joseph Yurosek
    Vice President