<PAGE>   1
                                                                   EXHIBIT 10.31

                                 AMENDMENT NO. 5


                                                   Dated as of February 12, 2001

To the banks, financial institutions and other
   institutional lenders (collectively, the "BANKS")
   party to the Credit Agreement referred to
   below, to Citicorp USA, Inc. as administrative
   agent for the Banks and as the Swing Line Bank,
   and to Bank of America, N.A., as documentation agent

Ladies and Gentlemen:

                  We refer to the Fourth Amended and Restated Revolving Credit
Agreement dated as of August 25, 2000 (as amended by Amendment No. 1 dated as of
September 29, 2000, Amendment No. 2 and Waiver dated as of November 30, 2000,
Amendment No. 3 and Waiver dated as of December 22, 2000, and Amendment No. 4
dated as of February 2, 2001, the "CREDIT AGREEMENT") among the undersigned and
you. Capitalized terms not otherwise defined in this Amendment No. 5 have the
same meanings as specified in the Credit Agreement.

                  The Borrower has requested that, on the terms and conditions
set forth herein, the Majority Banks agree to amend the Credit Agreement as
provided herein, and the parties hereto have agreed to so amend the Credit
Agreement, effective as of the effective date of this Amendment No. 5. It is
hereby agreed by you and us that the Credit Agreement is hereby amended as
follows:

                  Section 2.09(b) of the Credit Agreement is hereby amended in
its entirety as follows:

                           "(b)     Asset Sales Mandatory Prepayments. Upon any
         Asset Sale by the Borrower or any Subsidiary of the Borrower (i) 50% of
         all Net Cash Proceeds of such Asset Sale subsequent to February 1, 2001
         shall be retained by the Borrower until the aggregate amount of all
         such Net Cash Proceeds of such Asset Sales retained by the Borrower
         equals $10,000,000 (including the amount of any deemed redemption as
         provided below), and (ii) all other Net Cash Proceeds of all such Asset
         Sales (including the 50% of Net Cash Proceeds from Asset Sales not
         retained by the Borrower and regardless of the date thereof) shall be
         delivered directly to the Agent by the purchaser of the Assets upon
         closing of the respective Asset Sale, and any deferred cash proceeds of
         any such Asset Sale shall be delivered directly to the Agent (at which
         time the same shall become Net Cash Proceeds); provided, however, that
         if the Borrower does not make by March 15, 2001 the $4,421,250 interest
         payment due thereby with respect to the 4.5% subordinated notes of the
         Borrower issued under the Indenture dated February 15, 1996, the
         Borrower shall be deemed to have retained an additional $4,421,250 of
         Net Cash Proceeds from all such Asset Sales as of such date and an
         amount equal to all Net Cash Proceeds that the Borrower has received in
         excess of $5,578,750 (before giving effect to such deemed redemption)
         as of March 15, 2001 shall be paid on such date by the Borrower
         directly to the Agent, provided further that Net Cash Proceeds from
         retained asset shall be delivered to the Agent no later than 15
         Business Days after the end of each month, and in each case, such Net
         Cash Proceeds shall be


                                       1
<PAGE>   2

         applied in accordance with Section 2.09(f). The Agent shall not be
         required to release any of its liens on the Assets sold as provided for
         in Section 8.08 herein until it receives the Net Cash Proceeds of the
         respective Asset Sale as provided for in this Section 2.09(b) or unless
         other provisions satisfactory to the Agent are contained in the
         respective escrow instructions for the payment of Net Cash Proceeds."

                  This Amendment No. 5 shall become effective as of the date
first above written when, and only when, (i) the Agent shall have received by
5:00 pm (New York City time) on or before February 16, 2001, counterparts of
this Amendment No. 5 executed by the undersigned and the Majority Banks or, as
to any of the Banks, advice satisfactory to the Agent that such Bank has
executed this Amendment No. 5, and the consent attached hereto executed by each
Guarantor and (ii) the Borrower shall have paid by such date all amounts due and
payable under Section 9.04 of the Credit Agreement. This Amendment No. 5 is
subject to the provisions of Section 9.01 of the Credit Agreement.

                  On and after the effectiveness of this Amendment No. 5, each
reference in the Credit Agreement to "this Agreement", "hereunder", "hereof" or
words of like import referring to the Credit Agreement, and each reference in
the Notes and each of the other Loan Documents to "the Credit Agreement",
"thereunder", "thereof" or words of like import referring to the Credit
Agreement, shall mean and be a reference to the Credit Agreement, as amended by
this Amendment No. 5.

                  The Credit Agreement, as specifically amended by this
Amendment No. 5, is and shall continue to be in full force and effect and is
hereby in all respects ratified and confirmed. The execution, delivery and
effectiveness of this Amendment No. 5 shall not, except as expressly provided
herein, operate as a waiver of any right, power or remedy of any Bank or the
Agent under any of the Loan Documents, nor constitute a waiver of any provision
of any of the Loan Documents.

                  If you agree to the terms and provisions hereof, please
evidence such agreement by executing and telecopying one signature page to Susan
McManigal at Citibank, N.A. (Telecopier No. (212) 793-0642) and returning at
least three counterparts of this Amendment No. 5 to Patience Crowder at Shearman
& Sterling, 555 California Street, San Francisco, CA 94104 (Telecopier No. (415)
616-1199).

                  This Amendment No. 5 may be executed in any number of
counterparts and by different parties hereto in separate counterparts, each of
which when so executed shall be deemed to be an original and all of which taken
together shall constitute one and the same agreement. Delivery of an executed
counterpart of a signature page to this Amendment No. 5 by telecopier shall be
effective as delivery of a manually executed counterpart of this Amendment No.
5. This Amendment No. 5 shall be governed by, and construed in accordance with,
the laws of the State of New York.

                                        Very truly yours,

                                        PHYCOR, INC.


                                        By /s/ Tarpley B. Jones
                                          --------------------------------------
                                           Name: Tarpley B. Jones
                                           Title: Executive Vice President and
                                                  Chief Financial Officer


                                       2
<PAGE>   3


Agreed as of the date first above written:

CITIBANK, N.A.,
         as Issuing Bank

By: /s/ Susan McManigal
   ----------------------------------
   Title: Vice President

CITICORP USA, INC.
         as Agent, as Swing Line Bank and as Bank

By: /s/ Susan McManigal
   ----------------------------------
   Title: Vice President




AMSOUTH BANK, successor in interest by merger to,
FIRST AMERICAN NATIONAL BANK

By: /s/ Robert I. Hart
   ----------------------------------
   Title: Senior Vice President



BANK OF AMERICA, N.A.

By: /s/
   ----------------------------------
   Title: Managing Director





BANKERS TRUST COMPANY

By:
   ----------------------------------
   Title:




THE BANK OF NOVA SCOTIA, Atlanta Agency

By: /s/ William E. Zarrett
   ----------------------------------
   Title: Managing Director




CREDIT LYONNAIS NEW YORK BRANCH

By:
   ----------------------------------
   Title:




BANK ONE, NA (f/k/a THE FIRST NATIONAL BANK OF CHICAGO)

By: /s/
   ----------------------------------
   Title: First Vice President




AMROC INVESTMENTS, LLC

By:
   ----------------------------------
   Title:




PATRIARCH PARTNERS, LLC



By:
   ----------------------------------
   Title:



                                       3
<PAGE>   4


MELLON BANK, N.A.

By: /s/
   ----------------------------------
   Title: Assistant Vice President




COOPERATIEVE CENTRALE RAIFFEISEN
BOERENLEENBANK B.A., "RABOBANK NEDERLAND",
NEW YORK BRANCH

By:
   --------------------------------------
     Title:

By:
   --------------------------------------
     Title:




THE SUMITOMO BANK, LIMITED

By: /s/
   ----------------------------------
   Title: Senior Vice President



SUNTRUST BANK

By:
   ----------------------------------
   Title:




TORONTO DOMINION (TEXAS), INC.

By: /s/ Carolyn R. Faeth
   ----------------------------------
   Title: Vice President



WACHOVIA BANK

By:
   ----------------------------------
   Title:




CERBERUS PARTNERS, L.P.


By:
   ----------------------------------
   Title:



                                       4