<PAGE>
                                                                    EXHIBIT 10.4




                                   TERM B NOTE

                                                               __________, _____
$__________


         FOR VALUE RECEIVED, the undersigned, O'CHARLEY'S INC., a corporation
organized under the laws of Tennessee (the "Borrower"), promises to pay to the
order of _______________ (the "Lender"), at the place and times provided in the
Credit Agreement referred to below, the principal sum of _______________ DOLLARS
($__________) or, if less, the principal amount of all Term B Loans made by the
Lender pursuant to that certain Credit Agreement, dated as of January __, 2003
(as amended, restated, supplemented or otherwise modified, the "Credit
Agreement"), by and among the Borrower, the Lenders who are or may become a
party thereto, as Lenders (the "Lenders"), and Wachovia Bank, National
Association, as Administrative Agent (the "Administrative Agent"). Capitalized
terms used herein and not defined herein shall have the meanings assigned
thereto in the Credit Agreement.

         The unpaid principal amount of this Term B Note from time to time
outstanding is subject to mandatory repayment from time to time as provided in
the Credit Agreement and shall bear interest as provided in Section 5.1 of the
Credit Agreement. All payments of principal and interest on this Term B Note
shall be payable in lawful currency of the United States of America in
immediately available funds to the account designated in the Credit Agreement.

         This Term B Note is entitled to the benefits of, and evidences
Obligations incurred under, the Credit Agreement, to which reference is made for
a description of the security for this Term B Note and for a statement of the
terms and conditions on which the Borrower is permitted and required to make
prepayments and repayments of principal of the Obligations evidenced by this
Term B Note and on which such Obligations may be declared to be immediately due
and payable.

         THIS TERM B NOTE SHALL BE GOVERNED BY, CONSTRUED AND ENFORCED IN
ACCORDANCE WITH, AND ALL ISSUES RELATED TO THE LEGALITY, VALIDITY OR
ENFORCEABILITY HEREOF, SHALL BE DETERMINED UNDER THE LAWS OF NEW YORK (INCLUDING
SECTION 5-1401 AND SECTION 5-1402 OF THE GENERAL OBLIGATIONS LAW OF THE STATE OF
NEW YORK), WITHOUT REFERENCE TO THE CONFLICTS OF LAW PRINCIPLES THEREOF.

         The Debt evidenced by this Term B Note is senior in right of payment to
all Subordinated Debt referred to in the Credit Agreement.

         The Borrower hereby waives all requirements as to diligence,
presentment, demand of payment, protest and (except as required by the Credit
Agreement) notice of any kind with respect to this Term B Note.


<PAGE>

         IN WITNESS WHEREOF, the undersigned has executed this Term B Note under
seal as of the day and year first above written.

                                       O'CHARLEY'S INC.


                                       By:
                                          --------------------------------------
                                          Name:
                                                --------------------------------
                                          Title:
                                                --------------------------------













                                       2