<PAGE>   1
                     SEVERANCE AGREEMENT AND GENERAL RELEASE


     In consideration for the payments and additional benefits to be paid by
O'Charley's Inc., a Tennessee corporation (the "Company"), Charles F. McWhorter
("Employee") hereby releases the Company and its affiliates and all of its
officers, directors, employees, agents, attorneys and stockholders from all
claims or causes of action based on facts which are known by him or of which he
has been made aware, arising in the course of his employment or resulting from
the termination of his employment. This means that he cannot and will not file
any claim, charge, or lawsuit on the basis of any such claim or cause of action
for the purpose of obtaining any monetary award above and beyond the amounts
provided for in this Severance Agreement and General Release ("Agreement"),
reinstatement of his employment or for any equitable relief.

     Notwithstanding the foregoing, Employee acknowledges that this general
release includes, but is not limited to, all claims arising under federal, state
or local laws prohibiting employment discrimination and all claims growing out
of any legal restrictions on the Company's right to terminate its employees,
specifically including all claims of employment discrimination based on race,
color, religion, sex, and national origin, as provided under Title VII of the
Civil Rights Act of 1964, as amended, all claims of discrimination based on age,
as provided under the Age Discrimination in Employment Act of 1967, as amended,
all claims under the Employee Retirement Income Security Act (ERISA) and all
claims of employment discrimination under the Americans with Disabilities Act
(ADA) as well as claims under state law.

     In consideration of the agreements and covenants by Employee hereunder, the
Company hereby releases Employee from all claims or causes of action arising in
the course of his employment other than claims or causes of action based on
criminal conduct.

     Employee intends this Agreement to be binding upon himself, his estate,
heirs and assignees. The Company intends this Agreement to be binding upon it,
its successors and assignees. Employee understands and agrees that if he
breaches this Agreement in any material respect or if he files any claim or
lawsuit against the Company on the basis of a released claim, all payments and
benefits provided herein shall cease, and Employee or his estate shall be
required to reimburse the Company for all payments and benefits Employee
received under this Agreement prior to such time.

     In consideration of the foregoing, the Company and Employee hereby agree
and covenant as follows:

     1. Employee hereby resigns from employment with the Company effective
September 3, 1996.

     2. The Company will make severance payments to Employee for the period
ended December 29, 1996 as follows:



<PAGE>   2



          (a) Employee shall continue to receive his salary at the same rate as
     currently in effect through December 29, 1996;

          (b) Employee shall receive such bonus as Employee would have been
     entitled to receive for the year ended December 29, 1996 pursuant to the
     bonus formula contained in the 1996 Executive Officers' Bonus Plan (the
     "Bonus Plan"); and

          (c) Employee shall continue to receive his present automobile
     allowance through December 29, 1996.

All such severance payments will be subject to state and federal tax
withholding.

     3. Through December 29, 1996, the Company will continue to provide group
medical insurance to Employee on the same basis as the Company then makes such
insurance available to regular full-time employees of the Company, and will pay
for such coverage on Employee's behalf. Following December 29, 1996, Employee
shall be eligible for such coverage at Employee's expense for the period and to
the extent provided under COBRA.

     4. Pursuant to the terms of the 1990 Employee Stock Plan and the Senior
Management Stock Option Program (collectively, the "Option Plans"), Employee
currently has the following options to purchase the Company's Common Stock which
have vested in the amounts indicated:

<TABLE>
<CAPTION>
                                                              NUMBER OF SHARES
     DATE OF GRANT      NUMBER OF SHARES     EXERCISE PRICE        VESTED
     -------------      ----------------     --------------   ----------------
       <S>                   <C>                 <C>               <C>
        5/17/91               21,000             $ 3.08            21,000
        2/27/92                7,500             $ 4.17             6,000
        2/01/93                7,500             $ 6.33             4,500
       10/24/93              150,000             $ 8.42            60,600
        1/30/95               60,000             $11.50             9,240
</TABLE>

The Company hereby agrees that each of the foregoing options may be exercised as
to shares vested thereunder at any time during the remaining term of such
option. All shares not vested as of the effective date of this Agreement are
forfeited; provided, that with respect to the October 24, 1993 and January 30,
1995 options, Employee shall be deemed to have vested with respect to such
number of additional shares based on the vesting criteria set forth in the
Senior Management Stock Option Program for the year ended December 29, 1996.


                                        2

<PAGE>   3



     5. As Employee will no longer serve as an employee of the Company, the
Company will not make any further contributions for Employee's benefit to the
Company's SERP or 401(k) plans following the effective date of this Agreement.
The Company will pay to Employee promptly following December 29, 1996, however,
such amounts as the Company would have contributed to such plans for such
period.

     6. The Company will promptly reimburse Employee for any accountable
business expenses incurred by Employee prior to the date of this Agreement.

     7. Any vacation benefits earned by Employee as of the effective date of
this Agreement will be deemed completely paid as part of the severance, and no
further vacation, sick time, personal time or other paid time off will accrue
during the severance period.

     8. Neither the Company nor Employee shall make any disparaging statements
about the other (including, in the case of the Company, its officers, directors,
employees, accountants, attorneys or other representatives), except that this
provision does not limit or restrict the Company or its representatives or the
Employee from answering questions or testifying truthfully if subpoenaed by a
court of competent jurisdiction or governmental agency.

     9. Notwithstanding the general release contained herein, Employee shall
continue to be entitled to indemnification in the defense of any proceeding by a
third party to which he is a party because he was an officer or director of the
Company to the extent and in the manner required by the Tennessee Business
Corporation Act, the Company's Amended and Restated Charter and Bylaws.

     Employee has carefully read and fully understands all the provisions of
this Agreement, specifically including the general release of claims included
herein. Employee further acknowledges that this Agreement together with (i) the
Bonus Plan, (ii) the Option Plans and option agreements issued to Employee
thereunder, (iii) the documents which comprise the medical group insurance
coverage and (iv) the Company's Amended and Restated Charter and Bylaws, set
forth the entire agreement between the Company and Employee. In addition,
Employee acknowledges that he has been given a period of at least twenty-one
(21) days to consider this Agreement and that he is advised that he has the
right to consult with an attorney of his choice during this period. Finally,
Employee acknowledges that, in considering whether to sign this Agreement, he
has not relied upon any representation or statement by anyone, either written or
oral, not set forth in this document and that he has not been threatened or
coerced into signing this Agreement by any official of the Company and that he
has read, understood and fully and voluntarily accepts the terms of this
Agreement.

     EFFECTIVE DATE: Employee understands that this Agreement may be revoked by
Employee at any time during the seven (7) day period after he has signed it.
This Agreement shall not become effective until after the revocation period has
expired and no payment will be made until such period has expired.



                                        3

<PAGE>   4


     DATED THIS 9th day of September, 1996.


O'CHARLEY'S INC.


By:  /s/ Gregory L. Burns                                    9/9/96
     -------------------------------               ---------------------------
     Title:  Chief Executive Officer               Date



  /s/ Charles F. McWhorter                                   9/9/96
      ------------------------------               ---------------------------
      Charles F. McWhorter                         Date






                                        4