<PAGE>   1
                                                                   EXHIBIT 10.21


                                 PROMISSORY NOTE

Germantown, Tennessee                                              $460,000.00
December 15, 1997

         FOR VALUE RECEIVED, the undersigned (hereinafter referred to as
"Maker") hereby promises to pay to the order of Trust One Bank (hereinafter
referred to as "holder"), at the office of Holder, 2171 Judicial Drive, Suite
101, Germantown, Tennessee 38138, or at such other place as the Holder may from
time to time designate, the principal sum of Four Hundred Sixty Thousand and
NO/100 Dollars ($460,000.00) with interest at the fixed rate of nine and
three-quarters percent (9.75%) per annum. Maker shall pay fifty-nine (59) equal
monthly principal and interest installments of Six Thousand Fifty and 31/100
($6,050.31), based on a ten (10) year amortization, and one final payment in
full of any remaining principal, accrued interest and all other sums due under
this Note. Payments shall commence January 15, 1998, and shall continue on the
same day of each month thereafter through December 15, 2002. Interest shall be
calculated on a daily basis (computed on the basis of actual days elapsed over a
year of 360 days).

         PRE-PAYMENT. The indebtedness evidenced hereby may be pre-paid in whole
or in part at Maker's election at any time and from time to time without penalty
or premium. Pre-payments hereunder shall, at Holder's option, first be applied
to any accrued and unpaid interest, with the remainder being credited to
principal. Partial prepayments of principal shall not have the effect of
suspending or deferring the monthly payments herein provided for, and the same
shall continue to be due and payable on each due date subsequent to such partial
payment of principal, but shall operate to effect full payment of the principal
at an earlier date.

         TIME.  Time is of the essence of this Note.

         DEFAULT. If the Maker fails to pay when due any amount payable
hereunder and such failure shall continue for ten (10) days after receipt of
notice by Maker from the Holder specifying such default; or fails to perform,
defaults under, or breaches any covenant, agreement or undertaking in any Loan
Document, and such failure shall continue for ten (10) days after receipt of
notice by Maker from the Holder specifying such failure to perform, default or
breach; then, in any such event, there shall be deemed to be an "Event of
Default" hereunder. In such event, Holder, at its option, may declare the
outstanding principal balance of the indebtedness evidenced hereby, together
with any other sums advanced hereunder, or under the Loan Documents, together
with all unpaid interest accrued thereon, without notice to Maker, immediately
due, payable and collectable regardless of the date of maturity. Upon any Event
of Default, unpaid principal and interest shall bear interest thereafter until
paid at the highest rate allowed by applicable law. From and after any Event of
Default, Holder may exercise its remedies as a secured party under the Uniform
Commercial Code as adopted in Tennessee and cause the Trustee under the Trust
Deed to sell the Mortgaged property at a trustee sale or otherwise foreclose all
liens (including the lien of the Trust Deed) securing indebtedness evidenced by
this Note, whether by judicial proceeding or otherwise, all at the Holder's
option.

         SECURITY FOR THIS NOTE. This Note is secured, among other things, by
(i) a certain Deed of Trust and Security Agreement of even date herewith (herein
"Trust Deed"); (ii) financing statements of even date herewith; and (iii) any
other Loan Documents which may be required by Holder.



<PAGE>   2



         LATE CHARGE. In the event Holder, in its sole and exclusive discretion,
determines that it shall from time to time accept a payment more than fifteen
(15) days past its due date, as a condition preceding such acceptance, maker
agrees to hereby pay the Holder a late charge of an additional five percent (5%)
of the unpaid interest of the payment then due.

         ADDITIONAL GROUNDS FOR ACCELERATION. The indebtedness evidenced by this
note, may, at Holder's option, be declared due and payable if the Mortgaged
Property or any interest therein is leased (except in the ordinary course of
business), sold, assigned, conveyed, transferred (whether voluntary or
involuntary), unless the Holder first approves in writing the creditworthiness
of the lessee, optionee, transferee, vendor, assignee, grantee, encumbrancer,
mortgagee or partner. The Holder agrees that such approval shall not be
unreasonably withheld or delayed.

         ATTORNEY'S FEE AND EXPENSES. If an Event of Default shall have
occurred, and holder employs and attorney or incurs other expenses for the
collection of the amounts payable under this note or to protect the security
therefor, as described in the Loan Documents, whether suit be brought or not,
including those expenses and fees incurred in foreclosure, or court proceedings,
or any other litigation or proceeding affecting this Note or its security, or
reasonably incurred in any other way, then, in any such event, Maker shall pay
to Holder the sum of Holder's reasonable attorney's fees and collection
expenses.

         ADVANCES BY HOLDER. Should Holder either before or after an Event of
Default advance any sum pursuant to the provisions of this Note or the Loan
Documents, in order to pay any sum which the Maker is required to pay, or to pay
any sum to cure any default of Maker, or to do or perform any such act on behalf
of Maker, then Holder, at Holder's option, shall make such payment, or do or
perform any such act on behalf of Maker. All such payments made by Holder and
all costs and expenses incurred by Holder in doing or performing such acts shall
become immediately due and payable and, until paid, shall be and become part of
this indebtedness and shall bear interest at the highest rate of interest per
annum allowed by applicable law.

         NO WAIVER BY HOLDER. No failure to accelerate the indebtedness
evidenced hereby by reason of an Event of Default hereunder, acceptance of a
past due installment, or indulgences granted from time to time shall be
construed (i) as a novation of this Note or as reinstatement of the indebtedness
hereby evidenced or as a waiver of such right of acceleration, or the right of
Holder thereafter to insist upon strict compliance with the terms of this Note
or (ii) to prevent the exercise of such right of acceleration or any right
granted hereunder or by the laws of the State of Tennessee; and maker hereby
expressly waives benefit of any statute or rule of law or equity now provided,
or which may hereafter be provided, which would produce a result contrary to it
or in conflict with the foregoing. No extension of time for payment of this Note
or any installment hereunder, made by agreement with any person now or hereafter
liable for payment of this Note shall operate or release, discharge, modify,
change or affect the original liability of Maker under this Note, either in
whole or in part, unless Holder otherwise agrees in writing. This Note may not
be changed orally, but only by agreement in writing signed by the party against
whom enforcement of any waiver, change, modification or discharge is sought.



                                        2

<PAGE>   3



         WAIVER OF NOTICE. Maker and all who may become liable for same, jointly
and severally, waive presentment for payment, protest, notice of protest, notice
of non-payment, demand and all legal diligence in enforcing collection, and
hereby expressly agree that the Holder of this Note may defer or postpone
collection of the whole or any part thereof, either principal and/or interest,
or may extend or renew the whole or any part thereof, either principal and/or
interest, or may accept additional collateral or security for the payment of
this Note, or may release the whole or any part of any collateral security
and/or liens given to secure payment of this Note, or may release from liability
on account of this Note, maker or other parties, all without notice to them or
any of them; and such delay, postponement, renewal, extension, acceptance of
additional collateral or security and/or release shall not in any way operate to
release, modify, change or affect the obligation of maker or any other parties
liable for this Note, or any who may become liable for the payment hereof.

         LIMIT OF VALIDITY. It is the intention of the parties hereto to comply
strictly with the applicable usury laws; and, accordingly, in no event and upon
no contingency shall the Holder ever be entitled to receive, collect, or apply
as interest, any interest, fees, charges or other payments equivalent to
interest, in excess of the maximum rate which the Holder may lawfully charge
under applicable law in effect from time to time; and in the event that the
Holder ever receives, collects or applies as interest to any such excess, such
amount which, but for this provision, would be excessive interest, the same
shall be applied to the reduction of the principal amount of the indebtedness
hereby evidenced; and, if the principal amount of the indebtedness evidenced
hereby, and all lawful interest thereon, is paid in full, any remaining excess
shall forthwith be paid to the Maker, or any other party lawfully entitled
thereto. In determining whether or not the interest paid or payable under any
specific contingency exceeds the highest rate which the Holder hereof may
lawfully charge under applicable laws in effect from time to time, the parties
hereto shall, to the maximum extent permitted under applicable law, characterize
any non-principal payment as a reasonable loan charge rather than as interest.
Any provision hereof, or any other agreement that operates to bind, obligate, or
compel the Maker to pay interest in excess of such maximum rate shall be
construed to require the payment of the maximum rate only. The provisions of
this paragraph shall be given precedence over any other provision contained
herein and in any other agreement that is in conflict with the provisions of
this paragraph.

         GOVERNING LAW. This Note shall be governed and construed in accordance
with the laws of the State of Tennessee from time to time in effect, except to
the extent that applicable federal law may permit the charging of a higher rate
of interest than under applicable state law, in which event such applicable
federal laws, as amended and supplemented from time to time, shall govern and
control the maximum rate of interest permitted to be charged hereunder.

         SUCCESSORS AND ASSIGNS. The term "Holder" shall include the party named
herein as holder and any subsequent holder or holders of this Note. The terms of
this Note shall be binding upon and inure to the benefit of the parties hereto,
their respective heirs, personal representatives, successors and assigns, as if
they were in eery case named and expressed, and whether ever references are made
to either of the parties hereto the same shall be held to include and apply to
their heirs, legal executors, administrators, successors, and assigns, as if
they were in every case named and expressed.



                                        3

<PAGE>   4


         DEFINITIONS.  Whenever used in this Note the following words and terms
shall have the meanings respectfully ascribed to them.

         1. "Loan Documents" shall mean any and all instruments executed by the
Maker to evidence or secure or which pertain to the payment of the indebtedness
under this Note or the performance or discharge of the obligations under this
Note, including, without limiting the generality of the foregoing, the Trust
Deed, the financing statements and any and all other documents related to the
indebtedness now or hereafter executed by Maker.

         2. "Mortgaged property" shall be broadly meant to include all of the
property described in the Trust Deed of event date securing the indebtedness
evidenced by this Note.

         3. "Indebtedness' shall be broadly meant to include all sums owing
under this Note, including, without limiting the generality of the foregoing,
the unpaid principal balance, together with all accrued interest, advances
hereunder and all other sums owing under this Note.

         LAWFUL MONEY. Principal, interest, and all other sums due or that
become due under this Note are payable in the lawful money of the United States.

         TITLES. Titles of paragraphs are for convenience only and neither limit
nor amplify the provisions of this Note.

         SET-OFF. Holder shall have a right of set-off, in the full amount of
all of Maker's obligations to the Holder against any deposits, assets held by,
or other amounts owed by the Holder to, or held by the Holder for, the Maker, as
well as a lien on any and all property of the maker which is or may be in the
Holder's possession, at any time or from time to time, without notice to the
Maker or to any other person, any such notice being hereby expressly waived.

         NOTICE. Any notice required under the terms of this Note shall comply
with ss.7.6 of the Trust Deed.

                                             BACK YARD BURGERS, INC.



                                             By:
                                                -------------------------------

                                             Title:
                                                   ----------------------------


                                             By:
                                                -------------------------------

                                             Title:
                                                   ----------------------------



                                        4