<PAGE>

                                  EXHIBIT 10.51

             MODIFICATION, EXTENSION AND RENEWAL OF PROMISSORY NOTE

Date: Effective January 31, 2004

Maker: Schlotzsky's, Inc.

Maker's Mailing Address (including county): 203 Colorado Street
                                            Austin, Travis County, TX 78701

Payee: Jeffrey J. Wooley

Place for Payment (including county): 203 Colorado Street
                                      Austin, Travis County, TX 78701

Principal Amount: $112,500.00

Annual Interest Rate on Unpaid Principal from Date: 7%

Annual Interest Rate on Matured, Unpaid Amounts:

           Maximum non-usurious rate allowed by applicable law.

Terms of Payment (principal and interest)
           Interest only shall be paid monthly on the last day of each month.
           Principal shall be due and payable on January 31, 2005.

This note may be prepaid at any time without penalty.

Maker promises to pay to the order of Payee at the place for payment and
according to the terms of payment the principal amount plus interest at the
rates stated above. All unpaid amounts shall be due by the final scheduled
payment date.

If Maker defaults in the payment of this note or in the performance of any
obligation in any instrument securing or collateral to it, and the default
continues after Payee gives Maker notice of the default and the time within
which it must be cured, as may be required by law or by written agreement, then
Payee may declare the unpaid principal balance and earned interest on this note
immediately due. Maker and each surety, endorser, and guarantor waive all
demands for payment, presentations for payment, notices of intention to
accelerate maturity, notices of acceleration of maturity, protests, and notices
of protest, to the extent permitted by law.

<PAGE>

If this note or any instrument securing or collateral to it is given to an
attorney for collection or enforcement, or if suit is brought for collection or
enforcement, or if it is collected or enforced through probate, bankruptcy, or
other judicial proceeding, then Maker shall pay Payee all costs of collection
and enforcement, including reasonable attorney's fees and court costs, in
addition to other amounts due.

Interest on the debt evidenced by this note shall not exceed the maximum amount
of non-usurious interest that may be contracted for, taken, reserved, charged,
or received under law; any interest in excess of that maximum amount shall be
credited on the principal of the debt or, if that has been paid, refunded. On
any acceleration or required or permitted prepayment, any such excess shall be
canceled automatically as of the acceleration or prepayment or, if already paid,
credited on the principal of the debt or, if the principal of the debt has been
paid, refunded. This provision overrides other provisions in this and all other
instruments concerning the debt.

This Note is in renewal, extension and modification of that one certain
Promissory Note in the original principal amount of $112,500.00 between Maker
and Payee dated effective January 31, 2002, as extended by that prior
Modification, Extension and Renewal of Promissory Note dated effective December
30, 2002.

Each Maker is responsible for all obligations represented by this note.

When the context requires, singular nouns and pronouns include the plural.

Maker:

Schlotzsky's, Inc.

By: /s/ JOYCE CATES
    ------------------------------------------------
Name: Joyce Cates
Title: Senior Vice President - Franchise Operations

                                       2