<PAGE>

                                                                   EXHIBIT 10.50

             MODIFICATION, EXTENSION AND RENEWAL OF PROMISSORY NOTE

$1,000,000.00                                             As of January 16, 2004

      For value received, the undersigned (the "Makers," whether one or more),
promise to pay to the order of JOHN C. WOOLEY AND JEFFREY J. WOOLEY (the
"Payee"), at 203 Colorado, Austin, Travis County, Texas 78701, or such other
location as the Payee designates to the Makers in writing, the principal sum of
ONE MILLION AND NO/100 DOLLARS ($1,000,000.00), or the outstanding principal
amount advanced hereunder, whichever is less, in legal and lawful money of the
United States of America, with interest thereon from the date hereof through the
maturity date of this Note (whether by acceleration or otherwise) (the "Maturity
Date") at the rate of six percent (6.0%) per annum (calculated on the basis of
the actual number of days elapsed but computed as if each year consisted of 360
days), the interest being payable as hereinafter specified. After the Maturity
Date until paid, unpaid principal and accrued unpaid interest shall bear
interest at a rate per annum equal to the lesser of (i) eighteen percent (18%),
or (ii) the Maximum Lawful Rate. As used herein, the term "Maximum Lawful Rate"
shall mean the greater of (i) the highest non-usurious rate of interest
permitted by applicable United States law, or (ii) a rate per annum equal to the
applicable weekly ceiling described in Chapter 303 of the Texas Finance Code, as
amended, as such indicated rate ceiling is in effect from time to time, but in
no event greater than twenty-four (24.0%) per annum. Unless precluded by law,
changes in the Maximum Lawful Rate created by statute or governmental action
during the term of this Note shall be immediately applicable to this Note on the
effective date of such changes. If the applicable law ceases to provide for a
Maximum Lawful Rate, the Maximum Lawful Rate shall be equal to eighteen percent
(18%) per annum, unless the loan evidenced by this Note is subject to Regulation
Z of the Board of Governors of the Federal Reserve System, 12 C. F. R. Section
226 and is secured by a dwelling, in which case the Maximum Lawful Rate shall be
equal to twenty-four percent (24.0%) per annum.

      Notwithstanding the foregoing, if, at any time, the rate of interest
applicable to this Note (but for the limitation thereof to the Maximum Lawful
Rate) exceeds the Maximum Lawful Rate, the rate of interest to accrue on this
Note shall be limited to the Maximum Lawful Rate, but any subsequent reductions
in such rate of interest applicable to this Note (but for the limitation thereof
to the Maximum Lawful Rate) shall not reduce the rate of interest to accrue on
this Note below the Maximum Lawful Rate until the total amount of interest which
would have accrued if a varying rate per annum equal to the rate of interest
applicable to this Note (but for the limitation thereof to the Maximum Lawful
Rate) had at all times been in effect.

      If the Maximum Lawful Rate is increased by statute or other governmental
action subsequent to the date hereof, then the Makers agree that the new Maximum
Lawful Rate will be applicable hereto from the effective date of the new Maximum
Lawful Rate, unless such application is precluded by statute or governmental
action or by the general law of the jurisdiction governing the transaction
evidenced hereby.


                                       1
<PAGE>
      This Note is in renewal, extension, rearrangement and modification, but
not extinguishment, of that one certain Modification, Renewal and Extension of
Promissory Note ("Second Renewed Note") between Maker and Payee dated effective
October 31, 2003, and one certain Modification, Renewal and Extension of
Promissory Note ("First Renewed Note") between Maker and Payee dated effective
August 8, 2003, which renewed and extended the term of (but did not extinguish)
that certain Promissory Note (the "Original Note") in the original principal
amount of $1,000,000.00 between Maker and Payee dated effective April 8, 2003.
The only change the parties wish to make to the Second Renewed Note is to extend
the maturity date from January 16, 2004, to April 16, 2004. The parties intend
all other terms and the security for this indebtedness to remain the same. The
Original Note, First Renewed Note and Second Renewed Note are referred to herein
collectively as the "Renewed Notes".

TERMS OF PAYMENT:

      The entire amount of this Note, principal and accrued interest then
remaining unpaid, shall be due and payable on April 16, 2004. Interest shall be
calculated on the unpaid principal to the date each installment is paid and each
such payment shall be credited to the discharge of the interest accrued, the
reduction of principal, and other authorized charges, if any, in such manner and
order as the Payee shall determine in its sole discretion.

PAYMENT ON NON-BUSINESS DAYS:

      If any payment hereunder falls due on a Saturday, Sunday or public holiday
on which commercial banks in Austin, Texas are permitted or required by law to
be closed, the time for such payment shall be extended to the next day on which
the Payee is open for business, and such extension of time shall be included in
the calculation of interest accruing and payable hereunder.

PREPAYMENT:

      The Makers reserve the right to prepay this Note in any amount at any time
prior to maturity without penalty. Interest shall be calculated on the unpaid
principal to the date of any prepayment and any such prepayment shall be applied
first toward the payment of accrued interest and next to the principal
installments of this Note in the inverse order of maturity.

SECURITY FOR PAYMENT:

      Payment of this Note is secured by, and this Note is entitled to the
benefits of, all security agreements, assignments, deeds of trust, mortgages and
lien instruments executed by the Makers (or any of them), or other similar
instruments, guaranties, endorsements or other agreements, executed by any other
person or entity (the "Collateral Agreements," whether one or more) to secure,
guarantee or otherwise provide for the payment hereof, in favor of or for the
benefit of the Payee, including any previously executed and any now or hereafter
executed. Without limiting the foregoing, the Makers ratify and confirm that the
Collateral Agreements include, and this Note is entitled to the


                                       2
<PAGE>
benefits of "Indebtedness" under, that certain Security Agreement dated as of
April 8, 2003, executed by the Makers and Payee covering certain of the Makers'
personal property.

USE OF PROCEEDS:

      The Original Note, as modified by the Renewed Notes, represents funds
advanced to the Makers at the Makers' special instance and request and used to
provide working capital for the Makers.

REPRESENTATIONS AND WARRANTIES:

      The Makers expressly represent and warrant to the Payee that it is a
corporation duly organized and existing under the laws of the State of Texas;
that it possesses full power and authority to own its property and to conduct
its business as now conducted and as presently proposed to be conducted; that
the execution and delivery of this Note will not contravene any provisions of
its articles incorporation, bylaws or any other agreement relating to its form
of entity; that the officer executing this Note is the legally qualified and
acting officer of said corporation and is expressly authorized to execute this
Note by appropriate resolution of the Board of Directors of said corporation.

LIMITATION OF INTEREST:

      All agreements and transactions among the Makers and the Payee, whether
now existing or hereafter arising, whether contained herein or in any other
instrument, and whether written or oral, are hereby expressly limited so that in
no contingency or event whatsoever, whether by reason of acceleration of the
maturity hereof, late payment, prepayment, or otherwise, shall the amount of
interest contracted for, charged or received by the Payee from the Makers for
the use, forbearance, or detention of the principal indebtedness or interest
hereof, which remains unpaid from time to time, exceed the Maximum Lawful Rate,
it particularly being the intention of the parties hereto to conform strictly to
the applicable usury laws of the State of Texas (or applicable United States law
to the extent that it permits the Payee to contract for, charge or receive a
greater amount of interest than under Texas law). Any interest payable hereunder
or under any other instrument relating to the indebtedness evidenced hereby that
is in excess of the Maximum Lawful Rate, shall, in the event of acceleration of
maturity, late payment, prepayment, or otherwise, be applied to a reduction of
the unrepaid indebtedness hereunder and not to the payment of interest, or if
such excessive interest exceeds the unpaid balance of such unrepaid
indebtedness, such excess shall be refunded to the Makers. To the extent not
prohibited by applicable law, determination of the Maximum Lawful Rate shall at
all times be made by amortizing, prorating, allocating and spreading in equal
parts during the full term of this loan, all interest at any time contracted
for, charged or received from the Makers in connection with this loan, so that
the actual rate of interest on account of such indebtedness is uniform
throughout the term thereof.


                                       3
<PAGE>
SUCCESSORS AND ASSIGNS:

      As used herein, the term "Payee" shall include the successors and assigns
of the Payee and any subsequent owner and holder of this Note, and the term
"Makers" shall include co-makers, endorsers, guarantors, sureties and their
respective successors and assigns.

DEFAULT AND COLLECTION:

      Subject to the express notice and cure provisions contained in this Note,
it is expressly provided that, upon default in the punctual payment of this
Note, or any part hereof, principal or interest, as the same shall become due
and payable, or upon default in the performance of or compliance with any of the
terms of any of the Collateral Agreements, or if the Payee deems the Payee
insecure, either because the prospect of timely payment of this Note becomes
impaired, or because the prospect of timely performance of any of the Collateral
Agreements becomes impaired, at the option of the Payee, the entire indebtedness
evidenced hereby shall be matured, and in the event default is made in the
prompt payment of this Note when due or declared due, and the same is placed in
the hands of an attorney for collection, or suit is brought on the same, or the
same is collected through probate, bankruptcy or other judicial proceedings,
then the Makers jointly and severally agree and promise to pay all reasonable
attorney's fees, court costs and collection costs incurred by the Payee.

NOTICE AND CURE RIGHTS:

      In the event of any default under the Collateral Agreements or this Note,
the Makers and each Guarantors named below shall be entitled to receive written
notice of any such default and a period of fifteen (15) days after such notice
is sent by the Payee within which to cure such default prior to the Payee's
being entitled to exercise any remedy which may arise due to the occurrence of
such default, other than the right to withhold making further advances of funds
during the period any such default remains uncured. However, nothing herein
shall obligate the Payee to give the Makers more than one (1) notice of default
during any ninety (90) day period. The provisions of this paragraph shall
control over any inconsistent provision in any of the Collateral Agreements; and
any right to accelerate the maturity of this Note contained in any of the
Collateral Agreements is subject to prior compliance with this paragraph.

WAIVERS AND CONSENTS:

      Subject to the express notice and cure provisions contained in this Note,
each of the Makers waives presentment for payment, notice of intent to
accelerate, notice of acceleration, protest and notice of protest, dishonor and
diligence in collecting and the bringing of suit against any other party, and
agrees to all renewals, extensions, partial payments, releases and substitutions
of security, in whole or in part, with or without notice, before or after
maturity. The Payee may remedy any default, without waiving the same, or may
waive any default without waiving any prior or subsequent default.


                                       4
<PAGE>
GOVERNING LAWS AND VENUE:

      This Note is governed by and is to be construed and enforced in accordance
with the laws of the State of Texas and of the United States. The Makers agree
and consent to the jurisdiction of the District Courts of Travis County, Texas,
and of the United States District Court for the Western District of Texas
(Austin Division) and acknowledge that such courts shall constitute proper and
convenient forums for the resolution of any actions among the Makers and the
Payee with respect to the subject matter hereof, and agree that such courts
shall be the exclusive forums for the resolution of any actions among the Makers
and the Payee with respect to the subject matter hereof.

                                        Schlotzsky's, Inc.,
                                        a Texas corporation


                                        By:      /s/ JOYCE CATES
                                            --------------------------------
                                                 Joyce Cates,
                                                 Senior Vice President


                                       5