<PAGE>   1
                                                                    Exhibit 10.1

                             SECURED PROMISSORY NOTE
                             -----------------------

$666,666.00                                              Date: November 17, 2000

                FOR VALUE RECEIVED, the undersigned, Jeffrey A. Cole,
("Borrower"), hereby unconditionally promises to pay to the order of Cole
National Corporation, a Delaware corporation (the "Company"), the principal sum
of Six Hundred Sixty-Six Thousand Six Hundred Sixty-Six Dollars and 00/100
($666,666.00) in lawful money of the United States of America and in immediately
available funds, on January 18, 2004 and to pay simple interest (computed on the
basis of a 365 day year) on the unpaid principal amount hereof from and after
the date of this Secured Promissory Note until the entire principal amount
hereof has been paid in full, at the rate of 6.01% per annum. Interest is
payable on each anniversary of the date of this Secured Promissory Note with
respect to the period ending on such anniversary, and at maturity or upon full
prepayment of the principal hereof. Not later than five business days prior to
the date on which payment of interest is due, the Company shall give written
notice to the Borrower of the amount of the payment due on such date.

                For purposes of this Promissory Note, a "business day" shall
mean any day other than a Saturday, Sunday or federal holiday and shall consist
of the time period from 12:01 a.m. through 12:00 Midnight Eastern time.

                This Secured Promissory Note is delivered as payment of the
principal amount of that certain Amended and Restated Secured Promissory Note
dated March 15, 1998 from Borrower to Company. Payment of the principal and
interest on this Secured Promissory Note is secured with the property ("Pledged
Security") pursuant to the terms of the Stock Pledge and Security Agreement
dated November 17, 2000 between Borrower and the Company. This Secured
Promissory Note is subject to the following further terms and conditions:

                 1. PAYMENT AND PREPAYMENT.

                 (a) If the Borrower or any of his Permitted Transferees shall
                 sell any of the Pledged Security, such sale shall be made only
                 for cash, and Borrower agrees to promptly deliver to the
                 Company the consideration received by the Borrower on such sale
                 of shares of Stock (net of any taxes due as a result of such
                 sale) as partial payment of the unpaid


<PAGE>   2
                                                                               2


                 principal and accrued and unpaid interest on this Secured
                 Promissory Note until this Secured Promissory Note is paid in
                 full. In the event this Secured Promissory Note is not paid in
                 full, Borrower shall provide additional collateral in the event
                 the value or principal amount of the remaining Pledged Security
                 is less than the unpaid principal of this Secured Promissory
                 Note.

                 (b) All payments of principal and interest on this Secured
                 Promissory Note shall be made to the Company or its order in
                 lawful money of the United States of America and in immediately
                 available funds at the offices of the Company at its then
                 principal place of business (or at such other place as the
                 holder hereof shall notify Borrower in writing). Borrower may,
                 at his option, prepay this Secured Promissory Note in whole or
                 in part at any time or from time to time without penalty or
                 premium. Any prepayments of any portion of the principal amount
                 of this Secured Promissory Note shall be accompanied by payment
                 of all interest accrued but unpaid hereunder.

                 (c) Concurrently with any payment of any portion of this
                 Secured Promissory Note pursuant to paragraph 1(a) hereof or
                 any prepayment of any portion of the principal amount of this
                 Secured Promissory Note pursuant to paragraph 1(b) hereof, the
                 Company shall make a notation of such application or payment on
                 this Secured Promissory Note. If full payment of all unpaid
                 principal of and accrued and unpaid interest on this Secured
                 Promissory Note is made, this Secured Promissory Note shall be
                 cancelled. Any partial payment or prepayment shall be applied
                 first to accrued and unpaid interest hereon and then to the
                 unpaid installments of principal hereof in the inverse order of
                 their maturity.

                 (d) Borrower, at Borrower"s sole election, may repay any or all
                 of the unpaid principal amount of or interest on this Secured
                 Promissory Note, by delivery to the Company of Cole National
                 Group, Inc. 9-7/8% Senior Notes due 2006 or Cole National
                 Group, Inc. 8-5/8% Senior Notes due 2007, (collectively, the
                 "Bonds") in either case owned by Borrower, having a principal
                 amount equal to the principal amount or interest to be so
                 repaid, notwithstanding the prices at which the Bonds may trade
                 in the public markets. However, the foregoing sentence will not
                 apply, and Borrower will have no right to repay this Secured

<PAGE>   3
                                                                               3


                  Promissory Note by delivery of Cole National Group, Inc.
                  Senior Notes, if, at the time of delivery to the Company, Cole
                  National Group, Inc. is in default under the indentures
                  pursuant to which the Senior Notes were issued. Unless an
                  Event of Default shall have occurred, all interest paid on the
                  Bonds shall belong to Borrower.

                  2. CONVERSION DEMAND NOTE. In the event Borrower shall cease
to be employed by the Company as the result of:

                  (a) his death or permanent disability, then the unpaid
         principal of and accrued and unpaid interest of this Secured Promissory
         Note ("Unpaid Amounts") shall upon demand by the Company given in
         writing to the Borrower or his estate become due and payable one year
         from the date of such notice; or

                  (b) his voluntary termination of employment with the consent
         of the Board of Directors of the Company or through termination without
         cause, then the Unpaid Amounts shall upon demand by the Company given
         in writing to the Borrower become due and payable one year from the
         date of such notice; or

                  (c) his voluntary termination of employment without the
         consent of the Board of Directors of the Company or his termination by
         the Company with Cause (as defined in the Employment Agreement
         ("Employment Agreement") dated December 17, 1998 between the Company
         and Borrower), then the Unpaid Amounts shall upon demand by the Company
         given in writing to the Borrower become due and payable thirty (30)
         days from the date of such notice; or

                  (d) a "Change In Control" as defined in Employment Agreement
         triggering a payment to Borrower thereunder, then upon demand by the
         Company given in writing to the Borrower the Unpaid Amounts shall be
         paid in full at or prior to the time any Change in Control payment is
         made to Borrower.

         Notwithstanding anything in this Section 2, in no event shall the
Unpaid Amounts be due and payable later than January 18, 2004.

                  3. EVENTS OF DEFAULT. Upon the occurrence of any of the
following events ("Events of Default"):

                  (a) Failure to pay any principal of this Promissory Note,
                  including any prepayments required hereunder, when due which
                  shall remain unremedied for forty-five (45) days after notice
                  by the Company given in writing to the Borrower; or


<PAGE>   4
                                                                               4


                  (b) Failure to pay any interest due under this Secured
                  Promissory Note which shall remain unremedied for forty-five
                  (45) days after notice by the Company given in writing to the
                  Borrower; or

                  (c) A petition is filed by or against the Borrower seeking the
                  entry of an order for relief under the bankruptcy laws of the
                  United States, as now or hereafter amended or supplemented, or
                  there is an appointment of a permanent receiver or a permanent
                  trustee of all or substantially all the property of the
                  Borrower or an assignment is made by the Borrower for the
                  benefit of creditors;

then, and in any such event, the holder of this Secured Promissory Note may
declare, by notice of default given to Borrower, the entire principal amount of
this Secured Promissory Note to be forthwith due and payable, whereupon the
entire principal amount of this Secured Promissory Note outstanding and all
accrued and unpaid interest shall become due and payable without presentment,
demand, protest and notices of any kind or of dishonor, all of which are hereby
expressly waived. Upon the occurrence of an Event of Default, the accrued and
unpaid interest hereunder shall thereafter bear the same rate of interest as the
principal hereunder, but in no event shall such interest be charged which would
violate any applicable usury law. If an Event of Default shall occur hereunder,
Borrower shall pay costs of collection, including reasonable attorneys' fees,
incurred by the holder in the enforcement hereof.

                  No delay or failure by the holder of this Secured Promissory
Note in the exercise of any right or remedy shall preclude other or future
exercise thereof or the exercise of any other right or remedy.

                 4. ADDITIONAL RESTRICTIONS UPON OCCURRENCE OF AN EVENT OF
DEFAULT. In case an Event of Default shall occur and be continuing, the Borrower
agrees that he will not, without the prior written consent of the Company (which
consent shall not be unreasonably withheld), sell, assign, transfer, exchange,
or otherwise dispose of, or grant any option with respect to, any of the Pledged
Security (except for the purpose of curing any Event of Default hereunder)
pursuant to Section 1(a) hereof, nor will he create, incur or permit to exist
any pledge, lien, mortgage, hypothecation, security interest, charge, option or
any other encumbrances with respect to any of the Pledged Security, or any
interest therein, or any proceeds thereof.

                 5. MISCELLANEOUS.
<PAGE>   5
                                                                               5


                  (a) The provisions of this Secured Promissory Note shall be
                  governed by and construed in accordance with the laws of the
                  State of Delaware, without regard to the conflicts of laws
                  rules thereof.

                  (b) All notices and other communication hereunder shall be in
                  writing and will be deemed to have been duly given if
                  delivered in person or mailed by certified mail or guaranteed
                  overnight delivery service to the Company at its principal
                  executive offices and to the Borrower at the last address
                  reflected in the Company's records.

                  (c) The paragraph headings contained in this Promissory Note
                  are for reference purposes only and shall not affect in any
                  way the meaning or interpretations of the provisions thereof.

                  IN WITNESS WHEREOF, this Secured Promissory Note has been duly
executed and delivered by Borrower on the date first written above.

                                                /s/ Jeffrey A. Cole
                                                --------------------------------
                                                Borrower: Jeffrey A. Cole

Witness:

/s/ Cristina C. Garrett
--------------------------------
Cristina C. Garrett