<PAGE>
 
                                                                   EXHIBIT 10.10

                                PROMISSORY NOTE

$6,000,000.00  FEBRUARY 3, 1997
     BIRMINGHAM, ALABAMA


     FOR VALUE RECEIVED, the undersigned JUST FOR FEET, INC., an Alabama
corporation (the "Borrower"), hereby promises to pay to the order of COMPASS
BANK (the "Lender"), at its office at 15 South 20th Street, Birmingham, Alabama
35233, or at such other place as Lender may direct, in lawful money of the
United States of America constituting legal tender in payment of all debts and
dues, public and private, together with interest thereon calculated at the rate
and in the manner set forth herein, the principal amount of SIX MILLION AND
NO/100 DOLLARS ($6,000,000.00), or so much thereof as may be advanced and
outstanding hereunder.  Payment of principal and interest shall be in accordance
with the following provisions:

     1.  CONSTRUCTION PHASE.  From the date hereof until November 15, 1997 (the
"CONVERSION DATE"), the following provisions shall apply:

     (A) Borrower promises to pay interest monthly on or before the 15th day of
         each month, on the principal amount owing hereunder from time to time,
         computed daily in the manner and at the Applicable Rate set forth
         below; the first such interest payment shall be due and payable on
         March 15, 1997. (the "CONSTRUCTION PHASE MATURITY DATE").

     (B) The applicable interest rate (the "APPLICABLE RATE") during the
         Construction Phase shall be determined and paid and shall be equal to
         125 basis points (1.25%) in excess of "Libor" (as defined herein) from
         time to time in effect (the "ADJUSTED LIBOR RATE"). "LIBOR" refers to
         (a) the London Interbank Offered Rate for periods of thirty (30) days
         (the "LIBOR ADJUSTMENT PERIOD") as quoted on the Reuter's or Telerate
         Information System on the date of determination of the interest rate
         (or in the event no such quotation is available on such date, as quoted
         on the day most immediately preceding the date of determination on
         which such a quotation was available) or (b) in the event the Reuter's
         or Telerate Information System ceases to be available to Lender for any
         reason or ceases to provide information sufficient to determine the
         London Interbank Offered Rate for the applicable Libor Adjustment
         Period, the London Interbank Offered Rate for the applicable Libor
         Adjustment Period as published in the Wall Street Journal on the date
         of determination of the interest rate (or in the event no such
         quotation is available on such date, as quoted on the day most
         immediately preceding the date of determination on which such a
         quotation was available). The interest rate shall be determined as of
         the date any selection of the Adjusted Libor Rate becomes effective and
         as of the first day following the end of each succeeding Libor
         Adjustment Period during the remaining term of this Note for so long as
         the Adjusted Libor Rate is the applicable rate in effect (or in the
         event no such quotation is available on such date, as quoted on the day
         most immediately preceding the date of determination on which such a
         quotation was available) (the "INTEREST ADJUSTMENT DATES") and will be
         adjusted as of each Interest Adjustment Date to correspond to any
         change in Libor.

PROMISSORY NOTE
PAGE 1 
<PAGE>
 
     (C) Unless the maturity date of the Note is further extended pursuant to
         Section 2 hereof, all principal, interest and other charges hereunder
         shall be due and payable on November 15, 1997.

     2.  TERM PHASE.  In the event Borrower satisfies all Conditions for
Conversion to the Term Phase as set forth in Commitment Letter (hereafter
defined), then beginning on the first day of the month next following Borrower's
satisfaction of such Conditions for Conversion to the Term Phase (the "TERM
PHASE CONVERSION DATE") the following provisions shall apply:

     (A) At least five (5) days, but no more than ten (10) days prior to the
         Term Phase Conversion Date, Borrower shall have a one-time right to
         notify Lender in writing of its selection of one of the interest rate
         options set forth in Sections 2(b) or 2(c) below which shall be the
         applicable interest rate during the Term Phase. In the event that
         Lender has not received a written selection from Borrower of an
         interest rate option at least three days prior to the Term Phase
         Conversion Date, then Borrower shall be deemed to have selected the
         Libor Rate Option set forth in Section 2(b) below. Once either of the
         interest rate options in Section 2(b) or Section 2(c) has been selected
         (or deemed selected), the interest rate option shall not thereafter
         change for the term of the loan (provided, that if the Fixed Rate
         Option is selected, the applicable rate will revert back to the Libor
         Adjusted Rate at the end of sixty months from the commencement of the
         Term Phase as set forth below).

     (B) LIBOR RATE OPTION. If the provisions of this Section 2(b) shall be
         applicable, then the applicable interest rate during the Term Phase of
         this Note shall be determined and paid and shall be equal to the
         Adjusted Libor Rate as defined in Section 1(b) above.

     (C) FIXED RATE OPTION. The Fixed Rate Option shall be based on the fixed
         rate market (as determined by Lender in its sole discretion) on the
         Term Phase Conversion Date. Borrower understands that Lender will not
         be able to quote the fixed rate under this Section 2(c) more than ten
         (10) days prior to the Term Phase Conversion Date. If Borrower shall
         notify Lender in writing of its selection of the interest rate set
         forth in this Section 2(c), then the applicable interest rate during
         the period beginning on the Term Phase Conversion Date and ending sixty
         (60) months thereafter, shall be a fixed rate per annum and shall be
         equal to the fixed rate quoted by the Lender's investment department
         (which quoted rate shall be based on the Adjusted Libor Rate swap
         equivalent yield as determined and calculated by the Lender's
         investment department on the Conversion Date). Borrower understands
         that Lender may not be offering the Fixed Rate Option at the time of
         the Term Phase Conversion Date and, if such Fixed Rate Option is not
         offered at that time, then the interest rate for the Term Phase shall
         be the rate set forth in Section 2(b). Upon the expiration of sixty
         (60) months from the Term Phase Conversion Date, the applicable
         interest rate during the remaining portion of the Term Phase of this
         Note shall be determined and paid and shall be equal to the Adjusted
         Libor Rate as defined in Section 1(b) above.

     (D) If the principal balance of this Note at the Term Phase Conversion Date
         is $7,100,000.00, then throughout the Term Phase principal payments
         shall be made in monthly installments of Thirty-Nine Thousand Four
         Hundred Forty-Four and 44/100 Dollars ($39,444.44), commencing on the
         first day of the calendar month following the Term Phase Conversion
         Date, and a final installment equal to the total unpaid principal,
         interest and other charges, due and payable on the Term Maturity Date
         (defined below). If the principal balance of this Note at the Term
         Phase Conversion Date is other than $7,100,000.00 (the "Actual
         Principal Balance"), then throughout the Term Phase principal payments
         shall be made in monthly installments equal to 1/180 of such Actual
         Principal Balance. In addition to 

PROMISSORY NOTE
PAGE 2 

                                       2
<PAGE>
 
         principal, interest shall be paid monthly, commencing on the first day
         of the calendar month following the Term Phase Conversion Date.

     (E) All principal, accrued and unpaid interest and other outstanding
         charges hereunder shall be due and payable on November 15, 2007 (the
         "TERM MATURITY DATE").

3.   ADDITIONAL PROVISIONS REGARDING INTEREST. The interest rate applicable
under this Note shall not be less than five percent (5%).  Interest on all
principal amounts outstanding from time to time hereunder shall be calculated on
the basis of a 360-day year applied to the actual number of days upon which
principal is outstanding, by multiplying the product of the principal amount and
the applicable rate set forth herein by the actual number of days elapsed, and
dividing by 360.  In no event shall the rate of interest calculated hereunder
exceed the maximum rate allowed by law.  Any principal amounts outstanding
hereunder after maturity or earlier acceleration of this Note shall bear
interest at a floating rate of two percentage points (2%) in excess of Compass
Bank Prime.  "COMPASS BANK PRIME", as used in this Note, is a reference rate
established by the Lender for use in computing and adjusting interest, is
subject to increase, decrease, or change at the Lender's discretion, and is only
one of the reference rates or indices that Lender uses.  Borrower acknowledges
that the Lender may lend to others at rates of interest at, or greater or less
than, "Compass Bank Prime" or the rate provided herein.  Each change in the
interest rate resulting from a change in "Compass Bank Prime" shall become
effective on the day on which such change in "Compass Bank Prime" occurs.

     4.  PREPAYMENT. So long as the applicable interest rate under this Note is
equal to the Adjusted Libor Rate as defined in Section 1(b) above, then this
Note may be prepaid in whole or in part, and without charge, on any Interest
Adjustment Date. If the Fixed Rate Option is selected by Borrower, then so long
as the applicable interest rate is based on the fixed rate set forth in Section
2(c), this Note may be prepaid in whole, but not in part, and any prepayment in
full shall be accompanied by a prepayment fee calculated as follows:

     Loan year (of Term Phase)      Percentage of principal prepaid
     in which prepayment made       charged as prepayment fee
     ------------------------       -------------------------

              1st                             5%
              2nd                             4%
              3rd                             3%
              4th                             2%
              5th                             1% 

Notwithstanding the foregoing, no prepayment fee shall be charged for a
prepayment in full or in part during a fixed rate period under either of the
circumstances set forth in (A) or (B) below:

     A.  If Borrower prepays the loan with internally-generated funds that are
         not proceeds from another loan transaction; or
     B.  If:

         (i)  Borrower desires to further develop the Mortgaged Property with
              additional or expanded buildings and facilities (the "ADDITIONAL
              DEVELOPMENT FACILITIES"), and intends to finance such further
              development with a loan or loans to be secured by a mortgage(s) on
              the Additional Development Facilities, and

         (ii) either:

              (a)  Lender will not consent to the release of the 

PROMISSORY NOTE
PAGE 3 

                                       3
<PAGE>
 
                    Additional Development Facilities from its mortgage, or to a
                    second mortgage from another lender on the Additional
                    Development Facilities, or

               (b)  Lender will consent to a second mortgage, but Borrower's
                    other lender will not accept a second mortgage on the
                    Additional Development Facilities to secure its loan, or

               (c)  Lender and Borrower are unable to agree on the financing of
                    the Additional Development Facilities by Bank,

                         then, in any of such circumstances (a)-(c), and if
Borrower proceeds with the financing of the Additional Development Facilities
with another lender, Borrower may prepay the Loan in full without a prepayment
fee as a part of the closing on such other financing.

In the event of acceleration of this Note at any time and subsequent involuntary
or voluntary prepayment, and if the Fixed Rate Option is then in effect, any fee
set forth herein for a prepayment of the Note in full shall be applicable.

     5.   LOAN DOCUMENTS.  The indebtedness evidenced hereby is secured by,
inter alia, the Construction Loan Agreement (the "LOAN AGREEMENT") executed by
the Borrower in favor of the Lender as of the date hereof, and which
incorporates the Commitment Letter from Lender to Borrower dated September 24,
1996 (the "COMMITMENT LETTER"), the Future Advance Mortgage, Assignment of Rents
and Leases and Security Agreement from Borrower to Lender dated as of the date
hereof, and the other Loan Documents defined in the Loan Agreement
(collectively, the "LOAN DOCUMENTS").

     This Note is included in the indebtedness referred to in the Loan Documents
and is entitled to the benefits of those documents, but neither this reference
to those documents nor any provisions thereof shall affect or impair the
absolute and unconditional obligations of the Borrower to pay the principal of
and interest on this Note when due.

     6.   EVENTS OF DEFAULT.  Upon the occurrence of any one or more of the
following events ("EVENTS OF DEFAULT"):

          (A)  Default in the payment of the principal of or interest on this
     Note, as and when due and payable; and

          (B)  The occurrence of any Event of Default specified in the Loan
     Agreement, the other Loan Documents, or in any other instrument executed in
     connection with or securing this Note which is not cured within any cure
     period provided with respect thereto (if any),

then, or at any time thereafter during the continuance of any such event, the
Lender may, with or without notice to the Borrower, declare this Note and
indebtedness evidenced hereby to be forthwith due and payable, whereupon this
Note and the indebtedness evidenced hereby shall become forthwith due and
payable, both as to principal and interest, without presentment, demand,
protest, or other notice of any kind, all of which are hereby expressly waived,
anything contained herein or in any of the Loan Documents or in any other
instrument executed in connection with or securing this Note to the contrary
notwithstanding.

     7.   LATE FEE.  Any scheduled payment of principal and/or interest which is
not paid within 

PROMISSORY NOTE
PAGE 4 

                                       4
<PAGE>
 
ten (10) days from the date due will be subject to a late charge of five percent
(5%) of such scheduled payment.

     8.   WAIVERS.  Borrower hereby waives demand, presentment for payment,
notice of dishonor, protest, and notice of protest and diligence in collection
or bringing suit and agrees that the Lender may accept partial payment, or
release or exchange security or collateral, without discharging or releasing any
unreleased collateral or the obligations evidenced hereby.  Borrower further
waives any and all rights of exemption, both as to personal and real property,
under the constitution or laws of the United States, the State of Alabama or any
other state.

     9.   ATTORNEYS' FEES.  Borrower agrees to pay reasonable attorneys' fees
and costs incurred by the Lender in collecting or attempting to collect this
Note, whether by suit or otherwise.

     10.  MISCELLANEOUS.  As used herein, the terms "BORROWER" and "LENDER"
shall be deemed to include their respective successors, legal representatives
and assigns, whether by voluntary action of the parties or by operation of law.
This Note is given under the seal of all parties hereto, and it is intended that
this Note is and shall constitute and have the effect of a sealed instrument
according to law.  This Note has been negotiated, and is being executed and
delivered in Birmingham, in the State of Alabama, or if executed elsewhere,
shall become effective upon the Lender's receipt and acceptance of the executed
original of this Note in the State of Alabama; provided, however, that the
Lender shall have no obligation to give, nor shall Borrower be entitled to
receive, any notice of such acceptance for this Note to become a binding
obligation of Borrower.  Borrower hereby submits to jurisdiction in the State of
Alabama.  THIS NOTE SHALL BE GOVERNED BY AND BE CONSTRUED IN ACCORDANCE WITH THE
LAWS OF THE STATE OF ALABAMA. IT IS INTENDED, AND THE BORROWER AND THE LENDER
HEREOF SPECIFICALLY AGREE, THAT THE LAWS OF THE STATE OF ALABAMA GOVERNING
INTEREST SHALL APPLY TO THIS NOTE AND TO THIS TRANSACTION.  This Note may not be
modified except by written agreement signed by the Borrower and the Lender
hereof, or by their respective successors or assigns.

PROMISSORY NOTE
PAGE 5 

                                       5
<PAGE>
 
     IN WITNESS WHEREOF, Borrower has caused this Note to be executed, sealed
and delivered as of the date first set forth above.

                                    BORROWER:

ATTEST:                             JUST FOR FEET, INC.



By:                                 By:
     Its:                                Its:



STATE OF ALABAMA

COUNTY OF JEFFERSON

     I, _______________________________________________________, a notary public
in and for said county in said state, hereby certify that_______________________
whose name as __________________________________ of JUST FOR FEET, INC., a
corporation, is signed to the foregoing instrument and who is known to me,
acknowledged before me on this day that, being informed of the contents of such
instrument, he, as such officer and with full authority, executed the same
voluntarily for and as the act of said corporation.

     Given under my hand and official seal this 3/rd/ day of February, 1997.



 
                                         Notary Public

     [ Notarial Seal ]                   My Commission Expires:

PROMISSORY NOTE 
PAGE 6

                                       6