<PAGE>

                                                                 EXHIBIT 10.17.4



================================================================================
SECOND AMENDMENT TO AMENDED AND
RESTATED LOAN AND SECURITY AGREEMENT                   FLEET RETAIL FINANCE INC.
================================================================================

         This Second Amendment to Amended and Restated Loan and Security
Agreement (the "SECOND AMENDMENT") is made as of this 26th day of November, 2003
by and among

                  FLEET RETAIL FINANCE INC. (the "LENDER"), a Delaware
         corporation with offices at 40 Broad Street, Boston, Massachusetts
         02019,

                  and

                  BAKERS FOOTWEAR GROUP, INC., f/k/a Weiss and Neuman Shoe Co
         (the "BORROWER"), a Missouri corporation with its principal executive
         offices at 2815 Scott Avenue, Suite C, St. Louis, Missouri 63103,

in consideration of the mutual covenants contained herein and benefits to be
derived herefrom,

                                  WITNESSETH:

A. Reference is made to that certain Amended and Restated Loan Security
Agreement (as amended to date, the "LOAN AGREEMENT") dated as of June 11, 2002,
between the Borrower and the Lender.

B. The Borrower and the Lender desire to modify and amend the Loan Agreement to
extend the Maturity Date from December 31, 2004 to January 5, 2005, subject to
the terms and conditions of this Second Amendment.

         Accordingly, the Borrower and the Lender agree as follows:

         1. DEFINITIONS. Capitalized terms used herein and not otherwise defined
herein shall have the meanings assigned to such terms in the Loan Agreement.

         2. AMENDMENT TO ARTICLE 1.

                  (a) In Article 1 of the Loan Agreement, the definition of
         "Maturity Date" is hereby deleted in its entirety and replaced by the
         following:

                  ""Maturity Date": January 5, 2005."

         3. ADDITIONAL ACKNOWLEDGEMENTS AND REPRESENTATIONS. As an inducement
for the Lender to execute this Second Amendment, the Borrower hereby represents
and warrants that as of the date hereof (i) no Suspension Event has occurred and
is continuing; and (ii) none of the principal outstanding under any Permitted
Subordinated Indebtedness has matured or otherwise become due and payable and
the maturity date of all of such Permitted Subordinated Indebtedness is later
than the Maturity Date as amended by this Second Amendment.

<PAGE>
         4. RATIFICATION OF LOAN DOCUMENTS: NO CLAIMS AGAINST LENDER. Except as
provided herein, all terms and conditions of the Loan Agreement and of the other
Loan Documents remain in full force and effect. Each of the Borrower and the
Guarantor hereby ratifies, confirms, and re-affirms all and singular the terms
and conditions, including execution and delivery, of the Loan Documents. There
is no basis nor set of facts on which any amount (or any portion thereof) owed
by the Borrower or the Guarantor to the Lender could be reduced, offset, waived,
or forgiven, by rescission or otherwise; nor is there any claim, counterclaim,
offset, or defense (or other right, remedy, or basis having a similar effect)
available to the Borrower or to the Guarantor with regard to the respective
Liabilities of the Borrower and the Guarantor to the Lender; nor is there any
basis on which the terms and conditions of any of the respective Liabilities of
the Borrower and of the Guarantor to the Lender could be claimed to be other
than as stated on the written instruments which evidence such Liabilities. To
the extent that the Borrower or the Guarantor has (or ever had) any such claims
against the Lender, each hereby affirmatively WAIVES and RELEASES same.

         5. CONDITIONS TO EFFECTIVENESS. This Second Amendment shall not be
effective until each of the following conditions precedent have been fulfilled
to the satisfaction of the Lender:

                  (a) This Second Amendment shall have been duly executed and
         delivered by the respective parties hereto, shall be in full force and
         effect and shall be in form and substance satisfactory to the Lender;

                  (b) All action on the part of the Borrower necessary for the
         valid execution, delivery and performance by the Borrower of this
         Second Amendment shall have been duly and effectively taken and
         evidence thereof satisfactory to the Lender shall have been provided to
         the Lender;

                  (c) The Borrower shall have paid to the Lender all fees and
         expenses then due and owing pursuant to the Loan Agreement; and

                  (d) The Borrower shall have provided such additional
         instruments and documents to the Lender as the Lender and Lender's
         counsel may have reasonably requested, each in form and substance
         satisfactory to the Lender.

         6. MISCELLANEOUS

                  (a) This Second Amendment may be executed in several
         counterparts and by each party on a separate counterpart, each of which
         when so executed and delivered shall be an original, and all of which
         together shall constitute one instrument.

                  (b) This Second Amendment expresses the entire understanding
         of the parties with respect to the transactions contemplated hereby. No
         prior negotiations or discussions shall limit, modify, or otherwise
         affect the provisions hereof.

                  (c) Any determination that any provision of this Second
         Amendment or any application hereof is invalid, illegal, or
         unenforceable in any respect and in any instance shall not affect the
         validity, legality, or enforceability of such provision in any other
         instance, or the validity, legality, or enforceability of any other
         provisions of this Second Amendment.

                  (d) The Borrower shall pay on demand all reasonable costs and
         expenses of the Lender, including, without limitation, reasonable
         attorneys' fees in connection with the preparation, negotiation,
         execution, and delivery of this Second Amendment.

                                      -2-
<PAGE>
         (e) This Second Amendment shall be construed, governed, and enforced
pursuant to the internal laws of The Commonwealth of Massachusetts and shall
take effect as scaled instrument.


                        [SPACE LEFT INTENTIONALLY BLANK]



                                      -3-
<PAGE>
    IN WITNESS WHEREOF, the parties have hereunto caused this Second Amendment
to be executed and their seals to be hereto affixed as of the date first above
written.



                                           BAKERS FOOTWEAR GROUP, INC.,
                                           F/K/A WEISS AND NEUMAN SHOE CO.


                                           By: /s/ PETER EDISON
                                              ----------------------------------

                                           Name: Peter Edison
                                                --------------------------------

                                           Title: Chairman/CEO
                                                 -------------------------------


                                           FLEET RETAIL FINANCE INC.

                                           By: /s/ JAMES J. WOOD
                                              ----------------------------------

                                           Name: James J. Wood
                                                --------------------------------

                                           Title: Managing Director
                                                 -------------------------------



                                      S-1

<PAGE>

    The undersigned Guarantor hereby (i) consents to the terms and conditions of
this Second Amendment and hereby joins in the acknowledgments set forth in this
Second Amendment, all as of the date first above written, (ii) ratifies and
confirms the warranties and representations set forth in the Guaranty, and
acknowledges that pursuant to the terms of the Guaranty, the Guarantor
previously guaranteed the payment of the Liabilities of the Borrower to the
Lender to the extent set forth in such Guaranty, that this acknowledgment is
being executed as a confirmation of the Guarantor's obligations to Lender under
the Guaranty and that, subject to the limitations contained in the Guaranty, the
Guarantor shall remain liable for all of the Liabilities, now existing or
hereafter arising, whether or not any similar confirmation letter is executed
in the future and (iii) acknowledges and agrees that he has no offsets,
defenses, or counterclaims against the Lender with respect to his obligations
under the Guaranty or otherwise, and to the extent that the Guarantor has any
such offsets, defenses, or counterclaims, the Guarantor hereby WAIVES and
RELEASES the same.




                                              PETER EDISON
                                              ----------------------------------
                                              PETER EDISON





                                      S-2