<PAGE>   1

                                                                    EXHIBIT 4.11

                        MODIFICATION TO CREDIT AGREEMENT

         This Modification to Credit Agreement (this "Modification") dated the
1st day of May, 2001 (this "Modification"), is made by and between WALL STREET
DELI, INC., a Delaware corporation ("Borrower") and AMSOUTH BANK, an Alabama
banking corporation ("Lender").

                                    RECITALS

         A.       Borrower and Lender have heretofore entered into that certain
Credit Agreement dated as of June 19, 1996, as amended by that certain Amended
and Restated Credit Agreement dated February 2, 1999, that certain Modification
to Credit Agreement dated October 25, 1999, that certain Modification to Credit
Agreement dated October 29, 1999, that certain Modification to Credit Agreement
dated November 30, 1999, that certain Modification to Credit Agreement dated
January 27, 2000, that certain Modification to Credit Agreement dated March 3,
2000, that certain Modification to Credit Agreement dated November 10, 2000,
that certain Modification to Credit Agreement dated January 5, 2001, that
certain Modification to Credit Agreement dated February 5, 2001, and that
certain Modification to Credit Agreement dated March 23, 2001 (the "Credit
Agreement") pursuant to which the Lender agreed to make available to the
Borrower a credit facility in the original maximum principal amount of
$4,000,000 as evidenced by that certain Master Note from Borrower to Lender
dated December 26, 1996, as amended by that certain Amended and Restated Master
Note dated February 1, 1999, that certain Note Modification Agreement dated
October 25, 1999, that certain Note Modification Agreement dated January 5,
2001, and that certain Note Modification Agreement dated February 5, 2001 (the
"Note"). The Credit Agreement and Note shall hereinafter collectively be
referred to as the "Credit Documents"). Capitalized terms used herein and not
otherwise defined shall have the meanings given to them in the Credit Documents.

         B.       Borrower has requested certain amendments to the Credit
Documents. Lender has agreed to such amendments to the Credit Documents as more
particularly set forth herein and requires this Modification as evidence
thereof.

                                    AGREEMENT

         NOW, THEREFORE, in consideration of the foregoing Recitals and other
good and valuable consideration, the Borrower and Lender agree as follows:

         1.       The Recitals herein are true and correct.

         2.       The Credit Agreement is hereby amended by deleting Section 1.2
(aj) in its entirety and inserting in lieu thereof the following:

                  (a)      TERMINATION DATE means May 17, 2001, the maturity
         date of the Obligations, as such date may be extended from time to time
         pursuant to Section 2.5 or accelerated pursuant to Section 6.2.

         3.       As a condition to this Modification, Borrower shall pay to
Lender a modification fee of $1,000 due upon execution and delivery of this
Modification. Additionally, Borrower agrees to directly pay


                                       24
<PAGE>   2

and reimburse the Lender for all expenses, including the reasonable fees and
expenses of legal counsel, incurred by the Lender in connection with the
preparation of the documentation to evidence this Modification.

         4.       Borrower represents and warrants to the Lender that all
representations and warranties given by the Borrower to the Lender in Article
III of the Credit Agreement are true and correct as of the date hereof, except
to the extent affected by this Modification.

         5.       Except as herein modified, the Credit Agreement shall remain
in full force and effect, and the Credit Agreement as so modified, is hereby
ratified and affirmed in all respects. The Borrower confirms that it has no
offsets or defenses with respect to its obligations pursuant to the Credit
Agreement, as herein modified, and represents that this Modification has been
duly authorized, executed and delivered pursuant to all necessary action of the
Borrower. Furthermore, Borrower hereby releases, satisfies, conceals, waives,
acquits, and forever discharges the Lender, its directors, officers, employees,
agents, attorneys, successors and assigns, of and from any and all claims,
demands, actions, or causes of action of any kind or character, arising at any
time in the past, up to and including the date of this Modification, which
relate or pertain in any way to the Credit Documents or the indebtedness
evidenced thereby and/or the collection of such indebtedness.

         6.       This Modification shall inure to the benefit of and be binding
upon the parties hereto, and their respective successors and assignors.

         7.       This Modification may be executed in counterparts, each of
which shall be an original, but all of which when taken together shall
constitute one and the same instrument.

         8.       The Borrower irrevocably (a) acknowledges that this
Modification will be accepted by the Lender and performed by the Borrower in the
State of Alabama; (b) submits to the jurisdiction of each state or federal court
sitting in Jefferson County, Alabama (collectively, the "Courts") over any suit,
action or proceeding arising out of or relating to this Modification or any of
the other Credit Documents (an "Action"); (c) waives, to the fullest extent
permitted by law, any objection or defense that the Borrower may now or
hereafter have based on improper venue, lack of personal jurisdiction,
inconvenience of forum or any similar matter in any Action brought in any of the
Courts; (d) agrees that final judgment in any Action brought in any of the
Courts shall be conclusive and binding upon the Borrower and may be enforced in
any other court to the jurisdiction of which the Borrower is subject, by a suit
upon such judgment; (e) consents to the service of process on the Borrower in
any Action by the mailing of a copy thereof by registered or certified mail,
postage prepaid, to the Borrower at the Borrower's address designated in or
pursuant to Section 7.1; (f) agrees that service in accordance with Section
7.14(e) of the Credit Amendment shall in every respect be effective and binding
on the Borrower to the same extent as though served on the Borrower in person by
a person duly authorized to serve such process; and (g) AGREES THAT THE
PROVISIONS OF THIS PARAGRAPH, EVEN IF FOUND NOT TO BE STRICTLY ENFORCEABLE BY
ANY COURT, SHALL CONSTITUTE "FAIR WARNING" TO THE BORROWER THAT THE EXECUTION OF
THIS MODIFICATION MAY SUBJECT THE BORROWER TO THE JURISDICTION OF EACH STATE OR
FEDERAL COURT SITTING IN JEFFERSON COUNTY, ALABAMA WITH RESPECT TO ANY ACTIONS,
AND THAT IT IS FORESEEABLE BY THE BORROWER THAT THE BORROWER MAY BE SUBJECTED TO
THE JURISDICTION OF SUCH COURTS AND MAY BE SUED IN THE STATE OF ALABAMA IN ANY
ACTIONS. Nothing in this Paragraph shall limit or restrict the Lender's right to
serve process or bring Actions in courts otherwise than as herein provided.


                                       25
<PAGE>   3

         9.       The exercise by the Lender of any option given to it under the
Credit Agreement shall not constitute a waiver of the right to exercise any
other option. No failure or delay on the part of the Lender in exercising any
right, power or remedy under the Credit Agreement shall operate as a waiver
thereof, nor shall any single or partial exercise of any such right, power or
remedy preclude any further exercise thereof or the exercise of any other right,
power or remedy.

         10.      This Modification shall inure to the benefit of and be binding
upon the parties hereto and their respective successor and assigns.

         11.      Inapplicability or unenforceability of any provisions of this
Modification shall not limit or impair the operation or the validity of any
other provision of this Modification.

         IN WITNESS WHEREOF, the parties have caused this Modification to be
duly executed under seal as of the day and year first above written.

                                      BORROWER:

                                      WALL STREET DELI, INC.,
                                      a Delaware corporation


                                      By: /s/  Jeffrey V. Kaufman
                                         ---------------------------------------
                                      Printed Name: Jeffrey V. Kaufman
                                                   -----------------------------
                                      Its: Chief Executive Officer
                                          --------------------------------------

                                                          [Affix corporate seal]

                                      Attest: /s/ Thomas J. Sandeman
                                             -----------------------------------
                                      Printed Name: Thomas J. Sandeman
                                                   -----------------------------
                                      Title: Secretary
                                            ------------------------------------

                                                          [Affix corporate seal]

STATE OF ALABAMA           )
COUNTY OF JEFFERSON        )

         I, the undersigned, a Notary Public in and for said County, in said
State, hereby certify that Jeffrey V. Kaufman whose name as Chief Executive
Officer of Wall Street Deli, Inc., a Delaware corporation, is signed to the
foregoing instrument, and who is known to me, acknowledged before me on this day
that, being informed of the contents of the instrument, s/he, as such officer
and with full authority, executed the same voluntarily on the day the same bears
date.

         Given under my hand and official seal, this 1st day of May, 2001.


                                      /s/   Denis Mitchell
                                      ------------------------------------------
                                      Notary Public
                                      My commission expires: 2/4/04
                                                            --------------------

[SEAL]


                                       26
<PAGE>   4

                                      LENDER:

                                      AMSOUTH BANK,
                                      an Alabama banking corporation

                                      By:        /s/  Darlene E. Chandler
                                         ---------------------------------------
                                      Printed Name: Darlene E. Chandler
                                                   -----------------------------
                                      Title: Vice President
                                            ------------------------------------

STATE OF ALABAMA           )
COUNTY OF JEFFERSON        )

         I, the undersigned, a Notary Public in and for said County, in said
State, hereby certify that Darlene E. Chandler whose name as Vice President of
AmSouth Bank, an Alabama banking corporation, is signed to the foregoing
instrument, and who is known to me, acknowledged before me on this day that,
being informed of the contents of the instrument, s/he, as such officer and with
full authority, executed the same voluntarily on the day the same bears date.

         Given under my hand and official seal, this 2nd day of May, 2001.


                                      /s/   Hattie Evans
                                      Notary Public
                                      My commission expires: 1/17/04
                                                            --------------------

[SEAL]


                                       27