<PAGE>   1
                                                                    Exhibit 10.4

                            ASSIGNMENT AND ASSUMPTION OF LEASE


         This Assignment and Assumption of Lease ("ASSIGNMENT") is made this
18th day of May, 2001 by and between Famous Dave's Ribs-U, Inc., a Minnesota
corporation ("ASSIGNOR") and FUMUME II, LLC, a Delaware limited liability
company ("ASSIGNEE") with reference and respect to the following facts and
circumstances:

         A.       D & H Building Corporation, an Illinois corporation (the
                  "LANDLORD") is the Owner of that certain building which is
                  located at 739- 743 North Clark Street, Chicago, Illinois, and
                  more particularly described as Lots 9, 10, 11, and 17 on the
                  site plan attached to the Lease, as hereinafter described (the
                  "PREMISES").

         B.        Landlord and Assignor entered into a certain Lease Agreement
                   dated September 16, 1997, which lease was amended by that
                   certain First Amendment to Lease dated November 6, 1997 and
                   that certain Second Amendment to Lease dated November 10,
                   1998 (collectively, the "LEASE") for the Premises.

         C.       The Premises covered by the Lease are identified as 735 Clark
                  Street, Chicago, Illinois.

         D.       The Assignor desires to assign its right, title and interest
                  in, to and under the Lease and the Premises to Assignee, and
                  Assignee desires to accept such assignment upon and subject to
                  all of their terms and conditions hereinafter set forth.

         NOW, THEREFORE, in consideration of the foregoing facts and
circumstances, the mutual covenants and promises contained herein and after good
and valuable consideration, the receipt and sufficiency of which is acknowledged
by each of the parties, the parties do hereby agree to the following:

         1. Assignment and Assumption. Assignor hereby assigns to Assignee all
of its right, title and interest in, to and under the Lease and the Premises
(including all of Assignor's rights, title, and interest in and to any prepaid
rents as have been paid by Assignor pursuant to the Lease), and Assignee hereby
accepts such assignment, assumes all of Assignor's obligations under the Lease,
agrees to be bound by all of the provisions thereof and to perform all of the
obligations of the tenant thereunder from and after the effective date hereof.
Such assignment and assumption is made upon, and is subject to, all of the
terms, conditions and provisions of this Assignment.

         2. Condition of Premises. The Premises shall be delivered by Assignor
to Assignee in "As Is, Where Is" condition with all leasehold improvements
located thereon. All of such leasehold improvements owned by Assignor shall be
and become the property of Assignee subject to no liens, conditional sales
contracts, or other encumbrances except for such rights as the Landlord may have
in same as provided in the Lease.
<PAGE>   2
         3. Security Deposit. Assignor hereby transfers and assigns of all its
right, title and interest in the security deposit held by Landlord to the
Assignee which the parties acknowledge is in the amount of $84,833.33.

         4. Miscellaneous.

         4.1. Attorneys' and Other Fees. Should any party institute any action
or proceeding, to enforce or interpret this Assignment or any provision hereof,
for damages by reason of any alleged breach of this Assignment or of any
provision hereof, or for a declaration of rights hereunder, the prevailing party
in any such action or proceeding shall be entitled to receive from the other
party all costs and expenses, including actual attorneys' and other fees,
reasonably incurred in good faith by the prevailing party in connection with
such action or proceeding. The term "ATTORNEYS' AND OTHER FEES" shall mean and
include attorneys' fees, accountants' fees, and any and all consultants' and
other similar fees incurred in connection with the action or proceeding and
preparations therefor. The term "ACTION OR PROCEEDING" shall mean and include
actions, proceedings, suits, arbitrations, appeals and other similar
proceedings.

         4.2. Notices. Any notice, demand, request, covenant, approval or other
communication to be given by one party to the other shall be in writing (unless
some other form of notice is specifically provided for herein) and given by
personal service, telegram, or express mail, Federal Express, DHL or any other
similar form of airborne/overnight delivery service, or mailing in the United
States mail (certified and return receipt requested), addressed to the parties
at their respective addresses as follows:

                              If to Assignor:

                              Famous Dave's Ribs-U, Inc.
                              7657 Anagram Drive
                              Eden Prairie, Minnesota 55344

                              With a copy to:

                              Maslon, Edelman Borman & Brand, LLP
                              3300 Wells Fargo Center
                              90 South Seventh Street
                              Minneapolis, Minnesota 55402
                              Attention: William Mower

                              If to Assignee:
                              FUMUME, LLC
                              7657 Anagram Drive
                              Eden Prairie, Minnesota 55344
                              Attention: Martin O'Dowd



                              2
<PAGE>   3
Any such notice shall be deemed to have been given upon delivery or refusal of
acceptance of attempted delivery. Either party may change the address at which
it desires to receive notice upon giving written notice of such request to the
other party. Assignor and Assignee, and their respective counsel, hereby agree
that notices may be given hereunder by the parties' respective counsel, and that
if any communication is to be given hereunder by Assignor's or Assignee's
counsel, such counsel may communicate directly with all principals, as required
to comply with the foregoing provisions.

         4.3. TIME OF ESSENCE. TIME IS OF THE ESSENCE OF THIS Assignment AND
EACH AND EVERY TERM AND PROVISION HEREOF.

         4.4. Waiver. No waiver by any party hereto of any breach or default
shall be considered to be a waiver of any other breach or default. The waiver of
any condition shall not constitute a waiver of any breach or default with
respect to any covenant, representation or WARRANTY.


         4.5. Successors and Assigns. This Assignment shall inure to the benefit
of and be binding upon the parties hereto and their respective heirs, successors
and assigns.

         4.6. Governing Law. This Assignment shall be governed by, interpreted
under, and construed and enforced in accordance with the laws of the State of
Minnesota applicable to agreements made and to be performed wholly within the
State of Minnesota.

         4.7. Construction. Headings at the beginning of each Section and
subsection are solely for the convenience of the parties and are not a part of
this Assignment. Except as otherwise provided in this Assignment, all exhibits
referred to herein are attached hereto and are incorporated herein by this
reference. Unless otherwise indicated, all references herein to Articles,
Sections, subsections, paragraphs, subparagraphs or provisions are to those in
this Assignment. Any reference to a Section herein includes all subsections
thereof. This Assignment shall not be construed as if it had been prepared by
only Assignor or Assignee, but rather as if both Assignor and Assignee had
prepared the same. In the event any portion of this Assignment shall be declared
by a court of competent jurisdiction to be invalid, illegal or unenforceable,
such portion shall be deemed severed from this Assignment, and the remaining
parts hereof shall remain in full force and effect, as fully as though such
invalid, illegal or unenforceable portion had never been part of this
Assignment.

         4.8. Integration of Other Agreements. This Assignment sets forth the
entire agreement and understanding of the parties with respect to the matters
set forth herein and supersedes all previous written or oral understandings,
agreements, contracts, correspondence and documentation with respect thereto.
Any oral representations or modifications concerning this Assignment shall be of
no force and effect.

         4.9. Indemnification by Assignee. Assignee agrees to indemnify, defend
and hold Assignor free and harmless of, from and against any and all claims,
demands, damages, losses, liabilities, causes of action, costs or expenses
(including reasonable attorneys' fees),


                                        3
<PAGE>   4
directly or indirectly arising in connection with the breach of any covenant,
agreement, representation or warranty of Assignee under the terms of this
Agreement.

         4.10. Duplicate Originals; Counterparts. This Assignment may be
executed in any number of duplicate originals, all of which shall be of equal
legal force and effect. Additionally, this Assignment may be executed in
counterparts, but shall become effective only after a counterpart hereof has
been executed by each party; all said counterparts shall, when taken together,
constitute the entire single Assignment between the parties.

         4.11. Non-Waiver of Rights. No failure or delay of either party in the
exercise of any right given to such party hereunder shall constitute a waiver
thereof unless the time specified herein for exercise of such right has expired,
nor shall any single or partial exercise of any right preclude other or further
exercise thereof or of any other right.

         4.12. No Third Party Beneficiaries. Except as otherwise provided
herein, no person or entity shall be deemed to be a third party beneficiary
hereof, and nothing in this Assignment (either expressed or implied) is intended
to confer upon any person or entity, other than Assignor and/or Assignee (and
their respective nominees, successors and assigns), any rights, remedies,
obligations or liabilities under or by reason of this Assignment.


                        REMAINDER OF PAGE INTENTIONALLY BLANK


                                       4
<PAGE>   5
         IN WITNESS WHEREOF, each party has executed this Assignment on the date
set forth next to its signature below, but effective as of the date first set
forth above.


Date:  6-7-01                                     "Assignor"
      --------                                    FAMOUS DAVE'S RIBS-U, INC.

                                                  By:  /s/ Kenneth Stanecki
                                                       --------------------
                                                  Name:    Kenneth Stanecki
                                                  Title:   CFO


Date:  6-7-01                                     "Assignee"
      --------                                    FUMUME II, LLC

                                                  By:  /s/ Kenneth Stanecki
                                                       --------------------
                                                  Name:    Kenneth Stanecki
                                                  Title:   CFO

                                       5