<PAGE>
 
                                                                    EXHIBIT 10.4


                              FOUNDER'S AGREEMENT


         This Founder's Agreement, dated as of December 2, 1997, is entered into
by and between SFC Acquisition Corp., a Delaware corporation (the "Company") and
Keefe, Bruyette & Woods, Inc. in its own right and on behalf of certain of its
employees to be named at a later date ("Keefe", or the "Founder") (collectively,
the "Parties", individually, a "Party").

         Whereas the Company has been organized for the purposes of acquiring
all of the issued and outstanding capital stock (the "Acquisition") of Superior
Federal Bank, (the "Bank") a subsidiary of NB Holdings Corporation, and of
operating thereafter as a unitary thrift holding company;

         Whereas, the Founder and the Company have agreed to cooperate in
raising the capital necessary for the Acquisition and to provide for the payment
of the costs of the Acquisition;

         Whereas, the Parties acknowledge that neither the terms of this
Agreement nor the fact that they have entered into this Agreement shall (i) in
any way limit, control or determine the voting rights of the Founder as to any
capital stock of the Company that the Founder may acquire in connection with the
Acquisition; (ii) provide the Founder with any rights as to the management of or
control the affairs of the Company; or (iii) entitle the Founder to any share in
any profits of the Company;

         Whereas the capital required to complete the Acquisition shall be
raised by (i) direct investment of the Founder in the amount set forth on
Schedule "A" (the "Initial Investment"), (ii) a private placement of a number of
shares of common stock of the Company, par value $0.01 (the "Common Stock"),
sufficient to raise $85.0 million to $100.0 million of equity (including any
Founder's shares) in the Company (the "Private Placement"), and (iii) an
offering of debt instruments of the Company in the principal amount of $55.0
million to $70.0 million (the "Debt Offering");

         Whereas it is intended that the Private Placement, the Debt Offering
and the Acquisition shall be consummated at the offices of NB Holdings
Corporation in Charlotte, North Carolina at a date and time to be determined
(the "Closing");

         Whereas Keefe intends to provide investment banking services in
connection with the Private Placement and to serve as lead underwriter for the
Debt Offering;

         Whereas Keefe has incurred and expects to incur significant out-of-
pocket expenses in connection with the Acquisition;

         Now Therefore, the Company and the Founder agree as follows:

                                  ARTICLE ONE
                                CAPITALIZATION
                                --------------

         1.1 FOUNDER'S INVESTMENT. The Founder agrees, subject to the conditions
             --------------------   
set forth in
<PAGE>
 
Section 1.2, to purchase, and the Company agrees, subject to the terms and
conditions of a subscription agreement for the Common Stock (the "Subscription
Agreement"), to sell a number of shares of Common Stock equal to the amount
obtained by dividing 96% of that price per share of Common Stock as shall be set
forth in a Subscription Agreement for Common Stock to be prepared in connection
with the Private Placement (the "Offering Price") into the amount of the
Founder's Initial Investment as set forth on Schedule A. The Founder's
obligation to purchase, and the Company's obligation to sell, any shares
exceeding 10% of the Company's issued and outstanding Common Stock (as defined
herein) shall be subject to approval by the Office of Thrift Supervision, or
successor agency, if any, of any rebuttal of control submission of the Founder.
However, the Founder agrees to purchase, and the Company agrees to sell, the
maximum number of shares provided for in this Agreement, subject to any
reduction required to comply with applicable regulatory restrictions on such
purchase and sale.

         1.2 CONDITIONS TO PURCHASE AND SALE. The obligation of the Founder to
             -------------------------------  
purchase Common Stock as set forth in Section 1.1 hereof shall be subject to the
fulfillment of each of the following conditions: (i) the Private Placement
(apart from the funding by the Founder of its Initial Investment) and the Debt
Offering shall have been completed and the aggregate proceeds therefrom,
together with the Investment of the Founder, shall be sufficient to complete the
Acquisition; (ii) all conditions precedent to the consummation of the
Acquisition (other than any such condition requiring funding by the Founder of
its Initial Investments) shall have been fulfilled; and (iii) all regulatory
approvals and consents necessary for consummation of the Acquisition and for the
Company's operation as a unitary thrift holding company of the Bank shall have
been obtained or made and shall be in full force and effect and all waiting
periods required by any applicable law shall have expired. The obligation of the
Company to sell the Common Stock shall be subject to the Founder's delivery to
the Company of an executed Subscription Agreement.

                                  ARTICLE TWO
                   INVESTMENT BANKING SERVICES; UNDERWRITING
                   -----------------------------------------

         2.1 AGREEMENTS. Keefe shall provide placement agency services in
             ----------     
connection with the Private Placement and the Debt Offering pursuant to an
Agency Agreement substantially in the form of Exhibit 2.1 attached hereto.

         2.2 COMPENSATION. The Agency Agreement shall provide that Keefe shall
             ------------
be entitled to receive as compensation for services performed pursuant to the
Agency Agreement the sum of (i) an amount equal to 1.5% of the purchase price
paid to NB Holdings Corporation (or its designee) at the Closing; (ii) an amount
equal to 4% of the aggregate proceeds of the Private Placement, net of the
amount of the aggregate Founders' Initial Investments; and (iii) an amount equal
to 3.5% of the aggregate proceeds of the Debt Offering.

                                       2
<PAGE>
 
                                 ARTICLE THREE
                      RECOUPMENT OF PRE-CLOSING EXPENSES
                      ----------------------------------

         3.1 RECOUPMENT. Keefe shall be entitled to receive at Closing a number
             ----------
of additional shares of Common Stock equal to the sum of the Pre-closing
Expenses (as defined herein) distributed from the Escrow Account (as defined
herein) divided by the Offering Price, provided that the number of shares to
which the Founder shall be entitled pursuant to this Section shall not, when
aggregated with the number of shares to which the Founder shall be entitled
pursuant to Section 1.1, exceed 24.99% of the issued and outstanding shares of
Common Stock. If any reduction in the number of shares to which the Founder is
entitled under this Agreement is required in order to maintain the Founder's
stock ownership at a level below 25% of the issued and outstanding shares of
Common Stock, such reduction shall be made first out of shares to which the
Founder is entitled under Section 1.1 hereof.

         3.2 PRE-CLOSING EXPENSES. As used herein, the term Pre-closing Expenses
             --------------------
means expenses incurred in connection with the Acquisition, the Private
Placement and the Debt Offering, which shall include customary expenses of due
diligence, legal and accounting fees, consultancy fees and the cost of retaining
the services of a banking expert through the Closing. Upon execution of this
Agreement, the Founder will place in an escrow account with an agent mutually
agreeable to the Founder and the Company (the "Escrow Account") the sum of
$130,000 and agrees to deposit additional funds not to exceed $500,000 one month
from the date hereof or upon depletion of the Escrow Account, which ever is
sooner. All Pre-Closing Expenses shall be paid out of the Escrow Account. After
the Closing, one-half of any funds remaining in the Escrow Account shall be
reimbursed to the Founder.

                                 ARTICLE FOUR
                                 MISCELLANEOUS
                                 -------------

         4.1 BENEFIT. This Agreement shall inure to the benefit of and be
             -------
binding upon each of the Parties, and their respective successors. This
Agreement shall not be assignable by any Party without the prior written consent
of the other Party.

         4.2 GOVERNING LAW. This Agreement shall be governed by, and construed
             ------------- 
in accordance with the Laws of the State of Delaware without regard to any
conflict of Laws.

         4.3 COUNTERPARTS. This Agreement may be executed in counterparts, each
             ------------ 
of which shall be deemed to constitute an original. Each such counterpart shall
become effective when one counterpart has been signed by each Party thereto.

         4.4 HEADINGS. The headings of the various articles and sections of this
             --------
Agreement are for convenience of reference only and shall not be deemed a part
of this Agreement or considered in construing the provisions thereof.

                                       3
<PAGE>
 
         4.5   SEVERABILITY. Any term or provision of this Agreement that is
               ------------
prohibited or unenforceable in any jurisdiction shall, as to such jurisdiction,
be ineffective to the extent of such prohibition or unenforceability without
invalidating the remaining terms and provisions thereof or affecting the
validity or enforceability of such provision in any other jurisdiction, and if
any term or provision of this Agreement is held by any court of competent
jurisdiction to be void, voidable, invalid or unenforceable in any given
circumstance or situation, then all other terms and provisions, being severable,
shall remain in full force and effect in such circumstance or situation and the
term or provision shall remain valid and in effect in any other circumstances or
situation.

         4.6   CONSTRUCTION. Use of the masculine pronoun herein shall be deemed
               ------------ 
to refer to the feminine and neuter genders and the use of singular references
shall be deemed to include the plural and vice versa, as appropriate. No
inference in favor of or against any Party shall be drawn from the fact that
such Party or such Party's counsel has drafted any portion of this Agreement.

         4.7   EQUITABLE REMEDIES. The parties hereto agree that, in the event
               ------------------
of a breach of this Agreement by any Party, the Party not then in breach of this
Agreement may be without an adequate remedy at law owing to the unique nature of
the contemplated transactions. In recognition thereof, in addition to (and not
in lieu of) any remedies at law that may be available to a non-breaching Party,
the non-breaching Party shall be entitled to obtain equitable relief, including
the remedies of specific performance and injunction, in the event of a breach of
this Agreement by the Party in breach, and no attempt on the part of the non-
breaching Party to obtain such equitable relief shall be deemed to constitute an
election of remedies by the non-breaching Party that would preclude the non-
breaching Party from obtaining any remedies at law to which it would otherwise
be entitled.

         4.8   ATTORNEYS' FEES. If any Party hereto shall bring an action at law
               ---------------   
or in equity to enforce its rights under this Agreement, the prevailing Party in
such action shall be entitled to recover from the Party against whom enforcement
is sough its costs and expenses incurred in connection with such action
(including fees, disbursements and expenses of attorneys and costs of
investigation).

         4.9   NO WAIVER. No failure, delay or omission of or by any Party in
               ---------
exercising any right, power or remedy upon any breach or default of any other
Party shall impair any such rights, powers or remedies of the Party not in
breach or default, nor shall it be construed to be a waiver of any such right,
power or remedy, or an acquiescence in any similar breach or Default; nor shall
any waiver of any single breach or Default be deemed a waiver of any other
breach or default theretofore or thereafter occurring. Any waiver, permit,
consent or approval of any kind or character on the part of any Party of any
provisions of this Agreement must be in writing and be executed by the Parties
to this Agreement and shall be effective only to the extent specifically set
forth in such writing.

         4.10  REMEDIES CUMULATIVE. All remedies provided in this Agreement, by
               ------------------- 
law or otherwise, shall be cumulative and not alternative.

         4.11  AMENDMENT.  This Agreement may only be amended by a writing
               --------- 
signed by the Parties hereto.

                                       4
<PAGE>
 
         4.12  ENTIRE CONTRACT.   This Agreement and the documents and
               ----------------
instruments referred to herein constitute the entire contract between the
parties to this Agreement and supersede all other understandings with respect to
the subject matter of this Agreement.

KEEFE, BRUYETTE & WOODS, INC.               SFC ACQUISITION CORP.


By: /s/ Emmett J. Daly                      By: /s/ John C. H. Miller, Jr.
   ---------------------------                 ---------------------------------
Its: Senior Vice President                   Its: Acting Chairman 
------------------------------                   -------------------------------
                                       5