<PAGE>
 
                                                                    Exhibit 10.6


                              IFB HOLDINGS, INC.

                        RECOGNITION AND RETENTION PLAN


1.   Plan Purpose.

     The purpose of the Plan is to promote the long-term interests of the
Corporation and its stockholders by providing a means for attracting and
retaining directors, advisory directors and officers of the Corporation and its
Affiliates.

2.   Definitions.

     The following definitions are applicable to the Plan:

     "Award" - means the grant by the Committee of Restricted Stock, as provided
in the Plan.

     "Affiliate" - means any "parent corporation" or "subsidiary corporation" of
the Corporation, as such terms are defined in Section 424(e) and (f),
respectively, of the Code.

     "Bank" - means Investors Federal Bank, National Association and its
predecessors and  successors.

     "Board" or "Board of Directors" - means the board of directors of the
Corporation or its Affiliate, as applicable.

     "Change in Control" of the Bank or the Corporation means a change in
control of a nature that: (i) would be required to be reported in response to
Item 1(a) of the current report on Form 8-K, as in effect on the date hereof,
pursuant to Section 13 or 15(d) of the Securities Exchange Act of 1934 (the
"Exchange Act"); or (ii) results in a Change in Control of the Bank or the
Corporation within the meaning of the Bank Holding Company Act of 1956, as
amended ("BHCA"), and applicable rules and regulations promulgated thereunder,
as in effect at the time of the Change in Control; or (iii) without limitation
such a Change in Control shall be deemed to have occurred at such time as (a)
any "person" (as the term is used in Sections 13(d) and 14(d) of the Exchange
Act) is or becomes the "beneficial owner" (as defined in Rule 13d-3 under the
Exchange Act), directly or indirectly, of securities of the Corporation
representing 25% or more of the combined voting power of Corporation's
outstanding securities except for any securities purchased by the Bank's
employee stock ownership plan or trust; or (b) individuals who constitute the
Board on the date hereof (the "Incumbent Board") cease for any reason to
constitute at least a majority thereof, provided, however,  that this subsection
(b) shall not apply if the Incumbent Board is replaced by the appointment by a
Federal banking agency of a conservator or receiver for the Bank and, provided
further that any person becoming a director subsequent to the date hereof whose
election was approved by a vote of at least two-thirds of the directors
comprising the Incumbent Board, or whose nomination for election by the
Corporation's stockholders was approved by the same nominating committee serving
under an Incumbent Board, shall be, for purposes of this clause (b), considered
as though he were a member of the Incumbent Board; or (c) a plan of
reorganization, merger, consolidation, sale of all or substantially all the
assets of the Bank or the Corporation or similar transaction in which the Bank
or Corporation is not the surviving institution occurs; or (d) a proxy statement
soliciting proxies from stockholders of the Corporation, by someone other than
the current management of the Corporation, seeking stockholder approval of a
plan of reorganization, merger or consolidation of the Corporation or Bank or
similar transaction with one or more corporations as a result of which the
outstanding shares of the class of securities then subject to such plan or
transaction are to be exchanged for or converted into cash or property or
securities not issued by the Bank or Corporation shall be distributed and the
requisite number of proxies approving such plan of reorganization, merger or
consolidation of the Corporation or Bank are received and voted in favor of such
transactions; or (e) a tender offer is made for 25% or more of the outstanding
securities of the Bank or Corporation and the shareholders owning beneficially
or of record 25% or more of the outstanding securities of the Bank or
Corporation have tendered or 
<PAGE>
 
offered to sell their shares pursuant to such tender offer and such tendered
shares have been accepted by the tender offeror.

     "Code" - means the Internal Revenue Code of 1986, as amended.

     "Committee" - means the Committee referred to in Section 6 hereof.

     "Continuous Service" - means the absence of any interruption or termination
of service as a director, advisory director, officer or employee of the
Corporation or any Affiliate.  Service shall not be considered interrupted in
the case of sick leave, military leave or any other leave of absence approved by
the Corporation or any Affiliate or in the case of transfers between payroll
locations of the Corporation or between the Corporation, its subsidiaries or its
successor. With respect to any advisory director, continuous service shall mean
availability to perform such functions as may be required of such persons.

     "Corporation" - means IFB Holdings, Inc., a Delaware corporation.

     "Disability" - means the permanent and total inability by reason of mental
or physical infirmity, or both, of an employee to perform the work customarily
assigned to him.  Additionally, a medical doctor selected or approved by the
Board must advise the committee that it is either not possible to determine when
such Disability will terminate or that it appears probable that such Disability
will be permanent during the remainder of said Participant's lifetime.

     "ERISA" - means the Employee Retirement Income Security Act of 1974, as
amended.

     "Non-Employee Director" - means a director who (a) is not employed by the
Company or an Affiliate; (b) does not receive compensation directly or
indirectly as a consultant (or in any other capacity than as a director) greater
than $60,000; (c) does not have an interest in a transaction requiring
disclosure under Item 404(a) of Regulation S-K; or (d) is not engaged in a
business relationship for which disclosure would be required pursuant to Item
404(b) of Regulation S-K.

     "Normal Retirement" means retirement after reaching 65 years of age.
 
     "Outside Director" - means a director of the Corporation or an Affiliate
who is not an employee of the Corporation or an Affiliate.

     "Participant" - means any director, advisory director, officer or employee
of the Corporation or any Affiliate who is selected by the Committee to receive
an Award.

     "Plan" - means the Recognition and Retention Plan of the Corporation.

     "Restricted Period" - means the period of time selected by the Committee
for the purpose of determining when restrictions are in effect under Section 3
hereof with respect to Restricted Stock awarded under the Plan.

     "Restricted Stock" - means Shares which have been contingently awarded to a
Participant by the Committee subject to the restrictions referred to in Section
3 hereof, so long as such restrictions are in effect.

     "Shares" - means the common stock, par value $0.01 per share, of the
Corporation.


                                      B-2
<PAGE>
 
3.   Terms and Conditions of Restricted Stock.

      The Committee shall have full and complete authority, subject to the
limitations of the Plan, to grant awards of Restricted Stock and, in addition to
the terms and conditions contained in paragraphs (a) through (f) of this Section
3, to provide such other terms and conditions (which need not be identical among
Participants) in respect of such Awards, and the vesting thereof, as the
Committee shall determine.

     (a)     At the time of an award of Restricted Stock, the Committee shall
             establish for each Participant a Restricted Period during which or
             at the expiration of which, as the Committee shall determine and
             provide in the agreement referred to in paragraph (d) of this
             Section 3, the Shares awarded as Restricted Stock shall vest, and
             subject to any such other terms and conditions as the Committee
             shall provide, shares of Restricted Stock may not be sold,
             assigned, transferred, pledged, voted or otherwise encumbered by
             the Participant, except as hereinafter provided, during the
             Restricted Period. Except for such restrictions, and subject to
             paragraphs (c) and (e) of this Section 3 and Section 4 hereof, the
             Participant as owner of such shares shall have all the rights of a
             stockholder. The Committee shall have the authority, in its
             discretion, to accelerate the time at which any or all of the
             restrictions shall lapse with respect thereto, or to remove any or
             all of such restrictions.

     (b)     If a Participant ceases to maintain Continuous Service for any
             reason (other than death, Disability, Normal Retirement, or
             following a Change in Control), all Shares of Restricted Stock
             awarded to such Participant and which at the time of such
             termination of Continuous Service are subject to the restrictions
             imposed by paragraph (a) of this Section 3 shall upon such
             termination of Continuous Service be forfeited and returned to the
             Corporation. If a Participant ceases to maintain Continuous Service
             by reason of death, Disability, Normal Retirement, or following a
             Change in Control, Restricted Stock then still subject to
             restrictions imposed by paragraph (a) of this Section 3 will be
             free of those restrictions and shall be immediately vested.

     (c)     Each certificate in respect of Shares of Restricted Stock awarded
             under the Plan shall be registered in the name of the Participant
             or in the name of the Plan on behalf of the Participant and
             deposited by the Participant, together with a stock power endorsed
             in blank, with the Corporation and shall bear the following (or a
             similar) legend:

             "The transferability of this certificate and the shares of stock
             represented hereby are subject to the terms and conditions
             (including forfeiture) contained in the Recognition and Retention
             Plan of IFB Holdings, Inc. Copies of such Plan are on file in the
             office of the Secretary of IFB Holdings, Inc., 522 Washington
             Street, Chillicothe, Missouri 64601."

     (d)     At the time of any Award, the Participant shall enter into an
             agreement with the Corporation in a form specified by the
             Committee, agreeing to the terms and conditions of the Award and
             such other matters as the Committee, in its sole discretion, shall
             determine (the "Restricted Stock Agreement").

     (e)     After an Award has been granted but before such Award has been
             earned, the Participant shall receive any cash dividends paid with
             respect to such shares, or shall share in any pro-rata return of
             capital to all shareholders with respect to the Common Stock. Stock
             dividends declared by the Corporation and paid on Awards that have
             not yet been earned shall be subject to the same restrictions as
             the Restricted Stock and the certificate(s) or other instruments
             representing or evidencing such shares shall be legended in the
             manner provided in paragraph 3(c) and shall be delivered to the
             Escrow Agent for distribution to the Participant when the
             Restricted Stock upon which such dividends were paid are earned.
             Unless the Participant has made an election under Section 83(b) of
             the Code, cash dividends or other amounts so paid on shares that
             have not yet been earned by the Participant shall be treated as
             compensation income to the Participant when paid. If dividends are
             paid with respect


                                      B-3
<PAGE>
 
             to shares of Restricted Stock under the Plan that have been issued
             but not awarded, or that have been forfeited and returned to the
             Corporation or to a trust established to hold issued and unawarded
             or forfeited shares, the Committee can determine to award such
             dividends to any Participant or Participants under the Plan, to any
             other employee or director of the Corporation or the Bank, or can
             return such dividends to the Corporation.

     (f)     After an Award has been granted, the Participant as conditional
             owner of the Restricted Stock shall have the right to vote such
             shares.

     (g)     At the expiration of the restrictions imposed by paragraph (a) of
             this Section 3, the Corporation shall redeliver to the Participant
             (or where the relevant provision of paragraph (b) of this Section 3
             applies in the case of a deceased Participant, to his legal
             representative, beneficiary or heir) the certificate(s) and stock
             power deposited with it pursuant to paragraph (c) of this Section 3
             and the Shares represented by such certificate(s) shall be free of
             the restrictions referred to in paragraph (a) of this Section 3.

4.   Adjustments Upon Changes in Capitalization.

     In the event of any change in the outstanding Shares subsequent to the
effective date of the Plan by reason of any reorganization, recapitalization,
stock split, stock dividend, combination or exchange of shares, merger,
consolidation or any change in the corporate structure or Shares of the
Corporation, the maximum aggregate number and class of shares as to which Awards
may be granted under the Plan and the number and class of shares with respect to
which Awards theretofore have been granted under the Plan shall be appropriately
adjusted by the Committee, whose determination shall be conclusive.  Any shares
of stock or other securities received, as a result of any of the foregoing, by a
Participant with respect to Restricted Stock shall be subject to the same
restrictions and the certificate(s) or other instruments representing or
evidencing such shares or securities shall be legended and deposited with the
Corporation in the manner provided in Section 3 hereof.

5.   Assignments and Transfers.

     No Award nor any right or interest of a Participant under the Plan in any
instrument evidencing any Award under the Plan may be assigned, encumbered or
transferred except, in the event of the death of a Participant, by will or the
laws of descent and distribution or pursuant to a domestic relations order as
defined in the Code or Title I of ERISA or the rules thereunder.

6.   Administration.

     The Plan shall be administered by a Committee of the Board consisting of
either (i) at least two Non-Employee Directors of the Corporation, or (ii) the
entire Board of the Corporation.  Except as limited by the express provisions of
the Plan, the Committee shall have sole and complete authority and discretion,
to (i) select Participants and grant Awards; (ii) determine the number of shares
to be subject to types of Awards generally, as well as to individual Awards
granted under the Plan; (iii) determine the terms and conditions upon which
Awards shall be granted under the Plan; (iv) prescribe the form and terms of
instruments evidencing such grants; and (v) establish from time to time
regulations for the administration of the Plan, interpret the Plan, and make all
determinations deemed necessary or advisable for the administration of the Plan.

     A majority of the Committee shall constitute a quorum, and the acts of a
majority of the members present at any meeting at which a quorum is present, or
acts approved in writing by a majority of the Committee without a meeting, shall
be acts of the Committee.


                                      B-4
<PAGE>
 
7.   Shares Subject to Plan.

     Subject to adjustment by the operation of Section 4 hereof, the maximum
number of Shares with respect to which Awards may be made under the Plan is 4%
of the total Shares sold in the Bank's conversion to stock form.  The shares
with respect to which Awards may be made under the Plan may be either authorized
and unissued shares or issued shares reacquired and held as treasury shares.  An
Award shall not be considered to have been made under the Plan with respect to
Restricted Stock which is forfeited and new Awards may be granted under the Plan
with respect to the number of Shares as to which such forfeiture has occurred.

8.   Employee Rights Under the Plan.

     No director, officer or employee shall have a right to be selected as a
Participant nor, having been so selected, to be selected again as a Participant
and no director, officer, employee or other person shall have any claim or right
to be granted an Award under the Plan or under any other incentive or similar
plan of the Corporation or any Affiliate. Neither the Plan nor any action taken
thereunder shall be construed as giving any employee any right to be retained in
the employ of the Corporation, the Bank or any Affiliate.

9.   Withholding Tax.

     Upon the termination of the Restricted Period with respect to any shares of
Restricted Stock (or at any such earlier time that an election is made by the
Participant under Section 83(b) of the Code, or any successor provision thereto,
to include the value of such shares in taxable income), the Corporation may
withhold from any payment or distribution made under this Plan sufficient Shares
or may withhold or cause to be paid by Participant sufficient cash to cover any
applicable withholding and employment taxes.  The Corporation shall have the
right to deduct from all dividends paid with respect to shares of Restricted
Stock the amount of any taxes which the Corporation is required to withhold with
respect to such dividend payments.  No discretion or choice shall be conferred
upon any Participant with respect to the form, timing or method of any such tax
withholding.

10.  Amendment or Termination.

     The Board of Directors of the Corporation may amend, suspend or terminate
the Plan or any portion thereof at any time, provided, however, that no such
amendment, suspension or termination shall impair the rights of any Participant,
without his consent, in any Award theretofore made pursuant to the Plan.  Any
amendment or modification of the Plan or an outstanding Award under the Plan,
including but not limited to the acceleration of vesting of an outstanding Award
for reasons other than death, Disability, Normal Retirement, or termination
following a Change in Control, shall be approved by the Committee or the full
Board of the Corporation.

11.  Term of Plan.

     The Plan shall become effective on January 1, 1998 following its
ratification by stockholders of the Corporation.  It shall continue in effect
until the earlier of (i) ten years from the effective date unless sooner
terminated under Section 10 hereof, or (ii) the date on which all shares of
common stock available for award hereunder have vested in the recipients of such
Awards.



                                      B-5