<PAGE>   1

                            ARTICLES OF INCORPORATION
                                       OF
                           FIRST FEDERAL CAPITAL CORP.

                  The undersigned, being a natural person of the age of eighteen
(18) years or more, acting as incorporator of the corporation under the
Wisconsin Business Corporation Law, adopts the following Articles of
Incorporation for such corporation:

                                    ARTICLE I

                  The name of the corporation if First Federal Capital Corp.
(hereinafter referred to as the "Corporation").

                                   ARTICLE II

                  The period of existence of the Corporation is perpetual.

                                   ARTICLE III

                  The purpose or purposes for which the Corporation is organized
are to engage in any lawful activity within the purposes for which a corporation
may be organized, under the Wisconsin Business Corporation Law, Chapter 180 of
the Wisconsin Statutes.

                                   ARTICLE IV

                  Capital Stock. The total number of shares of capital stock
which the Corporation has authority to issue is 25,000,000, of which 5,000,000
shall be serial preferred stock, $.10 par value per share (hereinafter the
"Preferred Stock"), and of which 20,000,000 shall be common stock, par value
$.10 per share (hereinafter "Common Stock"). Except to the extent required by
governing law, rule or regulation, the shares of capital stock may be issued
from time to time by the Board of Directors without further approval of the
stockholders of the Corporation.

                  A description of the different classes and series (if any) of
the Corporation's capital stock and a statement of the designations, and the
relative rights, preferences, and limitations of the shares of each class of and
series (if any) of capital stock are as follows:

                  A. COMMON STOCK. Except as provided in this Article IV (or in
any resolution or resolutions adopted by the Board of Directors pursuant
hereto), the exclusive voting power shall be vested in the Common Stock, the
holders thereof being entitled to one vote for each share of such Common Stock
standing in the holder's name on the books of the Corporation. Subject to any
rights and preferences of any class of stock having preference over the Common
Stock, holders of Common Stock shall be entitled to such dividends as may be
declared by the Board of Directors out of funds lawfully available therefor.
Upon any liquidation, dissolution or winding up of the affairs of the
Corporation, whether voluntary or involuntary, holders of Common Stock shall be
entitled to receive pro rata the remaining assets



<PAGE>   2

of the Corporation after the holders of any class of stock having preference
over the Common Stock have been paid in full any sums to which they may be
entitled.

                  B. PREFERRED STOCK. Shares of Preferred Stock may be issued
from time to time in one or more series in any manner permitted by law pursuant
to a resolution or resolutions adopted by the Board of Directors under authority
hereby vested in it each series to be appropriately designated, prior to the
issuance of any shares thereof, by some distinguishing letter, number or title.
Before any shares of any such series are issued, the Board of Directors shall
have the full authority permitted by law to fix, and hereby is expressly
empowered to fix, resolution or resolutions full, limited, multiple, fractional
or no voting rights and such designations, preferences, qualifications,
privileges, limitations, restrictions, redemption rights, conversion rights and
other special or relative rights of the Preferred Stock or any series thereof
that may be desired.

                                    ARTICLE V

                  The name and address of the initial registered agent of the
corporation is Thomas W. Schini, 605 States Street, La Crosse, Wisconsin 54601
and the address of the corporation's registered office is the same.

                                   ARTICLE VI

                  The shareholders of the Corporation shall have no preemptive
right to acquire shares of any class or any securities convertible into shares
of any class of capital stock of the Corporation, now or hereafter authorized.

                                   ARTICLE VII

                  (a) The Board of Directors of the Corporation shall consist of
not less than seven (7) nor more than twenty (20) members, as determined in the
manner specified in the Corporation's Bylaws, as may be amended from time to
time. The names of the initial members of the Board of Directors are set forth
in Schedule A hereto and made a part hereof.

                  (b) The Board of Directors, other than those who may be
elected by the holders of any class or series of stock having preference over
the Common Stock as to dividends or upon liquidation, shall be divided into
three classes. At the first annual meeting of stockholders following the
effective date of these Articles of Incorporation, directors of the first class
shall be elected to hold office for a term expiring at the next succeeding
annual meeting, directors of the second class shall be elected to hold office
for a term expiring at the second succeeding annual meeting, and directors of
the third class shall be elected to hold office for a term expiring at the third
succeeding annual meeting, and, with respect to directors of each class, until
their respective successors are elected and qualified. At each subsequent annual
meeting of stockholders, directors elected to succeed those whose terms are
expiring shall be elected for a term of office to expire at the third succeeding
annual meeting of stockholders and until their respective successors are elected
and qualified.



                                      -2-
<PAGE>   3

                  (c) Subject to the rights of any class or series of stock
having preference over the Common Stock as to dividends or upon liquidation to
elect directors, any director may be removed from office only with cause by (i)
the affirmative vote of the holders of not less than two-thirds of the issued
and outstanding shares of capital stock of the Corporation entitled to vote
generally in an election of directors at a duly constituted meeting of
stockholders called expressly for the purpose or (ii) a majority of the total
number of directors. Except as may otherwise be provided by law, cause for
removal shall be construed to exist only if the director whose removal is
proposed (i) has been convicted of a felony by a court of competent jurisdiction
and such conviction is no longer subject to direct appeal or (ii) has been
adjudicated by a court of competent jurisdiction to be liable for gross
negligence or misconduct in the performance of such director's duty to the
Corporation and such adjudication is no longer subject to direct appeal. No
director may be removed as a director except for cause.

                                  ARTICLE VIII

                  LIABILITY OF DIRECTORS AND OFFICERS. A director or officer of
the Corporation shall not be personally liable for monetary damages for any
action taken, or any failure to take any action, as a director or officer except
to the extent that a director's or officer's liability for monetary damages may
not be limited by law. Any amendment to or repeal of this Article VIII shall not
adversely affect any right of a director or officer of the Corporation hereunder
or pursuant to the Business Corporation Law of the State of Wisconsin with
respect to any acts or omissions of such director or officer occurring prior to
such amendment or repeal.

                                   ARTICLE IX

                  BYLAWS. The Bylaws of the Corporation may be altered, amended
or repealed by the affirmative vote of a majority of the Board of Directors or
by the affirmative vote of the holders of not less than two-thirds of the issued
and outstanding shares of capital stock of the Corporation entitled to vote
generally in an election of directors.

                                    ARTICLE X


                  RESTRICTIONS ON OFFERS AND ACQUISITIONS.

                  (a) By virtue of this Article X(a), the Corporation elects to
be subject to (i) Sections 180.25(9) and 180.725 of the Business Corporation Law
of the State of Wisconsin, or any successor(s) thereto, as if the Corporation
was an "issuing public corporation" under such law and regardless whether it
then qualifies as such under such law and (ii) Section 180.726 of the Business
Corporation Law of the State of Wisconsin, or any successor thereto, as if the
Corporation was a "resident domestic corporation" under such law and regardless
whether it then qualifies as such under such law.

                  (b) Notwithstanding anything contained in the Corporation's
Articles of Incorporation or Bylaws to the contrary, for a period of five years
from the date of the closing of the conversion of First Federal Savings Bank La
Crosse-Madison from mutual to stock form, no



                                      -3-
<PAGE>   4

person shall directly or indirectly offer to acquire or acquire the beneficial
ownership of more than 10% of any class of an equity security of the
Corporation. This limitation shall not apply to (i) any offer with a view toward
public resale made exclusively to the Corporation by underwriters or a selling
group acting on its behalf, (ii)the purchase of shares by a tax-qualified
employee stock benefit plan established for the benefit of the employees of the
Corporation and its subsidiaries which is exempt from the approval requirements
under 12 C.F.R. ss.574.3(c)(1)(vi) or any successor thereto, and (iii) any offer
or acquisition approved in advance by the affirmative vote of two-thirds of the
entire Board of Directors.

                  (c) In the event shares are acquired in violation of Article
X(b), all shares beneficially owned by any person in excess of 10% shall be
considered "excess shares" and shall not be counted as shares entitled to vote
and shall not be voted by any person or counted as voting shares in connection
with any matters submitted to the stockholders for a vote.

                  (d) For purposes of Article X(b), the following definitions
apply:

                  (1) The term "person" includes an individual, a group acting
in concert, a corporation, a partnership, an association, a joint stock company,
a trust, an unincorporated organization or similar company, a syndicate or any
other group formed for the purpose of acquiring, holding or disposing of equity
securities of the Corporation.

                  (2) The term "offer" includes every offer to buy or otherwise
acquire, solicitation of any offer to sell, tender offer for, or request or
invitation for tenders of, a security or interest in a security for value.

                  (3) The term "acquire" includes every type of acquisition,
whether effected by purchase, exchange, operation of law or otherwise.

                  (4) The term "acting in concert" means (a) knowing
participation in a joint activity or conscious parallel action towards a common
goal whether or not pursuant to a express agreement, or (b) a combination or
pooling of voting or other interests in the securities of an issuer for a common
purpose pursuant to any contract, understanding, relationship, agreement or
other arrangement, whether written or otherwise.

                  (e) Paragraphs (b) through (d) of this Article X shall expires
and no longer be part of these Articles of Incorporation on the fifth
anniversary of the closing of the conversion of First Federal Servings Bank La
Crosse-Madison from mutual to stock form.

                                   ARTICLE XI

                  AMENDMENT. The Corporation reserves the right to amend, alter,
change or repeal any provision contained in these Articles of Incorporation, the
manner now or hereafter prescribed by the Business Corporation Law of the State
of Wisconsin, and all rights conferred upon stockholders herein are granted
subject to this reservation. Notwithstanding the foregoing, however, these
Articles of Incorporation may not be amended, altered, changed or repealed
without the vote of the holders of any class or series of Preferred Stock as may
be required by the



                                      -4-
<PAGE>   5

provisions establishing any such class or series and no amendment, alteration,
change or repeal of Articles VI, VII, VIII, IX, X or this Article XI may be made
unless it is approved by the holders of not less than two-thirds of the issued
and outstanding shares of capital stock of the Corporation entitled to vote
generally in election of directors.

                                   ARTICLE XII

                  The name and address of the incorporator is Thomas W. Schini,
605 State Street, La Crosse, Wisconsin 54601.

                  Dated at La Crosse, Wisconsin this 1st day of June, 1989.



                                           /s/ Thomas W. Schini
                                           -------------------------------------
                                           Thomas W. Schini, Incorporator



                                      -5-
<PAGE>   6



STATE OF WISCONSIN                  )
                                    )ss.
COUNTY OF MILWAUKEE                 )


                  Personally came before me this 1st day of June, 1989, the
aforenamed Thomas W. Schini, to me known to be the person who executed the
foregoing instrument and acknowledged the same.



                  [SEAL]


                                            /s/ Bradford R. Price
                                            ------------------------------------
                                            Notary Public, State of Wisconsin
                                            My commission is permanent.


                  This instrument was drafted by W. Charles Jackson, Esq. and is
returnable to:

                            W. Charles Jackson, Esq.
                            Michael Best & Friedrich
                            250 East Wisconsin Avenue
                           Milwaukee, Wisconsin 53202


                  This instrument is to be recorded in La Crosse County,
Wisconsin.




                                      -6-
<PAGE>   7



                     SCHEDULE A TO ARTICLE OF INCORPORATION
                                       OF
                           FIRST FEDERAL CAPITAL CORP.


      The initial members of the Board of Directors of the Corporation shall be:


      James O. Ash

      Daniel A. Chapman

      Henry C. Funk

      John F. Leinfelder

      Richard T. Lommen

      Patrick J. Luby

      David C. Mebane

      Dale A. Nordeen

      Phillip J. Quillin

      Robert B. Rennebohm

      Don P. Rundle

      Thomas W. Schini


<PAGE>   8



                            ARTICLES OF AMENDMENT TO
                        THE ARTICLES OF INCORPORATION OF
                           FIRST FEDERAL CAPITAL CORP


         1. The name of the corporation is "First Federal Capital Corp".

         2. On January 24, 1995, pursuant to Section 180.0602 of the Wisconsin
Business Corporation Law, and the authority conferred upon the Board of
Directors by the Articles of Incorporation of the Corporation, the Board of
Directors of the Corporation duly adopted the following resolutions creating a
series of 200,000 shares of Preferred Stock designated as Series A Junior
Participating Preferred Stock:

                  RESOLVED, that it is hereby declared to be in the best
         interests of the Corporation that the Articles of Incorporation of the
         Corporation be amended to create a new series of Preferred Stock to
         consist of 200,000 shares and to be designated as Series A Junior
         Participating Preferred Stock, and to determine the preferences,
         limitations and relative rights of the Series A Junior Participating
         Preferred Stock by adding a new subsection C to Article IV of such
         Articles of Incorporation to read in the form attached hereto as
         Appendix I; and

                  RESOLVED, that the amendment to the Articles of Incorporation
         of the Corporation attached hereto as Appendix I is hereby adopted and
         that the appropriate officers of the Corporation are authorized and
         directed to prepare and to file with the Secretary of State of The
         State of Wisconsin Articles of Amendment to give effect thereto.

         3. That Appendix I hereto constitutes the amendment referred to in the
foregoing resolutions. None of the shares of Series A Junior Participating
Preferred Stock authorized thereby has been issued as of the date hereof.

         4. That such amendment to the Articles of Incorporation of the
Corporation was adopted by the Board of Directors of the Corporation on January
24, 1995 in accordance with Section 180.1002 of the Wisconsin Business
Corporation Law. Shareholder action was not required.

         Dated:  January 26, 1995

                                       FIRST FEDERAL CAPITAL CORP

                                       By: /s/ Thomas W. Schini
                                           ------------------------------------
                                           Name:  Thomas W. Schini
                                           Title:  President and Chief Executive
                                                   Officer

         This document was drafted by Gerald L. Hawkins, Esq. of Elias, Matz,
Tiernan & Herrick L.L.P., 734 15th Street, N.W., Washington D.C. 20005
(202)347-0300.


<PAGE>   9


                                                                      APPENDIX I

         C. SERIES A JUNIOR PARTICIPATING PREFERRED STOCK.

         The Corporation has designated a series of Preferred Stock, without par
value, as Series A Junior Participating Preferred Stock. The preferences,
limitations and relative rights of the Series A Junior Participating Preferred
Stock shall be as follows:

         Section 1. Amount. The number of shares constituting the Series A
Junior Participating Preferred Stock shall be 200,000. Such number of shares may
be increased or decreased by resolution of the Board of Directors; provided,
that no decrease shall reduce the number of shares of Series A Junior
Participating Preferred Stock to a number less than that of the shares then
outstanding plus the number of shares issuable upon exercise of outstanding
rights, options or warrants or upon conversion of outstanding securities issued
by the Corporation.

         Section 2. Dividends and Distributions.

         (a) Subject to the prior and superior rights of the holders of any
shares of any Preferred Stock ranking prior and superior to the shares of Series
A Junior Participating Preferred Stock with respect to dividends, the holders of
shares of Series A Junior Participating Preferred Stock shall be entitled to
receive, when, as and if declared by the Board of Directors out of funds legally
available for the purpose, quarterly dividends payable in cash on the first day
of January, April, July and October in each year (each such date being referred
to herein as a "Quarterly Dividend Payment Date"), commencing on the first
Quarterly Dividend Payment Date after the first issuance of a share or fraction
of a share of Series A Junior Participating Preferred Stock, in an amount per
share (rounded to the nearest cent) equal to the greater of (a) $1.00 or (b)
subject to the provision for adjustment hereinafter set forth, 100 times the
aggregate per share amount of all cash dividends, and 100 times the aggregate
per share amount (payable in kind) on all non-cash dividends or other
distributions other than a dividend payable in shares of Common Stock or a
subdivision of the outstanding shares of Common Stock (by reclassification or
otherwise), declared on the Common Stock, par value $.01 per share, of the
Corporation (the "Common Stock") since the immediately preceding Quarterly
Dividend Payment Date, or, with respect to the first Quarterly Dividend Payment
Date, since the first issuance of any share or fraction of a share of Series A
Junior Participating Preferred Stock. In the event the Corporation shall at any
time after January 24, 1995 (the "Rights Declaration Date") (i) declare any
dividend on Common Stock payable in shares of Common Stock, (ii) subdivide the
outstanding Common Stock, or (iii) combine the outstanding Common Stock into a
smaller number of shares, then in each such case the amount to which holders of
shares of Series A Junior Participating Preferred Stock were entitled
immediately prior to such event under clause (b) of the preceding sentence shall
be adjusted by multiplying such amount by a fraction the numerator of which is
the number of shares of Common Stock outstanding immediately after such event
and the denominator of which is the number of shares of Common Stock that were
outstanding immediately prior to such event.



                                      I-1
<PAGE>   10

         (b) The Corporation shall declare a dividend or distribution on the
Series A Junior Participating Preferred Stock as provided in paragraph (a) above
immediately after it declares a dividend or distribution on the Common Stock
(other than a dividend payable in shares of Common Stock); provided that, in the
event no dividend or distribution shall have been declared on the Common Stock
during the period between any Quarterly Dividend Payment Date and the next
subsequent Quarterly Dividend Payment Date, a dividend of $1.00 per share on the
Series A Junior Participating Preferred Stock shall nevertheless be payable on
such subsequent Quarterly Dividend Payment Date.

         (c) Dividends shall begin to accrue and be cumulative on outstanding
shares of Series A Junior Participating Preferred Stock from the Quarterly
Dividend Payment Date next preceding the date of issue of such shares of Series
A Junior Participating Preferred Stock, unless the date of issue of such shares
is prior to the record date for the first Quarterly Dividend Payment Date, in
which case dividends on such shares shall begin to accrue from the date of issue
of such shares, or unless the date of issue is a Quarterly Dividend Payment Date
or is a date after the record date for the determination of holders of shares of
Series A Junior Participating Preferred Stock entitled to receive a quarterly
dividend and before such Quarterly Dividend Payment Date, in either of which
events such dividends shall begin to accrue and be cumulative from such
Quarterly Dividend Payment Date. Accrued but unpaid dividends shall not bear
interest. Dividends paid on the shares of Series A Junior Participating
Preferred Stock in an amount less than the total amount of such dividends at the
time accrued and payable on such shares shall be allocated pro rata on a
share-by-share basis among all such shares at the time outstanding. The Board of
Directors may fix a record date for the determination of holders of shares of
Series A Junior Participating Preferred Stock entitled to receive a payment of a
dividend or distribution declared thereon, which record date shall be no more
than 30 days prior to the date fixed for payment thereof.

         Section 3. Voting Rights. The holders of shares of Series A Junior
Participating Preferred Stock shall have the following voting rights:

         (a) Subject to the provision for adjustment hereinafter set forth, each
share of Series A Junior Participating Preferred Stock shall entitle the holder
thereof to 100 votes on all matters submitted to a vote of the stockholders of
the Corporation. In the event the Corporation shall at any time after the Rights
Declaration Date (i) declare any dividend on Common Stock payable in shares of
Common Stock, (ii) subdivide the outstanding Common Stock, or (iii) combine the
outstanding Common Stock into a smaller number of shares, then in each such case
the number of votes per share to which holders of shares of Series A Junior
Participating Preferred Stock were entitled immediately prior to such event
shall be adjusted by multiplying such number by a fraction the numerator of
which is the number of shares of Common Stock outstanding immediately after such
event and the denominator of which is the number of shares of Common Stock that
were outstanding immediately prior to such event.

         (b) Except as otherwise provided herein or by law, the holders of
shares of Series A Junior Participating Preferred Stock and the holders of
shares of Common Stock shall vote together as one class on all matters submitted
to a vote of stockholders of the Corporation.



                                      I-2
<PAGE>   11

         (c)(i) If at any time dividends on any Series A Junior Participating
Preferred Stock shall be in arrears in an amount equal to six (6) quarterly
dividends thereon, the occurrence of such contingency shall mark the beginning
on period (herein called a "default period") which shall extend until such time
when all accrued and unpaid dividends for all previous quarterly dividend
periods and for the current quarterly dividend period on all shares of Series A
Junior Participating Preferred Stock then outstanding shall have been declared
and paid or set apart for payment. During each default period, all holders of
Preferred Stock (including holders of the Series A Junior Participating
Preferred Stock) with dividends in arrears in an amount equal to six (6)
quarterly dividends thereon, voting as a class, irrespective of series, shall
have the right to elect two (2) Directors.

                  (ii) During any default period, such voting right of holders
         of Series A Junior Participating Preferred Stock may be exercised
         initially at a special meeting called pursuant to subparagraph (iii) of
         this Section 3(c) or at any annual meeting of stockholders, and
         thereafter at annual meetings of stockholders, provided that neither
         such voting right nor the right of the holders of any other series of
         Preferred Stock, if any, to increase, in certain cases, the authorized
         number of Directors shall be exercised unless the holders of one-third
         (1/3) in number of shares of Preferred Stock outstanding shall be
         present in person or by proxy. The absence of a quorum of the holders
         of Common Stock shall not affect the exercise by the holders of
         Preferred Sock of such voting right. At any meeting at which the holder
         s of Preferred Stock shall exercise such voting right initially during
         an existing default period, they shall have the right, voting as a
         class, to elect Directors to fill such vacancies, if any, in the Board
         of Directors as may then exist up to two (2) Directors or, if such
         right is exercised at an annual meeting, to elect two (2) Directors. If
         the number which may be so elected at any special meeting does not
         amount to the required number, the holders of the Preferred Stock shall
         have the right (subject to the Articles of Incorporation) to make such
         increase in the number of Directors as shall be necessary to permit the
         election by them of the required number. After the holders of the
         Preferred Stock shall have exercised their right to elect Directors in
         any default period and during the continuance of such period, the
         number of Directors shall not be increased or decreased except by vote
         of the holders of Preferred Stock as herein provided or pursuant to the
         rights of any equity securities ranking senior to or pari passu with
         the Series A Junior Participating Preferred Stock.

                  (iii) Unless the holders of Preferred Stock shall, during an
         existing default period, have previously exercised their right to elect
         Directors, the Board of Directors may order, or any stockholder or
         stockholders owning in the aggregate not less than ten percent (10%) of
         the total number of shares of Preferred Stock outstanding irrespective
         of series, may request, the calling of a special meeting of the holders
         of Preferred Stock, which meeting shall thereupon be called by the
         President, a Vice-President or the Secretary of the Corporation. Notice
         of such meeting and of any annual meeting at which holders of Preferred
         Stock are entitled to vote pursuant to this paragraph (iii) shall be
         given to each holder of record of Preferred Stock by mailing a copy of
         such notice to him at his last address as the same appears on the books
         of the Corporation. Such meeting shall be called for a time not earlier
         than ten (10) days and not later than sixty (60) days after such order
         or request or in default of the calling of such meeting within sixty
         (60)



                                      I-3
<PAGE>   12

         days after such order or request, such meeting may be called on similar
         notice by any stockholder or stockholders owning in the aggregate not
         less than ten percent (10%) of the total number of shares of Preferred
         Stock outstanding. Notwithstanding the provisions of this paragraph
         (c)(iii), no such special meeting shall be called during the period
         within sixty (60) days immediately preceding the date fixed for the
         next annual meeting of the stockholders.

                  (iv) In any default period, the holders of Common Stock, and
         other classes of stock of the Corporation, if applicable, shall
         continue to be entitled to elect the Whole Board of Directors until the
         holders of Preferred Stock shall have exercised their right to elect
         two (2) Directors voting as a class, after the exercise of which right
         (x) the Directors so elected by the holders of Preferred Stock shall
         continue in office until their successors shall have been elected by
         such holders or until the expiration of the default period, and (y) any
         vacancy in the Board of Directors may (except as provided in paragraph
         (c)(ii) of this Section 3) be filled by vote of a majority of the
         remaining Directors theretofore elected by the holders of the class of
         stock which elected the Director whose office shall have become vacant.
         References in this paragraph (c) of this Section 3 to Directors elected
         by the holders of a particular class of stock shall include Directors
         elected by such Directors to fill vacancies as provided in clause (y)
         of the foregoing sentence.

                  (v) Immediately upon the expiration of a default period, (x)
         the right of the holders of Preferred Stock as a class to elect
         Directors shall cease, (y) the term of any Directors elected by the
         holders of Preferred Stock as a class shall terminate and (z) the
         number of Directors shall be such number as may be provided for in the
         Articles of Incorporation or by-laws irrespective of any increase made
         pursuant to the provisions of paragraph (ii) of this Section 3 (such
         number being subject, however, to change thereafter in any manner
         provided by law or in the Articles of Incorporation or by-laws). Any
         vacancies in the Board of Directors effected by the provisions of
         clauses (y) and (z) in the preceding sentence may be filled by a
         majority of the remaining Directors.

         (d) Except as set forth herein, holders of Series A Junior
Participating Preferred Stock shall have no special voting rights and their
consent shall not be required (except to the extent they are entitled to vote
with holders of Common Stock as set forth herein) for taking any corporate
action.

         Section 4. Certain Restrictions.

         (a) Whenever quarterly dividends or other dividends or distributions
payable on the Series A Junior Participating Preferred Stock as provided in
Section 2 are in arrears, thereafter, and until all accrued and unpaid dividends
and distributions, whether or not declared, on shares of Series A Junior
Participating Preferred Stock outstanding shall have been paid in full, the
Corporation shall not

                  (i) declare or pay dividends on, make any other distributions
         on, or redeem or purchase or otherwise acquire for consideration any
         shares of stock ranking junior (either



                                      I-4
<PAGE>   13

         as to dividends or upon liquidation, dissolution or winding up) to the
         Series A Junior Participating Preferred Stock;

                  (ii) declare or pay dividends on or make any other
         distributions on any shares of stock ranking on a parity (either as to
         dividends or upon liquidation, dissolution or winding up) with the
         Series A Junior participating Preferred Stock, except dividends paid
         ratably on the Series A Junior Participating Preferred Stock and all
         such parity stock on which dividends are payable or in arrears in
         proportion to the total amounts to which the holders of all such shares
         are then entitled;

                  (iii) redeem or purchase or otherwise acquire for
         consideration shares of any stock ranking on a parity (either as to
         dividends or upon liquidation, dissolution or winding up) with the
         Series A Junior participating Preferred Stock, provided that the
         Corporation may at any time redeem, purchase or otherwise acquire
         shares of any such parity stock in exchange for shares of any stock of
         the Corporation ranking junior (either as to dividends or upon
         dissolution, liquidation or winding up) to the Series A Junior
         Participating Preferred Stock; or

                  (iv) purchase or otherwise acquire for consideration any
         shares of Series A Junior Participating Preferred Stock, or any shares
         of stock ranking on a parity with the Series A Junior Participating
         Preferred Stock, except in accordance with a purchase offer made in
         writing or by publication (as determined by the Board of Directors) to
         all holders of such shares upon such terms as the Board of Directors,
         after consideration of the respective annual dividend rates and other
         relative rights and preferences of the respective series and classes,
         shall determine in good faith will result in fair and equitable
         treatment among the respective series or classes.

         (b) The Corporation shall not permit any subsidiary of the Corporation
to purchase or otherwise acquire for consideration any shares of stock of the
Corporation unless the Corporation could, under paragraph (a) of this Section 4,
purchase or otherwise acquire such shares at such time and in such manner.

         Section 5. Reacquired Shares. Any shares of Series A Junior
Participating Preferred Stock purchased or otherwise acquired by the Corporation
in any manner whatsoever shall be retired promptly after the acquisition
thereof. All such shares shall upon their retirement become authorized but
unissued shares of Preferred Stock and may be reissued as part of a new series
of Preferred Stock to be created by resolution or resolutions of the Board of
Directors, subject to the conditions and restrictions on issuance set forth
herein.

         Section 6. Liquidation, Dissolution or Winding Up. Upon any liquidation
(voluntary or otherwise), dissolution or winding up of the Corporation, no
distribution shall be made (1) to the holders of shares of stock ranking junior
(either as to dividends or upon liquidation, dissolution or winding up) to the
Series A Junior Participating Preferred Stock unless, prior thereto, the holders
of shares of Series A Junior Participating Preferred Stock shall have received
$100 per share, plus an amount equal to accrued and unpaid dividends and
distributions thereon, whether or not declared, to the date of such payment;
provided that the holders of shares of



                                      I-5
<PAGE>   14

Series A Junior Participating Preferred Stock shall be entitled to receive an
aggregate amount per share, subject to the provision for adjustment hereinafter
set forth, equal to 100 times the aggregate amount to be distributed per share
to holders of shares of Common Stock, or (2) to the holders of shares of stock
ranking on a parity (either as to dividends or upon liquidation dissolution or
winding up) with the Series A Junior Participating Preferred Stock, except
distributions made ratably on the Series A Junior Participating Preferred Stock
and all such parity stock in proportion to the total amounts to which the
holders of all such shares are entitled upon such liquidation, dissolution or
winding up. In the event the Corporation shall at any time declare or pay any
dividend on the Common Stock payable in shares of Common Stock, or effect a
subdivision or combination or consolidation of the outstanding shares of Common
Stock (by reclassification or otherwise than by payment of a dividend in shares
of Common Stock) into a greater or lesser number of shares of Common Stock, then
in each such case the aggregate amount to which holders of shares of Series A
Junior Participating Preferred Stock were entitled immediately prior to such
event under the proviso in clause (1) of the preceding sentence shall be
adjusted by multiplying such amount by a fraction, the numerator of which is the
number of shares of Common Stock outstanding immediately after such event and
the denominator of which is the number of shares of Common Stock that were
outstanding immediately prior to such event.

         Section 7. Consolidation, Merger, etc. In case the Corporation shall
enter into any consolidation, merger, combination or other transaction in which
the shares of Common Stock are exchanged for or changed into other stock or
securities, cash and/or any other property, then in any such case the shares of
Series A Junior Participating Preferred Stock shall at the same time be
similarly exchanged or changed in an amount per share (subject to the provision
for adjustment hereinafter set forth) equal to 100 times the aggregate amount of
stock, securities, cash and/or any other property (payable in kind), as the case
may be, into which or for which each share of Common Stock is changed or
exchanged. In the event the Corporation shall at any time after the Rights
Declaration Date (i) declare any dividend on Common Stock payable in shares of
Common Stock, (ii) subdivide the outstanding Common Stock, or (iii) combine the
outstanding Common Stock into a smaller number of shares, then in each such case
the amount set forth in the preceding sentence with respect to the exchange or
change of shares of Series A Junior Participating Preferred Stock shall be
adjusted by multiplying such amount by a fraction the numerator of which is the
number of shares of Common Stock outstanding immediately after such event and
the denominator of which is the number of shares of Common Stock that were
outstanding immediately prior to such event.

         Section 8. No Redemption. The shares of Series A Junior Participating
Preferred Stock shall not be redeemable.

         Section 9. Ranking. The Series A Junior Participating Preferred Stock
shall rank junior to all other series of the Corporation's Preferred Stock as to
the payment of dividends and the distribution of assets, unless the terms of any
such series provide otherwise.

         Section 10. Amendment. The Articles of Incorporation of the Corporation
shall not be further amended in any manner which would materially alter or
change the powers, preferences or special rights of the Series A Junior
Participating Preferred Stock so as to affect them



                                      I-6
<PAGE>   15

adversely without the affirmative vote of the holders of a majority or more of
the outstanding shares of Series A Junior Participating Preferred Stock, voting
separately as a class.

         Section 11. Fractional Shares. Series A Junior Participating Preferred
Stock may be issued in fractions of one one-hundredth of a share (and integral
multiples thereof) which shall entitle the holder, in proportion to such
holder's fractional shares, to exercise voting rights, receive dividends,
participate in distributions and to have the benefit of all other rights of
holders of Series A Junior Participating Preferred Stock.





                                      I-7
<PAGE>   16
                              ARTICLES OF AMENDMENT
                                     TO THE
                            ARTICLES OF INCORPORATION
                                       OF
                           FIRST FEDERAL CAPITAL CORP.

         First Federal Capital Corp., a corporation organized and existing under
Chapter 180 of the Wisconsin Business Corporation Law (the "WBCL") (the
"Corporation"), does hereby certify that the Board of Directors of the
corporation (the "Board") duly adopted the necessary resolutions to increase the
number of authorized shares of Common Stock to 100,000,000 shares, $.10 par
value per share, pursuant to Section 180.1003 of the Wisconsin Statutes and the
authority conferred upon the Board by the Articles of Incorporation and, in
order to effectuate the foregoing, further approved the amendment of the first
paragraph of Article IV of the existing Articles of Incorporation of the
Corporation, to read as follows:

                  ARTICLE IV. CAPITAL STOCK. The total number of shares of
         capital stock which the Corporation has authority to issue is
         105,000,000, of which 5,000,000 shall be serial preferred stock, $.10
         par value per share (hereinafter the "Preferred Stock"), and of which
         100,000,000 shall be common stock, par value $.10 per share
         (hereinafter "Common Stock"). Except to the extent required by
         governing law, rule or regulation, the shares of capital stock may be
         issued from time to time by the Board of Directors without further
         approval of the stockholders of the Corporation.

         This Amendment of the Articles of Incorporation of the Corporation was
adopted by the Board of Directors of the Corporation at a meeting of the Board
of Directors held on January 26, 1999 and shareholder approval of this Amendment
was received on April 21, 1999 in accordance with Section 180.1003 of the
Wisconsin Statutes.

         Dated as of the 24th day of May, 1999.



                                        By:/s/ Bradford R. Price
                                           -------------------------------------
                                           Bradford R. Price, Executive Vice
                                           President and Secretary


               This document was drafted by and is returnable to:

                              Teresa M. Levy, Esq.
                           Michael Best &Friedrich LLP
                            100 East Wisconsin Avenue
                                   Suite 3300
                         Milwaukee, Wisconsin 53202-4108
                                 (414) 271-6560


<PAGE>   17



                              ARTICLES OF AMENDMENT
                                     TO THE
                            ARTICLES OF INCORPORATION
                                       OF
                           FIRST FEDERAL CAPITAL CORP.

         First Federal Capital Corp, a corporation organized and existing under
Chapter 180 of the Wisconsin Business Corporation Law (the "WBCL") (the
"Corporation"), does hereby certify that the Board of Directors of the
Corporation (the "Board") duly adopted the necessary resolutions to increase the
number of designated shares of Series A Junior Participating Preferred Stock, no
par value per share, pursuant to Section 180.1002 of the Wisconsin Statues and
the authority conferred upon the Board by the Articles of Incorporation; and to
effectuate the foregoing, further approved the amendment of the first sentence
of Section 1 of Subsection C of Article IV of the existing Articles of
Incorporation of the Corporation to read as follows:

         "SECTION 1. The number of shares constituting the Series A Junior
Participating Preferred Stock shall be 1,000,000."

         This Amendment to the Articles of Incorporation of the Corporation was
adopted by the Board of Directors of the Corporation by Unanimous Consent.

         Dated as of the 25th day of May, 1999.



                                        By:/s/ Bradford R. Price
                                           -------------------------------------
                                           Bradford R. Price, Executive Vice
                                           President and Secretary


               This document was drafted by and is returnable to:

                              Teresa M. Levy, Esq.
                           Michael Best &Friedrich LLP
                            100 East Wisconsin Avenue
                                   Suite 3300
                         Milwaukee, Wisconsin 53202-4108
                                 (414) 271-6560