<PAGE>   1
                                                                     EXHIBIT 4.7




================================================================================


                    PREFERRED SECURITIES GUARANTEE AGREEMENT


                                 BY AND BETWEEN


                                 BAYLAKE CORP.


                                      AND

                            WILMINGTON TRUST COMPANY


                        Dated as of _____________  ____, 2001



================================================================================



<PAGE>   2



                                TABLE OF CONTENTS

<TABLE>
<CAPTION>

                                                                                             Page

<S>      <C>                                                                                 <C>
ARTICLE I. DEFINITIONS AND INTERPRETATION.......................................................1
           Section 1.1. Definitions and Interpretation..........................................1

ARTICLE II. TRUST INDENTURE ACT.................................................................5
           Section 2.1. Trust Indenture Act; Application........................................5
           Section 2.2. The List of Holders of the Securities...................................5
           Section 2.3. Reports by the Preferred Guarantee Trustee..............................5
           Section 2.4. Periodic Reports to the Preferred Guarantee Trustee.....................5
           Section 2.5. Evidence of Compliance with Conditions Precedent........................6
           Section 2.6. Events of Default; Waiver...............................................6
           Section 2.7. Event of Default; Notice................................................6
           Section 2.8. Conflicting Interests...................................................6

ARTICLE III. POWERS, DUTIES AND RIGHTS OF THE PREFERRED GUARANTEE TRUSTEE.......................7
           Section 3.1. Powers and Duties of the Preferred Guarantee Trustee....................7
           Section 3.2. Certain Rights of the Preferred Guarantee Trustee.......................8
           Section 3.3. Not Responsible for Recitals or Issuance of Guarantee..................10

ARTICLE IV. THE PREFERRED GUARANTEE TRUSTEE....................................................10
           Section 4.1. The Preferred Guarantee Trustee; Eligibility...........................10
           Section 4.2. Appointment, Removal and Resignation of the Preferred
                        Guarantee Trustee......................................................11

ARTICLE V. GUARANTEE...........................................................................12
           Section 5.1. Guarantee..............................................................12
           Section 5.2. Waiver of Notice and Demand............................................12
           Section 5.3. Obligations not Affected...............................................12
           Section 5.4. Rights of the Holders..................................................13
           Section 5.5. Guarantee of Payment...................................................13
           Section 5.6. Subrogation............................................................13
           Section 5.7. Independent Obligations................................................14

ARTICLE VI. LIMITATION OF TRANSACTIONS; SUBORDINATION..........................................14
           Section 6.1. Limitation on Transactions.............................................14
           Section 6.2  Ranking................................................................14

ARTICLE VII. TERMINATION.......................................................................14
           Section 7.1. Termination............................................................14

ARTICLE VIII. INDEMNIFICATION..................................................................15
           Section 8.1. Exculpation............................................................15
</TABLE>



                                       i
<PAGE>   3


<TABLE>

<S>      <C>                                                                               <C>
           Section 8.2. Indemnification.......................................................15

ARTICLE IX. MISCELLANEOUS.....................................................................15
           Section 9.1. Successors and Assigns................................................15
           Section 9.2. Amendments............................................................15
           Section 9.3. Notices...............................................................15
           Section 9.4. Benefit...............................................................16
           Section 9.5. Governing Law.........................................................16
</TABLE>




                                       ii
<PAGE>   4



                              CROSS-REFERENCE TABLE

Section of                                                          Section of
Trust Indenture Act                                                 Guarantee
of 1939, as amended                                                 Agreement
-------------------                                                 ----------

310(a)....................................................................4.1(a)
310(b)...............................................................4.1(c), 2.8
310(c)............................................................Not Applicable
311(a)....................................................................2.2(b)
311(b)....................................................................2.2(b)
311(c)............................................................Not Applicable
312(a)................................................................... 2.2(a)
312(b)................................................................... 2.2(b)
313......................................................................... 2.3
314(a).......................................................................2.4
314(b)............................................................Not Applicable
314(c).......................................................................2.5
314(d)............................................................Not Applicable
314(e).............................................................1.1, 2.5, 3.2
314(f)..................................................................2.1, 3.2
315(a)....................................................................3.1(d)
315(b).......................................................................2.7
315(c).......................................................................3.1
315(d)....................................................................3.1(d)
316(a).............................................................1.1, 2.6, 5.4
316(b).......................................................................5.3
317(a).......................................................................3.1
317(b)............................................................Not Applicable
318(a)....................................................................2.1(a)
318(b).......................................................................2.1
318(c)....................................................................2.1(b)

Note: This Cross-Reference Table does not constitute part of this Agreement and
shall not affect the interpretation of any of its terms or provisions.


                                      iii
<PAGE>   5

                    PREFERRED SECURITIES GUARANTEE AGREEMENT



         THIS PREFERRED SECURITIES GUARANTEE AGREEMENT (this "Preferred
Securities Guarantee"), dated as of _________ __, 2001, is executed and
delivered by BAYLAKE CORP., a Wisconsin corporation (the "Guarantor"), and
WILMINGTON TRUST COMPANY, a trust company organized and existing under the laws
of Delaware, as trustee (the "Preferred Guarantee Trustee"), for the benefit of
the Holders (as defined herein) from time to time of the Preferred Securities
(as defined herein) of BAYLAKE CAPITAL TRUST I, a Delaware statutory business
trust (the "Trust").


                                    RECITALS


         WHEREAS, pursuant to an Amended and Restated Trust Agreement (the
"Trust Agreement"), dated as of ___________  __, 2001, among the trustees of the
Trust named therein, the Guarantor, as depositor, and the holders from time to
time of undivided beneficial interests in the assets of the Trust, the Trust is
issuing on the date hereof up to 1,380,000 preferred securities, having an
aggregate liquidation amount of $13,800,000, designated the __% Cumulative Trust
Preferred Securities (the "Preferred Securities");


         WHEREAS, as incentive for the Holders to purchase the Preferred
Securities, the Guarantor desires irrevocably and unconditionally to agree, to
the extent set forth in this Preferred Securities Guarantee, to pay to the
Holders of the Preferred Securities the Guarantee Payments (as defined herein)
and to make certain other payments on the terms and conditions set forth herein.

         NOW, THEREFORE, in consideration of the purchase by each Holder of
Preferred Securities, which purchase the Guarantor hereby agrees shall benefit
the Guarantor, the Guarantor executes and delivers this Preferred Securities
Guarantee for the benefit of the Holders.

                                   ARTICLE I.
                         DEFINITIONS AND INTERPRETATION

         SECTION 1.1. DEFINITIONS AND INTERPRETATION. In this Preferred
Securities Guarantee, unless the context otherwise requires:

         (a) capitalized terms used in this Preferred Securities Guarantee but
not defined in the preamble above have the respective meanings assigned to them
in this Section 1.1;

         (b) terms defined in the Trust Agreement as at the date of execution of
this Preferred Securities Guarantee have the same meaning when used in this
Preferred Securities Guarantee, unless otherwise defined in this Preferred
Securities Guarantee;

         (c) a term defined anywhere in this Preferred Securities Guarantee has
the same meaning throughout;



                                       1
<PAGE>   6

         (d) all references to "the Preferred Securities Guarantee" or "this
Preferred Securities Guarantee" are to this Preferred Securities Guarantee as
modified, supplemented or amended from time to time;

         (e) all references in this Preferred Securities Guarantee to Articles
and Sections are to Articles and Sections of this Preferred Securities
Guarantee, unless otherwise specified;

         (f) a term defined in the Trust Indenture Act has the same meaning when
used in this Preferred Securities Guarantee, unless otherwise defined in this
Preferred Securities Guarantee or unless the context otherwise requires; and

         (g) a reference to the singular includes the plural and vice versa.

         "Affiliate" has the same meaning as given to that term in Rule 405 of
the Securities Act of 1933, as amended, or any successor rule thereunder.

         "Business Day" means any day other than a Saturday, Sunday, a day on
which federal or state banking institutions in the Borough of Manhattan, The
City of New York are authorized or required by law, executive order or
regulation to close or a day on which the Corporate Trust Office of the
Preferred Guarantee Trustee is closed for business.

         "Corporate Trust Office" means the office of the Preferred Guarantee
Trustee at which the corporate trust business of the Preferred Guarantee Trustee
shall, at any particular time, be principally administered, which office at the
date of execution of this Preferred Securities Guarantee is located at Rodney
Square North, 1100 North Market Street, Wilmington, Delaware 19890-0001,
Attention: Corporate Trust Administration.

         "Covered Person" means any Holder or beneficial owner of Preferred
Securities.


         "Debentures" means the __ % Subordinated Debentures due March 31, 2031,
of the Debenture Issuer held by the Property Trustee of the Trust.


         "Debenture Issuer" means Baylake Corp., issuer of the Debentures under
the Indenture.

         "Event of Default" means a default by the Guarantor on any of its
payment or other obligations under this Preferred Securities Guarantee.

         "Guarantee Payments" means the following payments or distributions,
without duplication, with respect to the Preferred Securities, to the extent not
paid or made by the Trust: (i) any accrued and unpaid Distributions that are
required to be paid on such Preferred Securities, to the extent the Trust shall
have funds available therefor, (ii) the redemption price, including all accrued
and unpaid Distributions to the date of redemption (the "Redemption Price"), to
the extent the Trust has funds available therefor, with respect to any Preferred
Securities called for redemption by the Trust, and (iii) upon a voluntary or
involuntary dissolution, winding-up or termination of the Trust (other than in
connection with the distribution of the Debentures to the Holders in exchange
for the Preferred Securities as provided in the Trust Agreement), the lesser of
(A) the aggregate of the Liquidation Amount and all accrued and unpaid
Distributions on the



                                       2
<PAGE>   7

Preferred Securities to the date of payment, to the extent the Trust shall have
funds available therefor (the "Liquidation Distribution"), and (B) the amount of
assets of the Trust remaining available for distribution to Holders in
liquidation of the Trust.

         "Guarantor" means Baylake Corp., a Wisconsin corporation.

         "Holder" means a Person in whose name a Preferred Security is or
Preferred Securities are registered in the Securities Register; provided,
however, that, in determining whether the holders of the requisite percentage of
the Preferred Securities have given any request, notice, consent or waiver
hereunder, "Holder" shall not include the Guarantor, the Preferred Guarantee
Trustee or any of their respective Affiliates.

         "Indemnified Person" means the Preferred Guarantee Trustee, any
Affiliate of the Preferred Guarantee Trustee, or any officers, directors,
shareholders, members, partners, employees, representatives, nominees,
custodians or agents of the Preferred Guarantee Trustee.


         "Indenture" means the Indenture, dated as of ____________ __, 2001,
among the Debenture Issuer and Wilmington Trust Company, as trustee, and any
indenture supplemental thereto pursuant to which the Debentures are to be issued
to the Property Trustee of the Trust.


         "Liquidation Amount" means the stated value of $10 per Preferred
Security.

         "Liquidation Distribution" has the meaning provided therefor in the
definition of Guarantee Payments.

         "List of Holders" has the meaning set forth in Section 2.2 of this
Preferred Securities Guarantee.

         "Majority in Liquidation Amount of the Preferred Securities" means the
holders of more than 50% of the Liquidation Amount (including the stated value
that would be paid on redemption, liquidation or otherwise, plus accrued and
unpaid Distributions to the date upon which the voting percentages are
determined) of all of the Preferred Securities.

         "Officers' Certificate" means, with respect to any Person, a
certificate signed by two authorized officers of such Person, at least one of
whom shall be the principal executive officer, principal financial officer,
principal accounting officer, treasurer or any vice president of such Person.
Any Officers' Certificate delivered with respect to compliance with a condition
or covenant provided for in this Preferred Securities Guarantee shall include:

         (a) a statement that each officer signing the Officers' Certificate has
read the covenant or condition and the definition relating thereto;

         (b) a brief statement of the nature and scope of the examination or
investigation undertaken by each officer in rendering the Officers' Certificate;



                                       3
<PAGE>   8

         (c) a statement that each such officer has made such examination or
investigation as, in such officer's opinion, is necessary to enable such officer
to express an informed opinion as to whether or not such covenant or condition
has been complied with; and

         (d) a statement as to whether, in the opinion of each such officer,
such condition or covenant has been complied with.

         "Person" means a legal person, including any individual, corporation,
estate, partnership, joint venture, association, joint stock company, limited
liability company, trust, unincorporated association, or government or any
agency or political subdivision thereof, or any other entity of whatever nature.

         "Preferred Guarantee Trustee" means Wilmington Trust Company, until a
Successor Preferred Guarantee Trustee has been appointed and has accepted such
appointment pursuant to the terms of this Preferred Securities Guarantee and
thereafter means each such Successor Preferred Guarantee Trustee.

         "Preferred Securities" means the _____% Cumulative Trust Preferred
Securities representing undivided beneficial interests in the assets of the
Trust which rank pari passu with Common Securities issued by the Trust;
provided, however, that upon the occurrence of an Event of Default, the rights
of holders of Common Securities to payment in respect of distributions and
payments upon liquidation, redemption and otherwise are subordinated to the
rights of holders of Preferred Securities.

         "Redemption Price" has the meaning provided therefor in the definition
of Guarantee Payments.

         "Responsible Officer" means, with respect to the Preferred Guarantee
Trustee, any officer within the Corporate Trust Office of the Preferred
Guarantee Trustee with direct responsibility for the administration of this
Preferred Securities Guarantee, including any vice-president, any assistant
vice-president, any assistant secretary, the treasurer, any assistant treasurer
or other officer of the Corporate Trust Office of the Preferred Guarantee
Trustee customarily performing functions similar to those performed by any of
the above designated officers and also means, with respect to a particular
corporate trust matter, any other officer to whom such matter is referred
because of that officer's knowledge of and familiarity with the particular
subject.

         "Securities Register" and "Securities Registrar" have the meanings
assigned to such terms as in the Trust Agreement (as defined in the Indenture).

         "Successor Preferred Guarantee Trustee" means a successor Preferred
Guarantee Trustee possessing the qualifications to act as Preferred Guarantee
Trustee under Section 4.1.

         "Trust Indenture Act" means the Trust Indenture Act of 1939, as
amended, as in force at the date of which this instrument was executed;
provided, however, that in the event the Trust Indenture Act of 1939, as
amended, is amended after such date, "Trust Indenture Act" means, to the extent
required by any such amendment, the Trust Indenture Act of 1939, as so amended.




                                       4
<PAGE>   9


                                   ARTICLE II.
                               TRUST INDENTURE ACT

         SECTION 2.1. TRUST INDENTURE ACT; APPLICATION.

         (a) This Preferred Securities Guarantee is subject to the provisions of
the Trust Indenture Act that are required to be part of this Preferred
Securities Guarantee and shall, to the extent applicable, be governed by such
provisions.

         (b) If and to the extent that any provision of this Preferred
Securities Guarantee limits, qualifies or conflicts with the duties imposed by
Section 310 to 317, inclusive, of the Trust Indenture Act, such imposed duties
shall control.

         SECTION 2.2. THE LIST OF HOLDERS OF THE SECURITIES.

         (a) In the event the Preferred Guarantee Trustee is not also the
Securities Registrar, the Guarantor shall provide the Preferred Guarantee
Trustee with a list, in such form as the Preferred Guarantee Trustee may
reasonably require, of the names and addresses of the Holders of the Preferred
Securities (the "List of Holders") (i) within three Business Days after March
15, June 15, September 15 and December 15 in each year, and (ii) at any other
time within 30 days of receipt by the Guarantor of a written request for a List
of Holders as of a date no more than 15 days before such List of Holders is
given to the Preferred Guarantee Trustee; provided, that the Guarantor shall not
be obligated to provide such List of Holders at any time the List of Holders
does not differ from the most recent List of Holders given to the Preferred
Guarantee Trustee by the Guarantor. The Preferred Guarantee Trustee may destroy
any List of Holders previously given to it on receipt of a new List of Holders.

         (b) The Preferred Guarantee Trustee shall comply with its obligations
under Sections 311(a), 311(b) and Section 312(b) of the Trust Indenture Act.

         SECTION 2.3. REPORTS BY THE PREFERRED GUARANTEE TRUSTEE. On or before
October 15 of each year, commencing October 15, 2001, the Preferred Guarantee
Trustee shall provide to the Holders of the Preferred Securities such reports as
are required by Section 313 of the Trust Indenture Act, if any, in the form and
in the manner provided by Section 313 of the Trust Indenture Act. The Preferred
Guarantee Trustee shall also comply with the requirements of Section 313(d) of
the Trust Indenture Act.

         SECTION 2.4. PERIODIC REPORTS TO THE PREFERRED GUARANTEE TRUSTEE. The
Guarantor shall provide to the Preferred Guarantee Trustee such documents,
reports and information as required by Section 314 (if any) and the compliance
certificate required by Section 314 of the Trust Indenture Act in the form, in
the manner and at the times required by Section 314 of the Trust Indenture Act.


                                       5
<PAGE>   10

         SECTION 2.5. EVIDENCE OF COMPLIANCE WITH CONDITIONS PRECEDENT. The
Guarantor shall provide to the Preferred Guarantee Trustee such evidence of
compliance with any conditions precedent, if any, provided for in this Preferred
Securities Guarantee that relate to any of the matters set forth in Section
314(c) of the Trust Indenture Act. Any certificate or opinion required to be
given by an officer pursuant to Section 314(c)(1) may be given in the form of an
Officers' Certificate.

         SECTION 2.6. EVENTS OF DEFAULT; WAIVER. The Holders of a Majority in
Liquidation Amount of the Preferred Securities may, by vote, on behalf of the
Holders of all of the Preferred Securities, waive any past Event of Default and
its consequences. Upon such waiver, any such Event of Default shall cease to
exist, and any Event of Default arising therefrom shall be deemed to have been
cured, for every purpose of this Preferred Securities Guarantee, but no such
waiver shall extend to any subsequent or other default or Event of Default or
impair any right consequent thereon.

         SECTION 2.7. EVENT OF DEFAULT; NOTICE.

         (a) The Preferred Guarantee Trustee shall, within 90 days after the
occurrence of an Event of Default, transmit by mail, first class postage
prepaid, to the Holders of the Preferred Securities, notices of all Events of
Default actually known to a Responsible Officer of the Preferred Guarantee
Trustee, unless such defaults have been cured before the giving of such notice;
provided, that, except in the case of a default by Guarantor under the terms of
this Preferred Securities Guarantee on any of its payment obligations, the
Preferred Guarantee Trustee shall be protected in withholding such notice if and
so long as a Responsible Officer of the Preferred Guarantee Trustee in good
faith determines that the withholding of such notice is in the interests of the
Holders of the Preferred Securities.

         (b) The Preferred Guarantee Trustee shall not be deemed to have
knowledge of any Event of Default unless the Preferred Guarantee Trustee shall
have received written notice, or of which a Responsible Officer of the Preferred
Guarantee Trustee charged with the administration of the Trust Agreement shall
have obtained actual knowledge of such Event of Default.

         SECTION 2.8. CONFLICTING INTERESTS. The Trust Agreement shall be deemed
to be specifically described in this Preferred Securities Guarantee for the
purposes of clause (i) of the first proviso contained in Section 310(b) of the
Trust Indenture Act.




                                       6
<PAGE>   11


                                  ARTICLE III.
                            POWERS, DUTIES AND RIGHTS
                       OF THE PREFERRED GUARANTEE TRUSTEE

         SECTION 3.1. POWERS AND DUTIES OF THE PREFERRED GUARANTEE TRUSTEE.

         (a) This Preferred Securities Guarantee shall be held by the Preferred
Guarantee Trustee for the benefit of the Holders of the Preferred Securities,
and the Preferred Guarantee Trustee shall not transfer this Preferred Securities
Guarantee to any Person except a Holder of Preferred Securities exercising his
or her rights pursuant to Section 5.4(b) or to a Successor Preferred Guarantee
Trustee on acceptance by such Successor Preferred Guarantee Trustee of its
appointment to act as Successor Preferred Guarantee Trustee. The right, title
and interest of the Preferred Guarantee Trustee shall automatically vest in any
Successor Preferred Guarantee Trustee, and such vesting and cessation of title
shall be effective whether or not conveyancing documents have been executed and
delivered pursuant to the appointment of such Successor Preferred Guarantee
Trustee.

         (b) If an Event of Default actually known to a Responsible Officer of
the Preferred Guarantee Trustee has occurred and is continuing, the Preferred
Guarantee Trustee shall enforce this Preferred Securities Guarantee for the
benefit of the Holders of the Preferred Securities.

         (c) The Preferred Guarantee Trustee, before the occurrence of any Event
of Default and after the curing of all Events of Default that may have occurred,
shall undertake to perform only such duties as are specifically set forth in
this Preferred Securities Guarantee, and no implied covenants shall be read into
this Preferred Securities Guarantee against the Preferred Guarantee Trustee. In
case an Event of Default has occurred (that has not been cured or waived
pursuant to Section 2.6) and is actually known to a Responsible Officer of the
Preferred Guarantee Trustee, the Preferred Guarantee Trustee shall exercise such
of the rights and powers vested in it by this Preferred Securities Guarantee,
and use the same degree of care and skill in its exercise thereof, as a prudent
person would exercise or use under the circumstances in the conduct of his or
her own affairs.

         (d) No provision of this Preferred Securities Guarantee shall be
construed to relieve the Preferred Guarantee Trustee from liability for its own
negligent action, its own negligent failure to act, or its own willful
misconduct, except that:

             (i)   prior to the occurrence of any Event of Default and after the
         curing or waiving of all such Events of Default that may have occurred:

                   (A) the duties and obligations of the Preferred Guarantee
             Trustee shall be determined solely by the express provisions of
             this Preferred Securities Guarantee, and the Preferred Guarantee
             Trustee shall not be liable except for the performance of such
             duties and obligations as are specifically set forth in this
             Preferred Securities Guarantee, and no implied covenants or
             obligations shall be read into this Preferred Securities Guarantee
             against the Preferred Guarantee Trustee; and


                                       7
<PAGE>   12

                           (B) in the absence of bad faith on the part of the
                  Preferred Guarantee Trustee, the Preferred Guarantee Trustee
                  may conclusively rely, as to the truth of the statements and
                  the correctness of the opinions expressed therein, upon any
                  certificates or opinions furnished to the Preferred Guarantee
                  Trustee and conforming to the requirements of this Preferred
                  Securities Guarantee; but in the case of any such certificates
                  or opinions that by any provision hereof are specifically
                  required to be furnished to the Preferred Guarantee Trustee,
                  the Preferred Guarantee Trustee shall be under a duty to
                  examine the same to determine whether or not they conform to
                  the requirements of this Preferred Securities Guarantee;

                  (ii)  the Preferred Guarantee Trustee shall not be liable for
         any error of judgment made in good faith by a Responsible Officer of
         the Preferred Guarantee Trustee, unless it shall be proved that the
         Preferred Guarantee Trustee was negligent in ascertaining the pertinent
         facts upon which such judgment was made;

                  (iii) the Preferred Guarantee Trustee shall not be liable with
         respect to any action taken or omitted to be taken by it in good faith
         in accordance with the direction of the Holders of not less than a
         Majority in Liquidation Amount of the Preferred Securities relating to
         the time, method and place of conducting any proceeding for any remedy
         available to the Preferred Guarantee Trustee, or exercising any trust
         or power conferred upon the Preferred Guarantee Trustee under this
         Preferred Securities Guarantee; and

                  (iv)  no provision of this Preferred Securities Guarantee
         shall require the Preferred Guarantee Trustee to expend or risk its own
         funds or otherwise incur personal financial liability in the
         performance of any of its duties or in the exercise of any of its
         rights or powers, if the Preferred Guarantee Trustee shall have
         reasonable grounds for believing that the repayment of such funds or
         liability is not reasonably assured to it under the terms of this
         Preferred Securities Guarantee or indemnity, reasonably satisfactory to
         the Preferred Guarantee Trustee, against such risk or liability is not
         reasonably assured to it.

         SECTION 3.2. CERTAIN RIGHTS OF THE PREFERRED GUARANTEE TRUSTEE.

         (a) Subject to the provisions of Section 3.1:

             (i)   the Preferred Guarantee Trustee may conclusively rely, and
         shall be fully protected in acting or refraining from acting upon, any
         resolution, certificate, statement, instrument, opinion, report,
         notice, request, direction, consent, order, bond, debenture, note,
         other evidence of indebtedness or other paper or document believed by
         it to be genuine and to have been signed, sent or presented by the
         proper party or parties;

             (ii)  any direction or act of the Guarantor contemplated by this
         Preferred Securities Guarantee shall be sufficiently evidenced by an
         Officers' Certificate;



                                       8
<PAGE>   13

             (iii)  whenever, in the administration of this Preferred Securities
         Guarantee, the Preferred Guarantee Trustee shall deem it desirable that
         a matter be proved or established before taking, suffering or omitting
         any action hereunder, the Preferred Guarantee Trustee (unless other
         evidence is herein specifically prescribed) may, in the absence of bad
         faith on its part, request and conclusively rely upon an Officers'
         Certificate which, upon receipt of such request, shall be promptly
         delivered by the Guarantor;

             (iv)   the Preferred Guarantee Trustee shall have no duty to see to
         any recording, filing or registration of any instrument (or any
         rerecording, refiling or reregistration thereof);

             (v)    the Preferred Guarantee Trustee may consult with counsel,
         and the written advice or opinion of such counsel with respect to legal
         matters shall be full and complete authorization and protection in
         respect of any action taken, suffered or omitted by it hereunder in
         good faith and in accordance with such advice or opinion. Such counsel
         may be counsel to the Guarantor or any of its Affiliates and may
         include any of its employees. The Preferred Guarantee Trustee shall
         have the right at any time to seek instructions concerning the
         administration of this Preferred Securities Guarantee from any court of
         competent jurisdiction;

             (vi)   the Preferred Guarantee Trustee shall be under no obligation
         to exercise any of the rights or powers vested in it by this Preferred
         Securities Guarantee at the request or direction of any Holder, unless
         such Holder shall have provided to the Preferred Guarantee Trustee such
         security and indemnity, reasonably satisfactory to the Preferred
         Guarantee Trustee, against the costs, expenses (including reasonable
         attorneys' fees and expenses and the expenses of the Preferred
         Guarantee Trustee's agents, nominees or custodians) and liabilities
         that might be incurred by it in complying with such request or
         direction, including such reasonable advances as may be requested by
         the Preferred Guarantee Trustee; provided that, nothing contained in
         this Section 3.2(a)(vi) shall be taken to relieve the Preferred
         Guarantee Trustee, upon the occurrence and during the continuance of an
         Event of Default, of its obligation to exercise the rights and powers
         vested in it by this Preferred Securities Guarantee;

             (vii)  the Preferred Guarantee Trustee shall not be bound to make
         any investigation into the facts or matters stated in any resolution,
         certificate, statement, instrument, opinion, report, notice, request,
         direction, consent, order, bond, debenture, note, other evidence of
         indebtedness or other paper or document, but the Preferred Guarantee
         Trustee, in its discretion, may make such further inquiry or
         investigation into such facts or matters as it may see fit;

             (viii) the Preferred Guarantee Trustee may execute any of the
         trusts or powers hereunder or perform any duties hereunder either
         directly or by or through agents, nominees, custodians or attorneys,
         and the Preferred Guarantee Trustee shall not be responsible for any
         misconduct or negligence on the part of any agent or attorney appointed
         with due care by it hereunder;



                                       9
<PAGE>   14

             (ix)   no third party shall be required to inquire as to the
         authority of the Preferred Guarantee Trustee to so act or as to its
         compliance with any of the terms and provisions of this Preferred
         Securities Guarantee, both of which shall be conclusively evidenced by
         the Preferred Guarantee Trustee's or its agent's taking such action;

             (x)    whenever in the administration of this Preferred Securities
         Guarantee the Preferred Guarantee Trustee shall deem it desirable to
         receive instructions with respect to enforcing any remedy or right or
         taking any other action hereunder, the Preferred Guarantee Trustee (A)
         may request instructions from the Holders of a Majority in Liquidation
         Amount of the Preferred Securities, (B) may refrain from enforcing such
         remedy or right or taking such other action until such instructions are
         received, and (C) shall be protected in conclusively relying on or
         acting in accordance with such instructions.

         (b) No provision of this Preferred Securities Guarantee shall be deemed
to impose any duty or obligation on the Preferred Guarantee Trustee to perform
any act or acts or exercise any right, power, duty or obligation conferred or
imposed on it in any jurisdiction in which it shall be illegal, or in which the
Preferred Guarantee Trustee shall be unqualified or incompetent in accordance
with applicable law, to perform any such act or acts or to exercise any such
right, power, duty or obligation. No permissive power or authority available to
the Preferred Guarantee Trustee shall be construed to be a duty.

         SECTION 3.3. NOT RESPONSIBLE FOR RECITALS OR ISSUANCE OF GUARANTEE. The
Recitals contained in this Guarantee shall be taken as the statements of the
Guarantor, and the Preferred Guarantee Trustee does not assume any
responsibility for their correctness. The Preferred Guarantee Trustee makes no
representation as to the validity or sufficiency of this Preferred Securities
Guarantee.

                                   ARTICLE IV.
                         THE PREFERRED GUARANTEE TRUSTEE

         SECTION 4.1. THE PREFERRED GUARANTEE TRUSTEE; ELIGIBILITY.

         (a) There shall at all times be a Preferred Guarantee Trustee which
shall:

             (i)   not be an Affiliate of the Guarantor; and

             (ii)  be a Person permitted by the Securities and Exchange
         Commission to act as an institutional trustee under the Trust Indenture
         Act, authorized under such laws to exercise corporate trust powers,
         having a combined capital and surplus of at least $50,000,000, and
         subject to supervision or examination by federal, state, territorial or
         District of Columbia authority. If such Person publishes reports of
         condition at least annually, pursuant to law or to the requirements of
         its supervising or examining authority referred to above, then, for the
         purposes of this Section 4.1(a)(ii), the combined capital and surplus
         of such



                                       10
<PAGE>   15

         Person shall be deemed to be its combined capital and surplus as
         set forth in its most recent report of condition so published.

         (b) If at any time the Preferred Guarantee Trustee shall cease to be
eligible to so act under Section 4.1(a), the Preferred Guarantee Trustee shall
immediately resign in the manner and with the effect set out in Section 4.2(c).

         (c) If the Preferred Guarantee Trustee has or shall acquire any
"conflicting interest" within the meaning of Section 310(b) of the Trust
Indenture Act, the Preferred Guarantee Trustee and the Guarantor shall in all
respects comply with the provisions of Section 310(b) of the Trust Indenture
Act.

         SECTION 4.2. APPOINTMENT, REMOVAL AND RESIGNATION OF THE PREFERRED
GUARANTEE TRUSTEE.

         (a) Subject to Section 4.2(b), the Preferred Guarantee Trustee may be
appointed or removed without cause at any time by the Guarantor.

         (b) The Preferred Guarantee Trustee shall not be removed in accordance
with Section 4.2(a) until a Successor Preferred Guarantee Trustee has been
appointed and has accepted such appointment by written instrument executed by
such Successor Preferred Guarantee Trustee and delivered to the Guarantor.

         (c) The Preferred Guarantee Trustee appointed to office shall hold
office until a Successor Preferred Guarantee Trustee shall have been appointed
or until its removal or resignation. The Preferred Guarantee Trustee may resign
from office (without need for prior or subsequent accounting) by an instrument
in writing executed by the Preferred Guarantee Trustee and delivered to the
Guarantor, which resignation shall not take effect until a Successor Preferred
Guarantee Trustee has been appointed and has accepted such appointment by
instrument in writing executed by such Successor Preferred Guarantee Trustee and
delivered to the Guarantor and the resigning Preferred Guarantee Trustee.

         (d) If no Successor Preferred Guarantee Trustee shall have been
appointed and accepted appointment as provided in this Section 4.2 within 60
days after delivery to the Guarantor of an instrument of resignation, the
resigning Preferred Guarantee Trustee may petition any court of competent
jurisdiction for appointment of a Successor Preferred Guarantee Trustee. Such
court may thereupon, after prescribing such notice, if any, as it may deem
proper, appoint a Successor Preferred Guarantee Trustee.

         (e) No Preferred Guarantee Trustee shall be liable for the acts or
omissions to act of any Successor Preferred Guarantee Trustee.

         (f) Upon termination of this Preferred Securities Guarantee or removal
or resignation of the Preferred Guarantee Trustee pursuant to this Section 4.2,
the Guarantor shall pay to the Preferred Guarantee Trustee all fees and expenses
accrued to the date of such termination, removal or resignation.


                                       11
<PAGE>   16

                                   ARTICLE V.
                                    GUARANTEE

         SECTION 5.1. GUARANTEE. The Guarantor irrevocably and unconditionally
agrees to pay in full to the Holders the Guarantee Payments (without duplication
of amounts theretofore paid by the Trust), as and when due, regardless of any
defense, right of set-off or counterclaim that the Trust may have or assert. The
Guarantor's obligation to make a Guarantee Payment may be satisfied by direct
payment of the required amounts by the Guarantor to the Holders or by causing
the Trust to pay such amounts to the Holders.

         SECTION 5.2. WAIVER OF NOTICE AND DEMAND. The Guarantor hereby waives
notice of acceptance of this Preferred Securities Guarantee and of any liability
to which it applies or may apply, presentment, demand for payment, any right to
require a proceeding first against the Trust or any other Person before
proceeding against the Guarantor, protest, notice of nonpayment, notice of
dishonor, notice of redemption and all other notices and demands.

         SECTION 5.3. OBLIGATIONS NOT AFFECTED. The obligations, covenants,
agreements and duties of the Guarantor under this Preferred Securities Guarantee
shall in no way be affected or impaired by reason of the happening from time to
time of any of the following:

         (a) the release or waiver, by operation of law or otherwise, of the
performance or observance by the Trust of any express or implied agreement,
covenant, term or condition relating to the Preferred Securities to be performed
or observed by the Trust;

         (b) the extension of time for the payment by the Trust of all or any
portion of the Distributions, Redemption Price, Liquidation Distribution or any
other sums payable under the terms of the Preferred Securities or the extension
of time for the performance of any other obligation under, arising out of, or in
connection with, the Preferred Securities (other than an extension of time for
payment of Distributions, Redemption Price, Liquidation Distribution or other
sum payable that results from the extension of any interest payment period on
the Debentures or any extension of the maturity date of the Debentures permitted
by the Indenture);

         (c) any failure, omission, delay or lack of diligence on the part of
the Holders to enforce, assert or exercise any right, privilege, power or remedy
conferred on the Holders pursuant to the terms of the Preferred Securities, or
any action on the part of the Trust granting indulgence or extension of any
kind;

         (d) the voluntary or involuntary liquidation, dissolution, sale of any
collateral, receivership, insolvency, bankruptcy, assignment for the benefit of
creditors, reorganization, arrangement, composition or readjustment of debt of,
or other similar proceedings affecting, the Trust or any of the assets of the
Trust;

         (e) any invalidity of, or defect or deficiency in, the Preferred
Securities;



                                       12
<PAGE>   17
         (f) any failure or omission to receive any regulatory approval or
consent required in connection with the Preferred Securities (or the common
equity securities issued by the Trust), including the failure to receive any
approval of the Board of Governors of the Federal Reserve System required for
the redemption of the Preferred Securities;

         (g) the settlement or compromise of any obligation guaranteed hereby or
hereby incurred; or

         (h) any other circumstance whatsoever that might otherwise constitute a
legal or equitable discharge or defense of a guarantor, it being the intent of
this Section 5.3 that the obligations of the Guarantor hereunder shall be
absolute and unconditional under any and all circumstances.

         There shall be no obligation of the Holders to give notice to, or
obtain consent of, the Guarantor with respect to the happening of any of the
foregoing.

         SECTION 5.4. RIGHTS OF THE HOLDERS.

         (a) The Holders of a Majority in Liquidation Amount of the Preferred
Securities have the right to direct the time, method and place of conducting of
any proceeding for any remedy available to the Preferred Guarantee Trustee in
respect of this Preferred Securities Guarantee or exercising any trust or power
conferred upon the Preferred Guarantee Trustee under this Preferred Securities
Guarantee.

         (b) Any Holder of Preferred Securities may institute and prosecute a
legal proceeding directly against the Guarantor to enforce its rights under this
Preferred Securities Guarantee, without first instituting and prosecuting a
legal proceeding against the Trust, the Preferred Guarantee Trustee or any other
Person.

         SECTION 5.5. GUARANTEE OF PAYMENT. This Preferred Securities Guarantee
creates a guarantee of payment and not of collection.

         SECTION 5.6. SUBROGATION. The Guarantor shall be subrogated to all (if
any) rights of the Holders of the Preferred Securities against the Trust in
respect of any amounts paid to such Holders by the Guarantor under this
Preferred Securities Guarantee; provided, however, that the Guarantor shall not
(except to the extent required by mandatory provisions of law) be entitled to
enforce or exercise any right that it may acquire by way of subrogation or any
indemnity, reimbursement or other agreement, in all cases as a result of payment
under this Preferred Securities Guarantee, if, at the time of any such payment,
any amounts are due and unpaid under this Preferred Securities Guarantee. If any
amount shall be paid to the Guarantor in violation of the preceding sentence,
the Guarantor agrees to hold such amount in trust for the Holders and to pay
over such amount to the Holders.


                                       13
<PAGE>   18

         SECTION 5.7. INDEPENDENT OBLIGATIONS. The Guarantor acknowledges that
its obligations hereunder are independent of the obligations of the Trust with
respect to the Preferred Securities, and that the Guarantor shall be liable as
principal and as debtor hereunder to make Guarantee Payments pursuant to the
terms of this Preferred Securities Guarantee notwithstanding the occurrence of
any event referred to in subsections (a) through (h), inclusive, of Section 5.3
hereof.

                                  ARTICLE VI.
                   LIMITATION OF TRANSACTIONS; SUBORDINATION

         SECTION 6.1. LIMITATION ON TRANSACTIONS. So long as any of the
Preferred Securities remain outstanding, if there shall have occurred an Event
of Default under this Preferred Securities Guarantee Agreement, an event of
default under the Trust Agreement or during an Extension Period (as defined in
the Indenture), then (a) neither the Guarantor nor any of its Subsidiaries (as
defined in the Indenture) shall declare or pay any dividend on, make any
distributions with respect to, or redeem, purchase, acquire or make a
liquidation payment with respect to, any of its capital stock except as
permitted in such circumstances by the Indenture (b) the Guarantor shall not
make any payment of principal or interest on or repay, repurchase or redeem any
debt securities issued by the Guarantor which rank pari passu with or junior to
the Debentures other than payments under this Preferred Securities Guarantee and
(c) the Guarantor shall not redeem, purchase or acquire less then all of the
Outstanding Debentures or any of the Preferred Securities.

         SECTION 6.2 RANKING. This Preferred Securities Guarantee will
constitute an unsecured obligation of the Guarantor and will rank (a)
subordinate and junior in right of payment to all Senior Debt, Subordinated Debt
and Additional Senior Obligations (as defined in the Indenture) of the
Guarantor, (b) pari passu with the most senior preferred securities now or
hereafter issued by the Guarantor and with any guarantee now or hereafter
entered into by the Guarantor in respect of any preferred securities of any
Affiliate of the Guarantor, and (c) senior to the Guarantor's common stock.

                                  ARTICLE VII.
                                  TERMINATION

         SECTION 7.1. TERMINATION. This Preferred Securities Guarantee shall
terminate upon (a) full payment of the Redemption Price of all the Preferred
Securities, (b) full payment of the amounts payable in accordance with the Trust
Agreement upon liquidation of the Trust, or (c) distribution of the Debentures
to the Holders of the Preferred Securities. Notwithstanding the foregoing, this
Preferred Securities Guarantee shall continue to be effective or shall be
reinstated, as the case may be, if at any time any Holder of Preferred
Securities must restore payment of any sums paid under the Preferred Securities
or under this Preferred Securities Guarantee.




                                       14
<PAGE>   19


                                  ARTICLE VIII.
                                 INDEMNIFICATION

         SECTION 8.1. EXCULPATION.

         (a) No Indemnified Person shall be liable, responsible or accountable
in damages or otherwise to the Guarantor or any Covered Person for any loss,
damage or claim incurred by reason of any act or omission performed or omitted
by such Indemnified Person in good faith in accordance with this Preferred
Securities Guarantee and in a manner that such Indemnified Person reasonably
believed to be within the scope of the authority conferred on such Indemnified
Person by this Preferred Securities Guarantee or by law, except that an
Indemnified Person shall be liable for any such loss, damage or claim incurred
by reason of such Indemnified Person's negligence or willful misconduct with
respect to such acts or omissions.

         (b) An Indemnified Person shall be fully protected in relying in good
faith upon the records of the Guarantor and upon such information, opinions,
reports or statements presented to the Guarantor by any Person as to matters the
Indemnified Person reasonably believes are within such other Person's
professional or expert competence and who has been selected with reasonable care
by or on behalf of the Guarantor, including information, opinions, reports or
statements as to the value and amount of the assets, liabilities, profits,
losses, or any other facts pertinent to the existence and amount of assets from
which Distributions to the Holders of the Preferred Securities might properly be
paid.

         SECTION 8.2. INDEMNIFICATION. The Guarantor agrees to indemnify each
Indemnified Person for, and to hold each Indemnified Person harmless against,
any loss, liability or expense incurred without negligence or bad faith on its
part, arising out of or in connection with the acceptance or administration of
the trust or trusts hereunder, including the costs and expenses (including
reasonable legal fees and expenses) of defending itself against, or
investigating, any claim or liability in connection with the exercise or
performance of any of its powers or duties hereunder. The obligation to
indemnify as set forth in this Section 8.2 shall survive the termination of this
Preferred Securities Guarantee.

                                   ARTICLE IX.
                                  MISCELLANEOUS

         SECTION 9.1. SUCCESSORS AND ASSIGNS. All guarantees and agreements
contained in this Preferred Securities Guarantee shall bind the successors,
assigns, receivers, trustees and representatives of the Guarantor and shall
inure to the benefit of the Holders of the Preferred Securities then
outstanding.

         SECTION 9.2. AMENDMENTS. Except with respect to any changes that do not
adversely affect the rights of the Holders (in which case no consent of the
Holders will be required), this Preferred Securities Guarantee may only be
amended with the prior approval of the Holders of at least a Majority in
Liquidation Amount of the Preferred Securities. The provisions of Article VI of
the Trust Agreement with respect to meetings of the Holders of the Preferred
Securities apply to the giving of such approval.


                                       15
<PAGE>   20

         SECTION 9.3. NOTICES. All notices provided for in this Preferred
Securities Guarantee shall be in writing, duly signed by the party giving such
notice, and shall be delivered, telecopied or mailed by registered or certified
mail, as follows:

         (a) If given to the Preferred Guarantee Trustee, at the Preferred
Guarantee Trustee's mailing address set forth below (or such other address as
the Preferred Guarantee Trustee may give notice of to the Holders of the
Preferred Securities):

                             Wilmington Trust Company
                             Rodney Square North
                             1100 North Market Street
                             Wilmington, Delaware 19890-0001
                             Attention:   Corporate Trust Administration

         (b) If given to the Guarantor, at the Guarantor's mailing address set
forth below (or such other address as the Guarantor may give notice of to the
Holders of the Preferred Securities):

                             Baylake Corp.
                             217 North Fourth Avenue
                             Sturgeon Bay, Wisconsin 54235
                             Attention: Steven D. Jennerjohn, Treasurer

         (c) If given to any Holder of Preferred Securities, at the address set
forth on the books and records of the Trust.

         All such notices shall be deemed to have been given when received in
person, telecopied with receipt confirmed, or mailed by first class mail,
postage prepaid except that if a notice or other document is refused delivery or
cannot be delivered because of a changed address of which no notice was given,
such notice or other document shall be deemed to have been delivered on the date
of such refusal or inability to deliver.

         SECTION 9.4. BENEFIT. This Preferred Securities Guarantee is solely for
the benefit of the Holders of the Preferred Securities and, subject to Section
3.1(a), is not separately transferable from the Preferred Securities.

         SECTION 9.5. GOVERNING LAW. THIS PREFERRED SECURITIES GUARANTEE SHALL
BE GOVERNED BY, AND CONSTRUED AND INTERPRETED IN ACCORDANCE WITH, THE LAWS OF
THE STATE OF WISCONSIN.



                                       16
<PAGE>   21


         This Preferred Securities Guarantee is executed as of the day and year
first above written.


                                  BAYLAKE CORP.,
                                  as Guarantor




                                  By: __________________________________________
                                          Thomas L. Herlache
                                          President and Chief Executive Officer




                                  WILMINGTON TRUST COMPANY,
                                  as Preferred Guarantee Trustee




                                  By: __________________________________________

                                          Name:_________________________________

                                          Title: _______________________________




                                       17