<PAGE>
                                                                   Exhibit 10.6

                      COLLATERAL ASSIGNMENT OF FLOOD BOOK

THIS COLLATERAL ASSIGNMENT OF FLOOD BOOK (herein, "Assignment") is made
effective this 15 day of August, 2002 by BANKERS UNDERWRITERS, INC., A FLORIDA
CORPORATION (herein, "Pledgor"), in favor of INSURANCE MANAGEMENT SOLUTIONS
GROUP, INC., A FLORIDA CORPORATION (herein, "Lender").

                                   RECITALS:

WHEREAS, Lender has its principal place of business located in St. Petersburg,
Florida, and has established a revolving line of credit (collectively, with the
Note and Loan Documents, as defined herein, the "Credit Line") in favor of
Bankers Insurance Group, Inc., a Florida corporation (herein, "BIG") and
Pledgor (BIG and Pledgor herein, collectively, "Borrower"), in the amount of
seven million and no/100 dollar ($7,000,000.00), as evidenced by Borrower's
Revolving Line of Credit Master Promissory Note (herein, "Note") of even date
herewith and in like principal amount; and

WHEREAS, Pledgor, Bankers Insurance Company, a Florida corporation (herein,
"BIC"), and First Community Insurance Company, a Florida corporation (herein,
"FCIC") are each wholly-owned subsidiaries of BIG; and

WHEREAS, in connection with the establishment of the Credit Line, Borrower has
executed and delivered various applicable documents, all dated of even date
herewith (collectively together with this Assignment, the "Loan Documents")
including, but not limited to, the Note, Credit and Security Agreement, UCC-1
Financing Statement, and Further Assurances Agreement; and

WHEREAS, Pledgor, as a Florida general insurance agent serving both BIC and
FCIC, is the owner of a significant flood book of business (inclusive of all
agency agreements pertaining thereto herein, "Flood Book") consisting of in
excess of 350,000 flood insurance policies on properties located in various
states of the United States, which Flood Book is more particularly described on
Exhibit "A" attached hereto; and

WHEREAS, as a condition precedent to establishing the Credit Line, Lender has
required the execution of this Assignment;

NOW, THEREFORE, in consideration of the aforesaid Credit Line, as well as for
other good and valuable consideration, Pledgor hereby assigns the Flood Book to
Lender as collateral security for the repayment of all sums advanced on the
Credit Line and for the performance by Borrower of all covenants and agreements
under the Loan Documents, in accordance with the following terms and conditions:

     SECTION 1.  RECITALS.

          The statements contained in the recitals of fact set forth above
          (herein, "Recitals") are true and correct, and the Recitals by this
          reference are made a part of this Assignment.

     SECTION 2.  EXHIBITS.

          The exhibits attached to this Assignment are by this reference made a
          part of this Assignment.



                                     1 of 7

<PAGE>
SECTION 3. TERM OF ASSIGNMENT.

     a) This Assignment shall be effective upon the date set forth hereinabove.

     b) This Assignment shall continue in full force and effect until Borrower
        shall have satisfied in full all of its obligations to Lender under the
        Note and other Loan Documents.


SECTION 4. POWER OF ATTORNEY.

     a) Appointment.

          1) Until such time as the Credit Line has been paid in full or upon
             consent by Lender otherwise, and except as provided below, Pledgor
             irrevocably nominates, constitutes and appoints Lender its true and
             lawful attorney in fact, with full power of substitution and
             revocation for it, in its name, place and stead and either in the
             name of Lender or in the name of Pledgor to assign any agency
             agreements between Pledgor and agents responsible for the
             production of federal flood insurance policies as well as any other
             instruments, agreements, or certificates necessary or appropriate
             to give full force an effect to such assignment which assignment
             may be to Lender, to BIC, to FCIC, or to such other party as Lender
             may select, but Lender shall not be under any duty to exercise any
             such authority or power or in any way be responsible for the
             collection of any sums owing under the Flood Book (herein, "Power
             of Attorney").

          2) All rights, powers, and authority of said attorney-in-fact to
             exercise any and all rights and powers herein granted shall
             commence and be in full force and effect as of the date of this
             Assignment and such rights, powers, and authority shall remain in
             full force and effect thereafter until the termination of this
             Assignment.

     b) Limitation. Notwithstanding the foregoing or anything contained in this
        Assignment to the contrary, Lender shall not exercise its rights under
        this Power of Attorney or under this Agreement until an Event of Default
        (as defined below) occurs hereunder.

     c) Power. The Power of Attorney set forth herein is a power coupled with an
        interest.


SECTION 5. AFFIRMATIVE COVENANTS
     Pledgor hereby covenants with Lender that:

               (i) for so long as any amount remains outstanding under the
                   Credit Line; and

               (ii) unless Lender notifies Pledgor in writing it dispenses with
                    any one (1) or more of the requirements contained in the
                    Loan Documents,

     Pledgor shall keep the Flood Book free from any and all liens,
     encumbrances, and security interests, and shall pay and discharge when due
     all taxes, levies, and other charges upon them and defend them against all
     claims of any person.


SECTION 6. REPRESENTATIONS AND WARRANTIES.

     a) Pledgor hereby represents and warrants to Lender that:

          1) it is the lawful owner and holder of the entire interest in the
             Flood Book described above;

          2) it has good right to sell, transfer and assign the same as
             aforesaid;


                                     2 of 7


<PAGE>


  3)  there are no actions, suits, or proceedings pending or, to the best of
      Pledgor's knowledge and belief, threatened against or affecting it whether
      civil, criminal, administrative, or investigative, and it is not in
      default with respect to any judgment, decision, order, writ, injunction,
      decree, or demand of any court or governmental authority;

  4)  the consummation of the transactions hereby contemplated in performance of
      this Assignment or of any of the Loan Documents will not result in any
      breach of, or constitute a default under any mortgage, deed of trust,
      lien, bank loan or credit agreement, corporate charter or by-law, or other
      instrument to which Pledgor is a party, or by which it is bound or
      affected;

  5)  it is a corporation duly organized, validly existing and in good standing
      under the laws of the State of Florida;

  6)  it has the lawful power to own its properties and to engage in the
      business it conducts;

  7)  it is duly qualified and in good standing as a foreign corporation in the
      jurisdictions wherein the nature of the business transacted by it or
      property owned by it make such qualification necessary;

  8)  the execution of this Assignment and the consummation of the transactions
      contemplated herein, and the performance of this Assignment and the Loan
      Documents, will not result in a breach or violation of any law,
      regulation, ordinance, order, judgment or decree applicable to Pledgor or
      any affiliate of Pledgor;

  9)  as of the date hereof, and subject to the ordinary course of Pledgor's
      business, Pledgor has no knowledge of circumstances or events that will
      cause or is likely to cause any material adverse decline in the number of
      flood insurance policies which constitute the Flood Book, or any material
      adverse decline in the value of the Flood Book;

 10)  no affiliate of BIG owns or conducts any flood book of business, except
      for a nominal portion of flood book business written by Bankers Security
      Insurance Company and First Community Insurance Company; and

 11)  by means of this Assignment, Pledgor had granted Lender a first priority
      security interest in the Pledgor's entire interest in the Flood Book.

 b) All of the representations and warranties set forth in this Section 6 shall
    survive until all sums due under the Credit Line are satisfied in full.

SECTION 7. EVENTS OF DEFAULT

 a) The term "Event of Default" shall mean:

    1) Any breach by Pledgor of any of the terms of this Assignment; or

    2) The breach or the occurrence of any event of default by Borrower, or any
       affiliate of Borrower, under any of the Loan Documents.

SECTION 8. MISCELLANEOUS

 a) Further Assurances. From time to time, Pledgor will execute and deliver to
    Lender such additional documents and will provide such additional
    information as Lender may reasonably require to carry out the terms of this
    Assignment and be informed of Pledgor's status and affairs.



                                     3 of 7



<PAGE>


 b) Payment. The Pledgor agrees that as long as the Note is in full force and
    effect, it will make all payments, when due, by check duly mailed or
    delivered to Lender at the address indicated in the Notice section of this
    Assignment, or at such other place as Lender may designate to the Pledgor in
    writing, notwithstanding any contrary provisions herein or in Note with
    respect to the place of payment.

 c) Survival of Representations and Warranties. All representations and
    warranties contained herein or made in writing by the Pledgor in connection
    herewith shall survive the execution and delivery of this Assignment and of
    the Note.

 d) Successors and Assigns. Pledgor may not assign its rights or obligations
    hereunder without the prior written consent of Lender, except that Pledgor
    may assign all or a portion of its rights and obligations hereunder to BIG,
    but only in conjunction with an assignment to BIG of a corresponding portion
    of Pledgor's interest in the Flood Book in such a manner that Lender shall
    maintain its first priority security interest in the entirety of the Flood
    Book whether then owned by BUI or BIG (provided, however, that BUI shall
    first serve notice of its intent to make such an assignment to BIG and shall
    not effect such assignment until Lender shall have completed such acts as
    Lender shall believe reasonably necessary to maintain its first priority
    security interest in the entirety of the Flood Book). All covenants and
    agreements in this Assignment contained by or on behalf of the parties
    hereto shall bind and inure to the benefit of the respective successors and
    assigns of the parties hereto whether so expressed or not.

 e) Waiver by Lender. Lender shall have the right at all times to enforce the
    provisions of this Assignment and any other Loan Documents executed pursuant
    hereto in strict accordance with the terms hereof and thereof,
    notwithstanding any conduct or custom on the part of Lender in refraining
    from so doing at any time or times. The failure of Lender at any time or
    times to enforce its rights under such provisions, strictly in accordance
    with the same, shall not be construed as having created a custom in any way
    or manner contrary to specific provisions of this Assignment or as having in
    any way or manner modified or waived the same. All rights and remedies of
    Lender are cumulative and concurrent and the exercise of one right or remedy
    shall not be deemed a waiver or release of any other right or remedy.

 f) Inspect Records. Lender (or any person or persons designated by it) shall,
    in its sole discretion, have the right to call at any place of business of
    Pledgor at any reasonable time, and without hindrance or delay, inspect,
    audit, check and make extracts from Pledgor's books, records, journals,
    orders, receipts and any correspondence and other data relating to
    Collateral, to Pledgor's business or any other transactions between the
    Parties hereto.

 g) Costs/Expenses. All reasonable costs and expenses of the Credit Line shall
    be paid by Pledgor, including but not limited to, out-of-pocket expenses for
    payment of recording and filing fees, legal fees and expenses of counsel
    appointed by the Lender, together with any interest and penalties for the
    late payment thereof, all of which amounts shall be payable at the time of
    the execution of this Assignment or upon demand in the event they are
    hereafter incurred.

 h) Attorney's Fees. If either of the parties hereto should bring a Court action
    alleging breach of this Assignment or seeking to enforce, rescind, renounce,
    declare void or terminate this Assignment or any provisions thereof, the
    prevailing party shall be entitled to recover all of its legal expenses,
    including reasonable attorney's fees and costs (including legal expenses for
    any appeals taken),


                                     4 of 7
<PAGE>
        and to have the same awarded as part of the judgment in the proceeding
        in which such legal expenses and attorney's fees were incurred.

     i) INTENTIONALLY DELETED.

     j) Captions. The paragraph contains captions as to contents of the
        particular paragraphs herein are inserted only for convenience and are
        in no way to be construed as part of this Assignment or as a limitation
        of the scope of the particular paragraph in which they are referred.

     k) Construction of Assignment. Words of a gender used in this Assignment
        shall be held to include any other gender, the words in a singular
        number held to include the plural, when the sentence so requires.

     l) Counterparts. This Assignment may be executed in several counterparts,
        each of which so executed shall be deemed to be an original, and said
        counterparts shall together constitute and be one and the same
        instrument.

     m) Entire Assignment. This Assignment contains all of the oral and/or
        previously written agreements, representations, and arrangements
        between the parties hereto, and all right which the respective Parties
        may have had under any written agreements and/or oral agreements are
        hereby canceled and terminated, and all parties agree that there are no
        representations or warranties other than those set forth herein.

     n) Invalidation. Should any part of this Assignment for any reason be
        declared invalid, such decision shall not effect the validity of any
        remaining portion, which remaining portion shall remain in full force
        and effect as if this Assignment had been executed with the invalid
        portion thereof eliminated. It is, therefore, declared the intention of
        the parties hereto that each of them will have executed the remaining
        portion of this Assignment without including therein any such part,
        parts or portion which may, for any reason, be hereafter declared void.

     o) Modification. No change or modification of this Assignment shall be
        valid unless the same shall be in writing and signed by all of the
        Parties hereto.

     p) Notices. Any and all notices, designations, consents, offers,
        acceptances, or any other communication provided for herein shall be
        given in writing by hand delivery, by overnight carrier, by registered
        or certified mail or by facsimile transmission and shall be addressed
        as follows:

               As to Pledgor:      Bankers Underwriters, Inc.
                                   360 Central Avenue, 17th Floor
                                   St. Petersburg, Florida 33701
                                   Att: David B. Snyder
                                   Telephone: (727) 823-4000
                                   Telefax: (727) 823-6518

               As to Lender:       Insurance Management Solutions Group, Inc.
                                   360 Central Avenue, 16th Floor
                                   St. Petersburg, Florida 33701
                                   Att: David Howard, President


                                     5 of 7

<PAGE>
                              Tel#: (727) 803-2040
                              Fax: (727) 803-4093

          Notices sent by hand delivery shall be deemed effective on the date of
          hand delivery. Notices being sent by overnight carrier shall be deemed
          effective on the next business day after being placed into the hands
          of the overnight carrier. Notices sent by registered or certified mail
          shall be deemed effective on the third business day after being
          deposited into the post office. Notices sent by facsimile transmission
          shall be deemed to be effective on day when sent if sent prior to 5:00
          p.m. (the time being determined by the time zone of the recipient)
          otherwise they shall be deemed effective on the next business day.

       q) Representation Acknowledged.  The parties acknowledge that each party
          and its counsel have reviewed and revised this Assignment and that the
          normal rules of construction to the effect that any ambiguities are to
          be resolved against the drafting party shall not be employed in the
          interpretation of this Assignment or any amendments or exhibits
          hereto.

       r) Applicable Law/Venue.  This Assignment shall be construed in
          accordance with and governed by the laws of the State of Florida,
          without regard to choice of law provisions. Further, the venue for any
          action brought to enforce any of the provisions hereof shall be in a
          state court of competent jurisdiction in Pinellas County, Florida and
          any action commenced in any other forum may be removed to a state
          court of competent jurisdiction in Pinellas County, Florida.

IN WITNESS WHEREOF, the Parties hereto have set their hands and seals, the day
and year first above written.


"Pledgor"


BANKERS UNDERWRITERS, INC.,
a Florida corporation


By: /s/ David B. Snyder
    -----------------------
    Its Vice President
    Assistant Secretary
    (CORPORATE SEAL)


"Lender"


INSURANCE MANAGEMENT SOLUTIONS
GROUP, INC. a Florida corporation


By: /s/ D. M. Howard
    -----------------------
    Its President/CEO
    (CORPORATE SEAL)


                                     6 of 7
<PAGE>
                                  EXHIBIT "A"

                           DESCRIPTION OF COLLATERAL

     All of Pledgor's right, title and interest in and to all of the following
described Collateral, wherever located, and whether now owned or hereafter
acquired, together with all replacements therefor and proceeds (including but
without limitation, insurance policies) thereof:

                                         DESCRIPTION
                                            OF
              PLEDGOR                    COLLATERAL
------------------------------------------------------------------------------
Bankers Underwriters, Inc., a      All accounts and contract rights (as those
Florida corporation                terms are defined by the Uniform Commercial
                                   Code as adopted by the State of Florida)
                                   with insurance agents, including but not
                                   limited to general agency agreements with
                                   respect to the sale of federal flood
                                   insurance.







                                     7 of 7