<PAGE>   1
EXHIBIT 10.4

                             STOCK OPTION AGREEMENT


THIS STOCK OPTION AGREEMENT, made and entered into this 14 day of August, 2001,
between Bankers Insurance Group, Inc., a Florida corporation (hereinafter
"Optionor") and Insurance Management Solutions Group, Inc., a Florida
corporation (hereinafter "Optionee").





                                 R E C I T A L S




1.      Definitions. For purposes of this Agreement the terms listed below,
        wherever capitalized, shall be given the following definitions:

        1.1.    "Affiliate" shall mean any Person who controls, is controlled by
                or is under common control with another Person.

        1.2.    "Credit Line" shall mean the line of credit being established by
                Optionee in favor of Optionor in the amount of five million
                ($5,000,000) dollars and evidenced by Optionor's Master
                Promissory Note (herein, "Note") of even date herewith and in
                like principal amount.

        1.3.    "DOI" shall mean the New York Department of Insurance.

        1.4.    "FCIC" shall mean First Community Insurance Company, a New York
                domestic insurance company with its principal place of business
                located in St. Petersburg, Florida and a wholly owned subsidiary
                of Optionor.

        1.5.    "Loan Documents" shall mean the Note, Credit and Security
                Agreement, Collateral Assignment of Flood Book, Absolute
                Assignment of Flood Book, Further Assurances and Compliance
                Agreement and the within Stock Option Agreement all executed of
                even date herewith and which were executed and delivered to
                create the Credit Line.

        1.6.    "Note" shall mean the Master Promissory Note dated of even date
                herewith to evidence Optionor's obligations under the Credit
                Line.

        1.7.    "Option" shall mean the option to purchase the Option Stock
                granted by OPTIONOR to OPTIONEE hereunder.



1
<PAGE>   2

        1.8.    "Optionee" shall mean Insurance Management Solutions Group,
                Inc., a Florida corporation.

        1.9.    "Optionor" shall mean Bankers Insurance Group, Inc., a Florida
                corporation.

        1.10.   "Option Stock" shall mean the 10,898 shares of the common
                capital stock, $318 par value of FCIC as to which OPTIONEE is
                granted a purchase option hereunder.

        1.11.   "Person" shall mean any individual, corporation, partnership,
                association, joint-stock company, trust, un-incorporated
                organization, joint venture, court or government or political
                subdivision or agency thereof.

        1.12.   "Permitted Encumbrances" shall mean the liens on the Option
                Stock described in Exhibit A which is attached hereto and by
                reference made a part hereof.

        1.13.   "Transaction" shall mean the purchase and sale of the Option
                Stock hereunder and subsequent to the exercise of the Option.

2.      This Stock Option Agreement was required by Optionee as a condition of
        the establishment of the Credit Line.



NOW THEREFORE, FOR AND IN CONSIDERATION of the establishment of the Credit Line
as well as for other good and valuable consideration, it is hereby covenanted
and agreed among the parties hereto as follows:

                               1. GRANT OF OPTION

        1.1.    OPTIONOR hereby grants and conveys unto OPTIONEE or its assigns
                an irrevocable and exclusive option to purchase the Option Stock
                upon all the terms and conditions hereinafter set forth.

        1.2.    The term of this option to purchase shall commence on the day
                and year first above set forth and shall initially extend
                through and be irrevocable until all principal and accrued
                interest due under the Credit Line is paid in full and all
                obligations of Optionor under the Credit and Security Agreement
                have been satisfied in full.

        1.3.    This option shall only be exercisable on or after such time as
                (i) there shall be a default in payment of any sums due under
                the Note for more than 10 days after the date when they shall
                become due or (ii) there shall be any



2
<PAGE>   3

                other default under the Loan Documents if after notice thereof,
                such default has not been cured within thirty days of such
                notice.

        1.4.    Subject to the restrictions set forth in paragraph 1.3 hereof,
                OPTIONEE shall have the right to exercise the Option hereinabove
                granted at any time within the term of this Option by OPTIONEE
                giving written notice of such exercise delivered in person or
                mailed to OPTIONOR. Upon the giving of such notice of the
                exercise, this instrument shall thereupon constitute a firm and
                binding contract of purchase and sale between OPTIONEE and
                OPTIONOR at the price and upon the terms and conditions set
                forth in this Agreement.

                               2. PURCHASE PRICE

        2.1.    The purchase price for said Option Stock shall be the sum of Ten
                ($10.00) per share or One Hundred Eight Thousand Nine Hundred
                Eighty ($108,980) dollars.

        2.2.    The said purchase price of said Option Stock shall be paid in
                cash at the Closing.

                       3. REPRESENTATIONS AND WARRANTIES.

        3.1.    OPTIONOR represents and warrants to OPTIONEE that:

                3.1.1.  OPTIONOR is a corporation duly organized, validly
                        existing and in good standing under the laws of the
                        State of Florida. OPTIONOR has the corporate power and
                        is duly licensed or qualified to the extent required by
                        applicable law to carry on its business as now being
                        conducted and to own, hold and operate its properties
                        and assets.

                3.1.2.  The total authorized capital stock of the FCIC consists
                        of 10,898 shares of common stock, par value $318.00 per
                        share. Of this amount 10,898 shares have been issued and
                        are outstanding. Except for the rights of OPTIONEE with
                        respect to the Option Stock provided for herein and
                        except for the Permitted Encumbrances, OPTIONOR has no
                        stock appreciation rights options, warrants, rights,
                        agreements, understandings or commitments of any kind
                        entitling any person or persons to purchase, subscribe
                        for or otherwise acquire, or relating to the voting
                        rights, of any of OPTIONOR's common capital stock.

                3.1.3.  The execution, delivery and performance of this
                        Agreement by the OPTIONOR, and the consummation of the
                        transactions contemplated by this Agreement, will not
                        constitute a breach, violation or default, create a
                        lien, or give rise to any right of



3
<PAGE>   4

                        termination, cancellation, prepayment or acceleration,
                        under the certificate of incorporation or by-laws of the
                        OPTIONOR, or under any law, rule or regulation or any
                        judgment, decree, order, governmental permit or license,
                        or any note, mortgage, indenture, deed of trust,
                        license, lease, agreement or other instrument or
                        obligation to which OPTIONOR is a party or by which it
                        may be bound or affected.

                3.1.4.  Optionor is the legal owner and holder of the Option
                        Stock.COVENANTS OF OPTIONOR

                            4. COVENANTS OF OPTIONOR

        4.1.    From and after the date of the execution of this Agreement and
                throughout the term hereof:

                4.1.1.  OPTIONOR agrees that it shall not, without the consent
                        of OPTIONEE permit FCIC to:

                        4.1.1.1. Reorganize its capital structure;

                        4.1.1.2. Merge or consolidate with any other corporation
                                 or sell any of its assets except in the
                                 ordinary course of business; or

                        4.1.1.3. Issue any additional shares of stock.

                4.1.2.  OPTIONEE shall have the right to examine the books and
                        records of OPTIONOR and FCIC from time to time and
                        receive copies of all accounting reports and tax returns
                        prepared for, or on behalf of, FCIC subject to any DOI
                        policies governing dissemination of examination reports
                        to unaffiliated parties.

                4.1.3.  As soon as practicable, and in any event within twenty
                        (20) days after the end of each calendar month, furnish
                        to OPTIONEE a monthly unaudited financial statement of
                        OPTIONOR and FCIC and each of its subsidiaries,
                        including balance sheets and income statements, for the
                        calendar month just ended, and for the calendar year to
                        date; and

                4.1.4.  As soon as practicable, and in any event within 180 days
                        after the end of each fiscal year, furnish to OPTIONEE
                        the annual audit report of OPTIONOR and FCIC and each of
                        its subsidiaries, certified without material
                        qualification by independent certified public
                        accountants selected by OPTIONOR and acceptable to
                        OPTIONEE, prepared in accordance with generally accepted
                        accounting principles and practices applied on a basis
                        consistently maintained throughout the period involved,
                        together with relevant



4
<PAGE>   5

                        financial statements of Borrower for the twelve
                        (12)-month period just ended.

                                  5. CLOSING.

        5.1.    After delivery of the notice of intent to exercise the option
                pursuant to Section 1.4 hereof, and prior to the closing
                hereunder, OPTIONEE shall file with the DOI its completed
                application for approval of the Transaction, and OPTIONEE and
                OPTIONOR shall each use its good faith efforts to promptly
                obtain the written consent or approval of each person whose
                consent or approval is required to consummate the purchase and
                sale of the Option Stock, including all required DOI approvals.

        5.2.    OPTIONEE and OPTIONOR shall consummate and close the sale
                contemplated by this Agreement on or before the fifteenth
                calendar day following the date of the receipt by OPTIONEE of
                all required approvals from the DOI for the consummation of the
                Transaction. The closing shall occur at such time, date and
                location in St. Petersburg, Florida as is mutually agreed upon
                by OPTIONEE and OPTIONOR.

        5.3.    At closing hereunder, said Option Stock shall be conveyed free
                and clear of all liens and encumbrances other than the Permitted
                Encumbrances. Optionor has delivered to Optionee, simultaneously
                upon the execution hereof, stock powers and letters of
                instruction to the lenders holding first lien security interests
                in the Option Stock and otherwise disclosed in Exhibit A. Such
                powers and letters are to be held by Optionee and not used or
                delivered until the exercise of, and closing under, this Option.

        5.4.    Optionor shall make, execute and deliver such additional
                instruments as Optionee may reasonably request to convey the
                Optionor's interest in the Option Stock.

        5.5.    Optionor irrevocably nominates, constitutes and appoints
                Optionee its true and lawful attorney in fact, with full power
                of substitution and revocation for it, in its name, place and
                stead and either in the name of Optionee or in the name of
                Optionor to make, execute and deliver such stock powers, and
                other instruments or agreements as may be necessary or
                appropriate to close on the purchase of the Option Stock in
                accordance with the terms of this Agreement and to transfer the
                title to such stock from the name of the Optionor to the name of
                the Optionee or its assigns. All rights, powers and authority of
                said attorney-in-fact to exercise any and all rights and powers
                herein granted shall commence and be in full force and effect as
                of the date of this Agreement and such rights, powers and
                authority shall remain in full force and effect thereafter until
                the termination of this Agreement. However, Optionee shall not
                deliver the Absolute Assignment or otherwise exercise its rights
                under this Power of Attorney until there is an occurrence of an
                Event of Default hereunder. The power set forth herein is a
                power coupled with an interest.


5
<PAGE>   6

                               6. BOARD APPROVAL



        6.1.    This Agreement shall not be effective to bind the parties unless
                and until the Credit Line and the Loan Documents have been
                approved by the Optionee's Board of Directors and Audit
                Committee.

                               7. MISCELLANEOUS.

        7.1.    Attorney's Fees. If either of the parties hereto should bring a
                Court action alleging breach of this Agreement or seeking to
                enforce, rescind, renounce, declare void or terminate this
                Agreement or any provisions thereof, the prevailing party shall
                be entitled to recover all of its legal expenses, including
                reasonable attorney's fees and costs (including legal expenses
                for any appeals taken), and to have the same awarded as part of
                the judgment in the proceeding in which such legal expenses and
                attorney's fees were incurred.

        7.2.    Benefits. This Agreement shall be freely assignable and shall be
                binding upon the parties, their heirs, legal representatives,
                successors and assigns.

        7.3.    Captions. The paragraph captions as to contents of the
                particular paragraphs herein are inserted only for convenience
                and are in no way to be construed as part of this Agreement or
                as a limitation of the scope of the particular paragraph in
                which they are referred.

        7.4.    Construction of Agreement. Words of a gender used in this
                Agreement shall be held to include any other gender, the words
                in a singular number held to include the plural, when the
                sentence so requires.

        7.5.    Counterparts. This Agreement may be executed in several
                counterparts, each of which so executed shall be deemed to be an
                original, and said counterparts shall together constitute and be
                one and the same instrument.

        7.6.    Entire Agreement. This Agreement contains all of the oral and/or
                previously written agreements, representations, and arrangements
                between the parties hereto with respect to the subject matter
                hereof, and all right which the respective parties may have had
                with respect to the subject matter hereof under any written
                agreements and/or oral agreements are hereby canceled and
                terminated, and all parties agree that there are no
                representations or warranties other than those set forth herein.

        7.7.    Invalidation. Should any part of this Agreement for any reason
                be declared invalid, such decision shall not effect the validity
                of any remaining portion, which remaining portion shall remain
                in full force and



6
<PAGE>   7

                effect as if this Agreement had been executed with the invalid
                portion thereof eliminated. It is, therefore, declared the
                intention of the parties hereto that each of them will have
                executed the remaining portion of this Agreement without
                including therein any such part, parts or portion which may, for
                any reason, be hereafter declared void.

        7.8.    Modification. No change or modification of this Agreement shall
                be valid unless the same shall be in writing and signed by each
                of the parties hereto.

        7.9.    Applicable Law/Venue. This Agreement shall be construed in
                accordance with and governed by the laws of the State of
                Florida, without regard to choice of law provisions. Further,
                the venue for any action brought to enforce any of the
                provisions hereof shall be in a state court of competent
                jurisdiction in Pinellas County, Florida and any action
                commenced in any other forum may be removed to a state court of
                competent jurisdiction in Pinellas County, Florida.

        7.10.   Venue. The venue for any action brought to enforce the terms and
                conditions of this Agreement shall be any court of competent
                jurisdiction located in Pinellas County, Florida.


IN WITNESS WHEREOF, the parties have set their hands and seals hereunto and have
caused this Agreement to be executed in their names the day and year first above
written.

WITNESSES:                            Bankers Insurance Group, Inc.


  s/s Lisa M. Powell
  ------------------




  s/s Harold David Holland            BY:      s/s G. Kristin Delano
  ------------------------               --------------------------------------
                                      As Its:  Secretary
                                             ----------------------------------




WITNESSES:                            Insurance Management Solutions
                                      Group, Inc.

  s/s Harold David Holland
  ------------------------

  s/s Lisa Powell                     BY:      Richard G. Torra
  ---------------                         -------------------------------------
                                      As Its:  Corporate Secretary
                                             ----------------------------------




Exhibits: A        Permitted Encumbrances


7
<PAGE>   8



                                    EXHIBIT A

                             PERMITTED ENCUMBRANCES

Bankers Insurance Group - Western International Insurance Company -
Bonded Builders

                Loan:                $9,000,000.00 / Advanced $8,600,000.00
                Current Balance:     $8,426,940.65
                Lender:              Western International Insurance Company
                Borrower:            Bankers Insurance Group, Inc.
                Re:                  Financing for Bonded Builders acquisition
                Issue Date           April 16, 2000
                Maturity Date:       May 1, 2008
                Collateral
                Stock Pledged:       FCIC 3,023 shares owned by BIG
                                     BSIC 75,000 shares owned by BIG
                                     BHWA 1,000 shares owned by BIG

SouthTrust - Bankers Insurance Group, Inc.

                Loan:                $6,000,000.00
                Current Balance:     $3,372,305.40
                Lender:              SouthTrust
                Borrower:            BIG
                Issue Date           October 1, 1993
                Amended              December 20, 1997
                Maturity Date:       December 30, 2002
                Stock Pledged:       2,008,667 shares of BIG stock owned by BFC
                                     7,875 shares of FCIC shares owned by BIG


BFC means Bankers Financial Corporation, a Florida corporation
BIG means Bankers Insurance Group, Inc.
BHWA means Bankers Home Warranty Association, a Florida corporation
BSIC means Bankers Security Insurance Company, a Florida insurance company
FCIC mean First Community Insurance Company, a New York insurance company


8