<PAGE>   1


                                                                   EXHIBIT 10.17

                              SETTLEMENT AGREEMENT

         WHEREAS, subsequent to the merger of the John Vann Company with
Rushmore Financial Group, Inc. ("RFGI") in 1999, John A. Vann ("Vann") asserted
claims against RFGI and its board members (the "Board") in 1999-2000 arising out
of RFGI's representations leading to the merger, business activities and the
business decisions of the Board; and

         WHEREAS, bona fide disputes and controversies exist between the
parties, both as to liability and the amount thereof, if any, and by reason of
such disputes and controversies, the parties hereto desire to compromise and
settle all claims and causes of action of any kind whatsoever which Vann has
arising out of the pre-merger representations by RFGI, the business activities
of RFGI, and the business decisions made by the Board.

         NOW, THEREFORE, for and in consideration of the recitals set forth
above and the execution of this Settlement Agreement, the parties hereto agree
to the following:

         1.       Effective May 31, 2000, unless RFGI shall give notice to Vann
                  before 5:00 p.m. Dallas time, on June 1, 2000, that all of the
                  terms set forth in the Term Sheet dated 5/17/00 (attached
                  hereto as Exhibit "A") have not been approved by the Board of
                  RFGI and an officer of Spectrum Insurance ("Spectrum") by such
                  time, the parties agree that the business of Rushmore
                  Investors Advisors, Inc. ("RIA"), a wholly owned subsidiary of
                  RFGI, will be operated in such a way that:

                  (a). Vann will serve as RIA's chairman, CEO, and sole
                  director, and shall be granted full authority to manage the
                  business and operations of RIA, subject to Vann's and RIA's
                  continuing to conduct its operations in compliance with all
                  laws applicable to RFGI, as they may apply to RIA, and RIA,
                  including all federal and state securities and tax laws.

                  (b) RIA will retain possession and control over its own cash
                  flow, including but not limited to RIA's having complete
                  responsibility for the disposition of all cash receipts and
                  cash disbursements.

                  (c) To the extent RIA uses any of the services provided by
                  either RFGI or any RFGI subsidiary, then RFGI (or its
                  subsidiary) shall send an invoice at the then competitive
                  market rate charges for such services to RIA for such
                  services, and RIA shall pay in accordance with the terms of
                  such invoice.

                  (d) RIA shall pay for RIA's pro rata share of the cost of the
                  annual audit for RFGI to be performed by RFGI's independent
                  public accountants, Grant Thornton.

                  (e) RFGI shall arrange for the physical delivery to RIA of all
                  the computer hardware/software (including website and Advent
                  Link) and other assets, which are currently being used for the
                  business of RIA.


<PAGE>   2


                  (f) Vann will provide to RFGI, by the twelfth business day of
                  each month, all financial information regarding RIA's prior
                  month's activities that is reasonably necessary for RFGI to
                  prepare accurate financial statements for RIA in accordance
                  with GAAP and applicable regulatory principles. Such
                  information will include, but is not limited to, cash receipts
                  registers, cash disbursement registers, and statistical
                  information regarding assets under management.

                  (g) Vann will be the custodian of all RIA financial records
                  that relate to the time period for which Vann has
                  responsibility for RIA's own cash receipts and disbursements.
                  Vann agrees to cooperate with requests by governmental
                  authorities for such records, and agrees to maintain custody
                  of these records, for a time period encompassing the time
                  period for which the Federal Income Tax Return for the year
                  2000 is subject to audit by the Internal Revenue Service.

                  (h) Until the triggering even occurs (as described in either
                  Paragraphs 2 or 3 of this Settlement Agreement) RFGI cannot
                  terminate Vann from his position with RIA, except for good
                  cause. In the event of the perceived existence of "good cause"
                  grounds for termination or resignation, the party seeking to
                  terminate or resign shall give written notice to the other
                  party of the perceived grounds for "good cause", and the other
                  party receiving the notice shall have until March 31, 2001, to
                  cure such "good cause" grounds. Failure to cure on a timely
                  basis shall result in the termination or resignation. If such
                  termination or resignation occurs, and the parties are unable
                  to agree on whether or not it was appropriate, then the
                  dispute will be submitted to mediation in accordance with
                  Paragraph 9 of this Settlement Agreement.

                  2. On the first date when the publicly traded common stock of
         RFGI has a closing price of $6.00 (or more) per share, Vann and RFGI
         will exchange their stock, under the terms of which Vann will transfer
         all of his RFGI stock to RFGI, and, simultaneously, RFGI will transfer
         all of the RIA stock to Vann, subject to RFGI's holding such RIA stock
         as collateral under the Loan Agreement and Collateral Transfer and
         Security Agreement dated August 27, 1999, as described and modified in
         Paragraph 7 of this Settlement Agreement. In the event that on the
         first date when the publicly traded common stock of RFGI has a closing
         price of $6.00 (or more), (i.e., the triggering event occurs), and the
         actual closing price is $7.00 (or more) per share, to the extent that
         RFGI records a gain on the stock exchange transaction in accordance
         with GAAP, RFGI will forgive the then outstanding indebtedness
         described in Paragraph 7 of this Settlement Agreement by the amount of
         the gain.

                  3. If the election provided for in Paragraph 2 of this
         Settlement Agreement is not exercised prior to 1/1/01, because of
         RFGI's common stock not reaching the $6.00 (or more) per share closing
         price, then Vann shall have the right to require RFGI to forthwith
         effectuate a tax-free spin-off of RIA whereby RIA becomes a publicly
         traded investment advisory entity under the terms of the spin-off. The
         parties hereto shall utilize their best efforts to effect either the
         transfer or spin-off in the most effective, reasonable, and expedient
         way possible, and in no case shall either party cause such processes to
         be delayed unreasonably. RIA and RFGI shall each be responsible for
         paying one-half of the reasonable legal and


<PAGE>   3


         accounting fees incurred which are associated with the spin-off. At the
         time the spin-off is effectuated, Vann shall own in excess of 50.0% of
         the publicly traded RIA entity, (with the remainder of the stock in the
         new entity being owned by the remaining RFGI shareholders other than
         the RFGI control group ("the RFGI Control Group") whose shares will be
         transferred to Vann to reach the above 50.0% level) and Vann shall
         transfer all of his RFGI stock to the RFGI Control Group. Vann's shares
         in the publicly traded RIA entity after the spin-off shall be held by
         RFGI as collateral under the Loan Agreement and Collateral Transfer and
         Security Agreement dated August 27, 1999, as described and modified in
         Paragraph 7 of this Settlement Agreement.

                  4. Effective 5/31/00, subject to the RFGI board approving this
         Settlement Agreement on 6/1/00, Vann has resigned from his position as
         a member of RFGI's board of directors and from all other executive
         positions he holds for RFGI, and shall deliver resignation letters to
         that effect at the time this Settlement Agreement is executed. Vann
         shall continue to enjoy the same indemnity benefits as enjoyed by all
         other officers and directors of RFGI until such time as the triggering
         event in Paragraphs 2 or 3 occurs.

                  5. Effective 5/31/00, subject to the RFGI board approving this
         Settlement Agreement on 6/1/00, Vann's existing Employment Agreement
         with RFGI and the obligations of Vann set forth in Sections 3.18, 3.21,
         3.25, 3.26, 5.1, 5.2, 5.4, 5.5, 5.6, 5.8, 8.1, 8.4, 8.8, and 8.9 of the
         Merger Agreement, shall be terminated and be of no force or effect. If
         any conflict arises between this Settlement Agreement and the Merger
         Agreement, this Settlement Agreement shall govern and be controlling.
         It is further understood and agreed by the parties to this Settlement
         Agreement that after (i) the triggering event in Paragraph 2 of this
         Settlement Agreement occurs, (ii) the spin-off described in Paragraph 3
         is effectuated, (iii) the first date occurs when RFGI voluntarily seeks
         relief from creditors under any state or federal statute, regulation or
         code, including any state or federal bankruptcy statute, regulation or
         code, or any third-party, person, entity or governmental agency or
         regulatory body files to place RFGI in bankruptcy or receivership under
         any state or federal statute, regulation or code, (and such involuntary
         bankruptcy is not dismissed soon thereafter) or (iv) March 31, 2000,
         whichever date comes first, then Section 5.7 of the Merger Agreement is
         terminated and Vann and/or any entity in which he owns a controlling
         interest is not prohibited from conducting a financial services
         business or organization that engages or participates, directly or
         indirectly, in the sale of insurance, securities or other investment
         products and/or that solicits


<PAGE>   4


         business from any person or entity that is or was a client of RFGI or
         any of RFGI's subsidiaries or affiliates, specifically including
         Rushmore Investment Advisors, Inc. Notwithstanding any of the other
         provisions or language set forth in this Settlement Agreement, should
         RFGI voluntarily seek relief from creditors under any state or federal
         statute, regulation or code, including any state or federal bankruptcy
         statute, regulation or code, or any third-party, person, entity or
         governmental agency or regulatory body files to place RFGI in
         bankruptcy or receivership under any state or federal statute,
         regulation or code, (and such involuntary bankruptcy is not dismissed
         soon thereafter) then the Merger Agreement shall be terminated
         effective immediately and shall be of no force or effect.

                  6. Upon the first to occur of either (a) RFGI's common stock
         reaching $6.00 (or more) per share closing price, as provided in
         Paragraph 2 of this Settlement Agreement, or (b) Vann's electing the
         tax-free spin-off, as provided in Paragraph 3 of this Settlement
         Agreement, then upon the occurrence of the completion of such first
         event and transfer, (A) Vann (on behalf of himself, his heirs, assigns,
         affiliates, predecessors, successors, and any Vann-controlled entity)
         shall release, acquit, and forever discharge RFGI, its agents,
         servants, representatives, attorneys, employees, officers, directors,
         shareholders, successors, assigns, subsidiaries, and affiliates, and
         all persons (including but not limited to Mr. Rusty Moore), natural or
         corporate, in privity with them or any of them, from any and all claims
         or causes of action of any kind whatsoever, at common law, statutory or
         otherwise, which Vann has now existing against RFGI, it being intended
         that Vann will release all claims of any kind which he might have had
         against those hereby released whether previously threatened or asserted
         or not, and (B) RFGI (on behalf of itself, its officers, directors,
         shareholders, assigns, affiliates, predecessors, successors, agents,
         and representatives) shall release, acquit, and forever discharge Vann,
         his heirs, assigns, affiliates, predecessors, successors, agents,
         servants, representatives, attorneys, employees, and any
         Vann-controlled entity, natural or corporate, in privity with them or
         any of them, from any and all claims or causes of action of any kind
         whatsoever, at common law, statutory or otherwise, which RFGI has now
         existing against Vann, it being intended that RFGI will release all
         claims of any kind which it might have had against those hereby
         released whether previously threatened or asserted or not. The granting
         of these mutual releases shall occur so long as RFGI and RIA are each
         in compliance with the terms of this Settlement Agreement and all
         applicable laws and regulations. Until such time as the triggering
         event incurs in connection with Paragraphs 2 or 3 of this


<PAGE>   5


         Settlement Agreement, Vann and RFGI hereby covenant not to institute
         litigation or arbitration against each other or any RFGI agent,
         servants, representatives, attorneys, employees, officers, directors,
         shareholders, successors, assigns, subsidiaries, and affiliates.

                  7. It is acknowledged that Vann has previously obtained a loan
         from RFGI, under a Loan Agreement, Promissory Note, and Collateral
         Transfer and Security Agreement dated August 27, 1999, which loan
         currently has an outstanding principal balance in the amount of
         $270,000.00, plus accrued but unpaid interest in the amount of
         $10,319.18, such that the current total indebtedness owed on the loan
         is $280,319.18. The parties hereby agree that the terms of the Loan
         Agreement, Promissory Note, and Collateral Transfer and Security
         Agreement dated August 27, 1999, are all hereby restructured, such that
         Vann's repayment obligation to RFGI shall require him to pay 9%
         interest on the $280,319.18 current total indebtedness amount, and,
         therefore, shall pay to RFGI each month the amount of $2,102.39 on the
         1st day of the month beginning 7/1/00 until the occurrence of the first
         triggering event in either Paragraph 2 or 3 of this Agreement, and
         after such triggering event, Vann shall make 48 monthly payments to
         RFGI in the amount of $6,975.75 on the 1st day of the month following
         the triggering event, in order to pay off the balance of the
         indebtedness. Until the entirety of the debt is paid off, the
         collateral for the modified loan shall continue to be all of Vann's
         RFGI stock, until such time as the triggering event occurs under
         Paragraphs 2 or 3 of this Settlement Agreement, in which case the
         collateral for the loan shall be substituted, at which time the
         collateral for the unpaid loan shall either be all of the RIA stock or
         the stock in the publicly traded RIA entity after the spin-off, which
         Vann is to receive under said Paragraphs. After the execution of this
         Settlement Agreement, RFGI shall deliver the collateral to Will Pryor,
         as RFGI's trustee or escrow agent under the Collateral Transfer and
         Security Agreement, (or any other neutral who is mutually agreeable to
         Mike McCauley and Talmage Boston), for such trustee to hold until (i)
         Vann has defaulted on his payment obligations under the loan, been
         given notice of default and failed to cure such default, or (ii) all
         indebtedness owed on the loan has been paid in full. Upon the
         occurrence of either of these two events, the trustee shall deliver the
         collateral to RFGI in the event of default, or shall deliver the
         collateral to Vann in the event of payoff. RIA shall be responsible for
         the payment of all fees and costs charged or incurred by the trustee or
         escrow agent.


<PAGE>   6


                  8. Until such time as the occurrence of the first triggering
         event in either Paragraph 2 or 3 of the Agreement, Vann shall receive
         monthly compensation from RIA in the minimum amount of $50,000.00 per
         month, with $8,333.33 per month being Vann's salary, and the balance of
         the monthly payment being a bonus and/or additional compensation, which
         payments shall be paid from RIA's cash flow. Any amounts paid to Vann
         in excess of $50,000.00 per month shall require the approval of RFGI's
         Compensation Committee, which approval shall not be unreasonably
         withheld. In the event Vann believes that RFGI's Compensation Committee
         is unreasonably withholding consent to Vann's request for compensation
         in excess of $50,000.00 per month, then such dispute shall be mediated
         in accordance with Paragraph 9 of this Settlement Agreement.

                  9. In the event of a dispute between the parties arising from
         or relating to this Settlement Agreement, they hereby agree to a delay
         of four weeks before instituting any litigation or arbitration so that
         such dispute may be mediated by Mr. Will Pryor before the pursuit of
         any litigation or arbitration. In the event Mr. Pryor is unavailable
         within two (2) weeks after notice of the dispute, then the parties will
         attempt to mediate with a mutually agreeable mediator but only if it
         can be done within four weeks.

                  10. This Settlement Agreement may be executed in multiple
         counterparts, each of which will be deemed an original, and all of
         which constitute one and the same instrument.

                  11. This Settlement Agreement will be governed by, enforced,
         and interpreted in accordance with the laws of the State of Texas.

                  12. The failure of any party to enforce any of its rights or
         remedies under this Settlement Agreement with respect to any specific
         act or failure to act will not constitute a waiver of the right to such
         party to enforce such rights and remedies with respect to any other or
         subsequent or failure to act.

                  13. Neither this Settlement Agreement nor any provision of it
         may be changed, waived, discharged, or terminated orally, but only by
         an agreement in writing signed by the party against which the
         enforcement of such change, waiver, discharge, or termination is
         sought.

                  14. This Settlement Agreement constitutes the entire agreement
         between the parties with respect to the subject matter of this
         Settlement Agreement and there are no representations, understandings,
         or agreements relative to this Settlement Agreement which are not fully
         expressed in it.


<PAGE>   7


                  15. Each party warrants and represents to the other party
         that, before the execution of this Settlement Agreement, it has not
         transferred, conveyed, pledged, or in any manner assigned any of the
         claims, demands, actions, cause of action, suits, costs, losses,
         damages, expenses, or liabilities released herein.

                  16. Each party agrees to execute and deliver such documents
         and take such other action as will be reasonably required to carry out
         the provisions of this Settlement Agreement.

                  17. Any notice required to be given or made pursuant to this
         Settlement Agreement will be in writing and will be deemed properly and
         sufficiently given or made if delivered in person or if sent by
         registered or certified mail, return receipt requested, or by courier
         receipt of delivery or by facsimile transmission to the other party at
         the address or telecopier number set forth below:

                  If to John Vann, address to John Vann, 1424 Gables Court,
                  Plano, Texas 75075, fax 972/964-1928, with a copy to his
                  attorney Mike McCauley, McCauley MacDonald Devin & Huddleston,
                  3800 Renaissance Tower, 1201 Elm Street, Dallas, Texas 75270,
                  fax (214) 747-0942.

                  If to RFGI, address to Mr. Rusty Moore, Rushmore Financial
                  Group, Inc., 13355 Noel Road, One Galleria Tower, Suite 650,
                  Dallas, Texas 75240, fax (972) 450-6001, with a copy to Mr.
                  Talmage Boston, Winstead Sechrest & Minick, 5400 Renaissance
                  Tower, 1201 Elm, Dallas, Texas 75270, fax (214) 745-5859.

                  18. The person, address, or telecopier number to which notice
         must be given pursuant to the previous paragraph may be changed at any
         time upon giving two (2) business days' written notice to the other
         party.

                  19. The terms and agreements set forth herein are binding upon
         any successors and/or assigns of RFGI.

         IT IS EXPRESSLY UNDERSTOOD AND AGREED that the terms hereof are
         contractual and are not merely recitals and that the agreements herein
         contained and the consideration transferred are to compromise doubtful
         and disputed claims, avoid further litigation, and buy peace, and that
         no consideration delivered nor releases given shall be construed as an
         admission of liability, all liability being expressly denied.

         Vann and RFGI warrant that they and their respective representatives
         have read this agreement and fully understand it to be a compromise,
         settlement and release of all present, known claims, that Vann has or
         may have against the party or parties released, arising out of the
         matters described. RFGI also warrants that its representative is fully
         and legally competent to execute this agreement, and that he does so of
         his own


<PAGE>   8


         free will and accord without reliance on any representation of any kind
         or character not expressly set forth herein.

         SIGNED this 7th day of June, 2000.



                                           ----------------------------
                                           /s/ JOHN A. VANN


                                           /s/ RUSHMORE FINANCIAL GROUP, INC.
                                           BY:
                                              ----------------------------------
                                           ITS:
                                               ---------------------------------