<PAGE>   1
                                                                    EXHIBIT 10.2


                           COLLATERAL AGENCY AGREEMENT

                  This Collateral Agency Agreement (this "AGREEMENT") is made
and entered into as of May 14, 2001 by and among (i) BGLS Inc., a Delaware
corporation (the "COMPANY"), (ii) Brooke Group Holding Inc., a Delaware
corporation ("BROOKE HOLDING"), (iii) Vector Group Ltd., a Delaware corporation
("VECTOR"), (iv) New Valley Holdings, Inc., a Delaware corporation ("NV
HOLDINGS"), (parties (i) through (iv) the "COMPANY PARTIES"), (v) United States
Trust Company of New York, a New York banking corporation ("U.S. TRUST") as
collateral agent (the "COLLATERAL AGENT") appointed pursuant to this Agreement
with reference to that certain Note Purchase Agreement (the "NOTE PURCHASE
AGREEMENT") dated as of even date herewith under which the Company shall issue
to the purchasers (the "PURCHASERS") $60,000,000 in aggregate principal amount
of the Company's 10% Senior Secured Notes due March 31, 2006 (the "NOTES"), (vi)
the Purchasers under the Note Purchase Agreement and (vii) such other parties
who may become a party pursuant to SECTION 7.13 hereof. Defined terms appear in
SECTION 1.

                  WHEREAS, the obligations of the Company under the Notes and
the Note Purchase Agreement are secured by the Security Agreements;

                  WHEREAS, pursuant to the Security Agreements, the Company
Parties have granted to the Collateral Agent, for the benefit of the holders of
the Notes (the "HOLDERS"), a security interest in and liens on the Collateral
(as defined in the Note Purchase Agreement); and

                  WHEREAS, it is a condition precedent to the consummation of
the transactions contemplated by the Note Purchase Agreement that this Agreement
shall have been duly executed and delivered.

                  NOW THEREFORE, in consideration of the foregoing and the
mutual agreements contained herein, and for other good and valuable
consideration, the receipt and sufficiency of which are hereby acknowledged, the
parties hereto agree as follows:

         Section 1. TERMS AND DEFINITIONS.

                  1.01 DEFINITIONS. Capitalized terms used but not defined
herein have the respective meanings given to such terms in the Note Purchase
Agreement. As used in this Agreement, the following terms shall have the
following respective meanings:

                  "DEFAULT" and "EVENT OF DEFAULT" shall have the respective
meanings given to such terms in the Note Purchase Agreement.

                  "PERMITTED SECURITIES" has the meaning ascribed thereto in the
Account Control Agreement.

                  "RESPONSIBLE OFFICER" when used with respect to the Collateral
Agent, means any officer within the Corporate Trust Administration of the
Collateral Agent (or any successor group of the Collateral Agent) with direct
responsibility for the administration of the Note Documents and also means, with



<PAGE>   2

respect to a particular corporate trust matter, any other officer to whom such
matter is referred, but only for so long as it is referred, because of his
knowledge of and familiarity with the particular subject.

                  "SECURED OBLIGATIONS" shall have the meaning given to such
term in the Pledge Agreements.

                  1.02 INTERPRETATION.

                  (a) Unless the context otherwise indicates, words expressed in
the singular shall include the plural and vice versa.

                  (b) Headings of sections herein are solely for convenience of
reference, do not constitute a part hereof and shall not affect the meaning,
construction or effect hereof.

                  (c) The words "herein," "hereof," "hereby," "hereunder," and
other words of similar import refer to this Agreement as a whole and not to any
particular Section or subdivision hereof.

                  (d) Unless the context otherwise indicates, any reference to a
"Section" or "Schedule" refers to a Section of or a Schedule to this Agreement,
respectively.

                  1.03 HOLDERS' ACTION OR INSTRUCTIONS. Any action permitted to
be taken by the Holders under this Agreement, or any direction that the Holders
may give to the Collateral Agent, may be taken or given, as the case may be,
only by the Majority Holders unless specifically stated otherwise herein.

         Section 2. APPOINTMENT, COMPENSATION AND AUTHORIZATION OF THE
                    COLLATERAL AGENT.

                  2.01 APPOINTMENT OF THE COLLATERAL AGENT. U.S. Trust is hereby
appointed as Collateral Agent for the Holders and is irrevocably authorized and
empowered to (i) hold the Collateral for the benefit of the Holders, (ii)
exercise such authority, rights, powers, and duties hereunder as specifically
are delegated to and accepted by the Collateral Agent hereunder and (iii) take
such other action in connection with the foregoing as the Holders may from time
to time direct in accordance with the terms and conditions of this Agreement,
the Note Purchase Agreement and the Security Agreements. U.S. Trust hereby
accepts its appointment as Collateral Agent and agrees to perform the duties of
the Collateral Agent specified herein, in the Note Purchase Agreement and in the
Security Agreements and to exercise the powers granted hereby and thereby, in
either case in accordance with the terms hereof or thereof, as the case may be.

                  2.02 COMPENSATION OF THE COLLATERAL AGENT. As compensation for
its services as Collateral Agent, the Company agrees to pay or cause to be paid
the Collateral Agent on the Closing Date and, so long as this Agreement remains
in effect, annually on the first day of April, the sum agreed between the
Company and the Collateral Agent in writing and all the reasonable fees, costs
and expenses incurred in good faith by the Collateral Agent (including, without
limitation, the reasonable fees and disbursements of its counsel and other
advisers as the Collateral Agent reasonably elects to retain) (i) arising in
connection with the preparation, execution, delivery, performance, modification
and termination of this Agreement and the Security Agreements in good faith in
connection with the administration of the Collateral, the sale or other
disposition thereof pursuant to Security Agreements and the preservation,
protection, defense or enforcement of the Collateral Agent's rights under the


                                      -2-
<PAGE>   3


Security Agreements and in and to the Collateral or (ii) incurred in good faith
by the Collateral Agent in connection with the resignation or removal of the
Collateral Agent pursuant to SECTION 5.01. Additionally, the Company agrees to
(A) indemnify and hold harmless the Collateral Agent from any present or future
claim or liability for any stamp or other similar tax and any penalties or
interest with respect thereto, that may be assessed, levied or collected by any
jurisdiction in connection with this Agreement, the Note Purchase Agreement, the
Security Agreements or any Collateral and (B) pay or to reimburse the Collateral
Agent for any and all amounts in respect of all search, filing, recording and
registration fees, taxes, excise taxes and other similar imposts that may be
payable or determined to be payable in respect of the execution, delivery,
filing, performance and enforcement of this Agreement, the Note Purchase
Agreement, the Security Agreements and all documents (including, without
limitation, financing statements) provided for herein or therein.

                  2.03 AUTHORIZATION OF SECURITY AGREEMENTS.

                  The Purchasers hereby authorize the Collateral Agent's
entering into the Security Agreements on their behalf.

                  2.04 SECURITIES ACCOUNT NOTICES.

                  (a) The Collateral Agent shall deliver a Notice of Sole
Control promptly after receiving written instructions from the Majority Holders
to deliver such Notice of Sole Control. The Collateral Agent shall not deliver a
Notice of Sole Control absent delivery by the Majority Holders of such written
instructions.

                  (b) The Collateral Agent shall deliver a Notice of End of Sole
Control promptly after receiving the written instructions of the Majority
Holders to deliver such Notice of End of Sole Control. The Collateral Agent
shall not deliver a Notice of End of Sole Control absent delivery by the
Majority Holders of such written instructions.

                  (c) During the continuance of an Account Notice Event, the
Majority Holders shall provide the Collateral Agent with written instructions as
to Permitted Securities in which the balance of the funds on deposit in the
Securities Account shall be invested.

                  (d) During the continuance of an Account Notice Event, if the
Majority Holders deliver written instructions to the Collateral Agent for the
Collateral Agent to withdraw a specified amount of funds from the Securities
Account and deliver such funds to the Company in exchange for an Officer's
Certificate containing information to be specified by the Majority Holders, the
Collateral Agent shall withdraw such specified amount of funds from the
Securities Account and deliver such specified amount of funds to the Company in
exchange for an Officer's Certificate containing the specified information.



                                      -3-
<PAGE>   4

                  2.05 WITHDRAWALS FROM SECURITIES ACCOUNT DURING ACCOUNT NOTICE
EVENT.

                  In the event of an Account Notice Event, the Majority Holders
may deliver written instructions to the Collateral Agent to make the withdrawals
from the Securities Account specified in Section 13.11 of the Note Purchase
Agreement.

         Section 3. DUTIES, POWERS AND RIGHTS OF THE COLLATERAL AGENT.

                  3.01 SPECIFIC DUTIES OF THE COLLATERAL AGENT. The Collateral
Agent shall have the following duties:

                  (a) upon the receipt by it of written instructions of the
Majority Holders, execute and deliver on behalf of the Holders such documents as
the Majority Holders shall deem necessary or appropriate and provide to the
Collateral Agent from time to time to maintain the perfection of any lien in, to
or upon the Collateral or any portion thereof, that has been, are or will be
granted in favor of the Collateral Agent pursuant to the Security Agreements;

                  (b) accept, on behalf of the Holders, any part of the
Collateral delivered to it, including, without limitation, any certificated
securities, instruments and documents, and accept, on behalf of the Holders, any
new Collateral given as security for the Secured Obligations, and execute and
deliver, on behalf of the Holders, such documents or instruments as the Majority
Holders deem necessary or appropriate and provide to the Collateral Agent with
written instructions of the Majority Holders to evidence the creation of any
lien with respect thereto and to perfect such lien;

                  (c) upon the receipt by it of written instructions of the
Majority Holders, release the Collateral or any portion thereof from any liens
thereon that were created pursuant to the Security Agreements;

                  (d) furnish to the Holders, promptly upon receipt thereof,
duplicates of all reports, notices, requests, demands, certificates and other
documents received by it under this Agreement, the Security Agreements, the Note
Purchase Agreement or other documents provided for herein or therein;

                  (e) provide to the Holders a copy of all written notices
received from the Company or any Subsidiary Pledgor with respect to any capital
stock or securities that constitute Collateral and, upon receipt by it of
written instructions of the Majority Holders, exercise all rights and powers
determined by the Majority Holders that are appurtenant to any such capital
stock or securities that become a part of the Collateral, including, without
limitation, the right to vote stock, to receive dividends or other
distributions, and to grant or refrain from granting any consent or waiver, all
in accordance with such written instructions;

                  (f) inform the Holders in writing of the existence of any
Default or Event of Default promptly upon learning of the same; PROVIDED,
HOWEVER, that the Collateral Agent shall not be deemed to have any knowledge
whatsoever of any Default or Event of Default unless a Responsible Officer of
the Collateral Agent has actually received written notice stating that a Default
or an Event of Default has occurred from any of the Holders or the Company
Parties;



                                      -4-
<PAGE>   5

                  (g) upon receipt by it of written instructions of the Majority
Holders, take those actions determined by the Majority Holders as necessary to
protect and preserve the Collateral and realize on and foreclose upon the
Collateral, including, without limitation, initiating (at the expense of the
Company) and defending any and all actions or proceedings in any court of law or
equity that may be brought affecting any of the Collateral or any portion
thereof or otherwise pursue any remedies available to any Holder or to it in
respect of the Collateral or any portion thereof, which actions may include,
without limitation, initiating and conducting any public or private sale or
pursuing any other actions or remedies relating to the Collateral or any portion
thereof; provided, HOWEVER, that the Collateral Agent shall be under no
obligation to exercise any of its rights and powers under this Agreement unless
it shall have received security and indemnity satisfactory to it against any
loss, liability or expense;

                  (h) provide, at the written instructions of the Majority
Holders, notice required by the Note Purchase Agreement or the Security
Agreements, or by law, to the Company, or any other party entitled thereto, in
order to take any actions required or authorized to be taken under this
Agreement or specified in written instructions of the Majority Holders;

                  (i) receive any and all amounts of any kind paid pursuant to
the Security Agreements and receive proceeds of the Collateral subsequent to an
Event of Default and apply such amounts or proceeds as specified in SECTION
4.01; and

                  (j) take, or refrain from taking, such other actions (but only
such actions that are set forth in this Agreement) as the Majority Holders shall
from time to time direct by written instruction; PROVIDED, HOWEVER, that the
Collateral Agent may, in its sole discretion, refrain from taking such action
(other than an action required or necessary to discharge any duty under SECTION
5.01 below) if the taking of such action would expose it to liability, financial
or otherwise for which it does not receive adequate protection.

                  3.02 DUTIES LIMITED.

                  (a) The Collateral Agent shall be obligated to perform such
duties and only such duties as specifically set forth in this Agreement and no
implied covenants or obligations shall be read into this Agreement against the
Collateral Agent. The Collateral Agent shall be obligated to take any actions or
exercise any rights, powers or remedies which are discretionary with the
Collateral Agent under this Agreement only as may be specified in a written
notice from the Majority Holders; PROVIDED, HOWEVER, that the Collateral Agent
shall not be required to take any actions specified in a written notice if the
provisions of this Agreement expressly prohibit such action. The Collateral
Agent shall be deemed to have exercised reasonable care in the custody and
preservation of the Collateral in its possession if the Collateral is accorded
treatment substantially equal to that which the Collateral Agent accords to
similar property held by it as collateral agent or in a comparable capacity, it
being understood that the Collateral Agent shall not have any responsibility for
(a) ascertaining or taking any action with respect to calls, conversions,


                                      -5-
<PAGE>   6

exchanges, maturities or similar matters relative to any property held as
Collateral, whether or not the Collateral Agent has or is deemed to have
knowledge or notice of such matters or (b) taking any steps to maintain the
value of any property held as Collateral or to preserve rights against any
parties with respect thereto. Except as expressly provided herein or in the Note
Purchase Agreement or the Security Agreements, the Collateral Agent shall not
have any duty or obligation, express or implied, to:

                  (i) manage, control, use, maintain, sell, dispose of,
         purchase, bid for or otherwise deal with the Collateral or any portion
         thereof, or to otherwise take or refrain from taking any action under,
         or in connection with this Agreement, the Note Purchase Agreement or
         any Security Agreement, except to the extent required by law;

                  (ii) take any action that relates to, materially affects, or
         impairs the amounts that the Holders may recover from disposition of
         the Collateral, including, without limitation, any election or waiver
         of remedies available under the Security Agreements, or with respect to
         the Collateral or the manner of foreclosure upon the same; any
         determination of the order and timing of foreclosure upon any portion
         of the Collateral or of the amount of any credit bid to be entered at
         any public or private, judicial, or nonjudicial sale of the Collateral;
         the pursuit of any remedies against the Company or any of its
         Subsidiaries following the completion of foreclosure upon the
         Collateral; the compromise or settlement of any claims against the
         Company or any of its Subsidiaries, including without limitation the
         conduct of any negotiations relating to the same or with a view toward
         the termination of any pending foreclosure proceedings;

                  (iii) obtain or maintain insurance on the Collateral or any
         other insurance;

                  (iv) pay or discharge any tax, assessment or other
         governmental charge or any lien or encumbrance of any kind owing with
         respect to, or assessed or levied against, any part of the Collateral;

                  (v) take any action or omit to take any action provided for in
         the Security Agreements;

                  (vi) advance any monies for any purpose;

                  (vii) except at the specific written instructions of the
         Majority Holders, record or file the Security Agreements, any other
         document or any other instrument provided to it referred to herein or
         therein with respect to any lien; or

                  (viii) except as provided in Section 2.04, deliver a Notice of
         Sole Control or a Notice of End of Sole Control.

                  (b) In addition to and not in limitation of the provisions of
SECTION 3.02(A), under no circumstances shall the Collateral Agent have any duty
or obligation to take any actions hereunder other than those under Section 5.01,
even if instructed to do so by the Majority Holders or if expressly set forth
herein, if the Collateral Agent determines, in its sole and absolute discretion,
that such actions would subject it to liability or expense for which
satisfactory indemnity to the Collateral Agent has not been provided hereunder
or otherwise.



                                      -6-
<PAGE>   7

                  (c) Except as otherwise provided herein at the written
instruction of the Majority Holders, the Collateral Agent shall have no
obligation or liability in respect of the recording, rerecording, filing or
refiling of any instruments, documents, financing statements or continuation
statements or to take any other action hereunder with respect to the security
interests created hereby or pursuant to the Security Agreements, and the
Collateral Agent shall have no obligation to monitor the status of the security
interests as a perfected security interest created hereunder or under the
Security Agreements.

                  3.03 SPECIFIC POWERS OF THE COLLATERAL AGENT. In addition to
all powers necessary, appropriate, desirable or incidental to the Collateral
Agent's performance of the specific duties set forth in SECTION 3.01, the
Collateral Agent is hereby empowered and authorized to do, in its sole and
absolute discretion, any and all of the following in connection with its
performance of such duties; PROVIDED, HOWEVER, that in no event shall it have
any obligation to do so:

                  (a) establish bank accounts in its name with the right to be
the only party authorized to draw from such account or accounts;

                  (b) employ such persons, firms or professionals as it shall
reasonably deem appropriate or desirable in connection with the performance of
its duties hereunder, including, without limitation, appraisers, auctioneers,
stockbrokers, custodians of securities, fiduciaries, commercial banks,
investment banks, accountants and attorneys; and

                  (c) execute and deliver, as Collateral Agent and on behalf of
the Majority Holders, any agreements, escrow instructions, bills of sale,
releases, applications or any other documents related to or in any way connected
with any disposition of the Collateral, or any portion thereof, permitted under
this Agreement or directed by the Majority Holders in accordance with the terms
hereof; PROVIDED, HOWEVER, that in the event it is unwilling or unable for any
reason to execute and deliver such documents, then it promptly shall notify the
Holders of such unwillingness or inability and shall request execution and
delivery of such documents by the Holders.

                  3.04 WRITTEN INSTRUCTIONS. Any written request or written
instructions required or permitted to be given hereunder to the Collateral Agent
shall be given exclusively by the Majority Holders. In the event that the
Collateral Agent receives written instructions from the Majority Holders, that
the Collateral Agent determines, in its sole and absolute discretion, to be
ambiguous, inconsistent, in conflict with other instructions previously received
or otherwise insufficient to direct the actions of the Collateral Agent, then
the Collateral Agent shall have no obligation whatsoever to take or refrain from
taking any action pursuant to such written instructions, but shall instead do
the following:

                  (a) FIRST, seek additional written instructions from the
Majority Holders reasonably satisfactory to it; or

                  (b) SECOND, if the Collateral Agent is reasonably dissatisfied
with the further instructions or does not receive further instructions pursuant
to SECTION 3.04(A), resign as Collateral Agent in accordance with this
Agreement.



                                      -7-
<PAGE>   8


The Collateral Agent shall not be liable to any party hereto (or any Person
claiming by, through or under such party) by reason of its actions under this
SECTION 3.04.

                  3.05 RELIANCE. In acting with respect to this Agreement, the
Note Purchase Agreement or the Security Agreements, the Collateral Agent shall
be entitled to rely conclusively:

                  (a) on any communication reasonably believed by it to be
genuine and to have been made, sent or signed by the Person by whom it purports
to have been made, sent or signed;

                  (b) as to any matters of fact that might reasonably be
expected to be within the knowledge of the Holders, the Company Parties or the
Securities Intermediary, on a certificate signed by or on behalf of any of the
Holders, the Company Parties or the Securities Intermediary;

                  (c) on the advice or services of any persons, firms or
professionals employed by it pursuant to SECTION 3.03(B) and rely upon the
opinions and statements of any professional advisor so employed; and

                  (d) on any resolution, statement, certificate, instrument,
opinion, report, notice, request, consent, order, bond or other paper or
document that it reasonably believes to be genuine and to have been signed or
presented by the proper person or, in the case of cables, facsimile
transmissions, telecopies and telexes, to have been sent by the proper person.
The Collateral Agent shall not be responsible, may conclusively rely upon and
shall be protected, indemnified and held harmless by the Company Parties for the
sufficiency or accuracy of the form of, or the execution, validity, value or
genuineness of any document or property received, held or delivered by it
hereunder, or the signature or endorsement thereon, or for any description
therein, nor shall the Collateral Agent be responsible or liable in any respect
on account of the identity, authority or rights of the Persons executing or
delivering or purporting to execute or deliver any document, property or this
Agreement absent the gross negligence or willful misconduct of the Collateral
Agent.

                  The Collateral Agent shall not be liable to any party hereto
for any consequence of any such relying, acting, or refraining to act. Nothing
in this SECTION 3.05 shall impair the right of the Collateral Agent in its
discretion to take or omit to take any action that the Collateral Agent deems
proper to take or omit to take if such action or omission is not inconsistent
with any notice or direction from the Majority Holders; and that the Collateral
Agent shall not be under any obligation to take any action that is discretionary
to the Collateral Agent under this Agreement, the Note Purchase Agreement or the
Security Agreements except as may be specified in a written instruction of the
Majority Holders.

                  3.06 NO RESPONSIBILITY. The Collateral Agent does not assume
any responsibility for:

                  (a) any failure or delay in performance or breach by Vector,
the Company or any Subsidiaries of the Company of any of their respective
obligations under the Note Purchase Agreement or the Security Agreements or the
Securities Intermediary under the Account Control Agreement;




                                      -8-
<PAGE>   9

                  (b) the truth or accuracy of any representation or warranty or
statement given or made in connection with the Note Purchase Agreement or the
Security Agreements;

                  (c) the legality, validity, effectiveness, adequacy or
enforceability of the Note Purchase Agreement or the Security Agreements or the
priority or perfection of the Security Interest;

                  (d) the validity, enforceability or sufficiency of an
agreement or instrument, the sufficiency of any amount on deposit in any Group
Account to satisfy any obligation of any Company Party, any depreciation of
diminution in the value of any Collateral or any income thereon, or the effect
of any withdrawals or other transactions in the Group Accounts by or at the
direction of any Company Party; or

                  (e) the terms and conditions of any other document, instrument
or agreement in connection herewith, except those agreements to which it is a
party.

                  As to any event or occurrence in which neither the Collateral
Agent nor any Person acting on its behalf is a participant, the Collateral Agent
shall be conclusively presumed to have no knowledge of such event or occurrence,
absent gross negligence or willful misconduct, except to the extent that a
Responsible Officer of the Collateral Agent shall have received a written notice
from any of the Holders or any Company Party with respect thereto.

                  3.07 COLLATERAL AGENT PROTECTED. The Collateral Agent shall be
protected fully in acting or refraining to act upon any certificate, statement,
instrument, opinion, report, notice, request, consent, order, bond or paper or
document reasonably believed by it to be genuine and to have been signed or
presented by the proper party or parties. The Collateral Agent shall not be
liable for any error in judgment made in good faith by the Collateral Agent
unless the Collateral Agent was grossly negligent in ascertaining the pertinent
facts or acted intentionally in bad faith. The Collateral Agent may consult with
legal counsel with significant experience in transactions of the type
contemplated by the Note Purchase Agreement, and the advice of such counsel
shall constitute full and complete protection in respect of any action taken,
suffered or omitted by it under this Agreement in good faith and in accordance
with such advice of counsel. The Collateral Agent may execute any of its powers
hereunder or perform any duties hereunder either directly or through agents,
attorneys or custodians, and the Collateral Agent shall not be responsible for
any misconduct or negligence on the part of, or for the supervision of, any
agent, attorney or custodian appointed with due care by it hereunder; PROVIDED,
HOWEVER, that as between the other parties hereto and the Collateral Agent, all
such powers and duties are those of the Collateral Agent as provided hereunder.

                  3.08 LIMITATION ON LIABILITY.

                  (a) The Collateral Agent may not be relieved from liability
for its own grossly negligent action, its own grossly negligent failure to act,
or its own willful misconduct, except that:

                  (i) the Collateral Agent shall not be liable for any error of
         judgment made in good faith by a Responsible Officer, unless it is
         proved that the Collateral Agent was grossly negligent in ascertaining
         the pertinent facts; and


                                      -9-
<PAGE>   10


                  (ii) the Collateral Agent shall not be liable with respect to
         any action it takes or omits to take in good faith in accordance with a
         direction received by it pursuant to this Agreement or the other Note
         Documents;

                  3.09 LIABILITY FOR MONEY AND INTEREST. The Collateral Agent
shall not be liable for any interest or any money received by it except as the
Collateral Agent may agree in writing. Money held in trust by the Collateral
Agent need not be segregated from other funds except as required by law.

         Section 4. APPLICATION OF PROCEEDS OF COLLATERAL.

                  4.01 APPLICATION OF PROCEEDS OF COLLATERAL. The receipt of any
amounts on behalf of the Holders under the Note Purchase Agreement, the Security
Agreements or otherwise with respect to the Collateral and the proceeds of any
sale, enforcement or other disposition of any of the Collateral or any other
distribution in respect of the Collateral shall be applied by the Holders and
the Collateral Agent first, to the payment of all proper costs incurred by the
Collateral Agent in the collection thereof (including stamp or other taxes in
respect of the transfer or sale of any Collateral and the reasonable
compensation, expenses and the disbursements of the Collateral Agent and its
counselors) and then in accordance with Section 5.13 of the applicable Pledge
Agreement.

         Section 5. RESIGNATION OR REMOVAL OF COLLATERAL AGENT.

                  5.01 RESIGNATION OR REMOVAL OF THE COLLATERAL AGENT. The
Collateral Agent may, by written notice to the Holders, at any time resign its
agency under this Agreement. The Majority Holders may remove the Collateral
Agent by written notice to the Collateral Agent. No such resignation or removal
shall become effective, unless and until a successor Collateral Agent under this
Agreement is appointed and has accepted the appointment, with such successor
Collateral Agent to be appointed by the Majority Holders; PROVIDED, HOWEVER,
that if no successor Collateral Agent shall have been so appointed and shall
have accepted such appointment within thirty (30) days after the retiring
Collateral Agent's giving notice of resignation or after notice to the retiring
Collateral Agent of the retiring Collateral Agent's removal, as the case may be,
then the retiring Collateral Agent may apply to any court of competent
jurisdiction, at the expense of the Company, to appoint a successor Collateral
Agent to act until such time as a successor shall have been appointed by the
Holders. Upon the acceptance of any appointment as Collateral Agent hereunder by
a successor Collateral Agent, such successor Collateral Agent shall thereupon
succeed to and become vested with all the rights, powers, privileges and duties
of the retiring Collateral Agent, and the retiring Collateral Agent shall be
discharged from any further duties and obligations under this Agreement except
the duty to execute and deliver any documents necessary to vest or confirm the
vesting of such rights, powers, privileges, and duties in such successor
Collateral Agent and to deliver possession of any Collateral in the possession
of such retiring Collateral Agent to such successor Collateral Agent so long as
all sums owing to the Collateral Agent have been paid pursuant to Section 2.02.
After the retiring Collateral Agent's resignation or removal hereunder as
Collateral Agent, each reference herein to a place for giving of notice or
deliveries to the Collateral Agent shall be deemed to refer to the principal
office of the successor Collateral Agent or such other office of the successor
Collateral Agent as it may specify to each party hereto.



                                      -10-
<PAGE>   11

         Section 6. INDEMNIFICATION.

                  6.01 INDEMNIFICATION.

                  (a) The Company agrees to pay, indemnify and hold the
Collateral Agent and each director, officer, employee, agent, bailee or other
person acting on behalf of the Collateral Agent, and each stockholder of any
thereof, harmless from and against any and all liabilities, obligations, losses,
damages, penalties, actions, judgments, suits, costs, expenses (including,
without limitation, the reasonable fees and disbursements of counsel and other
advisers) or disbursements of any kind or nature whatsoever with respect to the
execution, delivery, enforcement, performance and administration of this
Agreement, including, without limitation, any amendment hereto, or to the
Security Agreements, or in connection with the transactions contemplated by this
Agreement, the Security Agreements and the Note Purchase Agreement (including
arising from the ordinary negligence of the person seeking indemnification),
unless arising from the gross negligence or willful misconduct of the person
seeking indemnification.

                  (b) In any suit, proceeding or action brought by the
Collateral Agent under or with respect to the Collateral for any sum owing
thereunder, or to enforce any provisions hereof or of the Security Agreements,
the Company shall save, indemnify and keep the Collateral Agent (including its
successors, assigns, directors, officers, employees, agents, bailees or any
other person acting on behalf of the Collateral Agent) and each stockholder of
any thereof and the Holders harmless from and against all expense, loss or
damage suffered by reason of any defense, set-off, counterclaim, recoupment or
reduction of liability whatsoever of the obligee thereunder, arising out of a
breach by the Company of any obligation hereunder or thereunder or arising out
of any other agreement, indebtedness or liability at any time owing to or in
favor of such obligee or its successors from the Company, and all such
obligations of the Company shall be and remain enforceable against and only
against such Person and shall not be enforceable against the Collateral Agent or
the Holders.

                  (c) The obligations of the Company under this SECTION 6.01
shall survive the termination or modification of the other provisions of this
Agreement and shall survive the commencement of a case under any applicable
bankruptcy law on behalf of or against the Company or any other proceeding for
the reorganization, management, adjustment of debt, dissolution or liquidation
on behalf of or against any such Person and shall survive any dissolution of any
such Person and shall survive the resignation or removal of the Collateral
Agent.

         Section 7. MISCELLANEOUS.

                  7.01 NOTICES. All notices and other communications provided
for in this Agreement shall be given or made by facsimile transmission, first
class mail, overnight delivery or personal delivery to the intended recipient to
the address specified below each party's name on the signature pages hereto, or
at such other address, or to the attention of such other officer, as any such
party shall have furnished in writing to each other party. Except as otherwise
provided in this Agreement, all such communications shall be deemed to have been
duly given when transmitted by facsimile transmission, subject to telephone
confirmation of receipt and the provision immediately thereafter of a copy by
first class mail, overnight delivery or personal delivery, or, in the case of a



                                      -11-
<PAGE>   12

mailed notice, when duly deposited in the U.S. mails, first class postage
prepaid, in each case given or addressed as aforesaid or upon actual receipt by
the Collateral Agent.

                  7.02 NO PARTNERSHIP OR JOINT VENTURE. Nothing contained in
this Agreement, and no action taken by the Collateral Agent or the Holders (or
any of them) pursuant hereto, is intended to constitute or shall be deemed to
constitute a partnership, association or joint venture.

                  7.03 COUNTERPARTS. This Agreement may be executed in two or
more counterparts, each of which shall be deemed an original, but all of which
together shall constitute one and the same instrument. Delivery of an executed
counterpart of a signature page of this Agreement by facsimile transmission
shall be effective as delivery of a manually executed counterpart of this
Agreement.

                  7.04 HEADINGS. The section and other headings contained in
this Agreement are for reference purposes only and shall not affect the meaning
or interpretation of this Agreement.

                  7.05 PAYMENTS. All payments hereunder shall be made in U.S.
dollars in immediately available funds.

                  7.06 REMEDIES CUMULATIVE, ETC. Each right, power and remedy
provided in this Agreement for the benefit of the Collateral Agent now or
hereafter existing at law or in equity or by statute or otherwise shall be
cumulative and concurrent and shall be in addition to every other right, power
or remedy provided for in this Agreement or now or hereafter existing at law or
in equity or by statute or otherwise. The exercise or partial exercise by the
Collateral Agent of any one or more of such rights, powers or remedies shall not
preclude the simultaneous or later exercise by the Collateral Agent of all such
other rights, powers or remedies, and no failure or delay on the part of the
Collateral Agent to exercise any such right, power or remedy shall operate as a
waiver thereof.

                  7.07 THE COMPANY'S SECURED OBLIGATIONS ABSOLUTE, ETC. The
obligations of the Company under this Agreement shall be absolute and
unconditional and shall remain in full force and effect without regard to, and
shall not be released, discharged or in any way impaired by any circumstance
whatsoever, including without limitation: (a) any amendment or modification of
the Notes, the Note Purchase Agreement, the Security Agreements or any
instrument provided for herein or therein, or any assignment, transfer or other
disposition of any thereof; (b) any waiver, consent, extension, indulgence or
other action or inaction under or in respect of any such instrument or any
exercise or non-exercise of any right, remedy, power or privilege under or in
respect of any such instrument or this Agreement; (c) any bankruptcy,
insolvency, reorganization, arrangement, readjustment, composition, liquidation
or similar proceeding with respect to the Company or any of its properties or
creditors; or (d) any limitation on the liability or obligations of the Company
under any such instrument or any invalidity or unenforceability, in whole or in
part, of any such instrument or any term thereof regardless of whether the
Company shall have notice or knowledge of any of the foregoing.

                  7.08 TERMINATION. This Agreement shall terminate upon the
receipt by the Collateral Agent of written notice from the Majority Holders of
the indefeasible payment (or prepayment) in full of the principal or accreted



                                      -12-
<PAGE>   13

value of, and the premium, if any, and interest on, all the Notes, in accordance
with the terms of the Note Purchase Agreement and the Notes, and the
indefeasible payment of all other amounts then owing to the Collateral Agent
hereunder, under the Note Purchase Agreement, the Notes and the Security
Agreements. At the time of such termination, the Collateral Agent, at the
request and expense of the Company, will execute and deliver to the Company a
proper instrument or instruments provided to it acknowledging the satisfaction
and termination of this Agreement, and will duly assign, transfer and deliver to
the Company such of the Collateral as has not yet theretofore been sold or
otherwise applied or released pursuant to this Agreement, together with any
moneys at the time held by the Collateral Agent or the Holders hereunder in
connection with the Collateral.

                  7.09 FURTHER ASSURANCES, ETC. The Company, at its expense,
shall duly execute, acknowledge and deliver all such instruments and take all
such action as the Collateral Agent may request in order further to effectuate
the purposes of this Agreement and to carry out the terms hereof.

                  7.10 CHOICE OF LAW. This Agreement shall be construed and
enforced in accordance with, and the rights of the parties shall be governed by,
the law of the State of New York, excluding choice-of-law principles of the law
of such State that would require the application of the laws of a jurisdiction
other than such State.

                  7.11 AMENDMENTS AND WAIVERS. Any terms of this Agreement may
be amended and the observance of any term hereof may be waived (either generally
or in a particular instance and either retroactively or prospectively) only with
the written consent of each Company Party (at any time prior to and during the
continuance of an Event of Default), the Collateral Agent and the Majority
Holders. Any amendment or waiver effected in accordance with this SECTION 7.11
shall be binding upon the Company Parties and the Collateral Agent.

                  7.12 ENTIRE AGREEMENT. This Agreement, together with the other
Security Agreements, embodies the entire agreement and understanding among the
parties hereto as to the subject matter hereof and supersedes all prior
agreements and understandings relating to the subject matter hereof.

                  7.13 SUCCESSORS AND ASSIGNS. This Agreement shall be binding
upon the successors and/or assigns of each party hereto.

            [THE REMAINDER OF THIS PAGE IS INTENTIONALLY LEFT BLANK]



                                      -13-
<PAGE>   14

                  IN WITNESS WHEREOF, the parties hereto have executed this
Agreement by their duly authorized officers as of the day and year first above
written.

                                      BGLS INC.



                                      By:   /s/ RICHARD J. LAMPEN
                                            ----------------------------------
                                            Name:Richard J. Lampen
                                            Title:  Executive Vice President

                                      ADDRESS FOR NOTICES:

                                      100 S.E. Second Street
                                      Miami, Florida  33131
                                      Telephone:  (305) 579-8000
                                      Facsimile:  (305) 579-8009

                                      Attention:  Richard J. Lampen
                                                  Executive Vice President



                                      -14-
<PAGE>   15


                                     BROOKE GROUP HOLDING INC.



                                     By:   /s/ RICHARD J. LAMPEN
                                           ----------------------------------
                                           Name: Richard J. Lampen
                                           Title: Executive Vice President

                                     ADDRESS FOR NOTICES:

                                     100 S.E. Second Street
                                     Miami, Florida  33131
                                     Telephone:  (305) 579-8000
                                     Facsimile:  (305) 579-8009

                                     Attention:  Richard J. Lampen
                                                 Executive Vice President



                                      -15-
<PAGE>   16


                                  VECTOR GROUP LTD.



                                  By:   /s/ RICHARD J. LAMPEN
                                        ---------------------------------
                                        Name: Richard J. Lampen
                                        Title: Executive Vice President

                                  ADDRESS FOR NOTICES:

                                  100 S.E. Second Street
                                  Miami, Florida  33131
                                  Telephone:  (305) 579-8000
                                  Facsimile:  (305) 579-8009

                                  Attention:  Richard J. Lampen
                                               Executive Vice President



                                      -16-
<PAGE>   17



                                      NEW VALLEY HOLDINGS, INC.



                                      By:   /s/ RICHARD J. LAMPEN
                                            ---------------------------------
                                            Name: Richard J. Lampen
                                            Title:  Executive Vice President

                                      ADDRESS FOR NOTICES:

                                      100 S.E. Second Street
                                      Miami, Florida  33131
                                      Telephone:  (305) 579-8000
                                      Facsimile:  (305) 579-8009

                                      Attention:  Richard J. Lampen
                                                  Executive Vice President



                                      -17-
<PAGE>   18


                                      UNITED STATES TRUST COMPANY OF NEW YORK,
                                            as Collateral Agent

                                      By:   /s/ PATRICIA GALLAGHER
                                            ----------------------------------
                                            Name: Patricia Gallagher
                                            Title:  Assistant Vice President

                                      ADDRESS FOR NOTICES:

                                      114 West 47th Street, 25th Floor
                                      New York, NY  10036-1532
                                      Telephone:  (212) 852-1664
                                      Facsimile:  (212) 852-1626

                                      Attention:  Patricia Gallagher



                                      -18-
<PAGE>   19



                                 TCW LEVERAGED INCOME TRUST, L.P.,
                                 By:   TCW Advisers (Bermuda), Ltd.,
                                       as its General Partner

                               By:     /s/ DARRYL L. SCHALL
                                       ------------------------------------
                                       Name: Darryl L. Schall
                                       Title: Managing Director

                                 By:   TCW Investment Management Company,
                                       as Investment Adviser


                                 By:   /s/ MARK ATTANASIO
                                       ------------------------------------
                                       Name: Mark Attanasio
                                       Title: Group Managing Director

                                 ADDRESS FOR NOTICES:

                                 Trust Company of the West
                                 11100 Santa Monica Boulevard, Suite 2000
                                 Los Angeles, CA 90025
                                 Telephone:  (310) 235-5956
                                 Facsimile:  (310)235-5966

                                 Attention:  Alena Tabora

                                 WITH A COPY TO:

                                 Milbank, Tweed, Hadley & McCloy LLP
                                 601 South Figueroa Street, 30th Floor
                                 Los Angeles, California 90017
                                 Telephone:  (213) 892-4333
                                 Facsimile:  (213) 629-5063

                                 Attention:  Kenneth J. Baronsky




                                      -19-
<PAGE>   20


                                  TCW LEVERAGED INCOME TRUST II, L.P.
                                               By:  TCW (LINC II), L.P.
                                                    as its General Partner

                                  By:   TCW Advisers (Bermuda), Ltd.,
                                        its General Partner


                                  By:   /s/ DARRYL L. SCHALL
                                        ------------------------------------
                                        Name: Darryl L. Schall
                                        Title: Managing Director

                                  By:   TCW Investment Management Company,
                                        as Investment Adviser


                                  By:   /s/ MARK ATTANASIO
                                        ------------------------------------
                                        Name: Mark Attanasio
                                        Title: Group Managing Director

                                  ADDRESS FOR NOTICES:

                                  Trust Company of the West
                                  11100 Santa Monica Boulevard, Suite 2000
                                  Los Angeles, CA 90025
                                  Telephone:  (310) 235-5956
                                  Facsimile:  (310)235-5966

                                  Attention:  Alena Tabora

                                  WITH A COPY TO:

                                  Milbank, Tweed, Hadley & McCloy LLP
                                  601 South Figueroa Street, 30th Floor
                                  Los Angeles, California 90017
                                  Telephone:  (213) 892-4333
                                  Facsimile:  (213) 629-5063

                                  Attention:  Kenneth J. Baronsky




                                      -20-
<PAGE>   21


                                   TCW LINC III CBO LTD.
                                   By: TCW Investment Management Company,
                                       as Collateral Manager


                                   By:   /s/ DARRYL L. SCHALL
                                         -------------------------------
                                         Name: Darryl L. Schall
                                         Title: Managing Director

                                   By:   /s/ MARK ATTANASIO
                                         -------------------------------
                                         Name: Mark Attanasio
                                         Title: Group Managing Director

                                   ADDRESS FOR NOTICES:

                                   Trust Company of the West
                                   11100 Santa Monica Boulevard, Suite 2000
                                   Los Angeles, CA 90025
                                   Telephone:  (310) 235-5956
                                   Facsimile:   (310)235-5966

                                   Attention:  Alena Tabora

                                   WITH A COPY TO:

                                   Milbank, Tweed, Hadley & McCloy LLP
                                   601 South Figueroa Street, 30th Floor
                                   Los Angeles, California 90017
                                   Telephone:  (213) 892-4333
                                   Facsimile:  (213) 629-5063

                                   Attention:  Kenneth J. Baronsky




                                      -21-
<PAGE>   22


                                  POWRs 1997-2 (Participating Obligations
                                               with Residuals 1997-2)

                                  By:   Citibank Global Asset Management,
                                        Its Investment Advisor

                                  By:   TCW Asset Management Company,
                                        Its Portfolio Manager


                                  By:   /s/ DARRYL L. SCHALL
                                        ---------------------------------
                                        Name: Darryl L. Schall
                                        Title: Managing Director

                                  By:   /s/ MARK ATTANASIO
                                        ---------------------------------
                                        Name: Mark Attanasio
                                        Title: Group Managing Director

                                  ADDRESS FOR NOTICES:

                                  Trust Company of the West
                                  11100 Santa Monica Boulevard, Suite 2000
                                  Los Angeles, CA 90025
                                  Telephone:  (310) 235-5956
                                  Facsimile:  (310) 235-5966

                                  Attention:  Alena Tabora

                                  WITH A COPY TO:

                                  Milbank, Tweed, Hadley & McCloy LLP
                                  601 South Figueroa Street, 30th Floor
                                  Los Angeles, California 90017
                                  Telephone:  (213) 892-4333
                                  Facsimile:  (213) 629-5063

                                  Attention:  Kenneth J. Baronsky




                                      -22-
<PAGE>   23


                                   CAPTIVA II FINANCE LTD.,
                                   By:   TCW Advisors, Inc.
                                         Its Financial Manager

                                   By:   /s/ DARRYL L. SCHALL
                                         ------------------------------
                                         Name: Darryl L. Schall
                                         Title: Managing Director

                                   By:   /s/ MARK ATTANASIO
                                         ------------------------------
                                         Name: Mark Attanasio
                                         Title: Group Managing Director

                                   ADDRESS FOR NOTICES:

                                   Trust Company of the West
                                   11100 Santa Monica Boulevard, Suite 2000
                                   Los Angeles, CA 90025
                                   Telephone:  (310) 235-5956
                                   Facsimile:  (310) 235-5966

                                   Attention:  Alena Tabora

                                   WITH A COPY TO:

                                   Milbank, Tweed, Hadley & McCloy LLP
                                   601 South Figueroa Street, 30th Floor
                                   Los Angeles, California 90017
                                   Telephone:  (213) 892-4333
                                   Facsimile:  (213) 629-5063

                                   Attention:  Kenneth J. Baronsky





                                      -23-
<PAGE>   24



                                  AIMCO CDO, SERIES 2000-A
                                  By:   Allstate Investment Management Company,
                                        Its Collateral Manager
                                  By:   TCW Asset Management Company,
                                        Its Investment Advisor


                                  By:   /s/ DARRYL L. SCHALL
                                        -------------------------------
                                        Name: Darryl L. Schall
                                        Title: Managing Director

                                  By:   /s/ MARK ATTANASIO
                                        -------------------------------
                                        Name: Mark Attanasio
                                        Title: Group Managing Director

                                  ADDRESS FOR NOTICES:

                                  Trust Company of the West
                                  11100 Santa Monica Boulevard, Suite 2000
                                  Los Angeles, CA 90025
                                  Telephone:  (310) 235-5956
                                  Facsimile:  (310)235-5966

                                  Attention:  Alena Tabora

                                  WITH A COPY TO:

                                  Milbank, Tweed, Hadley & McCloy LLP
                                  601 South Figueroa Street, 30th Floor
                                  Los Angeles, California 90017
                                  Telephone:  (213) 892-4333
                                  Facsimile:  (213) 629-5063

                                  Attention:  Kenneth J. Baronsky




                                      -24-