<PAGE>

                                                                    Exhibit 10.1
                         INVESTMENT ADVISORY AGREEMENT


     AGREEMENT made as of April 28, 2000 between the Imperial Special
Investments, Inc., a California corporation (hereinafter called the "Fund"), and
Imperial Asset Management, Inc., a California corporation registered under the
Investment Adviser's Act of 1940, with its principal office in Inglewood,
California (hereinafter called the "Investment Adviser").

     WHEREAS, the Fund is registered as a closed-end, non-diversified,
management investment company under the Investment Company Act of 1940, as
amended ("1940 Act"); and

     WHEREAS, the Fund desires to retain the Investment Adviser to furnish
certain investment advisory and related services described below in connection
with the management of the Fund, and the Investment Adviser represents that it
is willing and possesses the legal authority to furnish such services;

     NOW, THEREFORE, in consideration of the premises and mutual covenants
herein contained, it is agreed between the parties hereto as follows:

     1.   Appointment.  The Fund hereby appoints the Investment Adviser to act
          -----------
as investment adviser to the Fund for the period and on the terms set forth in
this Agreement.  The Investment Adviser accepts such appointment and agrees to
furnish the services herein set forth for the compensation herein provided.

     2.   Delivery of Documents.  The Fund has furnished the Investment Adviser
          ---------------------
with copies properly certified or authenticated of each of the following
documents:

          (a)  the Fund's Articles of Incorporation, dated April 24, 2000 and
filed with the Secretary of State of the State of California, and all amendments
thereto or restatements thereof (such Articles of Incorporation, as presently in
effect and as it shall from time to time be amended or restated, is herein
called the "Articles of Incorporation");

          (b)  the Fund's Bylaws and amendments thereto;

          (c)  resolutions of the Fund's Board of Directors authorizing the
appointment of the Investment Adviser and approving this Agreement;

          (d)  the Fund's original Notification of Registration on Form N-8A
under the 1940 Act as filed with the Securities and Exchange Commission on April
28, 2000 and all amendments thereto;

          (e)  the Fund's most recent Private Offering Memorandum (such Private
Offering Memorandum, as presently in effect, and all amendments and supplements
thereto are herein collectively called the "Private Offering Memorandum").
<PAGE>

     The Fund will promptly furnish the Investment Adviser with copies of all
amendments of or supplements to the foregoing documents.

     3.   Management.  Subject to the supervision of the Fund's Board of
          ----------
Directors, the Investment Adviser will provide or cause to be provided a
continuous investment program for the Fund, including investment research and
management with respect to all securities and investments and cash equivalents
in the Fund.  The Investment Adviser will determine or cause to be determined
from time to time what securities and other investments will be purchased,
retained or sold by the Fund and will place or cause to be placed orders for
purchase and sale on behalf of the Fund.

     The Investment Adviser will provide the services under this Agreement in
accordance with the Fund's investment objective, policies and restrictions as
stated in the Private Offering Memorandum, resolutions of the Fund's Board of
Directors, and any undertakings with regulatory authorities which are provided
by the Fund to the Investment Adviser.  The Investment Adviser further agrees
that it:

          (a)  will use the same skill and care in providing such services as it
uses in providing services to fiduciary accounts for which it has investment
responsibilities;

          (b)  will comply in all material respects with all applicable Rules
and Regulations of the Securities and Exchange Commission under the Investment
Company Act of 1940 and in addition will conduct its activities under this
Agreement in accordance with any applicable regulations pertaining to the
investment advisory activities of the Investment Adviser;

          (c)  will place or cause to be placed orders for the Fund either
directly with the issuer or with any broker or dealer and, in placing orders
with brokers and dealers, the Investment Adviser or any sub-investment adviser
employed by the Investment Adviser will attempt to obtain prompt execution of
orders in an effective manner at the most favorable price.  Consistent with this
obligation, when the execution and price offered by two or more brokers or
dealers are comparable, the Investment Adviser or any sub-investment adviser
employed by the Investment Adviser may, in its discretion, purchase and sell
portfolio securities to and from brokers and dealers who provide the Investment
Adviser or any such subinvestment adviser with research advice and other
services; and

          (d)  will treat confidentially and as proprietary information of the
Fund all records and other information relative to the Fund and prior, present,
or potential shareholders of the Fund learned by, or disclosed to, the
Investment Adviser in the course of its performance of its responsibilities and
duties under this Agreement, and will not use such records and information for
any purpose other than performance of its responsibilities and duties hereunder,
except after prior notification to and approval in writing by the Fund, which
approval shall not be unreasonably withheld and may not be withheld where the
Investment Adviser may be exposed

                                      -2-
<PAGE>

to civil, regulatory, or criminal sanctions for failure to comply when requested
to divulge such information by duly constituted authorities, or when so
requested by the Fund.

     4.   Use of Sub-Investment Adviser.  The Investment Adviser may, subject to
          -----------------------------
the approvals required under the 1940 Act, employ a sub-investment adviser to
assist the Investment Adviser in the performance of its duties under this
Agreement.  Such use does not relieve the Investment Adviser of any duty or
liability it would otherwise have under this Agreement.  Compensation of any
such sub-investment adviser for services provided and expenses assumed under any
agreement between the Investment Adviser and such sub-investment adviser
permitted under this paragraph is the sole responsibility of the Investment
Adviser.

     5.   Services Not Exclusive.  The investment management services furnished
          ----------------------
by the Investment Adviser hereunder are not to be deemed exclusive.  Except to
the extent necessary to perform the Investment Adviser's obligations under this
Agreement, nothing herein shall be deemed to limit or restrict the right of the
Investment Adviser, or any subsidiary or affiliate of the Investment Adviser, or
any employee of the Investment Adviser, to engage in any other business or to
devote time and attention to any other business, whether of a similar or
dissimilar nature, or to render services of any kind to any other person.

     6.   Books and Records.  In compliance with the requirements of Rule 3la-3
          -----------------
under the 1940 Act, the Investment Adviser hereby agrees that all records which
it maintains for the Fund are the property of the Fund and further agrees to
surrender promptly to the Fund any of such records upon the Fund's request.  The
Investment Adviser further agrees to preserve for the periods prescribed by Rule
3la-2 under the 1940 Act the records required to be maintained by Rule 3la-1
under the 1940 Act.

     7.   Expenses.  During the term of this Agreement, the Investment Adviser
          --------
will pay all expenses incurred by it in connection with its activities under
this Agreement other than the cost of securities (including brokerage
commissions or charges, if any) purchased for the Fund.  The Fund will be
responsible for all of the Fund's expenses and liabilities.

     8.   Compensation.  For the services provided and the expenses assumed
          ------------
pursuant to this Agreement, the Fund will pay the Investment Adviser and the
Investment Adviser will accept as full compensation therefor an annual fee paid
monthly on the first business day of each month equal to the lesser of (i)
$200,000 or (ii) such fee as may from time to time be agreed upon in writing by
the Fund and the Investment Adviser.  If the fee payable to the Investment
Adviser pursuant to this paragraph begins to accrue after the beginning of any
month or if this Agreement terminates before the end of any month, the fee for
the period from such date to the end of such month or from the beginning of such
month to the date of termination, as the case may be, shall be prorated
according to the proportion which such period bears to the full month in which
such effectiveness or termination occurs.

                                      -3-
<PAGE>

     9.   Limitation of Liability.  The Investment Adviser shall not be liable
          -----------------------
for any error of judgment or mistake of law or for any loss suffered by the Fund
in connection with the performance of this Agreement, except a loss resulting
from a breach of fiduciary duty under the Investment Company Act of 1940 with
respect to the receipt of compensation for services or a loss resulting from
willful misfeasance, bad faith or gross negligence on the part of the Investment
Adviser in the performance of its duties or from reckless disregard by it of its
obligations and duties under this Agreement.  In no case shall the Investment
Adviser be liable for actions taken or nonactions with respect to the
performance of services under this Agreement based upon specific information,
instructions, or requests given or made to the Investment Adviser by an officer
of the Fund thereunto duly authorized.  Notwithstanding the foregoing, nothing
in this paragraph should be deemed to be a waiver or limitation of any rights
the Fund may have under the Federal securities laws.

     10.  Duration and Termination.  This Agreement will become effective as of
          ------------------------
the date first written above, provided that it shall have been approved by vote
of a majority of the outstanding voting securities of the Fund, in accordance
with the requirements under the 1940 Act, and, unless sooner terminated as
provided herein, shall continue in effect until October 31, 2001.  Thereafter,
if not terminated, this Agreement shall continue in effect for successive
periods of twelve months each ending on October 31 of each year, provided such
                                                                 --------
continuance is specifically approved at least annually (a) by the vote of a
majority of those members of the Fund's Board of Directors who are not parties
to this Agreement or interested persons of any party to this Agreement, cast in
person at a meeting called for the purpose of voting on such approval, and (b)
by the vote of a majority of the Fund's Board of Directors or by the vote of a
majority of the outstanding voting securities of the Fund.  Notwithstanding the
foregoing, this Agreement may be terminated at any time on sixty days' written
notice, without the payment of any penalty, by the Fund (by vote of the Fund's
Board of Directors or by vote of a majority of the outstanding voting securities
of the Fund) or by the Investment Adviser.  This Agreement will immediately
terminate in the event of its assignment. (As used in this Agreement, the terms
"majority of the outstanding voting securities", "interested persons" and
"assignment" shall have the same meaning of such terms in the 1940 Act.)

     11.  Amendment of this Agreement.  No provision of this Agreement may be
          ---------------------------
changed, waived, discharged or terminated orally, but only by an instrument in
writing signed by the party against which enforcement of the change, waiver,
discharge or termination is sought.

     12.  Legal Advice.  The Investment Adviser shall notify the Fund at any
          ------------
time the Investment Adviser believes that it is in need of the advice of counsel
with regard to its responsibilities and duties pursuant to this Agreement; if
the Investment Adviser wishes to seek the advice of legal counsel to the Fund it
must first notify the Fund and seek its approval, which shall not be
unreasonably withheld, such advice to be at the expense of the Fund unless
relating to a matter involving the Investment Adviser's willful misfeasance, bad
faith, gross negligence or reckless disregard with respect to the Investment
Adviser's responsibilities and duties hereunder

                                      -4-
<PAGE>

and the Investment Adviser shall in no event be liable to the Fund or any
shareholder or beneficial owner of the Fund for any action reasonably taken
pursuant to such advice.

     13.  Miscellaneous.  The captions in this Agreement are included for
          -------------
convenience of reference only and in no way define or delimit any of the
provisions hereof or otherwise affect their construction or effect.

     Any notice required or permitted to be given by either party to the other
shall be deemed sufficient if sent by registered or certified mail, postage
prepaid, addressed by the party giving notice to the other party at the last
address furnished by the other party to the party giving notice:  if to the
Fund, at 9920 S. La Cienega Blvd., Suite 636, Inglewood, California  90301,
Attention: Controller Department; and if to the Investment Adviser, at 9920 S.
La Cienega Blvd., Suite 636, Inglewood, California  90301, Attention: Controller
Department.

     If any provision of this Agreement shall be held or made invalid by a court
decision, statute, rule or otherwise, the remainder of this Agreement shall not
be affected thereby.

     This Agreement shall be binding upon and shall inure to the benefit of the
parties hereto and their respective successors and shall be governed by the laws
of the State of California.

     The names "Imperial Special Investments, Inc." and "Director of Imperial
Special Investments, Inc." refer respectively to the Fund created and the
Directors, as directors but not individually or personally, acting from time to
time under Articles of Incorporation dated as of April 24, 2000 to which
reference is hereby made and a copy of which is on file at the office of the
Secretary of the State of California and elsewhere as required by law, and to
any and all amendments thereto so filed or hereafter filed.  The obligations of
"Imperial Special Investments, Inc." entered into in the name or on behalf
thereof by any of the Directors, representatives or agents are made not
individually, but in such capacities, and are not binding upon any of the
Directors, shareholders or representatives of the Fund personally, but bind only
the assets of the Fund, and all persons dealing with the Fund must look solely
to the assets of the Fund for the enforcement of any claims against the Fund.

                                      -5-
<PAGE>

     IN WITNESS WHEREOF, the parties hereto have caused this instrument to be
executed by their officers designated below as of the day and year first above
written.

                          Imperial Special Investments, Inc.

                          By: /s/ Richard M. Baker
                              ________________________________________________
                          Title: SVP & General Counsel


                          Imperial Asset Management, Inc.

                          By: /s/ Richard M. Baker
                              ________________________________________________
                          Title: SVP & General Counsel

                                      -6-