<PAGE>   1
                              AMRESCO CAPITAL TRUST

                                  EXHIBIT 99.4

                         CHARTER FOR THE AUDIT COMMITTEE
                                       OF
                           THE BOARD OF TRUST MANAGERS
                                       OF
                              AMRESCO CAPITAL TRUST
                                 (THE "COMPANY")

PURPOSE

         The Audit Committee assists the Board of Trust Managers in fulfilling
its oversight responsibilities relating to corporate accounting, financial
reporting, internal controls and the audit process. The Audit Committee also
will act as an avenue of communication between the Board of Trust Managers and
the Company's outside auditors, management and internal accountants.

COMPOSITION AND TERM

         The Audit Committee will be a committee of the Board of Trust Managers
comprised only of independent trust managers with at least three members. For
purposes of this Charter, an independent trust manager is defined as a trust
manager who has not been an employee of the Company in the prior three years,
does not receive compensation or payments in excess of a de minimus amount from
the Company (other than for service as a Trust Manager), is not a controlling
shareholder of the Company and does not have any other family or material ties
to the Company or management which, in the opinion of the Board or Audit
Committee members, would interfere with the exercise of independent judgment in
carrying out the responsibilities of a member of the Audit Committee. For
purposes of determining whether a Trust Manager is independent, any more
restrictive definition of "independent" contained in the listing requirements
for Nasdaq will be incorporated herein and deemed controlling. In all cases,
every member of the Audit Committee will have an ability to read and understand
fundamental financial statements. Additionally, at least one member of the Audit
Committee will have past employment experience in finance or accounting,
professional certifications in the areas of finance or accounting or other
comparable experience such as having been a chief executive officer, chief
financial officer or other senior officer with financial oversight
responsibilities denoting financial sophistication.

         The Audit Committee members will be appointed for one year terms at the
annual meeting of the Board. The chairman of the Audit Committee will also be
designated by the Board.

ADMINISTRATIVE MATTERS

         The Audit Committee will meet at such times and from time to time as it
deems to be appropriate, but not less than two times each year. A majority of
the members of the Audit Committee will constitute a quorum, with the majority
vote of those Committee members present at a meeting at which a quorum is
present being sufficient to adopt a resolution or otherwise take action. The
Audit Committee will report to the full Board of Trust Managers at the first
Board meeting following any Audit Committee meeting.


<PAGE>   2



         The Company's outside independent auditors and internal accountants
will attend at least one of the Audit Committee's meetings each year. The Audit
Committee may request members of management, or others to attend meetings and
provide pertinent information as the Audit Committee deems necessary. The Audit
Committee will also provide management, the outside independent auditors and
internal accountants with appropriate opportunities to meet separately with the
Audit Committee to discuss any matters that the Committee or these groups
believe should be discussed privately with the Audit Committee. The Committee
members will have the sole discretion in determining the meeting attendees and
agenda.

DUTIES AND RESPONSIBILITIES

         The duties and responsibilities of the Audit Committee include the
following:

         1. Consider and provide recommendations to the Board of Trust Managers
            as to:

               o    the scope of the audit conducted by the independent outside
                    auditors and their proposed fees;

               o    the selection (or nomination of the outside auditors to be
                    proposed for shareholder approval in any proxy statement),
                    retention and where appropriate, replacement of the
                    Company's firm of independent outside auditors;

               o    the independence of the Company's selected outside auditors;

               o    the risk assessment process proposed to be used by the
                    independent outside auditors in their examination; and

               o    the advisability of having the independent outside auditors
                    make specified studies and reports as to particular auditing
                    or other financial matters.

         2. Review the results of each of the independent outside auditors'
audits, including:

               o    the audit report, the presentation of the published
                    financial statements, the observations on internal controls
                    and accounting matters made by the outside auditors and any
                    other pertinent reports prepared by the independent outside
                    auditors and delivered to the Audit Committee;

               o    any material accounting issues identified by the independent
                    outside auditors; and

               o    other matters required to be communicated to the Audit
                    Committee under generally accepted auditing standards as
                    part of the independent outside auditor's audit.

         3. Review with management and the independent outside auditors such
accounting policies (and changes therein) and financial reporting issues that
could have a material impact on the Company's financial statements.


<PAGE>   3


         4. Review, through its Chairman or the Committee as a whole, with
financial management and the outside auditors the company's quarterly financial
results prior to the release of earnings and/or the company's quarterly
financial statements prior to filing or distribution.

         5. Review whether the risk assessment procedures described in the audit
report or other reports submitted to the Audit Committee and utilized by the
Company's internal accountants are adequate to attain the internal control and
financial reporting objectives required by applicable law or deemed appropriate
by the Company's management or the Board.

         6. Review the quality and staffing of the Company's internal
accountants.

         7. Obtain from the outside auditors on an annual basis a formal written
statement that the outside auditor is in fact independent and certifying that it
has delineated all relationships between the Company and the outside auditor
required to be disclosed under generally accepted auditing standards.

         8. Review periodically with the general counsel legal and regulatory
matters that may have a material impact on the Company's financial statements or
compliance policies and programs.

         9. Review and discuss with management and the independent outside
auditors the audited financial statements to be included in the Company's Annual
Report on Form 10-K and discuss with auditors the matters required to be
discussed by the relevant auditing standards. Based on this review and
discussions, the Committee will report to the Board and the shareholders whether
it recommends that the most recent year's audited financial statements be
included in the Company's Annual Report on Form 10-K.

         10. Make an annual self-assessment of the Audit Committee, including a
review of this charter.

         The Audit Committee will also undertake such additional activities
within the scope of its primary purpose as the Audit Committee may from time to
time determine. The Audit Committee, if it considers it necessary or
appropriate, may retain independent counsel, accountants or others to assist it
in carrying out its functions under this Charter.


OUTSIDE AUDITOR'S ACCOUNTABILITY

         Each engagement letter for the services of outside auditor to conduct
an audit and issue an Audit Report on the Company will contain a provision
advising the outside auditor of its ultimate accountability to the Board of
Trust Managers and the Audit Committee, as representatives of the Company's
shareholders, for the services to be rendered.