i
                                                                                                                            'il
                                                                                                                    'I
                                                                                                              1     '
                                                                                                I
                                                                                                I                          I
                                                                                                     .1i'l    I
                                                                                                1  @
                                                                                                illtltl'








                                                   Draft:  January 26, 1972















                             'REGIONAL MEDICAL PROORAMS SERVICE

                                          AND

                         domPUTER ASSISTED EKG  ANALYSIS SYSTEMS






























                                             Division of Professional and
                                              Technical Development
                                            @R4igional Medical Programs Service





















                                       TABLE OF CONTENTS









                         I. Introduction

                       II.  Need for,Improved   ZltctrbcardiogrAphid Services

                       111. Description of   CAE Systems

                       TV.  Current Functional Status of CAE Systems

                         V. Advantages   and Potential Advantages of,CAE Systems

                       VI.  Limitations


                      @VII. Potential RMPS Roles

                      VIII. Deployment

                       TX.  Recommendations

                            Appendix I













            1.  Introduction


                  The  staff  report  of  August  1971  entitled  "Computer  Assisted  EKG

            Analysis and the Regional Medical Programs" described the difficulties

            encountered  by  four  Regional  Medical  Programs  in  the  development   and

            implementation of a computer assisted electrocardiographic system (CAE).

            The information presented  was obtained from review of appropriate

            materials  gontained'in  grant   applications,   correspondence   filet,   site

            visit and progress reports,  Statements pertaining        to  the   functional

            capabilities  of  established  CAE  systems  were  derived  from  a   limited

            review  of  pertinent'literature,  and  attempted  to  define   the   practical

            advantages  and  the  insd4quacied  of  existing   systemi.   The   limitations

            i@erent.to a    report  based  solely  upon  these   sour.c*g are apparent,

            Accordingly, the report    was submitted to experts in the     field of    com-

            puterized  electrocardiography  both  within  and  without   RWS   with   the

            purpose of  identifying   major  deficiencies   and   inaccuracies   contained

            in  it,,and@t,o  offer  appropriate  suggestions.  Additibnally,  an   ad   hoc

            panel   of experts was convened   by RMPS to discuss the report     and  to

            .appritt Fd4PS As to the current state of the art.  A list of       patticipants

            At this  meeting$  and the agenda is included in Appendix I.


                  The@prosent,report@has   beenprop:ared@in,direct   response   to   the.

            .react:Lonsigeherated by, the  initial  paper  and  reflects  the   continued
            interest  on  the part of both,@the' Regional  Medical   Programs   Servide@@

            and individual,Retional Medical Programs with respect        to  CAE   systems.

            Three"sotircei,of  information have   been utilikad:








                                                                         2.



                1.  Background and descriptive material derived from the current

            medical literature included in the annotated bibliography accompanying

            this report.  (Appendix II)



                2.  Written communications from the project directors of four

            ongoing RKP supported CAE,progtame.


                3,  The stenographic transcript  of the proceeding of the ad hoc
            Advisory panel Assembled by RMPS on  November 30, 1971.


                The objectives of this report include the following:


                1.  To summarize the present functional capabilities of CAE
            systems with emphasis directed towards practical potentials and
            applications,, and realistic limitations.


                2,  To establish suitable roles for RMPS with respect to designi

            development an(i iL-ipleinentitioii of CAE systems.


                3.  To provid6 guidelines with respect to system selection,
            deployment and marketing strategies for use by RMPs interested in

            CAE systems.,.









                                                                          3.
                4.  To provide suitable information for use by the National
          Advisory Council, RMPS, in establishing policy with respect to sup-
          port of CAE systems.


                5.  To recommend appropriate RMPS policy in this area.


          II.  Need for Improved Electrocardiographic Services
               Disorders of the cardiovascular system are the leading causes of
          morbidity And mortality in the United States.  It has been estimated
          that over one million Americans die from acute coronary heart disease
          annually,.and an additional several hundred thousand succumb to cerebro-
          vancular@diseases.  The electrocardiogram is an essential tool in the
          evaluation.ahd management of these and other  disorders.  It  is  ac-
          knowledged by experts, however, that many areas of the country lack
          adequate electrocardiographic services.  The average practitioner
          though often familiar with the rudiments of EKG interpretation, relies
          uIpon the assistance of associates more thoroughly trained in this
          specialized area.  Ideally, an expert cardiologist should be available
          for this servicei but this is frequently not the case.  More comonlyi an
          internist or general practitioner with an interest in electrocardiog-
          raphy  assumes this function. Clearly, the quality of electtdcardio-
          graphic serv ces available in.any particular area of the country  raa
                                                                              y
          be quite variable.  Many potential solutions to  this problem  exist,

              -Train.irLg,more cardiologists or additional training for non-
          card ologist physicians are two approaches  It  must  be  realized,..







                                                                             4.

            however, that the existing deficiencies are as much due to mal
            distribution of trained personnel As they are to inadequacies in
            absolute nuImbers.  It is all too common to find an abundance of
            highly trained cardiologists concentrated around a sophisticated
            medical center while the surrounding community lacks such expertise.
            The problem is even more obvious when one examines the services
            available to peripheral comr4un ity hospitals in the more rural areas
            of the country.  It is unrealistic to expect more equitable distri-
            bution of existing manpower in the near future.  The extension of
            expert services available in modern medical centers into their
            surrounding regions through application of advanced technology is an
            attractive alternatives C    uter assisted EKG systems represents
                                       OMP

            one example of   this@approach.

               .Aside from  the  necessity of increasing the availability of EKG
                     there exists  a  need to improve the quality of EKG inter-
            pretation.  Conv entional electrocardiography suffers from severa
            shortcomings basic to which are the limitations of the human mind.
            Even the most skilled cardiologist is subject to fatigue.  This
            affects both his observations and his analysis.  Lack of consistency
            in the application of the multiplicity of diagnostic criteria is
            another characteristic that impairs the performance of electrocardi-
            ographers.  Finally, human EKG interpretation is biased by factors
            other than the analytic data provided in the tracings.  The electro-
            cardiographer@has variable access to information regarding the  ati nt s
            age  sex,, health status, medication, symptomatology  etc., and this









                                                                               5.



          influences his estimation of  various  abnormalities  of  EKG  wavefom.

          Though such clinical correlation is essential to accurate patient

          evaluation, it distorts the interpretation of objective data presented

          in the @. All of  these  factors  contribute  to  the  variability  of

          humn EKG interpretation.  This is reflected in the magnitude of

          disagreement found among experts analyzing the same EKG, and in    t  e

          incontittencids detected when the one interprdter reads the same

          tracing on different occasions.  Skilled electrbcardiogriphers exhibit

          a roprbd ucibility of EKG diagnosis of approximately 807.; the perform-
          ance of tion-experts is more variable.  The  digital  computer  is.well

          suited to EKG  analysis,      as will be  subsequently  discussed,  is

          capable of overcoming many.of the limitations inherent to human

          interpretation.


               Any proposed system designed to imprdv6 the availability and

          quality'of  electrocardiographic services must do so within the re

          straints imposed by existing economic rtsoureds.  The cost of a con-

          Ventioital ZKG,varies co@siddrably throughout the country  from  a  few

          dollars to thirt dollars or more.  These  figures  re  resent  the  fed
                                                                p

          charged by 'the@ptovider institution or physician and do,not necessarily

          reflect the true coot of the unit  cardiogram.  Limited  data  are  av ail,-
          abi6,a'nalyzitig component costs, i.e... technician salaries, software,

          maintenance, and overhead.  One important objective for any EKG

          service Lneluding,CAE systems must be cost-containment.      Additional
         'hardwart.:software and personnel,costo  should  not  add  significantly








                                                                              6.


           to existing expenditures.  The service must be provided at comparable

           costs, or preferably, with cost savings.



        III.  Descri]2ti6n of CAE Systems


               The digital.computer is capable of analyzing large volumes  of

           specific electrocardiographic variables related to wave contour,

           magnitude, duration, and frequency.  The recognition of deviations

           from programmed normal values in these parameters permits discrimination
           of abnormal EKGs from normal ones.  Identification of patterns of

           deviation can then-be translated into specific diagnostic statements,

           The computer, therefore, may serve only to select out abnormal tracings

           from a larger population of normal ones.  Or, the computer may offer

           diagnostic interpretations with appropriate alternatives and exclusions.


               Though,individual  systems differ with respect to specific hard-

           ware, they all share some  common  features:


               1. A device to detect  and  record  the electrical events of the

           heart and a preamplifier system to  amplify the millivolt-range
           physiologic signals to a level that can  be worked with conveniently

           (data acquisition cart).


               2.  A system for converting this signal into a mode suitable

           for computer input (analog-digital conversion).


               3.  A means of storing and transmitting the coded signal to

           the digital computer.









                                                                              7.


               4.  A digital computer and a suitable computer program


               Available systems differ with respect to design of acquisition

          carts (single channel versus three channel), input mode, computer

          facilities, and printout equ3.pment. The recording  technique  includes

          either standard twelve lead tracings, Frank orthogonal leads,  or  com-

          binations of these leads.  Several contour and rhythm programs are

          available, most using concept$ of binary logic.  The use of nwiti-

          variatt statistical analysis remains  largely  developmental.  Differ-

          ences also exist in the form of the computer printout (paper and

          microfilm), mounting, storage and retrieval capabilities  Finally,

          the types of service provided by any single system vary.


               Recogtizing the multiplicity of CAE systems currently in

          operation, and the variety of hardware, software, program  And  logis-

          tir-s involved the succeeding section concerned with current functional

          capabilities of CAE systems will confine itself to general  issues  and

          avoid detailed analysis of any single system.



         IV. :Current Functional.Status of PAE Systems


               A.  Range of Services


                    Stieening: Large volumes of EKGs can be  batch  processed  by
          existing CAR sy@tems,.rgpidly, reliably,, and at reasonable cost.
          Patients with abnormal or borderline tracings may be  selected  out'by
          the, comIouter for further evaluation.  In addition to separating normal








                                                                                   8


           from abnormal EKGS. the computer may provide specific diagnostic

           statements. EKGs determined to  be  normal  by  the  computer  appear  to

           require no further validation, since the incidence of false negative

           interpretations is less than 1'/..  Depending upon the rigidity of the

           diagnostic  criteria, the incidence of   false  positive  diagnosis   may

           be as  high as 20'/.;.therefore tracings with   suspected   abnormalities

           require physician validation.


                     Definitive Interpretation:  The succeeding sections

           dealing with the accuracy,  volume  capacity  and  cost  factors  of  CAE
           systems relate:.primarily to those,systems providing definitive inter-

           pretation.  Some general comments are in order at this point.  Several

           EKG wave contour and rhythm programs are currently available  and have

           met with varying  success.  Most  offer  standard  twelve  lead  tracings

           along with vecto.rcardiographic display.  Increased utilization of@

           the vectotdardiogram  (VCG)  should  be  considered  advantageous,  since

           it  provides  the  electrocardiographer  wIith additional useful information.

           CAE systems have proven to be  quite  flexible  in  that  changin'  diag-
                                                                            9

           nostic criteria  can  be  accommodated  for  by  appropriate  alterations

           in programming.


                     CAE systems have also shown flexibility with respect to

           deployment.  Computer centers have provided EKG services in a variety

           of environments including  large  medical  complexes,  small  and  medium

           sized hospitals, outpatient  clinics,  small  group  practices  and  solo

           office practices.









                                                                                  9.


                      multiple sources of  funding  have  provided  the  necessary

           working capital  including  NCHSR&D,  RMPS,  commercial  interests  and

           institutional   research And development grants.  CAE systems are
           presently available both  from  commercial  enterprises  and  from  several

           large  medical  centers*


                      Critical Care Unit Monitoring:  On-line EKG monitoring in

           coronary care (CCU),  intensive  care  (ICU),  and  respiratory  care-units

           may provide continuous surveillance  of  cardiac  rhythm  and  contour.,

           Othdr  physiologic  variables  often  simultaneously  monitored  include

           blood gases, blood  pressure,,@respir,atdry  rate  and  temperature.  Inade-

           quacies of,human  senses  with@respect  to  continuous  observation  of

           multiple  variables  have-been  well  documented.  The  application  of CAE

           systems;in these anviroments is largely developmental..


                      Exercise Teattni:  Several  programs  have   been'designed for

           continuous  analysis of EKG wavelcontour  and  rhythm  during  graded exer-

           eine tettin     These programs are essentially research orient         Com-
                       9                                                     ed.

           put6ritbd analysis@of  exercise testing is not widel
                                                                 y available, and

           must be considered in  a developmental   stage.


                      Comparative Interpretation:    Though several computer  pro-

           grams are  undergoing development, relatively few CAE systems    provide
           for even  Limited comparative  EKG  interpretation.   Improved   technology

           is available  for expansion of  storage  and  retrieval  capabilities Which

           may be experted',to,faci,litate comparative   analysis.   Digagreementi








                                                                                  10.


           exists, however,  as to which parameters should be compared.  Addition-

           ally, it has been stated by   some experts that comparative EKG analysis
           can be performed mote effectively by human observers.     All agree   that
           comparative evaluation  of multiple  EKGs adds valuable  information   to
           patient assessment, and  further development  of this aspect of CAE

           systems is to be expected.



                B.  Accuracy


                     Computer analysis of EKGs may be subdivided into t ree       a-
           tinct process66:  signal  acquisition  and  measurement,  comparison  of
           determined parameters against sets of normal values, and synthesis

           of patterns of deviation from normal values into diagnostic statements.


                     With respect to the  measurement process,   the computer appears
           capable of precision comparable to the most meticulous human      effort.
           Though some vitiation in wave measurement is inherent to     the  sampling
           procedure, it  is min   1.  The  computer  experiences  difficulties  in
           areas similarly troublesome to human  observers,  i.e.,  P  wave  recog-

           nition detection of-minute  r  waves,  and  determination  of  initiation,
           duration and termination of Q   waves, ST segment  and  T  wave.  Unlike
           their human counterparts, computers are  not  subject  to@ia.tigue,  thus

           assuring consistency-of repetitive determinations.


                     Recognition of deviations  from  normal  values  and  synthesis

           of dia nostic statements rely upon both the precision of wave measure-
                 9
           ment and the adequacy of  programmed normal values and logic.  Computers










             offer  the  distinct   advantage   of   being   ablelto   retain   large   volumes

             of   information,   including   the   multiple   parameters   required    for EKG

             analysis.  The major limitations are the validity of the progrmmed

             normal   values   and   diagnostic   criteria,   and,thdae    limitations    apply

             equAlly  to   both   computer   and   human   interpretation.   Most   CAE   systems

             in current use utilize binary logic and decision            tables   that    attempt

             to duplicate the analyti     c processes of the expert       electtocardiographer.

             The  net  effect   of   the   computer   interface   is   enhanced   reproducibility

             of interpretation resulting from rigid adherence to                progra-med in-

             structiorts.  Errors   of   omission   are   minimized   though   at   the   expense

             of 10-207*  false  positive statements.


                         If accuracy o   f CAE  systems   is defined as degree of agreement
             with   expert   human interpretation,   then   optimal   performance    should    be

             reflected  in   1007. computer-physician agreement.   In        practice,   less,

             than  the  ideal   is realized.   @Approximate figures from systems         serving
             large   inpatient    po   lation's containing   a   preponderance    of    abnormal
                                   PU
             tracings     indicate that $Si;  of@c,omputer analyzed EKGs,require no subse-
             queht tltoi&tiot@dr     modification$    i.e.,     the     Ohysician-computer &gee-
             meat iii)proac                                        been achieved          c
                           his 95%      Groatet,agreament     has'                  4ith  ontour

             analysis (Approx. 90'/.),'vhile     rhythm statftents ate less reliable

             (7-5-80%- agreement.


                                                                              disagre     'nt are
                      @'Two@seneral,reasong'for cotoputer-physician                     eme
             identifiable:     CAR iiystem.error and     physician    error.    Factors     affect-

                 hands  @formmee        hi-4       iou'ly been alluded to and include
                         per               e  prev     a








                                                                             12.

           fatigue, forgetfulness, imprecise measurement, inconsistency of diag-
           nostic criteria, and bias.  CAE system failure ma  be separated into,
                                                            y
           defects in data acquisition, data transmission, and  data  analysis.

                    The dominant sources  of errors of acquisition are related
           to inadequate recording 'techniques, improper lea.d.positioning, inad-
           vertant omission of leads  respiratory variations, artifact, and back-
           ground and transmission noise.  Many of these problems are resolvable
           through improved technician training, and modification of  the  data
           acquisition darts.  The design of more suitable' electrodes and improved
           methods for signal filtering are under development, and can be expected
           to enhance the quality of the recorded signal.  Similarly, distortion,
           noise, and,artifact introduced during telephonic transmission  appear
           to be potentially solvable technological problems.

                     Analytic errors may be further categorized into inadequacies
           of programming, errors  in measurement, and disagreement as to diag-
           nostic classification.  IErrors of measurement relate to the sampling
           process and, as previously mentioned, moat.often involve failure   to
           detedt P wavesand Q RS$ QT and ST-T segment duration.  Artifact,   back-
           ground noise, and unrecognized ectopic beats may further contribute

           to measurement error.

                     Instances where wave measurement  is accurate but diagnostic
           classification is improper ar.e usually due to remedial errors in pro-
                                ent with respect to d             lassification
           gramming.. Disagreem                      iagnostic c









                                                                              13.


           often reflects minor discrepancies between computer  and  electrocardi-

           ographer.  Physician inconsistency and ambiguity of diagnostic criteria

           are most often responsible.


                     In summary, though the degree of cardiologist-computer agree-

           ment is quite high, A significant incidence of major disagreement still
           exists, thus necessitating human validation of all  computer  analyzed

           EKGS.  Except in screening situations in which the computer can be

           relied upon to separate normal EKGO from equivocal  and  abnormal  ones
           independently, CAE systems serve to assist the electrocardiographer
           in his task. The value of the  physician-computer  interaction  will@be

           discussed in@a subsequent'section (V).



               C*  volume CaEactty


                    A decided  advantage of CAR a stems is their,ability to
                                                  y

           process large numbers of electrocardiograms rapidly  and  continuously.
           The volume capacity of any CAZ system is determined by three major
           variables:, data acquisition time, computer@time,.and  physician  vali-
           dation time.' Acquisition'time is affected by technician  skill, number
           of leads recorded, and type of dart utilized.  Using  carts  durrently

           available for recording three leads simultaneously, the acquisition

           time for a 'twelve lead tracing is comparable to recording a  standard

           twelve lead EKG on a one channel cart.  Addition of three Frank
           orthogonal leads increases recording time slightly.   The quality of
           the adquired signal is an important.determinan.t of overall system









                                                                       14

             erformance and this is directly influenced by the skill and motivation
             p
             of the recording-technician. Additional training is necessary to assure
             high quality signal acquisition.
               Among the factors controlling computer time are the following:.
             capacity of the computer  complexity of computer contour and rhythm
             programs, input mode (batch processing versus interrupted input), and
             range of services provided.  Comparative interpretation requires Addi-
             tional time to allow for search and retrieval of stored records.  A
             medium capacity, dedicated computer is capable of processing  apprbxi-
             mately two hundred EKGs'per hour including analysis and data printout
             without comparative interpretation.
                From a clinical standpoint, it  is advantageous to provide t e
             physician with an interpreted tracing with minimal delay.  The key
             determinant of turn-around time, i.e., time from inscription of the
             signal to delivery of_the definitive interpretation, is the additional
             time required for validation of the computer record.  The computer
             can provide an interpretation within minutes of the inscri tion
                                                                       p
             of the signal,  All computer diagnoses, however, excepting those
             determined to be entirely normal in screening situation, must be
             authenticated by an electrocardiographer.  At the present time this
             service may be provided by cardiologists associated with the computer
             center, local cardiologists, or physicians requesting the electro-
                                ce.                                  physi
             cardiographic servi     It is generally agreed that the      cian
             utilizer should be provided with a validated record, and that
             review by an electrocaidiog rapher is necessary.  It is the degree
             Af na&4afAnng offered by the CAE system to  the  electrocardiographer










           that determines its effective output.  Tracings  identified  as  being

           normal by the computer require no more than casual verification.

           Similarly, many abnormalities are analyzed so reliably by the CAE

           system that the reviewer can accept these  diagnostic  statements  with

           a high level of confidence.  Complex contour abnormalities and arrhyth-

           miss require closer scrutiny.  The computer interface accelerates

           analysis of routine tracings allowing  the  6lettrocatdiographer  more

           time to consider the mote complicated  ones.  The  interpretation  time

           of a cardiologist reading predominantly abnormal tracings may be

           halved, thus potentially doubling his  output.  Utilizing  batch  pro-
           ceasing, turn-around time for a computer  assisted  EKG  interpretation

           with verification  averagestwelve hours or less,



               D.: Cost FActors.,


                     From the standpoint:of the CAE systems provider it is    don-

           venient to examine component costs in three areas:  dat a   acquisition

           and transmission,  computer processing, and  physician  validation..  It

           =at be emphasized that.the.figures mentioned are only approximations,

           with gignifidant@variati6no existing among individual CAE systems@.
           .,some of the factors responsible for these differences Will be.identified.


                     Aside from the  e   ter,  the data acquisition cart  is  the
                                     OMPU
           single most expensive  hardware component of CAE systems.  'Cost Of
           purchase var ies from $6,000  to $10,000, whereas  rental  fees  average
           $2,000 to $3,000 yearly.  Assuming  the functional life of this equip-

           ment to be five to gevenlyears, cost tier unit EKG based upon an annual










          volume of two,th,oust,nd tracings is $1.00 to $2.00.  This is exclusive

          of  maintenance, overhead, supplies, and technician salaries.    These

          factors  must,be figured into the total cost of acquisition  and   apply

          to both systems utilizing computer  analysis, and to    conventional EKG

          services.  Where  technicians function  solely to record  EKGS,  the unit

          cost is increased  by an amount calculated by  dividing   technician

          salary by annual volume  of tracings.  In  institutions   generating large

          volumes of  =O,  technician  salaries  add  approxima tely $2.00 to unit

          acquisition costs.  Smaller  hospitals often use EKG technicians for

          a variety of functions in addition  to  EKG  recordin  thus obscuring
                                                              9

          the analysis  of acquisition costs.


                    Computer costs are  influenced by  computer  size, processing

          mode$ range of service provided, and data,display form.    The  most

          important determinant df@c'omputer costs is efficiency of  utilization.

          Maximal utilization in terms of volume of EKGs processed   results in,

          optimal unit cost. Cost-effectiveness  can  be  achieved  on a  medium

          sized (e.g., IBM 1800) dedicated cemputer with.an  annual  volume  of

          100,000 tracings.  Lesser.volumes are more economically handled using

          a medium sized computer on,a time-sharing basis.  Uninterrupted batch

          processing is the most efficient input mode.  Periodic "stat"   or

          emergency tracings must,be anticipated in any system designed   to pro-

          vide complete electrocardiographic services to inpatient pop ulations,

          though these interruptions impair the operating  efficiency  of  CAE

          systems. -The addition of data storage  and  retrieval  capabilities

          necessary for comparative interpretation increases unit cost, as   do









                                                                      17


           modifications in the form and content of  data  display.  Estimates  of

           $1.00 to $3.00 have been made for direct computer  costs  exclusive  of

           salaries for programmers, clerks, secretaries, maintenance, and over-

           head. Representative combined cost per  unit  EKG  including  adquisi-

           tion, transmission, prodeasing, printout and return varies from $5.00

           to $7.50. However, accurate estimates of  total  direct  and  indirect

           Costs are not available.


                Finally, there is the cost of validating the preprocessed

           EKG. As already mentioned the fee  charged  for  a  standard  eioctro-

           cardiogram shows marked regional variation and the  total  cost  trans-

           mitted to the patient@is not an accurate reflection  of  the  component

           costs of the service. The fees charged  by  physicians  for  ihterpre-

           tation of tracings are highly variable.  If the costs of computer

           preprocessing are-simply,added to the customary fed charged by the

           Olecttdcardi6grapher, cost  containment and reduction are not achieved.

           The obvious question remains:  How can the benefits of computer asoigted

           EKG analysis be realized wi thout inflating the.coit of  the  finished

           product?


                Most commercial organizations marketing CAIR  systems  essentially

           ignore the problem by hot including physician validation  as  par't,of

           the service they sell,, From both medical and  legal  viewpoints,  com-

           puter processed electrocardiograme,lacking  individual  verif,ication

           by a Okilled.6lectro6ardibgkaphtr are unacceptable.  Whether  or   not

           these'comp@ies can@provide CAE:Services   with constantly available

           physician validation at competitive prices is uncertain at  this  time.







            Two mechanisms to provide for validation may be offered.  The first
            is employment by the computer center of electrocardiographers  for  the
            purpose -of providing validation. A foreseeable  problem  is  the  avail-
                                                  A second appro
            ability of round-the-clock cover ge.                ach  is  validation
                                            a
            b  local cardiologists in the area being served by the computer center.
            y
            To avoid increasing EKG costs by addition of computer costs to local
            ones, same adjustment of physician fees may  become  necessary.  Theor-
            etically, since computer preprocessing is estimated to reduce the time
            required for physician interpretation, a reduction in unit cost may
            be possible. However, this will depend upon the complexity of the
            tracings, the need for comparison and the number  of  tracings  analyzed

                     CAE  systems situated in university medical center com-
            plexes have been able to provide EKG service with validation  on  a
            twenty-four hour a day basis and at  a cost of         $4.50 to  $6.50.
            There are several reasons for this. Most  important  are  the  investi-
            gative aspects of these programs and the interests of the participating
            cardiologists.  It.is obvious that the bulk of the money generated
            from the service charge covers.the cost of  acquisition  and  compu.ter
            processing; a relatively small fraction goes to  the  physicians  over-
            seeing the operation.  Secondly, several additional sources of fun  ng
            are available to these programs including support from private industry,
            foundations, and internal research  and  development  monies.  Thirdly,
            at least some of the supervisory function is assumed by members of

            the hous'
                    e staffs, though all tracings are  ultimately  reviewed  by
            senior staff members.  Fourthly, the current service charges    are









                                                                           19.

          adequate for support of the ongoing programs but fail to include

          initial developmental and marketing expenses.  Lastly, these programs

          operate on a,large annual volume of tracings.


                   In summary then, several viable CAE systems are available

          that are capable of delivering,high quality electrocardiographic
          services at reasonable cost.  Though current service charges provide

          adequate support for established, ongoing systemi they do not cook-

          ponsate for start-up costs which MAY be considerable.










                                                                               .20


          V.  Advantages and Potential Advantages of CAE Systems,


               A.  Quality of Electrocardiokraphic Services

                         Computer preprocessing of EKGs minimizes  the  adverse
          effects of physician fatigue* forgetfulness, and bias.  Rigid adherence
          to programmed criteria diminishes errors'of omission by  alerting  the
          physician to diagnostic possibilities otherwise overlooked,  Similarly,
          sources of physician misdiagnosis become identifiable upon  review  of
          the analog.data presented in the computer printout.  The computer inter-
          face allows for greater standarization of diagnostic  criteria,  thus
          reducing inter-observer and intra-observer variation, and   enhancing
          analytic re reducibility.  Additionally, technicians must be more
          meticulous during EKG recording to insure optimal signals for computer
          input; this enhances the quality of the data display and serves as  a
          built-in quality control mechanism.  Overall, the quality of the   EKG
          resulting from the cmputer-physician interaction is superior to   that

          furnished by either one separately.


               B.  utilization of Manpower
                             With respect to the EKG technician, utilization of the
          data acquisition carts is somewhat more complicated than conventional
          electtocdrdiagraphst-and additional training is required.  This is not
          difficult to achieve and technicians readily adapt to the  new  carts.
          Recording time may be slightly increased when fourteen or fifteen lead
          positions are required; though in general  the recording time on









                                                                                      21


           multichannel acquisition carts is comparable to standard machines,


                            The computer interface  permits  automation  of  cutting

           and,lmounting of tracings, thus  freeing  technicians  from  these  time-

           consuming  tasks,  Similarly,  computer  processing  provides   automated

           determination of impulse duration, magnitude,     direction   and frequency,

           these tasks having  previously  been  the  responsibility  of  the individual

           interpreting the EKG.  The  net  effect  is  a,significaut  savings  in

           personnel    time.


                            Two positive effects upon physician utilization are

           discernible.   Firstly, computer  assistance  enhances  the  speed  of inter

           proration  of,the@electrocardiographer, diminishing the    amount   of time

           necessary  for  the review  of  tracings,  This  it  translatable  into an

           increased   volume capacity.for the.physician  allowing  him   to examine

           more records    in less-time. The time saved can be invested in     other

           areas    demanding his attention*  Secondly*  CAE   systems   serve to expand,

           the,ou timddh   of the alettr6c&rdiographic   abilities    situated within the

           computet,center,   As a result, the general quality of electrocardiography

           in  the  community is improved, and areas lacking the essential manpower

           can  have   Access to high qualit  a               nie tly and at manageable
                                           y  ervicea, conve  n


           Costo@



                C.  @Regionalization


                            One attractive aspect of CAE systems from the,point of

           view of RMPS,,,  is the,-potential for   implementing.planned   delivery of










                                                                                22



          essential medical services on a regional basis,  This is   particularly
          true for those systems based in a medical.cdnter  environment.  The
          linkages established between.provider institutions and peiri heral
                                                                     p

          medical facilities antipersonnel have the potential for  developing
          beyond service relationships into viable communications  networks.  It'
          is difficult to estimate the effectiveness and quality  of  cooperative

          relationthips-foiwtored by existing CAE systems.


              D.  Increased Interest


                      It is probable that the availability of CAE  systems has
          and will continue to stimulate increased interest in electrocardiography
          among practicing physicians   This interest is expressed as   increased
          utilization of the electrocardiogram in patient eval uation in  both
          hospital and ambulatory'care enviro=ents, on-a more routine basis.


               I?,,  Coutinuinst Education


                      It has also been suggested that the CAE data display is
          an offective.educational device. Certainly  the  information  contained
          in the printout, i.e., analog data, including diagnostic  criteria  and
          tracing reproductions can increase the knowledge of the average physician.
          However, the extent to which it is utilized is de ttrmined by the motivation
          of the recipient. An evaluation of the educational aspects  of  CAE

          systems is not available.









                                                                                    23



          Fe,   CAE ftstems as a,Reference Point


                          As has been mentioned, little information is availi6

           with respect to cost and time,factors associated with the recording and

           interpretation  of  conventional  electrocardiograms.  Similarly,   relatively
           few studies attempt to define the  quality  of  standard  EKGS.  The  develop-
           mint of CAE programs has resulted in increased understanding of these
           and other aspects of electrocardiography.  This has been evidenced by
           material presented earlier in this paper dealing with cost factorso

           volume capacity$ and  sources of physician-computer disagreement.


                00  Data BAsLe


                          Much of  our current knowledge of electrocardiography is

           based upon analysis of  relatively small numbers of tracings.  In    partt
           this acc.ounts for the multiplicity of diagnostic   criteria  resently in
           uoqi   It is in this Aria that computer technology   will probably  have  the
           greatest impact. The enormous  capacity  of  computers  permits  analysis  and
           clinicovathologie correlation of  hundreds  of  thousands  of  electrocardio-
           grams* enabling us to more accurately define the limits of    normality   and
           the determinants of  abnormality.' The  application  of  computer  techniques
           hit already pointed  but, the limitations of conventional electrocardiography,,
           and will continue to add now knowledge.  Purthik research    in  these  areas,
           including th      lication of,analog computetog multivgriat6    anal sis,
                       a &pp                                                   y
           and high frequency  electrocardiography can be expected to alter,   the EKG

           of the future.










                                                                         24



          VI.    Limitations



                      The preceding pages have dealt almost entirely with   the

          positive aspects of CAE systems.  Unresolved problems do    6xist,   however,

          and need to be identified.



              A.  Costs



                    Provider


                      One persistent problem relates to program start-up.  Estab-

          lished CAE systems with adequate support, in   terms of operating volume.,  are

          able to sustain themselves on funds generat   ed by service charges comparable

          to the fees received for conventional EKGS.    In other  words,  A  potential

          subscriber can purchase the service from one   of several functioning

          systems.rather than,re@ying,on local resources without increasing the

          coat to his Pat ient.  But the funds accumulated by the Provider in this

          way cover.only cost of operation.  The money expended during program

          initiation, development and growth  may  not  be  recovered.  The  commercial

          interest or institution  organizing a CAE system must have access to

          sufficient   capit al to cover the costs of start-up.  In addition, a period

          of time must be planned for acquisition of an adequate market and over-

          Coming technical difficulties.  The magnitude of these variables have

          not yet been clearly defined.


                  Maintenance and operating costs will be determined by local,needs

          and by willingness of local physicians and institutions to subscribe to the










                                                                           25


                                                     uccessful because of their
         service.  Several CAE systems have been uns
          inability to establish an adequate market.


              Utilizer
                  Those individuals and institutions interested in subscribing
          to a CAE system,must first assess their needs and resources.  Partic'l-
         .Pation in such systems is. expensive,  Total direct and indirect costs
          have not yet been clearly delineated,  At the present time, verification
          by an experienced electrocakdiographer must be.included in any CAE
          system,  The, potential cost benefits detived from the system must be
          analyzed to separate  data acquisition savings, from the savings der  ed
          by reduction in the  time required for EKG interpretation.  The former
           is applicable to all  EIKG systems,  The cost to a physician's office, or
           a small EKG laboratory must be evaluated against the costs of alternative
           EKG interpretation systems.  Though CAE systems utiliz ation may be the
           most Advantageous arrangement in certain situations, other approaches
           existi and should be  considered when planning for improvement of
           electrocardiographic  Services.


               B., p
                     rforma
                    Because  of the rapid development of computer technology during
            the past decade, C!AE systems should be considered to be in,d transitional
            phase'with respect to'capabilities and range of services.  Disagrdement









                                                                           26


           among experts as to what constitutes optimal system  components,  both
           hardware and software, continues.  Further refinements of data acquisition
           carts, rhythm and comparativ             storage and retrieval ca abilities
                                       e programs,                          p
           are to be expected.  Increased interest in alternative approached, such
           as multivariate statistical analysis and increased usage of analog
           computers, will alter the nature of CAE systems  of  the  future.  How
           rapidly these changes will occur is difficult to predict.

                   Whether or not a completely automated EKG system is achievable is
           also debated.  Artifacts, complex arrhythmias, minute inconstant deflections,
           the need for comparative interpretation, and transmission difficulties
           are all challenges to this goal.  The advantages anticipated with utilizing
           a CAE system to provide physician assistance must be evaluated against the
           adequacy of existing services in the region or institution.  Emphasis
           should be directed towards@the delivery of high quality' electrociirdio-
           graphy by physicians and institutions ladking this  service  to  their
           patients.  Of the various benefits described in section  V,  the  most
           important, from the viewpoint of RMPS, is the potential for extending
           the outreach of the expertise within the computer center into the
           surrounding community.  The development of a CAE system to serve a
           single large medical center complex, without  linkage  to  surrounding
           institutions, is of lower priority. Though  the  quality  of  electro-
           cardiography as practiced in these centers may be enhanced by the
           .computer interface, such deployment would have little influence upon
           the EKG resources of the surrounding region, where the  need  is  more

           Apparent











                                                                       27


                    Related to this, though on a more philosophical level, ate the
          inherent limitations of the most popular CAE systems; i.ei, those pro-
          grammed to, reproduce human deciaion-making and logid.  Designed to mimic
          the expert electrocardiographer, such systems assume the limitations of
          conventional electrocardiography,  Some have suggested that greater
          benefit would be derived by employing computers to do what the human mind
          cannot do    to examine parameters and variables ignored in currect practice.

         VIII  Potential RMPS Roles


                In direct,response.to the question posed by RMPS, members of the
         ad hoc panel suggested several possible roles for involvement of
         individual M4Ps with CAE  systems.  The following suggestions reflect
         the opinions of the panel members and Are not necessarily in agreement with
         RMPS goals and policies.  They are presented for information only.


                A  Info=atign4,

                     1.  Assist itL the evaluation of local and@regional EKG

         resources and needs.

                     2,  Identify appropriate methods for fulfilling such needs.


               Be Uu-c-ationg
                        Serve'as a@source of information concerning the availability,
        capabilities, and potential advantages of CAE systems,
                    2.  Inform local physicians and institutions as to the existence

        of thi       of EKO service,,
              0 type

                    34  Encourage participation in such systems where local need is











                                                                          28



         demonstrated.

                     4.  Assist in overcoming local  resistance  to  implementation

         of CAR systeps,

                     5. Provide similar information  with  respect  to  alternative

         approaches for the delivery of EKG.service,



                C,    Organization:


                     1.  Encourage communication between the health care providers

         and health  institutions of the region so that c  on needs may   be

         recognized.

                     2,  Foster cooperative arrangements among local hospitals with

         the aim of  improving  the quality and availability of EKG servi6es, whether

         utilizing CAE  systems  or some alternative,



                D.    Consultation:

                     1.  Provide technical and organizational assistance to physicians

         and institutions interested in subscribing to or initiating a CAE system.

                     2.  Serve as liaison between interested parties and experts in

         the field of computer assisted electrocardiography.

                     3.  Provide similar consultative assistance to those interested

         in alternative EKG service system,,


                E.  Support:

                     1.  Make available limited monetary support to assist  in  ti)d

         planning and implementation of regional EKG services including CAE systems.

                     2.  Pro vide short term assistance until  these  systems  become










                                                                            2



          self-supporting.


                F.  Research and Development
                    1.  Encourage essential investigative work in areas related to
          CAE systems -- e.g., exercise testing programs, coronary care  unit

          surveillance, ate,
                    2*  Encourage and support proj6cts concerned with the extension
          of advanced technology into the special care unit environment.


          vtil Dep_loirment


              A. @Comouter Facilities

                   Both:cot=ekeial and institutional computer centers have  been
          utilized.  Private industry has played a large role in the development And
          implementation.@of reliable CAE,Isystems.  Sints they have proved successful*
          And because they can operate on a profit making basis, it  is  most appro-
          priate that CAB systems be marketed by commercial organizations,   Ongoing
          researc.h,and development are essential in all,viable industries  CAL,
          svotems being no  exIceptioa.  Continued development of system hardware,,and
          software should properly remain the responsibilit   of private industry
                                                            y

                   The use  of computer facilities within the medical center environ-
          ment offers additional advantages including a proximity  to expert electro-
          citaiogiapher, educational potential availability for EKG   research, an(.
          the possibility for establishment of a regional communications network.










                                                                                30


           The availability of both equipment and personnel, and the  capacity  for

           sharing computer time and programmer time are also attractive aspects of

           an institutional computer base.



                B.  Data Acquisition Term na a


                     A variety of sites for deployment of computer  terminals  have

           been explored.  Most often, mobile data acquisition carts have been

           utilized.  Efficiency is enhanced in situations where the carts are

           stationary and patients are required to come to some area set aside  for

           the recording of EKGO.  This is, however, impractical in many circum-

           stances.  Small and medium sized hospitals generally employ one or

           several mobile carts that record from patients in the  general  hospital

           area, special care units, and in emergency rooms.  Multiple terminals

           from a number of these institutions are linked to a central computer base

           to furnish the necessary unit volume.


                     Larger institutions may employ many,carts, both stationary and

           mobile.  A computer serving a single medical center complex may receive
           data from terminals situated in the areas already mentioned,  as  well as

           from outpatient facilities, cardiac cattiererization  laboratories,and

           associated extended care facilities.


                     Terminals situated in ambulatory care  facilities  including

           clinics, medical foundations, and HeAlth Maintenance  Organitations  are

           foreseeable.  Provided that the individual facility can generate a suit-

           able volume of tracings, the linkage of multiple ambulatory  care  areas








                                                                            31.


           to a central computer base may be practical.  This has not been

           demonstrated.


                                                                           rding
                    At this time, small groups and solo practitioners reco
           alimited number of tracings daily would be better served by some
           other arrangement. In  these  a.ituations,,the cost of data acquisition

           carts becomei prohibitive.



           IX.  Conclusions and Repommandations

                    .Electrocardiographic services are inadequate in many parts
           Of the counttyo and A need  for improving these services exists.  The
           magnitude of this need is quite variable,both among different regions
           and in different areas of a single region.  OrgAnization of, or iub-
           odription to, a CAE system may b6,ond approach towards fulfilling
           these need i.  $everal existing CAR systems have@demonstrated;their
           ability to improve both the quality and the availability of electro-
           cardiographic services, though problems of cost containment  continue.
           RIIP involvement with:CAE@aystems may be appropriate in some  instances

           with the following qualifications:

                le  Regional noode@with respect to electrocardiography must   be
           assessed as part of a more   SensIral'ovaluation of regional resources@for
           the Imanagement of cardiovascular disease,  Attention should,be directed
                                 need  and priorities established.  Improvement
           to areas of'st6atest
           ,of EKG@servicas :should not be considered alone but should be part of
           a  oordinated  plan for-enhancement of regional health services.









                                                                              32.


          Isolated attempts to upgrade regional EKG capabilities are not con-

          sidered to be appropriate roles for RMPS.


                2..  Regions concerned with upgrading EKG services should consider
          several courses of action.  Depending upon the availability of exist-
          ing resources this may be the establishment of cooperative arrange-
          ments between neighboring hospitals, linkages between local institutions
          and larger medical center complexes, or use of CAE systems.  The
          thorough consideration of all approa  ches, and selection of the' one

          best  suited to fulfill local needs, is of prime importance.


                3.  Institutions and individuals in regions lacking   the necessary
          personnel, technology,  or volume required for support of   an entire
          CAF, system may consider subscription to an established system   outside
          of the re ion if,such 'action@best serves local needs.  Individual
                     9
          RMPs may make available;to interested parties, pertinent information
          related to the existence of  and the capabilities of these systems.
          Such  information should,be accompanied by the following recnim-endations:


                     a.  The CAE system must provide for validation of   individual

          EKGs by an dxperienced,eledtrocardiographer.


                     b.  The service must  be available on a twenty-four hoar a

          day   basis.


                     c.  the services-offered by the CAE system must be compatible

          with the  needs of the..utilizer.









                                                                              3-3.



                     d. Careful consideration must be given  to'all  cost  factors

          including physician validation fees, equipment rental or@purchase

          costs, transmission charges, and maintenance.


               4.  Regions possessing the necessary technological expertise,

          ficilities.and capability for providing a volume of EKGs,sufficient

          to support a CAE systemmay receive RMP assistance.  This may take

          the form of technical consultation, organizational  counseling,  inati-@

          tutional liaison, and participation in the assessment of regional@

          needs, resources, and priorities. RMPs should  not  assume  the  respoh-

          sibility for provision of start-up and maintenance funds.    The potetr!-

          tial profit factor inherent@to CAE systems makes this  the  concern  of

          private industry.  Limited  monetary support b RIlPs may  be  justified
                                                        y

          during the organizational phases of  CAB  system  implementation.  Such

          funding is soacifically,intanded to assist in estalblishing suitable

          working relationships between participants, provide for technical

          consultation,,,and assist in the evaluation of the proposed system.


               5.  EKG iyetems,receiving RMP assistance should possess the

          following characteristics:


                    a.  The service  offered must be compatible@with the needs

          of the region and should  include both unit validation and  twenty.-four

          hour a day availabilit
                                ,y.


                    b.    Priori.ty is to be given to extension of the setvice@

          into the regio n so that small and medium sized institutions may

          ben6fit- computerization of electrocardiographic services within a








                                                                               34.


           medical center complex, serving only  the  components  of  that complex,
           is not,of   the,highest riority., The major objective  of   the EKG,
                                 p
           system@should be enhancing the  qi4ality-and  accessability  of EKG

           services throughout the region.


                     C.  Utilization of  proven hardware  software, and program
           Ming components, so that immediate  benefits  can  be  realized. Emphasis
           is to be directed at,delivery of service rather than research    and

           development.

                     d.  An ade4uate marketing   strategy, i.e.,.Provision  for
           the necessary  volume,' assurance of local support and utilization of
           the sygtemi and A   ser.vice'charg6 compatible with system viability.

                     e.  Total   cost to the patient,, including validation,
           should be comparable,to  the customary..EKG costs of  the region, and

           preferably less.

                6.  RMPS should  encourage  individual RMP,s to seek Assistance
           from acknowledged experts  in  the field during the planning and initi-
           ation phases of any   program. RMPs should act as liaison bet-4een the
           two so that appropr iate technical  consultation  is   available to project

           proposers.

                 7.  With respect  to  extension  of  computer  technology into the
           areas of exercise testing and oft-line critical care unit monitoring,,
           both areas must be considered developmental.  As such,     at the present
           time, they  lie outside of the direct,objectives of RIIPS,   i.e., im-
           proved delivery of medical card.











                                               APPENDIX I

                                    PARTICIPANTS OF AD HOC COMMITTEE
                               ON COMPUTER ANALYSIS OF ELECTROCARDIOGRAMS
                                AND  THE  REGIONAL  MEDICAL  PROGRAMS  SERVICE
                                            November 30, 1971

         Leonard SCHERLISO M.D. (Chairman)                   Samuel FOX,  M.D.
       .Professor  of   Medicine                             President-Elect,
         Read,-Division of Cardiology                        American College.of Cardiology
         School of Medicine                                  George Washington University
         University of Maryland                              Washington, D. C.
         Baltimore, Maryland
                                                             Hubert V. PIPBERGER, M.D.
         Cesar A. CACERESO M.D.                              Chief,
         President, Clinical Systems                         V.A. Research Center for CVDP
          Associates, Inc.                                   V.Ao Hospital
         Washingtoni D. Co                                   Washington, D. C.

         Thomas S. CARDILLOO M.D.                            Leon PORDY, 14.D.
          oj.edt Director                                    Associate Clinical Professor of
          art  Disease   Section                              Cardiology
         ochester Regional Medical Program                   Mt. Sinai School of Medicine
         University of Rochester Medical Center              New York, New York
         Rochester, New York
                                                             Mr, T. Allan PRYOR
         Leonard  DREIPUS,  M.D.                             Intermountain R6gionai'M@@edical
         Clinical Associate Professor                          Program
         Rahnemann Medical College and                       Salt Lake City, Utah
          Hospital of Philadelphia
         Philadelphia, Pennsylvania
                                                             Arthur E. RIKLI, X.D., M.P.HO
                                                             Missouri Regional Medical Program
         Robert V. ELLIOTT, M.D.                             'Columbia, Missouri
         c     ity Electrocardiographic
           Interpretative Service Computer
           Center                                            Ralph SMITH, M.D.
         'Denver  Colorado                                   Director
                                                             EKG Laboratory
                                                             Mayo Clinic
         Gordon E. ENGEBRETSON,   Ph*D.                      Rochester, Minnesota
         Associate Director
         "Cont,nuing   Education                             Mr. Morris WHITE
         Florida Regional Medical Program                    Hewlett Packard Company
         Tampa, Florida                                      Waltham, Massachusetts











                                     PARTICIPANTS  Continued




                              Harold X4RGULIES  M.D.
                              Director
                              Regional Medical Programs   Service


                              Edward  J.  HINM&N, M.D., M.P.H.
                              Director
                              Divisibn of  Professional  and
                                Technical  Development


                              John R. FARRELL, M.D.
                              Chief
                              Clinical.9pecialty  Branchi  DPTD


                              Kenneth  S.   GIMEL, M.D.
                              Health   Consultant
                              Clinical   Specialty Branch, DPM


                              Mr. Earle  R.  BELUE
                              Health   Consultant
                              Clinical   Specialty Branch, DPTD











                      HEALTH SERVICES AND MENTAL HEALTH ADMINISTRATION


                             RWIONAL MEDICAL PROGRAMS SERVICE



                                 AD HOC ADVISOLY COMMITTEE
                             TO REVIEW SUTEMENT ON COMPUT    -ER
                                              CTV.OC
                             ANAI,YSIS OF ilTog    ARDIO@


              Conference Room C     B Wing, 3rd Floor Parklawn Building
                    1000@00 - 3:10 Tuesday, November 10, 1971
                         Chairman:  Leonard Scherlis,, M. D,

                                        AGMA



          I.  Introduction and Statement of Purpose - Dr. Harold Margulies

         11.  Summary of Staff Report on Computer Analysis Of Electrocardiograms

               @(CAS)  Dr. Kenneth   Gimbal
    fail,  Q4egtious Conerning'the Current Functional Status of CAE a stems
                                                                              .y

                  A.  What are the   limitations of CAE systems currently being
                      utilized    with respect to accuracy,, precision and reliability?

                  B,  Can    physician validation of CAE be eliminated?

                  C.  What are the   actual'time savings in physician man-hours?
                      (Also in Allied Health and Clinical)

                  D,  Under what conditions are current systems cost-effective?
                      What are the total unit,coots?

                  E@, @what services should be considered essential     to the operation
                      of CAE?  (Optimal turn-around time; interval      between analysis
                      and
                          validationo@24 hour day availability?)

                  F.  What art practical@locatidno for the Acquisitioo units?
                      What it minimum volume to be cost effective

                  G*  What are alternatives for delivery of improved alectro-
                      cardiographic services in areas of need.

                  R.  What is the role of private entmriso,in the development
                      and Monitoring of   CAS?







                 1. Have CAE- programs met with acceptability from-the physician
                     community in which they are provided?  In your practice, do
                     you accept CAE without further verification?
                Jo, In existing programs, what percentage of operation can be
                    considered -research and development?
                K,  Under what conditions are CAE justified even if unit cost
                    exceeds cost for the same service in the same area?
                L.  Under what conditions would you use CAB as a screening tool?

       IV.  Additional questions or presentations (10 minute limit) from participants,

       V.  AdJourn