rheumatologic diseases manifest themselves in varying combinations of symptoms and signs , particularly at early stages , and therefore make differential diagnosis a challenge , especially for nonrheumatologists including general practitioners . since diagnosis at an early stage and adequate treatment improve prognosis , assistance in establishing diagnosis is desirable . given the substantial progress in computer science in the last years , the idea of computers taking the role of diagnostic support is not far - fetched . software applications have affected decision processes in clinical routine , for example , in controlling depth of anesthesia   or in detecting drug interactions . software tools to support physicians in the diagnostic process have been developed in almost every field of medicine . a widely utilized type is the so - called expert system , defined as artificial intelligence program designed to provide expert - level solutions to complex problems . there are three different approaches to knowledge - based systems depending on the form of knowledge representation : rule based , case based , and model based . in rule based reasoning , the knowledge is expressed by rules , often if then rules . the rules can be newly developed or can be extracted from decision tables or decision trees . in case of based reasoning , the inference engine searches the knowledge base for similar cases . finally , models , that is , biochemical or biophysical , can also form the knowledge . the approaches to intelligent computing systems are artificial neuron nets , genetic algorithm , and fuzzy systems . artificial neuron nets are built like biologic intelligent nervous systems and are regarded as learn - like . genetic algorithm mimics the process of natural evolution and is mainly used in search processes . fuzzy systems are usually based on rules , but the reasoning is approximate to cope with uncertainty and imprecision because the rules are given varying truth - value using fuzzy sets . thus , linguistic certainty or frequency levels , such as probable or seldom , derived from medical texts or experts can be incorporated into the knowledge base . the probability of a diagnosis is calculated with the accuracy of a test or a clinical finding and the prevalence of the disease . different methodologies are often combined , which are then called hybrid expert systems . already in 1959 , ledley and lusted anticipated the use of computers in supporting decisions and proposed different mathematical models to emulate the reasoning in medical diagnosis . they were archetypes for following expert systems , but they also demonstrated the challenges in the development of such tools . it was used for diagnosis and therapy of bacterial infection and has become the probably best - known expert system in medicine . it was designed to assist physicians in the diagnosis of complex and multiple diseases in internal medicine covering more than five hundred diseases . the problems encountered in developing internist-1 and its successors showed that a comprehensive knowledge base is needed for a correct diagnosis of complex diseases in internal medicine . several somewhat outdated review articles explored the development and application of expert systems in medicine in general [ 4 , 10 , 14 ] . in 1991 , bernelot moens and van der korst reviewed the literature assessing computer - assisted diagnosis of rheumatic diseases . meanwhile , also in rheumatology new expert systems have emerged and earlier expert systems have been improved to meet the many demands of modern rheumatology : establishing an early diagnosis with the highest probability to allow for a better outcome with the help of a prompt treatment . besides an overview of characteristics , comprehensiveness , and validation of existing diagnostic expert systems in rheumatology , this systematic review seeks to point out whether the current expert systems fulfill the expectations of clinicians in daily practice and finally what the characteristics of an optimal system would be . the research questions were as follows : what information is currently available on diagnostic expert systems in rheumatology , how do these systems work , what is their validity and their applicability in daily practice , and finally what is an optimal diagnostic expert system expected to be . in the optimal scenario we anticipated to find comprehensive reports on each individual diagnostic expert system including information on the precise diagnostic algorithm , the targeted diseases , a well - described validation cohort , and predictive values for diagnostic performance . the data would allow for a statistical comparison of the expert systems . in a suboptimal scenario , only descriptive reports of expert systems will be found allowing for a comprehensive overview of the past developments without statistical comparability . medline , embase , and cochrane library were searched using the following medical subject heading ( mesh ) terms : rheumatic diseases ,   rheumatology ,   arthritis ,   computer assisted diagnosis ,  and  expert systems . all publications referring to diagnostic expert systems in rheumatology or in a rheumatic subfield were included . reviews , editorials , and literature which described an expert system only used for education of healthcare providers and therefore not used in diagnostics were excluded . also , literature that referred to an expert system used for identifying solely the stage of a disease and hence not used for diagnosing a disease itself was excluded . records which described an expert system applied only to image analysis literature referring to data mining strategies using index diagnoses or solely epidemiological variables was excluded as well . figure 3 shows a flow diagram of the selection of studies . in case of uncertainties , inclusion or exclusion year of the last update of the system , number of considered rheumatic diseases , targeted diseases , information to feed the expert systems ( history , clinical exam , laboratory analyses , and imaging studies ) , methodology of the inference mechanism , and embedding of accepted disease criteria sets such as the american college of rheumatology ( acr ) or the european league against rheumatism ( eular ) criteria were extracted using standard forms . for the description of the validation method and the performance , the following information was extracted from the articles : number of cases used for the validation , determination of the resulting diagnosis , identification of the correct diagnosis , the reference diagnosis , percentage of correctly identified cases , sensitivity and specificity , positive predictive values , negative predictive values , positive likelihood ratio , and negative likelihood ratio . one record describing an expert system developed solely for education   and one record referring to an expert system that was not designed for clinical use   were excluded . six reviews or editorials were excluded [ 6 , 15 , 1922 ] . in the case of repeated reports , either the original or the more comprehensive article was included in the evaluation leading to a final number of 38 original articles ( figure 3 ) . in these 38 articles , 25 different expert systems and their successors or further developments are presented . most of the articles shown in this review presented the development and the methodology of expert systems . the number of considered diseases varies from one to 170 . both the amount and the nature of information to feed the expert systems vary according to the targeted disease group and inference mechanisms . the following methodologies of expert systems were observed : rule based , case based , model based , artificial neuron nets , fuzzy systems , bayes ' theorem , and other not further described algorithms or calculation tools ( figure 1 ) . 19 of the 25 expert systems ( 76% ) were validated . the number of cases used for the validation varied widely between 32 real cases and 12  000 simulated patients . different units of measurement were selected to report the performance of the expert systems , mostly the percentage of correctly diagnosed cases , sensitivity and specificity . the proportion of correctly diagnosed cases  the diagnostic accuracy  was between 43.1 and 99.9% . values for sensitivity and specificity ranged from 62 to 100 , and 88 to 98% , respectively . positive or negative predictive values and likelihood ratios were only surveyed for two expert systems [ 26 , 27 ] . binder et al . showed a positive likelihood ratio of 12.1 ( 95% ci 7.7019.1 ) and a negative likelihood ratio of 0.187 ( 95% ci 0.0990.351 ) . excluding this last report the reference standards were chosen differently : diagnoses according to established criteria , consensus diagnoses , discharge diagnoses , and diagnoses provided by a rheumatologist were used most often as reference . three expert systems presented certain criteria for the determination of the resulting diagnosis when several diagnoses were presented as a result or when a probability value was added to the diagnosis . an article that reports on the applicability of a rheumatological expert system in clinical routine could not be identified in the published literature . first , an overview over 25 different diagnostic expert systems designed for rheumatology is given . second , it is shown that the different designs and validation methods of the expert systems hinder the comparison of their performances . third , we found no publications reporting on the routine application of an expert system in rheumatology . artificial intelligence has achieved enormous progress in its development and computers have outclassed human beings in various fields , such as computer chess or ibm 's watson winning on the quiz show jeopardy !  given this progress in technology and the time period covered by this systematic review of over forty years , the low number of identified expert system is surprising . the reasons would be either low interest in supportive software or , more likely , the difficulties encountered in simulating the complex human diagnostic process . reported that developers of knowledge - based systems regarded pharmacovigilance , intensive care monitoring , and support for guidelines and clinical pathways as the most promising fields of knowledge - based systems . in other words , systems covering clearly defined decision rules or comparing databases . first of all , nonspecific findings occurring in multiple rheumatic diseases are common and consequently complicate the knowledge representation in expert systems . second , there is a lack of epidemiological data concerning the prevalence and incidence of rheumatic diseases as well as sensitivity and the specificity of single findings in diseases . third , even if available for a large population , such data vary greatly amongst ethnic groups and regions becoming an increasing problem in times of global migration . fourth , for many of the disease - specific findings , there are no internationally established standardized cut - off values . and finally , many rheumatic diseases can coexist with each other in overlap syndromes . nevertheless , the growing understanding of diseases and the corresponding findings or symptoms will facilitate the representation of medical knowledge and decision processes in the future . in consequence of the variation in the method of validation , first , the result of the expert systems to be compared with the reference diagnosis was presented in different ways . some expert systems indicated a probability value of the calculated resulting diagnosis , and others present a hypotheses list . final diagnoses in rheumatology often remain descriptive or incomplete and evolve over time as many of the rheumatic disorders present atypically and do not completely fulfill a diagnostic criteria set at the beginning . this issue is met by the presentation of the results as a hypotheses list or probability values , which can , as an important advantage , multiply the user 's own differential diagnosis and lead to more focused testing . yet for example , the diagnostic accuracy is erroneously high if a diagnosis at a low position in the hypotheses list or a diagnosis with a low probability value is accepted as a correct resulting diagnosis during the validation process . second , there is a lack of widely accepted reference standards for the correct diagnosis to compare the resulting diagnosis with . some authors used diagnoses in medical records or discharge diagnoses as a comparator assuming the correctness of their peers , some chose the consensus of rheumatologists , and others used diagnoses according to official diagnostic criteria sets . the latter is probably the most reliable way ; however , even if international consensus criteria exist , there are still many different criteria sets especially for rare diseases where the superiority of one set over the other and in particular the threshold for a diagnosis remains a matter of debate . in addition , many of these criteria sets were established to obtain homogenous cohorts in clinical trials leading to a low sensitivity in early or mild disease . another approach was the assessment of the interobserver variability by hernandez et al .   and martn - baranera et al . . here , the distance between the resulting diagnoses of clinicians and renoir was calculated without setting a reference diagnosis . by this means the uncertainty of the final diagnosis and the error proneness of clinicians were taken into account . the transferability of expert systems to the general population ( the external validity ) can be tested with a validation in a developer - independent clinical setting . only ai / rheum , cadiag , and rheuma [ 29 , 39 , 40 , 49 ] were validated this way , resulting in a lack of data on the transferability to daily practice of most of the presently available expert systems . besides the internal and external validity , the following features are , according to kawamoto et al . , highly associated with an expert system 's ability to improve clinical practice : the availability at the time and location of decision making , the integration into clinical workflow , and the provision of recommendations rather than a pure assessment . the wider use of computers in clinical routine , such as the possible use of tablet computers on ward rounds , will facilitate the integration into clinical workflow and enhance the availability at the time and location of decision making . the need of more detailed documentation for quality assurance may have a positive influence as well . are the only authors who reported the clinical use of their diagnostic expert system cadiag-4/rheuma - radio . the expert system was incorporated in the medical information system of the respective clinic . for the lack of accessibility of diagnostic support , the universally present search engines for the world wide web have become a popular alternative with an astonishing accuracy as shown by tang and ng   and lombardi et al . . and schewe and schreiber   regarded the time required for data input as the most limiting factor . considering the smaller amount of input data and consequently the shorter input time , specialized and restricted expert systems like the laboratory results analyzing system presented by binder et al . kaplan   presented a system with a provisional hypothesis list , which updates after every further input . here , the data input is limited ; hence , there is a risk of missed diagnoses due to the less thorough questioning . the required time for data input would decrease if the expert system was compatible with the institutional medical information system and consequently could allow direct access to all electronically stored patient data comprising patient history , physical exam , imaging studies , and laboratory analyses . . then again the data input and the required time depend on an intuitive user interface , which boegl et al . believed to have the biggest influence on the clinical success . the reason for the absence of expert systems in clinical use hitherto has been discussed in detail in the literature . mandl and kohane claimed that health information technology in general was in arrears compared to other industries . also they took the health information technology products as too specific and incompatible with each other . they stated that the lack of acceptance by the medical staff is the main problem in the application of knowledge - based systems in medicine . the different points of view of developers and clinicians show that a better cooperation is necessary . expert systems have to be adapted to clinical problems and to clinical workflow . on the other hand , clinicians should become more aware of the supportive possibilities of expert systems . in spite of computerized assistance , the user of the expert system needs rheumatologic fundamentals for the detection and the correct description of rheumatologic findings . cadiag , ai / rheum , renoir , rheumexpert , and mesicar were specifically developed for the assistance of nonrheumatologists [ 31 , 37 , 39 , 48 , 50 ] . these systems were designed to remind the nonspecialist of rare diseases or to indicate the cases which needed immediate treatment . yet , an expert system 's outcome highly depends on the entry of correct parameters . therefore , educational parts were added to some of the expert systems to increase the user 's diagnostic skills . these educational parts explain certain symptoms or show photographs of findings [ 30 , 42 , 51 ] . also , some systems provided a link to literature , such as medline , for further information [ 30 , 42 ] . a widely accepted system ideally covers the demands of generalists and specialists offering an easy understandable handling and not being too basic at the same time . the integration of widely accepted diagnostic criteria sets such as the acr or eular criteria into the diagnostic process would increase the acceptance and credibility of an expert system . it also reduces the influence of individual diagnostic strategies of the developers . nevertheless , only six of the identified expert systems reported the integration of such criteria sets into their expert database . the downside of diagnostic criteria originating primarily from classification criteria for the inclusion into clinical trials , however , is the generally low sensitivity in early disease . this insensitivity of some criteria , such as the 1987 ara criteria for rheumatoid arthritis , forced leitich et al . to modify the criteria using fuzzy sets to gain different levels of sensitivity . the recent development of official diagnostic criteria , which are more dedicated to the diagnosis in an early stage of the disease , will make their use in the design of expert systems more attractive . furthermore , some methodologies are ill suited to the use of diagnostic criteria , such as a mere probabilistic approach like bayes ' theorem or artificial neural network . these systems extract their knowledge base from patient data , such as symptoms and clinical findings , and the corresponding diagnoses assuming a correctness of the chosen diagnosis . diagnostic criteria can not be included in these systems without the combination with another methodology or an adaption of the reasoning process like the review of symptom weighing . other ways of knowledge representation facilitate the usage of official diagnostic criteria , like rule - based reasoning though the minority of the articles presenting a rule - based expert system reported an integration of official diagnostic criteria . although a thorough systematic search has been performed in the most relevant databases , some reports could have been missed if written in other languages than english or german . as most of the current literature is published in english at least as an abstract , we are confident that we did not miss relevant articles on diagnostic expert systems in rheumatology . the number of expert systems which have remained unpublished because of their expected commercial use or the abortion of the system at an early stage is hard to estimate . the reported expert systems showed a great variety in diseases spectrum , methodology , and validation status . this made a statistical comparison of the systems impossible . and finally , the important topic of patient reported outcomes which are of increasing importance not only in clinical trials and patient 's follow - up but also in the diagnostic process was beyond the scope of this review . in conclusion , this systematic review shows that the many attempts made for an ideal expert system in rheumatology in the past decades have not yet resulted in convincing validated tools allowing for reliable application in daily practice . nevertheless , the demand in support by expert systems is pressing as the knowledge about the rheumatic diseases increases and the therapeutic options especially in early disease stages are growing constantly . an ideal diagnostic expert system in rheumatology the expert system would allow for universal integration into the clinical workflow as well as rapid and intuitive data input . since rheumatologic diagnoses can not always be definite , the resulting diagnosis would have a probabilistic grade to indicate uncertainty . the system would also have an educational component to improve the nonexpert 's ability to recognize pathological findings . finally , accepted diagnostic criteria sets would be applied to increase the general validity of the system 's diagnostic process . based on the demand of such a tool and the progress made hitherto it seems to be a matter of time until new and promising expert systems enter clinical practice .
background . 
 the early detection of rheumatic diseases and the treatment to target have become of utmost importance to control the disease and improve its prognosis . 
 however , establishing a diagnosis in early stages is challenging as many diseases initially present with similar symptoms and signs . 
 expert systems are computer programs designed to support the human decision making and have been developed in almost every field of medicine . 
 methods . 
 this review focuses on the developments in the field of rheumatology to give a comprehensive insight . 
 medline , embase , and cochrane library were searched . 
 results . 
 reports of 25 expert systems with different design and field of application were found . 
 the performance of 19 of the identified expert systems was evaluated . 
 the proportion of correctly diagnosed cases was between 43.1 and 99.9% . 
 sensitivity and specificity ranged from 62 to 100 and 88 to 98% , respectively . 
 conclusions . 
 promising diagnostic expert systems with moderate to excellent performance were identified . 
 the validation process was in general underappreciated . 
 none of the systems , however , seemed to have succeeded in daily practice . 
 this review identifies optimal characteristics to increase the survival rate of expert systems and may serve as valuable information for future developments in the field .