1.2.3 Limitations of the Study

In the light of the above, it must be recognized that the research findings can not be extrapolated to all South African organizations. Only organizations who are known to be involved in or at least informed about ES technology were targeted for this research.

In addition, the general lack of empirical research data may hamper the ability to draw statistically meaningful comparisons. In particular, the lack of a previous large scale South African survey prevents the identification of historical trends. Hopefully, this study will remedy this situation.

Finally, it is conceivable that isolated "pockets of ES excellence" may have been missed through the selection of a particular sampling frame (see 3.4). That this is not inconceivable, even in the South African context, can be illustrated by the following (paraphrased) comment from ES consultant K.Carden [1992]:

"There may be things going on that hardly anyone knows about. For instance, everyone working in the [ES] field was surprised at the last [Expert 90] conference when a couple of speakers turned up to present a big, quite advanced system while no one even suspected that their organization was active in ES. And their work was really good, by any international standards. It was not that they had kept it specifically secret or so, they had just been quietly working on their own."

1.2.4 Objectives of the Research

The objective of this research is to obtain an up-to-date assessment of the status of ES usage and application in South African organizations.

The following specific aspects with respect to the ES use will be investigated.

- Demographic analysis of organizations engaged in ES activities: main activity, size and location;

- Level of ES activity: ES in planning, development or production stage.

- Nature of ES: brief description, primary activity and size.

- Usage level: length of time, frequency or reason for non-usage.

- Development concerns: who proposed and who developed, development investment, hardware platform, software tools, whether prototyping was used, knowledge acquisition process.

- Evaluation of ES: overall system success, main benefits and problems experienced.

- Opinions in respect of the future of ES technology.

- Miscellaneous issues: interface with other systems, strategic ES.

Specific attention will also be given to reasons for non-usage of ES technology in an attempt to identify barriers or resistance factors.

1.2.5 Research Hypotheses

Two primary research hypotheses are postulated in this research.

First primary research hypothesis H_I:

"Generally, very little ES activity in terms of system development and systems in actual use, is taking place in South Africa."

Second primary research hypothesis H_II:

"The general level of ES activity in South Africa is lower than that overseas"

In order to allow these hypotheses to be tested empirically, a number of sub-hypotheses will be formulated in 3.3 relating to the expected quantity, size and complexity of ES, the diversity of application areas, the type of problems and benefits experienced with the technology, the way in which development takes place, the general usage rate of the systems, and the type of barriers expressed.

1.3 RELEVANCE OF THIS RESEARCH IN THE SOUTH AFRICAN CONTEXT

From an academic point of view, this research can be seen as a natural progression of other South African theoretical [Mentz, 1988; Jacobson, 1989] and empirical [Keating, 1991] postgraduate research on ES. In particular, it must be emphasized that this is the first local in-depth survey of its kind and it will hopefully serve as a sound scientific basis to evaluate future developments in the ES area. Also, it will provide a foundation for further, more focused empirical research efforts.

To the practitioners this research may serve as more objective basis against which to assess the many subjective and often contradictory (refer 1.1) statements which are expressed in the literature. To a lesser extent it may also be seen as a more objective market research against which claims of vendors may be viewed. To ES users it will provide a standard against which they can measure their current situation. The need for this type of research was shown by the high degree of interest expressed by those practitioners who were initially contacted to assess the feasibility of this research: they all insisted on getting a full copy of the report.

However, the potential benefits of ES are of such a nature that society at large will also benefit from a scientific assessment. The following benefits of ES must be seen within South Africa's specific context.

This country has a dramatic dearth of highly skilled manpower in the areas of management and technical staff. Researchers at the University of Stellenbosch estimate an annual shortfall of 3500 managers and ten times as many technical staff [Anon., 1988] which reputedly constrains economic upswings by a lack of decision making abilities and technical skills. This may also contribute to South Africa's generally wide span of managerial control. South Africa's geographic isolation, complex educational problems and relatively small but sophisticated economy (from a global perspective) further compounds the relative scarcity of top experts in almost any given technical sub-domain. And although one must caution against the "cure-all" syndrome, proper implementation of ES technology could at least partially alleviate this shortage:

"Not only can it extend expert performance throughout an organisation or industry despite having lower skilled staff doing the work, but it can also bring novices up to speed faster." With the cautionary note: "These are some of the promises of knowledge-based systems. It will still take faith, dedication and money to turn the promises into reality." [Anon., 1988:46]

The management potential of ES could aslo seen to be illustrated in a perhaps more humorous way by an experiment whereby a management expert system developed by professor Duffy (University of the Witwatersrand) beat all ten teams of "real, human" businessmen in a business management game [Anon., 1989].

Another relevent aspect concerns the low growth in white collar productivity, despite the dramatic increases in corporate IT budgets. The problem seems to be one of information overload: more and more data is being delivered by the information systems but they do not aid in the consequent decision making. This is where ES have an important potential role to play: to automate the routine decision making process [Buckler, 1990b].

Further important considerations in motivating this research are intense competition for allocation of resources in the light of the many needs of a developing country. It is imperative that lessons be learnt from both successful and unsuccessful projects. South Africa is fortunate enough to be on a relatively high level in terms of computer technology and most organizations could thus take immediate advantage of ES opportunities without necessarily having to be on the leading (bleeding?) edge of the technology.

Lastly, the timing appears to be ideal for this research:

- The ES technology has reached a relative level of maturity and stability (although it is still developing fast). This is evidenced by the availability of a large number of development tools (some of South African origin), the wealth of overseas and local experience and a large body of both theoretical and practical literature.

- There seems to be less secrecy surrounding corporate ES projects than in the initial stages when immediate strategic advantages were expected.

- A sufficiently large amount of ES activity ("critical mass") is happening to draw meaningful conclusions, but not so much that the field becomes unmanageable.

- The technology is generally still "new" enough to enable clear identification of ES projects: system integration make soon advance to a stage where the distinction between ES and other information systems becomes too blurred.

- It is still early enough to learn from mistake.

1.4 STRUCTURE OF THE RESEARCH REPORT

Chapter 2 will be devoted to a literature review which will serve to identify the relevant issues for the survey. It will also amplify on the decision support aspects of ES. A very significant effort was made to identify as much previous empirical research as possible in order to take stock of their results and provide a basis for comparison.

Chapter 3 is concerned with the methodology of the empirical component of the research which consists of an initial exploratory phase (semi-structured interviews with a small number of practitioners) and a structured survey conducted by the mailing of a questionnaire.

Chapter 4 provides the initial survey results in the form of descriptive and comparative data analysis. This entails both global response analysis as well detailed question-by-question analysis in table and statistical test format. Some additional correlational analysis will also be performed.

This analysis will be validated and further interpreted in chapter 5 in terms of each of the research hypotheses as formulated in chapter 3. Attempts will be made to explain possibly unexpected results or apparent differences with other surveys.

Chapter 6 will conclude with an assessment of the overall achievement of the stated research objectives. The more important findings will be summarized, some limitations of the research re-iterated and suggestions made for possible future research.

The report also includes a list of references and a number of addenda which contain the full questionnaire, a provisional database of South African expert systems, detailed calculations in respect of the statistical chi-square and Spearman's rank correlation tests and a sample of comments made by respondents.

2.2 DEFINITION OF EXPERT SYSTEMS

As with many scientific disciplines, there is some difficulty in establishing a really satisfactory definition that is neither a tautology or self-referential.

For example, consider the classic ES definition proposed by Feigenbaum [Giarratano & Riley, 1989:1]:

"[An ES is] an intelligent computer program that uses knowledge and inference procedures to solve problems that are difficult enough to require significant human expertise for their solution."

This definition falls back on the "human expertise" concept which may turn out to be an even more elusive concept. Various other, similarly phrased definitions are mentioned in Keating [1990], but the fact remains that

"No one yet has a really satisfactory definition. [...] Perhaps expert systems have to be defined by their characteristics." [Anon., 1985:1]

Such a definition is proposed by Buchanan & Smith [1989:151]:

"An expert system is a computer program that:

a. Reasons with domain-specific knowledge that is symbolic as well as numerical (i.e. a knowledge-based system).

b. Uses domain-specific methods that are heuristic (plausible) as well as following procedures that are algorithmic (certain).

c. Performs well in its problem area.

d. Explains or makes understandable both what it knows and the reasons for its answers.

e. Retains flexibility."

The problem with definitions based on characteristics is that they tend to become rather long. Hence, the following definition has been adopted for purposes of the survey, notwithstanding the fact that it also refers to "human experts".

"Expert systems will be understood to be computerized advisory programs that attempt to imitate or substitute reasoning processes and knowledge of experts in solving specific types of problems." [Turban, 1990]

Although this definition can be subjected to some other criticism, it has been selected because it is readily understood by IS people and users and it does not refer to the technical aspects of the system (e.g. inferencing, heuristics etc.). As Turban points out, this definition is a content-free expression, i.e. it means different things to different people.

A final remark concerns the emergence of the term "knowledge-based system" (KBS). Initially it has been treated in the bulk of the literature as a synonym of ES, although perhaps more connotation-free in the commercial ES market place. There appears to be a recent tendency to increase its scope slightly to systems that perform tasks that do not require "experts" to perform, although they do require a certain level of competency in a specific problem domain [Methlie L.B., 1987:337]. Because of these differences in usage, the term KBS has been avoided as much as possible in this research.

2.3 EXPERT SYSTEM FUNDAMENTALS

Most of the material that follows is gleaned from Buchanan & Smith [1989], Turban [1990], and Giarratano & Riley [1989], unless indicated otherwise.

2.3.1 Structure of an ES.

The traditional three components of an ES are:

- the knowledge base which contains rules ('heuristics' or logic) and facts (problem situations, appropriate questions etc.) about the problem domain;

- the inference engine (the "brain" or "control centre" of the ES) which draws conclusions, recommends and motivates actions, etc. based on the information given by the user and the information in the knowledge base;

- the user interface which is the mechanism by with communication between the ES and user takes place - this can be in the form of structured text (menus or simple questions: True/False, Multiple Choice, One-word or number answers), unstructured text ("natural language interface") and other graphics or diagrammatic formats.

In practice, even an only moderately sophisticated ES will have a much more complex structure as illustrated in figure 1.

2.3.2 Categories of ES

2.3.2.1 Based on Technical Characteristics.

Traditionally, a distinction could easily be made on basis of the programming paradigm employed to develop the ES: i.e. rule-based versus model-based systems. The arrival of object-oriented programming resulted in yet another type of ES. As other technologies are being developed and incorporated into hybrid ES (neural networks) or systems become fully integrated with traditional systems (refer 2.5), the usefulness of this classification is decreasing.

Various other technical criteria can be used to classify ES: the nature of the software tools (developed using a shell, "traditional" or "AI" programming language), the hardware platform, the knowledge representation method etc. However, these typologies are not very useful from an IS point of view.

2.3.2.2 Based on Problem Area.

Perhaps more helpful is the widely used typology that was originally proposed by Hayes-Roth F., Waterman D.A. & Lenat D.B. [1983:13-16]. It is based on the type of problem that is being addressed by the system (figure 2). The problem with this classification is that some ES can perform two (or even more) functions simultaneously:

"Although the basic activities of ES are easy to describe, it is misleading to use them to categorize existing ES because many ES perform more than just one activity. For example, diagnosis often occurs with debugging, monitoring with control, and planning with design." [Waterman, 1986:39]

A more comprehensive list of various problem areas was proposed by Buchanan & Smith [1989]. It expands materially on the categories listed in figure 2, but divides them into two major classes: systems that interpreted data to analyze a situation (data interpretation, equipment diagnosis, troubleshooting, monitoring etc.) and systems that construct a solution within specified constraints (planning, scheduling, therapy management, design, configuration). There is here again no clear, unambiguous taxonomy of problem types that is independent of the actual methods used to approach the problems. Consequently, their categories are not entirely distinct.

2.3.2.3 Based on Application Area

Another commonly used classification scheme is based on the application areas: agriculture, chemistry, computer systems, electronics, engineering, geology, information management, manufacturing, mathematics, medicine, meteorology, military science, physics, process control and space technology [Waterman, 1986:40]. Unfortunately, it is sometimes difficult to decide exactly when an application area is sufficiently large to warrant separate a listing. This is illustrated by Hu's list of only five application areas: industrial, medicine, military, information management, law and agriculture. [1987:186].

2.3.2.4 Based on System Complexity

A novel, but empirically verified typology is proposed by Meyer & Curley [1991]. Their classification scheme attempts to determine the overall ES complexity based on the following two dimensions: the complexity of knowledge and the complexity of the technology [Figure 3]. Each dimension is measured using a fairly elaborate index:

Knowledge Complexity = f(domain breath, domain depth, rate of domain change, domain penetration, system outputs, breadth of information inputs, ambiguity of information inputs).

Technological Complexity = f(diversity of platforms, diversity of technology, database intensity, network intensity, KB programming effort, diversity of information sources, diffusion, systems integration).

This typology has a particular usefulness for IS managers. ES projects can be differentiated and compared across different applications and industries. IS management concerns seem more directly related to system complexity that to most of the other project variables. Appropriate management strategies and considerations can be identified for each quadrant.

2.3.3 ES Development Life Cycle and Knowledge Acquisition

A particular IS concern with ES development has been on how they fit in with the other systems.

It has been suggested that the traditional System Development Life Cycle (SDLC) [e.g. Whitten, Bentley & Barlow, 1989] approach does not usually work satisfactorily with an ES project and that therefore an ES specific SDLC needs to be adopted [Weitzel & Kerschberg, 1989]. The linear model in figure 4 summarizes the position of Giarratano & Riley [1989]. Although the overall steps resemble those of more conventional SDLCs, the detailed component tasks and objectives are very different and specific.

 

Turban [1990] on the other hand, argues that fairly standard SDLC methodologies can still be followed, regardless of the nature of the system being developed. However, even he acknowledges that the exact content of certain steps may be unique to ES development.

This is illustrated by perhaps the most important step in the ES SDLC: the knowledge acquisition process. Often, this task is performed by a specialist: the knowledge engineer (KE). The following methods are listed by Turban [1990, pp.458-475]:

Manual Methods

- Interview analysis: direct dialogue between the (subject) expert and the KE. This is the most common and one or more of the following methods may be used:

. working through example problems (cases);

. protocol analysis (step-by-step documentation of decision-making behaviour;

. discussion of cases in context of an ES prototype;

. the Repertory Grid Approach;

. directed interviews (usually supplementing the above);

. informal interviews (useful in the early stages).

- Observation of the expert at work, both physical (motor) performance as well as eye-movement can be tracked.

- Use of a questionnaire and/or organized expert's report (useful when the expert is not directly available to the KE).

- Analysis of documented knowledge such as electronic databases and domain-specific literature.

Computer-Aided Methods

Many ES development tools now incorporate automated rule induction mechanisms which can be used to create rules automatically, based on a large number of case studies. There are still a number of problems associated with these methodologies, not the least of which is the fact that those rules may seem completely unintelligible to humans.

A concern related to the SLDC is the fact that many computer-aided software engineering (CASE) and structured methodology tools have been designed for procedural languages. This causes problems, particularly when trying to design integrated mainstream systems that contain ES modules. However, it appears that newer CASE methodologies, especially those supporting object-orientated concepts, require less of an ad-hoc "patchwork" work-around. [Blackman, 1990].

2.3.4 Other ES Considerations

A primary consideration in the construction of the knowledge base is the representation of knowledge. The various approaches range from semantic nets, schemata and frames to propositional logic and objects. Giarratano & Riley [1989] give an excellent overview complete with examples and critique. In practice, it is found that larger systems use a combination of representation formats to combine power and speed.

Similarly, there are various inference methods available for the inference engine. The distinction between backward and forward chaining is perhaps the most familiar, although numerous other considerations such as the problems of resolution and meta-knowledge (i.e. knowledge about knowledge or the system itself) must be considered.

Finally, as ES will become more prominent in management decision-making areas, more attention will be given the various ways in which uncertainty is represented and manipulated. Kopsco [1988] gives a good overview of advantages and drawbacks of the available techniques (Bayesian approach, certainty factors, fuzzy logic, Shafer's belief functions) in the light of the requirements of management information systems.

2.4 EXPERT SYSTEMS AS DECISION SUPPORT SYSTEMS

2.4.1 Introduction and Definitions

Although the linkage between ES and Decision Support Systems (DSS) has been examined in an earlier research report [Mentz, 1988], the IS focus of this research and recent developments in the field require that at least a overview of the issues be given here as well.

Since virtually all ES provide support for decision-making in one area or another, every ES can be seen as a DSS in the wider sense of the word. However, in the IS literature, the term DSS has acquired a more specific meaning:

"A DSS is a computer-based system used by managers as an aid to decision making in semi-structured decision tasks through direct interaction with data and models." [Benbasat & Nault, 1990:203-204]

The important element of this definition is that it limits the scope to managerial environments. What motivates this distinction with the large number of ES in more technical or scientific areas?

"[Developing an] ES to produce a diagnosis the way a medical expert does, poses a very different set of problems than designing an expert system to replicate some aspect of managerial decision making." [Agarwal, 1990:126]

This is specifically illustrated in the discourse by Kim & Courtney [1988] where they demonstrate that the problem domain of management is much wider and shallower than that of most ES.

2.4.2 The DSS context

To place DSS in a wider context, the diagram in Figure 5 or one similar to it is often used: it illustrates that DSS are typically being used on the higher organizational (i.e. managerial) levels. More recent sources will typically add another - the strategic - level to incorporate Executive Information Systems (EIS) explicitly.

However, this theoretically attractive view often clashes with the reality: knowledge workers at all levels within the organization need decision support. In practice, the DSS may be seen to be useful right across the different management levels of the organization as is illustrated in Figure 6. This view suits the application of ES better since many managerial ES are in fact employed on the lower hierarchical levels of organizations, were the problem domains are better bounded (see infra).

 

2.4.3 Similarities and Differences Between DSS and ES

Although DSS and ES technologies appear very different at first sight, the difference between the two types of systems is perhaps more one of nomenclature than of a structural or technical nature. A closer look at the different structural components of both ES and DSS reveals that there are areas of substantial overlap. E.g. both have a strong inferencing mechanism, although a different emphasis may be given to exactly how the inferencing is done: more heuristically in ES as opposed to a more mathematically in DSS [Finlay, 1990].

In fact, the empirical study by Doukidis [1988], discussed in 2.7.4.2, shows clearly that most ES employ a substantial number of DSS concepts, especially from a technical viewpoint.

Given the nature and purpose of managerial ES, can they then not be considered to be some kind of DSS, or to put it another way:

"Inevitably, the question arises: Is an ES a DSS? There are a number of number of differences [...]. Expert systems deal with problems whose scope is narrow and relatively well defined. The system incorporates a set of rules and heuristics that are repeatedly used in the solution of the problem. The rules and relationships change with experience. Typically, an expert system has the ability to explain why it reached a conclusion. In contrast, a DSS is intended to operate in a broader and more diverse decision environment. It should be usable for ad hoc problems but usually does not incorporate a facility for explanation." [Kopcso, Pipino & Rybolt; 1988:67]

These differences are neatly listed in the figures 7 and 8. In respect to figure 7 it must be noted that advances in ES technologies have widened the scope of the problem domain, and the objective of many managerial ES is also to assist rather than replace the decision maker. The power of figure 8 lies in the wide comparison of attributes of a wide spectrum of computerized systems, including the EIS.

Many other authors have discussed the relationships between DSS and ES along similar lines. Some of the more original approaches can be found in Holtzman [1989] who offers a useful framework for deciding the most appropriate technology; and Benchimol, Lévine & Pomerol [1987] who discuss the contributions which the technologies made to each other.

2.4.4 Towards an Integration of ES and DSS

The real question is not what the differences or similarities between the two technologies are, but rather how they can best be utilized to improve decision making. Many authors suggest that the integration of the two technologies into what is called a Management Support System (MSS) is the solution.

Turban [1990] provides a comprehensive summary of the benefits that can be expected from such an integration (figure 8) and suggests a number of conceptual models in which this integration could be achieved.

- ES can attached to specific DSS components to make the subsystems of the DSS more "intelligent".

- ES can form a separate DSS component and take either its input from the DSS, provide its output to the DSS, or do both in a feedback loop.

- The ES can complement the DSS by supporting one separate, clearly identifiable link in the decision-making process. This step is typically one of the last ones which require judgement and creativity.

- The ES is used to generate alternative solutions which can then be explored using the existing DSS.

- The ES and DSS merge completely in a new, unified architecture.

Although the fully integrated models seem conceptually attractive, practical implementations are relatively difficult to realize. This underlies Liang's [1990] motivation for using the term Expert Support Systems (ESS) for ES which support managerial decision making in a more consultative way than the traditional ES. The ES and DSS differ in assigning the responsibilities to human versus computer in terms of supplying, modifying and managing the four categories of data, procedures, goals/constraints and strategies; ESS assign a joint human-computer responsibility for each of these categories [Benbasat & Nault, 1990]

2.5 RECENT TRENDS IN EXPERT SYSTEMS

2.5.1 Non-procedural Development Paradigms

Many of the early rule and model-based, ES were designed in a procedural development environment, but the suggested benefits of object-oriented system development led to a cautious movement towards object-oriented ES shells which often feature an sophisticated graphical user interface (GUI) [Higa & Liu, 1989]. This is also seen in the development of ES using object-oriented programming languages (C++), although object-oriented design can also be implemented in more traditional programming languages. Despite the recent hype, it must be realized that to a large extent

"object-oriented design is essentially what used to be called bottom-up design. Unfortunately, the term bottom-up never sounded quite as impressive as object-oriented" [Giarratano & Riley, 1989, p.42]

However, implementing attribute inheritance, defaults, demons etc. is greatly facilitated by the use of a specific object-orientated development tools.

Another non-procedural technological development which is being touted as highly promising is the integration of neural technology with ES. AI Expert has carried many technical articles on this issue [e.g. Hillman; 1990]. This technology is ideally suited to applications which require complex pattern recognition under circumstances of incomplete information and very large numbers of transactions [Trippi & Turban, 1989]. Successful neural network-based ES have been implemented in areas such as risk assessment (credit scoring, bond rating) and stock market analysis.

2.5.2 Further Integration with Other Systems

The advantages of integrating ES into DSS have been discussed in 2.4.4 is also representative of a wider trend which recognizes that the integration of ES with existing systems is the key to more widespread and more effective use of ES: businesses do not want to discard or replace existing systems completely [Broussell, 1990]. Standard software applications will start imbedding ES technology; this has already happened in the sophisticated user-interfaces of some recent PC application releases [Cashmore, 1989].

A high-growth area is the effort to create more intelligent database systems by incorporating ES technology into the DBMS. The considerable theoretical and practical interest in these Expert Database Systems is documented in Kerschberg [1989]. Often, a more neutral terminology is preferred by the industry e.g. knowledge-base management systems [Brodie & Mylopoulos, 1986].

Two other demonstrations of this trend towards integration are the application of traditional CASE tools to ES development [Blackman, 1990] and the coupling together of several smaller ES, each confined to a narrow domain, into one large(r), more comprehensive DSS [Turban, 1990].

2.5.3 Other Trends

The following trends are perhaps less pronounced but just as real. Firstly, the move towards more managerial decision-making oriented applications, with the resultant emergence of the expert support systems, has already been noted. These ESS work primarily as interactive aids to experts where the human provides the overall problem-solving direction or strategy as well as additional knowledge which is not incorporated in the system [Anon., 1987]

Two key bottlenecks in the development of ES concern the knowledge acquisition process and the maintenance and updating of the knowledge base. The first problem can be addressed by computer-assisted rule generation, either through the use of neural technology (2.5.1) or by an automated induction process (2.3.3).

The automation of the maintenance of the knowledge base is the main thrust behind the development of self-learning ES. Future ES will hopefully create and redefine their own rules as input data changes. This can be achieved by looking for examples and counter-examples as discussed by Dyson [1990] and Deng [1990].

One of the trends anticipated by the 1987 EDP Analyzer (now IS Analyzer) is the emergence of small, portable custom-developed stand-alone ES for use on PC. These would be "intelligent on-the-job aids" for various categories of white collar workers. Although these on-the-job aids have not yet materialized, a number of off-the-shelf ES are being marketed. The CSIR in South Africa is particularly active in this field.

Finally, with the technological advances and further integration of methodologies, ES are ready to tackle new problem domains. A fresh approach in this respect is the proposal of Young [1990] to use knowledge-based systems to support idea processing, i.e. facilitate the human creative processes involved in finding and developing new ideas.

2.6 OVERVIEW OF CURRENT ES APPLICATIONS

As discussed in 2.3.3, no perfect classification system for ES applications has yet been developed. An informal survey of an electronic CD-ROM based computer literature database for 1991 revealed between 100 and 200 ES being documented in about as many articles. Further research could be directed towards establishing a suitable classification scheme.

The purpose of this section is therefore to merely demonstrate the variety of application areas for which ES have been developed. The following books can be consulted for more details on commercial ES applications:

- Waterman: A Guide to Expert Systems [1986]; and

- Quinlan: Applications of Expert Systems Volume 1 [1987] and Volume 2 [1989].

Giarratano [1989] and Hu [1987] also have excellent chapters summarizing popular ES applications.

2.6.1 Accounting & Financial ES

- Prognosis: AUDITOR (debtors' risk);

- Planning: TAXADVISOR (personal tax & estate planning); TAXMAN (corporate tax); TIARA (internal audit); INVEST, FINANCIAL ADVISOR & PLANPOWER (investment planning); INVESTOR (tax shelter planning); EXPERTAX (tax planning & auditing); EY/ASQ (audit planning); etc.

- Interpretation/Prognosis: RIEM, CRITERION (stock selection); CLUES (insurance underwriting); AA (credit card authorization); ULTRUST (portfolio analysis); LINK ANALYSIS SYSTEM (criminal tax investigation); AUTOMATED UNDERREPORTER (tax return screening) LOAN PROBER (loan portfolio assessment);VATIA (VAT audit planning) etc.

- Monitoring: WATCHDOG (investment/portfolio).

2.6.2 Agricultural ES

- Planning: POMME (apple orchards).

- Diagnosis: PLANT/DS (soyabean disease).

- Prediction: PLANT/CD (damage from black cutworm).

2.6.3 Chemistry ES

- Interpretation: DENDRAL & C-13 (molecular structure); CRYSALIS (protein structure); GA1 (DNA); SEQ (nucleotide sequence).

- Design: CLONER (biological molecules); MOLGEN (gene-cloning experiments); OCSS, SYNCHEM & SECS (organic molecules).

- Planning: SPEX (molecular biology experiments).

- Remedy: TQMSTUNE (tune mass spectrometer)

2.6.4 Electronics ES

- Diagnosis: ACE (telephone networks); IN-ATE (oscilloscope); NDS (comms network); BDS (baseband distribution subsystem); FG502-TASP (Tektronix); FOREST (electronic equipment).

- Design: PALLADIO, PEACE & DAA (VLSI circuits); EURISKO (3-D micro-electronics); REDESIGN (digital circuits); SYN, TALIB & EL (integrated board circuits).

- Prognosis: CRITTER & DFT (VLSI performance).

- Instruction: CADHELP (CAD); SOPHIE (circuit fault diagnosis).

2.6.5 Engineering ES

- Diagnosis/Remedy: NPPC & REACTOR (reactor accidents); DELTA (GE locomotives); SPERIAL (structural damage assessment).

- Instruction: STEAMER (steam powerplant).

- Planning & Remedy: CONPHYDE (physical property estimation methods); SACON (structural analysis problems).

2.6.6 Geological ES

- Interpretation: PROSPECTOR (mineral exploration); DIPMETER (dipmeter logs); LITHO & ELAS (oil well logs).

- Diagnosis/Remedy: MUD (drilling problems).

2.6.7 Information Systems ES

- Configuration: XCON, XSEL & XSITE (DEC).

- Diagnosis/Remedy: TIMM/TUNER (VAX); CRIB (general hardware & software diagnostics); DART (IBM 4331); IDT (PDP 11/03); PROJCON (project management).

- Monitor/Control: YES/MVS (IBM MVS).

- Prognosis: PTRANS (DEC).

- Planning: ISA & IMACS (order scheduling, DEC).

- Design: MIXER (microprograms for TI990 chip); CODES (conceptual database design).

- Control: IR-NLI (Natural Language Interface for on-line databases); RABBIT & RUBRIC (database queries).

2.6.8 Legal ES

- Interpretation/Prognosis: DSCAS (building contractors DSC claims); LDS (product liability); LEGAL ANALYSIS SYSTEM (assault & battery); SAL (asbestos exposure claims).

2.6.9 Management ES

- Prognosis: DMCM (production cost estimation); SETELI (telecommunications investment assessment); INNOVATOR (evaluation of new products).

- Instruction: GAMEPLAN (corporate/political response to IS executive's actions).

- Planning: ESS (CIM scheduling); REXSYS (business recovery planning); SAGACITY (project management); TEAM (skilled labour management).

- Design: PPE (personel manuals & policies).

- Monitoring: BERT (statutory bank reports to US Treasury); INSPECTOR (internal forex trading).

2.6.10 Medical ES

- Diagnosis/Interpretation: PUFF (lung disease); ABEL (acid-base/electrolytes); AI/COAG (blood disease); AI/RHEUM (rheumatoid disease); CADUCEUS (internal medicine disease); ANGY (coronary vessels); AI/MM (renal physiology); CENTAUR (pulmonary function); CLOT (blood coagulation); DIAGNOSER & GALEN (congenital heart disease); EEG ANALYSIS; EMERGE (chest pain); HEART IMAGE INTERPRETER etc.

- Remedy/Diagnosis: BLUE BOX (depression); MYCIN (bacterial infections); ANNA (digitalis administration); CASNET/GLAUCOMA (glaucoma); DIALYSIS; HDDSS (Hodgkin's); HEADMED (psychopharmacological).

- Prognosis: DRUG INTERACTION CRITIC (drug interactions)

- Monitoring: VM (intensive care); BABY (newborn ICU); ONCOCIN (chemotherapy).

- Instruction: ATTENDING (anesthetic); GUIDON (bacterial infection).

2.6.11 Military ES

- Interpretation: ADEPT, ANALYST & AMUID (battlefield situation assessment); AIRID (aircraft identification); ASTA (radar identification); ATR (military target id.); HANNIBAL (comms situation assessment) etc.

- Planning/Prognosis: AIRPLAN (aircraft carrier); BATTLE (weapon allocation); TATR (attack enemy airfields); KNOBS (air mission planning).

- Prognosis: I&W (armed conflict).

- Control: EPES (F-16); EXPERT NAVIGATOR (tactical aircraft).

2.6.12 ES in Other Application Areas

- Planning: CARGuide (city street route planning); GCA (student curriculum); ISIS (factory job shop scheduling); KNEECAP (spacecraft crew activity planning); RBMS (flight scheduling); RPMS (generic resource scheduling).

- Monitoring: LES (shuttle liquid oxygen).

- Prognosis: WILLARD (thunderstorms).

- Design: ACES (cartographic map labeling).

- Diagnosis/Remedy: ADVISOR (novice MACSYMA users); PDS (machine proces diagnosis); FALCON (chemical process disturbances); FAITH (spacecraft troubleshooting)

- Interpretation: GAMMA (spectography).

- Control: ECESIS (space station life support system).

2.7 RESULTS FROM PREVIOUS ES SURVEYS

2.7.1 General Comments

The relative lack of systematic empirical ES surveys has been noted by Benbasat & Nault [1990] and Keating [1991]. An extensive literature search, however, revealed quite a substantial number of surveys. Unfortunately, it is extremely difficult to get hold of some of the original survey reports: e.g. the Frost & Sullivan report is available at a cost of US$ 3500! Where primary sources were not obtainable, secondary sources were used rather than having to ignore the studies completely. The following is believed to be one of the more extensive listings of large-scale questionnaire-type ES surveys to date.

Although a detailed discussion of the results of each survey is not possible, a significant effort has been made to present as much information as possible. It is hoped that this is to the benefit of readers who cannot get access to the wide range of literature sources which were consulted.

Benbasat & Nault [1990] point also out that there appear to be more methodological problems in empirical ES research than for any other empirical DSS research. This could explain some of the apparent contradictions between conclusions of various research.

The discussion will follow the somewhat ad-hoc but useful classification made in 1.2.2: general surveys that probe the extent (penetration) of ES across all organizations (population = all or most organizations); specific surveys directed at current ES users to probe the current state of the art (population = past, current or imminent ES users); and targeted surveys which focus on a specific aspect of ES or a specific class of users (population = sub-set of ES users). In practice, the distinction between the different types of surveys can be made on the basis of the sampling frame employed by the study or according to the percentage of respondents which are ES users.

2.7.2 Surveys That Probe the Extent to Which ES Are Being Used

2.7.2.1 Behestian-Ardekani [1988]: USA

This questionnaire was posted to a random sample DP managers and 47 valid responses were received.

Usage

Only two organizations were using ES, one non-user intended to develop an ES within the year while one other intended to purchase an ES within the year.

Reasons for not using ES

Reasons considered impeding the use of ES technology were:

- not cost-effective;

- user resistance;

- no necessity;

- limited technological capacity;

- too early;

- low priority;

- ES solve only simple problems.

2.7.2.2 Price-Waterhouse: UK

This study was reported in Keating [1991] and Price-Waterhouse [1989] and sampled the opinions of UK DP managers during September 1987.

Usage

9% of the respondents indicated that they were using 1 to 5 ES, while a further 2% indicated the use of more than 5 ES.

Responsibility for initiation and development

The responsibility for finding and developing applications seemed mainly the responsibility of the DP department although the involvement of the user was expected to increase substantially over the next three years [figure 10].

Success ratio

Of the 13 ES that were developed using shells, 7 (i.e. 46%) were put into use. Of the 6 custom-developed ("D.I.Y.") systems, 4 (i.e. 67%) were into use. The authors cite the greater flexibility and the larger investment as reason why relatively more custom programmed systems are successful.

Reasons for not using ES

The top ten reasons, in order of importance, given for holding back on the ES technology, together with the percentage of respondents who mentioned each reason, are listed below:

- lack of corporate awareness (66%);

- finding suitable applications (53%);

- cost justifying applications (37%);

- availability of technical skills (31%);

- integration with existing systems (29%);

- acceptance by users (25%);

- delivering practical systems (24%);

- capturing expert's knowledge (22%);

- maintaining captured knowledge (14%).

Expectations about the future

Only one-sixth of the respondents expect ES to become an "important" technology within 3 years as opposed to almost one-third who do not expect ES to become important.

2.7.2.3 Keating [1991]: R.S.A.

Keating mailed 450 questionnaires to a random sample drawn from the Computing SA 1989 DP installation handbook. A response rate of almost 20% (82 usable responses) was obtained. Although a detailed comparative analysis with the results from this survey will be conducted in chapter 4, many of the results will be detailed here as well because of their particular relevance. In some cases, a further tentative ranking had to be performed in order to improve readability and succintness.

Usage

Eighteen (22%) out of 82 respondents used expert systems (i.e. developed or implemented ES) with a total of 26 systems being described.

Organizational demographics

No pronounced differences between users and non-users (organizations) were found; although there were relatively less medium-sized users and a predominance of PWV users (72% of users versus 48% of total responses received).

Strategic importance

10 users reported that they had developed ES which were of strategic importance. Of all respondents, 48% envisaged developing strategically important ES in the future.

Responsibility for initiation and development

Although the majority (17 out of 26) of the systems were user-initiated, only two were developed by the user only; 11 ES were developed by DP and a further 8 by others.

Success

The general evaluation of the ES was very positive. Only 1 system was considered to be unsatisfactory, 4 satisfactory, 5 successful, 10 very successful and for the other 6 it was too soon to tell.

Development platform

Exactly half (13) of the ES were developed using a commercial system shell: VP Expert (5), Synapse (3), PC Plus (3), Crystal-3 (1) and Natural Expert (1).

The majority (18) of the systems were micro-based, with 2 mini- and 6 mainframe-based ES.

Operational status

Over 60% of the systems in place are actually used in the conducting of business: 7 are pilot systems, 10 are used for internal business purposes and a further 6 for external business purposes.

Level of integration with other applications

One-third (8) of the ES were reported as integrated systems with 15 being stand-alone systems.

General opinions of respondents

Respondents were asked to express their (dis)agreement with a number of statements on a 5-point Likert scale. The following statements attracted relatively strong opinions from users.

Notes: +x% = percentage that agree or agree strongly; -x% = percentage that disagree or disagree strongly; a cut-off value of x=50% has been used to select the statements below; the ranking is done by the author, NOT Keating.

- The term "expert system" creates unrealistically high expectations +78%

- Expert systems solve only simple problems +78%

- Expert systems will improve productivity +72%

- There is technical enthusiasm for ES +72%

- Experts must agree on solution +67%

- The legal aspects of expert systems will become very important +50%

- There is tremendous potential for expert systems in my organisation +50%

... [various other statements that did not attract a 50% or more agreement/disagreement consensus] ...

- Expert systems are of low priority -50%

- Expert systems are not cost effective -78%

- Expert systems are a fad -83%

- Expert systems will replace humans -89%

- There is no need for expert systems -89%

Issues for successful ES implementation

Respondents were asked to rate the significance of various issues towards a successful ES implementation on a 5-point scale (very significant to insignificant). The following issues are ranked on basis of the mode as ranked by the users (ranking by author, NOT Keating; percentage indicates proportion):

* Issues with mode = Very significant

- Acceptance by users (89%)

- Capturing expert's knowledge (83%)

- Delivering practical systems (78%)

- Ease of use (67%)

- Availability of technical skills (50%)

- Validating results (39%; ties with a 39% "significant" response)

- Maintaining captured knowledge (39%)

- Corporate awareness of potential (39%)

* Issues with mode = Significant

- Finding suitable applications (50%)

- Task is of a manageable size (50%)

- System will provide a high return (50%)

* Issues with mode = "Neutral"

- Integration with existing systems (33%)

Problems identified as potentially causing failure and actually experienced

The following problems were identified by at least 50% of the users as potentially causing ES to fail, or were actually experienced by at least 20% of the users. The first percentage indicates the proportion of users identifying it as a potential problem, the second figure denotes the proportion of users who actually experienced the problem.

Potential/Experienced

- Identifying suitable ES building tools 61% / 28%

- Commitment from a human expert 56% / 22%

- Lack of understanding about ES 56% / 11%

- Validating ES knowledge 50% / 22%

- Finding a suitable topic 50% / 22%

- Cost justification 50% / 17%

- Capturing human expertise 50% / 17%

- Maintaining knowledge 50% / 11%

- Interfaces to other systems 44% / 22%

- Long development cycle 33% / 22%

Comment: note that there is only partial agreement between what users see as potentially critical problems and the actual problems experienced. This seems to indicate that respondents actually anticipated and addressed these areas before they became problematic [Keating, 1991:103].

Benefits expected and actually experienced

The following responses were given to the expected benefits (first figure) and benefits actually experienced (second figure) of ES (own ranking & reorganizing)

Expected/Experienced

- Improving productivity 83% / 56%

- Making the expert knowledge available to a wide audience 78% / 33%

- Assimilating knowledge of many experts 78% / 22%

- Automating of repetitive tedious or complex operations 72% / 33%

- Achieving competitive advantage 67% / 39%

- Storing of valuable information 67% / 39%

- Freeing humans to deal with more interesting or complex tasks 67% / 33%

- Improved consistency 67% / 28%

- Substituting for human expertise 56% / 28%

- Training new experts 56% / 11%

- Personnel savings 28% / 22%

Comment: here again, there are less benefits actually experienced that were considered possible for the use of ES.

Overall conclusion

Keating her research as follows:

"The findings indicate that extensive use is not made of expert systems in South Africa, although respondents are aware of the potential strategic significance of expert systems to their organisations. The local opinions and experiences compare well with the results of international studies." [1991:ii]

2.7.2.4 Other Reported Surveys

A number of other surveys probing the extent of ES usage have been reported in the literature. Ambrioso [1990] estimates that 100 of the Fortune 1000 firms have mission-critical production applications that use ES technology. She suggests that successful production systems are typically used for applications such as allocation, scheduling and pricing.

On the other hand, Chapnick [1990] mentions, but does not reference, a survey amongst US manufacturing companies in which only 4% of the companies indicated that they use artificial intelligence in their operations.

2.7.2.5 Conclusion

It would seem from the above surveys that the penetration of ES is not as wide spread as many proponents of the technology suggest. A typical penetration rate of maximum 10% amongst the larger companies seems about par for the course, although the more recent surveys indicate an increasing interest.

2.7.3 Surveys Directed at Current ES Users

2.7.3.1 Philip & Schultz [1990]: USA

This survey was conducted amongst readers of the trade magazine AI Expert. Of the 1000 questionnaires that were mailed, 148 readers responded. Because of its sampling frame, methodology and scope, this survey will be used as a partial model for this research. Unfortunately, the full research results or questionnaire content could not be obtained from the researchers.

The following statistics are for the 89 respondents that were classified as ES users, i.e. implemented or developed ES. Statistics on ES typically refer to implemented systems only.

Usage

Of the respondents, 37% have implemented ES, 23% are busy developing applications and a further 24% are seriously considering ES. The total number of ES implemented was 180, under development 532 and the number of ES under serious consideration 735.

Organizational demographics

There is a surprising number (24%) of small (less than 20 employees) organizations amongst the users, while 26% are employ more than 2000 people.

Service organisations (26%) and manufacturing companies (23%) account for half of the users. Educational institutions (18%), government agencies (9%) and financial companies (5%) are the other significant players in the market.

Responsibility for initiation

Contrary to the findings of other surveys, only 12% of the major applications were proposed by MIS. The majority was proposed by users (28%), upper management (17%) or a functional unit (16%). No details are available on who actually developed the systems.

Reasons for developing ES

The following reasons for developing the systems were ranked as important by at least 40% of users:

- improve decision making/problem solving 67%

- improve productivity 63%

- improve service 53%

- improve product quality 51%

- improve diagnostics 45%

- preserve knowledge 44%

Areas of application

The following distribution among application areas was obtained: manufacturing (44%), management (29%), accounting (20%), finance (14%), marketing (14%), education (9%), distribution (8%) and engineering (6%). This is a very interesting departure from previous surveys, where applications were mostly confined to more technical or scientific areas.

Development platform

IBM PC or compatibles (83%) are the favourite hardware platform. Other hardware platforms being mentioned by 5% or more of users are Macintosh (19%), IBM 3090 (15%), Microvax (9%), Sun 386 (9%), HP 9000 (6%), VAX (5%) and DEC workstation (5%). The actual distribution amongst ES cannot be determined from the published results since organizations use many platforms for different systems.

The moving away from the traditional "AI" environment is evidenced in the type of software tools being used: shells (58%), conventional languages (43%), LISP (38%) and Prolog (33%).

73% of respondents used a prototype and only 7% did not, the rest did not answer or did not know.

Integration with other applications

A full 58% of applications interfaced with external programs written in conventional languages (mainly "C"); 36% with database management systems (DBMS, mainly dBase and Oracle), 18% with graphics software and 5% with a 4GL.

Size of systems

Figure 11 gives an overview of the size of the systems as measured in number of rules and development time required for completion.

Success of systems

The degree of satisfaction, measured on a Likert 5-point scale, was fairly high. "Satisfied" or "Very satisfied" was selected by 48% for the degree of user satisfaction, 38% for management satisfaction and 51% for developer satisfaction although 28% did not provide ratings for any of the groups.

The level of success was reflected in the frequency with which the systems are being used: 43% of systems is used more than once per day, a further 25% more than once per week. Another 13% used systems more than once per month with the balance indicated "other" frequencies or not responding to the question.

Problems experienced

The following lists serious problems or limitations experienced by at least 10% of the organizations:

- Performance speed 17%

- Interface with external program 16%

- Documentation 13%

- Interface with external files 14%

- Debugging aids 12%

- Ease of use 10%

- Ease of maintenance 10%

General opinions of respondents

The following opinions were ranked as important issues for the future of ES by at least 50% of the users (bold by author):

- Many future applications will need to interface to a DBMS 89%

- Applications provide an excellent tool for strategic information systems 84%

- Our firm will be using systems for decision support systems 76%

- There will be a significant increase in applications over the next two years in our firm 74%

- Applications reduce manpower needs 55%

- Our firm will be using systems for executive information systems 54%

The statement "ES is an overworked term and will fail over time" was ranked as important by 22% of respondents.

Overall conclusion

This survey reflects a status which is a significant evolution from the picture painted by earlier surveys, although it is unknown to what extent this has been influenced by the particular readership distribution of AI Expert.

ES seem to be maturing and joining the "mainstream" systems as reflected in the increased complexity, predominant use of non-AI software tools, high user involvement, and high level of interfacing with external programs. Even more significant is the large proportion of applications in the managerial area: management, finance and marketing account for 50% of the applications!

A much greater number of systems are in the planning and development stage, with an important place being reserved for them in future strategic, decision support and executive information systems.

2.7.3.2 The JIPDEC survey: Japan.

The Japan Information Processing Development Center (JIPDEC) conducted a survey on AI systems in January 1989. This survey was reported in both Motoda [1990] and Anon. [1990b].

Usage

ES account for about 75% of the 305 AI systems installed by respondents i.e. 229 ES, with approximately another 200 ES under development. The actual number of ES in use has not grown substantially when compared to the 168 ES reported by the 1986 survey.

Areas of application

The following break-down according to application type is provided:

- diagnosis 29%

- plant operation and production control 19%

- consulting 14%

- designing 9%

- investment management 6%

- distribution management 6%

2.7.3.3 The Frost & Sullivan Report

The Frost & Sullivan report nr E1266 [Anon., 1990c] is not an academic research survey but really a market study intending to survey and forecast the extent of the global ES industry. One may assume that their figures exclude development investments made by ES users internally. Although the objective value of their market forecasts might be disputed, the report is included here because it provides an interesting perspective on the status of ES in Europe.

USA

Frost & Sullivan take an optimistic view on the future as they expect the US ES market to demonstrate an almost tenfold increase between 1988 ($ 295 million) and 1994 ($ 2376 million).

According to their research, the market is currently evenly distributed between the public, financial, industrial, manufacturing and agricultural sectors. Of these, the financial market in particular is expected to show strong growth, with the defense and medical ES likely to follow closely. Growth in the agricultural and manufacturing sectors is expected to be low.

Product-wise, sales of hardware is expected to decrease sharply with the sales of shells and applications expected to perform much more strongly. The main trend will be towards integrated ES built into databases and conventional applications.

Europe

The following figures reflect the value of the ES market in 1988, as well as 1994 forecasts, for the major European countries:

1988 (1994 forecast)

United Kingdom $ 81 million ($ 584 million)

Germany $ 86 million ($ 643 million)

France $ 63 million ($ 468 million)

Italy $ 16 million ($ 218 million)

Total "big four" $246 million ($1913 million)

The biggest growth areas in Europe are expected to be in the applications, consultancy and customization areas, although regional patterns are slightly different.

2.7.3.4 Conclusion

A first important conclusion, highlighted especially in the Philip & Schultz survey but also mentioned in the other 2 reports, is the maturing of the ES market with a significant move towards business management applications.

A perhaps less expected result, not explicitly mentioned in the literature review is the global picture presented by the surveys. A popular (US) notion appears to be that the status of ES in Europe is a couple of years behind the USA, with perhaps Japan following much more closely in the context of their Fifth Generation Computer Initiative [Broussell, 1990]. In reality, the figures quoted by Frost & Sullivan show that the big four European countries are almost on par with the USA, while the number of known ES in Japan according to Motoda [1990] is only a fraction of the USA number.

2.7.4 Surveys Targeted at Specific ES Types or Users

2.7.4.1 Coopers & Lybrand: Insurance Industry Survey Series

Coopers & Lybrand have initiated a series of bi-annual surveys into the status of ES in the US insurance industry. Although their sampling frame changed over the 1986-1990 period, the results are still relatively comparable. The primary survey reports could not be obtained but the following secondary sources summarize their findings: Crofts [1989] quotes figures for the 1986 survey; O'Leary [1988] summarizes the 1988 survey and Jones [1990] provides a comprehensive overview of the findings of the 1990 survey.