Expert systems vendors and academics made many promises to business managers about the technology's potential. The authors conducted an in-depth South African survey which resulted in a database of 128 documented expert systems. For purposes of the Conference, a subset of those systems has been selected in the area of functional business management. Some of the various dimensions which are investigated in this paper are: specific areas of application, responsibility for initiation and development, use and successfulness, some technical considerations, strategic importance, benefits experienced and expectations about the future of expert systems.

Expert systems (ES) technology promised business managers, explicitly or implicitly, a panacea cure to solve all decision making problems in virtually every functional area. But these initial promises and enthusiasm were quickly tempered by the lack of mainstream business applications. Although a few successful pilot projects received substantial publicity in both technical and business literature, few organizations seem to follow through with a full-scale adoption. [Ambrioso, 1990]

The initial period of "flurry and excitement" in the early and mid-eighties was subsequently followed by what some have called the "AI winter". [Anon, 1990] However, the technology is still out there and being used in various situations, albeit under slightly different labels (e.g. knowledge-based systems) or integrated into mainstream technologies. In fact, the famed Feigenbaum insists that "there was one application of expert systems that over a period of only a few months saved more money [...] than the total amount invested in AI research since the beginning of time. That application was the expert system used to manage the logistics of Operation Desert Storm." [Metcalfe, 1993]

So, what is happening with ES in South Africa? Have they come and gone? Are they still being used? Were they never here in the first place? And, foremost, are there any ES used in a business and managerial context?

The following definition has been adopted for purposes of this paper:

"Expert systems will be understood to be computerized advisory programs that attempt to imitate or substitute reasoning processes and knowledge of experts in solving specific types of problems." [Turban, 1990]

Although this definition can be subjected to some criticism, it has been selected because it is readily understood by IS people and users and it does not refer to the technical aspects of the system (e.g. inferencing, heuristics etc.). As Turban points out, this definition is a content-free expression, i.e. it means different things to different people.

It must be noted that the term "knowledge-based system" (KBS) emerged during the last ten years or so. Initially it was treated as a synonym of ES, although perhaps more connotation-free in the commercial ES market place. Currently, there is a tendency to increase its scope slightly to include systems that perform tasks that do not require "experts" to perform, although they do require a certain level of competency in a specific problem domain [Methlie L.B., 1987:337]. Notwithstanding the fact that the term KBS is perhaps more favoured in a business environment, reference in what follows will only be made to ES since the original survey on which this paper is based used that terminology.

The three traditional components of an ES are:

_ the knowledge base which contains rules ('heuristics' or logic) and facts (problem situations, appropriate questions etc.) about the problem domain;

_ the inference engine (the "brain" or "control centre" of the ES) which draws conclusions, recommends and motivates actions, etc. based on the information given by the user and the information in the knowledge base;

_ the user interface which is the mechanism by with communication between the ES and user takes place - this can be in the form of structured text (menus or simple questions: True/False, Multiple Choice, One-word or number answers), unstructured text ("natural language interface") and other graphics or diagrammatic formats.

In practice, even an only moderately sophisticated ES will have a much more complex structure incorporating a black board, explanation facility, conflict resolver, exception handler, self-learning facility, scheduler, etc. Integrating object-orientation and neural technology with ES further compounds the situation.

Since virtually all ES provide support for decision-making in one area or another, every ES can be seen as a DSS in the wider sense of the word. However, in the IS literature, the term DSS has a much more specific meaning:

"A DSS is a computer-based system used by managers as an aid to decision making in semi-structured decision tasks through direct interaction with data and models." [Benbasat & Nault, 1990:203-204]

The important element of this definition is that it limits the scope to managerial environments. What motivates this distinction with the large number of ES in more technical or scientific areas?

"[Developing an] ES to produce a diagnosis the way a medical expert does, poses a very different set of problems than designing an expert system to replicate some aspect of managerial decision making." [Agarwal, 1990:126]

This is specifically illustrated in the discourse by Kim & Courtney [1988] where they demonstrate that the problem domain of management is much wider and shallower than that of most ES. Given the nature and purpose of managerial ES, can they then not be considered to be some kind of DSS, or to put it another way:

"Inevitably, the question arises: Is an ES a DSS? There are a number of number of differences [...]. Expert systems deal with problems whose scope is narrow and relatively well defined. The system incorporates a set of rules and heuristics that are repeatedly used in the solution of the problem. The rules and relationships change with experience. Typically, an expert system has the ability to explain why it reached a conclusion. In contrast, a DSS is intended to operate in a broader and more diverse decision environment. It should be usable for ad hoc problems but usually does not incorporate a facility for explanation." [Kopcso, Pipino & Rybolt; 1988:67]

These differences are neatly listed in figure 1, which also compares them to other types of information systems which are found in the business environment. Of course, the real question is not what the differences or similarities between the two technologies are, but rather how they can best be utilized to improve decision making.

This research can be seen as a natural progression of other South African theoretical [Mentz, 1988; Jacobson, 1989] and empirical [Keating, 1991] research on ES. In particular, it must be emphasized that this is the first local in-depth survey of its kind and it will hopefully serve as a sound scientific basis to evaluate future developments in the ES area. Also, it will provide a foundation for further, more focused empirical research efforts.

However, the potential benefits of ES are of such a nature that the business community will also benefit from a scientific assessment. The following benefits of ES must be seen within South Africa's specific context.

This country has a dramatic dearth of highly skilled manpower in the areas of management and technical staff. Some time ago, researchers at the University of Stellenbosch estimate an annual shortfall of 3500 managers and ten times as many technical staff [Anon., 1988] which reputedly constrains economic upswings by a lack of decision making abilities and technical skills. I am sure that these figures will be revisited in the course of the Conference. This may also contribute to South Africa's generally wide span of managerial control. South Africa's geographic isolation, complex educational problems and relatively small but sophisticated economy (from a global perspective) further compounds the relative scarcity of top experts in almost any given technical sub-domain. And although one must caution against the "cure-all" syndrome, proper implementation of ES technology could at least partially alleviate this shortage:

"Not only can it extend expert performance throughout an organisation or industry despite having lower skilled staff doing the work, but it can also bring novices up to speed faster." With the cautionary note: "These are some of the promises of knowledge-based systems. It will still take faith, dedication and money to turn the promises into reality." [Anon., 1988:46]

The management potential of ES could also seen to be illustrated in a perhaps more humorous way by an experiment whereby a management expert system developed by professor Duffy (University of the Witwatersrand) beat all ten teams of "real, human" businessmen in a business management game [Anon., 1989].

Another relevant aspect concerns the low growth in white collar productivity, despite the dramatic increases in corporate IT budgets. The problem seems to be one of information overload: more and more data is being delivered by the information systems but they do not aid in the consequent decision making. This is where ES have an important potential role to play: to automate the routine decision making process [Buckler, 1990].

Further important considerations in motivating this research are intense competition for allocation of resources in the light of the many needs of a developing country. It is imperative that lessons be learnt from both successful and unsuccessful projects. South Africa is fortunate enough to be on a relatively high level in terms of computer technology and most organizations could thus take immediate advantage of ES opportunities without necessarily having to be on the leading (bleeding?) edge of the technology.

Lastly, the timing appears to be ideal for this research:

_ The ES technology has reached a relative level of maturity and stability, as evidenced by the availability of many development tools (some of SA origin), the wealth of local experience and a substantial body of literature.

_ There seems to be less secrecy surrounding corporate ES projects than in the initial stages when immediate strategic advantages were expected.

_ A sufficiently large amount of ES activity ("critical mass") is happening to draw meaningful conclusions, but not so much that the field becomes unmanageable.

_ The technology is generally still "new" enough to enable clear identification of ES projects: system integration make soon advance to a stage where the distinction between ES and other information systems becomes too blurred.

_ It is still early enough to learn from our mistakes.

In 1992, the authors conducted as survey to investigate the current state of ES application and usage within South African organizations."saturation" surveys, directed at informed organizations or individuals who are already familiar with ES technology or which probe the current state of the art in the ES field. The model survey for this type is Philip & Schultz [1990] who surveyed recipients of the AI Expert magazine. A substantial proportion of the sampling frame was known to be involved with ES.

A total of 420 questionnaires were posted, from which 156 replies (or 37%) were received. This compares to the already high 20% response rate of Keating [1991:67] and the 14,7% recorded by Philip & Schultz [1990:57]. 80 respondents (or 52%; 3 questionnaires could not be used) provided information on one or more expert systems in use or under development. The users reported on a total of 128 unique expert systems. The questionnaire provided space for a maximum of 3 ES; 8 users indicated that they knew of more ES within their organization, adding up to another 26 non-documented ES. Although the full original research report [Van Belle, 1992] is available from the authors, the results have not (yet) been made available to the South African academic community due to specific circumstances.

However, the specific theme of the Conference prompted the authors to reflect on the value of the database of ES information since a significant sub-set of ES is applied in a commercial business environment. This presentation will therefore focus on those documented ES which are of immediate relevance to the delegates, namely the forty-six ES which in the area of functional business management. In what follows, the main thrust will be of a descriptive nature. Further statistical analysis is envisaged and colleagues are invited to offer suggestions in this respect.

It is hoped that, through this paper, the delegates of the Conference will gain a better understanding of the use of ES in South African business environments.

The management-related ES were targeted at a number of functional areas. These are summarized in Table 1, although some more or less arbitrary classification decisions had to be made by necessity. The list of (respondent supplied) ES descriptions is provided in addendum 12.1. For ease of reference, these ES will hereafter be referred to as managerial or management ES; for lack of a more appropriate term to indicate this subset.

Table 1: Area of ES Applications

This confirms the traditional view: "[T]here are several areas where knowledge-based systems are selling quite well. One is the financial services industry, where they can help process credit applications [...] Another is manufacturing and distribution, where companies are using them to manage order processing and scheduling." [Buckler, 1993:12] However, the six ES in human resources surprised the authors. Could this signal a niche market which South Africa is developing?

For comparison purposes, the reader may be interested in the other areas in which SA ES are deployed: engineering and manufacturing control (41); agriculture, forestry & ecology (9); computer systems (8); medical (4); military (4); and other (4). Note that some of military, forestry and manufacturing ES also support management-like decision making but their fields can arguably be said to be of a much more technical nature.

It is of interest to note who is responsible for initiating the ES project. This is indicated in Figure 2 below.

This is quite a startling difference to the other ES in the database as demonstrated in the table 2 below.

Table 2: Initiators of ES.

It is evident from this table that "upper management" and IS/DP play a much more significant role in instigating the ES than is the case for other (more technical) ES which are almost dominantly proposed by the end-users (often technicians or professionals). Although this result might perhaps have been expected on the basis of "gut-feel", this difference is also statistically significant at a 5% level of confidence (c =10,99; refer to Addendum 2).

Similarly, the responsibility for actually developing the ES shows marked differences. Figure 3 shows clearly that the managerial ES are mainly developed by IS/DP whilst other ES make much more use of consultants or end-users!

The acid test for measuring the relevance and appropriateness of ES technology is to be found in the use of the ES. Of the 46 ES, 29 were "live" i.e. in actual use.

The following non-exclusive reasons were given for non-usage (number of responses between brackets):

_ still under development or testing (10);

_ lack of user support (6);

_ no organization fit: does not fit into existing work flow or systems (3);

_ change in business needs and/or environment (1)

_ no specialized staff available (1);

_ don't know (1).

It is interesting to note the frequency of use as reflected in Figure 5.

Managerial ES have, by comparison, been in use slightly longer and being used more frequently than the other ES in the database.

Apart from the acid test "is the system used?", the respondents were also asked how they viewed the success of the ES. Interestingly, virtually all rated their ES as at least satisfactory: 9 ES were judged very successful, another 9 as successful, a further 10 as satisfactory and one ES each was classified unsatisfactory, unknown and no consensus respectively.

A peculiarity (?) seems to be that the relatively few user-initiated ES were less likely to be considered successful than those ES initiated by others as shown in Table 3. Cell numbers are too small to test this relationship statistically.

Table 3: Relationship between ES success and initiator.

Various measures can be employed to measure the size of an expert system. The most universally employed measure for classic' i.e. rule-based ES is the number of inference rules in its rule-base. Figure 6 gives the system sizes for all 46 ES and Figure 7 compares the respective sizes (in slightly aggregated format) of:

a) the managerial ES;

b) the other South African ES;

c) the findings for US systems [Schultz, 1990].

From this comparison, there is hardly any difference between the sub-sets of SA ES. When compared to the USA, though, there are relatively more South African "micro"-ES at the expense of "mini"-ES. This finding is statistically significant at a = 5%.

However, just as measuring information systems developed in third-generation language through the number of lines of code is not ideal, the number of rules criterium is not ideal either. This is especially so in the light of the growing importance of object-orientated and neural network-based ES. Another proxy for system size is the corporate investment in the ES through the development time. Although not quite as obvious as investment measured in Rand-cost, it is usually much more easy to obtain or gauge from the respondents. Table 4 details the findings and compares with the other and US systems respectively.

Table 4: Development time of ES.

On the whole, there are again a larger proportion of relatively small South African ES (statistically significant at a = 5%) when compared to the Schultz figures. However, this tendency is far less pronounced when one considers only the managerial ES.

A natural assumption would be to investigate whether the two proxies for system size (number of rules and development time respectively) are positively correlated. This is indeed statistically supported by a very wide margin (c =31.70; 4 degrees of freedom grouping data in a 3x3 grid) at a 1% level of confidence for the entire database. Although individual cell values are too small to conduct meaningfully statistical tests for the subsets, a visual inspection confirms that this holds true also for the respective subsets.

Another important practical aspect concerns the development and operational platforms for the ES. It is evident from Figure 8 that most managerial ES run on a PC environment and make use of a high-level ES shell. However, when compared to the other ES in the SA database, a relatively greater number make use of mainframe computer. Note that a number of ES use dual platforms.

It is further interesting to note that prototype ES are used for four out of every five systems, which is more than for the rest of the SA database (73%) but still less than the proportion in the US as reported by Schultz (92%; 1990).

A last concern, less technical but still development-related, is how the knowledge is `extracted' from the experts, to serve as input for the ES. This is shown in table 5.

Table 5: Knowledge acquisition methods.

The questionnaire queried users for a number of additional ES aspects. Although space does not allow a full discussion, a summary of the more pertinent issues should be of interest to delegates concerned with in ES technology in general. Note that the reported results come from all ES users, not those of the managerial ES alone.

The most important problems experienced were: the lack of commitment from a human expert; identifying and technical limitations of tools; cost justification; maintenance and validation of the knowledge base; the long development cycle and user acceptance/resistance. It must be noted that few problems were rated as particularly problematic by more than a third of the respondents.

When some further probing was done to identify more specific and immediate barriers, the following reasons were advanced by more than a third of the non-users:

_ There are many more urgent organizational needs/areas with a greater pay-off.

_ The backlog of other IS to be developed is too big.

_ No one has the time or know-how to investigate these opportunities further.

_ We do not have the technical manpower resources for development.

It seems that the need to put other systems in place first (first two reasons), together with a critical lack of skilled manpower (next three reasons), are the main barriers to ES development. It is evident that both issues are interrelated. This seems to be a common thread throughout the IS industry and again emphasizes the need for more IS training, improvements in IS productivity and perhaps better IS/IT management?

From the discussion it is clear that ES technology has penetrated the business world fairly deeply: 47 systems are documented. Although most are in the traditional ES arenas (production, finance, insurance and customer support), the authors were startled to find six applications in human resources management.

Overall, the characteristics of South African managerial ES are fairly similar to those of the other, more techno-scientific ES in the database. Where the profiles do differ statistically, there seems to be a fairly intuitively appealing explanation.

It is hoped that the delegates will have gained a better perspective on the potential for the technology in the managerial field, as well as the understanding that the South African business community is perhaps more ready than many academics suspect, to embrace the relevant technologies where they can yield effective results.

AMBROSIO J. "Expert Systems Make Their Mark in Corporations." Computerworld, Vol.24 No.32 (6 Aug 1990), p.10.

AGARWAL R. & TANNIRU M.R. "Knowledge Acquisition Using Structured Interviewing: An Empirical Investigation." Journal of Management Information Systems, Vol.7 No.1 (Summer 1990), pp.123-140.

ANON. 1988. Silicon Expertise. Knowledge-based Systems. Computer Mail, 29 Jan. 1988, pp.39-46.

ANON. 1989. Game for Experts: A computerised expert system developed in SA has beaten 10 teams of businessmen at their own game. Computer Mail, 29 Sept. 1989, pp.48-49.

ANON. "Expert Systems Shrug Off Slow Start for Good Growth According to Frost & Sullivan Report." Information Today, Vol.7 No.5 (May 1990), pp.43-44.

BENBASAT I. & NAULT B.R. "An Evaluation of Empirical Research in Managerial Support Systems." Decision Support Systems, Vol.6 No.3 (1990), pp.203-226.

BUCKLER, G. "The Experts Have Departed." Computing Canada, Vol.19 No.10 (May 10, 1993), p.12.

BUCKLER G. "Expert Systems Making Commercial Inroads: AI Now Viewed as Another Class of Development Tools." Computing Canada, Vol.16 No.15 (19 July 1990), pp.28-29.

JACOBSON M.A. 1989. The Use of Commercial Expert Systems in the Business Environment. Technical Report (Hons). Cape Town: University of Cape Town (Accounting Department), October 1989.

KEATING L.M. The Extent of Expert System Usage. MBA Research Report. Johannesburg: University of the Witwatersrand, 1991.

KIM J. & COURTNEY J.F. "A Survey of Knowledge Acquisition Techniques and Relevance to Managerial Problem Domains." Decision Support Systems, Vol4 No3, pp.269-284.

KOPCSO D., PIPINO L. & RYBOLT W. 1988. A Comparison of the Manipulation of Certainty Factors by Individuals and Expert System Shells. Journal of Management Information Systems, Vol.6 No.3 (Winter 1989-90), pp.66-81.

MENTZ M. 1988. The Integration of Expert Systems and Decision Support Systems to Solve Business Problems. Technical Report (Hons). Cape Town: University of Cape Town (Accounting Department), May 1988.

MENTZ M. The Integration of Expert Systems and Decision Support Systems to Solve Business Problems. Technical Report (Hons). University of Cape Town, May 1988.

METCALFE, R.M. "What Happened to Artificial Intelligence?" InfoWorld, Vol.15 No.15 (April 12, 1993), p.48.

METHLIE L.B. 1987. On Knowledge-based Decision Support Systems for Financial Diagnostics in HOLSAPPLE C.W. & WHINSTON A.B.(ed.) Decision Support Systems: Theory and Application. Berlin: Springer-Verlag, 1987, pp.335-372.

PHILIP G.C. & SCHULTZ H.K. "What's Happening With Expert Systems? A Survey of Expert System Users." AI Expert, Vol.5 No.11 (Nov.1990), pp.57-62.

TURBAN E. 1990. Decision Support and Expert Systems: Management Support Systems. New York: Macmillan, 1990.

VAN BELLE J.P. A Survey of the Current Status of Expert Systems in South Africa. Technical Report (B.Com Hons). Cape Town: University of Cape Town, 1992.

CHI-SQUARE TEST

===============

Topic: Who Initiated the ES Project?

Refer to table: 2

Columns represent: Management vs. other SA ES.

Rows represent: Initiator of ES Project.

Null Hypothesis: Independence between columns and rows.

OBSERVATIONS

============

Initiator: Mgt.ES Other ROW SUM:

---------------------- ----- ----- ----- ----- ----- ----- ========

End-users 9 29 38

Upper Management 10 6 16

IS/DP department 10 10 20

Consultants 5 12 17

Other 2 9 11

0

---------------------- ----- ----- ----- ----- ----- ----- ========

COLUMN SUM: 36 66 0 0 0 0 102

========

EXPECTED FREQUENCIES

====================

Mgt.ES Other ROW SUM:

---------------------- ----- ----- ----- ----- ----- ----- ========

End-users 13.4 24.6 0.0 0.0 0.0 0.0 38

Upper Management 5.6 10.4 0.0 0.0 0.0 0.0 16

IS/DP department 7.1 12.9 0.0 0.0 0.0 0.0 20

Consultants 6.0 11.0 0.0 0.0 0.0 0.0 17

Other 3.9 7.1 0.0 0.0 0.0 0.0 11

0.0 0.0 0.0 0.0 0.0 0.0 0

---------------------- ----- ----- ----- ----- ----- ----- ========

COLUMN SUM: 36 66 0 0 0 0 102

========

RELATIVE DIFFERENCES

====================

Mgt.ES Other ROW SUM:

---------------------- ----- ----- ----- ----- ----- ----- ========

End-users 1.45 0.79 0.00 0.00 0.00 0.00 2.24

Upper Management 3.36 1.83 0.00 0.00 0.00 0.00 5.19

IS/DP department 1.23 0.67 0.00 0.00 0.00 0.00 1.89

Consultants 0.17 0.09 0.00 0.00 0.00 0.00 0.26

Other 0.91 0.50 0.00 0.00 0.00 0.00 1.41

0.00 0.00 0.00 0.00 0.00 0.00 0.00

---------------------- ----- ----- ----- ----- ----- ----- ========

COLUMN SUM: 7.11 3.88 0.00 0.00 0.00 0.00 10.99

========

Chi-square test value: 10.99

Degrees of freedom: 4

Critical X-value (5%): 9.49

Critical X-value (1%): 13.28

Statistical conclusion: Reject null-hypothesis at 5% confidence level

Improve consistency 1 7 6,5

Make expert knowledge available 2 5 3

Store/preserve expert knowledge 3 2 0,5

Improve productivity 4 1 3

Free humans for complex tasks 5 5 0

Assimilate knowledge ¹ experts 6 9 3,5

Automate repetitive operations 7 5 2

Substitute human expertise 8 7 0,5

Training of new experts 9 11 2

Provide competitive advantage 10 2 7,5

Personnel saving 11 9 0,5

H_o = "There is no positive correlation between RSA 92 and RSA 91."

S d = 137,5; n = 11.

r' = 1-[6xS d]¸ [n(n-1)] = -0,38 (Spearman's rank-correlation coef).

z = r'Ö (n-1) = -1,19 (standard normal distribution test statistic).

Conclusion: There is no statistical basis to assume positive correlation between the results of this survey and those of Keating.

Note: No correction factor has been applied to r' to account for the large number of ties in RSA'91. This would not affect the statistical significance because of the observed negative correlation.

RSA USA ¹

Improve decision making 1 1 0

Store/preserve expert knowledge 2 3 1

Improve productivity 3 2 1

Gain experience with technology 4 5 1

Enhance product/service quality 5 4 1

H_o = "There is no positive correlation between the RSA and USA."

S d = 4; n = 5.

r' = 1-[6xS d]¸ [n(n-1)] = 0,80 (Spearman's rank-correlation coef).

z = r'Ö (n-1) = 1,60 (standard normal distribution test statistic).

Conclusion: There is a strong statistical suggestion of positive correlation between the results of this survey and those of Philip & Schultz, but it is not significant at a 5% level of confidence.

Note: This test value is very close to the critical z_0,05 = 1,64 (one-sided test since only positive correlation needs to be tested!).

In the RSA survey, "store/preserve knowledge" and "improving productivity" were very close; if Spearman's rank-correlation coefficient is calculated assuming a tie (without correction factor), r' increases to 0,88 resulting in a test value of 1,76 which is statistically significant at a = 5%.