Jean-Paul Van Belle 's  Life
Work & Study Publica- tions Family & Home Travel Running & Sports Other

EXPERT DECISION SUPPORT SYSTEMS

FOR FUNCTIONAL AREA MANAGEMENT:

A SOUTH AFRICAN STATUS REPORT

Jean-Paul Van Belle, 1995.

ABSTRACT

Notwithstanding the many claims made by expert systems (ES) vendors and academics, no systematic empirical study has been conducted to substantiate the impact of ES technology on the South African business environment. An in-depth survey on the actual use and implementation of expert systems to aid decision making in functional business management in South Africa was therefore conducted. The following dimensions were specifically investigated: functional application areas, responsibility for initiation and development, usage and success achieved, some technical considerations, strategic importance, benefits experienced and expectations about the future of expert systems.

OPSOMMING

Nieteenstaande die vele eise wat gemaak word deur ekspertsisteemverskaffers en akademici, is daar nog geen sistematiese empiriese studie rakende die impak van hierdie tegnologie op die Suid-Afrikaanse bedryfswêreld gedoen nie. Hierdie artikel beskryf die resultate van 'n ondersoek in Suid-Afrika oor die gebruik en implementering van ekspert sisteme om die besluitneming in funksionele ondernemingsbestuur te help. Die volgende dimensies is spesifiek ondersoek: funksionele toepassingsvelde, verantwoordelikheid vir inisiëring en ontwikkeling, mate van gebruik en sukses behaal, strategiese belangrikheid, voordele ervaar en verwagtings oor die toekoms van ekspertsisteme.

 

Notwithstanding the many claims made by expert systems (ES) vendors and academics, no systematic empirical study has been conducted to substantiate the impact of ES technology on the South African business environment. An in-depth survey on the actual use and implementation of expert systems to aid decision making in functional business management in South Africa was therefore conducted. The following dimensions were specifically investigated: functional application areas, responsibility for initiation and development, usage and success achieved, some technical considerations, strategic importance, benefits experienced and expectations about the future of expert systems.

OPSOMMING

Nieteenstaande die vele eise wat gemaak word deur ekspertsisteemverskaffers en akademici, is daar nog geen sistematiese empiriese studie rakende die impak van hierdie tegnologie op die Suid-Afrikaanse bedryfswêreld gedoen nie. Hierdie artikel beskryf die resultate van 'n ondersoek in Suid-Afrika oor die gebruik en implementering van ekspert sisteme om die besluitneming in funksionele ondernemingsbestuur te help. Die volgende dimensies is spesifiek ondersoek: funksionele toepassingsvelde, verantwoordelikheid vir inisiëring en ontwikkeling, mate van gebruik en sukses behaal, strategiese belangrikheid, voordele ervaar en verwagtings oor die toekoms van ekspertsisteme.

 

1 INTRODUCTION

Expert systems (ES) technology proponents promised business managers, explicitly or implicitly, a panacea cure to solve all decision making problems in virtually every functional area. But these initial promises and enthusiasm were quickly tempered by the lack of mainstream business applications. Although a few successful pilot projects received substantial publicity in both technical and business literature, few organizations seem to follow through with a full-scale adoption. [Ambrioso, 1990]

The initial period of "flurry and excitement" in the early and mid-eighties was subsequently followed by what some have called the "AI winter". [Anon, 1990] However, the technology is still out there and being used in various situations, albeit under slightly different labels (e.g. knowledge-based systems) or integrated into mainstream technologies. In fact, the famed Feigenbaum insists that "there was one application of expert systems that over a period of only a few months saved more money [...] than the total amount invested in AI research since the beginning of time. That application was the expert system used to manage the logistics of Operation Desert Storm." [Metcalfe, 1993]

So, what is happening with ES in South Africa? Have they come and gone? Are they still being used? Were they never here in the first place? And, foremost, are there any ES used in a business and managerial context?

2 DEFINITIONS

The following definition has been adopted for purposes of this paper:

Although this definition can be subjected to some criticism, it has been selected because it is readily understood by Information Systems (IS) professionals and users and it does not refer to the technical aspects of the system (e.g. inferencing, heuristics etc.). As Turban points out, this definition is a content-free expression, i.e. it means different things to different people.

About a decade ago the term "knowledge-based system" emerged. Initially it was treated as a synonym of ES, although perhaps more connotation-free in the commercial ES market place. Currently, there is a tendency to increase its scope slightly to include systems that perform tasks that do not require "experts" to perform, although they do require a certain level of competency in a specific problem domain [Methlie L.B., 1987:337]. Notwithstanding the fact that the term KBS is perhaps more favoured in a business environment, reference in what follows will only be made to ES since the original survey on which this paper is based used that terminology.

The three traditional components of an ES are:

In practice, even an only moderately sophisticated ES will have a much more complex structure incorporating a black board, explanation facility, conflict resolver, exception handler, self-learning facility, scheduler, etc. Integrating object-orientation and neural technology with ES further compounds the situation.

Since virtually all ES provide support for decision-making in one area or another, every ES can be seen as a DSS in the wider sense of the word. However, in the IS literature, the term DSS has a much more specific meaning:

The important element of this definition is that it limits the scope to managerial environments. What motivates this distinction with the large number of ES in more technical or scientific areas?

For purposes of this research, therefore, ES with a functional management focus will be seen as a DSS.

3 BENEFITS OF ES

Southern Africa is faced with a relative shortage of highly skilled manpower in the areas of management and technical staff which constrains economic upswings by a lack of decision making abilities and technical skills and also contribute to South Africa's generally wide span of managerial control. South Africa's geographic isolation, complex educational problems and relatively small but sophisticated economy (from a global perspective) further compounds the relative scarcity of top experts in almost any given technical sub-domain. And although one must caution against the "cure-all" syndrome, proper implementation of ES technology could at least partially alleviate this shortage:

The management potential of ES could also be illustrated in a perhaps more humorous way by an experiment whereby a management expert system developed by professor Duffy (University of the Witwatersrand) beat all ten teams of "real, human" businessmen in a business management game [Anon., 1989].

Another concern is the low growth in white collar productivity, despite the dramatic increases in corporate IT budgets. The problem seems to be one of information overload: more and more data is being delivered by the information systems but they do not aid in the consequent decision making. This is where ES have an important potential role to play: to automate the routine decision making process [Buckler, 1990].

 

4 METHODOLOGY

This research can be seen as a natural progression of other South African theoretical [Mentz, 1988; Jacobson, 1989] and empirical [Keating, 1991] research on ES. In particular, it must be emphasized that this is the first local in-depth survey of its kind and it will hopefully serve as a sound scientific basis to evaluate future developments in the ES area.

The authors conducted a survey to investigate the current state of ES application and usage within South African organizations. The sampling frame was "South African informed organizations or individuals who are already familiar with ES technology", as identified through the EXPERT conference delegates' register and the CSSA's Special Interest Groups.

A total of 420 questionnaires were posted, from which 156 replies (or 37%) were received. This compares to the already high 20% response rate of Keating [1991:67] and the 14,7% recorded in a similar USA survey [Philip & Schultz, 1990:57]. 80 respondents (or 52%; 3 questionnaires could not be used) provided information on one or more expert systems in use or under development. The users reported on a total of 128 unique expert systems. The questionnaire provided space for a maximum of 3 ES; 8 users indicated that they knew of more ES within their organization, adding up to another 26 non-documented ES.

Analysis of this database of ES information revealed a significant sub-set of ES that are applied in a commercial business environment. This research therefore focuses on those documented ES which are in the area of functional business management.

5 APPLICATION AREAS

A total of 46 fully documented management-related ES were found, used in a number of functional areas. These are summarized in Table 1, although some more or less arbitrary classification decisions had to be made by necessity. The list of (respondent supplied) ES descriptions is provided available in the addendum. For ease of reference, these ES will hereafter be referred to as managerial or management ES.

This confirms the traditional view: "[T]here are several areas where knowledge-based systems are selling quite well. One is the financial services industry, where they can help process credit applications [...] Another is manufacturing and distribution, where companies are using them to manage order processing and scheduling." [Buckler, 1993:12] However, the six ES in human resources came as a surprise to the authors. Could these be indicative of a niche market which South Africa is developing?

For comparison purposes, it is interesting to note the areas in which SA "technical and scientific" ES are deployed: engineering and manufacturing control (41); agriculture, forestry & ecology (9); computer systems (8); medical (4); military (4); and other (4). It must be noted that some of military, forestry and manufacturing ES also support management-like decision making but their fields can arguably be said to be of a much more technical nature.

6 PROJECT INITIATION AND DEVELOPMENT

Table 2 lists the initiators for the 36 ES projects where this data was available. More than a quarter of the ES are initiated by Upper management and the IS/DP department each, respectively. It is evident from this table that "upper management" and IS/DP play a much more significant role in instigating the ES than is the case for the other, more technical ES in the database which are almost dominantly proposed by the end-users (often technicians or professionals). This difference is statistically significant at a 5% level of confidence (χ test value is 10,99; 4 degrees of freedom).

Similarly, the responsibility for actually developing the ES shows marked differences. Figure 1 shows clearly that the managerial ES are mainly developed by the corporate IS/DP department while non-managerial ES are predominantly developed by sub-contractors. However, the 19% of "other ES" raises methodological problems for a statistically meaningful conclusion.

 

7 USAGE AND SUCCESSFULNESS

The acid test for measuring the relevance and appropriateness of ES technology is to be found in the actual use of the ES. Of the 46 ES, 29 were "live" i.e. in actual use. Managerial ES appear to have been in use slightly longer and are being used more frequently (see figure 3) than the other ES in the database, where only just more than half of the ES are in actual use. However, the differences are too small to be statistically conclusive.

The following (non-exclusive) reasons why managerial ES were not in use (number of responses between brackets):

_ still under development or testing (10);

_ lack of user support (6);

_ no organization fit: does not fit into existing work flow or systems (3);

_ change in business needs and/or environment (1)

_ no specialized staff available (1);

_ don't know (1).

Apart from the acid test "is the system used?", the respondents were also asked how they viewed the success of the ES. Interestingly, virtually all rated their ES as at least satisfactory:

_ 9 ES were judged very successful;

_ 9 as successful;

_ 10 as satisfactory and

A possible correlation between the party who initiated the initial ES and the system's perceived success was investigated. The cross-tabulation in table 3 suggests that the relatively few user-initiated ES were less likely to be considered successful than those ES initiated by others. Unfortunately, individual cell numbers are too small (all less than 5) to test this relationship statistically.

8 SOME TECHNICAL CHARACTERISTICS: SIZE AND PLATFORM

Various measures can be employed to measure the size of an expert system. The most universally employed measure for classic' i.e. rule-based ES is the number of inference rules in its rule-base. Figure 4 gives the system sizes for all 46 ES and Figure 5 compares the respective sizes (in slightly aggregated format) of:

a) the managerial ES;

b) the other South African ES;

c) the findings for US systems [Schultz, 1990].

From this comparison, there is hardly any difference between the sub-sets of SA ES. When compared to the USA, though, there are relatively more South African "micro"-ES at the expense of "mini"-ES. This finding is statistically significant at α = 5% (χ test value fol ALL SA ES is 7,90; 2 degrees of freedom).

However, just as measuring information systems developed in third-generation language through the number of lines of code is not ideal, the number of rules criterium is not ideal either. This is especially so in the light of the growing importance of object-orientated and neural network-based ES. Another proxy for system size is the corporate investment in the ES through the development time. Although not quite as obvious as investment measured in Rand-cost, it is usually much more easy to obtain or gauge from the respondents. Table 4 details the findings and compares with the other and US systems respectively.

On the whole, there are again a larger proportion of relatively small South African ES, statistically significant at α = 5%; (χ test value fol ALL SA ES is 12,91; 4 degrees of freedom; critical value for 5% = 9,49, critical value for 1% = 13,28) when compared to the Schultz [1990] data. However, this tendency is far less pronounced when one considers only the managerial ES.

A logical hypothesis from the foregoing would be to assume that the two proxies for system size (number of rules and development time respectively) are positively correlated. This is indeed statistically supported by a very wide margin (χ=31.70; 4 degrees of freedom grouping data in a 3x3 grid) at a 1% level of confidence for the entire database. Although individual cell values are too small to conduct meaningfully statistical tests for the subsets, a visual inspection confirms that this holds true also for the respective subsets.

Another important practical aspect concerns the development and operational platforms for the ES. It is evident from Figure 6 that most managerial ES run on a PC environment and make use of a high-level ES shell. However, when compared to the other ES in the SA database, a relatively greater number make use of mainframe computer. Note that some ES use dual platforms.

It is further interesting to note that prototype ES are used for four out of every five systems, which is more than for the rest of the SA database (73%) but still less than the proportion in the US as reported by Schultz (92%; 1990).

A last concern, less technical but still development-related, is how the knowledge is `extracted' from the experts, to serve as input for the ES. As is shown in table 5, three-quarter of the ES use individual interviews between the manager (expert) and the knowledge engineer (ES developer) with the analysis of written documents coming in as a distant but still distinct second place.

9 OTHER SURVEY RESULTS

Many other ES aspects were investigated by the authors. Although space does not allow a full discussion, a summary of the more pertinent issues should be of interest to the readers concerned with in the application of ES technology in organizations. Note that the results reported in this section are drawn from the entire ES database, not those of the managerial ES alone.

9.1 Strategic Importance of ES

Of the 65 users who gave a definite answer to "Have strategically important ES been developed", less than a quarter (15 or 23%) replied YES. However, when asked "whether strategically important ES are envisaged in the future", more than twice as many answer affirmatively (36 YES against 17 NO) although a further 21 don't know.

9.2 Benefits Gained from and Problems Experienced with ES

Users were also asked to indicate the importance of a number of (listed) benefits. The results in Table 6 have been ranked in approximate order of importance. The high ratings (most benefits scored a median value of "major" or higher) illustrate that respondents agree substantially with the benefits generally ascribed to ES in the literature. Equally interesting are the low rankings accorded to the competitive advantage and personnel savings respectively. The low ranking of the strategic benefit seems to concord with the small number of strategically important ES mentioned above. It is also clear that few users expect ES to substitute for humans, an implied threat which is often associated with the introduction of a new information systems technology and, almost just as often, proves to be unfounded in reality.

Note that the ranking of the benefits that were experienced does not correlate at all with the tentative results obtained by Keating [1991] with a Spearman's rank-correlation coefficient which is actually negative (r'= -0,38, n = 11). Her sample may have been too small for reliable statistical conclusions, though. However, there is a strong statistical correlation with the ranking obtained by Philip & Schultz [1990]: although the Spearman's rank-correlation coefficient of 0,80 fails to reject the null-hypothesis of no positive correlation between the RSA and USA ranking (n = 5, a different set of benefits was used as compared to Keating's list), the z test statistic of 1,60 is very close to the critical z0,05 of 1,64 (one-sided test since only positive correlation is investigated).

The most important problems experienced were: the lack of commitment from a human expert; identifying and technical limitations of tools; cost justification; maintenance and validation of the knowledge base; the long development cycle and user acceptance/resistance. It must be noted that few problems were rated as particularly problematic by more than a third of the respondents.

9.3 Opinions Regarding the Future of ES

When asked to express their agreement or disagreement with a number of statements, the following were the statements that most users agreed with:

9.4 Barriers to ES Use

The respondents who did not have any ES in their organizations were asked to identify potential barriers to the use of ES. There seemed to be a reasonable consensus among non-users that internal opportunities do exist for ES applications, but few indicated that there were concrete plans afoot to exploit these opportunities.

When some further probing was done to identify more specific and immediate barriers, the following reasons were advanced by more than a third of the non-users:

It seems that the need to put other systems in place first (first two reasons), together with a critical lack of skilled manpower (next three reasons), are the main barriers to ES development. It is evident that both issues are interrelated. This seems to be a common thread throughout the IS industry and again emphasizes the need for more IS training, improvements in IS productivity and perhaps better IS/IT management?

10 CONCLUSION

From the discussion it is clear that ES technology has penetrated the business world fairly deeply: 46 expert systems were documented along various dimensions. Although most are in the traditional ES arenas (production, finance, insurance and customer support), the authors were startled to find six applications in human resources management.

Overall, the characteristics of South African managerial ES are fairly similar to those of the other, more techno-scientific ES in the database. Where the profiles do differ statistically, there seems to be a fairly intuitively acceptable explanation.

It is hoped that this paper will give a better perspective on the potential for the technology in the managerial field, as well as the understanding that the South African business community is perhaps more ready than many academics suspect, to embrace the relevant technologies where they can yield effective results.

11 REFERENCES

 

AMBROSIO J. (1990) Expert Systems Make Their Mark in Corporations. Computerworld, Vol.24 No.32 p.10.

AGARWAL R. & TANNIRU M.R. (Summer 1990) Knowledge Acquisition Using Structured Interviewing: An Empirical Investigation. Journal of Management Information Systems, Vol.7 No.1, pp.123-140.

ANON. 1988. Silicon Expertise. Knowledge-based Systems. Computer Mail, pp.39-46.

ANON. 1989. Game for Experts: A computerised expert system developed in SA has beaten 10 teams of businessmen at their own game. Computer Mail, pp.48-49.

ANON. (1990, May) Expert Systems Shrug Off Slow Start for Good Growth According to Frost & Sullivan Report. Information Today, Vol.7 No.5, pp.43-44.

BENBASAT I. & NAULT B.R. (1990) An Evaluation of Empirical Research in Managerial Support Systems. Decision Support Systems, Vol.6 No.3, pp.203-226.

BUCKLER G. (1993) The Experts Have Departed. Computing Canada, Vol.19 No.10, p.12.

BUCKLER G. (1990) Expert Systems Making Commercial Inroads: AI Now Viewed as Another Class of Development Tools. Computing Canada, Vol.16 No.15, pp.28-29.

JACOBSON M.A. (1989) The Use of Commercial Expert Systems in the Business Environment. Technical Report (Hons). Cape Town: University of Cape Town (Accounting Department).

KEATING L.M. (1991) The Extent of Expert System Usage. MBA Research Report. Johannesburg: University of the Witwatersrand.

KIM J. & COURTNEY J.F. (1988) A Survey of Knowledge Acquisition Techniques and Relevance to Managerial Problem Domains." Decision Support Systems, Vol4 No3, pp.269-284.

KOPCSO D., PIPINO L. & RYBOLT W. (1989) A Comparison of the Manipulation of Certainty Factors by Individuals and Expert System Shells. Journal of Management Information Systems, Vol.6 No.3, pp.66-81.

MENTZ M. (1988) The Integration of Expert Systems and Decision Support Systems to Solve Business Problems. Technical Report (Hons). Cape Town: University of Cape Town.

METCALFE, R.M. (1993) What Happened to Artificial Intelligence? InfoWorld, Vol.15 No.15, p.48.

METHLIE L.B. (1987) On Knowledge-based Decision Support Systems for Financial Diagnostics in HOLSAPPLE C.W. & WHINSTON A.B.(ed.) Decision Support Systems: Theory and Application. Berlin: Springer-Verlag, pp.335-372.

PHILIP G.C. & SCHULTZ H.K. (1990) What's Happening With Expert Systems? A Survey of Expert System Users. AI Expert, Vol.5 No.11, pp.57-62.

TURBAN E. (1990) Decision Support and Expert Systems: Management Support Systems. New York: Macmillan.

VAN BELLE J.P. (1992) A Survey of the Current Status of Expert Systems in South Africa. Technical Report (B.Com Hons). Cape Town: University of Cape Town.