A Framework for the Analysis and Evaluation of Enterprise Models

 Jean-Paul W. G. D. Van Belle

Thesis Submitted for the Degree of Doctor of Philosophy to the Department of Information Systems University of Cape Town

 February 2003

 

Available Downloads

This thesis was submitted on 16 Feb 2003 and minor changes were made to various chapters, but I haven't gotten around to making the changes on the website yet. The full thesis (first version) excluding appendices) and the XML database with enterprise models are available for download only by special permission (because of copyright reasons) - the UNZIP requires a password.

However, the individual chapters are available as downloads.

  1. Introduction and Overview

  2. Modelling theory (lit survey)

  3. Reference disciplines for enterprise modelling

  4. Methodology used in the thesis

  5. A framework for enterprise model analysis

  6. A survey of enterprise models

  7. Syntactic analysis of enterprise models

  8. Semantic analysis of enterprise models

  9. Pragmatic analysis of enterprise models

  10. Other uses and applications of the framework for model analysis

  11. Conclusion

  12. References - those used in the thesis and the full list or sources which I used.

Abstract

The purpose of this study is the development and validation of a comprehensive framework for the analysis and evaluation of enterprise models.

The study starts with an extensive literature review of modelling concepts and an overview of the various reference disciplines concerned with enterprise modelling. This overview is more extensive than usual in order to accommodate readers from different backgrounds.

The proposed framework is based on the distinction between the syntactic, semantic and pragmatic model aspects and populated with evaluation criteria drawn from an extensive literature survey.

In order to validate and operationalize the framework, an exhaustive survey of enterprise models was conducted. From this survey, a database of more than twenty relatively large, publicly available enterprise models was constructed. A strong emphasis was placed on the interdisciplinary nature of this database and models were drawn from ontology research, linguistics, analysis patterns as well as the traditional fields of data modelling, data warehousing and enterprise systems. The resultant database not forms the test bed for the detailed framework-based analysis and its public availability in XML format should constitute a useful contribution to the modelling research community.

The bulk of the research is dedicated to implementing and validating specific analysis techniques to quantify the various model evaluation criteria of the framework. The aim for each of the analysis techniques is that it can, where possible, be automated and generalised to other modelling domains.

The syntactic measures and analysis techniques originate largely from the disciplines of systems engineering, graph theory and computer science. Various metrics to measure model size, hierarchy, architecture and complexity are tested and discussed. It is found that many are not particularly useful or valid for enterprise models. Hence some new measures are proposed to assist with model visualization and an original model signature consisting of three key metrics is proposed.

Perhaps the most significant contribution of the thesis lies in the development and validation of a significant number of semantic analysis techniques, drawing heavily on current developments in lexicography, linguistics and ontology research. Some novel and interesting techniques are proposed to measure, inter alia, domain coverage, model genericity, quality of documentation, perspicuity and model similarity. Especially model similarity is explored in depth by means of various similarity and clustering algorithms as well as ways to visualization the similarity relationships between models.

Finally, a number of pragmatic analyses techniques are applied to the models including face validity, degree of use, authority of model author, availability, cost, flexibility, adaptability, model currency, maturity and degree of support. This analysis relies mostly on the searching for and ranking of certain specific information details, often involving a degree of subjective interpretation, although for some criteria more specific quantitative procedures are suggested.

To aid future researchers, a separate chapter lists some promising analysis techniques that were investigated but found to be problematic from methodological perspective. More interestingly, this chapter also presents a very strong conceptual case for how the proposed framework and the analysis techniques associated with its various criteria can be applied to many other areas of information systems research. The case is presented on the ground of the underlying isomorphism between the various research areas and illustrated by suggesting the application of the framework to evaluate web sites, algorithms, software applications, programming languages, system development methodologies and user interfaces.

Acknowledgements

This work would not have been possible without the unwavering emotional, motivational and practical support from my dear wife Eva words will never be able to express my thanks and appreciation. I also thank my lovely children Anneke, Jonathan and Sylvia for their patience and understanding, especially in the last phases of the thesis where daddy was all work and no play. Only people who have been through the Ph D experience themselves, can appreciate the sacrifices that close and loved family members go through in order for the thesis to be written. Without my familys steadfast support, this research would have been still-born and this thesis represents almost as much the culmination of their hard work as mine.

Of course, I must also thank my parents, Roger and Monica Van Belle-Deleu, who gave me my life, education and the seeds of all my inspirations.

Thanks also to my supervisors, without whom this thesis would have been stillborn: Prof Jonathan Miller who fired the starting gun, Prof Paul Licker who counted the laps, and Prof Dewald Roode who manned the finish line.

My colleagues from the information systems department here at UCT deserve special thanks: Prof Derek Smith, Prof Mike Hart, Mike Eccles, Graham McLeod, and Jane Nash - for their support, guidance and practical assistance. Many deans who saw me through and provided financial supported whenever I called on them: here at UCT: Prof Peter Kritzinger, Prof Brian Kantor and Prof Douglas Pitt; and from my years at UWC Prof Andr Kritzinger and Prof Renfrew Christie.

The following people shared their time, office, knowledge and often granted me hospitality at their homes: Tom Russ (Information Sciences Institute, University of Southern California), Henry Kim (Industrial Engineering, University of Toronto), Wouter Jansweijer (Social Science Informatics, University of Amsterdam), John Kingston (AIAI, University of Edinburgh), Elizabeth Post (Lincoln University), Prof Richard Weston and Paul Gilders (Manufacturing Systems Integration Research Institute, University of Loughborough). Prof Glen Lowry (Convenor, Information Systems Research Group, University of Tasmania), Prof Peter Bernus (Griffith University, vice-chairman of the IFIP/IFAC Task Force on Architectures for Enterprise Integration) and Prof Andy Bytheway (UWC) helped me scope this research, provided valuable high-level direction and shared their invaluable experience. Constructive criticism was also received from Prof. Trevor Wegner (UCT), Mike Uschold (AIAI, University of Edinburgh) and Jean-Louis Van Belle (University of Antwerp).

I also wish to thank the following organisations who helped fund my research travel expenses and conference attendance: UCT, UWC, and the CSD/NRF. Both UCT and UWC granted me six months paid sabbaticals which enabled me to pursue and finalize my research and the CSD/NRF helped fund some of my research and conference expenses. Thanks also to the many conference organisers and delegates who attended my conference presentations and gave valuable constructive feedback.

Finally thanks to three special friends. Jean-Louis Van Belle, for his motivation by challenging me to finish before him (he owes me a Belgian beer). Dr Stephen Craven, for his excellent proofreading, his homebrews and serving as a living example that it is possible to do your thesis part-time. Last, but definitely not least, Caroline Brawner helped with some of the data entry, revitalized my energies by dragging me off to many mountain runs, proofread the entire thesis and provide some valuable criticism.

Jean-Paul Van Belle 's  Main Home Page

CV Publications Travel Sports