Towards an evaluation framework for complex social systems
IRD, University of Strathclyde
University of Strathclyde
Last modified: August 15, 2006
There is increasing recognition amongst both policy makers and practitioners that many initiatives, projects and organisations are complex systems whose non-linear interactions and co-evolution with their environment make them extremely difficult to understand, predict or control. Such systems can be highly creative, however the danger of 'destructive interference' must be recognised; well-intended policy or design of systems ranging from learning communities through innovation programmes to the UK National Health Service have often failed to deliver the anticipated benefits in practice.
While traditional control mechanisms are often ineffectual, completely abandoning such systems to ‘self-organisation’ without some kind of ongoing assurance of success is a considerable act of faith, one which most policy makers or funders are unwilling to contemplate. Even when the advantages of self-organisation are understood, there is difficulty in reconciling the complex systems approach with traditional funder, or project or organisational management expectations.
One potential way forward is to develop a method for evaluating complex social systems. Evaluation – systematic determination of the merit, worth and significance – of such systems is non trivial. Like traditional control mechanisms, current evaluation frameworks do not take cognisance of the complexity of the systems. Different players, be they individuals groups or organisations, will have different opinions on the value of the system. Understanding these differing values could however build a rich picture of the system enabling more informed analysis of the success of the system and identifying areas which need to be addressed. Developing an evaluation framework for complex systems is not a trivial task.
This paper combines review of evaluation methods and complex systems theory with input from practitioners and policy makers to examine how evaluation techniques might be usefully applied in example complex social systems. This is used to develop an investigatory framework for complex systems evaluation. This framework can be used not only to investigate evaluation of complex social systems but also can help to bridge the gap between complex systems theory and real life practice, enabling articulation of complexity concepts to practitioners in a relevant and contextualised manner.