Measuring quality?
by Ian Macduff
“And what is good, Phaedrus, and what is not good — Need we ask anyone to tell us these things?”
― Robert M. Pirsig, Zen and the Art of Motorcycle Maintenance: An Inquiry Into Values
“ISO 9001:2015 sets out the criteria for a quality management system and is the only standard in the family that can be certified to (although this is not a requirement). It can be used by any organization, large or small, regardless of its field of activity.”
“The standards provide guidance and tools for companies and organizations who want to ensure that their products and services consistently meet customer’s requirements, and that quality is consistently improved.”
This blog is part of a series leading up to the 2019 ODR Forum in Williamsburg, Virginia. There are two motivating themes or challenges that shape this entry. The first is that the overall objective of the 2019 Forum is to begin to measure success and progress (or not) in the last two decades of development in ODR and, as the Forum website states, to look at best practices for gathering the necessary data for that measurement. The second theme arose more by way of provocation at the recent (February 2019) annual commercial mediation competition in Paris, in which my colleague Greg Bond reminded me and the audience of mediators and law students that “quality” remains a metric, albeit elusive, that we’re obliged to consider.
On the first point, I’ll leave it to those who will be attending the Williamsburg Forum to determine the measures of success in ODR. The second point - Greg’s challenge - arose in the context of a panel discussion on the future[s] of mediation and, in particular, the ways in which digital technologies are already changing how we interact, solve problems, manage information, enhance access to legal and other resources and so on. I sought briefly to summarise some of the observations made and conclusions reached at the 2018 ODR Forum, and in particular to comment on the attention given to issues of ethics, equity, transparency, and accountability in the development of resources for online access to justice. Greg’s response was that, while he took issue with none of this - indeed, welcomed the prospect of enhanced access to dispute resolution resources - he was concerned at least to flag the importance of questions of “quality” in considering the services offered and institutions developed.
The immediate difficulty of course, as Pirsig found in his quest for “quality” was that it is a persistently elusive notion, neither wholly subjective nor wholly objective; neither wholly inherent in what we observe or entirely residing in the perceptions of the observer. As the ISO standards quoted above also suggest, “quality” is something that standards are designed to ensure . . . but quality itself is not necessarily reducible to a standard.
This question becomes important, too in the wider context of enhancing access to justice - whether through alternatives to courts or the development of digital pathways - given the mix of “drivers” and criteria for success in mediation or access to justice programmes. For fiscally-challenged justice systems, success - and quality - may well have more instrumental and efficiency criteria. For those who begin in a critique of or challenge to the “legalism” of conventional legal processes, success is likely to be measured in the enhanced autonomy and choices of disputants. For those who have been concerned that conventional processes typically exclude minority or indigenous peoples, the measure of quality will more likely turn on the degree to which indigenous processes are recognised and implemented. And so on.
The difficulty created by the last four decades of mediation, and all the more so by the contemporary digital, information-based development, is that the innovations are, almost by definition, scattered, distributed and networked, with a variable level of connection to the mother ship of law and state institutions. As Prof Luciano Floridi has discussed in a number of publications, a “distributed ethics” in the era of information brings a particular set of challenges to institutions, to their alternatives, and to conventional criteria of “quality”. Indeed, it’s almost written into the DNA of many of these alternative developments that the criteria of their legitimacy and success are not necessarily those of the mainstream institutions.
And yet it’s also clear that quality matters, for which reasons we see a plethora of standards of professional conduct; certifying bodies such as the International Mediation Institute, for example, have established clear criteria which need to be met by mediators and qualifying bodies; and may well become a question for courts to answer as to whether practitioners have met or failed to meet professional standards.
I doubt that Greg Bond’s challenge was an invitation to a conversation about Robert Pirsig’s “metaphysics of quality” or the question as to whether quality precedes any intellectual construction of it. But the fact that the question was raised is a reminder of the array of “qualities” that we might look for in the development of any practice or institution or innovation. As the question suggests, the quality - the attributes - to be measured are not just those of the instrumental efficacy of the innovation but rather those of the ends and the values that the innovation might be designed to meet. And those ends, those qualities, will be as diverse as individual autonomy, cultural appropriateness (but not appropriation), fiscal restraint, respect for public values (and not merely private ends), legal and constitutional coherence, gender and other forms of equity . . .
One way of thinking about this might be found in the theme running through Cal Newport’s book, Digital Minimalism: On living Better with Less Technology(2019): rather than thinking in terms of a rejection or denial of technology, the minimalists begin with the values that matter, then work out their relationship to technology and its uses on that basis. The risk of not doing so is that technology - and the designers behind it - come to provide the reasons for its expanded uses.