D. Davies (2024). The Unaccountability Machine. Why Big Systems Make Terrible Decisions — and How the World Lost Its Mind, Profile Books, 304 pp.
A passenger is refused permission to board an aircraft and makes a plea to a sympathetic flight attendant. Regrettably, the passenger’s grievance pertains to a company policy, and the flight attendant is unable to assist. This scenario exemplifies a scenario in which the onus of decision-making is entrusted not to an individual, but rather to a set of predetermined guidelines.
‘The unsettling thing about this conversation is that you progressively realise that the human being you are speaking to is only allowed to follow a set of processes and rules that pass on decisions made at a higher level of the corporate hierarchy. It’s often a frustrating experience; you want to get angry, but you can’t really blame the person you’re talking to. Somehow, the airline has constructed a state of affairs where it can speak to you with the anonymous voice of an amorphous corporation, but you have to talk back to it as if it were a person like yourself.’ (p. 16)
Davies poses the question of why errors and crises never appear to be attributable to any specific individual or entity. The blame is often attributed to ‘the system’, ‘the computer’, ‘the algorithm’, ‘the AI’, or similar entities. Davies terms this phenomenon an ‘accountability sink’, a structural element that absorbs or obscures the consequences of a decision, thereby preventing direct accountability.
The phenomenon of accountability sinks, as postulated by Davies, serves to impede individuals from both willing and unwillingly making or changing decisions and, consequently, being held directly accountable for them.
‘For an accountability sink to function, it has to break a link; it has to stop feedback from the person affected by the decision from affecting the operation of the system. The decision has to be fully determined by the policy, which means that it cannot be affected by any information that wasn’t anticipated.’ (p. 17)
Which means that:
‘The communication between the decision-maker and the decided-upon has been broken – they have created a handy sink into which negative feedback can be poured without any danger of it affecting anything.’ [1]
These sinks are everywhere in daily life, and everyone experiences them. Illustrative examples include the cancellation of a flight by an airline or the declaration of ineligibility for a benefit by a government agency. These occurrences can be conceptualized as broken connections between the individuals who are subject to the repercussions of the decisions made and the individuals responsible for formulating these decisions. This phenomenon effectively subverts the fundamental legal principle of accountability, as defined by Davies:
‘The extent to which you are able to change a decision is precisely the extent to which you can be accountable for it, and vice versa.’ (p. 17)
A further mechanism of accountability sinks is the manner in which decisions themselves cascade and cannot be traced back to their origins. Who should be held accountable for such a decision?
According to Davies, the proliferation of accountability sinks extends beyond the scope of aggrieved customers. In the field of finance, crises tend to reoccur, a phenomenon with which Davies is intimately acquainted, given his extensive experience in the sector. However, it is noteworthy that few bankers are held accountable for their actions. Politicians, too, have been known to complain of their inability to effect change when their promises prove fallible. Davies attributes the genesis of this predicament to the management revolution that took root in the aftermath of the Second World War, a period that witnessed the emergence of affordable computing power and, later, the ascendance of ‘data-driven decision-making’. Computers became a convenient accountability sink, which introduced a distortion in the feedback that underpins organizational decision-making, excluding, it should be noted, data-driven approaches that never were open for feedback. Algorithms and feedback do not really match. However, the increasing complexity of the world, the necessity for standardization, and the continuous management of complex environments have rendered computing and data-driven decisions inevitable.
The question, therefore, is not to return to an organization without non-human decision-making: it’s (1) ensuring that (data-driven) decision-making is open to feedback, (2) feedback information is communicated to the right organizational level to allow for changes in decisions, and (3) it is able to adapt to changing environments.
Davies’s view is that economics is flawed and that this is one of the reasons why accountability sinks have such negative consequences. Milton Friedman is Davies’s scapegoat. He identifies Friedman’s doctrine of ‘shareholder value maximization’ as a contributing factor to the accountability crisis. It created a situation in which managers and politicians could use the excuse of the primacy of the market in determining outcomes to obfuscate accountability. According to Davies, the doctrine was the main reason for the creation of the most significant accountability sink of recent decades. It led to the marginalization of other organizational objectives and the neglect of other stakeholders. Friedman’s approach entailed a simplification of the decision-making process and an over-reliance on the notion that ‘the market’ would integrate all available information into management and process optimization. While Friedman’s approach may have had its theoretical underpinnings, its practical implementation has been marred by a preoccupation with short-term goals, leading to phenomena such as short-termism, greed and excessive organizational misconduct, as I described in my book The Accountability Puzzle. [2] Davies really does have a point here.
Davies’s hero is Stafford Beer, a management consultant, information theorist and philosopher. Davies posits that the solution to the issue may be found in Beer’s ‘management cybernetics’. Beer proposed that the objective of resolving organizational issues is not the elimination of market failures, but rather the maintenance of a balanced feedback loop (and the flow of information to enable feedback) between decision-makers and those who are subject to their decisions. Beer’s seminal concept, the ‘purpose of a system is what it does’ (POSIWID), emphasizes the significance of an organization’s impact over its professed intentions. Davies delineates the five subsystems that Beer identified as indispensable for organizational functioning, employing these subsystems to meticulously analyse where organizations went wrong with accountability. These five subsystems are: (1) Operations, the part of the organization that makes changes in the environment and could function as a separate organization with its own management; (2) Regulations, enforcing rules for sharing and scheduling resources; (3) Integration, directing managing of individual operations of systems (1) and (2) to achieve purpose; (4) Intelligence, anticipating changes and directing subsystems (3) and (2) to plan for and cope with potential futures; and (5) Identity, making the decisions which determine POSIWID and the organization’s purpose. (pp. 115-136)
Beer’s proposition (and Davies’s adherence to this view) asserts that management information and action are synonymous. The definition of ‘information’ as outlined by Beer is contingent upon three factors: firstly, the timely arrival of the information; secondly, the information’s comprehensibility and relevance; and thirdly, its contribution to the decision-making process. This assertion, while valid from a cybernetics perspective, might not hold up under scrutiny from alternative viewpoints.
Davies’ argument is that the rise of modern shareholder capitalism and Friedman’s theory of ‘shareholder value maximization’ have resulted in organizations losing their subsystem 5, and even their subsystem 4, leading to a broken feedback loop, stagnating organizations incapable of adapting to changing environments, and accountability sinks that create cascading decision levels and obscure responsibility and accountability.
Practical recommendations are not specified, except for one. Davies’s most salient recommendation is to regulate the buyout industry to mandate private equity investors to guarantee the debts of takeover targets. This, it is argued, would serve to eliminate the accountability sink of limited liability. Ultimately, Davies calls for the overthrow of mainstream economics and economic orthodoxy, a sweeping statement that lacks specificity.
Davies’s book is very entertaining, interesting, and ingenious. It succinctly and (I think) correctly analyses the problems of economics and Friedman’s model of ‘shareholder value maximization’. The concept of accountability sinks is ingenious and very recognizable. Davies’s hypothesis that these ‘sinks’ are the result of a broken feedback loop also seems logical. But one question remains for me: what is the precise link between management cybernetics and ‘accountability sinks’?
Davies could have written his book without studying Stafford Beers and management cybernetics. Broken feedback loops have long been known, not only in cybernetics. His chapters on cybernetics are not the core of his hypothesis, although they provide him with a framework to explain ‘accountability sinks’. The problems of mainstream economics, Milton Friedman and his theory of ‘shareholder value maximization’, the impact of private equity investors and the need to regulate this buyout industry are the core of his work. These issues are, in his view, responsible for the growth of ‘accountability sinks’ by encouraging short-termism and removing the ability to adapt to changing environments. Cybernetics doesn’t even feature in his recommendations.
But is this the only explanation for the existence of the ‘accountability sink’? I do not think so. I suspect that other (possibly closely related) phenomena play a role, such as organizational behaviour, information and organizational cultures and climates, the neglect of the transformative power of information technology, the mistaken belief that ‘everyone’ knows how to process information, and others. These phenomena may all be the result of the emergence of ‘rationalist and unheroic’ bureaucrats who are ‘ill-equipped to face the problems, both domestic and international’ and who have therefore tried to hide their incompetence by following economic models and standardization efforts, defining stagnating policies and rules, creating multiple layers of bureaucracy to make decision processes opaque and difficult to follow, and initiating data-driven decision-making (citations from Joseph Schumpeter (1942). Capitalism, Socialism, and Democracy, Harper & Brothers, New York, p. 137-138.)
That said, Davies has written an interesting book and offers a valuable new concept: the ‘accountability sink’. It is a new way of looking at how organizations work, behave, and respond to environmental changes. The comparisons with artificial intelligence are obvious: delegating decisions to an algorithm is a very convenient way of creating a ‘sink.’ How to fight such practices will be a question that urgently asks for an answer.
[1] M. Rubenstein, and D. Davies (2024). ‘Decisions Nobody Made. Dan Davies Introduces His New Book. Plus: Earnings Season!’. Net Interest. Online source. https://www.netinterest.co/p/decisions-nobody-made. Archived at: https://archive.ph/YYRUQ.
[2] G.J. van Bussel (2021). An Accountability Puzzle. Organizations, Organizational Governance, and Accountability. Papers on Information and Archival Studies, II, Van Bussel Document Services: Helmond.
