D. Davies (2024). The Unaccountability Machine. Why Big Systems Make Terrible Decisions — and How the World Lost Its Mind, Profile Books, 304 pp.
A passenger is refused permission to board an aircraft and makes a plea to a sympathetic flight attendant. Regrettably, the passenger’s grievance pertains to a company policy, and the flight attendant is unable to assist. This scenario exemplifies a scenario in which the onus of decision-making is entrusted not to an individual, but rather to a set of predetermined guidelines.
‘The unsettling thing about this conversation is that you progressively realise that the human being you are speaking to is only allowed to follow a set of processes and rules that pass on decisions made at a higher level of the corporate hierarchy. It’s often a frustrating experience; you want to get angry, but you can’t really blame the person you’re talking to. Somehow, the airline has constructed a state of affairs where it can speak to you with the anonymous voice of an amorphous corporation, but you have to talk back to it as if it were a person like yourself.’ (p. 16)
Davies poses the question of why errors and crises never appear to be attributable to any specific individual or entity. The blame is often attributed to ‘the system’, ‘the computer’, ‘the algorithm’, ‘the AI’, or similar entities. Davies terms this phenomenon an ‘accountability sink’, a structural element that absorbs or obscures the consequences of a decision, thereby preventing direct accountability.
The phenomenon of accountability sinks, as postulated by Davies, serves to impede individuals from both willing and unwillingly making or changing decisions and, consequently, being held directly accountable for them.
Melvin Conway’s Law, first posited in 1968, states that the architectural structure of a software system mirrors the organizational structure of the entity responsible for its development.
In the event that an organization comprises multiple teams engaged in a single project, the resulting software system will likely exhibit a structure that reflects the communication patterns between those teams. Even the largest software development companies, such as Microsoft and Google, are susceptible to the influence of this ‘law’.
Although typically discussed in the context of software, the observation is applicable more broadly to systems in general. Conway posits that social problems such as poverty, healthcare, and education reflect (or mirror) the structures of government organizations.
Let’s take this one step further.
The architectural structure of the software may reflect the structural configuration of the organization that designed it. Ultimately, however, the implementation of the software reflects the structural configuration of the organization that uses it, even if the structural configurations of the developer and the user are disparate (or even contradictory).
Could this be a potential explanation for organizational difficulties encountered when implementing information and records management software?
In addition to the more obvious explanations regarding organizational and employee behaviour, of course.
I was inspired by Nohan Nayak’s insightful post below, in which he also raises the question of whether AI agents are influenced by Conway’s law. This is a very interesting question, as it would have significant implications if they are.
Further reading: Conway, M. E. (1968). ‘How do committees invent’, Datamation, Vol. 14, No. 4, 28-31. Bindrees, M. A., Pooley, R. J., Ibrahim, I. S., & Bental, D. S. (2014). ‘How public organisational structures influence software development processes’, Journal of Computer Science, Vol. 10, No. 12, 2593. Bailey, S. E., Godbole, S. S., Knutson, C. D., & Krein, J. L. (2013). ‘A decade of Conway’s Law: A literature review from 2003-2012’, 2013 3rd international workshop on replication in empirical software engineering research, IEEE, pp. 1-14.
Response to the following LinkedIn post:
Nohan Nayak, HowOrganizational Structure Impacts Software Design This relationship, known as Conway’s law, states that “organizations which design systems … are constrained to produce designs which are copies of the communication structures of these organizations”. There are several key ways in which organizational structure affects software design:
Communication and Coordination Software components that need to communicate frequently tend to be developed by teams that interact closely. Conversely, loosely coupled teams will produce more modular software designs.
Specialization and Expertise The division of labor and specialization of teams in an organization influences the modularity and layering of software systems. Teams focused on specific domains or technologies will produce components that are optimized for those areas but may not integrate as smoothly.
Decision-Making and Autonomy The decision-making structure of an organization, whether centralized or distributed, impacts the flexibility and extensibility of software. Centralized decision-making, such as in a chief programmer team, can produce more consistent designs but may limit innovation. Decentralized structures like egoless or democratic teams allow more experimentation but may lack coherence.
Organizational Inertia and Legacy Over time, an organization’s software tends to mirror the communication patterns and team structures that existed when it was originally built[1]. As an organization evolves, its software can become increasingly misaligned, requiring major refactoring efforts to untangle.
To mitigate the impact of organizational structure on software design, experts recommend:
– Aligning teams and components based on domain boundaries rather than technologies – Fostering cross-team communication and coordination through shared goals, tools, and processes – Empowering software architects to shape both the technical and organizational structures – Regularly reviewing and iterating the organization’s structure to keep it in sync with evolving software needs
How does organisational structure impact software design? And will AI Agents behave the same way?
This intriguing illustration sheds light on the dramatised organisational structures of Amazon, Google, Facebook, Microsoft, Apple, and Oracle, perfectly echoing 𝗖𝗼𝗻𝘄𝗮𝘆’𝘀 𝗟𝗮𝘄: “𝗦𝗼𝗳𝘁𝘄𝗮𝗿𝗲 𝗮𝗻𝗱 𝗽𝗿𝗼𝗱𝘂𝗰𝘁 𝗮𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗲 𝗶𝘀 𝗱𝗲𝘀𝘁𝗶𝗻𝗲𝗱 𝘁𝗼 𝗯𝗲 𝗮 𝗿𝗲𝗳𝗹𝗲𝗰𝘁𝗶𝗼𝗻 𝗼𝗳 𝘁𝗵𝗲 𝗼𝗿𝗴𝗮𝗻𝗶𝘀𝗮𝘁𝗶𝗼𝗻𝗮𝗹 𝘀𝘁𝗿𝘂𝗰𝘁𝘂𝗿𝗲 𝘁𝗵𝗮𝘁 𝗰𝗿𝗲𝗮𝘁𝗲𝗱 𝗶𝘁.”
This concept could become a critical hyperparameter in AI agent design, shaping their interactions and problem-solving approaches. Conway’s Law appears to be an emergent property of complex systems, but regardless, the fact that LLMs are trained on human decision-making makes it increasingly likely that Agents will emulate these same dynamics.
Karl Weick is een van de grote denkers binnen de organisatietheorie. Hij is hoogleraar Organizational Behaviour and Psychology aan de Ross School of Business van de universiteit van Michigan. Mijn eerste kennismaking met het werk van Weick was The social psychology of organizing uit 1969, dat een van de meest gelezen en bepalende boeken van de organisatiepsychologie werd en dat verplichte kost is voor iedereen die werkt of wil werken aan de verbetering van organisaties. Dit boek legt de grondslag voor Weick’s denken, dat vervolgens in al zijn publicaties verder wordt uitgewerkt, verdiept, verduidelijkt en geoperationaliseerd. Eenmaal gegrepen door Weick en overtuigt door zijn (veelvuldig) provocatieve, maar in de praktijk van organisaties gewortelde opvattingen, heb ik al zijn publicaties verzameld, gelezen, herlezen en (zo goed en kwaad als ik dat kon) toegepast in mijn eigen onderzoek- en adviespraktijk.
Volgens Weick bestaat er geen objectief waarneembare werkelijkheid. Organiseren is een sociaal proces van beïnvloeden en beïnvloed worden. Tijdens dit proces scheppen mensen langzamerhand een werkelijkheid. Uit de oneindige hoeveelheid variaties die optreden door veranderingen in de omgeving worden sommige wel en sommige niet geselecteerd door de leden van organisaties. Dit proces wordt ‘enactment’ genoemd en heeft in de organisatiekundige literatuur een status van algemene geldigheid verworven. Weick omschrijft het als volgt: ‘Managers construct, rearrange, single out, and demolish many ‘objective’ features of their surroundings. When people act they unrandomize variables, insert vestiges of orderliness, and literally create their own constraints’ (Social Psychology of Organizing, p. 243). ‘Enactment’ leidt tot een verbondenheid van een individu of groep mensen met de omgeving, maar het bereiken van overeenstemming over de juiste interpretatie van de ‘enacted environment’ vergt grote sociale inspanning. De interpretatie die uiteindelijk wordt gebruikt leidt tot een selectie van een dergelijke ‘enacted environment’ en leidt tot een bepaalde, op die selectie afgestemde reactie, die vervolgens in het ‘geheugen van de organisatie’ wordt opgeslagen. Die organisaties, waarin de leden geen overeenstemming kunnen bereiken over de te gebruiken interpretatie van de omgeving, zijn ten dode opgeschreven.
