Do Leaders Design Technical Debt?

Technical debt is normally discussed as an engineering problem. We think about ageing code, architectural compromises, deferred upgrades, rushed implementations and decisions that made sense under one set of circumstances but became expensive under another. Those are all legitimate forms of technical debt, but over the years I have become increasingly interested in another kind: the debt that accumulates even when nobody has made an obviously bad technical decision.

Sometimes a weakness becomes visible through normal production use. The organisation understands what happened, people agree that something should be improved and there is even a process for capturing the required work. The problem is known. The lesson has been documented. Nobody needs to rediscover it. Yet months later, nothing has changed.

That raises a slightly uncomfortable leadership question. If leaders influence priorities, measures, ownership, capacity, commercial models and the conditions under which work gets done, can leaders also design technical debt without ever touching the code? I think they can, although rarely in the deliberate sense suggested by the word design. More often, technical debt becomes the natural output of an operating model that rewards one set of behaviours while assuming another set will happen voluntarily.

I first began thinking about this differently after a critical customer incident.

A simple error and an expensive investigation

A batch process had stopped completing successfully, and because the process mattered to the customer, a sizeable group of technical specialists became involved in trying to understand why. The investigation followed the paths you would expect. Engineers were examining application behaviour, processing logic, infrastructure and the other areas where a failure of this type might reasonably originate.

I was involved because I was responsible for the customer outcome, but I was not there as the most technically specialised person in the room. My role during incidents like these was normally to understand enough of the discussion to challenge assumptions, make sure the right expertise was involved, communicate clearly with the customer and help the investigation keep moving in a useful direction.

At some point, one of the underlying data files was opened. Someone began moving quickly through hundreds of rows while the wider technical discussion continued, and I happened to keep watching the data. One row looked wrong. A couple of values appeared to have shifted into the wrong columns. We corrected them, ran the process again and it completed successfully.

After hours of specialist troubleshooting, the immediate cause was a small manual data error.

There is an easy lesson to take from a story like that: complicated problems sometimes have simple causes. That is true, but it was not the lesson that interested me most. I was much more interested in why such a small mistake had been capable of consuming so much specialist time.

The application had failed because it could not process the data it had been given, yet it had provided almost nothing useful to explain that. There was no meaningful error pointing towards malformed input, no clear indication of where the problem had occurred and no obvious diagnostic path leading the support team towards the affected data. The system knew enough to stop processing, but it did not give the people operating it enough information to understand why it had stopped.

That difference had turned an ordinary human mistake into a major troubleshooting exercise.

People will always make mistakes with data. They will copy information into the wrong field, enter an unexpected value, misunderstand a format or simply type something incorrectly. It is unrealistic to design systems around the assumption that perfect inputs will always arrive. Part of good software, in my view, is therefore not simply whether it works when everything is correct, but how usefully it fails when something is not.

Had the application identified the relevant input, record or validation failure, the original mistake would still have happened. The operational consequence, however, might have been measured in minutes rather than hours. Once the customer was working again, that seemed like the more useful lesson to take from the incident.

What happens to the lesson afterwards?

A process was subsequently created for feeding this type of experience back towards engineering. When Support encountered an unhelpful or generic error during an incident, an improvement could be raised so that better validation, logging or diagnostic messaging might eventually be added to the product.

Conceptually, I liked the idea because production should teach an organisation something. Support sees products behaving under conditions that are difficult to reproduce during development. Customers use software in ways designers did not anticipate. Inputs are imperfect, dependencies fail and rare combinations of events eventually stop being theoretical because, given enough customers and enough time, somebody encounters them. A difficult incident therefore contains information that can make the product better.

The problem was not the idea. It was what happened afterwards.

The improvement requests accumulated in a shared queue. There was no meaningful expectation around how quickly the queue should be addressed, no clear accountability for reducing it and no consistent mechanism for allocating the work. Engineers could pick up items from it, and some did, but much of the work was discretionary. Nothing was technically preventing the improvements from happening, while very little in the organisation was actively causing them to happen.

That queue gradually became interesting to me because every ticket represented something the organisation already knew. Somewhere, a production experience had exposed a weakness. A customer had experienced the consequence, Support had spent time investigating it, Engineering knowledge had been consumed and somebody had then taken the additional step of documenting what could be improved.

By that point, the organisation had already paid for the lesson through production experience. The queue was no longer simply a backlog of potential enhancements. It had become a ledger of weaknesses we already understood and lessons we had already purchased, but had not yet decided whether to use.

That is a very different problem from not knowing where technical debt exists.

When quality depends on professional pride

There was another aspect of this that made the situation more complicated. Some engineers naturally go beyond the minimum requirement. They think about what will happen when the happy path fails, add better validation because they know somebody will eventually need it, write useful error messages, improve logging or take a little extra time to make something easier to diagnose.

I have worked with many engineers like that. They do not necessarily need a formal process telling them that supportability matters because it is already part of how they think about good engineering. The difficulty begins when the organisation starts depending on that professional pride without formally recognising the work it produces.

Consider two engineers working under the same delivery pressure. Both implement the requested functionality successfully. One spends additional time improving error handling and diagnostic information before considering the work finished. The other delivers what was requested and moves on to the next item. From the perspective of visible feature delivery, the second engineer may appear faster.

The first engineer has also created value, but much of that value exists in future work that somebody will now never have to perform. An incident may be diagnosed in fifteen minutes rather than several hours. A support engineer may resolve something without escalating it. A customer may never need to involve senior stakeholders. Another developer may understand the problem immediately from the logs instead of reconstructing what happened from symptoms.

None of those avoided costs appears as another completed feature. If the organisation measures visible delivery much more clearly than future supportability, the behaviour it appears to reward becomes fairly predictable.

This does not require bad engineers or bad intentions. It simply means that one engineer is providing additional quality because they personally believe it matters, while another is delivering exactly what the organisation formally requested. If important engineering outcomes depend on people voluntarily supplying more than the operating model requires, then quality will inevitably become inconsistent.

The organisation effectively ends up with two definitions of quality: the quality it formally requires and the additional quality conscientious people provide because they believe the work should be done properly.

The gap between those two definitions is not simply an engineering issue. It is part of the operating model.

Outsourcing does not remove the incentive problem

The same issue can become even more visible when development work is performed by a systems integration partner.

An SI is normally contracted to deliver defined work. There is a scope, a set of requirements, acceptance criteria and a commercial agreement around what constitutes completion. Once that work has been accepted, the people who built it may move to another part of the programme, another customer or another project altogether. Their incentives are therefore quite reasonably connected to the thing they have been asked and paid to deliver.

If meaningful error handling, useful logging, operational documentation, observability or long-term supportability are not part of that definition, we should be careful about assuming they will somehow appear anyway.

The partner may never experience the eventual operational consequence. Six months after implementation, when something fails in production, a completely different group may spend hours investigating it. Support absorbs the effort, the customer experiences the disruption, internal engineering is pulled into an escalation and customer-facing teams manage the resulting loss of confidence. Meanwhile, the original implementation can still appear successful against the measures used to accept it.

This is where outsourcing can become more expensive than its commercial model initially suggests. The organisation may have purchased the implementation without purchasing enough of the operational quality required to live with that implementation afterwards.

I do not see that primarily as a criticism of the SI. If we pay somebody to complete the work defined in a ticket, it is difficult to criticise them for completing the work defined in the ticket. The more interesting question sits with the organisation buying the work: what did we actually ask them to deliver, and what did we define as finished?

If leadership decides which capabilities to outsource, how commercial agreements are structured, what acceptance criteria look like and which standards suppliers are expected to meet, then the long-term consequences of those choices belong partly to leadership as well.

Some technical debt may therefore be designed into the commercial and operating model before the first line of code is written.

The cost moves rather than disappears

One reason technical debt survives so easily is that its economics are often split across different parts of the organisation. Suppose another hour spent during development would have produced better validation and diagnostic information. Removing that hour can look like efficiency because the feature moves more quickly and the next piece of work can begin.

Months later, an unclear failure might consume many hours across Support, Engineering, Customer Success and leadership. A customer may lose productive time. An escalation may interrupt planned work. Engineers may be pulled away from current development to reconstruct what happened in software they last touched months earlier.

The original hour may therefore have been saved in one place and at one moment while the later cost appears somewhere entirely different. Support sees a more expensive incident, Engineering sees an interruption, Customer Success sees damaged confidence, the customer experiences disruption and leadership sees an escalation. Because those consequences are distributed across different teams and often arrive much later, the organisation can easily miss the connection between the original saving and the future cost it created.

This is not unique to software. Organisations frequently optimise one part of a system by moving cost into another. A department hits its target, a project meets its milestone or a budget is reduced while the wider organisation quietly absorbs the consequence somewhere less visible.

Technical debt is particularly good at hiding this way because its cost often arrives under a different name. Validation, logging, observability and meaningful error handling are therefore easy to undervalue because much of their return appears as work that never needs to happen: incidents resolved faster, escalations avoided, engineering interruptions prevented and customers who never have a reason to question the reliability of the organisation supporting them.

Prevented work is difficult to measure precisely because the most successful outcome is often that nothing happens.

Technical debt still involves legitimate trade-offs

None of this means every improvement discovered during an incident should automatically become a priority.

Engineering capacity is finite. Products contain thousands of potential improvements, many of which would be useful but economically difficult to justify. A diagnostic weakness that caused one minor incident in several years should not necessarily take priority over security work, contractual commitments, regulatory requirements or a major customer need.

Technical debt is partly the result of trade-offs, and leadership involves making trade-offs. I do not think good leadership means attempting to eliminate all technical debt, nor do I think every ticket raised after an incident deserves to be implemented.

The important distinction is between consciously accepting a trade-off and simply never making the decision.

If a known weakness is low risk and not worth the engineering investment, that can be a perfectly legitimate conclusion. If the improvement matters but the organisation lacks capacity, that is useful information too because it makes the capacity constraint visible. Perhaps priorities need to change. Perhaps additional investment is justified. Perhaps the same category of problem appearing repeatedly suggests that the answer is not another individual backlog ticket at all, but a change to engineering standards or the definition of completed work.

Ownership does not mean somebody must personally complete every improvement. It means somebody is responsible for ensuring that a decision exists.

The work might be fixed, prioritised, accepted as a known risk, incorporated into a wider standard or closed because the value no longer justifies the investment. What matters is that the feedback loop reaches a decision rather than stopping at a ticket.

Without that ownership, a queue can create the appearance of organisational learning while allowing the organisation to avoid acting on what it has learned. An incident can be thoroughly investigated, the root cause can be documented, an improvement can be raised and every formal process step can be followed, while the organisation itself remains essentially unchanged.

What production should teach leadership

I have therefore come to think that one of the most useful questions after an incident is not simply what caused it, but what should become different because it happened.

Production is where many assumptions made during design and development finally meet reality. Customers behave differently from test cases, data arrives in unexpected forms, dependencies behave differently under pressure and Support discovers that diagnostic information which appeared adequate during development may be nowhere near enough when a customer is waiting for an answer.

That experience is valuable information, but the organisation only benefits from it if there is a path from production back into the way the product is built and operated. Sometimes the lesson should result in a product change. Sometimes it should change an engineering standard. Sometimes documentation is enough. Sometimes the conclusion will quite reasonably be that the risk is understood and no further investment is justified.

The precise outcome matters less than the existence of a real feedback loop. Otherwise, incidents become isolated events. The organisation can become very effective at restoring service while remaining strangely capable of experiencing variations of the same problems again.

This is where I think the leadership aspect of technical debt becomes difficult to ignore. Leaders may never decide that error handling should be poor or that logging should be inadequate, but they do influence the conditions under which those things are created. They influence what gets measured, what receives capacity, where ownership sits, which standards are mandatory, how suppliers are contracted and what the organisation considers finished.

Those decisions shape behaviour long before a production incident exposes their consequences.

If feature delivery is highly visible while supportability remains mostly invisible, people will naturally optimise towards delivery. If operational improvements are captured but not owned, queues will grow. If quality depends on conscientious engineers voluntarily adding work that nobody formally requires, quality will vary according to who happens to be involved. If outsourced delivery is judged primarily by completion while long-term operational consequences remain with somebody else, both organisations can behave entirely rationally while future cost quietly accumulates.

None of this requires leadership to make a deliberate decision to create technical debt. It only requires an operating model in which accumulating that debt is easier than reducing it.

The original incident that changed my thinking was solved by correcting a very small error in a data file. That restored the customer's batch process. Better validation and error handling could have made the next similar problem easier to diagnose, but even that would not have addressed the wider question of why known improvements could enter an organisational process and then quietly stop moving.

That was not simply a coding problem. It was a question of ownership, incentives, standards, commercial design, visibility and priorities. In other words, it was also a leadership problem.

So when an incident is resolved, I now think there is value in looking beyond what failed and how quickly service was restored. What did the organisation learn from the experience? What, if anything, should become different because of that learning? And who owns deciding what happens next?

Technical debt is not always created because somebody took a shortcut. Sometimes it accumulates because everybody knows what the next step could be, but the organisation has been designed so that nobody needs to decide whether to take it.