Designing failure in advance is as important as designing for success.

Failing early preserves resources; failing late extracts maximum value from them.

In human-AI teams, explicit agreement on failure direction is essential, not optional.

When you build and operate in high-stakes, chaotic environments, you have to plan for failure. The Stoics had a name for thinking ahead about this: premeditatio malorum—the deliberate contemplation of what will go wrong. Modern systems science uses the term failure modes: the specific ways a system will break. The more clearly you can map your failure modes in advance, the more you can design your system to move through, over, or around them. Of course, not all failures are equal, and not all failure directions are the same. In this post, we will consider three axes for distinguishing between failures and how each one can inform different design choices for complex systems operating in high-stakes environments.

Failing Toward vs Failing Away

Systems that fail toward something have proactively identified a secondary point of stability outside their normal model of operation. These secondary points are not ideal, but they are enough to keep operating at a reduced level, and failing toward them increases the chance of accomplishing the objective even when the first approach falls short. Human-AI teams use this “failing toward” mode of failure. When operating at full capacity, these teams balance complex inputs, perform complicated operations, and share decision authority between human and artificial elements. Human-only decision-making is a natural secondary point of stability, so a well-designed human-AI team can be set up to break toward human control when things go wrong, with the AI component failing first and the human elements continuing to drive toward the mission.

Systems that fail away from something have recognized a specific risk and are designed to break directionally away from it. As a very simple model, imagine walking along the edge of a cliff. If you're going to trip, it’s clearly better to fall away from the cliff, so most people instinctively lean slightly inward as they walk. Aircraft coming in for a landing also use a failing away approach. The chief risk is contact with the ground, so when a landing sequence isn't tracking correctly, the preferred response is often to fly over the runway, regain altitude, and come around again. They’re not activating a specific backup plan yet, just moving away from catastrophic failure. The more precisely you understand your system's secondary points of stability and its areas of concentrated risk, the more deliberately you can design which way it falls.

Failing Early vs Failing Late

Failing early allows systems to abandon a path of action before reaching irreversible consequences. Because the system theoretically still has capacity to keep going, failing early can feel like quitting before the job is done—so care must be taken to proactively identify and capitalize on early signals that a course of action inevitably and irreversibly leads to failure. A classic example comes from the startup world. Startups have limited resources and can lack the structural resilience of more developed companies, so they are routinely counseled to run low-cost experiments, test ideas quickly, and abandon unworkable paths as early as possible, preserving scarce resources for something that works. In his classic book Good to Great, Jim Collins calls this firing bullets, then cannonballs: using low-cost failures (“bullets”) to triangulate the right direction before committing serious resources (“cannonballs”).

Systems that fail late, by comparison, run existing plans all the way to the end of their usable life, extending operations as long as possible. These systems typically have a scarce resource that already has been committed to a set path: The cannonballs Collins talks about are already being fired. In the hospital, oxygen tanks work this way. We want them to run all the way out, delivering as much oxygen as possible to the patients who need it. Importantly, failing late should never mean failing silently: Oxygen tanks have gauges and alarms that signal depletion before the tank is empty, allowing teams to extract maximum value from a resource while still planning ahead.

Failing Partially vs Failing Completely

Partial failure is useful when a failed system retains substantial ability to operate, even if in a reduced or differential capacity. In the emergency department, many video laryngoscopes have partial failure modes. Laryngoscopes are tools for placing a breathing tube into a patient’s airway, with a curved blade and (for video models) a small camera just under the tip. By mimicking the design of classic non-video blades, manufacturers of many video laryngoscopes built in the capacity to fail partially: If the video stops working, the laryngoscope still functions as a standard non-video model, allowing the procedure to continue.

Complete failure is useful when a failed system might still appear operational but actually is not, or when continuing to operate risks cascading or catastrophic consequences. Imagine a bridge that has suffered serious structural damage in an earthquake but is still standing. It looks like it could still bear traffic, and maybe a few cars could indeed pass, but the city chooses to close it entirely because the risk of collapse is too high to justify keeping it open. In this case, the complete failure of closing the bridge to all traffic prevents cascading failures of injured people and destroyed property.

For all these failure modes, the key is building a shared mental model of how and why a system is designed to fail in a particular direction. Teams that disagree, with one member pushing to abandon course while another is pushing to run it out, will generate serious friction at exactly the wrong moment. Having the conversation before the crisis, and updating it as conditions change, is not pessimism. It's preparation. This is especially true for human-AI teams, where the human and artificial elements may not share the same intuitions about failure structure at all, making explicit agreement about failure direction not just helpful, but essential.

There was a problem adding your email address. Please try again.

By submitting your information you agree to the Psychology Today Terms & Conditions and Privacy Policy

© Psychology Today