July 2024 Outage: Engineer Warns Bank of England of Critical Financial System Vulnerabilities

Patrick McKenzie, widely known as "patio11," delivered a significant address to the Bank of England this week, as he announced on social media: > "This week for Complex Systems we're switching up the format a bit, since I was invited to give a talk at the Bank of England, on how a systems engineer thinks about systemic risk in a way that might be additive to central bankers." McKenzie, a prominent figure in technology and finance, presented his insights at Threadneedle Street, highlighting critical vulnerabilities in financial infrastructure. His talk notably referenced the widespread July 2024 CrowdStrike outage that severely impacted major U.S. banks and airlines.

The July 2024 incident saw a configuration change in CrowdStrike Falcon endpoint monitoring software, pushed by a single engineer, lead to "Blue Screen of Death" events on countless Windows PCs. This effectively halted teller operations at several of the largest U.S. financial institutions for an entire working day. McKenzie emphasized the "near miss" nature of the event, noting that while electronic systems largely remained operational, a coordinated failure could have been far more catastrophic.

McKenzie argued that regulatory guidance, such as that from the Federal Financial Institutions Examination Council (FFIEC) in the U.S., inadvertently fosters "software monocultures" where many institutions rely on the same vendor for critical functions. This creates single points of failure, as demonstrated by the CrowdStrike incident where a data change, rather than code, caused widespread disruption. He urged regulators to engage more with engineers to understand implementation-level risks and promote a culture of blameless postmortems.

Beyond the outage, McKenzie also touched upon the evolving landscape of stablecoins, acknowledging their "almost 3x growth in volumes" for business-to-business (B2B) payments, particularly for transactions around $200,000. However, he maintained a skeptical stance on the broader crypto market, critically assessing Tether as a "hive of scum and villainy" that he believes will eventually face a "multi-tens-of-billions-dollar loss event." He also discussed the rapid advancements and potential concentration risks of AI in trading, noting increasing dependency on AI coding tools among engineers.

McKenzie's address underscored the urgent need for central bankers and policymakers to deeply understand the technical underpinnings of financial systems. He advocated for proactive measures, including red team exercises and identifying critical personnel, to mitigate risks posed by both accidental errors and malicious actors within an increasingly complex and interconnected global financial ecosystem.