Airlines, banks, hospitals and other risk-averse organizations around the world chose cybersecurity company CrowdStrike to protect their computer systems from hackers and data breaches.

But all it took was one faulty CrowdStrike software update to cause global disruptions Friday that grounded flights, knocked banks and media outlets offline, and disrupted hospitals, retailers and other services.

“This is a function of the very homogenous technology that goes into the backbone of all of our IT infrastructure,” said Gregory Falco, an assistant professor of engineering at Cornell University. “What really causes this mess is that we rely on very few companies, and everybody uses the same folks, so everyone goes down at the same time.”

The trouble with the update issued by CrowdStrike and affecting computers running Microsoft’s Windows operating system was not a hacking incident or cyberattack, according to CrowdStrike, which apologized and said a fix was on the way.

But it wasn’t an easy fix. It required “boots on the ground” to remediate, said Gartner analyst Eric Grenier.

“The fix is working, it’s just a very manual process and there’s no magic key to unlock it,” Grenier said. “I think that is probably what companies are struggling with the most here.”

While not everyone is a client of CrowdStrike and its platform known as Falcon, it is one of the leading cybersecurity providers, particularly in transportation, healthcare, banking and other sectors that have a lot at stake in keeping their computer systems working.

“They’re usually risk-averse organizations that don’t want something that’s crazy innovative, but that can work and also cover their butts when something goes wrong. That’s what CrowdStrike is,” Falco said. “And they’re looking around at their colleagues in other sectors and saying, ‘Oh, you know, this company also uses that, so I’m gonna need them, too.’”

Advertisement

Worrying about the fragility of a globally connected technology ecosystem is nothing new. It’s what drove fears in the 1990s of a technical glitch that could cause chaos at the turn of the millennium.

“This is basically what we were all worried about with Y2K, except it’s actually happened this time,” wrote Australian cybersecurity consultant Troy Hunt on the social platform X.

Across the world Friday, affected computers were showing the “blue screen of death” — a sign that something went wrong with Microsoft’s Windows operating system.

But what’s different now is “that these companies are even more entrenched,” Falco said. “We like to think that we have a lot of players available. But at the end of the day, the biggest companies use all the same stuff.”

Founded in 2011 and publicly traded since 2019, CrowdStrike describes itself in its annual report to financial regulators as having “reinvented cybersecurity for the cloud era and transformed the way cybersecurity is delivered and experienced by customers.” It emphasizes its use of artificial intelligence in helping to keep pace with adversaries. It reported having 29,000 subscribing customers at the start of the year.

The Austin, Texas-based firm is one of the more visible cybersecurity companies in the world and spends heavily on marketing, including Super Bowl ads. At cybersecurity conferences, it’s known for large booths displaying massive action-figure statues representing different state-sponsored hacking groups that CrowdStrike technology promises to defend against.

CrowdStrike CEO George Kurtz is among the most highly compensated in the world, recording more than $230 million in total compensation in the last three years. Kurtz is also a driver for a CrowdStrike-sponsored car racing team.

After his initial statement about the problem was criticized for lack of contrition, Kurtz apologized in a later social media post Friday and on NBC’s “Today Show.”

“We understand the gravity of the situation and are deeply sorry for the inconvenience and disruption,” he said on X.

Advertisement

Richard Stiennon, a cybersecurity industry analyst, said this was a historic mistake by CrowdStrike.

“This is easily the worst faux pas, technical faux pas or glitch of any security software provider ever,” said Stiennon, who has tracked the cybersecurity industry for 24 years.

While the problem is an easy technical fix, he said, it’s impact could be long-lasting for some organizations because of the hands-on work needed to fix each affected computer. “It’s really, really difficult to touch millions of machines. And people are on vacation right now, so, you know, the CEO will be coming back from his trip to the Bahamas in a couple of weeks and he won’t be able to use his computers.”

Stiennon said he did not think the outage revealed a bigger problem with the cybersecurity industry or CrowdStrike as a company.

“The markets are going to forgive them, the customers are going to forgive them, and this will blow over,” he said.

Forrester analyst Allie Mellen credited CrowdStrike for clearly telling customers what they need to do to fix the problem. But to restore trust, she said there will need to be a deeper look at what occurred and what changes can be made to prevent it from happening again.

“A lot of this is likely to come down to the testing and software development process and the work that they’ve put into testing these kinds of updates before deployment,” Mellen said. “But until we see the complete retrospective, we won’t know for sure what the failure was.”

___

Associated Press writer Alan Suderman in Richmond, Virginia, contributed to this report.

Advertisement

Transport providers, businesses and governments on Saturday are rushing to get all their systems back online after long disruptions following a widespread technology outage.

The biggest continuing effect has been on air travel. Carriers canceled thousands of flights on Friday and now have many of their planes and crews in the wrong place, while airports facing continued problems with checking in and security.

At the heart of the massive disruption is CrowdStrike, a cybersecurity firm that provides software to scores of companies worldwide. The company says the problem occurred when it deployed a faulty update to computers running Microsoft Windows, noting that the issue behind the outage was not a security incident or cyberattack.

Here’s the Latest:

Microsoft: 8.5 million devices on its Windows system were affected

Microsoft says 8.5 million devices running its Windows operating system were affected by a faulty cybersecurity update Friday that led to worldwide disruptions.

A Saturday blog post from Microsoft was the first estimate of the scope of the disruptions caused by cybersecurity firm CrowdStrike’s software update.

“We currently estimate that CrowdStrike’s update affected 8.5 million Windows devices, or less than one percent of all Windows machines,” said the blog post from Microsoft cybersecurity executive David Weston.

“While the percentage was small, the broad economic and societal impacts reflect the use of CrowdStrike by enterprises that run many critical services.”

Advertisement

Weston said such a significant disturbance is rare but “demonstrates the interconnected nature of our broad ecosystem.” Windows is the dominant operating system for personal computers around the world.

Austrian doctors’ group calls for better data protection for patients

In Austria, a leading doctors organization said the global IT outage exposed the vulnerability of health systems reliant on digital systems.

“Yesterday’s incidents underscore how important it is for hospitals to have analogue backups” to safeguard patient care, Harald Mayer, vice president of the Austrian Chamber of Doctors, said in a statement on the organization’s website.

The organization called on governments to impose high standards in patient data protection and security and on health providers to train staff and put systems in place to manage crises.

“Happily, where there were problems, these were kept small and short-lived and many areas of care were unaffected” in Austria, Mayer said.

Germany warns of scams after major IT outage

BERLIN — The German government’s IT security agency says numerous companies are still struggling with the consequences of a far-reaching technology outage.

“Many business processes and procedures have been disturbed by the breakdown of computer systems,” the BSI agency said on its website.

But the agency also said Saturday that many impacted areas have returned to normal.

It warned that cybercriminals were trying to take advantage of the situation through phishing, fake websites and other scams and that “unofficial” software code was in circulation.

Advertisement

The agency said it was not yet clear how faulty code ended up in the CrowdStrike software update blamed for triggering the outage.

European airports appear to be close to normal

LONDON — Europe’s busiest airport, Heathrow, said it is busy but operating normally on Saturday. The airport said in a statement that “all systems are back up and running and passengers are getting on with their journeys smoothly.“

Some 167 flights scheduled to depart from U.K. airports on Friday were canceled, while 171 flights due to land were axed.

Meanwhile, flights at Berlin Airport were departing on or close to schedule, German news agency dpa reported, citing an airport spokesman.

Nineteen flights took off in the early hours of Saturday after authorities exempted them from the usual ban on night flights.

On Friday, 150 of the 552 scheduled inbound and outbound flights at the airport were canceled over the IT outage, disrupting the plans of thousands of passengers at the start of the summer vacation season in the German capital.

German hospital slowly restoring its systems after widespread cancellations

BERLIN — The Schleswig-Holstein University Hospital in northern Germany, which on Friday canceled all elective surgery because of the global IT outage, said Saturday that it was gradually restoring its systems.

In a statement on its website, it forecast that operations at its two branches in Kiel and Luebeck would return to normal by Monday and that “elective surgery can take place as planned and our ambulances can return to service.”

Britain’s transport system still trying to get back on track

LONDON — Britain’s travel and transport industries are struggling to get back on schedule after the global security outage with airline passengers facing cancellations and delays on the first day of summer holidays for many school pupils.

Advertisement

Gatwick Airport said “a majority” of scheduled flights were expected to take off. Manchester Airport said passengers were being checked in manually and there could be last-minute cancellations.

The Port of Dover said it was seeing an influx of displaced air passengers, with hourlong waits to enter the port to catch ferries to France.

Meanwhile, Britain’s National Cyber Security Center warned people and businesses to be on the lookout for phishing attempts as “opportunistic malicious actors” try to take advantage of the outage.

The National Cyber Security Center’s former head, Ciaran Martin, said the worst of the crisis was over, “because the nature of the crisis is that it went very wrong very quickly. It was spotted quite quickly and essentially it was turned off.”

He told Sky News that some businesses would be able to get back to normal very quickly, but for sectors such as aviation it would take longer.

“If you’re in aviation, you’ve got people, planes and staffs all stranded in the wrong place… So we are looking at days. I’d be surprised if we’re looking at weeks.”

Germany airline expects most of its flights to run normally

BERLIN — Eurowings, a budget subsidiary of Lufthansa, said it expected to return to “largely scheduled” flight operations on Saturday.

On Friday, the global IT outage had forced the airline to cancel about 20% of its flights, mostly on domestic routes. Passengers were asked to take trains instead.

“Online check-in, check-in at the airport, boarding processes, booking and rebooking flights are all possible again,” the airline said Saturday on X. “However, due to the considerable extent of the global IT disruption there may still be isolated disruptions” for passengers, it said.

Advertisement

Delta Air Lines and its regional affiliates have canceled hundreds of flights

DALLAS — Delta Air Lines and its regional affiliates canceled more than a quarter of their schedule on the East Coast by midafternoon Friday, aviation data provider Cirium said.

More than 1,100 flights for Delta and its affiliates have been canceled.

United and United Express had canceled more than 500 flights, or 12% of their schedule, and American Airlines’ network had canceled 450 flights, 7.5% of its schedule.

Southwest and Alaska do not use the CrowdStrike software that led to the global internet outages and had canceled fewer than a half-dozen flights each.

Portland, Oregon, mayor declares an emergency over the outage

PORTLAND, Ore. — Mayor Ted Wheeler declared an emergency Friday after more than half of the city’s computer systems were affected by the global internet outage.

Wheeler said during a news conference that while emergency services calls weren’t interrupted, dispatchers were having to manually track 911 calls with pen and paper for a few hours. He said 266 of the city’s 487 computer systems were affected.

Border crossings into the US are delayed

SAN DIEGO — People seeking to enter the U.S. from both the north and the south found that the border crossings were delayed by the internet outage.

The San Ysidro Port of Entry was gridlocked Friday morning with pedestrians waiting three hours to cross, according to the San Diego Union-Tribune.

Even cars with people approved for a U.S. Customers and Border Protection “Trusted Traveler” program for low-risk passengers waited up to 90 minutes. The program, known as SENTRI, moves passengers more quickly through customs and passport control if they make an appointment for an interview and submit to a background check to travel through customs and passport control more quickly when they arrive in the U.S.

Advertisement

Meanwhile, at the U.S.-Canada border, Windsor Police reported long delays at the crossings at the Ambassador Bridge and the Detroit-Windsor tunnel.

WASHINGTON (AP) — A rapidly growing chorus of Democratic lawmakers called Friday for President Joe Biden to drop his reelection bid, even as the president insisted he’s ready to return to the campaign trail next week to counter what he called a “dark vision” laid out by Republican Donald Trump.

As more Democratic members of Congress urged him to drop out — bringing the total since his disastrous debate against Trump to nearly three dozen — Biden remained isolated at his beach house in Delaware after being diagnosed with COVID-19. The president, who has insisted he can beat Trump, was huddling with family and relying on a few longtime aides as he resisted efforts to shove him aside.

Late Friday, Ohio Sen. Sherrod Brown, a Democrat who is in a tough race for reelection, called for Biden to step aside.

Brown said in a statement that he agrees with “the many Ohioans” who have reached out to him. “I think the president should end his campaign,” he said.

And in a statement later Friday, Rep. Morgan McGarvey, D-Ky., also called on Biden to drop out while saying, “there is no joy in the recognition he should not be our nominee in November. But the stakes of this election are too high.”

Biden said Trump’s acceptance speech at the Republican National Convention showcased a “dark vision for the future.” The president, seeking to move the political conversation away from his fate and onto his rival’s agenda, said Friday he was planning to return to the campaign trail next week and insisted he has a path to victory over Trump, despite the worries of some of his party’s most eminent members.

“Together, as a party and as a country, we can and will defeat him at the ballot box,” Biden said. “The stakes are high, and the choice is clear. Together, we will win.”

Earlier in the day, his campaign chair, Jen O’Malley Dillion, acknowledged “slippage” in support for the president but insisted he’s “absolutely” remaining in the race and the campaign sees “multiple paths” to beating Trump.

Advertisement

“We have a lot of work to do to reassure the American people that, yes, he’s old, but he can win,” she told MSNBC’s “Morning Joe” show. She said voters concerned about Biden’s fitness to lead aren’t switching to vote for Trump.

Meanwhile, the Democratic National Committee’s rulemaking arm held a meeting Friday, pressing ahead with plans for a virtual roll call before Aug. 7 to nominate the presidential pick, ahead of the party’s convention later in the month in Chicago.

“President Biden deserves the respect to have important family conversations with members of the caucus and colleagues in the House and Senate and Democratic leadership and not be battling leaks and press statements,” Sen. Chris Coons of Delaware, Biden’s closest friend in Congress and his campaign co-chair, told The Associated Press.

It’s a pivotal few days for the president and his party: Trump has wrapped up an enthusiastic Republican National Convention in Milwaukee on Thursday. And Democrats, racing time, are considering the extraordinary possibility of Biden stepping aside for a new presidential nominee before their own convention.

Among the democrats expressing worries to allies about Biden’s chances were former President Barack Obama and Speaker Emerita Nancy Pelosi, who has privately told Biden the party could lose the ability to seize control of the House if he doesn’t step aside.

New Mexico Sen. Martin Heinrich called on Biden to exit the race, making him the third Senate Democrat to do so.

“By passing the torch, he would secure his legacy as one of our nation’s greatest leaders and allow us to unite behind a candidate who can best defeat Donald Trump and safeguard the future of our democracy,” said Heinrich, who’s up for reelection.

And Reps. Jared Huffman, Mark Veasey, Chuy Garcia and Mark Pocan, representing a wide swath of the caucus, together called on Biden to step aside.

“We must defeat Donald Trump to save our democracy,” they wrote.

Advertisement

Separately, Rep. Sean Casten of Illinois wrote in an op-ed that with “a heavy heart and much personal reflection” he, too, was calling on Biden to “pass the torch to a new generation.”

Campaign officials said Biden was even more committed to staying in the race. And senior West Wing aides have had no internal discussions or conversations with the president about dropping out.

On Friday, Biden picked up a key endorsement from the political arm of the Congressional Hispanic Caucus. CHC BOLD PAC said the Biden administration has shown “unwavering commitment” to Latinos and “the stakes couldn’t be higher” in this election.

But there is also time to reconsider. Biden has been told the campaign is having trouble raising money, and key Democrats see an opportunity as he is away from the campaign for a few days to encourage his exit. Among his Cabinet, some are resigned to the likelihood of him losing in November.

The reporting in this story is based in part on information from almost a dozen people who insisted on anonymity to discuss sensitive private deliberations. The Washington Post first reported on Obama’s involvement.

Biden, 81, tested positive for COVID-19 while traveling in Las Vegas earlier this week and experienced “mild symptoms” including “general malaise” from the infection, the White House said.

White House doctor Kevin O’Connor said Friday that the president still had a dry cough and hoarseness, but that his COVID symptoms had improved.

Biden noted his illness while making a joke about Trump on social media Friday night, posting: “I’m stuck at home with COVID, so I had the distinct misfortune of watching Donald Trump’s speech to the RNC. What the hell was he talking about?”

In Congress, Democratic lawmakers have begun having private conversations about lining up behind Harris as an alternative. One lawmaker said Biden’s own advisers are unable to reach a unanimous recommendation about what he should do. More in Congress are considering joining the others who have called for Biden to drop out. Some prefer an open process for choosing a new presidential nominee.

Advertisement

“It’s clear the issue won’t go away,” said Vermont Sen. Peter Welch, the other Senate Democrat who has publicly said Biden should exit the race. Welch said the current state of party angst — with lawmakers panicking and donors revolting — was “not sustainable.”

However, influential Democrats including Senate Majority Leader Chuck Schumer and House Democratic Leader Hakeem Jeffries are sending signals of concern.

“There is of course work to be done, and that in fact is the case because we are an evenly divided country,” Jeffries said in an interview on WNYC radio Friday.

But he also said, “The ticket that exists right now is the ticket that we can win on. … It’s his decision to make.”

To be sure, many want Biden to stay in the race. But among Democrats nationwide, nearly two-thirds say Biden should step aside and let his party nominate a different candidate, according to an AP-NORC Center for Public Affairs Research poll. That sharply undercuts Biden’s post-debate claim that “average Democrats” are still with him.

Amid the turmoil, a majority of Democrats think Vice President Kamala Harris would make a good president herself.

A poll from the AP-NORC Center for Public Affairs Research found that about 6 in 10 Democrats believe Harris would do a good job in the top slot. About 2 in 10 Democrats don’t believe she would, and another 2 in 10 say they don’t know enough to say.

___

Associated Press writers Joey Cappelletti in Lansing, Michigan, Ellen Knickmeyer in Aspen, Colorado, Steve Peoples in Milwaukee, and Josh Boak, Will Weissert, Mary Clare Jalonick, Seung Min Kim and Stephen Groves in Washington contributed to this report.

Advertisement