Malicious Life Podcast: What Happened at Uber?

Transcript

It’s every executive’s worst nightmare. 

“[Joe] And it was a period in which I didn’t know whether I would go to prison or not, and the government was arguing that I should go… I think they were initially arguing for 3 years.”

In 2016, Joe Sullivan was at the peak of his career. He had been eBay’s Senior Director of Trust and Safety, created Facebook’s bug bounty program as Chief Security Officer, and was appointed by President Obama to the Commission on Enhancing National Cybersecurity.

Then, within a few months, everything fell apart.

“[Joe] And so I researched prisons and evaluated and talked to my family about like, If I was at this prison would you be able to visit me. You know, like all the reality and and how do you stay safe in a prison environment and how do you keep your mind sharp and your body sharp and you know, like, everything about it. I just did my homework.”

How does someone go from being a highly successful CISO to contemplating life in prison? In Joe’s case, it all started with a phone call from one of Silicon Valley’s most notorious companies, known for its problematic culture and business ethics.

Early Career

Joe Sullivan was born in 1968 to a painter and sculptor father and a mystery writer mother. Maybe that’s why his teenage rebellion took an unusual direction.

“[Joe] I had it in my head from the time I was pretty young that I wanted to be a government lawyer. And when I think back, it’s kind of odd because I’d never met a lawyer. […] So I don’t know why but I had it in my head that I wanted to be a government lawyer and so that was my focus.”

After graduating from law school, Sullivan spent eight years at the Department of Justice. Back then, in the 1980s, personal computers were still rare in government offices. Sullivan, who used email regularly and knew how to type, quickly became known as the resident ‘computer whiz,’ which placed him on a unique career path.

“[Joe] And I convinced the Department of Justice to give me a direct internet connection to my office, which was unheard of in 1995 when I did that. And very shortly after that in 1997 I was asked to become something called the Computer and Telecommunication Crime Coordinator, which is a mouthful but it basically means you’re the dedicated high-tech prosecutor in that office and you get specialized training.”

When the high-tech industry began to grow in earnest in the late 90s and early 2000s, Sullivan left the public sector to join eBay, and then served as PayPal’s Associate General Counsel. In 2008, he joined Facebook, where his unique talents and experience earned him the prestigious role of CISO, responsible for the safety and privacy of hundreds of millions of people worldwide.

After seven years at Facebook, Sullivan was looking for a new challenge when he got an interesting phone call.

“[Joe] So in 2014 the CTO of Uber reached out to me because they were having…well they had a lot of negative headlines around security by the end of 2014. There had been more than one story about people inside the company abusing access to customer data. I think the most prominent example was when a reporter wrote a story about how she was going to visit the New York office of Uber and she showed up late and she apologized for being late, I think, the story went. And they said – oh we knew you’re going to be late because we were watching your trip. And that really freaked her out and she’s like – wait a minute, you’re the head of the office. You have visibility into my uber ride?”

Improper protection of user privacy was just one of Uber’s many problems. In New Delhi and London, women complained of sexual harassment and even rape by Uber drivers. In Denver, another driver was arrested for attempting to break into a passenger’s home after dropping her at the airport. These incidents, among others, prompted district attorneys in San Francisco and Los Angeles to sue the company for inadequate background checks for its drivers. 

And if that wasn’t enough, in May 2014, Uber suffered a serious breach. An unknown hacker or hackers used an unencrypted AWS access key mistakenly uploaded to GitHub to tap into a database containing information on some 50,000 of the company’s drivers. A year later, shortly after Sullivan joined Uber in April 2015, the Federal Trade Commission began investigating Uber’s data security practices.

And so Sullivan had a lot on his plate in his first few months at Uber. While the FTC’s investigation was ongoing, he expanded the company’s security team from just five employees to over 500 in less than two years, and he also established Uber’s first bug bounty program.

2016 Breach

During this hectic period, an anonymous email landed in Sullivan’s inbox.

“Hello Joe. I have found a major vulnerability in Uber. I was able to dump Uber’s database and many other things.”

By then, Uber’s bug bounty program was already operational, and the company regularly paid security researchers who disclosed vulnerabilities. So, Sullivan didn’t think much of it.

“[Joe] And so I punted it over to Rob […] whose job was to kind of interact with these people among other things.”

Rob is Rob Fletcher, Uber’s product security engineering manager.

“[Joe] He’d actually joined Uber security because he had done a responsible disclosure to Uber himself and then through that got to know the team.”

Rob took over the correspondence. The anonymous hacker, who called himself John Doughs, claimed that the database he downloaded was a massive one: records on some 57 million of Uber’s customers, and seven million of its drivers. Rob, who suspected this to be a hoax, asked for proof of the breach, which Doughs provided.

Rob treated the case as a run-of-the-mill vulnerability disclosure, and offered him $10,000, the maximum bounty Uber offered for such disclosures as part of its bug bounty program. But John Doughs wasn’t satisfied.

“Hi, Rob. […] we expect at least $100,000. I am sure you understand what this could’ve turned out to be if it was to get in the wrong hands. I mean, you guys had private keys, private data stored, backups of everything, config files, etc. This would’ve hurt the company a lot more than you think. Hopefully, this gives you an insight of what really could’ve went down if someone else had the intention of doing harm.”

If Rob noticed the somewhat darker undertones in Dough’s response, he didn’t comment about them. “Sounds good,” he wrote in reply, “I’ll start compiling a post mortem report and routing for approval.”

Since such an unusually high payment required authorization, Rob decided to involve other stakeholders at the company.

“[Joe] So I think the first meeting I joined legal and the communications team were already involved and onboard and embedded in the cross-functional team that was responding to the situation.”

Sullivan notified Travis Kalanick, Uber’s CEO, who approved the payment on condition that the hacker signed an agreement to destroy the data he obtained. Doughs agreed: he and his partner, going under the pseudonym “Scott Wilson”, signed a non-disclosure agreement in which they vowed to destroy any data in their possession. 

But then, a new problem surfaced: Uber wanted the payment to go through HackerOne, a bug bounty platform – but this turned out to be impossible since HackerOne required bounty recipients to disclose their true personal information. Ultimately, Uber paid the $100,000 to the hackers in BitCoin. 

But Sullivan and his team weren’t content with the situation: with the money paid and the perpetrators still anonymous, there was a risk that the hackers kept a copy of the stolen database and would try to sell it on the dark web. So Sullivan’s team used information gleaned from the hacker’s BitCoin address and other clues gathered during the email conversation to unearth the hackers’ true identities: they were Brandon Glover, a 22-year-old who was living with his family in a trailer park in Florida, and Vasile Mereacre, a 19-year-old Canadian immigrant from Moldova. Mereacre, who was later arrested together with his accomplice and testified in Sullivan’s trial, admitted that the two were “dumbstruck” when a few weeks later they received a new email from Rob Fletcher in which he addressed Brandon by his true name, and asked them to sign a fresh NDA with their true identities. “I happen to have one of my team members down in Florida right now and he will meet with you tomorrow to get the contract signed,” Fletcher added. 

“[Joe] And so my team sent a member of our team who was a former CIA interrogator, who worked on our team on investigations. […] He met with each of them, interviewed them and I think in each case generated a five-page single space psychological profile and analysis of whether they were telling the truth.”

 With the hackers’ true identities unmasked, Sullivan and his staff were confident that they had the leverage they needed to make sure the stolen data would not be leaked. 

“[Joe] We’ve ascertained that, you know, that our customer data has been secured and we celebrated. From the security team standpoint, it was an investigation really well done.”

Uber’s management, including the CEO, were pleased with the way Sullivan and his security team handled what could have been a major security breach, and praised them in the year-end performance reviews.

But then, less than a year later, on Thanksgiving of 2017:

“[Joe] I was on vacation with my family and I got a phone call basically telling me I needed to get on a Zoom right away and then I was told I was being fired.”

Troubles at Uber

In many ways, Uber was a reflection of its founder. 

Travis Kalanick started his business career early. As a teenager in Los Angeles, he sold knives door-to-door, and at 18 he founded his first company: “New Way Academy”, a SAT-prep tutoring service. He founded two more companies – the first went bankrupt, the second was sold to Akamai – before he hit the jackpot in 2009 with UberCab: an app-based ride hailing service. Ten months into the company’s existence, Kalanick became its CEO. 

Kalanick’s aggressive style of business-making was evident right from the start. In October 2010, the San Francisco Municipal Transportation Agency notified UberCab that it was in breach of the city’s regulations: it was operating much like a cab company – but did not have a taxi license, nor a taxi’s insurance. The SFMTA warned UberCab that if it continued operating, it was facing fines of up to 5000$ per ride. 

Kalanick was unphased: he considered the real reason for backlash to be the threat his company posed to the established taxi industry. 

“We’ve seen this before. New technology comes in and appears threatening to incumbent industries at first. At the end of the day, those industries see the benefit of that technology and ultimately find ways of using it in a productive manner, and embracing innovation.”

Kalanick decided to ignore the SFMTA cease and desist order and simply shortened the company’s name from UberCab to Uber to avoid accusations of falsely advertising as a taxi company.

In its early years, Uber expanded rapidly, raising over $2.5 billion from investors and hiring thousands of drivers in more than 250 cities worldwide. In many cities, Uber faced resistance similar to what it encountered in San Francisco. Regulators in Spain, Thailand, the Netherlands, and other countries aggressively tried to halt its services. In some places, Uber’s drivers faced violent reactions from local taxi drivers.

Kalanick’s response was equally aggressive. In 2017, the New York Times exposed a years-long worldwide program by Uber to deceive hostile authorities. The “Greyball” software identified officials trying to hail rides to build cases against the company, then either denied their requests or presented “ghost cars” that didn’t exist. If an official was accidentally picked up by an Uber driver, a representative would instruct the driver to end the ride. In other instances, drivers were told to ignore local regulations, with Uber promising to pay for any citations they received.

Sullivan says that he tried to avoid dealing with the inevitable fallout from such shenanigans. 

“[Joe] And so it was quite a mess from a regulatory standpoint, and I would get pulled into that stuff on occasion because you know, the people criticizing Uber from the outside would say we shouldn’t allow ridesharing because it’s too dangerous. And so that’s when I would get dragged in and have to kind of like say Okay, well let’s talk about what we’re doing on safety. But for the most part I tried to stay out of that of the world of regulatory and legal and focus on operational safety and security.”

Over time, Kalanick’s ruthlessness permeated Uber’s internal culture, turning the company into what many critics called a “toxic” workplace. The press reported untreated sexual harassment accusations, bullying, and chronic infighting. In one notorious incident, a senior executive suggested the company should hire private investigators to spy on a journalist who criticized Uber and dig up dirt on her personal life and family.

Eventually, investors and executives realized Kalanick’s toxic reputation was harming Uber more than helping it. A mutiny ensued, and Kalanick was forced to take a leave of absence and later step down as CEO. With Kalanick gone, the board seized the opportunity to clean up the company.

“[Joe] The Board had created a committee to dig deep into kind of everything, Travis and everything history. Let’s kind of like rip the band-aid off of everything and see if we can fix things. And during the summer of 2017 an outside law firm was digging into a lot of different things and one of the things they looked at was this 2016 incident and they interviewed me multiple times.”

After reviewing the case, the outside law firm came to the conclusion that Uber should have disclosed the breach to the FTC. In November of 2017, Dana Khosrowshahi, Uber’s new CEO, published the following statement in the company’s blog. 

“I recently learned that in late 2016 we became aware that two individuals outside the company had inappropriately accessed user data stored on a third-party cloud-based service that we use. […] You may be asking why we are just talking about this now, a year later. I had the same question, so I immediately asked for a thorough investigation of what happened and how we handled it. […]  Effective today, two of the individuals who led the response to this incident are no longer with the company.”

During Sullivan’s trial, when Khosrowshahi was asked why he fired Sullivan, he testified that he felt like he couldn’t trust his CISO anymore. Khosrowshahi says that Sullivan omitted several important facts about the incident in an email summary he sent him. 

“The numbers were big, and the fact that a download took place is a significant issue that I became aware of. The payment, which I believe was $100,000, that’s just a really big payment that doesn’t fit into a typical bug bounty program. Those were the significant factors that weren’t in this original email. […] I found that his email to me was either incomplete or misleading; I don’t know which was his intention, […] and that led me to conclude that I needed to bring in a different head of security. I need to trust my direct reports; I can’t launch an investigation every time someone tells me something.”

Sullivan says he shared all that information with the outside lawyers who questioned him. He suspects the real reason for his dismissal was entirely different.

“[Joe] The way my legal team saw it and the way they argued it was that the company wanted to portray the new CEO as the white Knight coming in and cleaning up, you know, the mess of the old administration and they needed to show that this was not the old Uber, that we don’t sweep things under the rug.”

Three years later, in 2020, the US Department of Justice announced criminal charges against Sullivan. “We allege Sullivan falsified documents to avoid the obligation to notify victims,” the official statement said, “and hid the severity of a serious data breach from the FTC, all to enrich his company.”

“[Ran] What were your feelings when you were told your were fired and then being accused of a cover-up?

[Joe] It was not a good day. It was not a good week. It was not a good month. Because I wanted to speak up and say no – but you know, everybody said – talk to lawyers first. And then I talked to lawyers and they said don’t say anything until we can review all the evidence and we understand it all. And so I just had to bite my tongue when you know every newspaper in the world had me on the front cover as orchestrating a cover-up.”

Analysis

Even without considering Sullivan’s impressive cybersecurity record, the case drew significant media attention as the first time a corporate executive stood trial over an external breach.

The first charge against Sullivan was Obstruction of Justice. The Department of Justice alleged he took deliberate steps to conceal the breach from the FTC, which was already investigating the 2014 breach when news of the new breach reached Sullivan.

There’s little doubt that Sullivan hid the breach from the FTC. The investigation showed he told a subordinate they “can’t let this get out,” and that, as far as anyone outside Uber was concerned, “this investigation does not exist.” The real question, however, is whether the incident should have been treated as a data breach in the first place.

The two men who stole Uber’s data were not “security researchers” but black hat hackers intending to blackmail the company. When asked by the Assistant U.S. Attorney – “Was it your intent to extort Uber?” Mereacre straightforwardly answered, “Yes.”

But Sullivan maintains that the hackers’ true intent wasn’t clear from their email communication, and everyone at Uber believed otherwise.

“[Joe] My team never viewed these two outsiders as blackmailing the company and trying to extort the company in the way that we think about, you know the never ending ransomware stories that we hear about today. My team understood that these were a couple of young people who didn’t have exposure to bug boundary programs.”

Reading the correspondence between Rob Fletcher and the hackers – the emails were published by the New York Times – one can certainly understand why Sullivan viewed the incident as a white hat vulnerability disclosure: during most of their correspondence, Doughs kept scolding Uber for its weak security. For example: 

 “Uber should have mandatory 2 step authentication on GitHub, ALL INTERNAL data was able to be downloaded and seen. Your security steps are very poorly done, the lack of negligence [sic] and care here is zero to none. Your employees are careless and don’t care about security.”

“[Joe] You know the team actually called this case, the internal name of the investigation was Preacher because they felt that whoever was on the other side was taking the tone of like lecturing us, like you guys shouldn’t have had this vulnerability you got to do better kind of stuff.”

However, It’s easy to see why the authorities were skeptical of Sullivan’s explanations. Given Uber’s notorious record of questionable “do what it takes” business ethics and toxic culture, it’s natural to assume Uber cared more about the impact news of the breach would have on its public image than about compliance with regulations. The fact that the hackers were ultimately paid ten times the usual amount Uber offers for vulnerability disclosures – and in Bitcoin, no less – suggests that Sullivan might not be telling the whole truth, to put it mildly. It seems more than likely that Uber was aware of the hackers’ true intentions – but tried to steer them towards its bug bounty program so as to make the breach seem like an ordinary vulnerability disclosure. 

But even if the Department of Justice suspected Sullivan was lying, it’s unclear why he alone was singled out as the sole executive responsible. As Sullivan noted—and the prosecution agreed—both Uber’s chief privacy lawyer and head of communications were closely involved, and it was CEO Travis Kalanick who ultimately approved the decision to not disclose the incident to the FTC and pay the hackers. 

“[Joe] At my sentencing hearing the judge turned to the prosecutors and said – I don’t understand why the CEO isn’t here. From where I sit, he’s at least as culpable if not more.”

The second charge against Sullivan was somewhat unusual: Misprision of a Felony. In plain English, Misprision is when someone knows of a felony but fails to report it to the appropriate authorities. This offense is unusual because it is an old and rarely used one. It was once part of the “Common Law”—the basis of modern legal systems in many countries—but is generally considered outdated and has been discarded in England, Australia, and various other legal systems.

The prosecution probably decided to invoke this unusual charge to send a message to the high-tech industry. As U.S. Attorney David Anderson was quoted in the New York Times: 

“Silicon Valley is not the Wild West. We expect good corporate citizenship. We expect prompt reporting of criminal conduct. We expect cooperation with our investigations. We will not tolerate corporate cover-ups. We will not tolerate illegal hush money payments.”

There are plenty of reasons why fighting such cover-ups is a good idea, as the Uber incident demonstrates. The two hackers who blackmailed Uber also targeted other companies: StubHub, SeatGeek, Lynda.com, and LinkedIn. By not informing authorities of the breach and providing the identifying information from its investigation, Uber put these other companies at risk.

However, as several legal commentators noted, the prosecution’s use of such a broad and sweeping offense could be problematic in the context of bug bounty programs. Sullivan described the issue in a BlackHat keynote talk he gave in 2023.

“If somebody comes into my front yard even when I have a sign that says “no trespassing”, […]  I can actually say [to the police] – oh no, that’s my cousin, it’s okay for them to come in. And HackerOne and Bugcrowd and all these companies, that’s the view they’ve taken forever with regard to security researchers, because if you think about it the first time a researcher finds a vulnerability in any company, do they already have a contractual relationship with the company where the company gave them permission? most of the time, especially back in 2016, the answer was no: researchers would go poke around at a bunch of sites, and if they found something then they’d go join the program – and by government definition, every one of those people has committed a felony, and if we as the company entered into agreement with them afterwards the company will have committed a felony.”

—

Sullivan’s trial concluded in October 2022, where he was found guilty on two charges of obstructing an FTC investigation. He says the waiting period for sentencing was particularly difficult.

“[Joe] And it was a period in which I didn’t know whether I would go to prison or not […] I was very fortunate. I had a lot of people that I could turn to and talk to through the process. And I had a therapist and the therapist told me one of the ways you can get through, like I said, how do I deal with the anxiety of the unknown – and he said well, one of the ways you get through it is you write down what the worst case scenario actually is, and you write down what the best case scenario actually is, and then you look at what overlaps and then you take those out of the picture for the worst case scenario and you just accept those are the things that you’re going to have to live with no matter what. And then you look at the worst case scenario and you figure out how do I deal with each of these and […] it was that you know on the one hand reality is even scarier but in the other hand, knowing what you’re facing kind of gets rid of like these you know like abstract thoughts of fear because it’s more concrete.”

The case sparked a heated debate among security professionals. Many, if not most of Sullivan’s colleagues, felt that he was tossed under the bus by Uber’s new management, and that the charges against him could deter security professionals from taking on the role of CISO—a job already jokingly referred to as “Chief Scapegoat Officer” or “Chief Sacrificial Officer.” Kiersten Todt, Chief of Staff of the U.S. Cybersecurity and Infrastructure Security Agency, wrote to the judge saying top executives had warned her that the verdict would “make it impossible to recruit smart people into the roles of CISOs and CSOs if imprisonment is on the table—and will set the industry back.” 

Others, like Jamil Farshchi, Equifax’s CISO, believe that Sullivan made a big mistake by taking part in the cover up. Farshchi wrote on his Linkedin profile, 

“Tribalism is a powerful force. It blinds us from facts. It deletes our objectivity. […] I don’t know what really happened at Uber back in 2016.[…][but] What I do know is that nobody is disputing that a breach of 57M people occurred, Uber concealed it, and that Joe Sullivan – the CISO at the time – was involved in the concealment. […]  The key lesson here is one that almost every CISO has experienced first-hand: when faced with a lose-lose decision, do the right thing (or at least the lawful one).”

Many of Sullivan’s colleagues wrote to the judge, praising his past work and asking for leniency. One letter was signed by forty current and former CISOs. Ultimately, the judge was lenient, sentencing Sullivan to three years of probation, 200 hours of community service, and a $50,000 fine.

Epilogue 

After leaving Uber, Sullivan served as Cloudflare’s CISO for five years. In late 2022 he joined Ukraine Friends: a humanitarian organization providing laptops to Ukrainian schoolchildren. 

“[Joe] The whole case and everything about it, I wouldn’t wish on anyone because it was really hard and you know, it hung over my shoulders for many years and weighed on me heavily. But there were silver linings to it. Number one is I have a balance in my life now that I’m doing nonprofit work, because you know like if the case never came along I’d probably just still be an executive working at some big company. But I get to do nonprofit work that’s really meaningful and the thing I would tell anybody is when you’re going through a hard time, go volunteer and help somebody else who’s in a worse situation than you. All of a sudden your problems don’t seem as big and so you know helping these kids in a war zone I feel good and it puts my problems in perspective. And there are a lot of people in the world who have it worse than I do.”

Nowadays, alongside his humanitarian work, Joe Sullivan sees himself as a sort of unofficial spokesperson for CISOs across the industry. 

“[Joe] A lot of CISOs have asked me to to talk more, to write more to speak up and so you know, maybe not the best representative of them because I’m the person with the felony conviction. But I’ve also seen the worst of what they’ve had, you know, and could face and so I feel a sense of calling Now.”