The Mudge vs Twitter Security Breakdown

Sep 1, 2022 · 4 min read · Security Strategy Leadership  ·
Share on:

The Peiter Zatko (aka "Mudge") vs Twitter situation has gotten very public. Mudge released a very long (84 pages) document that details what he says is a longstanding pattern of indifference to security. Here are the technical details of the alleged security failures.

NOTE: mDAU-free analysis. You're welcome.

The Backstory

In late August of 2022, Twitter's former Security Lead, Peiter Zatko (aka "Mudge"), released a whistleblower report about rampant security failures within the company. He claims that they have existed throughout his tenure at Twitter (Nov 2020 - Jan 2022), and that leadership knew about them and did nothing. Twitter, for a lack of a better word, says he's full of it. (It's actually gotten pretty acrimonious, with both sides flat-out calling each other liars.)

It is certain that there have been rampant security failures at Twitter for years. It was the July 2020 attack, where super high-profile users' accounts were compromised, that caused then-CEO Jack Dorsey to hire Mudge. During that time, Mudge made repeated and increasingly insistent attempts to get security on leadership's radar. He was ultimately unsuccessful, leading to his firing in January by new Twitter CEO Parag Agrawal.

The Twitter Infrastructure (And The Damage Done)

The whisleblower report contains a number of allegations that will probably sound real familiar to a lot of security folks. The full report is 84 pages long, so I'm doing a lot of condensing here. (Also there is a lot of politicking, character assassination, and a looooooong discussion about bots that I'm basically ignoring.)

Basically the allegations boil down to the following:

  1. Live production access is widespread and user activity is not effectively logged
  2. There is no comprehensive RBAC policy or process
  3. Additionally, testing and production are not separated

This is definitely not good. Twitter has appx 10,000 users, and the document alleges that roughly half of them have access to production. And testing live? Definitely a problem. This is an infrastructure design problem writ large (as we will see, approximately half a million servers large), but it's also one that's all too common in IT.

  1. Software used in the datacenter was not properly licensed
  2. Hardware/Software systems in the datacenter were no longer supported
  3. There was no design documentation for the application startup process

This isn't good either. (Noticing a trend?) This is indicative of an environment that grows ad-hoc, is not documented, and has gotten so large that nobody would know how to turn the whole thing back on if there was a catastrophic disaster that took everything offline. The document alleges that because of point 2 above, that "over 50% of Twitter's 500,000 data center servers with non-compliant kernels or operating systems, and many unable to support encryption at rest." Yikes.

And that's just (some of) the problems Mudge highlights in the datacenter. There are issues on the user side too:

  1. Endpoint management software had been installed on 92% of laptops, BUT
    - Patches and security updates had been disabled on over 30% of those devices.
  2. Endpoint management software did not actually white or black list software installs
  3. There was no mobile device management software whatsoever
  4. Endpoint devices were not backed up at all, and there was no policy defining where or how sensitive company data should be stored

All this is indicative of the worst kind of security regimen: The kind that is installed purely to tick a yes/no box on an audit. There is no effort at all towards an effective end user management system here- in fact users are able to install whatever software they want on laptops that have direct access to production (with no oversight or auditing).

What Mudge describes is an environment where development is not standardized, it's done from devices that are ineffectively secured and audited, half the staff can access production, production is not reliably up to date, and if there's a catastrophic event, you might not even be able to turn the product back on. Mudge also states that little-to-nothing was done about it over the years he was employed at Twitter, because executives were not incentivized to care about Security. So, if all that's true, you can see how he might be .. alarmed.