On Thursday evening, after a full day of chaos on the timeline, Elon Musk’s Twitter halted new enrollment into its $8-a-month Blue subscription offering. Everyone has the opportunity to slap another person. “verified” The widespread impersonation by celebrities, government officials and corporations of badges on their accounts had resulted in widespread use of the badge. It led to much confusion and even hoaxes, with accounts misrepresenting their identities. Eli Lilly, Tesla, Lockheed Martin This, along with others, had triggered an advertisement pullout and a general perception that the platform had fallen into chaos.

Musk realized the importance of this and tweeted that satirical accounts should now include “parody” In their bio and in their name. But if any of the fallout had come as a shock to Musk and his team, they can’t say they weren’t warned. 

Days before the November 9th launch, the company’s trust and safety team had prepared a seven-page list of recommendations intended to help Musk avoid the most obvious and damaging consequences of his plans for Blue. This document was obtained by PlatformerIt predicts with an eerie accuracy certain events that are to follow.

“Motivated scammers/bad actors could be willing to pay … to leverage increased amplification to achieve their ends where their upside exceeds the cost,” reads the document’s first recommendation, which the team labeled “P0” A concern that falls within the highest risk group is called “to denote”. 

“Impersonation of world leaders, advertisers, brand partners, election officials, and other high profile individuals” Another P0 risk was identified by the team. “Legacy verification provides a critical signal in enforcing impersonation rules, the loss of which is likely to lead to an increase in impersonation of high-profile accounts on Twitter.”  

“Removing privileges and exemptions from legacy verified accounts could cause confusion and loss of trust among high profile users.”

Musk had been considering a Blue annual subscription of $99 per year when the document was circulated internally on November 1st. He only reduced the price after an online exchange with Stephen King. It was a successful move. Intensifying There was a high risk of frauds when the impulse to buy $8 at $8 became a way to laugh at government officials and brands.

Additionally, the team noted that removing the verified badge from high-profile users and the related privileges if they paid would lead to them leaving Twitter permanently. “Removing privileges and exemptions from legacy verified accounts could cause confusion and loss of trust among high profile users,” They wrote. “We use the health-related protections … to manage against the risk of false-positive actions on high-profile users, under the assumption that the accounts have been heavily vetted. If that signal is deprecated, we run the risk of false positives or the loss of privileges such as higher rate limits resulting in escalation and user flight.” 

Twitter has not yet found a solution to these other risks, but the team discovered several others. First, Twitter does not have an automated system to remove verified badges. “Given that we will have a large amount of legacy verified users on the platform (400K Twitter customers), and that we anticipate we’ll need to debadge a large number of legacy verified accounts if they decide not to pay for Blue, this will require high operational lift without investment.”

(And this was). Before the company laid off 80 percent of its contractors, but we’ll get to that.)

The company’s trust and safety team did win support for some solutions, including retaining verification for some high-profile accounts using the “official badge.” 

The document mainly outlines a list of features that would make the product safer or easier to use. However, many of these have not yet been approved.

It was presented to Esther Crawford, a director of product management at the company who in recent weeks has risen to become one of Musk’s top lieutenants. Sources say that Musk and Alex Spiro were also briefed. Sources said that while Crawford appeared to be sympathetic to many concerns raised in the document, she declined to act on any suggestions that might delay Blue’s launch. (Crawford didn’t respond to a request of comment.

Despite all warnings, launch went as planned. Musk finally stopped the rollout after realizing the warnings of his safety and trust team.

The contractor wiping

A week after the initial round of layoffs, Saturday afternoon. had cut Twitter in half, Platformer It was reported by the first. a second massive wave of cuts had hit the company. This time, the cuts were aimed at Twitter’s contract workers. These losses were also more severe if you consider that they are a percentage. we reported, about 4,400 of 5,500 or so contractors — 80 percent of the team — had lost their jobs.

Content moderation, recruiting and ad sale, marketing, real estate were some of the functions that were affected. At the moment, it’s unclear how the loss of what may have been thousands of moderators This will impact the service. It seems that Twitter has significantly fewer people to monitor the site for any harmful material.

Unlike Twitter’s full-time employees, who at least got the courtesy of an email informing them that layoffs were coming a night before, contractors received nothing. Neither did their managers, who discovered one by one over the weekend that people they had been counting on to perform critical tasks had suddenly disappeared from the company’s systems. 

“One of my contractors just got deactivated without notice in the middle of making critical changes to our child safety workflows,” one manager noted in the company’s Slack channels. This is especially concerning because Twitter has struggled for years to adequately police child sexual abuse material on its platform, as we previously reported.

Over the course the day, similar messages were sent to Blind, which allows coworkers anonymously to discuss their workplaces. There are also external Slacks where employees can have more open discussions.

Many workers claimed that they learned their employment status from our tweets. They tried to log into Gmail and Slack and found they were no longer able to access them.

“Found out through your tweet and just happened to check,” One person involved in content moderation shared his experience with us. “What a lovely Saturday night.”

One person said the following: “If I didn’t see your tweet just now, I wouldn’t have even known.” 

Some employees shared with us that they were already preparing for the cuts since earlier this month’s layoffs. Many former contractors will be scrambling because of the sudden nature of the cuts. Platformer was first to reportVendors informed them by email that their medical benefits would be ending today, their last day on the job.

Meanwhile, the company’s Slack channels over the weekend told a story of already-low morale finding a new basement.

“I’m wondering when people will realize the value of Twitter was the people that worked here,” According to screenshots, one employee claimed that they had been taken by Platformer

Another: “In 2 weeks Twitter has gone from being the most welcoming and healthy workplace I’ve ever known to the most openly hostile and degrading I’ve ever known.”

Employees show great solidarity between one another. But not to the coterie of volunteer venture capitalists and on-loan engineers from Tesla and the Boring Company that have been carrying out Musk’s orders: those they refer to universally, including on Slack, as “the goons.” 

Code freeze

Twitter engineers were summoned to an emergency meeting Monday morning at 1:45 AM. Musk has just issued a new order: All production changes to Twitter systems must be stopped immediately. 

This was more than a code freeze. Engineers can only commit code, but they cannot deploy it. These are quite common and Twitter has had one since Musk’s takeover. Such freezes are generally intended to reduce the chances that a bug disrupts Twitter’s systems. 

This time, however, engineers were told they couldn’t even Write any code — “until further notice,” According to an internal email that was obtained by Platformer. If an exception is needed, “urgent change that is needed to resolve an issue with a production service, including any changes reflecting hard promised deadlines for clients,” According to the email, employees get “approval from VP level and Elon explicitly stating that the change needs to be made.” 

Slack confused even engineers who attended the late night meeting. “Is there a ticket I can reference?” An engineer was asked to help implement the freeze. “I don’t see any context.” “We don’t have much context as of now,” A colleague replied. “But this is coming from Elon’s team.”

In the meantime, we’re told engineers are writing code locally, on their laptops, and waiting for the freeze to end.

Meanwhile over the last day, Musk has made several public statements about the quality of Twitter’s code and service that have drawn rebukes from current engineers.

This is how another engineer explained it to me: “He doesn’t know what he’s talking about.”

“I’d like to apologize for Twitter being super slow in many countries. App is doing >1000 poorly batched RPCs just to render a home timeline!” Musk tweeted Sunday morning referring to remote procedure calls. Musk also complained about the sheer number of microservices Twitter has several features that can be used to keep the site running smoothly, and prevent it from going down in any way. 

Engineer Eric Frohnhoefer pushed back on Musk’s criticism, and offered a detailed thread Learn why Twitter loads slowly in certain locations. Musk fired Musk at the end of the day. Bloomberg reportedTogether with a second engineer who provided comment on the affair, he: “As the former tech lead for timelines infrastructure at Twitter, I can confidently say that this man has no idea wtf he’s talking about.”

This is how another engineer explained it to me:

The fact that he’s focusing on performance being worse in certain countries kind of shows that he doesn’t know what he’s talking about.

Everyone would have the same miserable experience if there was an issue with microservices communicating inside of a Data Center.

India is an exception to this rule. That’s because the payload gets delivered from further away (laws of physics come into effect) and that back-and-forth data transfer between the phone and the data center starts compounding.

Not to mention that places like India have a higher concentration of low power phones that tend to perform worse in general — as opposed to all of our overpowered iPhones and such.

Why is the code freeze? Although no one knows the answer, some speculate that Musk is paranoid about some engineers plotting to undermine the site.

Fallout

Eli Lilly suspended all its Twitter advertising campaigns Friday after the Blue disaster. Twitter could lose millions of dollars in revenue as a result. according to the Washington Post. (A “verified” A fake Eli Lilly account claimed that insulin would now be available for free. Twitter took six hours just to remove the tweet. 

This pharmaceutical giant is just one of many companies that has been using Twitter to pull ad dollars. Companies including Volkswagen and Pfizer have paused their campaigns, and large advertising firms like IPG’s Mediabrands and Omnicom Media Group are advising clients to do the same. 

The news has left Twitter’s ad teams — particularly those responsible for managing ad agency relationships — in a lurch, according to internal screenshots and conversations with current employees. 

“I know that many of your markets and clients are seeing large declines in Q4 and in particular L7D,” wrote Twitter’s global business lead in Slack. “Please add any commentary, questions, issues in this thread and I’ll endeavor to raise as many as possible TY!”

One employee replied that T-Mobile had asked to “pause the campaigns due to brand safety concerns.” (Three Days Later, former CEO at T-Mobile John Legere asked Musk Musk replied simply, “Let him run Twitter.” “no.”)

Another employee on Twitter said that General Motors had also requested to pause campaigns. “The initial reason they gave is elections, but it looks like an open-ended pause, because the team requested to meet next week to help them make a case to global on why they shouldn’t.” Later, the same employee was added: “Pause on [GM] til end of year confirmed and implemented. The reason now is brand safety.”

GroupM, the world’s largest media-buying agency, said to its clients that Twitter was high-risk. It has $60 billion annually in media spend. according to Digiday An email was sent to Platformer. Twitter’s agency partnerships lead explained the situation in Slack: “Given the recent senior departures in key operational areas (specifically Security, Trust & Safety, Compliance), GroupM have updated Twitter’s brand safety guidance to high risk. While they understand that our policies remain in place, they feel that Twitter’s ability to scale and manage infractions at speed is uncertain at this time.”

GroupM will reduce the risk grade if the following requirements are met. We are currently discussing these requirements with the leadership.

–Return to baseline levels of NSFW / toxic conversation on the platform

–Re-population of IT Security, Privacy, Trust & Safety senior staff

–Establishment of internal checks & balances

–Full transparency on future development plans of community guidelines / content moderation / anything affecting user security or brand safety

–Demonstrated commitment of effective content moderation, enforcing current Twitter Rules (e.g. account impersonation and timing of violative content removal, intolerance for hate speech, misinformation, and account theft

Advertisers seem to have a better understanding than Musk of Twitter’s needs, as demonstrated by the Blue rollout’s product recommendations. Massive cuts to content moderation, a paralyzing code lock, and hostility between the parties are all examples of the repercussions. “goons” The pre-Musk Tweeters have created a company which continues to face a greater crisis.

Musk’s announcement at mid-afternoon Monday afternoon prompted Monday’s Mid-afternoon he would begin disconnecting up to 80 percent of unspecified microservicesSome users claimed that SMS two-factor authentication had temporarily stopped working for them. Others reported experiencing partial site outages or difficulty downloading archives.

Some people are skilled in fixing all of those problems, but they no longer work for the company. Or they have been forbidden from shipping new code. Engineers were left wondering if new cracks would appear in the service, or how many and when.