Rackspace Hosted Exchange Failure Charge to Security Incident

Posted by

Rackspace hosted Exchange suffered a catastrophic failure starting December 2, 2022 and is still continuous as of 12:37 AM December 4th. Initially referred to as connectivity and login problems, the guidance was eventually updated to announce that they were dealing with a security event.

Rackspace Hosted Exchange Issues

The Rackspace system went down in the early morning hours of December 2, 2022. Initially there was no word from Rackspace about what the problem was, much less an ETA of when it would be dealt with.

Customers on Buy Twitter Verification reported that Rackspace was not reacting to support emails.

A Rackspace client privately messaged me over social networks on Friday to relate their experience:

“All hosted Exchange clients down over the past 16 hours.

Not sure the number of companies that is, but it’s significant.

They’re serving a 554 long hold-up bounce so people emailing in aren’t aware of the bounce for a number of hours.”

The official Rackspace status page offered a running upgrade of the failure but the initial posts had no details aside from there was an interruption and it was being examined.

The first authorities update was on December 2nd at 2:49 AM:

“We are examining a problem that is impacting our Hosted Exchange environments. More information will be posted as they become available.”

Thirteen minutes later Rackspace began calling it a “connectivity problem.”

“We are investigating reports of connection problems to our Exchange environments.

Users might experience a mistake upon accessing the Outlook Web App (Webmail) and syncing their email client(s).”

By 6:36 AM the Rackspace updates described the continuous issue as “connection and login issues” then later on that afternoon at 1:54 PM Rackspace announced they were still in the “examination stage” of the failure, still trying to determine what failed.

And they were still calling it “connection and login issues” in their Cloud Office environments at 4:51 PM that afternoon.

Rackspace Recommends Migrating to Microsoft 365

4 hours later on Rackspace referred to the situation as a “substantial failure”and began providing their consumers complimentary Microsoft Exchange Plan 1 licenses on Microsoft 365 as a workaround till they understood the problem and could bring the system back online.

The main guidance stated:

“We experienced a significant failure in our Hosted Exchange environment. We proactively closed down the environment to prevent any additional problems while we continue work to restore service. As we continue to resolve the source of the problem, we have an alternate option that will re-activate your ability to send out and get e-mails.

At no charge to you, we will be providing you access to Microsoft Exchange Strategy 1 licenses on Microsoft 365 until further notice.”

Rackspace Hosted Exchange Security Event

It was not till nearly 24 hours later on at 1:57 AM on December 3rd that Rackspace officially announced that their hosted Exchange service was struggling with a security occurrence.

The statement further exposed that the Rackspace service technicians had powered down and detached the Exchange environment.

Rackspace published:

“After further analysis, we have actually figured out that this is a security occurrence.

The known effect is separated to a part of our Hosted Exchange platform. We are taking needed actions to examine and safeguard our environments.”

Twelve hours later on that afternoon they upgraded the status page with more information that their security group and outside specialists were still dealing with fixing the outage.

Was Rackspace Service Impacted by a Vulnerability?

Rackspace has actually not launched details of the security event.

A security event usually involves a vulnerability and there are two severe vulnerabilities presently in the wile that were patched in November 2022.

These are the 2 most present vulnerabilities:

  • CVE-2022-41040
    Microsoft Exchange Server Server-Side Request Forgery (SSRF) Vulnerability
    A Server Side Request Forgery (SSRF) attack enables a hacker to check out and alter information on the server.
  • CVE-2022-41082
    Microsoft Exchange Server Remote Code Execution Vulnerability
    A Remote Code Execution Vulnerability is one in which an opponent is able to run harmful code on a server.

An advisory released in October 2022 explained the effect of the vulnerabilities:

“A validated remote attacker can perform SSRF attacks to escalate benefits and execute arbtirary PowerShell code on vulnerable Microsoft Exchange servers.

As the attack is targeted versus Microsoft Exchange Mailbox server, the attacker can possibly access to other resources through lateral motion into Exchange and Active Directory site environments.”

The Rackspace interruption updates have actually not indicated what the particular problem was, only that it was a security incident.

The most current status update since December 4th mentioned that the service is still down and consumers are encouraged to migrate to the Microsoft 365 service.

Rackspace posted the following on December 4, 2022 at 12:37 AM:

“We continue to make development in attending to the event. The availability of your service and security of your data is of high significance.

We have actually dedicated comprehensive internal resources and engaged first-rate external know-how in our efforts to reduce negative impacts to consumers.”

It’s possible that the above kept in mind vulnerabilities are related to the security event impacting the Rackspace Hosted Exchange service.

There has been no announcement of whether client details has actually been jeopardized. This event is still ongoing.

Featured image by Best SMM Panel/Orn Rin