Major ISP outages across the U.S.

Major ISP outages across the U.S.

James No Comments

*** The backlog in the delivery queue should be resolving as we ramp up delivery to the affected ISPs.

We are getting reports of major outages at:

  • Spectrum (also includes Charter, RoadRunner, Time Warner) across most of the East Coast and parts of the midwest.
  • Verizon outage in Texas and parts of the New England.
  • Cablevision/Optimum across U.S.
  • Cox.net out in New England area

Along with these major providers Level3, a backbone provider, is also experiencing outages.

While these outages are not directly affecting our servers, if you see mail stuck in queue or fail/bounce due to time outs it is due to being unable to connect to a mail provider due to these outages.

Service Outage

James No Comments

Update: Hardware failure, remediated with hot swap replacement.  Queued sends should begin to resolve over the next hour. Services are being restored.

As of 1:06 PM EST we began experiencing a service outage. Our engineers are currently working to restore all services.

Network Outage

James No Comments

Update 2: Resolution has been confirmed.

Update: Servers are beginning to be accessible. We will do a final update when it is confirmed resolved.

As of 2:35 pm EST one of our COLO datacenters is experiencing a network outage.

Router Failure

James No Comments

Update: The router has been replaced and Campaigns are getting pushed out. You may see some delay on the Statistics as they are getting caught up.

As of 9:48 a.m. EST a router failed causing our servers to be cut off from the internet. We are in the process of replacing it. The expected resolution time is 11:30 a.m. EST.

AOL Issues

James No Comments

Update: The AOL issues are beginning to go away and should normalize over the next few days.

If you are experiencing a high number of bounces at AOL, know that our deliverability team is currently working with them to get this resolved.

Delayed Previews

James No Comments

Update: Previews are now working as intended. Any Preview sent before 10:10 pm EST will need to be re-sent.

Campaign Previews are currently delayed. Our team is looking into the cause of this and will push out any delayed Previews as soon as possible. The delay has not affected live Campaigns.

Comcast Bounces

James No Comments

As of this morning, IP reputation monitor ReturnPath is not returning Sender Scores for IP addresses across the internet causing Comcast’s automatic blocking systems to block all mail.

Facebook Discontinues Email Forwarding

James No Comments

On May 1 Facebook will discontinue forwarding @facebook.com email addresses to forwarding accounts. We will no longer attempt to deliver email addresses that contain @facebook.com to avoid deliverability or bounce issues for our customers.

The @facebook.com email address will still be on your Mailing Lists unless you Deactivate them.

Open and Click tracking delayed

James No Comments

Update: Tracking is getting pushed out to Statistics right now. It may take a few hours to be completely caught up.

Investigating – Early this morning our servers that push the Statistic data for Campaigns is delayed in pushing the Open and Click data to your Campaign Statistics.

High Rate of Bounces at Microsoft Accounts

James No Comments

Update – Deliverability has made progress with the Microsoft team. Bounces are coming down. The initial find is the Microsoft spam algorithm incorrectly flagged IP addresses as “spammy” due to the mail stream increases beyond the threshold’s limits. We will continue to mitigate with them until all mail is flowing freely again.

Investigating – We are suddenly seeing high bounce rates coming from Microsoft accounts (hotmail.com, msn.com, outlook.com, live.com) tonight. Our deliverability team is currently in communication with Microsoft to determine the cause of this.