Update: Service has been fully restored.
We are aware of reports that the front facing interface is currently inaccessible for some users. The team is currently working to resolve this as soon as possible.
Update: Service has been fully restored.
We are aware of reports that the front facing interface is currently inaccessible for some users. The team is currently working to resolve this as soon as possible.
Update: Hardware replacement was completed by 4:25 pm EST 2/24/2022.
We have scheduled down time for hardware migration at 2:30 PM EST on 2/24/2022. Estimated time is 60 minutes. Updates will be posted here.
We will have scheduled down time for hardware replacement at 3PM EST today. Estimated time is 30 minutes. Updates will be posted here.
**Update 1
We got started at 3:13 and hardware has been swapped, performing testing now.
**Update 2
Hardware has been replaced along with additional hardware that was out of tolerance.
Service should now be restoring, and we will continue to monitor.
You may see some down periods as we confirm configurations.
**Update 3
Replaced hardware seems to have stabilized, but we will be monitoring for any continuing issues.
**Update 4
Brief downtime now for some reboots to initialize.
**Update **
Recovery is complete and services are back online. Campaigns that were scheduled to go out during the downtime were paused and will need to be manually restarted or rescheduled.
SSL certificates for each custom url domain are being generated. If you encounter a SSL error, please wait a few minutes and check again.
**Update**
Contingency hardware is online, and backup restoration is in process. The weather event has prevented our datacenter provider from correcting the hardware issue in person within a known time frame.
Our contingency plan has continued, and the failover server is being restored from our early morning backup.
Any scheduled campaigns for today, should begin to be delivered as soon as restoration is complete.
ETA restoration of service is 1200 EST.
We are aware of an intermittent network issue effecting our data center, we are working to stabilize the service.
Due to winter weather events in the DFW area immediate recover has been hampered.
We are proceeding with secondary recovery offsite and will provide updates as they become available.
This is now resolved.
There is currently a hardware problem that is being investigated. You may experience intermittent outages while we work on fixing this.
Update: This error has been resolved.
We are aware of an error with the Campaign Content Editor as a result of a recent udpate. The team is working to resolve as quickly as possible.
*** The backlog in the delivery queue should be resolving as we ramp up delivery to the affected ISPs.
We are getting reports of major outages at:
Along with these major providers Level3, a backbone provider, is also experiencing outages.
While these outages are not directly affecting our servers, if you see mail stuck in queue or fail/bounce due to time outs it is due to being unable to connect to a mail provider due to these outages.
Update: Hardware failure, remediated with hot swap replacement. Queued sends should begin to resolve over the next hour. Services are being restored.
As of 1:06 PM EST we began experiencing a service outage. Our engineers are currently working to restore all services.
Update 2: Resolution has been confirmed.
Update: Servers are beginning to be accessible. We will do a final update when it is confirmed resolved.
As of 2:35 pm EST one of our COLO datacenters is experiencing a network outage.
Update: The router has been replaced and Campaigns are getting pushed out. You may see some delay on the Statistics as they are getting caught up.
As of 9:48 a.m. EST a router failed causing our servers to be cut off from the internet. We are in the process of replacing it. The expected resolution time is 11:30 a.m. EST.