As of 11:15 am central, all systems are back online.
Posted Jan 26, 2016 - 11:54 CST
Monitoring
Starting at 7:45am central, TrustedForm experienced a major outage resulting from a problem with the database cluster where we temporarily store unclaimed certificates and snapshots for 3 days. The root cause of the database problem could not be diagnosed, so we opted to mitigate damage by resetting the entire cluster. During that process certificates generated in the last 3 days were lost. For a portion of the certificates generated in the last 3 days that were already claimed, this outage has no impact. Some portion of those claims, however, may be missing page snapshots and Video Replay. Claims on certs that were generated between 7:45 and 11:30 am central today are likely irretrievably lost.
There is no higher priority for us than ensuring the integrity of our customer's data. We are working on salvaging whatever data we can and have engaged our database vendor to help us investigate the root cause of this issue. We will update this incident with additional information as it becomes available.
Posted Jan 26, 2016 - 11:53 CST
Investigating
We are currently investigating issues with our TrustedForm service. You may notice that calls to claim TrustedForm certs return an error and that the website may respond with a Heavy Load notice. We do not believe cert issuance to be affected. We will update here as soon as we have more information.