03-November Server Issue
Incident Report for Harvest
Resolved
Overnight we performed exhaustive verification of all databases after yesterday's service outage. We have verified that all databases in all of our locations are 100% consistent, and we have now restored the database cluster back to full power. Sincere apologies for yesterday's issue, it was caused by failures of two different components, and we are implementing controls to prevent this from happening again.
Posted Nov 04, 2014 - 14:40 UTC
Monitoring
Earlier today we experienced failure of a core server, and then failure in the software designed to prevent service issues when servers fail. Unfortunately this resulted in an outage in all of our services while we manually restored things. Currently we have all services operating normally again, however, we are still working to bring our cluster back up to full power at this time and will close this incident once that is complete. We are very sorry for the interruption in service, please bear with us while this is resolved.
Posted Nov 03, 2014 - 21:40 UTC
Identified
We've identified the problem, stabilized the systems, and are working to bring everything back up to full power now!
Posted Nov 03, 2014 - 20:45 UTC
Investigating
We're investigating a problem with one of our servers. Thank you for your patience while we get this sorted out!
Posted Nov 03, 2014 - 20:10 UTC