On May 19, 2025, between 3:40 PM UTC and 6:30 PM UTC, ButterCMS experienced an availability issue that affected its dashboard functionality and the ability to serve new content/pages. The issue did not impact the delivery of previously published content/pages, with only approximately 3% of the CDN experiencing problems.
Impact: - Dashboard Operation: Users encountered problems accessing and using the dashboard - unavailable during the incident. - New Content Serving: The system experienced difficulties in serving new content/pages. - Previously Cached Content: Remained available through the CDN with approximately 3% of the CDN experiencing issues.
Root Cause: The incident was caused by a problem with one of the cache databases, which became unavailable due to a network configuration issue. This was compounded by: - Heavy and resource-intensive operations, which increased traffic to the system. - Without properly functioning cache databases, the main database was directly burdened, leading to further performance degradation.
Resolution: - The cache database configuration was corrected, and the network problem was resolved. - A code fix was implemented to optimize application endpoints, improving response times and reducing the system load.
Next Steps: - Continuous monitoring of the cache database to ensure network stability. - Regular optimization of heavy operations to avoid system overload. - Exploration of further enhancements to the caching strategy to increase resilience.
Posted May 19, 2025 - 21:13 UTC
Monitoring
A fix has been implemented and we are monitoring the results.
Posted May 19, 2025 - 19:54 UTC
Identified
The issue has been identified and a fix is being implemented.
Posted May 19, 2025 - 19:19 UTC
Update
We are continuing to investigate this issue.
Posted May 19, 2025 - 18:25 UTC
Update
We are continuing to investigate this issue.
Posted May 19, 2025 - 18:13 UTC
Investigating
We are currently investigating an issue affecting ButterCMS. Users may experience a higher ratio of errors. Our team is working to identify the root cause, and we will provide an update as soon as more information is available.
We apologize for any inconvenience caused and appreciate your patience.