Following the implementation of several key improvements, we have successfully stabilized the westeurope-1 environment and returned the Elasticsearch cluster to a healthy Green status. While disk usage has been significantly reduced to optimal levels, a backfill process is currently running to ensure all project data is fully synchronized and up to date. We have completed an extended monitoring period to confirm system stability, and all services are now fully operational.
Resolved
Following the implementation of several key improvements, we have successfully stabilized the westeurope-1 environment and returned the Elasticsearch cluster to a healthy Green status. While disk usage has been significantly reduced to optimal levels, a backfill process is currently running to ensure all project data is fully synchronized and up to date. We have completed an extended monitoring period to confirm system stability, and all services are now fully operational.
Monitoring
With some improvements made, we are continuing to monitor the incident.
Monitoring
Disk usage across the Elasticsearch cluster has decreased and cluster health is now green.
A backfill is running to ensure all projects are up to date, and the system is being monitored to confirm stability.
Identified
The underlying issue has been identified and a fix is now being worked on. The backlog of the impacted files is now being processed and the situation is gradually improving. We will update once the components are fully operational.
Investigating
We are currently observing a degraded performance issue with file uploads to Semantic Search. However, this is limited to new file uploads, file deletes and viewing recently uploaded files.
