Degraded performance for sequences and time series on EUR-C1
Resolved
Lasted for 1d

The engineering team has continued their monitoring during the day, and have concluded that the situation has stabilized and that the service performance and quality is at the desired level. Cognite has closed this incident.

Wed, Jun 7, 2023, 11:32 AM
11 months ago
Affected components

No components marked as affected

Updates

Resolved

The engineering team has continued their monitoring during the day, and have concluded that the situation has stabilized and that the service performance and quality is at the desired level. Cognite has closed this incident.

Wed, Jun 7, 2023, 11:32 AM
4h earlier...

Monitoring

Cognite Engineering has been monitoring the services since yesterday. The latency and error rates seem to have been stable since yesterday UTC 17:30. The load on the system is at a level where we think there is little risk for yesterday's problems to repeat. We are continuing to monitor and analyze the root cause.

Wed, Jun 7, 2023, 07:18 AM
11h earlier...

Monitoring

The engineering team has spent additional hours working on a resolution for the incident. This incident is likely due to a change in traffic or traffic pattern. Some of the traffic has been turned off now, and the service looks healthier. We will continue to monitor, but no action will be taken until tomorrow if this remains stable. The incident is not considered resolved, but with the current traffic pattern and load, we believe that the service quality and performance is at an acceptable level.

Tue, Jun 6, 2023, 07:35 PM
7h earlier...

Identified

Cognite Engineering has deployed configuration changes that seem to reduce the problem significantly. The engineering team is still working on additional improvements. End users should see reduced error rates and fewer requests with slow response times.

Tue, Jun 6, 2023, 12:35 PM
44m earlier...

Investigating

The end-user experience is both elevated response times, and increased error rates. The incident is re-classified as "partial outage". Cognite is still working on the investigation.

Tue, Jun 6, 2023, 11:50 AM
25m earlier...

Investigating

Cognite Engineering is investigating an issue where the performance of sequences and time series is reduced. We see P90 latency of up to 1 minute for parts of the services.

Tue, Jun 6, 2023, 11:25 AM
Powered by