Incident causing errors and high latencies for time series API in the USA-E1 cluster.
Resolved
Degraded performance
Lasted for 6h

The engineering team has monitored the time series service's performance over the last few hours, and have concluded that the system is healthy. The time series API showed elevated error rates and increased response times for a period of 53 minutes. No data was lost, but freshness was poor, and users had to resubmit requests that failed in the first attempt.

Wed, Nov 2, 2022, 08:17 PM
1 year ago
Affected components
Updates

Resolved

The engineering team has monitored the time series service's performance over the last few hours, and have concluded that the system is healthy. The time series API showed elevated error rates and increased response times for a period of 53 minutes. No data was lost, but freshness was poor, and users had to resubmit requests that failed in the first attempt.

Wed, Nov 2, 2022, 08:17 PM
5h earlier...

Monitoring

The engineering team has restored the service performance and time series is not under close monitoring for us to verify that the problems causing this incident have been remediated. Users should no longer experience elevated error rates or response times.

The time series API will be monitored for a while longer before we make a decision on closing the incident.

Wed, Nov 2, 2022, 02:18 PM
21m earlier...

Identified

Cognite engineering has identified an incident where the time series API is giving high latencies and an increased error rate due to high load. Investigation is ongoing.

Wed, Nov 2, 2022, 01:57 PM
Powered by