Resolved
All nodes are healthy again and request limits have been restored to normal levels.
Monitoring
The engineering team is still working on repairing the failing node pool. The time series performance is still significantly reduced by applying rate limiting and throttling of incoming traffic to the API.
Monitoring
We are experiencing reduced time series performance in this CDF cluster due to repeated k8s node failures. This lower rate limits and more throttling is applied. We are working to repair the node pool.