今天上k8s集群,kubectl命令操作卡顿,执行kubectl get node 命令返回有时很快,有时能卡好几十秒。
然后检查apiserver日志,发现了以下信息
v1beta1.metrics.k8s.io failed with: failing or missing response from https://10.106.18.58:443/apis/metrics.k8s.io/v1beta1: Get https://10.106.18.58:443/apis/metrics.k8s.io/v1beta1: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: Error trying to reach service: 'dial tcp 10.106.18.58:443: i/o timeout', Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]] I1118 11:49:26.791429 1 controller.go:127] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue. v1beta1.metrics.k8s.io failed with: Operation cannot be fulfilled on apiservices.apiregistration.k8s.io "v1beta1.metrics.k8s.io": the object has been modified; please apply your changes to the latest version and try again
然后尝试在apiserver中添加如下配置
–enable-aggregator-routing=true
然后再使用kubectl命令操作正常,没有再卡顿了。
—enable- aggregate -routing=true CLI标志添加到kube-apiserver,这样发送到Metrics Server的请求就可以在两个实例之间实现负载平衡。