kubectl-资源监控
metrics
metrics是什么
- metrics是一个监控系统资源使用的插件,可以监控node节点上的cpu、内存的使用率,或pod对资源的占用率,通过对资源占用的了解,可以更加合理的部署容器应用
- metrics从kubernetes 1.8开始,资源使用情况的监控可以通过Metrics Api 的形式获取,具体的组件为Metrics Server,用来替换之前的heapster,heapster从1.11开始逐渐被废弃。
安装metrics
metrics是扩展的apiserver,依赖于kube-aggregator,因此需要在apiserver中开启相关参数,开启聚合API
-
开启聚合API
移动
/etc/kubernetes/manifests/kube-apiserver.yaml文件,并备份修改[root@k8s-master k8s-ymls]# mv /etc/kubernetes/manifests/kube-apiserver.yaml ./ [root@k8s-master k8s-ymls]# kubectl get nodes The connection to the server 10.10.10.100:6443 was refused - did you specify the right host or port? [root@k8s-master k8s-ymls]# cp kube-apiserver.yaml kube-apiserver.back [root@k8s-master k8s-ymls]# vim kube-apiserver.yaml apiVersion: v1 ... - --tls-private-key-file=/etc/kubernetes/pki/apiserver.key #添加下行参数 - --enable-aggregator-routing=true image: hub.godone.top/pub/kube-apiserver:v1.34.2 ...还原
kube-apiserver.yaml文件,重启kubelet[root@k8s-master k8s-ymls]# mv kube-apiserver.yaml /etc/kubernetes/manifests/ [root@k8s-master k8s-ymls]# systemctl restart kubelet.service -
签发证书
修改所有节点的
/var/lib/kubelet/config.yaml文件,在最后最佳一行参数echo 'serverTLSBootstrap: true' >> /var/lib/kubelet/config.yaml # 重启kubelet服务 systemctl restart kubelet在master节点查看未签发的证书
[root@k8s-master k8s-ymls]# kubectl get certificatesigningrequests.certificates.k8s.io NAME AGE SIGNERNAME REQUESTOR REQUESTEDDURATION CONDITION csr-6qgnx 2m22s kubernetes.io/kubelet-serving system:node:k8s-node2 <none> Pending csr-slcq7 75s kubernetes.io/kubelet-serving system:node:k8s-master <none> Pending csr-tphqb 3m24s kubernetes.io/kubelet-serving system:node:k8s-node1 <none> Pending签发证书
[root@k8s-master k8s-ymls]# kubectl get certificatesigningrequests.certificates.k8s.io NAME AGE SIGNERNAME REQUESTOR REQUESTEDDURATION CONDITION csr-6qgnx 2m22s kubernetes.io/kubelet-serving system:node:k8s-node2 <none> Pending csr-slcq7 75s kubernetes.io/kubelet-serving system:node:k8s-master <none> Pending csr-tphqb 3m24s kubernetes.io/kubelet-serving system:node:k8s-node1 <none> Pending [root@k8s-master k8s-ymls]# kubectl certificate approve csr-6qgnx certificatesigningrequest.certificates.k8s.io/csr-6qgnx approved [root@k8s-master k8s-ymls]# kubectl certificate approve csr-tphqb certificatesigningrequest.certificates.k8s.io/csr-tphqb approved [root@k8s-master k8s-ymls]# kubectl get certificatesigningrequests.certificates.k8s.io NAME AGE SIGNERNAME REQUESTOR REQUESTEDDURATION CONDITION csr-6qgnx 6m13s kubernetes.io/kubelet-serving system:node:k8s-node2 <none> Approved,Issued csr-slcq7 75s kubernetes.io/kubelet-serving system:node:k8s-master <none> Approved,Issued csr-tphqb 7m15s kubernetes.io/kubelet-serving system:node:k8s-node1 <none> Approved,Issued -
部署资源
kubectl apply -f kube-metrics.yml--- apiVersion: v1 kind: ServiceAccount metadata: labels: k8s-app: metrics-server name: metrics-server namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: labels: k8s-app: metrics-server rbac.authorization.k8s.io/aggregate-to-admin: "true" rbac.authorization.k8s.io/aggregate-to-edit: "true" rbac.authorization.k8s.io/aggregate-to-view: "true" name: system:aggregated-metrics-reader rules: - apiGroups: - metrics.k8s.io resources: - pods - nodes verbs: - get - list - watch --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: labels: k8s-app: metrics-server name: system:metrics-server rules: - apiGroups: - "" resources: - nodes/metrics verbs: - get - apiGroups: - "" resources: - pods - nodes verbs: - get - list - watch --- apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: labels: k8s-app: metrics-server name: metrics-server-auth-reader namespace: kube-system roleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: extension-apiserver-authentication-reader subjects: - kind: ServiceAccount name: metrics-server namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: labels: k8s-app: metrics-server name: metrics-server:system:auth-delegator roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:auth-delegator subjects: - kind: ServiceAccount name: metrics-server namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: labels: k8s-app: metrics-server name: system:metrics-server roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:metrics-server subjects: - kind: ServiceAccount name: metrics-server namespace: kube-system --- apiVersion: v1 kind: Service metadata: labels: k8s-app: metrics-server name: metrics-server namespace: kube-system spec: ports: - appProtocol: https name: https port: 443 protocol: TCP targetPort: https selector: k8s-app: metrics-server --- apiVersion: apps/v1 kind: Deployment metadata: labels: k8s-app: metrics-server name: metrics-server namespace: kube-system spec: selector: matchLabels: k8s-app: metrics-server strategy: rollingUpdate: maxUnavailable: 0 template: metadata: labels: k8s-app: metrics-server spec: containers: - args: - --cert-dir=/tmp - --secure-port=10250 - --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname - --kubelet-use-node-status-port - --metric-resolution=15s image: hub.godone.top/pub/metrics-server:v0.8.0 imagePullPolicy: IfNotPresent livenessProbe: failureThreshold: 3 httpGet: path: /livez port: https scheme: HTTPS periodSeconds: 10 name: metrics-server ports: - containerPort: 10250 name: https protocol: TCP readinessProbe: failureThreshold: 3 httpGet: path: /readyz port: https scheme: HTTPS initialDelaySeconds: 20 periodSeconds: 10 resources: requests: cpu: 100m memory: 200Mi securityContext: allowPrivilegeEscalation: false capabilities: drop: - ALL readOnlyRootFilesystem: true runAsNonRoot: true runAsUser: 1000 seccompProfile: type: RuntimeDefault volumeMounts: - mountPath: /tmp name: tmp-dir nodeSelector: kubernetes.io/os: linux priorityClassName: system-cluster-critical serviceAccountName: metrics-server volumes: - emptyDir: {} name: tmp-dir --- apiVersion: apiregistration.k8s.io/v1 kind: APIService metadata: labels: k8s-app: metrics-server name: v1beta1.metrics.k8s.io spec: group: metrics.k8s.io groupPriorityMinimum: 100 insecureSkipTLSVerify: true service: name: metrics-server namespace: kube-system version: v1beta1 versionPriority: 100查询资源状态
kubectl -n kube-system get podskubectl get apiservices.apiregistration.k8s.io
查看资源监控
[root@k8s-master k8s-ymls]# kubectl top nodes
NAME CPU(cores) CPU(%) MEMORY(bytes) MEMORY(%)
k8s-master 47m 2% 890Mi 54%
k8s-node1 58m 2% 432Mi 26%
k8s-node2 45m 2% 478Mi 29%