当前位置: 首页 > news >正文

K8S节点出现Evicted状态“被驱逐”

 在Kubernetes集群中,Pod状态为“被驱逐(evicted)”表示Pod无法在当前节点上继续运行,已被集群从节点上移除。

问题分析:

节点磁盘空间不足 ,使用df -h查看磁盘使用情况

 可以看到根目录 / 已100%满,导致pod被驱逐

查看占用空间最大的目录,可以看到/var占用了20G

[root@k8s-master apply]# du -sh /* 2>/dev/null | sort -hr
20G     /var
4.2G    /usr
3.9G    /root
491M    /opt
295M    /home
193M    /boot
172M    /etc
150M    /run
124M    /kubectl-argo-rollouts-linux-amd64
28K     /monitor
16K     /lost+found
4.0K    /srv
4.0K    /mnt
4.0K    /media
4.0K    /id_rsa.pub
4.0K    /afs
0       /tmp
0       /sys
0       /sbin
0       /proc
0       /lib64
0       /lib
0       /dev
0       /bin

清理/var 目录

进入/var逐层分析,检测var下的子目录大小

[root@k8s-master apply]# cd /var
[root@k8s-master var]# du -sh * 2>/dev/null | sort -hr
19G     lib
676M    log
332M    cache
36K     spool
20K     tmp
12K     kerberos
12K     db
8.0K    empty
4.0K    yp
4.0K    preserve
4.0K    opt
4.0K    nis
4.0K    local-path-provisioner
4.0K    local
4.0K    games
4.0K    ftp
4.0K    crash
4.0K    adm
0       run
0       mail
0       lock

清理Docker数据

 删除所有未使用的容器、镜像、卷和缓存

[root@k8s-master var]# docker system prune -a -f[root@k8s-master var]# docker ps 
CONTAINER ID   IMAGE                                                           COMMAND                  CREATED        STATUS        PORTS                               NAMES
07f99d0b452f   hub.17usoft.com/gsairgroup/tool-portal:b684c29d                 "docker-entrypoint.s…"   17 hours ago   Up 17 hours   3000/tcp, 8080/tcp                  nervous_euler
1107187c9e61   apolloconfig/apollo-adminservice                                "/apollo-adminservic…"   25 hours ago   Up 25 hours   0.0.0.0:8090->8090/tcp              apollo-adminservice
f75ec993e063   apolloconfig/apollo-portal                                      "/apollo-portal/scri…"   25 hours ago   Up 25 hours   0.0.0.0:8070->8070/tcp              apollo-portal
a3cc9a54bc6d   apolloconfig/apollo-configservice                               "/apollo-configservi…"   25 hours ago   Up 25 hours   0.0.0.0:8080->8080/tcp              apollo-configservice
feca1915a4c7   mysql:8.0                                                       "docker-entrypoint.s…"   25 hours ago   Up 25 hours   0.0.0.0:3306->3306/tcp, 33060/tcp   apollo-mysql
07a829c9ae88   hub.17usoft.com/gsairgroup/timeline-service:master-fe94ee5      "/bin/sh"                34 hours ago   Up 34 hours   8080/tcp, 9090/tcp                  wonderful_lichterman
00483430d7d3   5e785d005ccc                                                    "/usr/bin/kube-contr…"   39 hours ago   Up 39 hours                                       k8s_calico-kube-controllers_calico-kube-controllers-697d846cf4-wplzw_kube-system_7b3c76be-3dc8-466c-b2b9-21b5f6bf3c50_0
9f883bb71d98   8d147537fb7d                                                    "/coredns -conf /etc…"   39 hours ago   Up 39 hours                                       k8s_coredns_coredns-7d89d9b6b8-585rc_kube-system_9ced5570-6a86-4397-8f1a-ae080e21c38f_0
fa3084652e29   8d147537fb7d                                                    "/coredns -conf /etc…"   39 hours ago   Up 39 hours                                       k8s_coredns_coredns-7d89d9b6b8-clvgn_kube-system_8a1a76b2-7ce1-4e8b-8345-1c874b8bc198_0
2863213c1950   registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.5   "/pause"                 39 hours ago   Up 39 hours                                       k8s_POD_coredns-7d89d9b6b8-clvgn_kube-system_8a1a76b2-7ce1-4e8b-8345-1c874b8bc198_0
0c0faa9863bb   registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.5   "/pause"                 39 hours ago   Up 39 hours                                       k8s_POD_coredns-7d89d9b6b8-585rc_kube-system_9ced5570-6a86-4397-8f1a-ae080e21c38f_0
19088b072072   registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.5   "/pause"                 39 hours ago   Up 39 hours                                       k8s_POD_calico-kube-controllers-697d846cf4-wplzw_kube-system_7b3c76be-3dc8-466c-b2b9-21b5f6bf3c50_026c4f07db245   08616d26b8e7                                                    "start_runit"            39 hours ago   Up 39 hours                                       k8s_calico-node_calico-node-gq2wj_kube-system_1a026e1d-9424-4884-93d3-b74a8bedb74b_0
3350890701b5   registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.5   "/pause"                 39 hours ago   Up 39 hours                                       k8s_POD_calico-node-gq2wj_kube-system_1a026e1d-9424-4884-93d3-b74a8bedb74b_0
1ee7c444baf0   77c8bfac1781                                                    "/usr/local/bin/kube…"   40 hours ago   Up 40 hours                                       k8s_kube-proxy_kube-proxy-dtqdn_kube-system_94f28854-13c6-4483-8f16-26651d0ad9f7_0
d7cf20062e40   registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.5   "/pause"                 40 hours ago   Up 40 hours                                       k8s_POD_kube-proxy-dtqdn_kube-system_94f28854-13c6-4483-8f16-26651d0ad9f7_0
ad961ccfa202   fce326961ae2                                                    "etcd --advertise-cl…"   40 hours ago   Up 40 hours                                       k8s_etcd_etcd-k8s-master_kube-system_5709e96ef17ce9e30e67cf45d0377549_2
95247db9a58e   registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.5   "/pause"                 40 hours ago   Up 40 hours                                       k8s_POD_etcd-k8s-master_kube-system_5709e96ef17ce9e30e67cf45d0377549_0
04d1716d923b   d4893b67e97f                                                    "kube-scheduler --au…"   40 hours ago   Up 40 hours                                       k8s_kube-scheduler_kube-scheduler-k8s-master_kube-system_56e5f657c0f5842192c9009d22729ac6_22
414292bdca76   c7ab721dfdae                                                    "kube-controller-man…"   40 hours ago   Up 40 hours                                       k8s_kube-controller-manager_kube-controller-manager-k8s-master_kube-system_7319ae284ccecb49c0164089eaf03446_22
953c9b59080a   2b5e9c96248f                                                    "kube-apiserver --ad…"   40 hours ago   Up 40 hours                                       k8s_kube-apiserver_kube-apiserver-k8s-master_kube-system_1156d3f2edec396a386c715bc506a05e_3
92df1a9b0841   registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.5   "/pause"                 40 hours ago   Up 40 hours                                       k8s_POD_kube-scheduler-k8s-master_kube-system_56e5f657c0f5842192c9009d22729ac6_0
91d3e9d0bc25   registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.5   "/pause"                 40 hours ago   Up 40 hours                                       k8s_POD_kube-apiserver-k8s-master_kube-system_1156d3f2edec396a386c715bc506a05e_0
9e56d1c18ce0   registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.5   "/pause"                 40 hours ago   Up 40 hours                                       k8s_POD_kube-controller-manager-k8s-master_kube-system_7319ae284ccecb49c0164089eaf03446_0
d051c56fa19d   2eaf94c71682                                                    "/usr/bin/docker-ent…"   2 weeks ago    Up 40 hours   0.0.0.0:9000->9000/tcp              minio
0cfd3f86b76e   moby/buildkit:buildx-stable-1                                   "buildkitd"              3 weeks ago    Up 40 hours                                       buildx_buildkit_default
[root@k8s-master var]# docker images
REPOSITORY                                                                    TAG               IMAGE ID       CREATED        SIZE
hub.17usoft.com/gsairgroup/tool-portal                                        b684c29d          c48fc7daaf38   17 hours ago   815MB
mysql                                                                         8.0               c6f7ec307380   3 days ago     772MB
moby/buildkit                                                                 buildx-stable-1   3c2d38015344   3 weeks ago    209MB
quay.io/minio/minio                                                           latest            2eaf94c71682   5 weeks ago    182MB
apolloconfig/apollo-portal                                                    latest            cc502c758c3b   2 months ago   291MB
apolloconfig/apollo-adminservice                                              latest            0fc2047c9a04   2 months ago   307MB
apolloconfig/apollo-configservice                                             latest            185f23e7f2d6   2 months ago   311MB
swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/calico/kube-controllers    v3.25.0           5e785d005ccc   2 years ago    71.6MB
swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/calico/node                v3.25.0           08616d26b8e7   2 years ago    245MB
registry.cn-hangzhou.aliyuncs.com/google_containers/kube-apiserver            v1.22.17          2b5e9c96248f   2 years ago    128MB
registry.cn-hangzhou.aliyuncs.com/google_containers/kube-controller-manager   v1.22.17          c7ab721dfdae   2 years ago    122MB
registry.cn-hangzhou.aliyuncs.com/google_containers/kube-scheduler            v1.22.17          d4893b67e97f   2 years ago    52.7MB
registry.cn-hangzhou.aliyuncs.com/google_containers/kube-proxy                v1.22.17          77c8bfac1781   2 years ago    104MB
registry.cn-hangzhou.aliyuncs.com/google_containers/etcd                      3.5.6-0           fce326961ae2   2 years ago    299MB
hub.17usoft.com/gsairgroup/timeline-service                                   master-fe94ee5    db53ed5a2383   2 years ago    48.8MB
registry.cn-hangzhou.aliyuncs.com/google_containers/coredns                   v1.8.4            8d147537fb7d   3 years ago    47.6MB
registry.cn-hangzhou.aliyuncs.com/google_containers/pause                     3.5               ed210e3e4a5b   4 years ago    683kB

可以看到 根目录的使用降下来了

容忍 DiskPressure 污点 

 在此添加容忍配置

应用新配置可以看到pod正常运行

 

相关文章:

  • 【Java面试笔记:基础】8.对比Vector、ArrayList、LinkedList有何区别?
  • 模型 替罪羊效应
  • 【Go】重难点知识汇总
  • vue使用笔记
  • 零基础上手Python数据分析 (21):图表选择困难症?常用可视化类型详解与应用场景指南
  • 基于STM32的室内环境监测系统
  • uView的u-modal不显示问题
  • 【数字图像处理】立体视觉基础(2)
  • 【OSG学习笔记】Day 6: Day 6: 几何体(Geometry)的创建与自定义
  • Graph Database Self-Managed Neo4j 知识图谱存储实践1:安装和基础知识学习
  • 如何高效定位Bug:测试工程师的问题诊断艺术
  • redis数据类型-地理空间GEO
  • 【Chrome插件开发】某视频网站的m4s视频/音频下载方案,及其Chrome插件实现-v250415
  • 美乐迪电玩客户端打包与资源替换实战教程
  • 2025软件测试常用面试问题及参考答案(附文档)
  • 如何解决消费者高频触发订单更新?
  • WeakSet:JavaScript 中容易被忽视的“弱集合”
  • spark和Hadoop的对比和联系
  • DCDC芯片,boost升压电路设计,MT3608 芯片深度解析:从架构到设计的全维度技术手册
  • 文档管理 Document Management
  • 美联储官员:若特朗普高额关税致失业率飙升,将支持降息
  • 苗旋已任民航局空管局局长、党委副书记
  • 吃菜和吃肉,哪个更“增肌”?
  • 董明珠的接班人还是董明珠
  • 贵州省交通运输厅原副厅长冯伟已任遵义市领导
  • 我们的免疫系统,是世界上最好的“医生”