docker容器监控自动恢复
关于实现对docker容器监控以及自动恢复,这里介绍两种实现方案。
方案1:
实现思路:
找到(根据正则表达式)所有待监控的docker容器,此处筛选逻辑根据docker运行状态找到已停止(Exit)类型的容器
代码清单1:
#!/bin/bash# 获取时间信息,格式为:"年-月-日T时:分:秒"
ctime1=`date +%Y-%m-%dT%k:%M:%S`echo "--------------------------------------------------- Script start at : $ctime1 "# 正则表达式匹配所有已biz-aaa/biz-bbb/biz-ccc开头,且以":v9"结尾的镜像名,结果中排除容器"container_to_be_excluded",获取位于索引为18位置上的docker名称
#set1=($(sudo docker ps -a | grep -E "biz-(aaa|bbb|ccc).*:v9" | grep -v ' container_to_be_excluded' | awk '{print $18}'))# 正则表达式匹配所有已biz-aaa/biz-bbb/biz-ccc开头,且以":v9"结尾的镜像名,且进一步过滤出已停止的容器(status = "Exit"),此时其docker名位于索引为16的位置处
set1=($(sudo docker ps -a | grep -E "biz-(aaa|bbb|ccc).*:v9" | grep "Exit" | awk '{print $16}'))# 获取数组长度
set1_size=${#set1[@]}# A).判断数组长度是否为0,注意数字的判断请使用:
## -eq : 相等
## -ne : 不同
# B).如果判断字符串,则为:
## = : 相等
## != : 不同
if [[ $set1_size -eq 0 ]]; thenecho ">>>> All business docker processes are health!"
else echo ">>>> Some business docker processes are not health, will be restarted!"sleep 60;# 迭代数组,可以使用${set1[@]},或${set1[?]}# for循环如果一行搞定,可以这样使用: for i in ${set1[@]};do ${CODE_LOGICAL}; donefor i in ${set1[@]};do echo ">>>>>>>> Docker instance - $i will be restarted right now"echo ">>>>>>>> Will execute command : docker restart $i"docker restart $i# Wait some time for resource to be releasedsleep 30# Re-check if docker process is upset2=($(sudo docker ps -a | grep $i | grep "Exit" | awk '{print $16}'))set2_size=${#set2[@]}if (( set2_size != 0 )); then docker restart $ifidonefictime2=`date +%Y-%m-%dT%k:%M:%S`
echo "--------------------------------------------------- Script end at : $ctime2 "
方案2:
实现思路:
找到(根据正则表达式)所有待监控的docker容器,"docker ps -a"匹配出所有需要监控的容器列表,"docker ps"匹配出当前存活的容器列表,差集即为有问题需要重启的容器
代码清单2:
#!/bin/bashset1=($(sudo docker ps -a | grep -E "biz-(aaa|bbb|ccc).*:v9" | grep -v ' container_to_be_excluded' | awk '{print $18}'))
set2=($(sudo docker ps | grep -E "biz-(aaa|bbb|ccc).*:v9" | grep -v ' container_to_be_excluded' | awk '{print $18}'))set1_size=${#set1[@]}
set2_size=${#set2[@]}# Sort sets
printf "%s\n" "${set1[@]}" | sort > set1_sorted.txt
printf "%s\n" "${set2[@]}" | sort > set2_sorted.txtdifference=($(comm -23 set1_sorted.txt set2_sorted.txt))if [[ $set1_size -eq $set2_size ]]; thenecho ">>>> All business docker processes are health!"
else echo ">>>> Some business docker processes are not health, will be restarted!"for i in ${difference[@]};do echo ">>>>>>>> Docker instance - $i will be restarted right now"docker restart $i#echo "-------------------------" $idonefirm -f set1_sorted.txt set2_sorted.txt
配置crontab调度,实现简单定时调度
crontab -e : 编辑crontab
查看已配置的crontab:crontab -l
*/10 * * * * /aaa/bbb/my_docker_ha.sh >> /aaa/bbb/logs/my_docker_ha.log 2>&1