Zabbix對Kafka?topic積壓數(shù)據(jù)監(jiān)控的問題(bug優(yōu)化)
簡述
《Zabbix對Kafka topic積壓數(shù)據(jù)監(jiān)控》一文的目的是通過Zabbix自動發(fā)現(xiàn)實現(xiàn)對多個消費者組的Topic及Partition的Lag進行監(jiān)控。因在實際監(jiān)控中發(fā)現(xiàn)有問題,為給感興趣的讀者不留坑,特通過此文對監(jiān)控進行優(yōu)化調整。
分區(qū)自動發(fā)現(xiàn)
# 未優(yōu)化前的計算方式: # 自動發(fā)現(xiàn)配置文件 vim consumer-groups.conf #按消費者組(Group)|Topic格式,寫入自動發(fā)現(xiàn)配置文件 test-group|test # 執(zhí)行腳本自動發(fā)現(xiàn)指定消費者和topic的分區(qū) bash consumer-groups.sh discovery { "data": [ { "{#GROUP}":"test-group", "{#TOPICP}":"test", "{#PARTITION}":"0" }, { "{#GROUP}":"test-group", "{#TOPICP}":"test", "{#PARTITION}":"1" }, { "{#GROUP}":"test-group", "{#TOPICP}":"test", "{#PARTITION}":"3" }, { "{#GROUP}":"test-group", "{#TOPICP}":"test", "{#PARTITION}":"2" } ] }
經過上線驗證,當自動發(fā)現(xiàn)配置文件只有一個test-group|test
是沒有問題的,但當我們按需求再接入test-group|test1
(即test-group消費者組的第二個Topic)時,自動發(fā)現(xiàn)的結果如下:
# 未優(yōu)化前的計算方式: vim consumer-groups.conf #按消費者組(Group)|Topic格式,寫入自動發(fā)現(xiàn)配置文件 test-group|test test-group|test1 # 執(zhí)行腳本自動發(fā)現(xiàn)指定消費者和topic的分區(qū) bash consumer-groups.sh discovery { "data": [ { "{#GROUP}":"test-group", "{#TOPICP}":"test", "{#PARTITION}":"0" }, { "{#GROUP}":"test-group", "{#TOPICP}":"test", "{#PARTITION}":"1" }, { "{#GROUP}":"test-group", "{#TOPICP}":"test", "{#PARTITION}":"3" }, { "{#GROUP}":"test-group", "{#TOPICP}":"test", "{#PARTITION}":"2" } { "{#GROUP}":"test-group", "{#TOPICP}":"test1", "{#PARTITION}":"0" }, { "{#GROUP}":"test-group", "{#TOPICP}":"test2", "{#PARTITION}":"1" }, { "{#GROUP}":"test-group", "{#TOPICP}":"test3", "{#PARTITION}":"2" } ] }
了解Zabbix自動發(fā)現(xiàn)格式的同學會發(fā)現(xiàn),每個Topic的Partition會出現(xiàn)','
,這種格式是不符合規(guī)范,這就是導致我們的監(jiān)控項會出現(xiàn)問題,因此我們需要進一步修改腳本。
經修改后,最終效果應該如下:
# 優(yōu)化后的計算方式: vim consumer-groups.conf #按消費者組(Group)|Topic格式,寫入自動發(fā)現(xiàn)配置文件 test-group|test test-group|test1 # 執(zhí)行腳本自動發(fā)現(xiàn)指定消費者和topic的分區(qū) bash consumer-groups.sh discovery { "data": [ { "{#GROUP}":"test-group", "{#TOPICP}":"test", "{#PARTITION}":"0" }, { "{#GROUP}":"test-group", "{#TOPICP}":"test", "{#PARTITION}":"1" }, { "{#GROUP}":"test-group", "{#TOPICP}":"test", "{#PARTITION}":"3" }, { "{#GROUP}":"test-group", "{#TOPICP}":"test", "{#PARTITION}":"2" }, { "{#GROUP}":"test-group", "{#TOPICP}":"test1", "{#PARTITION}":"0" }, { "{#GROUP}":"test-group", "{#TOPICP}":"test1", "{#PARTITION}":"1" }, { "{#GROUP}":"test-group", "{#TOPICP}":"test1", "{#PARTITION}":"2" } ] }
獲取監(jiān)控項“test-group/test/分區(qū)X”的Lag
經過自動發(fā)現(xiàn)后的數(shù)據(jù),我們可以進一步獲取不同分區(qū)的lag
# 優(yōu)化后的計算方式: # test-group test分區(qū)0 lag bash consumer-groups.sh lag test-group test 0 # test-group test分區(qū)1 lag bash consumer-groups.sh lag test-group test 1 # test-group test1分區(qū)0 lag bash consumer-groups.sh lag test-group test1 0
通過命令可以看到,我們的參數(shù)通過消費者組、Topic、Partition來獲取最終的lag值,如果不加消費者區(qū)分,那么無法區(qū)分不同消費者組和不同Topic相應的lag結果:
# 未優(yōu)化前的計算方式: # 獲取分區(qū)0 lag bash consumer-groups.sh lag 0 # 獲取分區(qū)1 lag bash consumer-groups.sh lag 1 # 獲取分區(qū)2 lag bash consumer-groups.sh lag 2 # 獲取分區(qū)3 lag bash consumer-groups.sh lag 3
最終優(yōu)化后腳本
# 自動發(fā)現(xiàn)配置文件 vim consumer-groups.conf #按消費者組(Group)|Topic格式,寫入自動發(fā)現(xiàn)配置文件 test-group|test test-group|test1 # 自動發(fā)現(xiàn)、lag計算腳本 vim consumer-groups.sh #!/bin/bash ##comment: 根據(jù)消費者組監(jiān)控topic lag,進行監(jiān)控告警 #配置文件說明 #消費者組|Topic #test-group|test #獲取topic 信息 cal_topic() { if [ $# -ne 2 ]; then echo "parameter num error, 讀取topic信息失敗" exit 1 else /usr/local/kafka/bin/./kafka-consumer-groups.sh --bootstrap-server 192.168.3.55:9092 --describe --group $1 |grep -w $2|grep -v none fi } #topic+分區(qū)自動發(fā)現(xiàn) topic_discovery() { printf "{\n" printf "\t\"data\": [\n" m=0 num=`cat /etc/zabbix/monitor_scripts/consumer-groups.conf|wc -l` for line in `cat /etc/zabbix/monitor_scripts/consumer-groups.conf` do m=`expr $m + 1` group=`echo ${line} | awk -F'|' '{print $1}'` topic=`echo ${line} | awk -F'|' '{print $2}'` cal_topic $group $topic > /tmp/consumer-group-tmp count=`cat /tmp/consumer-group-tmp|wc -l` n=0 while read line do n=`expr $n + 1` #判斷最后一行 if [ $n -eq $count ] && [ $m -eq $num ]; then topicp=`echo $line | awk '{print $1}'` partition=`echo $line | awk '{print $2}'` printf "\t\t{ \"{#GROUP}\":\"${group}\", \"{#TOPICP}\":\"${topicp}\", \"{#PARTITION}\":\"${partition}\" }\n" else topicp=`echo $line | awk '{print $1}'` partition=`echo $line | awk '{print $2}'` printf "\t\t{ \"{#GROUP}\":\"${group}\", \"{#TOPICP}\":\"${topicp}\", \"{#PARTITION}\":\"${partition}\" },\n" fi done < /tmp/consumer-group-tmp done printf "\t]\n" printf "}\n" } if [ $1 == "discovery" ]; then topic_discovery elif [ $1 == "lag" ];then cal_topic $2 $3 > /tmp/consumer-group cat /tmp/consumer-group |awk -v t=$3 -v p=$4 '{if($1==t && $2==p ){print $5}}' else echo "Usage: /data/scripts/consumer-group.sh discovery | lag" fi # 手動運行 ## 自動發(fā)現(xiàn) bash consumer-groups.sh discovery ## test-group test分區(qū)0 lag bash consumer-groups.sh lag test-group test 0
接入Zabbix
1.Zabbix配置文件
vim userparameter_kafka.conf UserParameter=topic_discovery,bash /data/scripts/consumer-groups.sh discovery UserParameter=topic_log[*],bash /data/scripts/consumer-groups.sh lag "$1" "$2" "$3"
2.Zabbix自動發(fā)現(xiàn)
3.監(jiān)控項配置
4.告警信息
告警主機:Kafka_192.168.3.55 主機IP:192.168.3.55 主機組:Kafka 告警時間:2022.03.21 00:23:10 告警等級:Average 告警信息:test-group/test/分區(qū)1:數(shù)據(jù)積壓100 告警項目:topic_lag[test-group,test,1] 問題詳情: test-group/test/1: 62
到此這篇關于Zabbix對Kafkatopic積壓數(shù)據(jù)監(jiān)控的文章就介紹到這了,更多相關Zabbix積壓數(shù)據(jù)監(jiān)控內容請搜索本站以前的文章或繼續(xù)瀏覽下面的相關文章希望大家以后多多支持本站!
版權聲明:本站文章來源標注為YINGSOO的內容版權均為本站所有,歡迎引用、轉載,請保持原文完整并注明來源及原文鏈接。禁止復制或仿造本網(wǎng)站,禁止在非www.sddonglingsh.com所屬的服務器上建立鏡像,否則將依法追究法律責任。本站部分內容來源于網(wǎng)友推薦、互聯(lián)網(wǎng)收集整理而來,僅供學習參考,不代表本站立場,如有內容涉嫌侵權,請聯(lián)系alex-e#qq.com處理。