0%

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
cd /etc/yum.repos.d/
wget http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
rpm -ivh epel-release-latest-7.noarch.rpm

yum install jq

...
==============================================================================================================================================
Package 架构 版本 源 大小
==============================================================================================================================================
正在安装:
jq x86_64 1.5-1.el7 epel 153 k
为依赖而安装:
oniguruma x86_64 5.9.5-3.el7 epel 129 k
...

在Prometheus监控Targets页面,显示的都是监控标的列表。我们如何验证访问此/metrics的结果呢?

monitoring/kube-apiserver

这种https://172.16.64.233:6443/metrics URL,是需要带证书才能访问的:

1
2
3
4
5
6
7
8
9
curl -k --key /etc/kubernetes/pki/apiserver-kubelet-client.key \
--cert /etc/kubernetes/pki/apiserver-kubelet-client.crt \
https://172.16.64.233:6443/metrics
...
# HELP go_gc_duration_seconds A summary of the GC invocation durations.
# TYPE go_gc_duration_seconds summary
go_gc_duration_seconds{quantile="0"} 1.2317e-05
go_gc_duration_seconds{quantile="0.25"} 3.1153e-05
go_gc_duration_seconds{quantile="0.5"} 4.6201e-05

monitoring/kube-state-metrics

在页面上显示的Endpoint是https://10.244.169.167:9443/metrics

然后查看kube-state-metrics-serviceMonitor.yaml

发现有个配置bearerTokenFile: /var/run/secrets/kubernetes.io/serviceaccount/token

进入到pod prometheus-k8s-0, 通过/var/run/secrets/kubernetes.io/serviceaccount/token文件拿到token。

然后执行下面的命令就可以拿到指标数据了。

更多详情请看此文 Prometheus monitoring/kube-state-metrics监控指标

1
curl -k https://10.244.169.167:9443/metrics -H "Authorization: Bearer eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJtb25pdG9yaW5nIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6InByb21ldGhldXMtazhzLXRva2VuLXNtNmdkIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6InByb21ldGhldXMtazhzIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiMTk2OTkyOTItNDY2Yi00NWQ4LWJmYmYtYzkyZjIwNjczOWY3Iiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Om1vbml0b3Jpbmc6cHJvbWV0aGV1cy1rOHMifQ.iggZ4ZxmD0y04OQfDlo4P6zRgzn0ryVhcdhlgncpnBY5BJ39Xz0a2AA51ePa78R2njFDjPcecgDJRcqPv76X3o-C-G7EZvN_Ru8zSdB51YxqlLNoIW5hy6Jr27aw74lMslg1MYX_31kkRTqD9DxVn6lq6Uqf4Djebj_E-2maiwl863GCeNRfS1X6KM8idsVknLlpdVINbM8U_l1Yuw-auNzelAk1NQlBdbJqsm1CZKIg_YBsT-KbiyTsbjX2v0uL1D6-Q5Xs9NZMLEAa7dfwz_EOYMDnIGbv-eyhD-924H4_pGOIoQ0dCBP01cxFm7pLJPGouwLaEwPs5BRS0B6u-w"

monitoring/kubelet

https://172.16.64.233:10250/metrics

monitoring/node-exporter

https://172.16.64.233:9100/metrics

在Ubuntu环境里,运行程序时,报错:

1
Gtk-Message: 09:10:26.571: Failed to load module "canberra-gtk-module"

查看一下模块位置

1
2
locate libcanberra-gtk-module
/usr/lib/x86_64-linux-gnu/gtk-3.0/modules/libcanberra-gtk-module.so

发现已经安装了,却加载不了。

重新安装一下吧。

1
2
3
4
5
6
7
8
9
10
11
12
13
sudo apt-get install libcanberra-gtk-module
[sudo] password for simon:
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
libcanberra-gtk0
The following NEW packages will be installed:
libcanberra-gtk-module libcanberra-gtk0
0 upgraded, 2 newly installed, 0 to remove and 175 not upgraded.
Need to get 17.8 kB of archives.
After this operation, 92.2 kB of additional disk space will be used.
Do you want to continue? [Y/n] y

安装成功,再次启动程序,就不会报错了。