K8s 采集 MySQL 监控指标

将 MySQL 的访问凭据保存到 Secret

为 mysql-exporter 创建 secret。

apiVersion: v1
kind: Secret
metadata:
  name: mysql-secret
  namespace: tools
type: Opaque
data:
  datasource: temp_user:123456@tcp(127.0.0.1:3306)/

使用 StatefulSet 创建 mysql-exporter

apiVersion: apps/v1
kind: StatefulSet
metadata:
  labels:
		app.kubernetes.io/name: mysql-exporter
    app.kubernetes.io/component: prometheus-exporter
  name: mysql-exporter
  namespace: tools
spec:
  podManagementPolicy: OrderedReady
  replicas: 0
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      k8s-app: mysql-exporter
  serviceName: ""
  template:
    metadata:
      labels:
    		app.kubernetes.io/name: mysql-exporter
        app.kubernetes.io/component: prometheus-exporter
    spec:
      containers:
      - env:
        - name: TZ
          value: Asia/Shanghai
        - name: DATA_SOURCE_NAME
          valueFrom:
            secretKeyRef:
              key: datasource
              name: mysql-secret
        image: ccr.ccs.tencentyun.com/rig-agent/mysqld-exporter:v0.12.1
        imagePullPolicy: IfNotPresent
        name: mysql-exporter
        ports:
        - containerPort: 9104
          name: metric-port
          protocol: TCP
        resources:
          limits:
            cpu: 250m
            memory: 512Mi
          requests:
            cpu: 250m
            memory: 512Mi
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      imagePullSecrets:
      - name: qcloudregistrykey
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30
  updateStrategy:
    rollingUpdate:
      partition: 0
    type: RollingUpdate

确认 mysql-exporter 的端点访问正常：http://127.0.0.1:9104/metrics

创建 Service 绑定 mysql-exporter

apiVersion: v1
kind: Service
metadata:
  annotations:
    prometheus.io/mysql-exporter-prometheus-enabled: "true"
    prometheus.io/port: "9104"
  name: mysql-exporter
  namespace: tools
spec:
  clusterIP: none
  ports:
  - name: tcp-9104-9104
    port: 9104
    protocol: TCP
    targetPort: 9104
  selector:
		app.kubernetes.io/name: mysql-exporter
    app.kubernetes.io/component: prometheus-exporter
  sessionAffinity: None
  type: ClusterIP

加入 Prometheus 监控配置

基于 K8s 服务发现机制解析 mysql-exporter 的 endpoints，从而获得所有 Pod 的实例信息。

global:
  scrape_interval: 15s
  evaluation_interval: 15s

scrape_configs:
  - job_name: "mysql-exporter"
    scrape_interval: 5s
    kubernetes_sd_configs:
    - role: endpoints
    relabel_configs:
  	# 匹配 mysql-exporter 绑定 Service 的标签 prometheus.io/mysql-exporter-prometheus-enabled
    - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_mysql_exporter_prometheus_enabled]
      action: keep
      regex: true
  	# 匹配 mysql-exporter 绑定 Service 的标签 prometheus.io/port
    - source_labels: [__address__, __meta_kubernetes_service_annotation_prometheus_io_port]
      action: replace
      target_label: __address__
      regex: ([^:]+)(?::\d+)?;(\d+)
      replacement: $1:$2
    - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scheme]
      action: replace
      target_label: __scheme__
      regex: (https?)
    - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_path]
      action: replace
      target_label: __metrics_path__
      regex: (.+)
    - action: labelmap
      regex: __meta_kubernetes_service_label_(.+)
    - source_labels: [__meta_kubernetes_namespace]
      action: replace
      target_label: namespace
    - source_labels: [__meta_kubernetes_service_name]
      action: replace
      target_label: service_name

确认 Prometheus 的目标 MySQL 节点状态为 UP

从 Grafana 导入 MySQL 监控面板

Grafana Labs 7362
效果如下