Links

MySQL

Setup

To enable the Prometheus metric for MySQL you can use MySQL exporter Here. You can either run MySQL exporter using an environment variable or using the docker.
Using an environment variable
export DATA_SOURCE_NAME='user:password@(hostname:3306)/' ./mysqld_exporter <flags>
Using docker
docker network create my-mysql-network docker pull prom/mysqld-exporter
docker run -d -p 9104:9104 --network my-mysql-network -e DATA_SOURCE_NAME="user:password@(hostname:3306)/" prom/mysqld-exporter
Please refer to the exporter link above to enable flags in the exporter.
Verify the following metrics in the Prometheus to confirm the exporter is attached to the MySQL server properly.
  • mysql_version_info
  • mysql_global_status_connections

Metrics

Request and Errors

Metric
Key Performance Indicator (KPI)
Request
mysql_global_status_connections
mysql_global_status_handlers_total
Request Rate
rate(mysql_global_status_connections[5m])
rate(sum without(handler) (mysql_global_status_handlers_total{handler=~"commit"})[5m])
Error
mysql_global_status_connection_errors_total
Error Ratio
rate(mysql_global_status_connection_errors_total[5m])/ rate(mysql_global_status_connections[5m])

Resource

Metric
Key Performance Indicator (KPI)
Connections
mysql_global_status_max_used_connections
Connection Usage
mysql_global_status_max_used_connections / mysql_global_variables_max_connections
Temp Tables Created
mysql_global_status_created_tmp_tables
Temp Tables Usage
avg_over_time(mysql_global_status_created_tmp_tables[5m)
Network Bytes Received
mysql_global_status_bytes_received
Network Bytes Transmitted
mysql_global_status_bytes_sent
Data transfer rate
avg_over_time(mysql_global_status_bytes_received[5m)
avg_over_time(mysql_global_status_bytes_received[5m)
Open Files
mysql_global_status_open_files
Open Files Usage
mysql_global_status_open_files / mysql_global_variables_open_files_limit

Alerts

KPI
Alert
Request Rate
RequestRateAnomaly
Error Ratio
ErrorRatioBreach and ErrorBuildup based on an availability SLO of 99.9
Connection Usage , Open Files Usage
Saturation with severity level of warning and critical when utilization exceeds 80% and 90% respectively
Temp Tables Usage
ResourceRateAnomaly
Network Bytes
ResourceRateAnomaly

Failure Alerts

MySQLDown
MySQL is not Running
mysql_up != 1
MySqlRestarted
MySQL server restarted
mysql_global_status_uptime < 60
MySqlSlowQueries
MySQL queries are running slow
rate(mysql_global_status_sloq_queries[1m]) * 60 > 0
MysqlSlaveSqlThreadNotRunning
MySQL slave is not running though it is enabled
mysql_slave_status_master_server_id > 0 and on (instance, asserts_env, asserts_site) mysql_slave_status_slave_sql_running == 0

Dashboards

Following MySQL dashboard shows the Summary level metrics
Further details metrics are shown in the below dashboard
InnoDB Specific metrics are shown in the below dashboard