MySQL on k8s 云原生环境部署
一、概述
MySQL
是一个关系型数据库管理系统,由瑞典MySQL AB 公司开发,属于 Oracle 旗下产品。MySQL 是最流行的关系型数据库管理系统之一,在 WEB 应用方面,MySQL是最好的 RDBMS (Relational Database Management System,关系数据库管理系统) 应用软件之一。这里主要讲mysql部署在k8s上,mysql部署在k8s上的优势主要有以下几点:
- 资源隔离
- 动态弹性扩缩容
- 环境一致性
- 运维方便
官方文档:docs.oracle.com/en-us/iaas/… MySQL 原理介绍也可以参考我这篇文章:MySQL原理介绍
二、开始部署(一主两从)
1)添加源
helm repo add bitnami https://charts.bitnami.com/bitnami
helm pull bitnami/mysql
tar -xf mysql-9.3.3.tgz
helm pull bitnami/mysql
tar -xf mysql-9.3.3.tgz
2)修改配置
- 修改
mysql/values.yaml
…
image:
registry: myharbor.com
repository: bigdata/mysql
tag: 8.0.30-debian-11-r15
…
architecture: replication
…
primary:
persistence:
enabled: true
size: 10Gi
storageClass: “mysql-local-storage”
# 目录需要提前在宿主机上创建
local:
– name: mysql-0
host: “local-168-182-110”
path: “/opt/bigdata/servers/mysql/data/data1”
service:
type: NodePort
nodePorts:
mysql: “30306”
secondary:
replicaCount: 2
persistence:
enabled: true
size: 10Gi
storageClass: “mysql-local-storage”
# 目录需要提前在宿主机上创建
local:
– name: mysql-1
host: “local-168-182-111”
path: “/opt/bigdata/servers/mysql/data/data1”
– name: mysql-2
host: “local-168-182-112”
path: “/opt/bigdata/servers/mysql/data/data1”
service:
type: NodePort
nodePorts:
mysql: “30307”
…
metrics:
## @param metrics.enabled Start a side-car prometheus exporter
##
enabled: true
image:
registry: myharbor.com
repository: bigdata/mysqld-exporter
tag: 0.14.0-debian-11-r33
image:
registry: myharbor.com
repository: bigdata/mysql
tag: 8.0.30-debian-11-r15
…
architecture: replication
…
primary:
persistence:
enabled: true
size: 10Gi
storageClass: “mysql-local-storage”
# 目录需要提前在宿主机上创建
local:
– name: mysql-0
host: “local-168-182-110”
path: “/opt/bigdata/servers/mysql/data/data1”
service:
type: NodePort
nodePorts:
mysql: “30306”
secondary:
replicaCount: 2
persistence:
enabled: true
size: 10Gi
storageClass: “mysql-local-storage”
# 目录需要提前在宿主机上创建
local:
– name: mysql-1
host: “local-168-182-111”
path: “/opt/bigdata/servers/mysql/data/data1”
– name: mysql-2
host: “local-168-182-112”
path: “/opt/bigdata/servers/mysql/data/data1”
service:
type: NodePort
nodePorts:
mysql: “30307”
…
metrics:
## @param metrics.enabled Start a side-car prometheus exporter
##
enabled: true
image:
registry: myharbor.com
repository: bigdata/mysqld-exporter
tag: 0.14.0-debian-11-r33
- 添加
mysql/templates/pv.yaml
{{- range .Values.primary.persistence.local }}
—
apiVersion: v1
kind: PersistentVolume
metadata:
name: {{ .name }}
labels:
name: {{ .name }}
spec:
storageClassName: {{ $.Values.primary.persistence.storageClass }}
capacity:
storage: {{ $.Values.primary.persistence.size }}
accessModes:
– ReadWriteOnce
local:
path: {{ .path }}
nodeAffinity:
required:
nodeSelectorTerms:
– matchExpressions:
– key: kubernetes.io/hostname
operator: In
values:
– {{ .host }}
—
{{- end }}
{{- range .Values.secondary.persistence.local }}
—
apiVersion: v1
kind: PersistentVolume
metadata:
name: {{ .name }}
labels:
name: {{ .name }}
spec:
storageClassName: {{ $.Values.secondary.persistence.storageClass }}
capacity:
storage: {{ $.Values.secondary.persistence.size }}
accessModes:
– ReadWriteOnce
local:
path: {{ .path }}
nodeAffinity:
required:
nodeSelectorTerms:
– matchExpressions:
– key: kubernetes.io/hostname
operator: In
values:
– {{ .host }}
—
{{- end }}
—
apiVersion: v1
kind: PersistentVolume
metadata:
name: {{ .name }}
labels:
name: {{ .name }}
spec:
storageClassName: {{ $.Values.primary.persistence.storageClass }}
capacity:
storage: {{ $.Values.primary.persistence.size }}
accessModes:
– ReadWriteOnce
local:
path: {{ .path }}
nodeAffinity:
required:
nodeSelectorTerms:
– matchExpressions:
– key: kubernetes.io/hostname
operator: In
values:
– {{ .host }}
—
{{- end }}
{{- range .Values.secondary.persistence.local }}
—
apiVersion: v1
kind: PersistentVolume
metadata:
name: {{ .name }}
labels:
name: {{ .name }}
spec:
storageClassName: {{ $.Values.secondary.persistence.storageClass }}
capacity:
storage: {{ $.Values.secondary.persistence.size }}
accessModes:
– ReadWriteOnce
local:
path: {{ .path }}
nodeAffinity:
required:
nodeSelectorTerms:
– matchExpressions:
– key: kubernetes.io/hostname
operator: In
values:
– {{ .host }}
—
{{- end }}
- 添加
mysql/templates/storage-class.yaml
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: {{ .Values.primary.persistence.storageClass }}
provisioner: kubernetes.io/no-provisioner
apiVersion: storage.k8s.io/v1
metadata:
name: {{ .Values.primary.persistence.storageClass }}
provisioner: kubernetes.io/no-provisioner
3)开始安装
# 创建持久化目录
mkdir -p /opt/bigdata/servers/mysql/data/data1
# 先准备好镜像
docker pull docker.io/bitnami/mysql:8.0.30-debian-11-r15
docker tag docker.io/bitnami/mysql:8.0.30-debian-11-r15 myharbor.com/bigdata/mysql:8.0.30-debian-11-r15
docker push myharbor.com/bigdata/mysql:8.0.30-debian-11-r15
# mysqld-exporter
docker pull docker.io/bitnami/mysqld-exporter:0.14.0-debian-11-r33
docker tag docker.io/bitnami/mysqld-exporter:0.14.0-debian-11-r33 myharbor.com/bigdata/mysqld-exporter:0.14.0-debian-11-r33
docker push myharbor.com/bigdata/mysqld-exporter:0.14.0-debian-11-r33
# 开始安装
helm install mysql ./mysql -n mysql –create-namespace
mkdir -p /opt/bigdata/servers/mysql/data/data1
# 先准备好镜像
docker pull docker.io/bitnami/mysql:8.0.30-debian-11-r15
docker tag docker.io/bitnami/mysql:8.0.30-debian-11-r15 myharbor.com/bigdata/mysql:8.0.30-debian-11-r15
docker push myharbor.com/bigdata/mysql:8.0.30-debian-11-r15
# mysqld-exporter
docker pull docker.io/bitnami/mysqld-exporter:0.14.0-debian-11-r33
docker tag docker.io/bitnami/mysqld-exporter:0.14.0-debian-11-r33 myharbor.com/bigdata/mysqld-exporter:0.14.0-debian-11-r33
docker push myharbor.com/bigdata/mysqld-exporter:0.14.0-debian-11-r33
# 开始安装
helm install mysql ./mysql -n mysql –create-namespace
NOTES
NAME: mysql
LAST DEPLOYED: Mon Sep 19 23:57:18 2022
NAMESPACE: mysql
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
CHART NAME: mysql
CHART VERSION: 9.3.3
APP VERSION: 8.0.30
** Please be patient while the chart is being deployed **
Tip:
Watch the deployment status using the command: kubectl get pods -w –namespace mysql
Services:
echo Primary: mysql-primary.mysql.svc.cluster.local:3306
echo Secondary: mysql-secondary.mysql.svc.cluster.local:3306
Execute the following to get the administrator credentials:
echo Username: root
MYSQL_ROOT_PASSWORD=$(kubectl get secret –namespace mysql mysql -o jsonpath=”{.data.mysql-root-password}” | base64 -d)
To connect to your database:
1. Run a pod that you can use as a client:
kubectl run mysql-client –rm –tty -i –restart=’Never’ –image myharbor.com/bigdata/mysql:8.0.30-debian-11-r15 –namespace mysql –env MYSQL_ROOT_PASSWORD=$MYSQL_ROOT_PASSWORD –command — bash
2. To connect to primary service (read/write):
mysql -h mysql-primary.mysql.svc.cluster.local -uroot -p”$MYSQL_ROOT_PASSWORD”
3. To connect to secondary service (read-only):
mysql -h mysql-secondary.mysql.svc.cluster.local -uroot -p”$MYSQL_ROOT_PASSWORD”
To access the MySQL Prometheus metrics from outside the cluster execute the following commands:
kubectl port-forward –namespace mysql svc/mysql-metrics 9104:9104 &
curl http://127.0.0.1:9104/metrics
LAST DEPLOYED: Mon Sep 19 23:57:18 2022
NAMESPACE: mysql
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
CHART NAME: mysql
CHART VERSION: 9.3.3
APP VERSION: 8.0.30
** Please be patient while the chart is being deployed **
Tip:
Watch the deployment status using the command: kubectl get pods -w –namespace mysql
Services:
echo Primary: mysql-primary.mysql.svc.cluster.local:3306
echo Secondary: mysql-secondary.mysql.svc.cluster.local:3306
Execute the following to get the administrator credentials:
echo Username: root
MYSQL_ROOT_PASSWORD=$(kubectl get secret –namespace mysql mysql -o jsonpath=”{.data.mysql-root-password}” | base64 -d)
To connect to your database:
1. Run a pod that you can use as a client:
kubectl run mysql-client –rm –tty -i –restart=’Never’ –image myharbor.com/bigdata/mysql:8.0.30-debian-11-r15 –namespace mysql –env MYSQL_ROOT_PASSWORD=$MYSQL_ROOT_PASSWORD –command — bash
2. To connect to primary service (read/write):
mysql -h mysql-primary.mysql.svc.cluster.local -uroot -p”$MYSQL_ROOT_PASSWORD”
3. To connect to secondary service (read-only):
mysql -h mysql-secondary.mysql.svc.cluster.local -uroot -p”$MYSQL_ROOT_PASSWORD”
To access the MySQL Prometheus metrics from outside the cluster execute the following commands:
kubectl port-forward –namespace mysql svc/mysql-metrics 9104:9104 &
curl http://127.0.0.1:9104/metrics
查看pod状态
kubectl get pods,svc -n mysql -owide
4)测试验证
【温馨提示】从库(slave)是只读的。就是简单的读写测试,还有就是pod挂了,能否正常拉起等等。这个测试验证比较简单。这里就不一步步的演示了。
5)Prometheus监控
Prometheus:
可以通过命令查看采集数据
kubectl get –raw http://10.244.0.74:9104/metrics
kubectl get –raw http://10.244.1.125:9104/metrics
kubectl get –raw http://10.244.2.178:9104/metrics
kubectl get –raw http://10.244.1.125:9104/metrics
kubectl get –raw http://10.244.2.178:9104/metrics
Grafana 账号:admin
,密码通过下面命令获取
导入grafana模板,集群资源监控:7362
官方模块下载地址:grafana.com/grafana/das…
6)卸载
helm uninstall mysql -n mysql
kubectl delete pod -n mysql `kubectl get pod -n mysql |awk ‘NR>1{print $1}’` –force
kubectl patch ns mysql -p ‘{“metadata”:{“finalizers”:null}}’
kubectl delete ns mysql –force
kubectl delete pod -n mysql `kubectl get pod -n mysql |awk ‘NR>1{print $1}’` –force
kubectl patch ns mysql -p ‘{“metadata”:{“finalizers”:null}}’
kubectl delete ns mysql –force
这里只是实现了mysql 的主从,没有实现高可用,官方目前没有出mysql ok k8s高可用的实现方案,虽然网上也有高可用的实现方案,但是我们公司没真正去落地使用,所以不予评价网上的高可用的方案,有兴趣的小伙伴可以去试试,其实mysql在大数据领域一般只是作为元数据存储,主挂了,影响不是很大。目前也在研究mysql on k8s 的高可用实现方案,如果后期有mysql on k8s高可用真正落地到生产环境中使用,到时候再来分享一下。
MySQL on k8s 环境部署就先这里了,更多关于MySQL k8s 环境部署的资料请关注其它相关文章!