zifuy 的个人笔记

Kubernetes部署Nacos集群参考

Nacos方便了配置文件的管理、更新、发布。

数据库准备

这里数据库独立于Kubernetes集群之外

说明：先配置好数据库mysql主从，并初始化数据库。数据库主从目的是在主数据库无法工作时Slave可继续提供支持

建库和授权

create database nacos_devtest character set utf8 collate utf8_bin;

set global validate_password_policy=0;

set global validate_password_length=4;

grant all privileges on *.* to nacos@"%" identified by 'nacos';

flush privileges;

初始化数据库并导入sql

wget https://github.com/alibaba/nacos/releases/download/1.2.0-beta.1/nacos-server-1.2.0-beta.1.tar.gz

解压后 使用 conf/nacos-mysql.sql 文件初始化数据库

mysql> source /root/nacos/conf/nacos-mysql.sql

部署基于NFS持久化存储的Nacos集群

NFS持久化存储可以参考本博客Kubernetes持久化存储部分，本篇文章不做详述。

这里以有状态应用的方式部署具有3个节点Nacos集群

cat nacos-statefulset.yaml

---
apiVersion: v1
kind: Service
metadata:
  name: nacos
  labels:
    app: nacos
spec:
  ports:
    - port: 8848
      name: server
      targetPort: 8848
  selector:
    app: nacos
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: nacos-cm
data:
  mysql.master.service.host: "10.13.1.6"
  mysql.slave.service.host: "10.13.1.6"
  mysql.master.db.name: "nacos_devtest"
  mysql.master.port: "3306"
  mysql.slave.port: "3306"
  mysql.master.user: "nacos"
  mysql.master.password: "nacos"
  nacos.isito.mcp.server.enable: "true" #开启istio.mcp协议支持
  management.endpoints.web.exposure.include: "*"  #开启prometheus监控
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: nacos
spec:
  serviceName: nacos
  replicas: 3
  template:
    metadata:
      labels:
        app: nacos
      annotations:
        pod.alpha.kubernetes.io/initialized: "true"
    spec:
      containers:
        - name: k8snacos
          imagePullPolicy: Always
          image: learn.harbor.com/app/nacos:1.1.4
          resources:
            requests:
              memory: "2Gi"
              cpu: "500m"
          ports:
            - containerPort: 8848
              name: client
          env:
            - name: NACOS_REPLICAS
              value: "3"
            - name: SERVICE_NAME
              value: "nacos"
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: metadata.namespace
            - name: MYSQL_MASTER_SERVICE_HOST
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: mysql.master.service.host
            - name: MYSQL_SLAVE_SERVICE_HOST
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: mysql.slave.service.host
            - name: MYSQL_MASTER_SERVICE_DB_NAME
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: mysql.master.db.name
            - name: MYSQL_MASTER_SERVICE_PORT
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: mysql.master.port
            - name: MYSQL_SLAVE_SERVICE_PORT
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: mysql.slave.port
            - name: MYSQL_MASTER_SERVICE_USER
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: mysql.master.user
            - name: MYSQL_MASTER_SERVICE_PASSWORD
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: mysql.master.password
            - name: NACOS_ISTIO_MCP_SERVER_ENABLE
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: nacos.isito.mcp.server.enable
            - name: MANAGEMENT_ENDPOINTS_WEB_EXPOSURE_INCLUDE
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: management.endpoints.web.exposure.include
            - name: NACOS_SERVER_PORT
              value: "8848"
            - name: PREFER_HOST_MODE
              value: "hostname"
            - name: NACOS_SERVERS
              value: "nacos-0.nacos.default.svc.cluster.local:8848 nacos-1.nacos.default.svc.cluster.local:8848 nacos-2.nacos.default.svc.cluster.local:8848"
          volumeMounts:
            - name: datadir
              mountPath: /home/nacos/data
            - name: logdir
              mountPath: /home/nacos/logs
  volumeClaimTemplates:
  - metadata:
      name: datadir
      annotations:
        volume.beta.kubernetes.io/storage-class: course-nfs-storage
    spec:
      accessModes: [ "ReadWriteMany" ]
      resources:
        requests:
          storage: 5Gi
  - metadata:
      name: logdir
      annotations:
        volume.beta.kubernetes.io/storage-class: course-nfs-storage
    spec:
      accessModes: [ "ReadWriteMany" ]
      resources:
        requests:
          storage: 5Gi
  selector:
    matchLabels:
      app: nacos

上面事先做了基于NFS的持久化存储。需要注意

这里NFS作用是即使应用故障服务无法启动，数据仍然保存在NFS后端的 /home/nacos/data目录中，除此之外Mysql中也有保留

非持久化存储部署

即数据只在Mysql中

---
apiVersion: v1
kind: Service
metadata:
  name: nacos
  labels:
    app: nacos
spec:
  ports:
    - port: 8848
      name: server
      targetPort: 8848
  selector:
    app: nacos
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: nacos-cm
data:
  mysql.master.service.host: "10.13.1.6"
  mysql.slave.service.host: "10.13.1.6"
  mysql.master.db.name: "nacos_devtest"
  mysql.master.port: "3306"
  mysql.slave.port: "3306"
  mysql.master.user: "nacos"
  mysql.master.password: "nacos"
  nacos.isito.mcp.server.enable: "true" #开启istio.mcp协议支持
  management.endpoints.web.exposure.include: "*"  #开启prometheus监控
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: nacos
spec:
  serviceName: nacos
  replicas: 3
  template:
    metadata:
      labels:
        app: nacos
      annotations:
        pod.alpha.kubernetes.io/initialized: "true"
    spec:
      containers:
        - name: k8snacos
          imagePullPolicy: Always
          image: learn.harbor.com/app/nacos:1.1.4
          resources:
            requests:
              memory: "2Gi"
              cpu: "500m"
          ports:
            - containerPort: 8848
              name: client
          env:
            - name: NACOS_REPLICAS
              value: "3"
            - name: SERVICE_NAME
              value: "nacos"
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: metadata.namespace
            - name: MYSQL_MASTER_SERVICE_HOST
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: mysql.master.service.host
            - name: MYSQL_SLAVE_SERVICE_HOST
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: mysql.slave.service.host
            - name: MYSQL_MASTER_SERVICE_DB_NAME
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: mysql.master.db.name
            - name: MYSQL_MASTER_SERVICE_PORT
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: mysql.master.port
            - name: MYSQL_SLAVE_SERVICE_PORT
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: mysql.slave.port
            - name: MYSQL_MASTER_SERVICE_USER
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: mysql.master.user
            - name: MYSQL_MASTER_SERVICE_PASSWORD
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: mysql.master.password
            - name: NACOS_ISTIO_MCP_SERVER_ENABLE
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: nacos.isito.mcp.server.enable
            - name: MANAGEMENT_ENDPOINTS_WEB_EXPOSURE_INCLUDE
              valueFrom:
                configMapKeyRef:
                  name: nacos-cm
                  key: management.endpoints.web.exposure.include
            - name: NACOS_SERVER_PORT
              value: "8848"
            - name: PREFER_HOST_MODE
              value: "hostname"
            - name: NACOS_SERVERS
              value: "nacos-0.nacos.default.svc.cluster.local:8848 nacos-1.nacos.default.svc.cluster.local:8848 nacos-2.nacos.default.svc.cluster.local:8848"
  selector:
    matchLabels:
      app: nacos

集群功能验证

服务注册
curl -X PUT 'http://cluster-ip:8848/nacos/v1/ns/instance?serviceName=nacos.naming.serviceName&ip=20.18.7.10&port=8080'
服务发现
curl -X GET 'http://cluster-ip:8848/nacos/v1/ns/instances?serviceName=nacos.naming.serviceName'
发布配置
curl -X POST "http://cluster-ip:8848/nacos/v1/cs/configs?dataId=nacos.cfg.dataId&group=test&content=helloWorld"
获取配置
curl -X GET "http://cluster-ip:8848/nacos/v1/cs/configs?dataId=nacos.cfg.dataId&group=test"

考虑问题

集群内的应用调用配置的接口,这里需要考虑尽可能的内网集群内调用，减少因环境和网络原因造成的配置调用的问题

待续---