k8s环境实训系统真实部署
一、文档说明
1.1 文档
该文档用于信息学院数据中心k8s环境下部署实训系统
1.2 资源环境
1.3 密级与安全
数据库口令、JWT 密钥等敏感信息一律使用 Secret 保存,不写入 ConfigMap 或软件包。
本文档含明文口令,属机密资料,须限制知悉范围并定期轮换。
二、系统概览与架构
2.1 技术栈
2.2 部署组件清单
2.3 部署架构
浏览器 → 硬件防火墙 → KubeSphere 网关(Ingress)→ 前端 Service:80(Nginx 托管 dist);Nginx 将 /prod-api 反向代理到后端 Service:8080。后端连接集群内 MySQL、Redis,文件落在 PVC。
2.4 命名空间与资源规划
三、依赖的集群基线
部署前逐项确认:
账号与端口:
四、前置准备
4.1 软件包准备
后端软件包
app.jar(Spring Boot 可执行包)。前端构建产物
dist/与nginx.conf。数据库初始化脚本(建库建表 SQL)【待补充】。
项目配置项(数据库名、Redis 地址、上传目录、域名)【待补充】。
4.2 基础镜像准备
部署不使用自定义镜像,但以下公共基础镜像须预先离线导入集群(经 Harbor 或 ctr import):
ctr -n k8s.io images import base-images.tar # 或逐镜像经 Harbor 拉取
crictl images | grep -E 'temurin|nginx|busybox|mysql|redis'4.3 文件服务器准备
在 172.22.30.122 上创建软件包目录并上传:
mkdir -p /var/www/html/files
restorecon -Rv /var/www/html/files4.4 中间件规划
五、软件包分发
5.1 上传软件包
# 后端:直接上传 app.jar
scp app.jar root@172.22.30.122:/var/www/html/files/app.jar
# 前端:打包 dist 与 nginx.conf
tar -czf dist.tar.gz -C frontend dist
scp dist.tar.gz root@172.22.30.122:/var/www/html/files/dist.tar.gz
scp frontend/nginx.conf root@172.22.30.122:/var/www/html/files/nginx.conf5.2 验证分发
curl -I http://172.22.30.122/files/app.jar
curl -I http://172.22.30.122/files/dist.tar.gz预期:均返回 200 OK。
5.3 软件包更新
替换文件服务器上的 app.jar / dist.tar.gz,再滚动重启对应 Deployment,initContainer 会重新拉取最新软件包:
kubectl -n train-system rollout restart deployment/train-backend
kubectl -n train-system rollout restart deployment/train-frontend六、KubeSphere 部署
6.1 创建企业空间与项目
KubeSphere 界面:工作台 → 企业空间 → 创建「train-ws」→ 进入后「项目」→ 创建「train-system」。
kubectl 等价操作:
kubectl create namespace train-system6.2 部署 MySQL
Secret(保存 root 口令):
apiVersion: v1
kind: Secret
metadata:
name: train-mysql-secret
namespace: train-system
type: Opaque
stringData:
MYSQL_ROOT_PASSWORD: "<root口令>"StatefulSet + Service:
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: train-mysql
namespace: train-system
spec:
serviceName: train-mysql
replicas: 1
selector:
matchLabels: { app: train-mysql }
template:
metadata:
labels: { app: train-mysql }
spec:
containers:
- name: mysql
image: mysql:8.0
ports:
- containerPort: 3306
env:
- name: MYSQL_ROOT_PASSWORD
valueFrom:
secretKeyRef: { name: train-mysql-secret, key: MYSQL_ROOT_PASSWORD }
volumeMounts:
- name: data
mountPath: /var/lib/mysql
volumeClaimTemplates:
- metadata: { name: data }
spec:
accessModes: ["ReadWriteOnce"]
storageClassName: nfs-storage
resources:
requests: { storage: 10Gi }
---
apiVersion: v1
kind: Service
metadata:
name: train-mysql
namespace: train-system
spec:
selector: { app: train-mysql }
ports:
- port: 3306
targetPort: 3306初始化数据库:
kubectl -n train-system exec -it train-mysql-0 -- \
mysql -uroot -p -e "CREATE DATABASE train DEFAULT CHARACTER SET utf8mb4;"
# 再导入项目初始化 SQL【待补充】6.3 部署 Redis
apiVersion: apps/v1
kind: Deployment
metadata:
name: train-redis
namespace: train-system
spec:
replicas: 1
selector:
matchLabels: { app: train-redis }
template:
metadata:
labels: { app: train-redis }
spec:
containers:
- name: redis
image: redis:6.2-alpine
args: ["--requirepass", "$(REDIS_PASSWORD)"]
env:
- name: REDIS_PASSWORD
valueFrom:
secretKeyRef: { name: train-redis-secret, key: REDIS_PASSWORD }
ports:
- containerPort: 6379
---
apiVersion: v1
kind: Service
metadata:
name: train-redis
namespace: train-system
spec:
selector: { app: train-redis }
ports:
- port: 6379
targetPort: 63796.4 部署后端(基础镜像 + app.jar)
步骤 1:配置 ConfigMap(application.yml,占位符按项目实际调整):
apiVersion: v1
kind: ConfigMap
metadata:
name: train-backend-config
namespace: train-system
data:
application.yml: |
server:
port: 8080
spring:
datasource:
url: jdbc:mysql://train-mysql:3306/train?useUnicode=true&characterEncoding=utf8&useSSL=false
username: root
password: ${DB_PASSWORD}
redis:
host: train-redis
port: 6379
password: ${REDIS_PASSWORD}
ruoyi:
uploadPath: /data/upload步骤 2:Secret(口令与 JWT 密钥):
apiVersion: v1
kind: Secret
metadata:
name: train-backend-secret
namespace: train-system
type: Opaque
stringData:
DB_PASSWORD: "<与 train-mysql-secret 一致>"
REDIS_PASSWORD: "<与 train-redis-secret 一致>"
JWT_SECRET: "<随机长字符串>"步骤 3:Deployment + Service(initContainer 拉取 app.jar 到 emptyDir):
apiVersion: apps/v1
kind: Deployment
metadata:
name: train-backend
namespace: train-system
spec:
replicas: 2
selector:
matchLabels: { app: train-backend }
template:
metadata:
labels: { app: train-backend }
spec:
initContainers:
- name: fetch-app
image: busybox
command:
- sh
- -c
- wget -O /opt/app/app.jar http://172.22.30.122/files/app.jar && ls -l /opt/app
volumeMounts:
- name: app
mountPath: /opt/app
containers:
- name: backend
image: eclipse-temurin:17-jre
command: ["java", "-jar", "/opt/app/app.jar", "--spring.config.additional-location=/app/config/"]
ports:
- containerPort: 8080
env:
- name: DB_PASSWORD
valueFrom:
secretKeyRef: { name: train-backend-secret, key: DB_PASSWORD }
- name: REDIS_PASSWORD
valueFrom:
secretKeyRef: { name: train-backend-secret, key: REDIS_PASSWORD }
- name: JWT_SECRET
valueFrom:
secretKeyRef: { name: train-backend-secret, key: JWT_SECRET }
volumeMounts:
- name: app
mountPath: /opt/app
- name: config
mountPath: /app/config
- name: upload
mountPath: /data/upload
livenessProbe:
httpGet: { path: /actuator/health, port: 8080 }
initialDelaySeconds: 60
readinessProbe:
httpGet: { path: /actuator/health, port: 8080 }
initialDelaySeconds: 30
volumes:
- name: app
emptyDir: {}
- name: config
configMap: { name: train-backend-config }
- name: upload
persistentVolumeClaim: { claimName: train-upload-pvc }
---
apiVersion: v1
kind: Service
metadata:
name: train-backend
namespace: train-system
spec:
selector: { app: train-backend }
ports:
- port: 8080
targetPort: 8080
app.jar由 initContainer 在每次 Pod 启动时从文件服务器拉取到 emptyDir,更新软件包后滚动重启即可生效。健康检查路径以项目是否启用 actuator 为准。
6.5 部署前端(基础镜像 + dist)
步骤 1:nginx 配置 ConfigMap:
apiVersion: v1
kind: ConfigMap
metadata:
name: train-frontend-config
namespace: train-system
data:
default.conf: |
server {
listen 80;
server_name _;
root /usr/share/nginx/html;
index index.html;
location / {
try_files $uri $uri/ /index.html;
}
location /prod-api/ {
proxy_pass http://train-backend:8080/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}步骤 2:Deployment + Service(initContainer 拉取并解压 dist):
apiVersion: apps/v1
kind: Deployment
metadata:
name: train-frontend
namespace: train-system
spec:
replicas: 2
selector:
matchLabels: { app: train-frontend }
template:
metadata:
labels: { app: train-frontend }
spec:
initContainers:
- name: fetch-dist
image: busybox
command:
- sh
- -c
- wget -O /tmp/dist.tar.gz http://172.22.30.122/files/dist.tar.gz && tar -xzf /tmp/dist.tar.gz -C /opt/html
volumeMounts:
- name: html
mountPath: /opt/html
containers:
- name: frontend
image: nginx:1.24-alpine
ports:
- containerPort: 80
volumeMounts:
- name: html
mountPath: /usr/share/nginx/html
- name: nginx-conf
mountPath: /etc/nginx/conf.d/default.conf
subPath: default.conf
livenessProbe:
httpGet: { path: /, port: 80 }
volumes:
- name: html
emptyDir: {}
- name: nginx-conf
configMap: { name: train-frontend-config }
---
apiVersion: v1
kind: Service
metadata:
name: train-frontend
namespace: train-system
spec:
selector: { app: train-frontend }
ports:
- port: 80
targetPort: 806.6 服务暴露(应用路由)
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: train-ingress
namespace: train-system
annotations:
nginx.ingress.kubernetes.io/proxy-body-size: "100m"
spec:
ingressClassName: nginx
rules:
- host: <实训系统域名>
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: train-frontend
port: { number: 80 }
/prod-api已由前端 Nginx 反向代理到后端,Ingress 只需暴露前端;域名与网关选择 【待补充】。
6.7 KubeSphere 界面操作对照
命令行创建的 YAML 会自动显示在 KubeSphere 界面;同一资源建议固定一种管理入口,避免配置漂移。
七、部署顺序与命令汇总
部署按「密钥 → 中间件 → 后端 → 前端 → 路由」顺序执行:
NS=train-system
# 1) 密钥
kubectl -n $NS apply -f train-mysql-secret.yaml
kubectl -n $NS apply -f train-redis-secret.yaml
kubectl -n $NS apply -f train-backend-secret.yaml
# 2) 中间件
kubectl -n $NS apply -f mysql.yaml
kubectl -n $NS apply -f redis.yaml
kubectl -n $NS rollout status statefulset/train-mysql
kubectl -n $NS rollout status deployment/train-redis
# 3) 上传目录 PVC
kubectl -n $NS apply -f train-upload-pvc.yaml
# 4) 后端
kubectl -n $NS apply -f train-backend-config.yaml
kubectl -n $NS apply -f backend.yaml
kubectl -n $NS rollout status deployment/train-backend
# 5) 前端
kubectl -n $NS apply -f train-frontend-config.yaml
kubectl -n $NS apply -f frontend.yaml
kubectl -n $NS rollout status deployment/train-frontend
# 6) 路由
kubectl -n $NS apply -f train-ingress.yaml上传目录 PVC(train-upload-pvc.yaml,多副本共享须 ReadWriteMany):
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: train-upload-pvc
namespace: train-system
spec:
accessModes: ["ReadWriteMany"]
storageClassName: nfs-storage
resources:
requests: { storage: 10Gi }