CI/CD架构实战：Jenkins+GitLab企业级持续集成与持续部署

一、CI/CD概述

1.1 CI/CD核心概念

CI/CD（Continuous Integration/Continuous Deployment）实现代码到生产的自动化流程。

CI/CD核心流程:
  Continuous Integration (持续集成):
    - 代码提交触发构建
    - 自动运行测试
    - 快速反馈问题
    - 保证代码质量
  
  Continuous Deployment (持续部署):
    - 自动打包应用
    - 自动部署到环境
    - 自动化验证
    - 快速发布
  
  CI/CD工具链:
    - Jenkins: 构建和部署编排
    - GitLab: 代码仓库和触发
    - Docker: 容器化打包
    - Kubernetes: 容器编排

1.2 Jenkins+GitLab架构

Jenkins + GitLab架构:
  GitLab:
    - 代码仓库
    - GitHook触发
    - Merge Request管理
    - Issue跟踪
  
  Jenkins:
    - 接收Webhook
    - 执行Pipeline
    - 运行测试
    - 构建镜像
    - 部署应用
  
  流程:
    1. 开发者push代码到GitLab
    2. GitLab发送Webhook给Jenkins
    3. Jenkins触发Pipeline
    4. 构建、测试、打包、部署

二、Jenkins安装配置

2.1 Jenkins安装

# 使用Docker安装Jenkins
docker run -d \
  -p 8080:8080 \
  -p 50000:50000 \
  -v jenkins_home:/var/jenkins_home \
  --name jenkins \
  jenkins/jenkins:lts

# 或使用apt安装
wget -q -O - https://pkg.jenkins.io/debian-stable/jenkins.io.key | sudo apt-key add -
sudo sh -c 'echo deb http://pkg.jenkins.io/debian-stable binary/ > /etc/apt/sources.list.d/jenkins.list'
sudo apt-get update
sudo apt-get install jenkins

# 启动Jenkins
sudo systemctl start jenkins
sudo systemctl enable jenkins
sudo systemctl status jenkins

2.2 初始配置

# 获取初始密码
sudo cat /var/lib/jenkins/secrets/initialAdminPassword

# 访问Jenkins
# http://localhost:8080

# 安装推荐插件
# - Git Plugin
# - GitLab Plugin
# - Pipeline Plugin
# - Docker Pipeline Plugin
# - Kubernetes Plugin

2.3 GitLab集成配置

Jenkins配置GitLab凭据

# Jenkins -> Manage Jenkins -> Credentials
# 添加GitLab Personal Access Token

# GitLab生成Token:
# User Settings -> Access Tokens
# 权限: api, read_repository, write_repository

配置GitLab Connection

# Jenkins -> Manage Jenkins -> Configure System
# GitLab配置:
  Connection Name: GitLab
  GitLab Host URL: https://gitlab.example.com
  Credentials: GitLab Token
  Test Connection: Success

三、GitLab CI配置

3.1 .gitlab-ci.yml基础

完整的GitLab CI配置

# .gitlab-ci.yml

stages:
  - build
  - test
  - package
  - deploy

variables:
  DOCKER_REGISTRY: registry.example.com
  DOCKER_IMAGE: $CI_PROJECT_NAME
  DOCKER_TAG: $CI_COMMIT_REF_SLUG

# 构建阶段
build:
  stage: build
  image: node:16
  script:
    - echo "构建应用..."
    - npm install
    - npm run build
  artifacts:
    paths:
      - dist/
    expire_in: 1 week
  only:
    - main
    - develop

# 单元测试
unit_test:
  stage: test
  image: node:16
  script:
    - echo "运行单元测试..."
    - npm test
  coverage: '/覆盖率:\s+\d+\.\d+%/'
  allow_failure: true

# 集成测试
integration_test:
  stage: test
  image: node:16
  services:
    - postgres:13
    - redis:6
  variables:
    POSTGRES_DB: testdb
    REDIS_URL: redis://redis:6379
  script:
    - echo "运行集成测试..."
    - npm run test:integration
  only:
    - main

# 打包Docker镜像
docker_build:
  stage: package
  image: docker:20
  services:
    - docker:20-dind
  before_script:
    - echo $CI_REGISTRY_PASSWORD | docker login -u $CI_REGISTRY_USER --password-stdin $CI_REGISTRY
  script:
    - docker build -t $DOCKER_REGISTRY/$DOCKER_IMAGE:$DOCKER_TAG .
    - docker push $DOCKER_REGISTRY/$DOCKER_IMAGE:$DOCKER_TAG
  only:
    - main
    - tags

# 部署到开发环境
deploy_dev:
  stage: deploy
  image: alpine:latest
  script:
    - echo "部署到开发环境..."
    - apk add --no-cache curl
    - curl -X POST "$JENKINS_WEBHOOK_URL?job=deploy-dev&token=$DEPLOY_TOKEN"
  only:
    - develop
  environment:
    name: development
    url: https://dev.example.com

# 部署到生产环境
deploy_prod:
  stage: deploy
  image: alpine:latest
  script:
    - echo "部署到生产环境..."
    - apk add --no-cache curl
    - curl -X POST "$JENKINS_WEBHOOK_URL?job=deploy-prod&token=$DEPLOY_TOKEN"
  only:
    - main
  when: manual
  environment:
    name: production
    url: https://example.com

四、Jenkins Pipeline

4.1 声明式Pipeline

完整的Jenkinsfile

// Jenkinsfile - 声明式Pipeline

pipeline {
    agent any
    
    options {
        // Pipeline选项
        buildDiscarder(logRotator(numToKeepStr: '30'))
        disableConcurrentBuilds()
        timeout(time: 30, unit: 'MINUTES')
        timestamps()
    }
    
    environment {
        // 环境变量
        DOCKER_REGISTRY = 'registry.example.com'
        IMAGE_NAME = "${env.JOB_NAME}"
        IMAGE_TAG = "${env.BUILD_NUMBER}"
        DEPLOY_PATH = '/opt/app'
    }
    
    parameters {
        // 构建参数
        choice(name: 'ENV', choices: ['dev', 'test', 'prod'], description: '部署环境')
        string(name: 'VERSION', defaultValue: '', description: '版本号')
        booleanParam(name: 'SKIP_TESTS', defaultValue: false, description: '跳过测试')
    }
    
    stages {
        // 阶段1: 检出代码
        stage('Checkout') {
            steps {
                echo '检出代码...'
                checkout scm
                
                script {
                    env.GIT_COMMIT = sh(
                        script: 'git rev-parse --short HEAD',
                        returnStdout: true
                    ).trim()
                    
                    env.BRANCH_NAME = env.BRANCH_NAME ?: env.GIT_BRANCH?.replaceAll('origin/', '')
                }
            }
        }
        
        // 阶段2: 构建
        stage('Build') {
            steps {
                echo "构建 ${params.VERSION ?: env.GIT_COMMIT}"
                
                sh '''
                    echo "安装依赖..."
                    npm install
                    
                    echo "构建应用..."
                    npm run build
                '''
            }
        }
        
        // 阶段3: 测试
        stage('Test') {
            when {
                expression { !params.SKIP_TESTS }
            }
            parallel {
                stage('Unit Tests') {
                    steps {
                        script {
                            try {
                                sh 'npm run test:unit'
                                publishTestResults testResultsPattern: 'test-results/**/*.xml'
                            } catch (Exception e) {
                                echo "单元测试失败: ${e}"
                                currentBuild.result = 'UNSTABLE'
                            }
                        }
                    }
                }
                
                stage('Integration Tests') {
                    steps {
                        sh 'npm run test:integration'
                    }
                }
            }
        }
        
        // 阶段4: 代码质量检查
        stage('Code Quality') {
            steps {
                script {
                    // ESLint检查
                    sh 'npm run lint' | ignoreErrors()
                    
                    // 发布代码覆盖率
                    publishCoverage adapters: [
                        coberturaAdapter('coverage/cobertura-coverage.xml')
                    ]
                }
            }
        }
        
        // 阶段5: 打包Docker镜像
        stage('Docker Build') {
            steps {
                script {
                    def imageName = "${env.DOCKER_REGISTRY}/${env.IMAGE_NAME}:${params.VERSION ?: env.GIT_COMMIT}"
                    
                    sh "docker build -t ${imageName} ."
                    
                    withCredentials([usernamePassword(
                        credentialsId: 'docker-registry',
                        usernameVariable: 'DOCKER_USER',
                        passwordVariable: 'DOCKER_PASS'
                    )]) {
                        sh "echo \$DOCKER_PASS | docker login -u \$DOCKER_USER --password-stdin ${env.DOCKER_REGISTRY}"
                        sh "docker push ${imageName}"
                    }
                    
                    env.DOCKER_IMAGE = imageName
                }
            }
        }
        
        // 阶段6: 部署
        stage('Deploy') {
            steps {
                script {
                    def deployScript = """
                        #!/bin/bash
                        ssh deploy@${params.ENV}-server "
                            # 拉取镜像
                            docker pull ${env.DOCKER_IMAGE}
                            
                            # 停止旧容器
                            docker stop ${env.IMAGE_NAME} || true
                            
                            # 启动新容器
                            docker run -d \\
                                --name ${env.IMAGE_NAME} \\
                                --restart unless-stopped \\
                                -p 8080:8080 \\
                                ${env.DOCKER_IMAGE}
                        "
                    """
                    
                    sh deployScript
                }
            }
        }
        
        // 阶段7: 健康检查
        stage('Health Check') {
            steps {
                script {
                    def healthUrl = getHealthUrl(params.ENV)
                    
                    retry(5) {
                        sleep(10)
                        
                        def response = sh(
                            script: "curl -s -o /dev/null -w '%{http_code}' ${healthUrl}",
                            returnStdout: true
                        ).trim()
                        
                        if (response != '200') {
                            error "健康检查失败: HTTP ${response}"
                        }
                        
                        echo "健康检查通过: ${healthUrl}"
                    }
                }
            }
        }
    }
    
    post {
        // 成功后的处理
        success {
            echo '部署成功!'
            emailext(
                subject: "构建成功: ${env.JOB_NAME} #${env.BUILD_NUMBER}",
                body: "构建成功，已部署到${params.ENV}环境",
                to: 'devops@example.com'
            )
        }
        
        // 失败后的处理
        failure {
            echo '部署失败!'
            emailext(
                subject: "构建失败: ${env.JOB_NAME} #${env.BUILD_NUMBER}",
                body: "构建失败，请检查日志",
                to: 'devops@example.com',
                attachLog: true
            )
            
            // 回滚
            script {
                sh '''
                    ssh deploy@${params.ENV}-server "
                        docker stop ${IMAGE_NAME} || true
                        docker run -d --name ${IMAGE_NAME} --restart unless-stopped -p 8080:8080 ${DOCKER_REGISTRY}/${IMAGE_NAME}:previous
                    "
                '''
            }
        }
        
        // 清理
        always {
            cleanWs()
        }
    }
}

// 辅助函数
def getHealthUrl(env) {
    def urls = [
        dev: 'http://dev.example.com:8080/health',
        test: 'http://test.example.com:8080/health',
        prod: 'https://example.com/health'
    ]
    return urls[env] ?: urls.dev
}

五、多环境部署Pipeline

5.1 分环境Pipeline

// Jenkinsfile - 多环境部署

pipeline {
    agent any
    
    environment {
        DOCKER_REGISTRY = 'registry.example.com'
        APP_NAME = 'myapp'
    }
    
    parameters {
        choice(
            name: 'DEPLOY_ENV',
            choices: ['dev', 'test', 'prod'],
            description: '选择部署环境'
        )
    }
    
    stages {
        // 构建阶段
        stage('Build') {
            steps {
                echo "构建应用..."
                sh 'npm install && npm run build'
            }
        }
        
        // 测试阶段
        stage('Test') {
            when {
                anyOf {
                    params.DEPLOY_ENV == 'dev'
                    params.DEPLOY_ENV == 'test'
                }
            }
            steps {
                echo "运行测试..."
                sh 'npm test'
            }
        }
        
        // 打包阶段
        stage('Package') {
            steps {
                echo "打包Docker镜像..."
                sh """
                    docker build -t ${env.DOCKER_REGISTRY}/${env.APP_NAME}:${env.BUILD_NUMBER} .
                    docker push ${env.DOCKER_REGISTRY}/${env.APP_NAME}:${env.BUILD_NUMBER}
                """
            }
        }
        
        // 部署阶段
        stage('Deploy') {
            steps {
                script {
                    // 根据环境选择部署策略
                    switch(params.DEPLOY_ENV) {
                        case 'dev':
                            deployToDev()
                            break
                        case 'test':
                            deployToTest()
                            break
                        case 'prod':
                            deployToProd()
                            break
                    }
                }
            }
        }
    }
}

// 部署函数
def deployToDev() {
    echo '部署到开发环境...'
    sh '''
        kubectl set image deployment/myapp myapp=registry.example.com/myapp:BUILD_NUMBER -n dev
        kubectl rollout status deployment/myapp -n dev
    '''
}

def deployToTest() {
    echo '部署到测试环境...'
    sh '''
        kubectl set image deployment/myapp myapp=registry.example.com/myapp:BUILD_NUMBER -n test
        kubectl rollout status deployment/myapp -n test
    '''
}

def deployToProd() {
    echo '部署到生产环境...'
    sh '''
        # 蓝绿部署
        kubectl apply -f manifests/production/
        kubectl rollout status deployment/myapp -n production
    '''
}

六、自动化测试

6.1 单元测试集成

// Jenkinsfile - 测试阶段

stage('Unit Tests') {
    steps {
        script {
            def testResults = [:]
            
            // 前端测试
            testResults['Frontend'] = {
                dir('frontend') {
                    sh 'npm test'
                    archiveArtifacts artifacts: 'frontend/coverage/**/*'
                }
            }
            
            // 后端测试
            testResults['Backend'] = {
                dir('backend') {
                    sh 'mvn test'
                    archiveArtifacts artifacts: 'backend/target/surefire-reports/**/*'
                }
            }
            
            // 并行测试
            parallel testResults
        }
    }
}

6.2 E2E测试

stage('E2E Tests') {
    agent {
        docker {
            image 'cypress/browsers:latest'
            args '-v /tmp:/tmp'
        }
    }
    steps {
        echo '运行E2E测试...'
        sh '''
            npm install
            npm run test:e2e -- --browser chrome
        '''
        archiveArtifacts artifacts: 'cypress/videos/**/*, cypress/screenshots/**/*'
    }
}

6.3 性能测试

stage('Performance Tests') {
    steps {
        script {
            def perfResults = []
            
            // 压力测试
            sh '''
                echo "运行压力测试..."
                wrk -t12 -c1000 -d30s http://test-server:8080/api/test
            '''
            
            // 性能监控
            perfTest([
                runner: 'Gatling',
                simulation: 'MySimulation',
                reportFiles: 'build/reports/**/*.html'
            ])
        }
    }
}

七、Docker构建和推送

7.1 Dockerfile最佳实践

# Dockerfile - 多阶段构建

# 构建阶段
FROM node:16 AS builder

WORKDIR /app

# 复制依赖文件
COPY package*.json ./
RUN npm ci --only=production

# 复制源代码
COPY . .
RUN npm run build

# 运行阶段
FROM node:16-slim

WORKDIR /app

# 从构建阶段复制文件
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules
COPY --from=builder /app/package*.json ./

# 创建非root用户
RUN groupadd -r appuser && useradd -r -g appuser appuser
RUN chown -R appuser:appuser /app
USER appuser

# 暴露端口
EXPOSE 8080

# 健康检查
HEALTHCHECK --interval=30s --timeout=3s \
    CMD node healthcheck.js || exit 1

# 启动应用
CMD ["node", "dist/index.js"]

7.2 Jenkins构建和推送

stage('Docker Build & Push') {
    steps {
        script {
            def imageName = "${env.DOCKER_REGISTRY}/${env.IMAGE_NAME}:${env.BUILD_NUMBER}"
            def latestImage = "${env.DOCKER_REGISTRY}/${env.IMAGE_NAME}:latest"
            
            // 构建镜像
            sh """
                docker build \
                    --build-arg NODE_ENV=production \
                    -t ${imageName} \
                    -t ${latestImage} \
                    .
            """
            
            // 推送到仓库
            withCredentials([usernamePassword(
                credentialsId: 'docker-registry',
                usernameVariable: 'DOCKER_USER',
                passwordVariable: 'DOCKER_PASS'
            )]) {
                sh """
                    echo \$DOCKER_PASS | docker login -u \$DOCKER_USER --password-stdin ${env.DOCKER_REGISTRY}
                    docker push ${imageName}
                    docker push ${latestImage}
                """
            }
            
            // 清理本地镜像
            sh "docker rmi ${imageName} ${latestImage} || true"
        }
    }
}

八、Kubernetes部署

8.1 Deployment配置

# manifests/deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp
  namespace: production
spec:
  replicas: 3
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  selector:
    matchLabels:
      app: myapp
  template:
    metadata:
      labels:
        app: myapp
    spec:
      containers:
      - name: myapp
        image: registry.example.com/myapp:BUILD_NUMBER
        ports:
        - containerPort: 8080
        env:
        - name: NODE_ENV
          value: "production"
        - name: DATABASE_URL
          valueFrom:
            secretKeyRef:
              name: db-secret
              key: url
        resources:
          requests:
            memory: "256Mi"
            cpu: "250m"
          limits:
            memory: "512Mi"
            cpu: "500m"
        livenessProbe:
          httpGet:
            path: /health
            port: 8080
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /ready
            port: 8080
          initialDelaySeconds: 5
          periodSeconds: 5

8.2 Jenkins部署到K8s

stage('Deploy to Kubernetes') {
    steps {
        script {
            // 更新镜像标签
            sh """
                sed -i 's/BUILD_NUMBER/${env.BUILD_NUMBER}/g' manifests/deployment.yaml
            """
            
            // 应用配置
            sh """
                kubectl apply -f manifests/deployment.yaml
            """
            
            // 等待滚动更新完成
            sh """
                kubectl rollout status deployment/myapp -n production --timeout=300s
            """
            
            // 验证部署
            sh """
                kubectl get pods -n production -l app=myapp
                kubectl logs -n production -l app=myapp --tail=100
            """
        }
    }
}

九、通知和告警

9.1 钉钉通知

post {
    success {
        script {
            def message = """
            构建成功！
            项目: ${env.JOB_NAME}
            构建号: ${env.BUILD_NUMBER}
            分支: ${env.BRANCH_NAME}
            提交: ${env.GIT_COMMIT}
            环境: ${params.ENV}
            """
            
            sh """
                curl -X POST "$DINGTALK_WEBHOOK" \\
                    -H 'Content-Type: application/json' \\
                    -d '{
                        "msgtype": "text",
                        "text": {"content": "${message}"}
                    }'
            """
        }
    }
    
    failure {
        script {
            def message = """
            构建失败！
            项目: ${env.JOB_NAME}
            构建号: ${env.BUILD_NUMBER}
            错误: 请查看Jenkins日志
            """
            
            sh """
                curl -X POST "$DINGTALK_WEBHOOK" \\
                    -H 'Content-Type: application/json' \\
                    -d '{
                        "msgtype": "text",
                        "text": {"content": "${message}"}
                    }'
            """
        }
    }
}

9.2 邮件通知

post {
    success {
        emailext(
            subject: "✅ 构建成功: ${env.JOB_NAME} #${env.BUILD_NUMBER}",
            body: """
                <h2>构建成功</h2>
                <p><strong>项目:</strong> ${env.JOB_NAME}</p>
                <p><strong>构建号:</strong> ${env.BUILD_NUMBER}</p>
                <p><strong>分支:</strong> ${env.BRANCH_NAME}</p>
                <p><strong>环境:</strong> ${params.ENV}</p>
                <p><a href="${env.BUILD_URL}">查看详情</a></p>
            """,
            mimeType: 'text/html',
            to: 'devops@example.com'
        )
    }
}

十、最佳实践

10.1 Pipeline最佳实践

CI/CD最佳实践:
  1. 版本控制:
    - Jenkinsfile提交到Git
    - 使用GitOps理念
    - 配置即代码
  
  2. 环境隔离:
    - 开发、测试、生产分离
    - 独立Pipeline
    - 权限控制
  
  3. 安全性:
    - 使用凭据管理
    - 避免硬编码密码
    - 定期轮换密钥
  
  4. 监控告警:
    - 构建状态通知
    - 部署状态告警
    - 性能监控
  
  5. 回滚机制:
    - 自动回滚
    - 快速恢复
    - 版本管理

10.2 GitLab CI最佳实践

GitLab CI最佳实践:
  1. 缓存优化:
    - 缓存依赖包
    - 加速构建
    - 减少网络开销
  
  2. 并行执行:
    - 测试并行化
    - 提高效率
    - 缩短时间
  
  3. 条件执行:
    - 按分支执行
    - 按标签执行
    - 手动触发
  
  4. 工件管理:
    - 保存构建产物
    - 设置过期时间
    - 版本管理

十一、故障排查

11.1 常见问题

# Jenkins常见问题
# 1. 连接GitLab失败
# 检查Credentials和网络

# 2. Pipeline失败
# 查看日志和错误信息

# 3. Docker构建失败
# 检查Dockerfile和网络

# 查看Jenkins日志
sudo tail -f /var/log/jenkins/jenkins.log

# 查看构建日志
# Jenkins -> 构建 -> Console Output

11.2 调试技巧

// 使用debug输出
script {
    echo "Debug: BUILD_NUMBER = ${env.BUILD_NUMBER}"
    echo "Debug: DOCKER_IMAGE = ${env.DOCKER_IMAGE}"
    
    // 打印环境变量
    sh 'env | sort'
}

// 使用try-catch捕获错误
try {
    sh 'npm test'
} catch (Exception e) {
    echo "测试失败: ${e}"
    currentBuild.result = 'UNSTABLE'
}

十二、总结

Jenkins+GitLab CI/CD方案实现了：

核心要点

CI/CD流程：构建、测试、打包、部署
Pipeline自动化：Declarative、Scripted
多环境支持：dev、test、prod
监控告警：钉钉、邮件通知

技术要点

Jenkins Pipeline：声明式与脚本式
GitLab CI：gitlab-ci.yml 配置
Docker：构建、推送、打包
Kubernetes：部署、回滚

实践建议

Jenkinsfile纳入版本控制
实施严格的测试与质量检查
完善监控、日志与告警
建立自动回滚机制
定期演练与优化流程

通过Jenkins+GitLab，可实现软件交付的自动化与一致性。

第330集CI/CD架构实战：Jenkins+GitLab企业级持续集成与持续部署完整解决方案

CI/CD架构实战：Jenkins+GitLab企业级持续集成与持续部署

一、CI/CD概述

1.1 CI/CD核心概念

1.2 Jenkins+GitLab架构

二、Jenkins安装配置

2.1 Jenkins安装

2.2 初始配置

2.3 GitLab集成配置

Jenkins配置GitLab凭据

配置GitLab Connection

三、GitLab CI配置

3.1 .gitlab-ci.yml基础

完整的GitLab CI配置

四、Jenkins Pipeline

4.1 声明式Pipeline

完整的Jenkinsfile

五、多环境部署Pipeline

5.1 分环境Pipeline

六、自动化测试

6.1 单元测试集成

6.2 E2E测试

6.3 性能测试

七、Docker构建和推送

7.1 Dockerfile最佳实践

7.2 Jenkins构建和推送

八、Kubernetes部署

8.1 Deployment配置

8.2 Jenkins部署到K8s

九、通知和告警

9.1 钉钉通知

9.2 邮件通知

十、最佳实践

10.1 Pipeline最佳实践

10.2 GitLab CI最佳实践

十一、故障排查

11.1 常见问题

11.2 调试技巧

十二、总结

核心要点

技术要点

实践建议