第329集自动化发版架构实战:Shell+SCP+SSH实现无缝部署的企业级CI/CD解决方案 | 字数总计: 4.3k | 阅读时长: 20分钟 | 阅读量:
自动化发版架构实战:Shell+SCP+SSH实现无缝部署的企业级CI/CD解决方案 一、自动化发版概述 1.1 发版流程核心 自动化发版通过脚本完成构建、打包、传输和部署。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 自动化发版流程: 1 . 代码构建: - 编译代码 - 运行测试 - 打包制品 2 . 文件传输: - 使用SCP传输文件 - 验证文件完整性 - 备份当前版本 3 . 远程部署: - SSH连接服务器 - 停止服务 - 更新文件 - 启动服务 - 健康检查 4 . 回滚机制: - 检测部署失败 - 自动回滚 - 通知告警
1.2 Shell+SCP+SSH优势 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 Shell脚本优势: 简单直接: - 轻量级,无额外依赖 - 易于编写和理解 - 灵活控制流程 功能强大: - 文件操作 - 流程控制 - 错误处理 SSH/SCP优势: 安全可靠: - 加密传输 - 密钥认证 - 远程执行 操作便捷: - 单一命令传输 - 批量服务器管理 - 自动化运维
二、Shell脚本基础 2.1 发布脚本框架 基础发布脚本 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 #!/bin/bash set -e set -u DEPLOY_USER="deploy" DEPLOY_HOST="192.168.1.10" DEPLOY_PATH="/opt/app" APP_NAME="myapp" APP_VERSION="${1:-latest} " log () { echo "[$(date '+%Y-%m-%d %H:%M:%S') ] $@ " } error () { log "ERROR: $@ " >&2 exit 1 } success () { log "SUCCESS: $@ " } red () { echo -e "\033[31m$@ \033[0m" ; }green () { echo -e "\033[32m$@ \033[0m" ; }yellow () { echo -e "\033[33m$@ \033[0m" ; }log "开始部署 $APP_NAME 版本 $APP_VERSION "
2.2 分阶段发布脚本 完整多阶段发布脚本 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 #!/bin/bash CONFIG_FILE="deploy.conf" SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]} " ) " && pwd) " source "$SCRIPT_DIR /$CONFIG_FILE " || error "无法加载配置文件" DEPLOY_TIMESTAMP=$(date +%Y%m%d%H%M%S) BACKUP_DIR="$DEPLOY_PATH /backup" LOG_FILE="$SCRIPT_DIR /logs/deploy_${DEPLOY_TIMESTAMP} .log" mkdir -p "$SCRIPT_DIR /logs" exec > >(tee -a "$LOG_FILE " )exec 2>&1notify () { local msg="$1 " echo "$msg " } pre_check () { log "=== 步骤1: 前置检查 ===" if ! ssh -o ConnectTimeout=5 "$DEPLOY_USER @$DEPLOY_HOST " "echo 'SSH连接成功'" > /dev/null 2>&1; then error "无法连接到服务器 $DEPLOY_HOST " fi success "SSH连接检查通过" local package_file="dist/${APP_NAME} -${APP_VERSION} .tar.gz" if [ ! -f "$package_file " ]; then error "部署包不存在: $package_file " fi success "部署包检查通过: $package_file " local available=$(ssh "$DEPLOY_USER @$DEPLOY_HOST " "df $DEPLOY_PATH | tail -1 | awk '{print \$4}'" ) if [ "$available " -lt 1048576 ]; then error "服务器磁盘空间不足" fi success "磁盘空间检查通过" } package () { log "=== 步骤2: 打包 ===" local package_file="dist/${APP_NAME} -${APP_VERSION} .tar.gz" if [ -f "$package_file " ]; then success "使用已有包: $package_file " echo "$package_file " else error "请先构建应用程序" fi } backup () { log "=== 步骤3: 备份当前版本 ===" if ssh "$DEPLOY_USER @$DEPLOY_HOST " "test -d $DEPLOY_PATH /current" ; then ssh "$DEPLOY_USER @$DEPLOY_HOST " "mkdir -p $BACKUP_DIR " local backup_name="backup_${DEPLOY_TIMESTAMP} .tar.gz" ssh "$DEPLOY_USER @$DEPLOY_HOST " "cd $DEPLOY_PATH && tar -czf $BACKUP_DIR /$backup_name current/" if [ $? -eq 0 ]; then success "备份完成: $backup_name " ssh "$DEPLOY_USER @$DEPLOY_HOST " "echo '$backup_name ' > $DEPLOY_PATH /.last_backup" else error "备份失败" fi else log "首次部署,跳过备份" fi } transfer () { log "=== 步骤4: 文件传输 ===" local package_file="$1 " local remote_file="$DEPLOY_PATH /releases/${APP_NAME} -${APP_VERSION} .tar.gz" ssh "$DEPLOY_USER @$DEPLOY_HOST " "mkdir -p $DEPLOY_PATH /releases $DEPLOY_PATH /backup" log "传输文件: $package_file -> $DEPLOY_HOST :$remote_file " if scp "$package_file " "$DEPLOY_USER @$DEPLOY_HOST :$remote_file " ; then success "文件传输成功" local local_md5=$(md5sum "$package_file " | awk '{print $1}' ) local remote_md5=$(ssh "$DEPLOY_USER @$DEPLOY_HOST " "md5sum $remote_file | awk '{print \$1}'" ) if [ "$local_md5 " == "$remote_md5 " ]; then success "文件完整性验证通过" else error "文件完整性验证失败" fi else error "文件传输失败" fi } deploy_remote () { log "=== 步骤5: 远程部署 ===" local remote_file="$DEPLOY_PATH /releases/${APP_NAME} -${APP_VERSION} .tar.gz" local deploy_dir="$DEPLOY_PATH /releases/${APP_VERSION} " ssh "$DEPLOY_USER @$DEPLOY_HOST " << 'DEPLOY_SCRIPT' set -e mkdir -p $DEPLOY_DIR cd $DEPLOY_DIR tar -xzf $REMOTE_FILE if systemctl is-active --quiet $APP_NAME ; then systemctl stop $APP_NAME echo "服务已停止" fi ln -sfn $DEPLOY_DIR $DEPLOY_PATH /current systemctl start $APP_NAME echo "服务已启动" sleep 3 if systemctl is-active --quiet $APP_NAME ; then echo "服务运行正常" else echo "服务启动失败" >&2 exit 1 fi DEPLOY_SCRIPT if [ $? -eq 0 ]; then success "远程部署完成" else error "远程部署失败" fi } health_check () { log "=== 步骤6: 健康检查 ===" sleep 5 if ssh "$DEPLOY_USER @$DEPLOY_HOST " "systemctl is-active --quiet $APP_NAME " ; then success "服务运行状态正常" else error "服务未运行" fi local health_url="http://$DEPLOY_HOST :8080/health" log "HTTP健康检查: $health_url " local status_code=$(curl -s -o /dev/null -w "%{http_code}" "$health_url " || echo "000" ) if [ "$status_code " == "200" ]; then success "HTTP健康检查通过" else error "HTTP健康检查失败,状态码: $status_code " fi } main () { local package_file pre_check package_file=$(package) backup transfer "$package_file " deploy_remote health_check notify "✅ 部署成功: $APP_NAME -$APP_VERSION " } trap 'echo "部署失败于: $LINENO, 错误码: $?"' ERRtrap 'cleanup' EXITcleanup () { log "清理临时文件..." } main "$@ "
三、SCP文件传输 3.1 SCP基础用法 文件传输脚本 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 #!/bin/bash HOSTS=( "deploy@192.168.1.10" "deploy@192.168.1.11" "deploy@192.168.1.12" ) LOCAL_FILE="dist/app.tar.gz" REMOTE_DIR="/opt/app/releases" transfer_file () { local host=$1 log "传输到: $host " scp "$LOCAL_FILE " "$host :$REMOTE_DIR /" if [ $? -eq 0 ]; then success "传输成功: $host " return 0 else error "传输失败: $host " return 1 fi } for host in "${HOSTS[@]} " ; do if ! transfer_file "$host " ; then error "文件传输失败,中止部署" exit 1 fi done
3.2 高级SCP传输 带压缩和断点续传 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 #!/bin/bash transfer_with_compression () { local local_file="$1 " local remote_host="$2 " local remote_path="$3 " log "压缩传输: $local_file " gzip -c "$local_file " | ssh "$remote_host " "gunzip > $remote_path " rsync -avz --progress "$local_file " "$remote_host :$remote_path " } sync_directory () { local local_dir="$1 " local remote_host="$2 " local remote_dir="$3 " rsync -avz --delete \ --exclude '*.log' \ --exclude 'tmp/' \ "$local_dir " "$remote_host :$remote_dir " }
四、SSH远程执行 4.1 SSH批量执行 远程命令执行框架 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 #!/bin/bash execute_on_host () { local host="$1 " local command ="$2 " log "执行命令: $host -> $command " ssh "$host " << 'REMOTE_CMD' set -e export PATH=/usr/local/bin:$PATH eval "$COMMAND " REMOTE_CMD } execute_on_all_hosts () { local command ="$1 " for host in "${HOSTS[@]} " ; do log "执行于: $host " if ssh "$host " "$command " ; then success "$host : 执行成功" else error "$host : 执行失败" return 1 fi done }
4.2 远程服务管理 服务管理脚本 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 #!/bin/bash stop_service () { local host="$1 " local service="$2 " log "停止服务: $host -> $service " ssh "$host " << EOF sudo systemctl stop $service if [ \$? -eq 0 ]; then echo "服务 $service 已停止" else echo "服务 $service 停止失败" >&2 exit 1 fi EOF } start_service () { local host="$1 " local service="$2 " log "启动服务: $host -> $service " ssh "$host " << EOF sudo systemctl start $service # 等待服务启动 sleep 2 if systemctl is-active --quiet $service; then echo "服务 $service 已启动" else echo "服务 $service 启动失败" >&2 exit 1 fi EOF } restart_service () { local host="$1 " local service="$2 " log "重启服务: $host -> $service " ssh "$host " << EOF sudo systemctl restart $service # 等待服务就绪 sleep 3 if systemctl is-active --quiet $service; then echo "服务 $service 已重启" else echo "服务 $service 重启失败" >&2 exit 1 fi EOF } health_check () { local host="$1 " local service="$2 " local check_url="$3 " log "健康检查: $host " if ! ssh "$host " "systemctl is-active --quiet $service " ; then error "服务未运行: $service " return 1 fi if [ -n "$check_url " ]; then local status_code=$(ssh "$host " "curl -s -o /dev/null -w '%{http_code}' $check_url || echo '000'" ) if [ "$status_code " == "200" ]; then success "HTTP健康检查通过" else error "HTTP健康检查失败: $status_code " return 1 fi fi success "健康检查通过" } main () { local host="deploy@192.168.1.10" local service="myapp" local check_url="http://localhost:8080/health" stop_service "$host " "$service " start_service "$host " "$service " health_check "$host " "$service " "$check_url " } main "$@ "
五、零停机部署 5.1 蓝绿部署 蓝绿部署脚本 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 #!/bin/bash DEPLOY_HOST="deploy@192.168.1.10" BASE_DIR="/opt/app" APP_NAME="myapp" VERSION="$1 " log () { echo "[$(date '+%Y-%m-%d %H:%M:%S') ] $@ " } get_current_env () { ssh "$DEPLOY_HOST " "readlink -f $BASE_DIR /current | xargs basename" } switch_traffic () { local target="$1 " log "切换到: $target 环境" ssh "$DEPLOY_HOST " << EOF # 停止旧环境 systemctl stop ${APP_NAME}-blue 2>/dev/null || true systemctl stop ${APP_NAME}-green 2>/dev/null || true # 启动目标环境 systemctl start ${APP_NAME}-${target} # 更新软链接 ln -sfn $BASE_DIR/${target} $BASE_DIR/current # 验证 if systemctl is-active --quiet ${APP_NAME}-${target}; then echo "流量已切换到 ${target}" else echo "切换失败" >&2 exit 1 fi EOF } deploy_to_idle () { local current=$(get_current_env) local target if [ "$current " == "blue" ]; then target="green" else target="blue" fi log "部署到: $target 环境" scp "dist/${APP_NAME} -${VERSION} .tar.gz" "$DEPLOY_HOST :$BASE_DIR /releases/" ssh "$DEPLOY_HOST " << DEPLOY_SCRIPT cd $BASE_DIR mkdir -p ${target} tar -xzf releases/${APP_NAME}-${VERSION}.tar.gz -C ${target} cd ${target} # 安装依赖等 # ... # 启动服务 systemctl start ${APP_NAME}-${target} # 健康检查 sleep 5 if curl -f http://localhost:8080/health > /dev/null 2>&1; then echo "部署成功" else echo "部署失败" >&2 exit 1 fi DEPLOY_SCRIPT } main () { local target_env deploy_to_idle if [ "$(get_current_env) " == "blue" ]; then target_env="green" else target_env="blue" fi log "健康检查: $target_env " if ssh "$DEPLOY_HOST " "curl -f http://localhost:8080/health" > /dev/null 2>&1; then success "新版本健康检查通过" else error "新版本健康检查失败,不切换" exit 1 fi switch_traffic "$target_env " success "部署完成" } main "$@ "
5.2 灰度发布 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 #!/bin/bash DEPLOY_HOSTS=( "deploy@192.168.1.10" "deploy@192.168.1.11" ) PRODUCTION_HOSTS=( "deploy@192.168.1.20" "deploy@192.168.1.21" ) deploy_canary () { log "=== 灰度部署 ===" for host in "${DEPLOY_HOSTS[@]} " ; do log "部署到灰度服务器: $host " done log "等待验证期: 5分钟" sleep 300 if check_canary_health; then success "灰度验证通过" return 0 else error "灰度验证失败" return 1 fi } deploy_production () { log "=== 生产部署 ===" for host in "${PRODUCTION_HOSTS[@]} " ; do log "部署到生产: $host " done } main () { if ! deploy_canary; then error "灰度部署失败,中止" exit 1 fi read -p "灰度验证完成,是否继续生产部署? (y/n) " -n 1 -r echo if [[ ! $REPLY =~ ^[Yy]$ ]]; then log "部署已取消" exit 0 fi deploy_production } main "$@ "
六、企业级部署方案 6.1 多环境部署脚本 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 #!/bin/bash declare -A ENV_CONFIGSENV_CONFIGS[dev]="deploy@192.168.1.10" ENV_CONFIGS[test ]="deploy@192.168.1.20" ENV_CONFIGS[prod]="deploy@192.168.1.30" deploy_to_env () { local env ="$1 " local version="$2 " log "部署到环境: $env " local host="${ENV_CONFIGS[$env]} " if [ -z "$host " ]; then error "未知环境: $env " return 1 fi ssh "$host " "bash -s" << EOF set -e echo "部署 $version 到 $env" # 备份 systemctl stop myapp # 更新 tar -xzf /tmp/app.tar.gz -C /opt/app/current # 启动 systemctl start myapp # 健康检查 sleep 3 if systemctl is-active --quiet myapp; then echo "部署成功" else echo "部署失败" >&2 exit 1 fi EOF } main () { local version="$1 " local envs="${2:-dev,test,prod} " IFS=',' read -ra env_array <<< "$envs " for env in "${env_array[@]} " ; do if deploy_to_env "$env " "$version " ; then success "$env 部署成功" if [ "$env " == "prod" ]; then read -p "生产部署成功,请确认是否一切正常? (y/n) " -n 1 -r echo if [[ ! $REPLY =~ ^[Yy]$ ]]; then log "需要回滚" fi fi else error "$env 部署失败" break fi done } main "$@ "
七、回滚机制 7.1 自动回滚脚本 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 #!/bin/bash rollback () { local host="$1 " local backup_file="$2 " log "回滚: $host " ssh "$host " << ROLLBACK_SCRIPT set -e # 停止服务 systemctl stop myapp # 恢复备份 cd $DEPLOY_PATH tar -xzf backup/$backup_file # 重新启动服务 systemctl start myapp # 验证 sleep 3 if systemctl is-active --quiet myapp && curl -f http://localhost:8080/health; then echo "回滚成功" else echo "回滚失败" >&2 exit 1 fi ROLLBACK_SCRIPT } auto_rollback_on_failure () { local exit_code=$? local backup_file="$DEPLOY_PATH /.last_backup" if [ $exit_code -ne 0 ]; then log "部署失败,开始回滚..." if [ -f "$backup_file " ]; then local backup_name=$(cat "$backup_file " ) rollback "$DEPLOY_HOST " "$backup_name " notify "回滚完成" else error "未找到备份文件,无法回滚" fi fi } trap 'auto_rollback_on_failure' ERR
八、最佳实践 8.1 部署规范 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 部署脚本最佳实践: 1 . 错误处理: - 使用set -e 立即退出 - 捕获所有可能的错误 - 提供清晰的错误信息 2 . 日志记录: - 记录所有操作 - 保存历史日志 - 便于问题追踪 3 . 幂等性: - 可重复执行 - 不产生副作用 - 状态检查 4 . 安全性: - 使用SSH密钥 - 验证文件完整性 - 限制权限 5 . 可观测性: - 健康检查 - 性能监控 - 告警通知
8.2 配置管理 deploy.conf配置文件 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 ENV="${DEPLOY_ENV:-production} " declare -A SERVERSSERVERS[dev]="deploy@192.168.1.10" SERVERS[test ]="deploy@192.168.1.20" SERVERS[prod]="deploy@192.168.1.30 deploy@192.168.1.31 deploy@192.168.1.32" APP_NAME="myapp" APP_VERSION="1.0.0" DEPLOY_PATH="/opt/app" SSH_USER="deploy" SSH_KEY="~/.ssh/deploy_key" SSH_OPTS="-o StrictHostKeyChecking=no -o ConnectTimeout=10" DEPLOY_TIMEOUT="300" HEALTH_CHECK_RETRIES="5" HEALTH_CHECK_INTERVAL="10" NOTIFY_WEBHOOK="https://oapi.dingtalk.com/robot/send?access_token=xxx" case "$ENV " in dev) MAX_DEPLOY_WORKERS=1 ;; test ) MAX_DEPLOY_WORKERS=2 ;; prod) MAX_DEPLOY_WORKERS=3 ;; esac
九、总结 基于Shell+SCP+SSH的自动化发版方案具备:
核心要点
Shell脚本 :流程控制、错误处理
SCP传输 :安全文件传输、完整性验证
SSH执行 :远程命令执行、服务管理
部署策略 :零停机、蓝绿、灰度
技术要点
脚本框架 :模块化、可复用
错误处理 :自动回滚、健壮性
监控告警 :健康检查、通知
版本管理 :备份、回滚
实践建议
使用标准的脚本框架与模板
强制启用错误处理与回滚
完整的日志记录与审计
安全的密钥与权限管理
通过健康检查与告警提升可观测性
通过脚本化、安全与自动化,可确保部署的可靠性。