企业级基础设施架构实战：服务器资源规划、微服务中间件部署与完整技术栈配置方案

一、基础设施架构概述

1.1 服务器资源配置

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
服务器资源配置方案:
  应用服务器 (2台):
    CPU: 4核
    内存: 16GB
    磁盘: 500GB SSD
    用途: 应用服务部署
    操作系统: CentOS 7
  
  数据库服务器 (1台):
    CPU: 2核
    内存: 4GB
    磁盘: 500GB SSD
    用途: MySQL + Redis
    操作系统: CentOS 7
  
  中间件服务器 (2台):
    CPU: 4核
    内存: 16GB
    磁盘: 500GB SSD
    用途: Kafka + Zookeeper
    操作系统: CentOS 7
  
  服务注册中心:
    Nacos: 3个节点
    ES: 3个节点
    Cassandra: 3个节点

1.2 技术栈组成

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
完整技术栈:
  基础组件:
    - Nacos: 服务注册与配置中心
    - Nginx: 反向代理和负载均衡
    - 操作系统: CentOS 7
  
  消息中间件:
    - Kafka: 3个broker节点
    - Zookeeper: 3个节点
    - Maxwell: CDC数据同步
  
  数据库:
    - MySQL 8.0: 主数据库
    - Redis: 缓存和会话存储
    - Cassandra: 时序数据存储
  
  搜索引擎:
    - Elasticsearch: 全文搜索和日志分析
    - Kibana: 数据可视化
  
  对象存储:
    - OSS/Aliyun OSS: 文件存储
    - 图片、文档等静态资源

二、服务器资源规划

2.1 资源分配方案

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
#!/bin/bash
# resource_planning.sh - 资源规划脚本

echo "=== 企业级服务器资源规划 ==="
echo ""

# 应用服务器1 (Server-01)
echo "--- 应用服务器 1 (Server-01) ---"
echo "硬件配置:"
echo "  CPU: 4核心"
echo "  内存: 16GB"
echo "  磁盘: 500GB SSD"
echo ""
echo "部署服务:"
echo "  - Nacos Server (端口: 8848)"
echo "  - 应用服务1"
echo "  - Nginx (端口: 80, 443)"
echo ""
echo "资源预留:"
echo "  系统预留: 10% (1.6GB内存, 50GB磁盘)"
echo "  应用使用: 80% (12.8GB内存, 350GB磁盘)"
echo "  预留缓冲: 10% (1.6GB内存, 100GB磁盘)"
echo ""

# 应用服务器2 (Server-02)
echo "--- 应用服务器 2 (Server-02) ---"
echo "硬件配置:"
echo "  CPU: 4核心"
echo "  内存: 16GB"
echo "  磁盘: 500GB SSD"
echo ""
echo "部署服务:"
echo "  - Nacos Server (端口: 8848)"
echo "  - 应用服务2"
echo "  - Elasticsearch (端口: 9200)"
echo ""
echo "资源预留:"
echo "  ES预留: 8GB内存"
echo "  应用预留: 6GB内存"
echo "  系统预留: 2GB内存"
echo ""

# 数据库服务器 (DB-Server)
echo "--- 数据库服务器 (DB-Server) ---"
echo "硬件配置:"
echo "  CPU: 2核心"
echo "  内存: 4GB"
echo "  磁盘: 500GB SSD"
echo ""
echo "部署服务:"
echo "  - MySQL (端口: 3306)"
echo "  - Redis (端口: 6379)"
echo ""
echo "资源分配:"
echo "  MySQL: 2.5GB内存"
echo "  Redis: 1GB内存"
echo "  系统: 0.5GB内存"
echo ""

# 中间件服务器1 (MQ-01)
echo "--- 中间件服务器 1 (MQ-01) ---"
echo "硬件配置:"
echo "  CPU: 4核心"
echo "  内存: 16GB"
echo "  磁盘: 500GB SSD"
echo ""
echo "部署服务:"
echo "  - Kafka Broker 1"
echo "  - Zookeeper 1"
echo "  - Cassandra 1"
echo ""
echo "资源分配:"
echo "  Kafka: 4GB内存"
echo "  Zookeeper: 1GB内存"
echo "  Cassandra: 6GB内存"
echo "  系统预留: 5GB内存"
echo ""

# 中间件服务器2 (MQ-02)
echo "--- 中间件服务器 2 (MQ-02) ---"
echo "硬件配置:"
echo "  CPU: 4核心"
echo "  内存: 16GB"
echo "  磁盘: 500GB SSD"
echo ""
echo "部署服务:"
echo "  - Kafka Broker 2"
echo "  - Zookeeper 2"
echo "  - Maxwell"
echo ""
echo "资源分配:"
echo "  Kafka: 4GB内存"
echo "  Zookeeper: 1GB内存"
echo "  Maxwell: 2GB内存"
echo "  系统预留: 9GB内存"
echo ""

echo "资源规划完成"

2.2 服务器监控脚本

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
#!/bin/bash
# server_monitor.sh - 服务器资源监控

monitor_server() {
    local host=$1
    local name=$2
    
    echo "=== 监控服务器: $name ==="
    
    # CPU使用率
    cpu_usage=$(ssh $host "top -bn1 | grep 'Cpu(s)' | sed 's/.*, *\([0-9.]*\)%* id.*/\1/' | awk '{print 100 - \$1}'")
    echo "CPU使用率: ${cpu_usage}%"
    
    # 内存使用
    memory=$(ssh $host "free -h | grep Mem")
    echo "内存: $memory"
    
    # 磁盘使用
    disk=$(ssh $host "df -h | grep '/$'")
    echo "磁盘: $disk"
    
    # 进程数量
    process_count=$(ssh $host "ps aux | wc -l")
    echo "进程数: $process_count"
    
    echo ""
}

# 监控所有服务器
monitor_server "server-01" "应用服务器1"
monitor_server "server-02" "应用服务器2"
monitor_server "db-server" "数据库服务器"
monitor_server "mq-01" "中间件服务器1"
monitor_server "mq-02" "中间件服务器2"

三、MySQL部署配置

3.1 MySQL安装配置

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
#!/bin/bash
# install_mysql.sh - MySQL安装脚本

echo "=== 安装MySQL 8.0 ==="

# 1. 下载MySQL RPM包
wget https://dev.mysql.com/get/mysql80-community-release-el7-3.noarch.rpm

# 2. 安装RPM
rpm -ivh mysql80-community-release-el7-3.noarch.rpm

# 3. 安装MySQL
yum install -y mysql-community-server

# 4. 启动MySQL
systemctl start mysqld
systemctl enable mysqld

# 5. 获取临时密码
temp_password=$(grep 'temporary password' /var/log/mysqld.log | awk '{print $NF}')
echo "临时密码: $temp_password"

# 6. 安全配置
mysql_secure_installation

# 7. 配置MySQL
cat > /etc/my.cnf <<EOF
[mysqld]
# 基本配置
port = 3306
bind-address = 0.0.0.0

# 字符集
character-set-server = utf8mb4
collation-server = utf8mb4_unicode_ci

# 存储引擎
default-storage-engine = InnoDB

# 内存配置 (总内存4GB)
innodb_buffer_pool_size = 2G
innodb_log_file_size = 256M
innodb_log_buffer_size = 64M
innodb_flush_log_at_trx_commit = 1
innodb_flush_method = O_DIRECT

# 连接配置
max_connections = 200
max_connect_errors = 10000
wait_timeout = 28800
interactive_timeout = 28800

# 查询缓存 (MySQL 8.0已移除)
# query_cache_type = 0
# query_cache_size = 0

# 日志配置
slow_query_log = 1
slow_query_log_file = /var/log/mysql/slow-query.log
long_query_time = 2
log_bin = /var/log/mysql/mysql-bin.log
binlog_format = ROW
expire_logs_days = 7
max_binlog_size = 100M

# 表名大小写
lower_case_table_names = 1

# 错误日志
log-error = /var/log/mysql/error.log

[client]
default-character-set = utf8mb4
EOF

# 8. 重启MySQL
systemctl restart mysqld

echo "MySQL安装完成"

3.2 MySQL性能优化

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
-- my.cnf优化配置

[mysqld]
# InnoDB优化
innodb_buffer_pool_size = 2G
innodb_buffer_pool_instances = 2
innodb_log_file_size = 256M
innodb_log_files_in_group = 2
innodb_file_per_table = 1
innodb_flush_log_at_trx_commit = 1
innodb_flush_method = O_DIRECT

# 连接优化
max_connections = 200
thread_cache_size = 50
table_open_cache = 2000

# 查询优化
tmp_table_size = 64M
max_heap_table_size = 64M
sort_buffer_size = 2M
read_buffer_size = 1M
read_rnd_buffer_size = 2M

# 其他优化
max_allowed_packet = 64M

四、Redis部署配置

4.1 Redis安装配置

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
#!/bin/bash
# install_redis.sh - Redis安装脚本

echo "=== 安装Redis 6.2 ==="

# 1. 下载Redis
cd /usr/local/src
wget http://download.redis.io/releases/redis-6.2.7.tar.gz

# 2. 解压
tar -xzf redis-6.2.7.tar.gz
cd redis-6.2.7

# 3. 编译安装
make && make install

# 4. 创建配置目录
mkdir -p /etc/redis
mkdir -p /var/log/redis
mkdir -p /var/lib/redis

# 5. 复制配置文件
cp redis.conf /etc/redis/redis.conf

# 6. 配置文件
cat > /etc/redis/redis.conf <<EOF
# 基本配置
bind 0.0.0.0
port 6379
daemonize yes
pidfile /var/run/redis/redis-server.pid
logfile /var/log/redis/redis.log

# 持久化配置
save 900 1
save 300 10
save 60 10000
stop-writes-on-bgsave-error yes
rdbcompression yes
dbfilename dump.rdb
dir /var/lib/redis

# AOF配置
appendonly yes
appendfilename "appendonly.aof"
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb

# 内存配置
maxmemory 1gb
maxmemory-policy allkeys-lru

# 性能配置
tcp-backlog 511
timeout 0
tcp-keepalive 300

# 客户端配置
maxclients 1000

# 日志级别
loglevel notice
EOF

# 7. 创建systemd服务
cat > /etc/systemd/system/redis.service <<EOF
[Unit]
Description=Redis In-Memory Data Store
After=network.target

[Service]
User=redis
Group=redis
ExecStart=/usr/local/bin/redis-server /etc/redis/redis.conf
ExecStop=/usr/local/bin/redis-cli shutdown
Restart=always

[Install]
WantedBy=multi-user.target
EOF

# 8. 创建redis用户
useradd -r -s /bin/false redis
chown -R redis:redis /var/lib/redis
chown -R redis:redis /var/log/redis

# 9. 启动Redis
systemctl daemon-reload
systemctl start redis
systemctl enable redis

echo "Redis安装完成"

4.2 Redis性能优化

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
# redis.conf优化配置

# 内存优化
maxmemory 1gb
maxmemory-policy allkeys-lru
maxmemory-samples 5

# 持久化优化
save ""  # 禁用RDB（如果使用AOF）
appendonly yes
appendfsync everysec
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb

# 网络优化
tcp-backlog 511
timeout 300

# 客户端优化
maxclients 1000

五、Kafka集群部署

5.1 Kafka安装配置

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
#!/bin/bash
# install_kafka.sh - Kafka集群安装

echo "=== 安装Kafka 2.13 ==="

# 1. 下载Kafka
cd /usr/local/src
wget https://downloads.apache.org/kafka/3.3.0/kafka_2.13-3.3.0.tgz

# 2. 解压
tar -xzf kafka_2.13-3.3.0.tgz
mv kafka_2.13-3.3.0 /usr/local/kafka

# 3. 配置文件
cat > /usr/local/kafka/config/server.properties <<EOF
# Broker ID (每个节点不同: 0, 1, 2)
broker.id=0

# 监听地址
listeners=PLAINTEXT://0.0.0.0:9092
advertised.listeners=PLAINTEXT://mq-01:9092

# 日志目录
log.dirs=/var/kafka-logs

# Zookeeper配置
zookeeper.connect=mq-01:2181,mq-02:2181

# 性能优化
num.network.threads=8
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600

# 副本配置
default.replication.factor=2
min.insync.replicas=2

# 日志保留
log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000

# 压缩
compression.type=snappy

# 内存配置
num.replica.fetchers=4
replica.fetch.max.bytes=10485760
replica.fetch.wait.max.ms=500

# 事务配置
transaction.state.log.replication.factor=2
transaction.state.log.min.isr=2
transaction.max.timeout.ms=900000

# 其他配置
offsets.topic.replication.factor=2
transaction.state.log.num.partitions=16
EOF

# 4. 创建systemd服务
cat > /etc/systemd/system/kafka.service <<EOF
[Unit]
Description=Apache Kafka Server
Documentation=http://kafka.apache.org/documentation.html
Requires=zookeeper.service
After=zookeeper.service

[Service]
Type=simple
User=kafka
Group=kafka
Environment=JAVA_HOME=/usr/lib/jvm/java-11-openjdk
Environment="KAFKA_HEAP_OPTS=-Xmx4G -Xms4G"
ExecStart=/usr/local/kafka/bin/kafka-server-start.sh /usr/local/kafka/config/server.properties
ExecStop=/usr/local/kafka/bin/kafka-server-stop.sh
Restart=on-abnormal

[Install]
WantedBy=multi-user.target
EOF

# 5. 创建用户
useradd -r -s /bin/false kafka
chown -R kafka:kafka /usr/local/kafka
chown -R kafka:kafka /var/kafka-logs

# 6. 启动Kafka
systemctl daemon-reload
systemctl start kafka
systemctl enable kafka

echo "Kafka安装完成"

六、Zookeeper集群部署

6.1 Zookeeper安装配置

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
#!/bin/bash
# install_zookeeper.sh - Zookeeper集群安装

echo "=== 安装Zookeeper 3.8 ==="

# 1. 下载Zookeeper
cd /usr/local/src
wget https://archive.apache.org/dist/zookeeper/zookeeper-3.8.0/apache-zookeeper-3.8.0-bin.tar.gz

# 2. 解压
tar -xzf apache-zookeeper-3.8.0-bin.tar.gz
mv apache-zookeeper-3.8.0-bin /usr/local/zookeeper

# 3. 创建数据目录
mkdir -p /var/zookeeper
mkdir -p /var/zookeeper/data
mkdir -p /var/zookeeper/logs

# 4. 配置文件
cat > /usr/local/zookeeper/conf/zoo.cfg <<EOF
# 基本配置
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/var/zookeeper/data
dataLogDir=/var/zookeeper/logs
clientPort=2181
maxClientCnxns=60

# 性能优化
preAllocSize=65536
snapCount=100000
autopurge.snapRetainCount=3
autopurge.purgeInterval=1

# 集群配置
server.1=mq-01:2888:3888
server.2=mq-02:2888:3888
EOF

# 5. 创建myid文件 (每个节点不同: 1, 2)
echo "1" > /var/zookeeper/data/myid

# 6. 创建systemd服务
cat > /etc/systemd/system/zookeeper.service <<EOF
[Unit]
Description=Apache Zookeeper server
Documentation=http://zookeeper.apache.org
After=network.target

[Service]
Type=simple
User=zookeeper
Group=zookeeper
Environment=JAVA_HOME=/usr/lib/jvm/java-11-openjdk
Environment="JVMFLAGS=-Xmx1G -Xms1G"
ExecStart=/usr/local/zookeeper/bin/zkServer.sh start
ExecStop=/usr/local/zookeeper/bin/zkServer.sh stop
Restart=on-failure

[Install]
WantedBy=multi-user.target
EOF

# 7. 创建用户
useradd -r -s /bin/false zookeeper
chown -R zookeeper:zookeeper /usr/local/zookeeper
chown -R zookeeper:zookeeper /var/zookeeper

# 8. 启动Zookeeper
systemctl daemon-reload
systemctl start zookeeper
systemctl enable zookeeper

echo "Zookeeper安装完成"

七、Elasticsearch部署

7.1 ES安装配置

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
#!/bin/bash
# install_elasticsearch.sh - Elasticsearch安装

echo "=== 安装Elasticsearch 7.17 ==="

# 1. 下载ES
cd /usr/local/src
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.17.9-linux-x86_64.tar.gz

# 2. 解压
tar -xzf elasticsearch-7.17.9-linux-x86_64.tar.gz
mv elasticsearch-7.17.9 /usr/local/elasticsearch

# 3. 配置文件
cat > /usr/local/elasticsearch/config/elasticsearch.yml <<EOF
# 集群配置
cluster.name: my-application
node.name: node-1
node.master: true
node.data: true

# 网络配置
network.host: 0.0.0.0
http.port: 9200

# 发现配置
discovery.seed_hosts: ["es-01:9300", "es-02:9300"]
cluster.initial_master_nodes: ["node-1"]

# 性能优化
bootstrap.memory_lock: true

# 索引设置
action.auto_create_index: true
EOF

# 4. JVM配置
cat > /usr/local/elasticsearch/config/jvm.options <<EOF
# Xms和Xmx设置为物理内存的50%
-Xms4g
-Xmx4g

# GC配置
-XX:+UseG1GC
-XX:MaxGCPauseMillis=200
-XX:+ParallelRefProcEnabled
-XX:MaxDirectMemorySize=2g

# 其他配置
-XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=/var/log/elasticsearch
EOF

# 5. 创建用户
useradd -r -s /bin/false elasticsearch
chown -R elasticsearch:elasticsearch /usr/local/elasticsearch

# 6. 配置系统limits
cat >> /etc/security/limits.conf <<EOF
elasticsearch soft nofile 65536
elasticsearch hard nofile 65536
elasticsearch soft memlock unlimited
elasticsearch hard memlock unlimited
EOF

# 7. 启动ES
systemctl start elasticsearch
systemctl enable elasticsearch

echo "Elasticsearch安装完成"

八、Nacos部署

8.1 Nacos安装配置

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
#!/bin/bash
# install_nacos.sh - Nacos安装

echo "=== 安装Nacos 2.2 ==="

# 1. 下载Nacos
cd /usr/local/src
wget https://github.com/alibaba/nacos/releases/download/2.2.0/nacos-server-2.2.0.tar.gz

# 2. 解压
tar -xzf nacos-server-2.2.0.tar.gz
mv nacos /usr/local/

# 3. 集群配置
cat > /usr/local/nacos/conf/cluster.conf <<EOF
server-01:8848
server-02:8848
EOF

# 4. MySQL配置
mysql -u root -p < /usr/local/nacos/conf/nacos-mysql.sql

# 5. 配置文件
cat > /usr/local/nacos/conf/application.properties <<EOF
# 服务器配置
server.servlet.contextPath=/nacos
server.servlet.encoding=UTF-8
server.contextPath=/nacos

# 数据库配置
spring.datasource.platform=mysql
db.num=1
db.url.0=jdbc:mysql://db-server:3306/nacos_config?characterEncoding=utf8&serverTimezone=UTC
db.user=root
db.password=your_password

# Nacos核心配置
nacos.cmdb.dumpTaskInterval=3600
nacos.cmdb.taskTimerThreadCount=1
nacos.cmdb.loadDataAtStart=false

# 集群配置
nacos.core.auth.caching.enabled=true
nacos.core.auth.caching.caches.cache.size=4096
nacos.core.auth.default.token.expire.seconds=18000
nacos.core.auth.enable.userAgentAuthWhite=true
nacos.core.auth.server.identity.key=serverIdentity
nacos.core.auth.server.identity.value=security
nacos.core.auth.plugin.nacos.token.secret.key=VGhpc0lzTXlDdXN0b21TZWNyZXRLZXkwMTIzNDU2Nzg=
EOF

# 6. 启动Nacos
cd /usr/local/nacos/bin
./startup.sh -m cluster

echo "Nacos安装完成"

九、Maxwell部署

9.1 Maxwell安装配置

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
#!/bin/bash
# install_maxwell.sh - Maxwell安装

echo "=== 安装Maxwell ==="

# 1. 下载Maxwell
cd /usr/local/src
wget https://github.com/zendesk/maxwell/releases/download/v1.38.1/maxwell-1.38.1.tar.gz

# 2. 解压
tar -xzf maxwell-1.38.1.tar.gz
mv maxwell-1.38.1 /usr/local/maxwell

# 3. 配置文件
cat > /usr/local/maxwell/config.properties <<EOF
# Maxwell配置
host=db-server
user=maxwell
password=your_password
port=3306
schema_database=maxwell
producer=kafka
kafka.bootstrap.servers=mq-01:9092,mq-02:9092
kafka_topic=mysql_maxwell
output_binlog_position=true
output_ddl=true
EOF

# 4. 创建systemd服务
cat > /etc/systemd/system/maxwell.service <<EOF
[Unit]
Description=Maxwell MySQL Binlog Replicator
After=network.target kafka.service

[Service]
Type=simple
User=maxwell
Group=maxwell
Environment="MAXWELL_HOME=/usr/local/maxwell"
WorkingDirectory=/usr/local/maxwell
ExecStart=/usr/local/maxwell/bin/maxwell --config /usr/local/maxwell/config.properties
Restart=on-failure

[Install]
WantedBy=multi-user.target
EOF

# 5. 启动Maxwell
systemctl daemon-reload
systemctl start maxwell
systemctl enable maxwell

echo "Maxwell安装完成"

十、技术栈架构图

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
graph TB
    subgraph "应用层"
        A[Nginx] --> B[应用服务1]
        A --> C[应用服务2]
    end
    
    subgraph "注册中心"
        D[Nacos集群]
    end
    
    subgraph "消息中间件"
        E[Kafka集群] --> F[Zookeeper集群]
        G[Maxwell CDC] --> E
    end
    
    subgraph "数据层"
        H[MySQL] --> G
        I[Redis]
        J[Cassandra]
    end
    
    subgraph "搜索分析"
        K[Elasticsearch集群]
        L[Kibana]
    end
    
    subgraph "存储层"
        M[OSS对象存储]
    end
    
    B --> D
    C --> D
    B --> E
    C --> E
    B --> H
    C --> H
    B --> I
    C --> I
    E --> K
    L --> K
    B --> M
    C --> M

十一、监控和运维

11.1 统一监控脚本

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
#!/bin/bash
# infrastructure_monitor.sh - 基础设施监控

echo "=== 基础设施监控 ==="
echo ""

# MySQL监控
echo "--- MySQL监控 ---"
mysql -h db-server -u root -p -e "SHOW STATUS LIKE 'Threads_connected';" 2>/dev/null

# Redis监控
echo "--- Redis监控 ---"
redis-cli -h db-server INFO memory | grep used_memory_human

# Kafka监控
echo "--- Kafka监控 ---"
/usr/local/kafka/bin/kafka-topics.sh --bootstrap-server mq-01:9092 --list

# Zookeeper监控
echo "--- Zookeeper监控 ---"
echo ruok | nc mq-01 2181

# Elasticsearch监控
echo "--- Elasticsearch监控 ---"
curl -s http://es-01:9200/_cluster/health | jq .

# Nacos监控
echo "--- Nacos监控 ---"
curl -s http://server-01:8848/nacos/v1/console/health

echo ""
echo "监控完成"

十二、总结

12.1 架构要点

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
基础设施架构要点:
  服务器规划:
    - 2台应用服务器: 负载均衡和高可用
    - 1台数据库服务器: 集中管理数据
    - 2台中间件服务器: 消息队列集群
  
  技术栈选型:
    - 微服务治理: Nacos
    - 消息中间件: Kafka + Zookeeper
    - 数据库: MySQL + Redis
    - 搜索引擎: Elasticsearch
    - CDC: Maxwell
    - 可视化: Kibana
    - 对象存储: OSS
  
  高可用方案:
    - 应用层: 多实例部署
    - 中间件: 集群模式
    - 数据库: 主从复制
    - 消息: 副本机制

12.2 最佳实践

  1. 资源预留: 系统预留20%资源
  2. 监控告警: 搭建统一监控
  3. 日志管理: 统一日志收集与分析
  4. 备份策略: 定期备份与恢复演练
  5. 容量规划: 按需扩展

以上配置满足中小型生产环境的可用性、稳定性与扩展性。