
Patroni方案简介Patroni是一个基于zk、etcd、consul等的pg ha模板可以使用python来创建和定制高可用性解决方案。Patroni使用分布式key-value数据库作为数据存储主节点故障时进行主节点重新选举。通过PG内置的流复制支持同步和异步复制。由于数据库信息记录在ETCD中通过增加部署实例数可以避免脑裂且该方案自动化程度较高可以自动初始化PG实例当STANDBY实例关闭后支持自动尝试拉起支持当主库宕机后自动选取新主库。ETCD是一个基于RAFT协议的分布式Key-Value数据库基于Go语言编写Patroni监控本地的PG数据库状态并将相关信息写入ETCD。每个Patroni都能读写ETCD的KEY从而获取其他主机的PG数据库实例信息。环境说明操作系统CentOS Linux 7.9数据库版本PostgreSQL 13.1Python版本Python3.8ETCD版本3.3Patroni版本2.0.1部署规划主机名IP组件备注PG1192.168.21.135PostgreSQL、Patroni、ETCDMASTERPG2192.168.21.136PostgreSQL、Patroni、ETCDSTANDBYPG3192.168.21.137PostgreSQL、Patroni、ETCDSTANDBY搭建部署1.安装PostgreSQL以及搭建流复制略2. 安装etcd集群# PG1节点 sudo yum install -y etcd vim etc/etcd/etcd.conf ETCD_NAMEnode1 ETCD_DATA_DIR/var/lib/etcd/node1.etcd ETCD_LISTEN_CLIENT_URLShttp://127.0.0.1:2379,http://192.168.21.135:2379 ETCD_ADVERTISE_CLIENT_URLShttp://127.0.0.1:2379,http://192.168.21.135:2379 ETCD_LISTEN_PEER_URLShttp://192.168.21.135:2380 ETCD_INITIAL_ADVERTISE_PEER_URLShttp://192.168.21.135:2380 ETCD_INITIAL_CLUSTERnode1http://192.168.21.135:2380,node2http://192.168.21.136:2380,node3http://192.168.21.137:2380 ETCD_INITIAL_CLUSTER_TOKENetcd-cluster ETCD_INITIAL_CLUSTER_STATEnew # PG2节点 sudo yum install -y etcd vim /etc/etcd/etcd.conf ETCD_NAMEnode2 ETCD_DATA_DIR/var/lib/etcd/node2.etcd ETCD_LISTEN_CLIENT_URLShttp://127.0.0.1:2379,http://192.168.21.136:2379 ETCD_ADVERTISE_CLIENT_URLShttp://127.0.0.1:2379,http://192.168.21.136:2379 ETCD_LISTEN_PEER_URLShttp://192.168.21.136:2380 ETCD_INITIAL_ADVERTISE_PEER_URLShttp://192.168.21.136:2380 ETCD_INITIAL_CLUSTERnode1http://192.168.21.135:2380,node2http://192.168.21.136:2380,node3http://192.168.21.137:2380 ETCD_INITIAL_CLUSTER_TOKENetcd-cluster ETCD_INITIAL_CLUSTER_STATEnew # PG3节点 sudo yum install -y etcd vim /etc/etcd/etcd.conf ETCD_NAMEnode3 ETCD_DATA_DIR/var/lib/etcd/node3.etcd ETCD_LISTEN_CLIENT_URLShttp://127.0.0.1:2379,http://192.168.21.137:2379 ETCD_ADVERTISE_CLIENT_URLShttp://127.0.0.1:2379,http://192.168.21.137:2379 ETCD_LISTEN_PEER_URLShttp://192.168.21.137:2380 ETCD_INITIAL_ADVERTISE_PEER_URLShttp://192.168.21.137:2380 ETCD_INITIAL_CLUSTERnode1http://192.168.21.135:2380,node2http://192.168.21.136:2380,node3http://192.168.21.137:2380 ETCD_INITIAL_CLUSTER_TOKENetcd-cluster ETCD_INITIAL_CLUSTER_STATEnew3.安装patroni# 这里采用编译安装的方式 # 安装依赖包 yum install -y libffi-devel libuuid libuuid-devel # 安装python tar -xzvf python-3.8.7.tgz cd python-3.8.7 ./configure make -j 7 make -j 7 install rm -rf /usr/bin/python ln -s /usr/local/bin/python3 /usr/bin/pyhon # 处理yum的问题 vim /usr/bin/yum 修改#!/usr/bin/python 为#!/usr/bin/python2 # 安装patroni pip3 install psycopg2-binary pip3 install patroni[etcd]4.创建patroni配置文件# PG1节点 mkdir /etc/patroni vim /etc/patroni/patroni.conf scope: etcd_patroni_pg namespace: /data/ name: pg1 restapi: listen: 192.168.21.135:8008 connect_address: 192.168.21.135:8008 etcd: host: 192.168.21.135:2379 bootstrap: dcs: ttl: 30 loop_wait: 10 retry_timeout: 10 maximum_lag_on_failover: 1048576 master_start_timeout: 300 synchronous_mode: false postgresql: use_pg_rewind: true use_slots: true parameters: listen_address: * port: 5432 max_connections: 2000 hot_standby: on max_locks_per_transactions: 128 max_prepared_transactions: 2000 max_replication_slots: 10 max_wal_sender: 10 max_worker_precesses: 256 track_commit_timestamp: on wal_keep_size: 0 wal_level: replica wal_log_hints: on log_directory: log log_destination: stderr postgresql: listen: 0.0.0.0:5432 connect_address: 192.168.21.135:5432 data_dir: /data/pgdata bin_dir: /opt/pg13/bin pgpass: /home/postgres/.pgpass authentication: replication: username: replica password: replica superuser: username: postgres password: root tags: nofailover: false noloadbalance: false clonefrom: false nosync: false# PG2节点 mkdir /etc/patroni vim /etc/patroni/partoni.conf scope: etcd_patroni_pg namespace: /data/ name: pg2 restapi: listen: 192.168.21.136:8008 connect_address: 192.168.21.136:8008 etcd: host: 192.168.21.136:2379 bootstrap: dcs: ttl: 30 loop_wait: 10 retry_timeout: 10 maximum_lag_on_failover: 1048576 master_start_timeout: 300 synchronous_mode: false postgresql: use_pg_rewind: true use_slots: true parameters: listen_address: * port: 5432 max_connections: 2000 hot_standby: on max_locks_per_transactions: 128 max_prepared_transactions: 2000 max_replication_slots: 10 max_wal_sender: 10 max_worker_precesses: 256 track_commit_timestamp: on wal_keep_size: 0 wal_level: replica wal_log_hints: on log_directory: log log_destination: stderr postgresql: listen: 0.0.0.0:5432 connect_address: 192.168.21.136:5432 data_dir: /data/pgdata bin_dir: /opt/pg13/bin pgpass: /home/postgres/.pgpass authentication: replication: username: replica password: replica superuser: username: postgres password: root tags: nofailover: false noloadbalance: false clonefrom: false nosync: false# PG3节点 mkdir /etc/patroni vim /etc/patroni/patroni.conf scope: etcd_patroni_pg namespace: /data/ name: pg3 restapi: listen: 192.168.21.137:8008 connect_address: 192.168.21.137:8008 etcd: host: 192.168.21.137:2379 bootstrap: dcs: ttl: 30 loop_wait: 10 retry_timeout: 10 maximum_lag_on_failover: 1048576 master_start_timeout: 300 synchronous_mode: false postgresql: use_pg_rewind: true use_slots: true parameters: listen_address: * port: 5432 max_connections: 2000 hot_standby: on max_locks_per_transactions: 128 max_prepared_transactions: 2000 max_replication_slots: 10 max_wal_sender: 10 max_worker_precesses: 256 track_commit_timestamp: on wal_keep_size: 0 wal_level: replica wal_log_hints: on log_directory: log log_destination: stderr postgresql: listen: 0.0.0.0:5432 connect_address: 192.168.21.137:5432 data_dir: /data/pgdata bin_dir: /opt/pg13/bin pgpass: /home/postgres/.pgpass authentication: replication: username: replica password: replica superuser: username: postgres password: root tags: nofailover: false noloadbalance: false clonefrom: false nosync: false5.将patroni加入服务开机启动vim /etc/systemd/system/patroni.service [Unit] Descriptionpatroni - a high-availability PostgreSQL Documentationhttps://patroni.readthedocs.io/en/latest/index.html Aftersyslog.target network.target etcd.target Wantsnetwork-online.target [Service] Typesimple Userpostgres Grouppostgres PermissionsStartOnlytrue ExecStart/usr/local/bin/patroni /etc/patroni/patroni.yml ExecReload/bin/kill -HUP $MAINPID LimitNOFILE65536 KillModeprocess KillSignalSIGINT Restarton-abnormal RestartSec30s TimeoutSec0 [Install] WantedBymulti-user.target6.启动集群启动数据库 Pg_ctl start $PGDATA 启动etcd Sudo systemctl start etcd 启动patroni: Sudo systemctl start patroni检查验证1. 查看集群节点状态patronictl -c /etc/patroni/patroni.yml listRole显示当前节点所属状态TL显示当前集群的时间线Lag in MB:显示当前集群主备延迟大小2.手工触发主备切换选择某一可用的从节点使其成为主节点角色 patronictl -c /etc/patroni/patroni.yml switchover 查看集群状态 patronictl -c /etc/patroni/patroni.yml list原始旧主库自动切换为新主的备库TL时间线变化。切换后旧主库会自动跟随新主库。3.手工停止主节点Patroni再次发生主备切换重新启动patroni组件后集群状态pg2自动跟随新主库pg3。当某一节点与主库不同步或者节点异常运行时可以使用此方法初始化节点信息以重新加入集群。patronictl -c /etc/patroni/patroni.yml reinit pgsql1.选择以下需要添加的节点名称pg22.你确定要重新初始化成员 pg2[y/N]y3.成功为成员pgsql_node3执行初始化4. 手动停止PG主节点实例从上面的过程可以看出停止主库的后patroni组件很快就检测到主库宕机然后尝试将主库重新启动。如果启动不成功则触发主备切换。绑定VIPPatroni集群在应用连接时可以连接多个IP在使用时先判断当前实例是否为MASTER如果是则业务继续进行否则需要尝试其他IP。PATRONI可以将VIP绑定在MASTER节点当主备切换时VIP漂移到新主节点。1. 修改patroni配置文件每个节点都需要scope: etcd_patroni_pg1 namespace: /data/ name: pg1 restapi: listen: 192.168.21.135:8008 connect_address: 192.168.21.135:8008 etcd: host: 192.168.21.135:2379 bootstrap: dcs: ttl: 30 loop_wait: 10 retry_timeout: 10 maximum_lag_on_failover: 1048576 master_start_timeout: 300 synchronous_mode: false postgresql: use_pg_rewind: true use_slots: true parameters: listen_address: * port: 5432 max_connections: 2000 hot_standby: on max_locks_per_transaction: 128 max_prepared_transactions: 2000 max_replication_slots: 10 max_wal_sender: 10 max_worker_precesses: 256 track_commit_timestamp: on wal_keep_size: 0 wal_level: replica wal_log_hints: on log_directory: log log_destination: stderr postgresql: listen: 0.0.0.0:5432 connect_address: 192.168.21.135:5432 data_dir: /data/pgdata bin_dir: /opt/pg13/bin authentication: replication: username: replica password: replica superuser: username: postgres password: root pgpass: /home/postgresql/.pgpass callbacks: on_start: /etc/patroni/patroni_callback.sh on_stop: /etc/patroni/patroni_callback.sh on_role_change: /etc/patroni/patroni_callback.sh tags: nofailover: false noloadbalance: false clonefrom: false nosync: false注意callbacks:on_start: /etc/patroni/patroni_callback.shon_stop: /etc/patroni/patroni_callback.shon_role_change: /etc/patroni/patroni_callback.sh此段为绑定VIP需要的shell脚本。2. shell脚本#!/bin/bash readonly cb_name$1 readonly role$2 readonly scope$3 function usage() { echo Usage: $0 on_start|on_stop|on_role_change role scope; exit 1; } echo this is patroni callback $cb_name $role $scope case $cb_name in on_stop) sudo ip addr del 192.168.21.250/24 dev ens33 label ens33:1 sudo arping -q -A -c 1 -I ens33 192.168.21.250 #sudo iptables -F ;; on_start) ;; on_role_change) if [[ $role master ]]; then sudo ip addr add 192.168.21.250/24 brd 192.168.21.255 dev ens33 label ens33:1 sudo arping -q -A -c 1 -I ens33 192.168.21.250 #sudo iptables -F elif [[ $role slave ]]||[[ $role replica ]]||[[ $role logical ]]; then sudo ip addr del 192.168.21.250/24 dev ens33 label ens33:1 sudo arping -q -A -c 1 -I enp0s8 192.168.21.250 #sudo iptables -F fi ;; *) usage ;; esac总结Patroni用于构建高可用 PostgreSQL 集群的工具它通过自动化的方式来管理主备切换、数据复制和集群健康检查。参考资料乖乖PG高可用咱不用主从流复制了咱试试patroni基于Patronietcd流复制搭建PostgreSQL高可用 - 墨天轮使用Patroni和HAProxy创建高度可用的PostgreSQL集群-腾讯云开发者社区-腾讯云https://www.linode.com/docs/guides/create-a-highly-available-postgresql-cluster-using-patroni-and-haproxy/