主机名 | CPU | 内存 | 存储 | 备注 |
---|---|---|---|---|
dfq-cloudberrydb-01 | 8c | 8G | 30G SSD | 部署Coordinator |
dfq-cloudberrydb-02 | 8c | 8G | 30G SSD | 部署Segment(Primary+Mirror) |
dfq-cloudberrydb-03 | 8c | 8G | 30G SSD | 部署Segment(Primary+Mirrir) |
3台服务器信息如下:
[root@dfq-cloudberrydb-01 ~]# cat /etc/os-release
NAME="Rocky Linux"
VERSION="8.6 (Green Obsidian)"
ID="rocky"
ID_LIKE="rhel centos fedora"
VERSION_ID="8.6"
PLATFORM_ID="platform:el8"
PRETTY_NAME="Rocky Linux 8.6 (Green Obsidian)"
ANSI_COLOR="0;32"
CPE_NAME="cpe:/o:rocky:rocky:8:GA"
HOME_URL="https://rockylinux.org/"
BUG_REPORT_URL="https://bugs.rockylinux.org/"
ROCKY_SUPPORT_PRODUCT="Rocky Linux"
ROCKY_SUPPORT_PRODUCT_VERSION="8"
REDHAT_SUPPORT_PRODUCT="Rocky Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="8"
下载CloudberryDB源码:
git clone https://github.com/cloudberrydb/cloudberrydb.git
cd cloudberrydb
git checkout 1.5.3
安装依赖项
sudo dnf groupinstall -y "Development Tools"
sudo dnf install -y epel-release
sudo dnf install -y apr-devel bison bzip2-devel cmake3 flex gcc gcc-c++ krb5-devel libcurl-devel libevent-devel libkadm5 libxml2-devel libzstd-devel openssl-devel perl-ExtUtils-Embed python3-devel python3-pip readline-devel xerces-c-devel zlib-devel
~/cloudberrydb/deploy/build/README.Rhel-Rocky.bash
pip3.9 install -i https://pypi.tuna.tsinghua.edu.cn/simple -r ~/cloudberrydb/python-dependencies.txt
编译CloudberryDB,设置安装路径为/usr/local/cloudberrydb
cd cloudberrydb
./configure --with-perl --with-python --with-libxml --with-gssapi --prefix=/usr/local/cloudberrydb
make -j8
make install
下面的步骤需要在三台服务器都一次执行一次,使用root用户来执行命令
解压编译好的CloudberryDB安装包到/usr/local目录下,然后使用下面的命令分别修改每台服务器的主机名:
hostnamectl set-hostname dfq-cloudberrydb-0x
禁用SELINUX和防火墙
setenforce 0
systemctl stop firewalld
systemctl disable firewalld
修改/etc/hosts
文件,包含CloudberryDB所有主机别名与其网络IP地址的映射:
10.197.165.181 dfq-cloudberrydb-01
10.197.165.182 dfq-cloudberrydb-02
10.197.165.183 dfq-cloudberrydb-03
配置内核参数,在/etc/sysctl.conf
文件中添加下面的参数:
# kernel.shmall = _PHYS_PAGES / 2
kernel.shmall = 197951838
# kernel.shmmax = kernel.shmall * PAGE_SIZE
kernel.shmmax = 810810728448
kernel.shmmni = 4096
vm.overcommit_memory = 2
vm.overcommit_ratio = 95
net.ipv4.ip_local_port_range = 10000 65535
kernel.sem = 250 2048000 200 8192
kernel.sysrq = 1
kernel.core_uses_pid = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.msgmni = 2048
net.ipv4.tcp_syncookies = 1
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.tcp_max_syn_backlog = 4096
net.ipv4.conf.all.arp_filter = 1
net.ipv4.ipfrag_high_thresh = 41943040
net.ipv4.ipfrag_low_thresh = 31457280
net.ipv4.ipfrag_time = 60
net.core.netdev_max_backlog = 10000
net.core.rmem_max = 2097152
net.core.wmem_max = 2097152
vm.swappiness = 10
vm.zone_reclaim_mode = 0
vm.dirty_expire_centisecs = 500
vm.dirty_writeback_centisecs = 100
vm.dirty_background_ratio = 0
vm.dirty_ratio = 0
vm.dirty_background_bytes = 1610612736
vm.dirty_bytes = 4294967296
kernel.core_pattern=/var/core/core.%h.%t
执行下面的命令使参数生效:
sysctl -p
资源限制设置,在/etc/security/limits.conf
文件中添加下面的参数,对软硬件资源用量进行限制:
*soft nofile 524288
*hard nofile 524288
*soft nproc 131072
*hard nproc 131072
* soft core unlimited
创建gpadmin管理用户:
groupadd -g 520 gpadmin
useradd -g 520 -u 520 -m -d /home/gpadmin/ -s /bin/bash gpadmin
passwd gpadmin
编译好的CloudberryDB软件包在/usr/local/cloudberrydb
路径下,为gpadmin用户授予安装目录的权限:
chown -R gpadmin:gpadmin /usr/local/cloudberrydb
在所有的节点安装依赖:
sudo dnf install -y apr-devel bison bzip2-devel cmake3 epel-release flex gcc gcc-c++ krb5-devel libcurl-devel libevent-devel libkadm5 libxml2-devel libzstd-devel openssl-devel python39 python39-devel python39-psutil python3-pip perl-ExtUtils-MakeMaker.noarch perl-ExtUtils-Embed.noarch readline-devel rsync xerces-c-devel zlib-devel python3-psutil python3-pyyaml python3-psycopg2
sudo yum --enablerepo=powertools install -y libyaml-devel
su - gpadmin
pip3.9 install -i https://pypi.tuna.tsinghua.edu.cn/simple psutil==5.7.0 pygresql==5.2 pyyaml==5.3.1
sudo rm -rf /usr/bin/python3
sudo ln -s /usr/bin/python3.9 /usr/bin/python3
配置节点间的互信,切换到gpadmin用户,后续的操作均使用gpadmin用户进行:
su - gpadmin
创建节点配置信息,在/home/gpadmin/all_hosts
里填入下面的信息:
dfq-cloudberrydb-01
dfq-cloudberrydb-02
dfq-cloudberrydb-03
在/home/gpadmin/seg_hosts
里填入segment节点的信息:
dfq-cloudberrydb-02
dfq-cloudberrydb-03
在每台主机上执行下面的命令生成SSH密钥:
ssh-keygen
在每台主机上执行下面的命令来配置免密:
ssh-copy-id dfq-cloudberrydb-01
ssh-copy-id dfq-cloudberrydb-02
ssh-copy-id dfq-cloudberrydb-03
在Coordinator节点使用gpadmin用户来操作
在所有的节点执行:
在~/.bashrc
文件中添加下面的命令:
source /usr/local/cloudberrydb/greenplum_path.sh
执行下面的命令使命令生效:
source ~/.bashrc
在Coordinator节点使用下面的命令来为Segment节点创建数据目录和mirror目录:
gpssh -f seg_hosts
mkdir -p /data/primary
mkdir -p /data/mirror
在Coordinator节点上创建数据目录:
mkdir -p /data/coordinator
在Coordinator节点的主机上往~/.bashrc
文件中添加下面的命令:
export COORDINATOR_DATA_DIRECTORY=/data/coordinator/gpseg-1
执行下面的命令使文件生效:
source ~/.bashrc
在Coordinator节点所在主机上,复制模板文件:
cp $GPHOME/docs/cli_help/gpconfigs/gpinitsystem_config .
修改gpinitsystem_config
文件:
DATA_DIRECTORY
修改为Segment计算节点的数据目录:/data/primary
COORDINATOR_HOSTNAME
修改为dfq-cloudberrydb-01
COORDINATOR_DIRECTORY
修改为Coordinator节点数据目录:/data/coordinator
MIRROR_DATA_DIRECTORY
修改为Mirror的数据目录:/data/mirror
在gpadmin用户下,使用下面的命令初始化CloudberryDB:
gpinitsystem -c gpinitsystem_config -h /home/gpadmin/seg_hosts
初始化完成后,将下面的环境变量配置到~/.bashrc
里
export PGDATABASE=postgres
export PGHOST=127.0.0.1
export PGUSER=gpadmin
export PGPORT=5432
重新加载~/.bashrc
source ~/.bashrc
使用psql登录数据库
[gpadmin@dfq-cloudberrydb-01 ~]$ psql
psql (14.4, server 14.4)
Type "help" for help.
postgres=# SELECT version();
version
-----------------------------------------------------------------------------------
PostgreSQL 14.4 (Cloudberry Database 1.5.3+dev.57.ge1d276f80e build dev) on x86_64-pc-li
nux-gnu, compiled by gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-22), 64-bit compiled on Jun
19 2024 09:20:42
(1 row)