概述

greenplum扩容segment可以分为2类:

  • 单机上的seg增加,我们称之为“纵向扩展”
  • 单机不变,通过增加机器,我们称之为“横向扩展”

下面详细介绍下这两种扩容方式的具体步骤和方法

gpexpand命令

首先我们来学习一下 gpexpand 命令,基本使用方法如下:

  1. gpexpand
  2. [-f <hosts_file>]
  3. | -i <input_file> [-B <batch_size>] [-V]
  4. | {-d <hh:mm:ss> | -e '<YYYY-MM-DD hh:mm:ss>'}
  5. [-analyze] [-n <parallel_processes>]
  6. | --rollback
  7. | --clean
  8. [-D <database_name>][--verbose] [--silent]

命令参数:

  1. -B <batch_size>
  2. 并发数可以指定同时运行ssh命令的数量,默认时16,参数范围是1-128
  3. 一般的默认值即可
  4. -D <database_name>
  5. 指定要在其中创建扩展模式的数据库和表。如果未提供此选项,则为使用环境变量PGDATABASE。数据库模板不能使用template1template0
  6. -i | --input <input_file>
  7. 指定扩展配置文件的名称
  8. <hostname>:<address>:<port>:<fselocation>:<dbid>:<content>:<preferred_role>:<replication_port>
  9. 主机名:地址:端口:segment data目录,注意是全路径:dbid号:角色
  10. -n <parallel_processes>
  11. 同时进行重分布的表的个数,有效的参数为1-16,每一个表重分布必须要有两个数据库连接,设置之前,要确保最大连接数max_connections这个参数可以满足这个值
  12. -S | --simple_progress
  13. 简单模式,不会显示提示
  14. -t | --tardir <directory>
  15. 指定要放置tar文件的段主机上的临时目录。
  16. -v | --verbose
  17. 调试模式,打印出详细的执行计划
  18. -V | --novacuum
  19. 创建schema副本时,不对数据字典进行vacuum

纵向拓展

原Greenplum集群状态

原集群的一个master节点和2个数据节点,每一个数据节点上有4个primary计算节点(为了显示清晰,就没有增加mirror节点了)具体信息如下:

  1. postgres=# SELECT * from gp_segment_configuration ;
  2. dbid | content | role | preferred_role | mode | status | port | hostname | address | replication_port | san_mounts
  3. ------+---------+------+----------------+------+--------+-------+----------+---------+------------------+------------
  4. 1 | -1 | p | p | s | u | 5432 | gw_mdw1 | gw_mdw1 | |
  5. 2 | 0 | p | p | s | u | 40000 | gw_sdw1 | gw_sdw1 | |
  6. 6 | 4 | p | p | s | u | 40000 | gw_sdw2 | gw_sdw2 | |
  7. 3 | 1 | p | p | s | u | 40001 | gw_sdw1 | gw_sdw1 | |
  8. 7 | 5 | p | p | s | u | 40001 | gw_sdw2 | gw_sdw2 | |
  9. 4 | 2 | p | p | s | u | 40002 | gw_sdw1 | gw_sdw1 | |
  10. 8 | 6 | p | p | s | u | 40002 | gw_sdw2 | gw_sdw2 | |
  11. 5 | 3 | p | p | s | u | 40003 | gw_sdw1 | gw_sdw1 | |
  12. 9 | 7 | p | p | s | u | 40003 | gw_sdw2 | gw_sdw2 | |
  13. (9 rows)

现在准备通过在每台机器上再增加1个节点,来扩容segment,简单的来说就是纵向扩容

生成参数文件

指定一个数据库,在此数据库下生成schema

再创建一个seg_hosts的文件,即所有数据节点的主机名

  1. [gpadmin@gw_mdw1 ~]$ cat seg_hosts
  2. gw_sdw1
  3. gw_sdw2

执行命令,生成参数文件

  1. [gpadmin@gw_mdw1 ~]$ gpexpand -f seg_hosts -D test
  2. 20190327:23:18:01:007122 gpexpand:gw_mdw1:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 4.3.1.0 build 6'
  3. 20190327:23:18:01:007122 gpexpand:gw_mdw1:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 8.2.15 (Greenplum Database 4.3.1.0 build 6) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.4.2 compiled on Jun 11 2014 17:23:40'
  4. 20190327:23:18:01:007122 gpexpand:gw_mdw1:gpadmin-[INFO]:-Querying gpexpand schema for current expansion state
  5. System Expansion is used to add segments to an existing GPDB array.
  6. gpexpand did not detect a System Expansion that is in progress.
  7. Before initiating a System Expansion, you need to provision and burn-in
  8. the new hardware. Please be sure to run gpcheckperf/gpcheckos to make
  9. sure the new hardware is working properly.
  10. Please refer to the Admin Guide for more information.
  11. Would you like to initiate a new System Expansion Yy|Nn (default=N):
  12. > y <------确认添加
  13. How many new primary segments per host do you want to add? (default=0):
  14. > 1 <------每台机器上添加1个计算节点
  15. Enter new primary data directory 1:
  16. > /data/primary <------增加的计算节点存放的目录
  17. Generating configuration file...
  18. 20190327:23:19:03:007122 gpexpand:gw_mdw1:gpadmin-[INFO]:-Generating input file...
  19. Input configuration files were written to 'gpexpand_inputfile_20190327_231903' and 'None'.
  20. Please review the file and make sure that it is correct then re-run
  21. with: gpexpand -i gpexpand_inputfile_20190327_231903 -D test
  22. 20190327:23:19:03:007122 gpexpand:gw_mdw1:gpadmin-[INFO]:-Exiting...

参数文件的内容如下:

  1. [gpadmin@gw_mdw1 ~]$ cat gpexpand_inputfile_20190327_231903
  2. gw_sdw1:gw_sdw1:40004:/data/primary/gpseg8:10:8:p
  3. gw_sdw2:gw_sdw2:40004:/data/primary/gpseg9:11:9:p

如果存放的目录有所改变,可以手动去修改此文件,将该计算节点存放在自己想要放在的位置

利用参数文件执行拓展命令

  1. [gpadmin@gw_mdw1 ~]$ gpexpand -i gpexpand_inputfile_20190327_231903 -D test
  2. 20190327:23:20:19:007227 gpexpand:gw_mdw1:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 4.3.1.0 build 6'
  3. 20190327:23:20:19:007227 gpexpand:gw_mdw1:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 8.2.15 (Greenplum Database 4.3.1.0 build 6) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.4.2 compiled on Jun 11 2014 17:23:40'
  4. .......(省略信息)
  5. 20190327:23:22:19:007227 gpexpand:gw_mdw1:gpadmin-[INFO]:-************************************************
  6. 20190327:23:22:19:007227 gpexpand:gw_mdw1:gpadmin-[INFO]:-Initialization of the system expansion complete.
  7. 20190327:23:22:19:007227 gpexpand:gw_mdw1:gpadmin-[INFO]:-To begin table expansion onto the new segments
  8. 20190327:23:22:19:007227 gpexpand:gw_mdw1:gpadmin-[INFO]:-rerun gpexpand
  9. 20190327:23:22:19:007227 gpexpand:gw_mdw1:gpadmin-[INFO]:-************************************************
  10. 20190327:23:22:19:007227 gpexpand:gw_mdw1:gpadmin-[INFO]:-Exiting...

查看新添加的状态

此时去数据库里查看相关节点信息,可以看到已经增加了两个节点了。

  1. postgres=# SELECT * from gp_segment_configuration ;
  2. dbid | content | role | preferred_role | mode | status | port | hostname | address | replication_port | san_mounts
  3. ------+---------+------+----------------+------+--------+-------+----------+---------+------------------+------------
  4. 1 | -1 | p | p | s | u | 5432 | gw_mdw1 | gw_mdw1 | |
  5. 2 | 0 | p | p | s | u | 40000 | gw_sdw1 | gw_sdw1 | |
  6. 6 | 4 | p | p | s | u | 40000 | gw_sdw2 | gw_sdw2 | |
  7. 3 | 1 | p | p | s | u | 40001 | gw_sdw1 | gw_sdw1 | |
  8. 7 | 5 | p | p | s | u | 40001 | gw_sdw2 | gw_sdw2 | |
  9. 4 | 2 | p | p | s | u | 40002 | gw_sdw1 | gw_sdw1 | |
  10. 8 | 6 | p | p | s | u | 40002 | gw_sdw2 | gw_sdw2 | |
  11. 5 | 3 | p | p | s | u | 40003 | gw_sdw1 | gw_sdw1 | |
  12. 9 | 7 | p | p | s | u | 40003 | gw_sdw2 | gw_sdw2 | |
  13. 10 | 8 | p | p | s | u | 40004 | gw_sdw1 | gw_sdw1 | |
  14. 11 | 9 | p | p | s | u | 40004 | gw_sdw2 | gw_sdw2 | |
  15. (11 rows)

数据重分布

执行重分布命令将数据重分布

  1. [gpadmin@gw_mdw1 ~]$ gpexpand -a -d 1:00:00 -D test

此时还没有完成,去指定的数据库下面查看重分布状态

  1. test=# select * from gpexpand.status;
  2. status | updated
  3. ------------+----------------------------
  4. SETUP | 2019-03-27 23:21:49.299565
  5. SETUP DONE | 2019-03-27 23:21:53.737479
  6. (2 rows)

还有多少个表需要重分布,这里我的已经同步完成,因为是测试环境没有什么数据量

  1. test=# select * from gpexpand.expansion_progress ;
  2. name | value
  3. ------------------------------+-------
  4. Estimated Expansion Rate |
  5. Estimated Time to Completion |
  6. (2 rows)

完成重分布之后,将schema删除

  1. [gpadmin@gw_mdw1 ~]$ gpexpand -c -D test

横向拓展

这次我们不仅在每台机器上添加1个计算节点,还添加一个数据节点(也就是一个新机器)

生成配置文件

这次我们在主机的名单中再增加一个新的机器gw_sdw3,此机器是新添加进来的一个节点。只是安装了软件,没有任何数据。

  1. [gpadmin@gw_mdw1 ~]$ cat seg_hosts
  2. gw_sdw1
  3. gw_sdw2
  4. gw_sdw3

使用同样的方法执行,每一节点也选择增加1个计算节点

  1. [gpadmin@gw_mdw1 ~]$ gpexpand -f seg_hosts -D test

生成文件如下:
可以看到,虽然只是添加一个,greenplum查看到gw_sdw3上并没有一个节点,就自动为其一次添加6个,是它与其他两个机器数量一致

  1. [gpadmin@gw_mdw1 ~]$ cat gpexpand_inputfile_20190328_014748
  2. gw_sdw3:gw_sdw3:40000:/data/primary/gpseg10:12:10:p
  3. gw_sdw3:gw_sdw3:40001:/data/primary/gpseg11:13:11:p
  4. gw_sdw3:gw_sdw3:40002:/data/primary/gpseg12:14:12:p
  5. gw_sdw3:gw_sdw3:40003:/data/primary/gpseg13:15:13:p
  6. gw_sdw3:gw_sdw3:40004:/data/primary/gpseg14:16:14:p
  7. gw_sdw1:gw_sdw1:40005:/data/primary/gpseg15:17:15:p
  8. gw_sdw2:gw_sdw2:40005:/data/primary/gpseg16:18:16:p
  9. gw_sdw3:gw_sdw3:40005:/data/primary/gpseg17:19:17:p

执行扩容命令

执行扩容命令,和上面一样

  1. [gpadmin@gw_mdw1 ~]$ gpexpand -i gpexpand_inputfile_20190328_014748 -D test
  2. 20190328:23:50:44:001498 gpexpand:gw_mdw1:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 4.3.1.0 build 6'
  3. 20190328:23:50:44:001498 gpexpand:gw_mdw1:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 8.2.15 (Greenplum Database 4.3.1.0 build 6) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.4.2 compiled on Jun 11 2014 17:23:40'
  4. ....(忽略信息)
  5. 20190328:23:54:59:001498 gpexpand:gw_mdw1:gpadmin-[INFO]:-Starting Greenplum Database
  6. 20190328:23:55:32:001498 gpexpand:gw_mdw1:gpadmin-[INFO]:-************************************************
  7. 20190328:23:55:32:001498 gpexpand:gw_mdw1:gpadmin-[INFO]:-Initialization of the system expansion complete.
  8. 20190328:23:55:32:001498 gpexpand:gw_mdw1:gpadmin-[INFO]:-To begin table expansion onto the new segments
  9. 20190328:23:55:32:001498 gpexpand:gw_mdw1:gpadmin-[INFO]:-rerun gpexpand
  10. 20190328:23:55:32:001498 gpexpand:gw_mdw1:gpadmin-[INFO]:-************************************************
  11. 20190328:23:55:32:001498 gpexpand:gw_mdw1:gpadmin-[INFO]:-Exiting...

查看数据库的节点状态,发现已经增加了8个节点,其中2个是纵向添加,6个是横向添加

  1. postgres=# SELECT * from gp_segment_configuration ;
  2. dbid | content | role | preferred_role | mode | status | port | hostname | address | replication_port | san_mounts
  3. ------+---------+------+----------------+------+--------+-------+----------+---------+------------------+------------
  4. 1 | -1 | p | p | s | u | 5432 | gw_mdw1 | gw_mdw1 | |
  5. 2 | 0 | p | p | s | u | 40000 | gw_sdw1 | gw_sdw1 | |
  6. 6 | 4 | p | p | s | u | 40000 | gw_sdw2 | gw_sdw2 | |
  7. 3 | 1 | p | p | s | u | 40001 | gw_sdw1 | gw_sdw1 | |
  8. 7 | 5 | p | p | s | u | 40001 | gw_sdw2 | gw_sdw2 | |
  9. 4 | 2 | p | p | s | u | 40002 | gw_sdw1 | gw_sdw1 | |
  10. 8 | 6 | p | p | s | u | 40002 | gw_sdw2 | gw_sdw2 | |
  11. 5 | 3 | p | p | s | u | 40003 | gw_sdw1 | gw_sdw1 | |
  12. 9 | 7 | p | p | s | u | 40003 | gw_sdw2 | gw_sdw2 | |
  13. 10 | 8 | p | p | s | u | 40004 | gw_sdw1 | gw_sdw1 | |
  14. 12 | 10 | p | p | s | u | 40000 | gw_sdw3 | gw_sdw3 | |
  15. 11 | 9 | p | p | s | u | 40004 | gw_sdw2 | gw_sdw2 | |
  16. 13 | 11 | p | p | s | u | 40001 | gw_sdw3 | gw_sdw3 | |
  17. 14 | 12 | p | p | s | u | 40002 | gw_sdw3 | gw_sdw3 | |
  18. 15 | 13 | p | p | s | u | 40003 | gw_sdw3 | gw_sdw3 | |
  19. 16 | 14 | p | p | s | u | 40004 | gw_sdw3 | gw_sdw3 | |
  20. 17 | 15 | p | p | s | u | 40005 | gw_sdw1 | gw_sdw1 | |
  21. 18 | 16 | p | p | s | u | 40005 | gw_sdw2 | gw_sdw2 | |
  22. 19 | 17 | p | p | s | u | 40005 | gw_sdw3 | gw_sdw3 | |
  23. (19 rows)

重分布表

执行表的重分布命令

  1. [gpadmin@gw_mdw1 ~]$ gpexpand -a -d 1:00:00 -D test
  2. 20190329:00:00:56:002723 gpexpand:gw_mdw1:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 4.3.1.0 build 6'
  3. 20190329:00:00:56:002723 gpexpand:gw_mdw1:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 8.2.15 (Greenplum Database 4.3.1.0 build 6) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.4.2 compiled on Jun 11 2014 17:23:40'
  4. 20190329:00:00:56:002723 gpexpand:gw_mdw1:gpadmin-[INFO]:-Querying gpexpand schema for current expansion state
  5. 20190329:00:01:03:002723 gpexpand:gw_mdw1:gpadmin-[INFO]:-EXPANSION COMPLETED SUCCESSFULLY
  6. 20190329:00:01:03:002723 gpexpand:gw_mdw1:gpadmin-[INFO]:-Exiting...

去指定数据库下查看,重分布状态

  1. test=# select * from gpexpand.status;
  2. status | updated
  3. --------------------+----------------------------
  4. SETUP | 2019-03-28 23:54:39.325544
  5. SETUP DONE | 2019-03-28 23:54:48.213527
  6. EXPANSION STARTED | 2019-03-29 00:00:58.467156
  7. EXPANSION COMPLETE | 2019-03-29 00:00:58.979106
  8. (4 rows)

完成重分布之后,将schema删除

  1. [gpadmin@gw_mdw1 ~]$ gpexpand -c -D test

至此,两种方式的扩容测试都演示完毕

0 评论  
添加一条新评论