Glusterfs常用操作整理

集群节点
扩展集群
1,必须做hosts域名解析
其实通过IP地址也能做集群,但是不建议这种方式.

[root@glusterfs01 pv1]# tail -n3 /etc/hosts
10.0.0.150 glusterfs01
10.0.0.151 glusterfs02
10.0.0.152 glusterfs03
10.0.0.153 glusterfs04

2, 添加节点到集群中,在当前所有集群节点中都需要执行

[root@glusterfs01 ~]# gluster peer probe glusterfs04

增加一份复制

gluster volume  add-brick gv0  replica 4 glusterfs04:/data/pv1
[root@glusterfs01 ~]# gluster volume info

Volume Name: gv0
Type: Replicate
Volume ID: 0bc7d90c-1358-4f93-9a70-9eb092273aaa
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 4 = 4
Transport-type: tcp
Bricks:
Brick1: glusterfs01:/data/pv1
Brick2: glusterfs02:/data/pv1
Brick3: glusterfs03:/data/pv1
Brick4: glusterfs04:/data/pv1
Options Reconfigured:
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off

PS:客户端是并行向所有节点写数据。

3,查看对等状态

gluster peer status

查看集群节点信息

gluster pool list

删除节点
从集群中删除节点

gluster peer detach glusterfs04

卷
创建卷

distribute volume分布式卷(默认)

命令:

gluster volume create NEW-VOLNAME [transport [tcp | rdma | tcp,rdma]] NEW-BRICK...
示例:

gluster volume create test-volume  transport tcp glutserfs01:/data/pv1 gluster02:/data/pv1
Creation of test-volume has been successful
Please start the volume to access data

Striped Glusterfs Volume 条带卷
命令:

gluster volume create NEW-VOLNAME [stripe COUNT] [transport [tcp | dma | tcp,rdma]] NEW-BRICK...
示例:

gluster volume create test-volume stripe 2 transport tcp glutserfs01:/data/pv1 gluster02:/data/pv1
Creation of test-volume has been successful
Please start the volume to access data

Replicated Glusterfs Volume 复制卷(最少两个节点)
命令:

gluster volume create NEW-VOLNAME [replica COUNT] [transport [tcp | rdma | tcp,rdma]] NEW-BRICK...
示例:

gluster volume create test-volume replica 2 glutserfs01:/data/pv1 gluster02:/data/pv1
Creation of test-volume has been successful
Please start the volume to access data

Distributed Replicated Glusterfs Volume 分布式复制卷
命令:

gluster volume create NEW-VOLNAME [replica COUNT] [transport [tcp | rdma | tcp，rdma]] NEW-BRICK ......

示例:
两个节点组成分布式,镜像3份

gluster volume create test-volume replica 3 transport tcp glutserfs01:/data/pv1 gluster02:/data/pv1 glutserfs03:/data/pv1 gluster04:/data/pv1 glutserfs05:/data/pv1 glusterfs06:/data/pv1
Creation of test-volume has been successful
Please start the volume to access data

glusterfs01和glusterfs02 组成分布式
glusterfs03和glusterfs04 组成分布式
glusterfs05和glusterfs06 组成分布式
然后在组成镜像

三个节点分布式镜像2份

gluster volume create test-volume replica 2 transport tcp glutserfs01:/data/pv1 gluster02:/data/pv1 glutserfs03:/data/pv1 gluster04:/data/pv1 glutserfs05:/data/pv1 glusterfs06:/data/pv1
Creation of test-volume has been successful
Please start the volume to access data

glusterfs01 和glusterfs02 和glusterfs03 组成分布式
glusterfs04 和glusterfs05 和glusterfs06 组成分布式
然后在组成镜像

Distributed Striped Glusterfs Volume 分布式条带卷
先组条带在组分布
命令:

gluster volume create NEW-VOLNAME [stripe COUNT] [transport [tcp | rdma | tcp,rdma]] NEW-BRICK...

示例:

gluster volume create test-volume stripe 4 transport tcp
 glutserfs01:/data/pv1 glutserfs02:/data/pv1 glutserfs03:/data/pv1 glutserfs04:/data/pv1 glutserfs05:/data/pv1 glutserfs06:/data/pv1 glutserfs07:/data/pv1 glutserfs08:/data/pv1
Creation of test-volume has been successful
Please start the volume to access data.

前4个组成一个条带,然后在组成分布式,组成多少分布式和条带式根据条带数量完成的

性能分析
开启性能分析

gluster volume profile VOLUME_NAME start

关闭性能分析

gluster volume profile VOLUME_NAME stop

显示I/O信息:

[root@glusterfs01 pv1]# gluster volume profile gv0 info
Brick: glusterfs01:/data/pv1
----------------------------
Cumulative Stats:
 %-latency   Avg-latency   Min-Latency   Max-Latency   No. of calls         Fop
 ---------   -----------   -----------   -----------   ------------        ----
      0.00       0.00 us       0.00 us       0.00 us             24      FORGET
      0.00       0.00 us       0.00 us       0.00 us             36     RELEASE
      0.00       0.00 us       0.00 us       0.00 us             80  RELEASEDIR

    Duration: 3338 seconds
   Data Read: 0 bytes
Data Written: 0 bytes

Interval 0 Stats:
 %-latency   Avg-latency   Min-Latency   Max-Latency   No. of calls         Fop
 ---------   -----------   -----------   -----------   ------------        ----
      0.00       0.00 us       0.00 us       0.00 us             24      FORGET
      0.00       0.00 us       0.00 us       0.00 us             36     RELEASE
      0.00       0.00 us       0.00 us       0.00 us             80  RELEASEDIR

    Duration: 3338 seconds
   Data Read: 0 bytes
Data Written: 0 bytes

Brick: glusterfs03:/data/pv1
----------------------------
Cumulative Stats:
 %-latency   Avg-latency   Min-Latency   Max-Latency   No. of calls         Fop
 ---------   -----------   -----------   -----------   ------------        ----
      0.00       0.00 us       0.00 us       0.00 us             24      FORGET
      0.00       0.00 us       0.00 us       0.00 us             36     RELEASE
      0.00       0.00 us       0.00 us       0.00 us             77  RELEASEDIR

    Duration: 3334 seconds
   Data Read: 0 bytes
Data Written: 0 bytes

Interval 0 Stats:
 %-latency   Avg-latency   Min-Latency   Max-Latency   No. of calls         Fop
 ---------   -----------   -----------   -----------   ------------        ----
      0.00       0.00 us       0.00 us       0.00 us             24      FORGET
      0.00       0.00 us       0.00 us       0.00 us             36     RELEASE
      0.00       0.00 us       0.00 us       0.00 us             77  RELEASEDIR

    Duration: 3334 seconds
   Data Read: 0 bytes
Data Written: 0 bytes

Brick: glusterfs02:/data/pv1
----------------------------
Cumulative Stats:
 %-latency   Avg-latency   Min-Latency   Max-Latency   No. of calls         Fop
 ---------   -----------   -----------   -----------   ------------        ----
      0.00       0.00 us       0.00 us       0.00 us             24      FORGET
      0.00       0.00 us       0.00 us       0.00 us             36     RELEASE
      0.00       0.00 us       0.00 us       0.00 us             80  RELEASEDIR

    Duration: 3336 seconds
   Data Read: 0 bytes
Data Written: 0 bytes

Interval 0 Stats:
 %-latency   Avg-latency   Min-Latency   Max-Latency   No. of calls         Fop
 ---------   -----------   -----------   -----------   ------------        ----
      0.00       0.00 us       0.00 us       0.00 us             24      FORGET
      0.00       0.00 us       0.00 us       0.00 us             36     RELEASE
      0.00       0.00 us       0.00 us       0.00 us             80  RELEASEDIR

    Duration: 3336 seconds
   Data Read: 0 bytes
Data Written: 0 bytes

Brick: glusterfs04:/data/pv1
----------------------------
Cumulative Stats:
 %-latency   Avg-latency   Min-Latency   Max-Latency   No. of calls         Fop
 ---------   -----------   -----------   -----------   ------------        ----
      0.00       0.00 us       0.00 us       0.00 us             21      FORGET
      0.00       0.00 us       0.00 us       0.00 us             34     RELEASE
      0.00       0.00 us       0.00 us       0.00 us             25  RELEASEDIR

    Duration: 2764 seconds
   Data Read: 0 bytes
Data Written: 0 bytes

Interval 0 Stats:
 %-latency   Avg-latency   Min-Latency   Max-Latency   No. of calls         Fop
 ---------   -----------   -----------   -----------   ------------        ----
      0.00       0.00 us       0.00 us       0.00 us             21      FORGET
      0.00       0.00 us       0.00 us       0.00 us             34     RELEASE
      0.00       0.00 us       0.00 us       0.00 us             25  RELEASEDIR

    Duration: 2764 seconds
   Data Read: 0 bytes
Data Written: 0 bytes