Hengshi Documentation

Pre-deployment Preparation

1. Specify namespace, such as hengshi

kubectl create namespace hengshi

2. Replace the $(POD_NAMESPACE) variable in gpdb.yaml with the current namespace, such as hengshi

sed -i 's/$(POD_NAMESPACE)/hengshi/g' gpdb.yaml

3. Modify pvc

• The following services require volume modification:
  • Change storageClassName: csi-rbd to the storageclass of the current cluster
  • Change storage: to the storage size for each service

metadb.yaml
gpdb.yaml
zookeeper.yaml

engine

Deployment (engine)

1. Initialize and start the engine

kubectl -n hengshi apply -f gpdb.yaml
kubectl -n hengshi exec -it master-0 -- /entrypoint.sh -m initsystem
kubectl -n hengshi exec -it master-0 -- /entrypoint.sh -m startsystem

2. Deploy

kubectl -n hengshi apply -f configmap.yaml
kubectl -n hengshi apply -f service.yaml
kubectl -n hengshi apply -f zookeeper.yaml
kubectl -n hengshi apply -f metadb.yaml
kubectl -n hengshi apply -f gpdb.yaml
kubectl -n hengshi apply -f hengshi.yaml
kubectl -n hengshi apply -f ingress.yaml #Deploy ingress as needed

3. Safely stop services (metadb, engine)

kubectl -n hengshi exec -it metadb-0 -- /docker-entrypoint.sh stop metadb single
kubectl -n hengshi exec -it master-0 -- /entrypoint.sh -m stopsystem

Restart (engine)

kubectl -n hengshi exec -it master-0 -- /entrypoint.sh gpstop -r

Scale out (engine)

1. Modify StatefulSet/segment

kubectl -n hengshi edit StatefulSet/segment

• Fill the SEGMENTS field with the appname of all segments after scaling out (for example, from 2 to 4)
• Change replicas: to the number of all segments after scaling out

apiVersion: v1
kind: ConfigMap
metadata:
  name: greenplum
data:
  MASTER: "master-0"
  SEGMENTS: |  #List of 4 segments
    segment-0
    segment-1
    segment-2
    segment-3
...
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: segment
spec:
  replicas: 4 #For example, 4 segments after scaling out

• Then execute kubectl -n hengshi apply -f gpdb.yaml
• Then wait for all new and existing segment pods to become running

1. Write new_host_file (list of new segments, for example, from the original 2 segments (0,1) to 4 segments (0,1,2,3))

kubectl -n hengshi exec -it master-0 /bin/bash
cd /opt/hsdata/ && mkdir expand && cd expand
cat <<EOF > new_host_file
segment-2
segment-3
EOF

3. Perform scale-out operation

kubectl -n hengshi exec -it master-0 /bin/bash
cd /opt/hsdata/expand
psql postgres -c "create database expand"
gpexpand -f new_host_file  -D expand
>y
>0 #This will generate a file named gpexpand_inputfile_yyyymmdd_xxxxxx

gpexpand -i gpexpand_inputfile_yyyymmdd_xxxxxx -D expand

4. Rollback on failure (engine)

kubectl -n hengshi exec -it master-0 /bin/bash
cd /opt/hsdata/expand
gpstart -aR
gpexpand -r -D expand

Data migration for engine

1. Export old engine data

# dump db data
kubectl exec -it $old-gp /bin/bash
source $HS_HOME/engine-cluster
pg_dumpall > /opt/hsdata/engine.back.sql
exit

2. Copy data to the new machine

# cp db data
kubectl cp $old-gp:/opt/hsdata/engine.back.sql engine.back.sql
kubectl cp engine.back.sql $master-0:/opt/hsdata/engine.back.sql

3. Import data into the new environment

# load db data
kubectl exec -it $master-0 /bin/bash
source $HS_HOME/engine-cluster
psql postgres < /opt/hsdata/engine.back.sql
rm /opt/hsdata/engine.back.sql

hengshi

Deploy hengshi service

kubectl apply -f configmap.yaml
kubectl apply -f service.yaml
kubectl apply -f zookeeper.yaml
kubectl apply -f metadb.yaml
kubectl apply -f hengshi.yaml

Configuration for 3.x with gpdb5 (upgrading hengshi from 2.x to 3.x without upgrading gpdb)

• Edit configuration file of master-0

kubectl -n hengshi exec -it metadb-0 -- /bin/bash
cd /opt/hengshi/conf; test -f hengshi-sense-env.sh || cp hengshi-sense-env.sh.sample hengshi-sense-env.sh
cat<<EOF >> hengshi-sense-env.sh
ENGINE_QUERY_USER=hengshi_query
ENGINE_QUERY_PASSWORD=query202020
ENGINE_QUERY_QUEUE=hengshi_query_queue
QUERY_QUEUE_ACTIVE_NUM=10
ENGINE_ETL_USER=hengshi_etl
ENGINE_ETL_PASSWORD=etl202020
ENGINE_ETL_QUEUE=hengshi_etl_queue
ETL_QUEUE_ACTIVE_NUM=4
EOF

• Update gpdb configuration

kubectl -n hengshi get pod
kubectl cp hengshi-sense-xxxxxxxxx-xxxxx:/opt/hengshi/bin/engine.sh engine.31.sh
kubectl cp engine.31.sh master-0:/opt/hengshi/bin/engine.31.sh
kubectl -n hengshi exec -it metadb-0 -- /bin/bash
cd /opt/hengshi/bin
./engine.31.sh config

• Update configmap

set_kv_config() {
    local config_file="$1"
    local param="$2"
    local val="$3"
    # edit param=val if exist or insert new param=val
    grep -E "^\s*${param}\s*:" "${config_file}" > /dev/null \
                || sed -i "$ a ${param}: ${val}" "${config_file}"
}
set_kv_config configmap.yaml ENGINE_QUERY_USER hengshi_query
set_kv_config configmap.yaml ENGINE_QUERY_PASSWORD query202020
set_kv_config configmap.yaml ENGINE_QUERY_QUEUE hengshi_query_queue
set_kv_config configmap.yaml ENGINE_ETL_USER hengshi_etl
set_kv_config configmap.yaml ENGINE_ETL_PASSWORD etl202020
set_kv_config configmap.yaml ENGINE_ETL_QUEUE hengshi_etl_queue

kubectl -n hengshi apply -f configmap.yaml

• Restart hengshi pod

kubectl rollout restart deployment/hengshi-sense