Table of Contents generated with DocToc

Installation Instructions

Machine Configuration

  • Minimum Configuration

    • Memory 24G
    • CPU 8 cores or more (2-4 people concurrently)
    • Disk 50G
  • Recommended Configuration

    • Memory 32G
    • CPU 16 cores or more (6-12 people concurrently)
    • Disk 500G

Environment Dependencies

  1. Centos 7.5 (64-bit)
  2. OpenJDK 1.8
  3. Python version 2.7 or above
  4. Do not install using root, it is recommended to create a user to install/run this product (hengshi user as an example below)

Prepare Files

  • Download the installation package hengshi-sense-[version]-*.zip (Please consult technical support for the download address)
  • Decompress to generate the hengshi-[version] directory (taking hengshi-sense-[version].zip as an example)
mkdir hengshi-sense-[version] && unzip -qod hengshi-sense-[version]/ hengshi-sense-[version].zip

Initial Installation

Using cloud service PostgreSQL to replace Hengshi metadb

Standalone installation process

Cluster installation process

Software Upgrade

Standalone upgrade process

Cluster upgrade process

Engine Configuration

Post-installation configuration

Start/Stop/Check

Standalone/Cluster service operations

Backup and Restore Data

Backup/Restore data

Containerized Deployment

kubernetes

Configure Autostart (Optional)

  • First, run the following command to configure the autostart file (user must have sudo permissions):
$ /opt/hengshi/bin/hengshi-sense-bin init-os systemd
  • Then, use the following command to enable/disable startup:
$ sudo systemctl enable hengshi

$ sudo systemctl disable hengshi
  • In addition, you can use the following commands to control Hengshi:
sudo systemctl stop hengshi

sudo systemctl start hengshi

sudo systemctl status hengshi

Query Engine Upgrade

GP5 -> GP6

Engine Configuration for High Availability

High availability engine configuration

Engine Scaling

Engine expansion

Configuration File

Configuration File Description

There are two configuration files: (they do not exist by default)

  • conf/hengshi-sense-env.sh (all configuration items should be modified in this file except the segment host list for the acceleration engine)
  • conf/engine-segment-hosts (only used to configure the segment list for the acceleration engine)

Note:

  • If conf/hengshi-sense-env.sh does not exist, you can generate this file by copying the conf/hengshi-sense-env.sh.sample template.
  • If it is an already installed system, the new configuration takes effect by restarting the hengshi service.
Acceleration Engine Configuration Method

This configuration is only necessary when installing the acceleration engine.

  • master
    master-engine management module, only one machine, its ip or hostname is configured through the HS_ENGINE_HOST in the conf/hengshi-sense-env.sh configuration file
  • segment
    segment-engine data query module, can be on multiple machines, its ip or hostname is configured through the conf/engine-segment-hosts configuration file, each line is an ip or hostname of a segment. By default, the system will generate configuration containing only localhost
Data Directory Configuration Suggestions
  • For initial installation, the program and system data need at least 3G of space
  • If there are many uploaded files that take up a lot of space, it is recommended to configure the HS_HENGSHI_DATA directory to a large disk. System data, uploaded data, and acceleration engine data are all stored in this directory
  • If you need the acceleration engine to speed up a large amount of data, it is recommended to configure the HS_ENGINE_HOME directory to a large disk. After configuration, the engine acceleration data will be independent of HS_HENGSHI_DATA and stored in this directory

Configuration Variable Reference

Without the configuration file the system will run with default parameters

All configuration items need to be written into the conf/hengshi-sense-env.sh file

  • Table-1
Parameter Purpose
JAVA_HOME Set the runtime Java environment directory (needs to be set to a Java 1.8 environment)
HS_HENGSHI_PORT Change Hengshi WEB service port, default 8080 (BI prohibits root startup, below 1024 port please configure proxy forwarding, such as Nginx, etc.)
HS_HENGSHI_DATA The root directory where hengshi stores all data, defaults to the installation directory, you can configure this variable when there is a large partition specifically for data storage
HS_PG_HOST Host of the internal database program, if not using the built-in one, such as RDS, can be set through this variable
HS_PG_PORT The listening port of the internal database program (54320), if it conflicts with existing service ports it can be changed through this variable
HS_PG_DB Database name of the internal database program
HS_PG_USR Username for the internal database program
HS_PG_PWD Password for the internal database program
HS_ENGINE_HOME Path for engine data files
HS_ENGINE_HOST Engine host, after configuration the master module of the engine will be deployed on the corresponding machine, default is localhost
HS_ENGINE_SEGMENTS Engine worker nodes, array, default is (localhost); example for multiple nodes: (Node-A Node-B Node-C)
HS_ENGINE_PORT Engine port

Others

File Upload Limit

The product requires that uploaded files do not exceed 50MB. If you configure a reverse proxy, you need to configure the file upload size; taking Nginx as an example:

# For configuration file such as /etc/nginx/nginx.conf, configure in the corresponding server
client_max_body_size 100m;
timeout settings
# For configuration file such as /etc/nginx/nginx.conf, configure in the corresponding server
proxy_connect_timeout  600;
proxy_send_timeout 600;
proxy_read_timeout 600;
send_timeout        600;
Serving through ports 80, 443

The product cannot be started by root, cannot bind ports below 1024. If you need to listen on ports 80, 443, it is recommended to use Nginx for reverse proxying.

  • Create configuration file /etc/nginx/conf.d/hengshi.conf, content as follows:
# hengshi proxy
upstream hengshi_proxy {
        server $IP:$PORT;
}
server {
        server_name $SERVNAME;
        access_log /var/log/nginx/access.log main;
        listen 80;
        location / {
                proxy_pass http://hengshi_proxy;
        }
}
server {
        server_name $SERVNAME
        listen 443 ssl http2 ;
        access_log /var/log/nginx/access.log main;
        ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;
        ssl_ciphers '...';
        ssl_prefer_server_ciphers on;
        ssl_session_timeout 5m;
        ssl_session_cache shared:SSL:50m;
        ssl_session_tickets off;
        ssl_certificate $PATH_TO_CER
        ssl_certificate_key $PATH_TO_CER_KEY
        add_header Strict-Transport-Security "max-age=31536000" always;
        location / {
                proxy_pass http://hengshi_proxy;
        }
}

where variables need to be replaced with actual values

  • IP: The IP of the machine where hengshi is deployed
  • PORT: The port bound by the hengshi service
  • SERVNAME: The hostname that Nginx is used to serve
  • PATH_TO_CER, PATH_TO_CER_KEY: The absolute path to the SSL service's cer/key files respectively
More JDBC Drivers

If the version of the database to connect to is listed in the "Special Data Source Version JDBC Driver Table", you need to download the respective version of the driver and perform the following operations:

  1. Download the corresponding driver zip
  2. Decompress the corresponding zip
  3. Copy the decompressed content to the plugins directory under the installation directory
  4. Restart the hengshi service

Special Data Source Version JDBC Driver Table:

Server Version Download Address
hive-1.1 https://download.hengshi.io/3rd/hive-1.1-jdbc-minimal.zip

Note: After placing a JDBC driver in the plugins directory, you can only connect to the corresponding version of the service. Connectivity to other versions of the service cannot be guaranteed. For example, if you download the hive-1.1 driver and put it in the plugins directory, only connectivity to hive-1.1 can be guaranteed, and connectivity to other hive versions is no longer guaranteed.