On
id=”js_tags” class=”article-tag__list”> included in the collection #redis 1

When writing open source projects, I thought of supporting a variety of Redis deployment methods, so if it is helpful to investigate the architecture selection of this production environment

,

please click to pay attention to support my continued writing, thank you ~

an engine version

It is recommended to use a newer engine version to support more features,

Redis 6.0 new feature description

  • module system to add multiple APIs.

  • Supports SSL/TLS encryption.

  • Support for the new Redis protocol:

    RESP3.

  • The server supports multi-mode client-side caching.

  • Support for multi-threaded IO.

  • Diskless replication is supported in replicas.

  • Redis-benchmark has added Redis cluster mode.

  • Support for overriding Systemd.

  • Support for the Disable module.

Redis 5.0 new feature description

    cloud database

    Redis

  • version 5.0 greatly optimizes the kernel, runs more stably, and adds Stream, account management, Audit logs and other features to meet your needs in more scenarios.

  • New data type:

    Stream.

    For detailed instructions, see Redis Streams.

  • Added account management function.

    • Added log management function, support audit log, running log, and slow log,

  • you can log to manage query read and write operations, sensitive operations (such as KEYS, FLUSHALL), and management command usage records, and slow logs.

  • Added snapshot-based cache analysis.

  • APIs for new Timers, Cluster, and Dictionary modules.

  • Add LFU and LRU information to the RDB.

  • The cluster manager was ported from Ruby (redis-trib.rb) to C code in redis-cli.

  • Added ordered set commands ZPOPMIN, ZPOPMAX, BZPOPMIN

    , and BZPOPMAX.

  • Upgrade Active Defragmentation to v2.

  • Enhanced HyperLogLog implementation.

  • Optimized memory statistics reporting.

  • The HELP subcommand has been added to many commands that have subcommands.

  • Improved performance when clients connect and disconnect frequently.

  • Upgrade Jemalloc to version 5.1.

  • NEW COMMANDS CLIENT ID AND CLIENT UNBLOCK.

  • The LOLWUT command for art has been added.

  • Deprecation of the slave term (with the exception of cases that require API backward compatibility).

  • Several optimizations have been made to the network layer.

  • Some Lua-related improvements have been made.

  • Added Dynamic HZ to balance idle CPU usage and responsiveness.

  • The Redis core code has been refactored and improved in many ways.

You need to choose according to your business needs:

  • cluster architecture can easily break through the single-threaded bottleneck of Redis itself to meet the requirements of large capacity and high performance.

  • The master-slave architecture provides high-performance cache services and high data reliability.

    • The read/write splitting

  • architecture provides highly available, high-performance, and highly flexible read/write splitting services to solve the business requirements of centralized hotspot data and high concurrent reads, and maximize user O&M costs.

2.1 Master-slave architecture – double replicas

are built in master-replica mode. When the primary node fails, the system automatically switches to the standby node within 30 seconds to ensure smooth business operation.

Reliability

< ul class="list-paddingleft-2">

  • service reliably adopts a dual-machine master-replica architecture, and the master-slave nodes are located on different physical machines.

    The master node provides external access, and users can add, delete, modify, and check data through the Redis command line and the common client.

    When the master node fails, the HA system automatically switches the master and slave to ensure smooth business operation.

  • Data Reliability enables data persistence by default, and all data is placed on disk.

    Supports data backup,

    allowing users to roll back instances or clone instances for backup sets, effectively resolving problems such as data misoperation.

    • Usage scenario

    < ul class="list-paddingleft-2">

  • Redis provides a persistence mechanism and backup and restoration mechanism as a business standard version for persistent data storage to greatly ensure data reliability.

  • For services with controllable performance pressure for a single Redis service, Redis natively adopts a single-threaded mechanism, and the service with a performance of less than 100,000 QPS is recommended.

    If you need higher performance requirements, use the cluster version.

  • Redis commands are relatively simple, and services with fewer sorting and calculation commands will become the main bottleneck due to Redis’ single-threaded mechanism.

    For example, it is recommended that you use the cluster version configuration for services with more sorting and computing classes.

    • 2.2 Master-slave architecture – single copy

    can give full play to performance advantages in pure cache scenarios without data reliability requirements.

    When

    the node fails, the

    system re-pulls a Redis process (no data), and when the node fails to automatically switchover, the application needs to rewarm the data to avoid the impact on the back-end database. The single-copy architecture cannot provide data reliability, and if a node failure occurs, you need to rewarm up the business, so it is recommended to choose a two-copy architecture for sensitive services that require high data reliability.
    Since Redis natively adopts a single-threaded mechanism, the CPU is single-core capability, and the performance is recommended for 80,000 QPS. If you need higher performance requirements, choose a cluster edition configuration.
    Due to Redis’ single-threaded mechanism, the CPU is the main bottleneck. For example, it is recommended that you use the cluster version configuration for services with more sorting and computing classes.

    2.3 Cluster Edition – Dual Replicas

    can easily break through the single-threaded bottleneck of Redis itself and meet the requirements of large capacity and high performance. Dual-replica Cluster Edition instances adopt a cluster architecture, and each sharded server adopts master-replica dual replica mode. Cluster Edition supports both proxy and direct connection connection modes, and you can select a connection mode that suits your business needs according to the instructions in this section.

    The local disk instance of the proxy mode cluster architecture adopts proxy mode by default, which supports access to the Redis cluster through a unified connection address (domain name), and the client’s request is forwarded to each data shard through the proxy server, and the proxy server, data shard, and configuration server do not provide a separate connection address. Reduces application development difficulty and code complexity. The service architecture diagram and components of the proxy pattern are described below.

    direct connection mode

    Because all requests are forwarded through the proxy server, the proxy mode reduces the difficulty of business development and slightly affects the response speed of Redis service. If your business requires very high response speed, you can use the direct-connect mode to bypass the proxy server and directly connect to the backend data shard, thereby reducing network overhead and service response time. The service architecture and description of the direct-connect mode are as follows.

    Prerequisites
    Use Jedis, PhpRedis and other clients that support Redis Cluster.

    • using a client that does not support Redis Cluster, it may not get the data it needs because the client cannot redirect the request to the correct shard.

    • Jedis’ support for Redis Cluster is based on the JedisCluster class, see Jedis’s documentation for details.

    • You can find more clients that support Redis Cluster in the client list on the Redis website.

    Sample code using custom connection pooling 

     import redis.clients.jedis.*; import java.util.HashSet; import java.util.Set; public  class main {private  static final int DEFAULT_TIMEOUT =  2000; private static final int DEFAULT_REDIRECTIONS = 5; private static final JedisPoolConfig DEFAULT_CONFIG = new JedisPoolConfig(); 

     public static void main(String args[]){  JedisPoolConfig config = new JedisPoolConfig();   config.setMaxIdle(200 );   config.setMaxTotal(300 );  config.setTestOnBorrow(false);  config.setTestOnReturn(false); 
      String  host = "r-bp1xxxxxxxxxxxx.redis.rds.aliyuncs.com";  int port = 6379;   String password = "xxxxx" ; 
     Set jedisClusterNode = new HashSet();  jedisClusterNode.add(new HostAndPort(host, port));  JedisCluster jc = new JedisCluster(jedisClusterNode, DEFAULT_TIMEOUT, DEFAULT_TIMEOUT, DEFAULT_REDIRECTIONS,password, "clientName", config); }}

    2.4 cluster-single replica

    2.5 read/write splitting

    For business scenarios where read, write, and write less, it provides highly available, high-performance, and flexible read/write splitting services to meet the business requirements of centralized hot data and high concurrent reads, and maximize O&M costs. The read/write splitting version is mainly composed of primary and standby nodes, read-only nodes, proxy nodes, and high-availability systems.

    < img src="https://mmbiz.qpic.cn/mmbiz_png/sXiaukvjR0RDv7vocJgiaTnkickgnHH1YQw12XDUKhb7GLYAHW47dvvPKfqZBDryiabic2AFiacRGicfHMEL2ibDicAJia2A/640" >

    feature

    1. automatically monitors the health status of all data nodes through a self-developed high-availability system to escort the availability of the entire instance.
      Automatically selects a new primary node and rebuilds the replication topology when the primary node is unavailable.
      When a read-only node is abnormal, the high-availability system can automatically detect and restart the new node to complete data synchronization and take the abnormal node offline.
    2. Proxy nodes are aware of the service status of each read-only instance in real time.

      During an abnormal period of a read-only instance, Proxy automatically reduces the service weight of the node, and

      after the read-only node fails more than a certain number of consecutive times, it stops the service entitlement of the abnormal node and has the ability to continue monitoring and restarting the node service.
    1. read/write splitting adopts a chained replication architecture, which can increase the overall instance performance linearly by expanding the number of read-only instances, and at the same time customize and optimize the Redis replication process based on the source code level , which can maximize the system stability of linear replication and make full use of the physical resources of each read-only node.

    Use scenario

    < ul class="list-paddingleft-2">

  • read request QPS (Queries Per Second) is under high pressure

    • The standard version of Redis cannot support large QPS, if the service type is read, write more and write less, you need to use the deployment method of multiple read-only nodes to break through the performance bottleneck of Redis single-thread. Redis Cluster Edition provides configurations of 1, 3, and 5 read-only nodes, which can improve QPS by nearly 5 times compared with the standard version.
    Services that require high compatibility with Redis protocol
    The read/write splitting version is fully compatible with Redis protocol commands, which can migrate self-built Redis databases to read/write splitting version, and support one-click smooth upgrade from Redis Standard Edition (dual copies) to read/write splitting version.

    Recommended and Usage Instructions

    < ul class="list-paddingleft-2">

  • When a read-only node fails, the request will be forwarded to other nodes; If all read-only nodes are unavailable, requests are all forwarded to the primary node. Read-only node exceptions may cause the load on the primary node to increase and the response time to become longer, so we recommend that you use multiple read-only nodes in scenarios with high read load.
  • In some scenarios, the full synchronization of read-only nodes is triggered, such as after the primary node triggers a high-availability switchover. During full synchronization, read-only nodes do not provide services and return -LOADING Redis is loading the dataset in memory\r\n.

    • If you like this article, please click on the

    upper right corner to share the article to the circle
    of friends If you want to know the technical points of learning, please leave a message to Ruofei to arrange sharing

    end


     
     

     
     public number (zhisheng)  reply to  Face, ClickHouse, ES, Flink, Spring, Java, Kafka, Monitoring < keywords such as span class="js_darkmode__148"> to view more articles corresponding to keywords. 
     
     
    like + Looking, less bugs 👇