What should Redis cluster plan do?

Usually, in order to improve the response speed of the website, hot data is always stored in memory, rather than directly read from the back-end database. Redis is a good caching tool. For large-scale website applications, the amount of hot data is often huge, and it is normal for tens of gigabytes to hundreds of gigabytes. In this case, how to construct Redis correctly?

First of all, whether we use our own physical hosts or cloud service hosts, memory resources are often limited. Expanding the scale is not a good idea. We need scale-out, and scale-out requires multiple hosts to provide services collaboratively, that is, distributed multiple Redis instances work collaboratively.

Secondly, at present, the cost of hardware resources is reduced, and multi-core CPU and hosts with tens of gigabytes of memory are very common. For Redis whose main process is single-threaded, it seems a bit wasteful to run only one instance. At the same time, managing a huge memory is not as efficient as managing a relatively small memory. Therefore, in practical use, usually a machine runs multiple instances of Redis at the same time.

plan

1.Redis official cluster scheme

Rediscoluster is a server fragmentation technology, which was officially provided in version 3.0.

Radice

In the cluster, the concept of slot is adopted for fragmentation, and a * * * is divided into 16384 slots, which is somewhat similar to the pre mentioned above.

Split thinking. For each key-value pair entering Redis, it is hashed according to the key and assigned to one of these 16384 slots. The hash algorithm used is also relatively simple.

Single, that is, after CRC 16, 16384 takes the modulus.

Each node in the Redis cluster is responsible for sharing a part of these 16384 slots, that is, each

The slot is responsible for processing a corresponding node. When dynamically adding or deleting nodes, 16384 slots need to be reallocated, and the key values in the slots need to be migrated. Of course, this process, at present.

At present, it is still in a semi-automatic state and needs manual intervention.

Redis cluster ensures that the nodes corresponding to slot 16384 work normally. If a node fails, the slot it is responsible for will fail and the whole cluster will not work.

for

In order to increase the accessibility of the cluster, the officially recommended scheme is to configure the nodes into a master-slave structure, that is, a master node with n slave nodes. At this point, if the master node is lost.

Effective, Redis cluster will choose a slave node as the master node according to the election algorithm, and the whole cluster will continue to provide services to the outside world. This is very similar to Redis mentioned in the last article.

In the fragmentation scenario, the server nodes are built into a master-slave structure through Sentinel monitoring, but Rediscoluster itself provides failover and fault tolerance.

Radice

The new node identification ability, fault judgment ability and failover ability of a Cluster are that each node in the cluster is communicating with other nodes, which is called cluster bus (cluster

They use a special port number, that is, the foreign service port number plus 10000. For example, if the port number of a node is 6379, then the port number it communicates with other nodes is

16379。 The communication between nodes adopts special binary protocol.

For clients, the whole cluster is regarded as a whole, and clients can connect any one.

Nodes operate just like a single Redis instance. When the key operated by the client is not assigned to the node, Redis will return a steering instruction pointing to the correct node.

It's a bit like a 302 redirect jump on a browser page.

Redis cluster was officially launched after Redis 3.0, and the time was late. At present, there are not many cases that can prove successful in mass production environment, and it takes time to test.

2. The Radish Harding Cluster

Redis 3 officially launched the official cluster technology, which solved the collaborative service problem of multiple Redis instances. Rediscoluster can be said to be the embodiment of server-side fragmentation technology, that is, the key values are reasonably allocated to each instance fragment according to a certain algorithm, and at the same time, each instance node coordinates and communicates, and * * * undertakes services consistent with the outside world.

Multi-Redis instance service is much more complicated than single Redis instance, involving technical issues such as positioning, coordination, fault tolerance and capacity expansion. Here, we introduce a lightweight client-side Redis fragmentation technology.

Radice

Fragment can be said to be Redis

Before the advent of Cluster, the multi-Redis instance clustering method was widely used in the industry. The main idea is to use hash algorithm to hash the key of Redis data, and through hash function, specifically

The key of will be mapped to a specific Redis node. In this way, the client knows which Redis node to operate the data on. The fragmentation architecture is as shown in the figure:

Fortunately, java redis client driver jedis already supports redis fragmentation function, that is, ShardedJedis and ShardedJedisPool combine cache pool.

The Redis fragmentation implementation of Jedis has the following characteristics:

choose

Use a consistent hash algorithm (consistent

Hashing), hash the key and node name at the same time, and then map and match. The algorithm adopted is mur burn _ hash. Use consistent hashes instead of simple classes.

The main reason of modular mapping similar to hash is that when nodes increase or decrease, there will be no rehash caused by re-matching. Consistent hashing only affects the key distribution of neighboring nodes, with little influence.

2.

In order to avoid the node allocation pressure caused by consistent hashing only affecting neighboring nodes, ShardedJedis will virtualize each Redis node according to its name (no, Jedis will give it a default name).

160 virtual nodes are hashed. According to the weight, virtual nodes with multiples of 160 can also be virtualized. Using virtual nodes for mapping and matching can increase or decrease Redis nodes.

If so, the key is redistributed more evenly in each Redis node, instead of only affecting neighboring nodes.

3.ShardedJedis supports keyTagPattern mode, that is, a part of keyTag is extracted and sliced, and a group of related keys are put into the same Redis node by reasonably naming keys, which is very important to avoid accessing related data across nodes.

Redis fragmentation adopts the client-side harding mode, and the server-side Redis is still a relatively independent Redis instance node without any change. At the same time, we don't need to add additional intermediate processing components, which is a very lightweight and flexible Redis multi-instance clustering method.

Of course, the lightweight and flexible way of Redis fragmentation will inevitably damage other functions of the cluster. For example, in the expansion, when you want to add a Redis node, even if the consistent hash is used, there will still be keys that cannot be matched and lost, so key value migration is needed.

As a lightweight client, it is unrealistic to slice and handle the key migration of Redis, which requires the application layer to allow data loss in Redis or reload data from the back-end database. But sometimes, breaking through the cache layer and accessing the database layer directly will cause great pressure on system access. Is there any other way to improve this situation?

Radice

The author gives a more convenient method-pre-harding, that is, deploying as many Redis instances as possible according to the system scale in advance, which occupy very little system resources and can be used by a physical machine.

Multiple departments, let them all participate in fragmentation. When expansion is needed, select an instance as the master node, and the newly added Redis node as the slave node for data replication. After data synchronization, modify

Fragment configuration, let the fragment pointing to the original instance point to the extended Redis node on the new machine, and adjust the new Redis node to the main node, so that the original instance can not be used again.

Prehardening

It is to allocate enough fragments in advance, and only replace the Redis instance that originally belonged to a fragment with a new Redis instance with larger capacity when expanding. Fragments participating in the fragmentation have not changed, so they also

There is no phenomenon that the key value is transferred from one area to another partition, but the key value belonging to the same partition is synchronized from the original Redis instance to the new Redis instance.

Not only increase

Deleting Redis nodes leads to the loss of key values, and the greater obstacle comes from the sudden downtime of Redis nodes. The article "Persistence of Redis" mentioned that in order not to affect the performance of Redis, try not to open it.

The file saving function of AOF and RDB can constitute the main standby mode of Redis. When the primary Redis goes down, the data will not be lost, and the standby Redis has a backup.

In this way, our architectural model has changed.

The Redis node slice contains a primary Redis and a standby Redis. When the main Redis goes down, the standby Redis takes over and rises to the main Redis to continue providing services. owner

Standby * * * forms a Redis node, which ensures the high availability of the node through automatic failover. Then, the fragmentation architecture evolved into:

Redis Sentinel provides the function of Redis monitoring and failover in active and standby mode, which realizes the high availability of the system.

Under high traffic, even if there is fragmentation, a single node is still under great access pressure. At this time, it needs to be further decomposed. Generally speaking, there is a great difference between reading and writing when an application accesses Redis, and reading is often several times as much as writing. At this time, we can separate reading from writing. Reading provides more examples.

You can use master-slave mode to separate reading from writing. The master is responsible for writing and the slave is responsible for reading. A master hangs multiple slaves at the same time. Under the sentinel monitoring, it can also ensure the automatic monitoring of node faults.

3. Using proxy middleware to realize large-scale Redis cluster

The above introduces two ways of multi-Redis server clustering, namely client-based Redis fragmentation and server-based Redis clustering.

The advantage of client-side fragmentation technology is that the Redis instances on the server are independent and unrelated, and each Redis instance runs like a single server, which is very easy to expand linearly and the system is very flexible. Its disadvantages are:

Because the fragmentation processing is put on the client, it will bring challenges to the operation and maintenance when the scale is expanded.

When the topology of the server Redis instance group changes, each client needs to update and adjust.

The connection cannot be enjoyed, the application scale increases, and the waste of resources restricts optimization.

The advantage of sharding's Redis cluster is that when the topology of the server's Redis cluster changes, the client does not need to know, and the client uses the Redis cluster like a single Redis server, which makes the operation and maintenance management more convenient.

However, the official version of Redis cluster has not been released for a long time, and the stability and performance of the system need time to test, especially in large-scale use occasions.

Can you combine the advantages of both? That is to say, server instances can be independent of each other, support linear scaling, and at the same time, fragmentation can be handled centrally, which is convenient for unified management? Twemproxy, the Redis proxy middleware introduced in this paper, is such a technology that uses middleware to segment.

Twemproxy is located between the client and the server, and forwards the request sent by the client to the real Redis server after some processing (such as fragmentation). In other words, the client does not directly access the Redis server, but indirectly accesses it through the twemproxy proxy middleware.

Referring to the Redis fragmentation architecture, the Redis cluster architecture for adding proxy middleware is as follows:

The internal processing of twemproxy middleware is stateless, and it can easily cluster itself, thus avoiding a single point of pressure or failure.

Twemproxy, also known as Nutcracker, originated from the development practice of redis/memcached cluster in twitter system, which runs well and the code is dedicated to the open source community. Lightweight and efficient, developed in C language, the project website is: github-Twitter/twemproxy: Afast, memcached and redis lightweight agents.

Twemproxy backend not only supports redis, but also supports memcached, which is caused by the specific environment of twitter system.

Due to the use of middleware, twemproxy can enjoy the connection with the back-end system through * *, which reduces the number of connections directly connected by the client to the back-end server. At the same time, it also provides a fragmentation function to support the horizontal expansion of the back-end server cluster. Unified operation and maintenance management also brings convenience.

Of course, due to the use of middleware proxy, compared with the method of connecting the client directly to the server, the performance will be lost, and the measured results will be reduced by about 20%.