What is a consistent hash algorithm?

Consistent hash algorithm is a distributed hash (DHT) algorithm proposed by MIT in 1997. Its design goal is to solve the hot issues in the Internet, and its original intention is very similar to that of carp.

Consistent hashing is a special hashing algorithm, which is widely used in the field of load balancing because of its balanced and persistent mapping characteristics. For example, nginx and memcached both use consistent hashing as a cluster load balancing scheme.

The goal of consistent hashing algorithm is when k request keys initiate a request. Adding or deleting nodes in the background will only cause K/N keys to be remapped. That is, the consistent hash algorithm, when the background node is stable, every request of the same key is mapped to the same node. When the background nodes increase or decrease, the algorithm tries to map k keys to the same nodes as before.

superiority

Scalability. The consistent hash algorithm ensures the minimum change of data storage when adding or reducing servers, and greatly saves the overhead of data movement compared with the traditional hash algorithm.

Better adapt to the rapid growth of data. A consistent hash algorithm is used to distribute data. When the data keeps growing, some virtual nodes may contain a large amount of data, resulting in uneven data distribution on virtual nodes. At this time, you can split the virtual node containing more data, just split the original virtual node in two, without re-hashing or splitting all the data.

After the virtual nodes are split, if the physical server load is still unbalanced, it is only necessary to adjust the storage allocation of some virtual nodes among servers. This can dynamically expand the number of physical servers with the growth of data, and the cost is much smaller than that of traditional hash algorithm to redistribute all data.

Above content reference: Baidu Encyclopedia-Consistent Hash