Redis Cluster - How to create a cluster without redis-trib.rb file

The redis documentation here mentions about creating the redis cluster using a redis-trib.rb script. However, just to know how it works internally, one should know how to create the cluster without the redis-trib.rb script.

This blog assumes that you have 6 redis instances running on ports 7000, 7001, 7002, 7003, 7004, and 7005.

The redis cluster creation involves the below steps.

1. Starting all the redis nodes in cluster mode.
2. Making all the nodes meet each other.
3. Assign slots to the nodes.
4. Assign slaves to masters.

Step 1: Assuming the redis nodes are started in cluster mode, you need to check whether redis is indeed running using 'ps -ef | grep redis' command.

Step 2: Making the nodes meet each other.
This is done using the cluster meet command as mentioned here.
redis-cli -c -p 7000 cluster meet 127.0.0.1 7001
redis-cli -c -p 7000 cluster meet 127.0.0.1 7002
redis-cli -c -p 7000 cluster meet 127.0.0.1 7003
redis-cli -c -p 7000 cluster meet 127.0.0.1 7004
redis-cli -c -p 7000 cluster meet 127.0.0.1 7005

This way, all the nodes will meet the node 7000, and in turn, each other.

Now all the nodes have met each other. Execute "redis-cli -p 7000 cluster info" command to check the output. It should show the cluster_state as failed, and cluster_slots_assigned as 0.

Step 3: Assign Slots to the nodes.
This is done using the "cluster addslots" command as mentioned here.
Below we use inline shell script to assign the slots to the three nodes and ignore the output by directing it to "/dev/null". If we don't direct the output anywhere, it will still work, only that it will show "OK" many times.

for slot in {0..5461}; do redis-cli -p 7000 CLUSTER ADDSLOTS $slot > /dev/null; done;
for slot in {5462..10923}; do redis-cli -p 7001 CLUSTER ADDSLOTS $slot > /dev/null;; done;
for slot in {10924..16383}; do redis-cli -p 7002 CLUSTER ADDSLOTS $slot > /dev/null;; done;

Each command may take some time, around a minute. Executing "cluster info" after it shows that cluster state is "ok" and cluster_slots_assigned are 16384.

Step 4: Assign Slaves to masters.
Note that we had assigned slots to 7000, 7001, 7002, and plan to make the other nodes as their slaves.
Slaves are assigned using "cluster replicate" command as described here.
The syntax of the command is
"redis-cli -c -p <port_of_slave> cluster replicate <node_id_of_its_master>"
To find the node id of masters, we can use "cluster nodes" command and note the ids corresponding to ports 7000, 7001, 7002.

After that we will execute (Note your node ids will be different than below)

redis-cli -p 7003 cluster replicate 0db8943cccb9b8c3963af78019baf8d2db827f14 

redis-cli -p 7004 cluster replicate add93eed2b5f05371afab5d2611381ebc363d9c7

redis-cli -p 7005 cluster replicate 12e0e6c2e59b8ac284d413623b7d73fd3ec56383

Executing "redis-cli -p 7000 cluster nodes" after it will show that you have slaves also.

Congratulations, your cluster is up and running.

Now, you can add some dummy data in it and start playing around with it.

Redis Error - Can't save in background : fork: Cannot allocate memory

Redis periodically saves its data in memory onto the disk for backup in the form of rdb file. This is helpful in recovering the data if redis restarts or for backup purposes.
Sometimes while saving the data, redis may give the error: "Can't save in background : fork: Cannot allocate memory".

This error can come if redis has enough memory for the data, but not for saving the rdb file.
For this, you need to check a system param named "sysctl vm.overcommit_memory". Its value should be set to 1 instead of 0 by the command "sysctl vm.overcommit_memory=1" as shown below.

After setting this value, the background save can be successfully started, and it does not give an error.
The Below are the logs after the value was set to 1 and after starting background save.

Note that this error only comes when all three conditions hold

• redis memory is close to or more than half of allocated memory/system memory(memory is 32+GB and 27 GB is left).
• vm.overcommit_memory is set to 0
• when redis does a background save.

Also, to make sure that this setting persists even when the redis machine restarts, it needs to be set in "/etc/sysctl.conf" file.

More details here