finding out the memory occupied by a single key in redis

Many times while using redis, we need to find out how much memory is occupied by a particular key.

This is very useful because redis keeps everything in memory and we need to make the best use of whatever memory we have to reduce the infrastructure cost.

Redis provides a command 'DEBUG' which can be used for this purpose.

This command gives us information like size about a particular key. However this length is the serialized length(i.e. the bytes used to store this key in the rdb backup file).

The below shows how we can find the information about two keys(one string and one hash) in redis.

In the above, the serialized length is the number of bytes which are used by redis for storing the value in the backup file.

More often than not, we are not interested in the serialized length but the memory used by that key.

Unfortunately redis does not provide any such information by its own(as per my knowledge), but thanks to the open source community, an excellent such utility 'redis-rdb-tools' is written which gives us a very good estimate of the memory used by redis for a particular key.

The link to the utility is here.

Redis stores different keys in different formats, and the utility reverse engineers a key with its type and value, to get the approximate amount of memory used by a key. Although the utility underestimates the memory occupied by a key and the real memory used by a key is a little higher(can be higher by upto 20%), but even then it's very important because it can find out the large keys, and it can generate a CSV with the information about all the keys. The CSV file generated can be further analyzed for more insights into the data.

Usage:

After installing the redis-rdb-tools as described here, we can use it to find out the memory used by a key.

finding out memory for a key from running redis.

redis-3.2.4 $ redis-memory-for-key -s localhost -p 6379 mystringkey

Key "mystringkey"

Bytes 88

Type string

redis-3.2.4 $ redis-memory-for-key -s localhost -p 6379 myhashkey

Key "myhashkey"

Bytes 115

Type hash

Encoding ziplist

Number of Elements 2

Length of Largest Element 6

redis-3.2.4 $ 

finding out memory for a key from a rdb file.

redis-3.2.4 $ rdb -c memory dump.rdb -k mystringkey

database,type,key,size_in_bytes,encoding,num_elements,len_largest_element

0,string,"mystringkey",88,string,13,13

finding out memory for all keys for a pattern.

redis-3.2.4 $ rdb -c memory dump.rdb -k my.*

database,type,key,size_in_bytes,encoding,num_elements,len_largest_element

0,list,"mylistkey",219,quicklist,7,6

0,sortedset,"mysortedsetkey",143,ziplist,6,5

0,hash,"myhashkey",115,ziplist,2,6

0,string,"mystringkey",88,string,13,13

0,string,"myhllkey",168,string,90,90

0,set,"mysetkey",452,hashtable,4,6

finding out memory for all keys for a pattern and exporting to a csv file.

redis-3.2.4 $ rdb -c memory dump.rdb -k my.* -f memory.csv

redis-3.2.4 $ head memory.csv 

database,type,key,size_in_bytes,encoding,num_elements,len_largest_element

0,list,"mylistkey",219,quicklist,7,6

0,sortedset,"mysortedsetkey",143,ziplist,6,5

0,hash,"myhashkey",115,ziplist,2,6

0,string,"mystringkey",88,string,13,13

0,string,"myhllkey",168,string,90,90

0,set,"mysetkey",452,hashtable,4,6

After we have the csv file from the above method, we can do all sorts of analysis after the excel tricks by finding out the largest N keys of a particular type, keys occupying highest amount of memory etc.

:)