In this session we are going to talk about Data Persistence in Redis. We have two approaches
One is called RDB and the other one is called AOF and both are having its own advantages and disadvantages
which we are going to discuss.
So which actually brings a third approach, which is called a Hybrid Approach.
So you use both.
You can use RDB with combination of AOF
Okay. So that is going to be the third approach
so how RDB works is, you enable RDB by making the change in the Configuration File.
or you can also use the Command Line to provide the
Configuration related instruction using Redis cli.
Okay so what RDB is going to do is, it is going to perform the point-in-time snapshot of data which is
available in Redis. So by that what I mean is, you define a, you know, kind of Snapshotting policy and then
based on whatever the Snapshotting policy, which has been defined, Redis is going to take a Snapshot
of the Data which is available in that point of time. Okay.
Based on the configuration which you have specified and whenever you restart Redis or whenever you
know some crashes occurs, you can always restore from that point -in saved Snapshot.
Okay.
So if you focus here on the second line, which is called Save 60 then space 1000
So this is one of the way to define RDB Snapshotting policy.
So you tell Redis to save every 60 seconds.
If there are 1000, you know, keys has been updated or keys has been inserted.
Okay
so you can have many number of, you know, such configuration defined in the config file.
We'll see how to refine these things, but you can have something like one line, one piece of line is this
and then you can have another line of code, which says save every 5 seconds when you have 100000 keys
inserted or updated and then you can have third line which says, save every 1 second. If you have 1 million
keys inserted or updated. So something like that
and then the second approach which is called AOF.
So AOF is just append only file. So what it does is, basically you are telling Redis is to make a
note of all the operations, all the right operations, which has been performed in Redis
And basically it creates one file and it keeps on writing all the operation, which has been performed
by Redis.
So whenever Redis goes down you actually use this file and you, you bring all the instructions.
You basically replay whatever has been logged in AOF
Okay.
So when you are, let's say writing a Query to Redis by using Set command. So you define a Set,
you define a Key and you define a Value.
So once you run that command, the same command also going to get logged in this AOF
File. So whenever Redis is shut down and it is started again.
Whatever is there in this AOF file.
This is going to be played back.
So that is a mechanism how Redis is
restoring all the data, which is stored using AOF mechanism. Ok, in AOF
we have something called Defining fsync policy. So we can define how frequently we want to sync
We want to keep writing the logs.
Remember whatever the policies which we define, is going to impact the performance.
So if you are asking Redis to, you know, always sync with AOF.
So basically AOF resides on the Disk and whenever you are asking that for each operation which Redis is
performing, it should be writing to Disk, then the performers is going to be very slow.
OK.
If you want Increase in Performance, you can define a policy which is Every Second.
And then there is another policy, which is called Never. Ok in that case most of the Operating System
itself performs commits, which is usually around every 30 seconds. So the performance is going to be
faster. Okay, we'll be discussing about all of these approaches and we will see how difference, you know, performance
which we are able to achieve.
So the point is, there is a Persistence Mechanism available which is called RDB and RDB is not
you know, you are not going to get point of failure, in a data recovery.
So it is Point In Time Data Recovery.
So basically you define a Configuration and based on that you are going to be able to restore whatever
the latest data available in that RDB.
and AOF, you can actually make it like, almost point of failure, you know, you can restore, you can restore
without losing any data in Redis.
So now let's discuss more about RDB advantages and disadvantages.
So RDB basically is a Compact, Single File.
So by that what I mean is, when you perform RDB Snapshotting, Redis runs a child process which is called
Fork.
So that process snapshots all the Redis data in a single RDB file, which is also called
RDB.dump. Ok
that single file actually consist all of the data, which is available in Redis. OK.
So this is very good, if you want to perform any external backups. Let's say every 24 hours or every 6
hours. You want to, you want to backup or you want to perform External Backup.
OK.
So RDBs are very good in those kind of scenarios and the performance of RDB backup is very good.
The reason is, there is no overhead on Redis. The only thing which Redis has to do is just creating
a child process, which takes care of the Snapshotting and also when you are backing up using RDB, the restart, when you are
restarting Redis , it is going to be faster because it can quickly load whatever there in RDB to
Redis memory.
OK so faster approach when you compare it with AOF and disadvantage in case of RDB is
that these are Chances of Data Loss.
So it is not, you know, not giving you point of failure of data recovery.
So whatever the, whenever the latest snapshot has happened, you are going to restore only from that point
of time. You are not going to restore from the point of failure.
So yes there is a chances of Data Loss, when you are using only RDB approach and also when you
have very huge Data Set. Since RDB has to create a Child Process or Fork Process to perform Snapshotting.
So for few milliseconds, it may has to temporarily suspend or stop writing operations, but it is only
going to happen, when you have really large Data set. It doesn't, you will not even notice when the Data set
is not very large.
Now let's discuss about AOF
So AOF is Append Only File. Append Only File is highly durable, when you compare it with RDB
and in AOF you can configure different fsync policies. So it gives you more flexibility when compared with
RDB.
So you can define how frequent you want your data to be saved or log to AOF.
and in RDB, usually we have chances of corruption, when there is any power loss. in AOF the chances
of corruption is very less.
The reason being is AOF consist of
So all the instruction, which you have given to Redis Line by Line.
So basically if you ran 1000, you know, Set instructions AOF is going to keep all those thousand lines.
Line by Line and going to keep on appending in a single file.
So in case there is a power outage, the chances of corruption of that file is very less.
And even though if there is a corruption happens, there is a tool which is called
redis-check-aof
You can use this tool to fix any problem with this AOF file
and since AOF file is more like, you know, your data file or any other text file. You can open this file
and if you want to remove or you want to, you know, do not want to run some of the lines when Redis restarts
you can remove those lines.
This is especially going to be useful, when let's say accidentally you run Flushall or Flush command and
then you just want to restore everything back from AOF.
So when you run Flushall command,
AOF is also going to make an entry o Flushall command in AOF File.
Okay but let's say when you want to restart data, you don't want all the data to be flushed. You can
remove that Flushall line from this AOF File and then you can restore from this AOF File
OK.
So basically this AOF File gives you liberty to pass the file and play with the, you know, the set of instructions
or the commands which are available in this file. So if we discuss about disadvantages when compared
to RDB.
So usually the size of AOF is going to be bigger than RDB. RDB compresses or performs snapshotting, compresses
the data.
So the size is lesser when compared to AOF and AOF can also be slower than RDB, again depending
on the fsync policies which you have defined.
So if you have defined fsync policy of Always Sync. In those cases all the instruction has to be
written to AOF and then AOF can grow bigger in size and AOF can be slower OK.
So let's see all of these,
I mean both of these things in action and then we will also see how we can take advantage of Hybrid
approach.