So in this session
we are going to learn about setting up a Master Slave Configuration, which is going
to be a Production like setup and also we will try to simulate a Production Failure.
So we are going to shut down one Master Server and then we will be creating a New Master Server on the
fly and the existing Slave Server, we are going to point to the New Master Server
and in this example I'm going to use three servers. So first server is 10.128.0.4 and
Host name is itpanther02 and Second Server is going to be a Slave Server, which is having IP address
of 10.128.0.5 and
Host name is itpanther03. Okay.
So we will set up a Master Slave Configuration on these two servers and then we are going to shut down
a Master Server, which is 10.128.0.4 and then we are going to log into a New Server
which is, 10.128.0.6 and there are we are going to set up Redis again. A brand new installation
and then we are going to, you know, move these existing server or point this existing server, Slave Server
to this New Master Server.
And of course we'll be making sure that whatever data which has been written on, you know, previously Master
and Slave, that should be available in the New Master Server.
Okay.
So basically we should be having 0 Data Loss. So let's begin.
So you can verify this is a first server, which is going to be a Master Server. The IP address is
10.128.0.4
Right now,
I am in /home/redisadmin directory and I have downloaded the software which is Redis 5
And also I have extracted this software and now we have a Redis-stable directory.
So let's go into this directory
and let's make changes into Redis.conf, Configuration File.
So the first thing which we need to do, is change the bind address to the actual IP address, where we
are running this Redis Server.
So it is going to be in this case 10.128.0.4
and if you want to change the port, you can change it.
In my case, I'm just going to leave it to default which is 6 3 7 9
and log file I'm going to change it to /tmp/redis.log
OK.
So we want logs to be written to /tmp/redis.log
and also we want RDB back up to be created to
Directory which is called, /home/redisadmin
OK.
So you see, db file name is set up to dump.rdb and if you want, you can change this name as well.
So in this example I'm not going to change it. I'm going to leave it as it is and directory is going to
be /home/redisadmin
So the RDB back up will be creating into this directory. So that's all required on the Master Configuration.
Now I'm going to save and exit
okay.
Now let's log into SlaveS erver.
So either you can open a new session, new and log In there or otherwise you can just use ssh to log
into any other sever and in my case I have set up Passwordless authentication .
So basically I'll set up a ssh key authentication. So I don't need to provide password.
OK.
In your case if you are getting some error related to authentication, just make sure you have setup ssh
authentication or otherwise you can open a New Session and there you can provide the IP details of
Slave Server. So I'm going to now log into itpanther03, which is having IP address of
10.128.0.5
Okay now we can verify the IP address by using Host name space -i
Command and you can see the IP address is 10.128.0.5
And currently we are in to /home/redisadmin directory
And if I do ls-lrt, you can see currently we do not have Redis software available here.
So what I will do is since I already have the Software on itpanther02 server. So I'm going
to copy from there.
So I'll exit from here and we are back to itpanther02 server.
So here we have the software.
So I will use scp command to copy the software to the Remote Server.
So command to do that is scp -r Redis-stable.
I want to copy this Complete Directory and sub- directories.
And now we need to provide the Remote Server where you want to copy this.
So it is going to be itpanther03
So I'm providing the Host name but you can do the same with IP name as well.
And we want to copy to the Home Directory.
So the way to do that is, you need to provide : after Host name and then you use ~ character
and then /. Hit enter. and Software copy is in progress.
okay.
Let me clear the screen and let me go to itpanther03 server.
We verify Host name -i
We verify the IP.
Now we can do ls-lrt. We can see Redis-stable is already moved here.
Now I'm going to make the configuration changes here. Okay.
So by default whatever configuration which we had done their on Master, has come here because we just
copy pasted the software on this machine.
However in this case we need to make sure we change the bind address because this is a different server.
So we need to provide the IP address of this server, which is in this case going to be
10.128.0.5
and I want all the logs to be going to /tmp/redis.log
So this is already good. I do not want to make any changes here.
And now next thing which we need to change here is related to you know, the RDB backup.
So I think this is also OK in this case. So we will just leave it to the same directory because I had
the same directory standard followed on all the servers.
OK.
So I'm just going to leave it as it is.
And now we need to set up Replica. So on Master Server we have not made any changes to this line or
this directive. Here we need to make changes.
So uncomment the line and after Replica of, you need to provide the IP address of the Master Server
which is in this case going to be 10.128.0.4
Okay.
And Master Port, as we know we haven't made any changes in the port.
So by default it is going to be running on Port 6 3 7 9
So that's.
Now we can save and exit and now we have software installed on, I mean the configuration is done on both the
server, which is Master Server and the Slave Server.
So now we can go ahead and
start these Servers.
Okay.
So let me start Master first.
So I will go back to
you know, Master Server.
We will verify the IP address.
So this is the IP address of Master Server.
So I will go to Redis -stable directory.
And here we need to go inside src directory
okay.
Here you see we have Redis -server utility.
So this is what we need to start.
Let me clear the screen first.
So we are into this directory and now I'm doing /./Redis- Server
And when we are starting Redis, we also need to provide the full path of the configuration file.
Redis.conf file.
So I know that is located at /home/redisadmin/redis-stable/redis.conf
And also I want to start this as a background process.
So in the end, I'm giving & symbol. So that's it. If we hit enter, now Masters should start
and also we can verify by checking the logs.
So as we remember in the Configuration, we have given the, that logs should be saved to
/tmp/redis.log
So let's check the logs.
Okay.
So we see Redis is started. It is saying Ready to accept connections. So everything looks okay here.
So now I'm going to
move to Slave Server which, is 10.128.0.5
okay.
And you can verify the IP address.
So this is the correct IP address.
Now let's go to Redis table.
Go to cd src
and here we can do Redis-server and also provide the full path of the wherever your configuration
file is.
So in this case it is at
/home/redisadmin/redis-stable/redis.conf
And also I'm going to use &
and let's see the log by going to /tmp/redis.log
So now on the Slave Server, when we went to redis.log, you can see that it is saying that Master
Replica Sync is successful.
So basically when we started Slave Server it was knowing that it is a Slave Server and it has to
connect to a Master Server.
So it tried connecting connecting to the Master Server and the connection was successful.
So now we will use
redis-cli utility to connect to Master Node.
So just remember using redis-cli, you can connect to any number of servers. You just need
to provide the correct IP address to which you want to connect to.
So I do not, if I want to connect to Master Server. I do not need to go to the Master Server.
Okay.
I can just stay on this server which is itpanther03, which is a Slave Server and
I still can connect to any server.
So I will do redis-cli then space -h
Here you need to provide the Host name. Either Host name or IP address. So let's provide IP address in this
case and also if you are running Redis on non default port, which is the port
Other than 6 3 7 9 then also you can provide that port by using -p and whatever the port
you are running Redis on.
So if you are running on a default port which is 6 3 7 9, then you do not need to provide the Port but
if you still want to be explicit, then you still can provide the port.
OK.
So it is going to work the same way regardless of whether I use -P or not.
I mean whatever, whether I use this port part or not but let's say in case I had changed the port
from 6 3 7 9 to some other port and then I tried to connect to this node without providing the port
information, then it will try to connect to 6 3 7 9 port and it will eventually say, that it is not able
to connect. So in those cases you need to provide the port number.
All right so let's connect. Hit enter and we should be able to connect to the Master Node.
Okay so we see we are able to connect to Master Node and let's do dbsize. So dbsize tells us what is
the total number of keys, which are available right now in Redis. So since it is a Brand New Fresh
Installation, we do not have any data stored.
Okay we can verify the same with keys *.
So we do not have anything or any keys stored.
I will store a key, I will store a list rather with Color and let's name it, let's provide some of the values.
Hit enter.
So now we have 6 values stored in a key which is called Color and if you want to see all the items which
is inside the Color Key. So we can use L range and the key name which is Color and Starting index which
is going to be 0 and then Ending index. Since I want to print everything I need to use -1. so
-1 means is basically print everything in this case.
Okay so if we hit enter, we can see we have all the information.
Now I'm going to exit by using Quit and then now I'm going to connect to Slave Node.
Okay so we need to change the IP address. Slave Node IP address is 10.128.0.5. I have change the IP address.
Hit enter.
Now you are connected to the Slave Node. So whatever you have written there on the Master Node, you should
be able to get the same data here on the Slave Node.
So I will do L range and provided the key name and the starting and end range.
Okay.
So once I do, I can see that I have got the same information on the Slave Node also.
Now to verify that it is really a Slave Node and it is not allowing any you know, right connection you
can try to give a or
try issuing an
Now to make sure this is a really a Slave Node, we can actually issue a Set command. It should give
us an error saying that, It is a Read Only Replica. So let's do a Set command and try to update a key
which is called Product.
And let's try to give a Value Name which is Chair. Hit enter and you see we are getting the error that this
is a Read Only Replica.
Okay.
So this confirms that this Server or this Node, which we are connected to right now is actually a Slave Node.
Okay so I will quit again from here
and let me clear the screen. Let's do Host name -i
So you see we are actually on Slave Node Server and also now let's see the you know,
existence of Slash.
Basically we want to see whether the dump.rdb file is created or not. So if you remember
we have given path of /home/redisadmin
So here we should be able to see
dump.rdb file
Yes we see the file is already available here.
OK
now we are good to mimic you know, Master fail scenario. So we will log into
itpanther02 server.
so now let's verify that we are on
yeah, we are on the Master node.
Okay.
And now I'm going to shut down this server.
So once we do that, we no longer will be able to, basically connect to this session. So I'm going to open
a new session.
Or let me open a Duplicate Session
okay.
So on this session I'm going to switch to Slave
Node.
Okay.
So here if you verify this is a Slave Node and here you can verify this a Master Node.
Now I'm going to do sudo space shutdown and before that let's also do tail -F on Slave Node. So
we can actually see the error.
Okay when we are shutting down the Master Server.
Okay so let me shut it down.
Okay so shut down command has issued and now this server is going to get shut down.
Now let's see the logs here.
So whatever message you saw here.
Actually this is not related to Redis log.
This is basically because we open the session as a duplicate session.
So this is the message from itpanther02 server.
Anyhow we can run this command again.
So we are able to see the latest logs
Okay so as of now Master server, still is in the process of shutting down.
So we do not see any error here as of now.
Okay.
So now we see that we get the error, which means
that server has been disconnected.
So since we opened it as a Duplicate session or somehow this session is not allowing us to see anything
and this session
Also has went to a inactive state. So rather I will open a new session and then we log into the Slave
Node.
So this is a new session which I have opened and this is a Slave Node. You can verify the IP address.
Let me close all the previous sessions.
Okay so this is what we have right now.
And just remember that we have shut down the Master server.
So let's verify the logs on Slave server.
Now you see that Slave is getting some time out errors that it is basically not able to connect to Master
node
okay.
So this is an indication that Master is not available.
And also you can use various Redis-cli and use -h and try connecting to the Master Node
you notice
when I hit enter, I don't see anything or any message.
So basically it has just, we are not able to connect to this Redis Client, which is Master Node.
So i press control c to exit
and if we verify by connecting to Slave Node.
We still can connect to Slave Node and we can see the keys stored here.
And we also can see everything, which is stored here. It is still a Slave Node, because if you try to update
any value, you won't be able to do it.
Okay so now let's connect to a New Server which in this case is going to be, bind address
And here we will do the setup.
Okay so right now we are on Slave Node.
I'm going to do ssh itpanther04 and now if we do Host name -i , you can see will log
to
10.128.0.6
Now here right now, I do not see any Redis software.
So this is you can assume, it is a fresh server, where we do not have Redis software also available.
So I'm going to go back to Slave Node.
From there I will copy the software.
And also I will copy dump.rdb
okay.
So here we will do scp -r, let's copy this software
and now we will copy dump.rdb as well.
Okay.
Because if you remember dump.rdb is having the or this is the basically a snapshot.
Okay.
And before we actually copy this dump.rdb, we should actually make sure that this is the latest
dump file.
Okay so this is the latest RDB file.
Now the way to ensure that is, actually you can use a safe command again.
That will make sure that it creates a latest dump.rdb file. So for that I will connect to
Redis-cli like and provide the Slave IP address.
Syntax was incorrect.
So that's why we couldn't connect okay.
So here I will write or I will hit save command. Type save.
Hit Enter. Now it replies Okay. So now it has created the latest dump.rdb file. So we can quit
from here. So you can use either Quit command or exit command. Both works the same way.
OK.
Now you see dump.rdb size previously and now there is a difference. So basically this was, this might
be a little not up to date.
This is the up to date, most up to date.
So now we are good to copy this software. So we will copy this dump.rdb to Remote Server
which is itpanther04 and also we'll copy it to the home directory. Okay.
So this is copied and now we will connect to new server which is itpanther04
You can verify the IP address.
So now we are connected to this New Server.
okay.
So here again go inside where Redis-stable and make sure the configuration is as per the New
Node.
Okay.
So i am opening Redis.conf file and here
make sure whatever the bind address,
yeah whatever the bind address, IP addresses is. This should be the new server's IP address.
So here I have changed it from five to six.
Okay because IP address of this server is 6
And if you want to make any changes here in the log file you can make.
But I just follow the standard. So I'm keeping you in /tmp/redis.log
and the most important step is, this directory should be pointing to the directory where you have the
latest dump.rdb file.
Because when you start this server it is going to pick that latest dump.rdb to copy everything
and restore.
OK so if it is not able to find dump.rdb, it is going to assume that there is no data and it
is just going to start blank server, basically which will not have any data and after that make sure in the
replica of
Since we copied this software from Slave Node. So we have replica of enabled. We need to disable it, because
we don't want this Sever to become a Slave Node. This ever is going to be a Master Node.
OK.
So I have commented the replica of line. Ok you can verify that here, in this line. So I will save and exit
and now we can go to src directory and we are good to
start Redis.
OK.
So again to start Redis, you do ./Redis- server
after going inside src directory. Let me clear the screen.
And this is the current directory, which we are now
and also provide the full path of the Redis.conf file.
okay.
Now we can check the logs
OK.
So this server has started properly and also now we can connect to this Server and we can verify that we
have the data restored.
Okay.
So we can see whatever dump.rdb file was there, it has restored data from there.
OK.
We see all the data which we had put in Color key.
We can see here, but please remember right now this node is working as a, you know, independent Master Node and
still the previously Slave node is not pointing to this new Master Node.
So we need to connect to that previous Slave node, by using redis-cli -h provide the
IP address of previous Slave node, which is 10.128.0.5
Hit enter and here you need to change this and point it to the New Node. So the way to do that is you
Slave of command, which is slaveof and then you provide the New Master's IP address, which is going to
be 10.128.0.6
And also you provide the Port Name. So once you hit enter, it says Okay. Now you can quit from here.
Okay, so now let's log into the New Master, which is on IP 10.128.0.6
and now let's see the value again, which we had previously in the Color Key and now to verify that we are
able to write or perform write operations on the Master Node. I am going to add some for some more values
into this Color Key.
You see we are successfully able to write more values to this key.
Now I'm going to quit from here and we will go back to the Slave Node and we will see that new data
has been replicated there or not.
Okay so Slave Node is running at 10.128.0.5
And if I do L push here. You can see I'm not able to you know, write data
here because
this is a Slave Node. However I will do L range now and see what is the new value of Color Key.
Okay, so now we see list has been updated and now we are having couple of more items, which we had added.