1 00:00:04,900 --> 00:00:10,540 So in this session let's talk about AOF, which is another approach of Data Persistence in Redis. 2 00:00:14,770 --> 00:00:24,130 So just a quick recap. In our previous session, we used RDB Persistence Mode and we specified we made 3 00:00:24,130 --> 00:00:30,070 some changes in the Configuration Directory, where we specified that the Dump should be kept at 4 00:00:30,070 --> 00:00:31,630 Redis_backup location 5 00:00:35,160 --> 00:00:38,350 and if we go on this, 6 00:00:38,360 --> 00:00:43,480 I mean if we go at this folder, we can see there is a file which is called dump_redis.rdb and 7 00:00:43,910 --> 00:00:50,540 this File Name was also specified in, Redis.conf, Configuration File. OK. 8 00:00:50,570 --> 00:00:54,140 So let's stop the Redis process 9 00:01:10,150 --> 00:01:13,840 and then let's go back to the Configuration File 10 00:01:22,630 --> 00:01:23,830 Let me clear the screen. 11 00:01:23,830 --> 00:01:24,130 OK. 12 00:01:24,160 --> 00:01:34,000 So we need to go inside stable folder, Redis-stable folder and here we have Redis.conf 13 00:01:34,000 --> 00:01:34,330 File. 14 00:01:34,710 --> 00:01:34,990 OK. 15 00:01:35,020 --> 00:01:45,060 So I will open this file and first thing which I'll do is, I'm going to comment these statements 16 00:01:45,090 --> 00:01:50,090 because we do not want to use RDB Snapshotting. 17 00:01:50,100 --> 00:01:51,810 Basically we are going to use AOF. 18 00:01:51,900 --> 00:02:02,960 So in that case you can comment these lines, because if you keep these specified, then it is still going 19 00:02:02,960 --> 00:02:07,460 to snapshot based on whatever the configuration which has been written here. 20 00:02:14,520 --> 00:02:14,850 OK. 21 00:02:14,880 --> 00:02:21,390 So once that is done, now it is not going to, you know, perform RDB Snapshotting. 22 00:02:25,470 --> 00:02:30,630 Remaining all other things you can leave as it is, because it doesn't really matter, because you haven't 23 00:02:30,630 --> 00:02:39,120 really provided the Snapshotting Mechanism or the Check Point. It will Snapshotting should perform. 24 00:02:39,150 --> 00:02:44,140 So even though if you leave this file, it is not going to make any difference. 25 00:02:44,400 --> 00:02:50,250 And now this is the directory where we have specified and this was specifying the location of the 26 00:02:50,280 --> 00:02:52,980 dump_redis.rdb. 27 00:02:52,980 --> 00:03:01,170 So by default, AOF file is also going to be stored at the same directory. Okay whatever directory is specified 28 00:03:01,170 --> 00:03:01,570 here. 29 00:03:01,650 --> 00:03:05,940 It is the same directory, inside which AOF files are going to be stored. 30 00:03:07,030 --> 00:03:10,320 However, you do have option to change the Name of the AOF file 31 00:03:14,000 --> 00:03:21,370 So if you scroll down, let's try to find the information related to AOF 32 00:03:25,210 --> 00:03:27,770 Or rather I will just search for the AOF Keyword. 33 00:03:28,600 --> 00:03:34,190 And now we move on to the location, where we have AOF file name specified. 34 00:03:34,470 --> 00:03:40,320 Okay so this is the name of the file, or AOF file which is going to be created. 35 00:03:40,800 --> 00:03:52,940 So I'm also going to change this to different file name. i will call it Redis_backup.aof 36 00:03:52,990 --> 00:03:54,460 Okay. 37 00:03:54,470 --> 00:03:56,980 So this is the AOF file, which is going to get created. 38 00:03:57,140 --> 00:04:01,340 And right now if you see, by default appendonly is set to no. 39 00:04:02,010 --> 00:04:02,290 Okay. 40 00:04:02,330 --> 00:04:05,960 That's why Redis doesn't perform any AOF backup. 41 00:04:05,960 --> 00:04:09,280 So I'm going to change it to Yes. Okay. 42 00:04:09,290 --> 00:04:17,510 So what these two configuration will do is, it is going to tell to Redis that append only or AOF 43 00:04:18,050 --> 00:04:26,180 Persistence is enabled and also all the instructions, which is going to be stored in 44 00:04:26,180 --> 00:04:33,110 append only file, that should be stored at this location. The location which we specified in directory and 45 00:04:33,110 --> 00:04:33,850 in this file. 46 00:04:37,960 --> 00:04:38,370 okay. 47 00:04:38,390 --> 00:04:40,370 If you scroll down you are going to 48 00:04:43,970 --> 00:04:50,720 get the location where append fsync has been specified. So by default appendfsync is set to every 49 00:04:50,720 --> 00:05:00,280 second, which means AOF is basically going to, every second is going to log everything. 50 00:05:00,360 --> 00:05:05,850 So whatever has been or whatever has been written to Redis or whatever keys have been changed, those 51 00:05:05,850 --> 00:05:08,910 are going to be logged in to AOF file. 52 00:05:08,960 --> 00:05:16,360 every one second. If you change it to Always. In that case, it is going to be very slow I mean the performance 53 00:05:16,360 --> 00:05:22,630 is going to be impacted because whenever you perform or whenever you make any key changes that is going 54 00:05:22,630 --> 00:05:25,950 to be logged into that File, AOF File. 55 00:05:26,230 --> 00:05:29,890 So usually the best approach is to, just keep Every Second. 56 00:05:29,920 --> 00:05:37,330 So in this case even if your system crashes, the only data which you are going to lose is maximum of 57 00:05:37,330 --> 00:05:38,490 one second of data. 58 00:05:39,330 --> 00:05:39,570 OK. 59 00:05:39,610 --> 00:05:42,320 So if that is OK you can go ahead with this approach. 60 00:05:42,370 --> 00:05:49,370 If this is not okay, then you can or you do not want any data loss, then you can go ahead with append fsync 61 00:05:49,380 --> 00:05:51,330 set to always. 62 00:05:51,340 --> 00:05:53,860 So in this case I'm just going to leave it to Every Second 63 00:05:57,820 --> 00:06:02,290 and we are going to save it and go back. 64 00:06:02,290 --> 00:06:02,540 Okay. 65 00:06:02,560 --> 00:06:07,990 So right now, we're in this location but it doesn't really matter, which location we start Redis from. 66 00:06:07,990 --> 00:06:13,300 So I have moved to Home Directly of this user. I'm going, I'm going to start Redis. 67 00:06:18,500 --> 00:06:24,110 So when I'm starting Redis, I'm also providing the full path of the Redis Configuration File. If you 68 00:06:24,110 --> 00:06:28,550 do not provide this, Redis is just going to assume, there is no Default configuration File. 69 00:06:30,260 --> 00:06:40,100 Okay so whatever changes you have provided, is not going to get reflected. 70 00:06:40,360 --> 00:06:46,340 So I have ran this and let's connect to Redis by using redis cli 71 00:07:02,550 --> 00:07:02,850 OK. 72 00:07:02,880 --> 00:07:04,940 So right now this DB is empty 73 00:07:12,890 --> 00:07:19,180 and if I go back to this location, you see there is already a redis_backup.aof 74 00:07:20,590 --> 00:07:23,500 file , which has been created and right now the size is 0 75 00:07:23,530 --> 00:07:27,670 So if I cat this file and if I see what is the content in this file. 76 00:07:27,670 --> 00:07:29,140 So right now we do not see anything. 77 00:07:29,190 --> 00:07:29,440 Okay. 78 00:07:29,440 --> 00:07:31,270 This file is completely empty. 79 00:07:32,970 --> 00:07:40,130 Now let's perform some Redis write operations. So yeah lets connect to Redis again using 80 00:07:40,130 --> 00:07:40,720 Redis cli 81 00:07:45,550 --> 00:08:00,020 and I'm going to write some Country Short Codes along with the Country Names. 82 00:08:00,310 --> 00:08:05,470 So basically whatever instruction, which has been written after delay of 1 sec, is going to get written 83 00:08:05,470 --> 00:08:08,150 to redis_backup.aof 84 00:08:08,240 --> 00:08:18,430 So if I exit from here and if I do ls-lrt here again. Now you see redis_backup.aof 85 00:08:18,430 --> 00:08:27,150 size has been changed to 9. Okay let's see the size 86 00:08:27,170 --> 00:08:28,720 Yeah it has been changed to 91. 87 00:08:28,730 --> 00:08:32,930 And let me also get rid of this 88 00:08:32,940 --> 00:08:33,620 dump redis.rdb 89 00:08:33,870 --> 00:08:35,900 We are not no longer using this. 90 00:08:40,220 --> 00:08:41,130 OK. 91 00:08:41,360 --> 00:08:41,550 Yeah. 92 00:08:41,580 --> 00:08:46,070 So basically now we have only 1 file, which is called redis_backup.aof 93 00:08:47,740 --> 00:08:49,440 And if I perform More Operation 94 00:09:01,850 --> 00:09:08,460 and if I exit and if I see the size again. You see the size has been changed again. 95 00:09:08,780 --> 00:09:10,620 The reason is whatever 96 00:09:10,670 --> 00:09:12,430 commands, which you have been writing here. 97 00:09:12,440 --> 00:09:16,220 That is actually being inserted into this log file. 98 00:09:17,380 --> 00:09:17,860 OK. 99 00:09:17,870 --> 00:09:25,230 So this is a Text kind of File. So you can open this file using cat Editor any other editor. 100 00:09:25,640 --> 00:09:32,960 And here you can see all the commands, which we have basically requested to Redis, has been saved here. 101 00:09:35,120 --> 00:09:35,530 OK. 102 00:09:35,540 --> 00:09:39,210 So this is how the data gets written to this 103 00:09:39,260 --> 00:09:41,390 redis_backup.aof File. 104 00:09:46,890 --> 00:09:48,770 Now let's insert some large data. 105 00:09:59,570 --> 00:10:04,110 So whatever the data is there in this country_code_names.csv file, I'm 106 00:10:04,130 --> 00:10:05,690 going to write it to Redis. 107 00:10:24,570 --> 00:10:24,830 OK. 108 00:10:24,870 --> 00:10:34,600 So there are 249 Key Value Pairs, which has been changed. Now if I go back to Redis_backup directory. 109 00:10:38,960 --> 00:10:45,350 and I see the size of redis_backup.aof file has changed again. 110 00:10:45,350 --> 00:10:51,560 And if we see the content in that file. We can see contends has also been updated and we can see 111 00:10:51,560 --> 00:10:58,920 the statements, which are key value pairs, which is available in that AOF file. 112 00:10:59,460 --> 00:11:07,830 So now what Redis is going to do is, it will keep on appending all the instructions which we are passing 113 00:11:07,830 --> 00:11:10,460 to Redis and going to make an entry here. 114 00:11:12,420 --> 00:11:18,270 And if I want to, let's say, whenever Redis is crashing and we are restoring. All the data which is available 115 00:11:18,270 --> 00:11:25,050 here in redis_backup.aof is going to be played back and then this is how Redis is going 116 00:11:25,050 --> 00:11:26,810 to restore all the data. 117 00:11:27,120 --> 00:11:31,410 So let's perform a simulation of Redis crash, by killing this process 118 00:11:38,320 --> 00:11:39,550 so i have killed Redis. 119 00:11:39,600 --> 00:11:45,890 a process now. Actually we try to connect to Redis, we see we'll not be able to connect to Redis 120 00:11:47,160 --> 00:11:47,460 OK. 121 00:11:47,460 --> 00:11:50,940 So connection is refused, which means Redis not running 122 00:11:57,380 --> 00:12:04,040 and now let's start Redis again by, giving Redis-server command. 123 00:12:08,880 --> 00:12:15,080 and also we need to provide full path of the Redis.conf configuration file 124 00:12:18,710 --> 00:12:18,990 OK. 125 00:12:19,020 --> 00:12:21,330 So this has started Redis process. 126 00:12:21,330 --> 00:12:25,910 Now if I connect using Redis cli, I'll be, I should be able to connect 127 00:12:29,630 --> 00:12:29,980 okay. 128 00:12:29,990 --> 00:12:37,040 And if I do dbsize, you can see whatever changes which were there, it has already been restored and Redis 129 00:12:37,040 --> 00:12:39,880 has played back that AOF File 130 00:12:39,890 --> 00:12:41,690 And we do not see any Data Loss. 131 00:12:45,750 --> 00:12:45,980 okay. 132 00:12:46,010 --> 00:12:48,590 So whatever data which we had inserted, it is there. 133 00:12:58,640 --> 00:13:05,930 and also as we learn that this Redis or whatever AOF File, which we have created it is possible which 134 00:13:05,930 --> 00:13:13,410 means you can change the instructions in this. So let's try to change some instructions and then restore 135 00:13:13,410 --> 00:13:18,490 this file. So I will log into Redis Client again 136 00:13:28,690 --> 00:13:29,140 okay. 137 00:13:29,160 --> 00:13:35,850 Just make a note of this dbsize. So right now we have 248 number of keys which is there in Redis 138 00:13:35,880 --> 00:13:42,240 And now I'll exit from here and I will make changes in this file. 139 00:13:42,540 --> 00:13:42,840 Okay. 140 00:13:42,840 --> 00:13:43,260 So 141 00:13:57,790 --> 00:14:17,840 let's remove some of the entries. 142 00:14:17,860 --> 00:14:23,180 All right, so let me remove these Entries. So whatever we see here, 143 00:14:23,440 --> 00:14:26,190 this set instructions. I'm going to remove. 144 00:14:42,550 --> 00:14:46,640 So couple of set instructions, I have removed from this AOF file 145 00:14:51,500 --> 00:15:00,740 and now if we stop Redis and start Redis. It is going to be restored using this file and also the 146 00:15:00,740 --> 00:15:04,610 total number of keys, which is now available in this file, has been reduced. 147 00:15:04,610 --> 00:15:06,190 It is not going to be 248 148 00:15:06,200 --> 00:15:07,190 So let's check that 149 00:15:23,410 --> 00:15:24,490 so i have stopped 150 00:15:24,500 --> 00:15:24,990 Redis 151 00:15:25,000 --> 00:15:26,680 And then I have started Redis again. 152 00:15:38,110 --> 00:15:42,400 And let's say, if we are able to connect to Redis, by using Redis cli 153 00:15:45,770 --> 00:15:46,270 okay. 154 00:15:46,300 --> 00:15:48,440 So we are able to connect to Redis. 155 00:15:48,460 --> 00:15:49,810 And if I dbsize, 156 00:15:50,320 --> 00:15:56,020 So now you can see, since we deleted some entries from that 157 00:15:56,060 --> 00:16:01,900 redis_backup.aof file. This time when I see dbsize, after restarting 158 00:16:01,900 --> 00:16:09,380 Redis. I see less number of entries here or less number of output when I do a dbsize 159 00:16:09,380 --> 00:16:09,710 command.