WEBVTT 00:04.380 --> 00:10.440 I have just explained the concept of partition in the topic and the partition is actually increased 00:10.440 --> 00:16.410 performance of entire Kafka last, and they optimized the read and write operations because now they 00:16.410 --> 00:18.390 are spread among different Saras. 00:18.600 --> 00:25.080 And also they increase the full tolerance because in case when some rules fail, Özer will continue 00:25.140 --> 00:27.470 processing of messages inside of a specific topic. 00:27.720 --> 00:33.210 And no, let me explain you how messages are stored inside of the partitions recorded in the previous 00:33.210 --> 00:39.870 practice section, we have created a topic called c.D and we didn't specify one of the partitions we 00:39.870 --> 00:41.400 want to create for that topic. 00:41.670 --> 00:45.510 And by default, in such case, only a single partition will be created. 00:45.600 --> 00:51.720 And you have seen that the Kafka broker has created a folder with may see this Bessel zero that the 00:51.720 --> 00:54.590 story's all messages for this particular partition. 00:55.020 --> 01:00.060 That means that every partition is simply a separate folder with files, nothing else. 01:00.420 --> 01:07.140 And if there are multiple partitions spread among different brokers, then every broker may have one 01:07.140 --> 01:12.930 or multiple folders for every partition like it zero it is once, it is two and so on. 01:13.110 --> 01:19.020 If you create a topic with multiple partitions and if there are multiple partitions on different computers, 01:19.200 --> 01:23.070 producers might write messages to different partitions. 01:23.400 --> 01:29.670 For example, in this case there is topic eight with three partitions partition zero one and two. 01:29.880 --> 01:35.490 Notice that the number of the partition is always zero and you're not able to adjust it. 01:35.620 --> 01:38.400 It is behavior does the behavior of a class. 01:38.520 --> 01:41.040 It creates partitions starting from number zero. 01:41.310 --> 01:47.520 For example, if you create the topic with ten partitions, then last the partition number will be nine 01:47.520 --> 01:50.490 and the number of the fourth partition will be zero. 01:50.820 --> 01:54.270 So in this case, there are three partitions, zero, one and two. 01:54.480 --> 01:58.410 And here you see how those partitions are distributed among those brokers. 01:58.860 --> 02:06.340 And notice that the offset numbers instead of every partition are unique and starting from zero. 02:07.050 --> 02:12.990 So, for example, here in this partition, there are only two messages and offset of those messages 02:13.020 --> 02:14.490 are zero and one. 02:15.090 --> 02:20.510 So when Thrillist message arrives to empty partition, it gets offset zero. 02:20.940 --> 02:25.800 Next one will get offset one and so on, two, three, four and so on. 02:26.580 --> 02:33.870 Offset numbers must be unique across partition, not Dobek, because here in this fourth partition, 02:33.870 --> 02:39.750 you see that that also has offset numbers from that to just zero and one same as in this partition. 02:40.200 --> 02:44.160 But those messages are completely different from those messages. 02:44.310 --> 02:49.380 They are simply different messages and same you see here in Partition two. 02:49.650 --> 02:54.000 It's currently four messages with or says from zero to three. 02:54.510 --> 03:01.380 And main idea I want to communicate to you using this diagram is that area partition must have unique 03:01.380 --> 03:03.000 numbers across all messages. 03:03.000 --> 03:05.550 Inside of it is very, very important. 03:05.970 --> 03:11.220 But of the numbers of the messages across entire topic should not be unique in this case. 03:11.220 --> 03:17.190 You see that this message, this one and this one are different messages, but they have same offset 03:17.190 --> 03:20.130 number, but they are located in different partitions. 03:20.880 --> 03:25.710 So producers may write messages to different partitions. 03:25.950 --> 03:30.750 For example, for use of no one may write messages to partition zero. 03:31.200 --> 03:38.160 Producer number two may write messages to partition zero and partition one and producible number three. 03:38.310 --> 03:40.830 Might write messages only to partition to. 03:41.960 --> 03:46.850 Again, every producer decides which party to choose to write. 03:47.380 --> 03:50.630 It is very, very important producer decides that. 03:50.990 --> 03:56.480 And every message that arrives to specific politician, for example, let's say we sent a new message 03:56.480 --> 04:03.230 to partition one, it will be abandoned to the last message that already exists in this partition and 04:03.230 --> 04:07.020 new message in this partition, one will get of that number three. 04:07.670 --> 04:13.430 So now from this diagram, you see that using different partitions, we achieve parallel performance 04:13.430 --> 04:15.520 of writing and regional operations. 04:15.530 --> 04:22.090 So we may write to different partitions in parallel on different brockers same relates to operations. 04:22.550 --> 04:30.230 But now you may ask my question, but what if Broecker one will fail and partition one with all messages 04:30.230 --> 04:32.330 inside of it will simply disappear? 04:33.260 --> 04:36.210 In such case, those messages will be simply lost. 04:36.230 --> 04:42.170 I am talking about message zero one and two instead of partition one and you will not be able to consume 04:42.170 --> 04:43.570 those messages anymore. 04:43.760 --> 04:49.190 They were simply lost because they were stored only in a single place on a single broker. 04:49.910 --> 04:52.100 And there was of course, a solution for this problem. 04:52.250 --> 04:57.200 And you are able to replicate messages inside of every partition. 04:57.440 --> 05:03.470 And if you want to, you are able to replicate every single message inside of the topic multiple times, 05:03.470 --> 05:05.840 for example, five times or ten times. 05:06.170 --> 05:11.120 And in such case, copies of the same message will be stored on different broadcasts. 05:11.420 --> 05:17.250 And in such case, if one of those brokers fails, also, it will be able to sell the same messages 05:17.330 --> 05:20.920 to consumers, this idea behind replication. 05:21.050 --> 05:26.900 And now let's discuss the application in greater details and talk about leader and let's do the text 05:27.080 --> 05:27.380 by.