WEBVTT 00:04.450 --> 00:11.320 We were able to successfully start the producer and produce hundred messages, and we did that two times, 00:11.620 --> 00:15.430 and at the consumer we have ensured that we have received those messages. 00:15.580 --> 00:19.420 And here in the terminal, we have seen meaningful logs. 00:19.930 --> 00:27.580 And first of all, when producer starts, it generates such information with a set of parameters. 00:27.580 --> 00:28.810 It has started a week. 00:29.230 --> 00:33.010 But again, that we have specified only three parameters here. 00:33.020 --> 00:37.380 They are most observers, key serializer and Waialua serializer. 00:37.630 --> 00:40.840 And I have described what those parameters mean. 00:41.230 --> 00:45.700 And no, let me briefly talk about some of the parameters. 00:45.700 --> 00:51.100 I want to cover all of them, because there are many of them and a lot of them are actually used very 00:51.100 --> 00:54.500 often in order to describe those parameters. 00:54.520 --> 00:56.950 Let me go to the recommendation here. 00:57.190 --> 01:00.070 And we have actually opened before producer conflicts. 01:00.070 --> 01:07.150 And if you have not yet done that, please go to documentation and scroll down to the section configuration 01:07.150 --> 01:08.140 of the producer. 01:08.140 --> 01:15.510 A producer can fix this one and you'll jump to different parameters you are able to set for your producer. 01:15.760 --> 01:19.060 Notice that for every parameter there is importance. 01:19.060 --> 01:23.290 And I'll talk about parameters that have high importance. 01:23.590 --> 01:28.570 Some parameters have medium importance and some of them have low importance. 01:28.820 --> 01:32.530 OK, let's talk about parameters of this high importance in greater details. 01:32.950 --> 01:39.550 We have already talked about serializer and various serializer, but let me quickly sum up what those 01:39.760 --> 01:40.720 parameters mean. 01:41.020 --> 01:48.320 They specify which serializer interface should be used in order to encode key and value and usually 01:48.370 --> 01:49.960 key and value are strings. 01:49.960 --> 01:57.720 But if you want to encode, for example, into you should use integer serializer instead of string serializer, 01:57.880 --> 01:59.410 we will try the delay down. 01:59.770 --> 02:03.660 Next parameter is acknowledgments and is default. 02:03.660 --> 02:07.000 Timbalier is set the one here you'll see this information. 02:07.000 --> 02:12.140 Default value is one and possible values are zero one and all. 02:12.520 --> 02:15.190 Let me quickly explain what this value means. 02:15.430 --> 02:22.480 And for that, let me quickly go to diagrams and find diagram where we have talked about leaders and 02:22.690 --> 02:24.710 followers probably on this one. 02:24.730 --> 02:29.860 Let me make it a bit smaller like this and let me explain what this parameter means. 02:30.310 --> 02:36.070 Let's suppose that there is a topic with some protections and there is an obligation of factor configurator 02:36.070 --> 02:37.690 at the topic level basics. 02:37.840 --> 02:40.500 And for example, the application count is set to three. 02:40.660 --> 02:45.150 That's how we basically configure to our top economic in this example. 02:45.490 --> 02:51.070 And in such case, there was a leader and two followers and the leader is responsible for communication 02:51.070 --> 02:54.970 with producers and consumers and it performs all right. 02:54.970 --> 02:57.280 And really operations to specific Patison. 02:57.850 --> 03:01.840 And those follow rules just synchronize messages with leader. 03:01.960 --> 03:06.220 Actually, leader sends a copy of every message to each of the followers. 03:06.340 --> 03:11.770 And in such case, every message will be copied actually two times it will be stored at leader in this 03:11.770 --> 03:17.620 partitions zero and it will be copied to partitions zero to broker a zero and two partitions zero at 03:17.620 --> 03:19.450 Broka two and no. 03:19.450 --> 03:21.310 Let's get back to this parameter. 03:21.310 --> 03:22.570 UX acknowledgments. 03:23.020 --> 03:29.830 If it is set to zero, then producer will not wait for any acknowledgement from the server, from the 03:29.830 --> 03:30.760 broker at all. 03:31.120 --> 03:34.360 It means that every message will be just sent and that's it. 03:34.660 --> 03:40.390 If there is some network outage or Breuker is not temporarily available to process this message, this 03:40.390 --> 03:41.950 message will be simply lost. 03:42.550 --> 03:46.000 And this way it does not recommend it to use the disk at all. 03:46.150 --> 03:50.920 I mean, set equal to zero, it is recommended to use setting either one or. 03:50.920 --> 03:58.180 Oh, let me describe the difference between equal one and equal all set in only leader of the partition. 03:58.180 --> 04:05.230 In our case, Broecker one will send acknowledgement to producer after it will successfully write message 04:05.230 --> 04:08.170 to specific partition in our example, partition zero. 04:08.500 --> 04:16.900 But if you will use set in equal all this one, it will mean that leader will wait for a full synchronization 04:17.050 --> 04:18.370 with followers. 04:18.520 --> 04:25.150 In this example was Rocknroll zero and Broka two and only after it will receive acknowledgments from 04:25.450 --> 04:30.790 or zero and Broecker two that they have successfully read a specific message to their partitions. 04:31.270 --> 04:38.020 Only after that this Breuker leader for this partition will send acknowledgement to producer. 04:38.530 --> 04:46.300 That's what they said in equal all means, and it is actually most fault tolerant and with such that 04:46.630 --> 04:52.960 you will ensure that the message that is produced by producer will be written to Ledra and all sync 04:53.000 --> 04:53.590 replicas. 04:53.680 --> 04:56.890 That is the strongest available option among all of them. 04:57.340 --> 04:59.500 Let's quickly go to our project and observe. 04:59.500 --> 05:01.750 Well, of course, here it is, actually. 05:02.520 --> 05:07.620 Here on the list, and you'll see here one, and that means that this producer will wait for acknowledgement 05:07.620 --> 05:13.470 only from leader when a leader will write a specific record to specific protection grade. 05:13.500 --> 05:13.590 Great. 05:13.890 --> 05:14.870 This parameter. 05:14.890 --> 05:18.400 OK, let's scroll down to next parameter buster observers. 05:18.480 --> 05:22.180 That is actually mandatory and it doesn't have any default value. 05:22.380 --> 05:25.550 That's why you need to specify at least one silver. 05:25.710 --> 05:32.220 And in our example, we have specified actually three severals like this in the value we have separated 05:32.220 --> 05:33.960 them using commas like this. 05:34.200 --> 05:40.380 And here you see this parameter on the list and it shows us actually array of serros. 05:40.860 --> 05:41.250 Great. 05:41.460 --> 05:44.190 Next parameter is the buffer of that memorium. 05:44.340 --> 05:51.840 And this parameter specifies how many Mamoru producer may use to buffer records that are waiting to 05:51.840 --> 05:54.800 be sent to the celebra and default value is here. 05:54.810 --> 06:00.180 It is equal to 32 megabytes and the importance of this parameter is ultra-high. 06:00.510 --> 06:05.250 Next parameter compression type and by default there was no compression. 06:05.250 --> 06:06.060 Here you see that? 06:06.210 --> 06:10.740 And if I switch to our project, you'll see that here in this output as well. 06:10.890 --> 06:12.870 Compression the type equal number. 06:13.050 --> 06:18.720 And that means that there is no compression applied to while sending messages from producers to brokers. 06:18.810 --> 06:20.070 And that's actually not good. 06:20.070 --> 06:24.510 And it is always recommended to use compression in order to increase throughput. 06:24.510 --> 06:34.320 And the same benefits and possible values are josipa snappy Elzy four and zesty radwaste parameter specifiers. 06:34.480 --> 06:42.410 How many times producer will try to send the same message if there was some error and if it does not 06:42.510 --> 06:44.960 receive acknowledgement from the leader? 06:45.090 --> 06:49.620 If there was such sediq acknowledgment is set to zero. 06:49.620 --> 06:52.050 It means that we don't wait for any acknowledgement. 06:52.320 --> 06:57.600 Then a parameter actually doesn't matter and we simply send messages and that's it. 06:57.810 --> 07:04.800 But if acknowledgements is set to one or all, then retries does matter and default value is here. 07:04.800 --> 07:09.240 It is very, very large and producer will try to send every message many, many times. 07:09.240 --> 07:14.360 And if l go to our project and scroll down, I'll find retries somewhere here. 07:14.400 --> 07:15.150 Here was this. 07:15.150 --> 07:16.690 Well, it is very, very large. 07:17.100 --> 07:17.460 Great. 07:17.610 --> 07:18.330 Let's proceed. 07:18.330 --> 07:23.850 And the next parameter of high importance is actually as a cell configuration parameter. 07:24.000 --> 07:31.110 And there was a set of such parameters and by default, connection between producer and the broker is 07:31.110 --> 07:34.500 actually unencrypted and that's not always safe. 07:34.770 --> 07:42.630 And if the brokers and producers connect over open Internet connection and someone might intercept messages 07:42.630 --> 07:49.230 and the message of importance, then you should stop SSL encryption between producers and brokers. 07:49.230 --> 07:55.260 And for that you need to use several parameters like this one and all for other parameters related to 07:55.260 --> 07:57.000 configuration of SSL. 07:57.570 --> 08:03.780 Next parameter I want to talk about here is that batch size is important is the medium actually. 08:03.780 --> 08:09.330 And we have talked about all parameters of high importance, but which size is important parameter. 08:09.330 --> 08:11.340 And let me talk about it right now. 08:12.030 --> 08:15.030 Size is by default a 16 kilobytes. 08:15.030 --> 08:20.720 And if I go here, you'll see both sides at the beginning of this output here as this parameter by each 08:20.730 --> 08:22.410 size and its default value. 08:22.410 --> 08:29.220 Sixteen kilobytes and thus forming Riquelme that we have sent multiple messages one by one, and we 08:29.220 --> 08:31.620 have actually sent 100 messages. 08:31.770 --> 08:36.300 And it would not be nice if we sent every of those messages separately. 08:36.350 --> 08:43.800 A separate request to broker in such case that we need to actually 100 different requests and 100 different 08:43.800 --> 08:44.420 packages. 08:44.760 --> 08:51.990 That's not fair and it does not save several sources of brokers and does not efficiently utilize Ben, 08:52.110 --> 08:58.140 because every separate packet of dissent has its own header and that actually decreases throughput. 08:58.140 --> 09:00.210 And of course, it is not efficient. 09:00.540 --> 09:08.730 That's why batch size is used and the producer will try to batch Rackers together into fewer requests 09:09.000 --> 09:10.140 if it is possible. 09:10.530 --> 09:13.980 If multiple records are sent to the same partition. 09:13.980 --> 09:18.320 It is important, you know, that the partitions may reside on different brokers. 09:18.330 --> 09:24.900 And that's why producer may batch of records to different producers that host different partitions. 09:25.050 --> 09:32.400 And in general, if batch size is used, producer will send fewer packets and it will try to combine 09:32.400 --> 09:33.980 multiple messages together. 09:34.200 --> 09:40.950 But please note that in such case, messages may arrive a bit later after they were actually created 09:40.950 --> 09:46.650 on producer because producer will keep them for some time in cash and not below that. 09:46.650 --> 09:52.560 The small budget size will not make a batch and really efficient and it may reduce throughput. 09:52.770 --> 09:58.470 But advantage of the small budget size is that the messages will be sent almost without delay. 09:58.500 --> 10:00.690 They will be kept for a long time in cash. 10:01.260 --> 10:01.920 But if. 10:02.140 --> 10:09.400 There was a large bite sized configurable then producer will utilize memory and it may go out of memory 10:09.400 --> 10:14.320 if there are really many messages waiting in a buffer and that's the way it is actually recommended 10:14.320 --> 10:17.810 to keep default volume that is equal to 16 kilobytes. 10:17.950 --> 10:23.800 But again, if you want, you could increase it and make it like a several megabytes if you want this 10:24.130 --> 10:24.490 size. 10:24.910 --> 10:27.250 There are also a bunch of other parameters. 10:27.410 --> 10:33.460 If you have spare time, I recommend you to quickly go over this document and read about remaining parameters. 10:33.760 --> 10:35.000 Actually lost parameter. 10:35.020 --> 10:37.390 I want to talk about here is client ID. 10:37.720 --> 10:41.140 This parameter is set to empty string by default. 10:41.140 --> 10:44.650 And if I go to this output here, you'll see that. 10:44.650 --> 10:47.470 Let me scroll up and find the client ID. 10:47.590 --> 10:48.270 Here it is. 10:48.520 --> 10:56.320 And using this parameter, you are able to specify custom idea of the producer and if there are many, 10:56.320 --> 11:02.410 many different producers and if you want to know which producer has produced specific message, you 11:02.410 --> 11:08.080 could add this parameter in the configuration of producer and then you will be able to track, request 11:08.080 --> 11:12.490 and understand to which brokers send particular messages altogether. 11:12.520 --> 11:18.670 That's all what I wanted to tell you about parameters you could configure on a producer level and all 11:18.670 --> 11:23.170 of those parameters you could setup, same as we did with bootstrap servers. 11:23.290 --> 11:29.830 Serializer and various serializer simply use probes to put their own name or parameter that there should 11:29.830 --> 11:32.950 be equal to name specified here in documentation. 11:32.950 --> 11:35.710 For example, delivery time out dot miles. 11:35.890 --> 11:37.810 Then look which value is expected. 11:37.810 --> 11:44.230 For example, for this parameter, we need to specify time out in milliseconds and default value is 11:44.230 --> 11:44.710 this one. 11:44.710 --> 11:50.230 It is two minutes and every parameter you specify here will override default values. 11:50.890 --> 11:52.210 OK, that's all for this lecture. 11:52.210 --> 11:53.640 And next, let's deforming. 11:53.920 --> 12:01.980 Let's try to produce numbers, integers instead of strings here and see what we will get here in a terminal. 12:02.140 --> 12:02.970 I'll see you next. 12:02.980 --> 12:03.460 Bye bye.