So now you understand that the topic may have multiple partitions, those partitions are spread among
different brokers and managers inside of every politician must have unique of that number and of set
numbers.
I started from zero.
So our first message inside of every politician has of set number zero new messages appended at the
end of existing messages inside of every partition.
But there is a problem if any of the brokers fails is partition simply disappears and messages that
were stored in that partition also disappear.
That's why it is possible to create replicas of partitions.
And in.
Let me talk about that.
So here you see a diagram with three brokers, brokers zero, broker one and broker two.
And there is topic A and notice that in this topic there is partition zero.
Let's suppose that there was just single partition so far and you see the same partition partition zero
that was created on brokers zero and broker to.
But those partitions are created as follows partitions.
So this broker and this broker, followers and broker one is leader for this partition and job of followers
is simply to get new messages from the leader and ride them into specific partition.
That's all.
They don't accept any riden requests for this partition from producers.
Also, they don't serve consumers.
Producers and consumers communicate only with Broker One when they want to write or read from Partition
Zero.
And that's why partition zero on this broker one is called Leader Partition, or this broker is called
leader for this particular partition.
And again, when, for example, new message arrives to partition zero and broker one, it accepts write
request from producer creates new message here in its partition zero and replicates this same message.
Message number two, for example, to partition zero and broker a zero here and to broker to partition
zero here.
And same message number two is created here, here and here.
And now there are two replicas of the same message.
And if broker one fails now one of those brokers is our broker or broker to become leader for the same
partition, partition zero and no, for example, Broker Zero will still continue the operation and
will accept new writing requests to this partition and accept Reid's request from consumers.
That's how you are able to achieve for tolerance using replication.
Again, Main India behind this application is that leader performs most operations.
It communicates with producers and consumers and also it sends copies of every message to all followers.
Followers simply sit and relax and wait for new messages.
And as soon as new message arrives from the leader, their job is to write this message to partition.
And that's all, nothing else.
That's why you need to blend resources of the brokers accordingly, because if there will be multiple
replicas for every partition, for every topic then logged on brokers who will be responsible for lead,
the role will increase hugely increase.
So please keep all of that in mind.
All that is recommended to create the at least the two replicas.
I mean, if you want to create the full all around architecture, you need to have at least multiple
brokers.
And if you want to create two copies of every message, you need to have at least three brokers and
you need to configure a so-called replication factor on a topic level.
Basically, it means that you are not able to set up a replication factor for every specific partition
inside of the topic.
Instead, you must configure it on a topic level basis and by default, replication factor is a set
to one.
And that means that every message is stored only once, only on one broker.
And if you want to achieve such architecture where every message from a leader will be replicated twice
to two different brokers, then replication a factor must be set to three.
And that means that every message will be stored three times on three different brokers.
It is actually a recommended number for production environments and you should not go further.
And the set of application de facto to a four or five and so on.
So three is completely enough and this number will tolerate shutdown down of two brokers of three that
we're storing replicas of every message in this case, broker one and broker two may fail and broker
a zero will still continue operation and it will still keep old messages in partition zero.
And if I go back to this diagram where we had three partitions, partition zero, one and two, and
if you stop replication of fact on three, then basically we will have nine partitions instead of three,
but six of them will be, let's say, passive, and they will be created as four in partitions and the
leaders will write messages, replicate messages to those following partitions.
But as you see, with replication factor in place, quantity of partitions basically multiplies by replication
of factor.
So here now we have three partitions, but with a replication factor.
Three, we need to have nine partitions.
OK, that's all about Leedham, leader of partition, and that's all about the replication.
I hope it is clear.
And if you use multiple brokers, multiple partitions in each topic and.
A factor you might achieve really nice results and you may build real nice for tolerant zylon to Kafka.
Kloster again, this all about this topic.
And next, let's going to talk about control.
I'll explain you what is controller and what is his job.
So we'll see you next.