OK, now, you know, structure of the message to distort and Kafka brokers, and now is a good time
to go on and talk about how topics are actually stored on brokers and what is partition.
So every topic may exist on different brokers that are included in a Kafka class.
For example, here on this diagram, you see that topic A is present at Broker Zero and and broker one.
And Topic B is present on Broker one, broker and broker at three.
Another topic, topic C is present only at broker to notice that on this diagram I have removed the
zookeeper for simplicity.
But it is always there without zookeeper, Kafka broker is not able to operate.
So you might ask me a question.
Why do we need the same topic on different brokers?
Why don't just to create a single topic?
A single broker answer is pretty simple for fault tolerance.
If in such case Broker two will fail, old messages in Topic C will be lost and no one will be able
to produce new messages to Topic C and consumer messages from this topic because broker that was serving
old messages inside of this topic, topic C is gone.
In this case, when we talk about topic A and if, for example, a broker or zero fails, broker one
will be able to still save you messages that will arrive to topic EHP and it will able to sell reasonable
requests from consumers that will ask for data from this topic.
A but how?
Actually messages are spread among different brokers in such case, when same topic is present on different
brokers.
For that, Kafka uses partitions.
And here in this example, you see that topic A has two partitions partition and partition one one partition
is located and broker zero and second one is located.
Broker one.
If we will talk about topic B, it has in this example three partitions starting from partition number
zero.
So this partition is located and broker number one, this partition number one is located at another
broker, broker two and Partition two is located at the last broker number three.
Topic C has this example, only a single partition partition zero, and it is located only on a single
broker.
Also this diagram you see one more topic topic deal that has to partition partition zero and partition
one that allocated both on one broker, a broker or zero.
It is also possible also you are able to create the topics with hundreds of different partitions that
will be spread among multiple brokers.
But what is an idea behind creation of those partitions?
Why don't just create the topic was just single partition as shown here in this example?
Because, as you know, every message is read into the file and file is stored on a hard drive of every
computer.
And if there will be a lot of messages that are produced simultaneously by many producers, single computer
will be able simply to write the such amount of data weekly to the hard drive.
This way, if there will be multiple partitions spread among different computers, they will perform
right operations of matter just much, much more quicker.
The same relates to read operations.
There are many consumers and if, for example, one thousand consumers will try to read data from Topic
C Partitions zero, this program will simply go down due to lack of resources.
But if this topic will have, for example, hundreds of partitions spread among different brokers,
this job of supplying data to consumers will be much, much easier.
That is the reason for creation of different positions that are created basically on different brokers.
Also, it makes topic for tolerant.
And in this example, you, for example, broker a zero will fail topic A will still be present on Broecker
one and it will still accept new messages and accept the new read request from consumers.
That's why partitions are needed.
Also in this diagram, I have shown you this topic.
Barbaras in order to make you understand that multiple partitions in the same topic may exist on a same
block.
It is possible.
OK, let's proceed.
And next, let me explain you how messages are spread among different partitions when they are produced
by producers.
So see you next.
My.