So now we're going to deploy ElasticSearch Helme chart in our cluster, and the first step for that
is to go and look up the chart for ElasticSearch.
So whatever application you are installing, there is a chance that there is a Helme chart already for
it.
And this one is actually from elastic itself, which is another advantage of Helme charts, because
when official organization creates them, you can rely on the fact that they can do it better or better
than you putting all these configuration files and logic together.
So this is the repository for elastic Helme charts and they have one for each of different applications.
We're going to use Kibwana later.
Right now, we need the ElasticSearch chart and the installation is very simple.
You have a guide here, but I will put all the necessary comments in the repository so you can just
copy them from the commands file from there.
But just so you see, this is documentation with the installation commands.
And here in this section you see all the parameters that you can externally configure for your cluster.
So you have a pretty long list of things that you can actually change in the default values of the cluster,
which is pretty neat.
We're not going to need most of these parameters because we're not going to configure much.
But one thing that we need to configure is the volume.
So by default, our elasticsearch will just get some temporary volume.
But this is not what we need because obviously we need to process that log data that elastics stauss
permanently.
So even when the pods die, we want the data to be available.
If you don't know the concept of volumes yet, you can actually check out my YouTube video where I explained
that in detail.
But what you need to know for our setup is that we need to process the data and we're going to do that
using storage or physical storage on Lenard's servers in the set up is going to look like this.
We're going to install ElasticSearch Helme chart.
We're going to configure this attribute externally as a parameter and set it to Lenard's storage class.
Well, this is going to do is it's going to dynamically or automatically create volumes that we need
for each of three replicas of ElasticSearch and it will back them up with actual physical storage on
LYNARD servers.
So even if the pods die or we delete the whole elasticsearch chart and reinstall it or upgraded or whatever,
the data will still be there.
And the great thing about it is that with storage class for Lynard Block storage, you have minimal
effort for setting this up.
You don't have to create volumes manually.
You have to set up the storage and then connect them to the volume.
Everything just happens dynamically.
We just need to provide this value right here.
And the way we do that is also very simple.
In order to override any of these values in Helme chart installation, we need to create a values YAML
file where we just list all of these parameters and set them to our custom values or what we want the
values to be.
So here I have the values YAML file already created, so I'm basically overriding some of the resource
size attributes, which is not very interesting here.
This is the most interesting part.
The volume claim template is basically an attribute of stateful set.
So we're writing that exactly as it would be in stateful set.
Right.
So we have exis mode and resources here as well.
So basically how much storage you want to be able for each pod?
So each pod replica of ElasticSearch application will get storage size defined here.
And this is the interesting part.
Storage class name is basically Lynard Block Storage and that's it.
This configuration here will be enough to dynamically create the volumes we need to connect them to
lenard's storage, actual storage and save all the data and per system there.
That's it.
So now we have all the resources, we have the file.
We have to come in.
So let's go ahead and install elastic search for that.
We need just to commence.
The first one is we need to edit repository to help with.
Again, a package manager for communities, we need to add a repository that holds or contains our desired
chart and this is a plastic repository.
So let's go and do that.
It's been edited and now we have access to the chart, so again, note we're using Helme three.
This is the comment.
I'm going to explain that.
Yeah.
Second, so Helme installed is the come in to install the chart.
This is the name that we are getting.
So you can basically call it anything you want.
And this is the name of the chart.
And here is how we are going to overwrite those parameters using our values file.
So pretty simple.
We just say minus F, which is file.
And here we are going to pass our values YAML file and this will install ElasticSearch chart.
Note that you will find links to all the charts we use in this course in the git repository in Linc's
file.
Also important thing I have to note here is help install chart name will install the latest chart version
so it can be different from the version I am installing right now.
If you want to have the same versions that I have to be consistent with the tutorial, you can fixate
the version of each chart we install with Desh version Fleg.
I have added the commands with the exact versions in Git Repository so you can just easily copy them
from their.
You would just get some status data and now we can actually watch the components and different stuff
getting created.
And this may need some time to start running because it's pulling the images and doing all the setup
and now everything is running.
These are just the parts.
But we can also see what other components got created.
So let's do it all and let's actually filter on elastic.
So here we see our three pod replicas.
We see the services ElasticSearch Master and Master Headlam services and the stateful said that got
created shortly to these two services.
And the reason why you have two of them is that this is the service that load balances, so to say,
requests to the pod replicas.
So, for example, when you have queries from the data to get some information, you would be talking
to this service and this service will then forward that request to one of those pods, whichever is
least busy.
Now, the second one is important because when you were making update requests to ElasticSearch, meaning
your writing to its database, we want to address the master pod specifically because the master pod
is the only one that is allowed to write.
As I mentioned, they're both working on the same data or database.
So if we allow all the pods to write to the same data, we will get data inconsistencies.
That's why only the master pod is allowed to write and the others are basically synchronizing the data
from the master to get up to date data state.
Again, I explain the whole concept of how this works, including the concept of attached volumes in
far more detail in my YouTube video, which you can see in the link so you can learn more about this
concept from that video.
So overall, as you see with just one simple command of helm install, we have the whole elasticsearch
cluster running in our communities cluster and we don't need much elasticsearch specific knowledge in
order to make that happen.
And you can also see the overview of Deployed ElasticSearch in Dashboard.
So here, if you scroll down, you see the parts of ElasticSearch got created.
By the way, the name of ElasticSearch Mester for each pod may be misleading because only one of those
three pods is actually the master, which is usually the first one, the one with the zero index, the
other two, or if you had four replicas, the other three would all be slave pod, so to say.
And here you also have the stateful said, no, we configure the storage bakin for our elasticsearch.
Right.
So that means it must have created volumes in our cluster.
And these happen dynamically without our manual interaction, which is the best part, because doing
that manually, which is also possible, but it will be much more effort.
So we have the persistent volumes here.
Each pod gets its own persistent volume.
That's important to note.
And each of these volumes is connected to their respective pod using these claims.
So it's like an intermediate sort of sea between them.
The Connector and these claims are also dynamically created using this template that we defined right
here and here.
You see in the configuration storage class is Lynard Block Storage.
Now, that also means that a physical storage must have been created on Lenos service.
So if you go back to the node and volumes, you see three storage use or storage beacons were created
in the region that might cluster is running.
We have this size for each of those storage volumes, which we define here as well.
This is the storage and this is actually the path to actual folder or actual storage that will contain
our information on Lenard's systems.
To give you a graphical overview and to end a physical storage in some folder is created which gets
attached to work or node, which is right here.
So we have three working nodes.
Each one of them gets this storage.
Then respectively, three persistent volumes will get created in the cluster, which are backed up by
the storage, and they will then be passed into each pod using this claim.
So that's how the whole string of connection actually works.
And as I mentioned, if.
Stay or even if I remove the stateful set of ElasticSearch, these capillaries volumes and the LYNARD
volumes will all be there, even though the stateful said and the pots aren't running anymore, so your
data will be persisted.