I'm also going to show you some of the important parts about the Demons said configuration as well,
so you have a bit of a more understanding of how it works.
So first, let's check the demon said we can simply do edit.
And here you see the YAML file or the configuration file of the demon set.
And of course, you don't have to understand all this stuff.
One thing that I want to point out is this part here.
Right.
So the volumes these are all the volumes that are defined for the pod replicas of the demon said in
here you see something called Vare Liebe Docker Containers.
And the Perth host, Perth, so to say, is slash bar, slash the slash dockers slash container's.
So this is an important part because every worker node has exactly this folder on the server where the
container logs are being saved.
So each container will get its own uniquely named folder within these containers folder and that will
contain all the logs that the container is logging, including our Java and node web containers.
And this is how fluid can access the logs of those containers.
So that's an important part here.
And obviously this pod volume then gets passed on to the container volume at this level.
Right.
So we see it here as well.
And the second interesting part is this volume right here, which is called Fluence the config.
And this is a config map.
Right.
Which was also dynamically created using this Helme chart.
And the config maps name is fluently forward to see him.
So this is the configuration that we have to adjust.
And in this configuration, we define how fluent this should process the logs that it reads from these
containers folder.
Right.
So let's actually go ahead and look at the contents of this configuration file.
In the config maps section.
You see we have a fluency for the configuration map.
This is the one we'll be adjusting.
The other one is for the aggregator.
And this is that fluent stateful set which was also deployed.
We're not going to touch that or adjust that.
So we're going to leave that alone.
So let's go to the folder and here are the contents of this configuration file.
We can also do the edit here so we can see better so fluently config.
This is the configuration file that gets mounted into each fluent part so that fluent knows how to process
the logs and what to do with them.
This configuration has three four most important tags and with those takes you can basically do most
of the configuration.
So the first one here at the top is basically tech, which would use a couple of times.
So all of this log entries are loaded into fluent right.
And these log entries actually have takes.
Each log entry has a name tag that contains the application name, pod name, namespace name, etc.
So using this method, regular expression, we can filter out or target specific applications or multiple
applications with the similar name, with the regular expressions.
So here what we are saying is we're targeting old logs produced by the fluent container itself and we're
just throwing them away.
This is what type NUL means, so we're not interested in them.
We don't want to collect or process them.
These are just some health checks.
We're not interested in them.
This is another interesting part here.
You see the source you saw in the volume configuration that we are reading, container logs from a path
called slash bar, slash, leap, slash dockers, slash containers.
And this is another path where those log entries are temporarily stored.
So we use that as a source for the fluency to process the data.
Again, we exclude any influence the logs.
And here we're trying to pass each log entry using a regular expression.
We're not going to go into detail here because we're going to change his regular expression.
But that's what the parse tag is for.
So you can pass each log entry of a different format using different regular expressions so you can
have JSON format.
You can have, for example, Ingenix will have its own format and so on.
In other tag that you see here is filter.
So what we're doing is we are targeting again, just like in match.
We are targeting all the logs which have Coronado's dot something tech.
And this is going to be all the logs coming from here because you see this line here, we are adding.
A prefix to each log entry called Communities Dot, so that we can filter them right here.
So what this filter here does is basically it catches all those logs coming from here and it gets them
some metadata from Carbonetti.
So it enriches these base logs with some additional data.
This data will be, for example, a pod name and pod ID, which the logs belong to.
This can also be namespace and so on.
And this final part here, basically what it does, again, you see the match take here, it matches
everything.
So it kind of it's like a funnel.
So it comes from top down.
So here we match everything and we're just forwarding them.
We're also going to change that apart.
So you don't have to understand each detail here.
But that's the configuration file.
It's not very beginner friendly, I would say.
But then again, you just have to understand a couple of takes and how this works, and you should be
good to go.
So now let's go and actually check the logs of fluently parts themselves and let's see what it actually
does using this configuration out of the box.
So let's go to our fluent teapots, these three right here, and let's actually open two of them so
we can see the logs.
And here, as you see, a lot of stuff gets locked here, so first of all, as you see here, there
is a whole bunch of Péter not matched messages.
Right, or warnings from fluently.
So what this means is actually it read all those log entries from the containers and using the pattern
matching or the regular expression that we have in this default configuration, which is this one right
here in the past, we have this regular expression.
This one doesn't match the log entry formats because our log entries are in JSON indices expression
for something other than Jason.
So that's why we have all these warnings.
We're going to fix that problem by changing that parser.
So let's go back to the configuration.
And we can edit it directly here, so I'm going to remove that pass completely and we're going to write
format, Jason, this is not going to pass the log entries.
We're going to do that later.
But this is just for formatting logs.
So if I update it, whenever I update a config map, I have to restart fluent parts so that they can
reload the new config map.
You can do that by an automated script.
You can do that by configuring the demon set to automatically reload it.
But I'm just going to restart the demon set with cubes.
It'll roll out restart.
And this is the name of our demon said, and this will actually restart demon set in its pots.
So here we see the status of pots restarting.
It will take a couple of seconds.
So now the pots have restarted.
We can actually check them again.
So here are the logs.
So we have a more clear output of the logs so we can actually see what's going on here.
So right at the top, we have a couple of interesting stuff.
The first thing to note here are these lines.
Here you see a couple of fluent plugins are loaded on the start up.
One of them that we are going to use is ElasticSearch.
This is the plugin we're going to need to connect to the plastic surgeon, push or forward those logs
to ElasticSearch.
Another one is this community's metadata filter.
This is what we saw in the configuration file right here.
This is a Carbonetti metadata plugin which enriches each log entry with Khubani data like pod name,
container name, port ID and so on.
And we also have multi format parser means.
If you have logs from different containers which are in different formats, like one of them is logging
in Jason, another one is logging in some other format and so on.
You can actually define a couple of different formats using this plugin.
And other thing to note here is that you have an output of the configuration file that is loaded by
fluency.
This is basically what we saw or what we edited here.
And the third interesting part in our logs are these entries right here, you see following tail off
and then there's Pece.
So all of this basically is a list of different container logs that fluently was able to collect or
read out of this log location.
So here you see var log containers elasticsearch master Ingenix.
So basically it has logs of every single container that is running in our cluster.
And here we also have the health check logs so we can actually clean up all the stuff that we don't
need so we can actually turn off the health check logs and also remove taling or collecting logs from
containers that we're not interested.
So basically we just want to collect the logs of Java and know that that's it in real production scenarios
in my differ.
So you might want to collect logs of Aldermaston processes or all the services like ElasticSearch and
so on, or you may want to limit your log collection only on your applications.
And the final thing that you may already noticed is this one right here you see following tale of var
log containers, Java EBP.
So this specific flew antipode right here is actually following and processing the logs of Java YPP
and there is no node EP on the list.
So if we go to next Fluence and depart and we check its logs, you see that it has, for example, Kibwana
logs in the third influenzae pod is actually the one that is collecting and processing our node.
EBP logs this one right here.
So how come one fluency part is collecting node web logs and another one is collecting Java epilogues?
The way it works is that, as I mentioned, one fluency part is running on every worker, node and node
and maybe running on one worker node in Java is running on another worker node so fluently will actually
exis and process logs of only those containers that are running on the same worker node as that specific
fluency pod.