And these next few videos we're going to look at ElasticSearch, I'm going to use Lustick Search seven
point ten.
Now I'm going to set up ElasticSearch DataSource, which connects to another server where I'm running
the ElasticSearch process, you might already have ElasticSearch on your network, but I'm going to
just install a minimal set.
Anyway, the demonstration elasticsearch is another popular tool use for monitoring infrastructure.
You may prefer every other data there is in this video.
I'm going to set it up and connect to do some sample queries.
And then in the next few videos, I'll set up Volbeat to read log files, which is similar to Lokey
and also demonstrate Metric Beat, where I'll monitor metrics from a Windows server.
And at the end you will be able to set up an ElasticSearch server in a simple configuration and set
up Volbeat Metrically Collectors'.
So ElasticSearch is very sophisticated and warrants its own course.
So I'm going to get to show you some of the basics.
It has collectors and those collectors are pushing data to the main service.
And then we have an ElasticSearch data source which will read from the main service.
Now ElasticSearch runs in the job of VM.
I'm going to recommend a minimum of two gigabytes of RAM.
You can run this on one gig of rent, but I haven't had much success doing that.
So what I've done is gone and created a new server for myself on Digital Ocean.
Two gigs of RAM and everything else doesn't really matter.
That's the most important thing for me.
And it is Ubuntu twenty point zero four and I have connected.
This is S.H..
Okay, so the instructions for installing ElasticSearch can be found here, so I'll be following the
Debian package manager instructions.
So this is where I got my information from, but I have put it on this page so I can copy and paste
it easier.
First thing I want the public and key.
Right, click presenter done instore dependencies, most systems will have this already, but I'm just
going to do it anyway.
I didn't need it.
OK, save the repository definition.
Press enter.
OK, and now I can update and install ElasticSearch package using IAPT.
OK, that is the elasticsearch server installed, but I haven't started or configured it yet, so we
can verify just by calling status that it is there and it can be found, but it is not running.
So that's OK.
That was expected.
Let's start the service now.
Now, if you used one gig of RAM, it might not start at this point, but I'm using to gigs of RAM.
OK, that looks good.
Let's check the status again.
Very good, active running.
So.
Actually, I can just verify that by using locally.
And we go, there is a response on the server.
Now, if you had problems, you can read the log files from General.
You elasticsearch looks pretty good to me if you had any areas that would show up there, OK, when
it was installed, it created a new user called ElasticSearch Pegasi ElasticSearch and they are the
processes that it's running.
OK, so now it's time to configure ElasticSearch, but we can KDAY into the ATC folder.
So c.D, ATC ElasticSearch, Alice El-Hage.
And these are the files, the configuration that I'll need to edit here is Lastic search y amount,
Goudeau and I.
ElasticSearch, why email they this scroll down and.
By default, the only way you can query the ElasticSearch server is through localhost.
As I just demonstrated, using KOLD before one twenty seven zero zero one, if I want to make this server
accessible from other servers, I need to change the bar and address.
So I'm going to use zero zero zero zero, which will bind to all network interfaces.
I also need to add.
Transport host, local host, and I'll leave the port as default, so control X, yes, and restart.
Now, the reason for doing that is I will need my data source to be able to connect to this external
server where I'm running ElasticSearch.
So if I didn't do that, then the only way I could connect to the server was if I had given or running
on the same server as well on localhost.
OK, so that is still restarting.
Starting and restarting and take 10 seconds, roughly my experience status or a good active of running.
Now, this also means that I can access this from the Internet.
I can access that external IP address of my ElasticSearch server con nine to hundred enter and it will
show a response in the browser.
Now, we should lock this down because there are services on the Internet that will spam this elasticsearch
server.
It doesn't take long to be discovered.
So straight away when I'm going to do is I'm going to restrict deport nine two hundred using IP tables.
So down here, I'm going to clear allow local host Access Port 9100, I'm also going to allow my Ghafar
server, which is at i.p.
That currently and I'm going to drop everything else.
Very good.
This time there are the rules there except drop.
Now, I shouldn't be able to access that in the browser that will time out.
OK, very good.
OK, so before we can connect to it in California, we'll need to create an index on it and have some
data in it, for example.
So let's have a look at the data source configuration.
So data sources do at a data source elasticsearch there is select that now.
Mihiro is going to be that Colen nine 200.
Now, the Ippei of Microfauna Server has been added to the IP tables, so they should bind it.
Down here, we have to set the index nine, which we haven't created yet, and also a time filled name
which has credit and I'm going to connect to seven plus.
So if I try to save that now, it's not going to work.
That's OK.
Let's do some configurations to ElasticSearch to create at least an index that we can query.
OK, so going back up here, I'm going to create an example index in more or less the search server
called index one down there.
So copy that and we'll put index address nine two hundred, index one.
And it's putting a new index presenter.
Very good.
Acknowledged true index one.
We can view the index metadata.
It's some information about the index, the.
I've used the pretty query string that just prints it out indented like that, you can leave that off
if you like, like so and it's not quite so easy to read.
It's up to you.
Let's add some data to the index so that we've got something to read inside Ravana.
OK, so clear, and if we go so pull content type application, Jason posting, that's post there to
localhost nine to the index, it's a doc.
And the data here inside these single quotes is the Jason.
Now, I'm creating two new metrics in there named ABC with a value one to three, another one called
Najm with the value X, Y, Z and a timestamp key in the timestamp.
I'm just getting the current dict written in oiseau format.
It's like every other data source of Carfagna will rate it once a timestamp against every row.
So this is what I'm doing here.
So.
Presenter OK, so that was created a successful one, but it's true.
Now just to demonstrate what this line does, again, if I was to just write data, I second in the
console, it just writes the current data in UTC.
So that's what I'm inserting into the timestamp column just up there in my index one.
So anyway, we can now view the contents of the index.
OK, there we go.
There's the contents of the index and the value that I just inserted is the ABC one, two, three name
and a timestamp thing.
Whatever was at the time we executed that line.
OK, so let's look at that value there and that value.
They're now going back into the data source configuration ElasticSearch.
The index name is Index one, and that's my timestamp field I'm using at Timestamp, the scientist that
we go index.
OK, excellent.
Right now explore ElasticSearch.
OK, so straight away it's shown me that there's something there, count one behind my head so we can
zoom into that and look at that as raw data.
And there we go.
There's my line that I just inserted before indexed one TalkTalk ABC one, two, three nine x, y,
z.
At that time stamp, it's to another one.
OK, this time I can do something else like five, six am, and, oh, anything really does matter.
A successful one by refresh that at five seconds and look at that for the last five minutes, there
are two rows.
Now there's the second row there straight away.
You could right now a Python script or anything else that can send data or anything.
You like as many metrics as you like to that elasticsearch server.
And you can read them in Safonov now and you can create a dashboard from that.
So just very quickly, dashboards and each write a new dashboard at a new panel.
These elasticsearch would go and save that.
ABC and we go or read another one.
So as I.
ElasticSearch as a table.
And we go.
Look at the raw data.
Go save that five.
I go and there we go.
So excellent.
Last five minutes or so already.
That is an example of putting data into LASTIC search and reading it in a dashboard microphone.
OK, now you might have a lot of indexes in your elasticsearch so you can check those pool cat indices.
I have one called index one.
Now, I can also delete an index.
So experiment as much as you like.
And then when you're finished you can always delate index, one, two, three, etc..
I'm going to leave it there for now.
OK, remember also for your service on the Internet like mine, and you use IP tables and you restart
your server IP tables, rules won't be sites, so you'll need to save them time.
Yes, I'm just installing I type was persistent.
Yes, yes.
OK, very good.
Now, when I reboot my server ever, the IP table's rules will be reapplied.
Okay, so excellent.
So whether you use ElasticSearch or not is up to you.
ElasticSearch has many different options for licensing, so you can either try the free version which
I used or you can pay for it or register, etc., etc. far.
OK.
In the next video I'll set up File Beat and I'll read system logs from my server.
So it will be similar to what I did with when setting up the Lokey datasource impromptu but being done
using Lastic search.
Volbeat excellent.