For this shell scripting exercise we're going to write a script named show Dasch attackers.
SH And this script is going to require a file be provided as an argument and if a file is not provided
or we can't read it for some reason then what we're going to do is have the script display an error
message an exit with an exit status of one.
This group counts the number of failed log in attempts by IP address.
If there are any IP addresses with more than 10 log in attempts the number of attempts made the IP address
from which those attempts were made and the location of that IP address will be displayed.
By the way we're going to be using a command called Geo IP look up to determine the location of that
IP address.
We may or may not have covered that in a previous demonstration or lesson.
So just be aware of that command give it an IP address and I'll tell you where it thinks that IP address
originates from.
We're going to make this group produce output and CSP.
And of course V stands for comma separated values and we're going to give the output a header of Count
comma I.P comma location.
So here I am on a system and the sample data that we're going to use for this exercise is included in
the course download.
I've already placed a copy of it in the shared vagrant folder for this project.
So if I C.D into forward slash vagrant then I can see all the files that are in that folder on my local
physical machine.
The first thing I do when working on these types of problems is to look at the data that I'm dealing
with.
I want to know what the input looks like first then I can start to look at ways to transform that data
so that it's easy to work with or so that it meets my requirements so let me just look at the contents
of this file.
That's a lot of data.
One thing I notice a lot or I see a lot of are lines that say failed password for root.
I'm going to bet that people are not only trying to log in as the root user but probably some other
common users as well.
Let's see if that's the case.
So a common thing here on these lines for failed password for root is the word failed with a capital
F..
So let me just grep out that pattern grip failed syslog sample here on our screen only see route.
I think I saw a variation or two above when this output was scrolling by.
So let's exclude route to see what else we get.
If we look at the last five lines or so of output we see log in attempts for the buntu user the admin
user the LP user the admin user again and a user named A.
What I notice is that the lines aren't exactly in the same format.
What sticks out to me here is that the words in valid user exist on most of those lines here but they
don't exist on all of the lines.
For example the LP line that third line from the bottom here says for LP from.
But if you look at the next line it says for invalid user admin from the important piece of information
that we want to isolate on each of these lines is the IP address.
One way we could do that is to split the line on the word from.
So let's grep for failed.
We'll pipe this to awk and use the word from as a field separator.
And actually I'm going to use from space so we don't end up with that extra space here and then we'll
just print the second part of that.
So we should end up with the IP address and the rest of the information on the line.
Let's see what happens.
Now we're left with four columns all separated by a single space.
The first column is the IP address.
The second column is the word port followed by the actual port number itself.
And then finally the protocol which here is S-sh to now from here we can print the first column either
with the cut or AWC.
So let me just do this pipe this to awk print dollar sign 1 and that gives us the IP address.
Or instead of using awk here we can use cut cut SD and we're going to separate on a space so that's
our delimiter and we want the first field.
And so we get the same output.
So let's go back and talk about another way to solve this problem.
If we count the number of fields from the left to right we end up with the IP address being in different
fields depending on whether or not the user was valid or invalid.
But if we count the number of fields from the right to left or from the end of the line towards the
beginning of the line you'll see that the IP address is always the fourth column from the end.
That means we can use alks special variable of n f which represents the total number of fields on a
line and then do a little subtraction to end up with the IP address.
So here we can do this
arc print and then we're going to take the number of fields and subtract 3 from that number and we should
end up with a column that has the IP address in it.
So I've demonstrated two ways to extract the IP address.
There are other ways in if you came up with another way that's totally fine as long as you have extracted
the IP address from all those lines in that file.
So this approach that I used here makes the most sense to me and it seems a little bit simpler so that's
what I'm going to use going forward.
We know we can use the unique command to count the number of occurrences of a line.
We also know that unique Richar sorted input.
So let's first assort our list of IP addresses and then send it to unique.
So I'm just going to sort this list doesn't have to be a numeric sort it can be any sort as long as
it's sorted unique doesn't care.
So then all we can do here is pass it into unique and tell unique to count the occurrences of each line.
And of course the only thing that are on these lines are IP addresses so it'll count the occurrences
of these IP addresses.
Now that we have this bit of data let's sort this numerically.
So we'll just pipe this to sort dash in.
Actually let's reverse this order and put the most failed attempts at the top of the list.
So we can just add a dash r to our sort command to reverse it and we end up with the most failed logging
attempts first and the least failed logging attempts last.
By the way this simple log file contains entries from just one day.
That means there were six thousand seven hundred and forty nine failed log in attempts from the IP address
of 182.
One hundred sixty seven dot 59.
Now this could mean a couple of different things.
The first thing that comes to my mind is someone was performing a brute force attack.
However another possibility is that something is wrong with an account that we're using for some sort
of automated process.
Perhaps one of our servers and another data center is connecting to the system over S-sh to do some
work.
But maybe the S-sh key was accidentally changed or deleted or the password for the account was changed
or some other configuration issue has cropped up here.
So what I'm going to do is find the location of this IP address.
Now there's a command called Geo IP look up that returns the location of an IP address and so let's
run that now
that IP address is associated with China.
If you happen to have servers in China or people who work from China this still might not be an attack
but just a misconfiguration of some sort.
However let's assume our people only work in the United States Canada and Europe.
Also let's assume our data centers are located in New York London in Amsterdam and this particular case
I would interpret this activity as a brute force attack.
It would be nice to have this location information for any IP addresses who fail to log into our servers
more than let's say 10 times.
If we look at the data we have we have two columns a count in column one and an IP address in column
two.
We could loop through this output and test to see if the count is greater than 10.
And then if it is performed the geo IP look up on that associated IP address.
Now let's take the command.
We worked out here on the command line.
Put it into a script and start working on this last bit of logic.
We'll give our script a header here.
What I'm going to do is actually use a variable to define or limit in that way if we decide or limit
changes in the future we can quickly update that variable at the top of our script.
Like I said I'm going to use a variable.
You could actually create this script with an option and have the user specify that if you'd like but
I'm just going to keep it simple and leave it at a hard coded number here.
However what we are going to do is ask the user to provide us a file and so that will be the first argument
on the command line.
And if it doesn't exist we need to tell them about it.
OK this is our little check here if the file doesn't exist or we can't open it or read it then we're
going to tell them that we can't open the file they provide it or it's going to be blank if they don't
provide a file and then we're going to exit with the exit status of one.
Now what we need to do is loop through this data that we generated here.
So I'm just going to paste that command we worked out and you remember that the command generated two
columns of data.
The first column being a count and then the second column being an IP address.
So what we can do is pipe this to while read assign the first column to the variable named count and
the second column to a variable named i.p.
So here we can just do a simple check here if the count is greater than the limit we set
then we're going to determine a location we'll use that Geo IP lookup command against the IP address
and then we'll display this information.
Well my common typing mistakes here is to put the dollar sign outside of the quotation marks instead
of inside and I've corrected that here.
Let's see if I have any more I'm just going to go to the top of the script for a quick look.
OK that looks good.
Let's go ahead and save our changes and test our script.
We'll run it against the sample data we have here.
So this is a good start but we need to clean up this output a bit.
First let's remove that little bit of redundant information there GOP country Ed. Let's remove those
words from the geo IP lookup output we could use cut and use a comma as the field separator but that
would leave a space before the country.
However we could use awk and include the space after the comma as the field separator so let's do that
instead.
While we're here let's turn this output into see S V output and let's display a heter
will display the count.
The IP address and the location here for our Hetter.
Now we're going to change this geo IP or look up output going to pipe that to arc with a field separator
of a comma and a space and then that will leave us with the data we need here in the second field.
And also while I'm here I'm going to change this echo command here to have commas in between the data.
And that looks good.
Let's see what this brings us show attackers this log sample.
Ok grace we have a header of Cal Typee location.
Then the first line we have a count of 60 7:49 with an IP of 180 to 167 to 59 and a location of China.
So this is exactly how we want our output to be displayed.
While we're here let's make sure that our little file tests that we wrote at the top of our script works.
So let's provide some fake data to it.
OK that looks good.
We get an exit status of one.
And let's make sure that if we don't provide a file that this also works.
Sure enough it says hey I can't open a file that doesn't exist.
And we also get an exit status of one.
So this completes the exercise walkthrough for this script.