Hi and welcome to the last episode of this curse.
Today we will be building a distributed search engine a very little one to do so.
We're going to be using struct slices in statesman for loops range classes channels guaranteeing select
statements in a very very short amount of lines the search engine consists in one database which is
collection of users as you can see the user is just a structure with an email and a name and the database
is a collection of those with random bodies.
So what we have to do is to implement some sort of function that given an input which is going to be
an email it should return the name warns that even.
Right.
And we're gonna be using this to structure that we will call worker.
So let's get started.
The first thing we have to do is to create a worker structure so let's do time worker structure and
this worker will have a copy of our database which will call users.
So we have the users and let's build a constructor for these workers.
So new worker is going to receive the database.
And it will return our worker pointer.
Right.
So return worker and we'll pass the user as an argument.
Right.
And now we need to build defined function which is going to be a method of the worker structure.
So there's gonna be worker and the find function will receive an email as a string and then will return
a user.
OK.
So how do we do this.
We need to iterate over our database.
So what is going to be raise over not new users will have a user is going to be removed.
Why is the index of users so is going to be like this.
But as we want to return a pointer is going to be the address user.
Right.
And now we need to do the checking.
So if user that email is equal to the e-mail that we just pass we will return the user and after if
we didn't return anything which is return nil.
Right.
We have an error in line 3 4.
Because this is plural.
Yes.
OK.
Well we still can run anything because we know the main function.
So let's write a main function.
So first of all we need to create the worker because going to be new worker in the argument is gonna
be the database.
Think of all the database something as external perhaps.
We're just putting in the code for this example and we can easily call w that fine and some e-mail right
now where this e-mail came from.
We're gonna take it from the arguments when we call the program.
We can pass arguments and those arguments can be retrieved using the OS package.
So let's important always package R's and let's say the first one is not number zero because number
zero is the name of the program.
So there's is gonna be.
Right.
So we will say log print looking for e-mail right.
There's gonna be a user can happen.
Two things.
The first one is a user is not Neil which means we found something.
So we will print the e mail is owned by this.
We're going to pass user the email and user the name right now.
If we couldn't find the e-mail we will say log print.
The e-mail was not found.
OK.
And now we can try this thing.
So let's go round and try to find some e-mail it says.
OK.
The e-mail was not found.
If I type an invalid e-mail it's not found.
But if I go to my database and I try to find for me estimate for example
Smith's example dot com it says the e mail me that's made an example that com is owned by me as me so
good.
But this is not distributed at all.
So we need to make these distributed and to do so we need to run multiple instances of worker at the
same time Crawley give them a portion of the database to each of those so they can iterate over their
own database so we all need to do some changes the first thing is to make these a synchronous.
So for doing we will start by creating some channel so there's going to be a channel of user and the
response of the method of defined function is not going to be returned as we're doing here in line 59
but what will happen is that the worker will go right into the channel.
So when it's about the channel we will pass it here and we are passing the channel here is going to
be a channel of user and we need to create it and destruct.
So it is going to be a channel of pointers to user and finally we do channel tunnel.
Okay.
Right.
And now wanting to change a signature of this function.
So instead of returning our user we're not returning it.
We are just sending it to the child.
We can remove the return and remove the return and we're not returning the user anymore.
So you're gonna be doing this.
And we also want these to run in the background because it's synchronized it's concurrent right.
So this is gonna be running in the background and now we can safely wait for the channel to pass a message.
So the user is not coming from the find function but it's coming from the channel.
Right.
So the fine will right into the channel whenever she can do it and we're gonna be listening here we're
rich from the channel and we can come into this house because we don't need it anymore.
So we have the user and we're printed right.
An undefined channel.
Yeah.
Because this has to be a property of the worker.
Okay.
So let's try it again.
Good.
We have it.
But what happens if we go and we pass a new valid email address.
There is an error but it's not an error.
We are controlling is kind of random errors.
Go error.
And what it's saying is that the Gore team is asleep.
So the compiler is smart enough to detect that this function is gonna be waiting forever.
Why is going to be waiting forever.
Because dysfunction basically return and never wrote to the channels.
So we need a way to say OK if after a while I can switch from the channel will timeout.
OK.
So to do so we will use the select statements and we will say OK in the case this channel return print
is no way to defend a timeout and to do it we will use the time package and what time does is kind of
after and if you see the signature of this function it receives a duration and returns a channel of
time messages.
So we'll say OK after one second we can assume that we couldn't find anything.
So we'll reduce this thing and we'll print the e-mail was that far.
OK.
Let's try to run this.
If you take a closer look to this there is one second between these first message and the second message
so let's increment the time to three seconds for example and let's run again and there is three seconds
between each of those which means that we are timing out for a fixed amount of time.
Let's turn it back to one if we want shorter time we can set OK.
Let's do it for a hundred measly seconds so the output is gonna be way faster.
Now you see it's kind of good enough so you can even keep it for 100 milliseconds.
OK.
So we have a timeout now.
Now this thing started looking like client server architecture where I make requests I waste of time
and then get response is during specific amount of time.
I got nothing.
I can assume nothing happens.
The e-mail was not found.
Right.
So now we're in pretty good shape to basically this to win this thing among many workers.
So if we see the size of the database is one two three four 18.
So we have 18 registries here.
What I want to do is to give the first half to one worker and the second half of the data to the other
workers so they don't have to repeat the job.
It wouldn't make sense to create many workers doing exactly the same thing.
So what I'm doing is what in databases they call shutting splitting the database into different portions
and each of the workers will work in a particular portion.
How do I do this.
First of all literary factor this into a way simpler notation.
So we're gonna do metal Cheney here on creating the worker but I'm not holding the reference to the
worker.
I'm just calling find out over this expression.
OK.
I'm gonna do the same for four days now instead of putting the whole database when I'm gonna do this
passive slice of 0 to 9 and 9 to 18.
So this is the same as doing 0 2 9 9 2 18 but you can do that with this syntax which is shorter.
So I'm passing the first half of database to one worker and the second half to the other worker and
the effect should be the same.
Nothing was found and me asking me it was fine.
Right.
But think about these when you have millions of registries then you can take advantage of the distributed
side of this implementation.
OK.
So again if I want to do it for three workers I could do something like from zero to six from six to
twelve from 12 to the end.
Right.
And it's probably gonna work as well.
Now just if you're curious you can pass.
Let's say we want to name the workers.
So
number three here we could name them.
But the need to receive a name here was going to be a string here same here.
Name String and we can pass the name here.
Name and one specific worker finds it.
We can just print it here.
Okay so print w the name right.
And some station.
Okay so it's I just want to know who was the worker.
This is gonna be number three right.
So let's try with Alex Hunter Davies.
It should be worker number one.
Alex Hunter the Davies number one right.
So this is cool.
We basically at this point we already have our distributed search engine.
We basically split the database into three fractious segments and each of my worker is working over
one of these subsets.
Now this is pretty boring because if I don't know the exact email I'll probably can't find the name
of the owner.
So what I want to do now is to make this search engine a little bit more smart and pass team a putter
like a fragment of the e-mail.
It makes sure that I can get the answer back anyway.
Now if I pass a fraction of the email I might get more than one answer.
So let's see how we do with that as well.
First of all instead of doing this comparison we will use the strings package in the string package
has these functions contains and contains basically verifies that in this stream the email that this
fragment is contained.
Okay.
And there is pretty much the same Eve user that email contains the key word that I'm gonna give you.
Put it in the channel again.
So let's try it.
This is still working but unable to pass Alexander.
And if you take a look Alexander is owned by Alexander Davies.
So first I will need to use the email Yeah but here I have two e-mails with Alexander I have Alexander
Davis and Jackson and I just got them one.
So what happened after I read from this channel.
The program basically quit.
So the only thing that I need to do which you are really probably guess is I need to put this into a
four and this is basically going to be reading from the channel all the time until I got to the timeout
and that's it.
It's that simple.
So let me run this again.
Yeah.
What happened is I got the two Alex Sanders and then not found forever.
So when I reached this point instead of printing not found I will say return basically quit.
And now I have to Alexander's.
And also I can be less specific and said OK give me all the e-mails that contain that sworn word and
get Jackson Jackson on Robinson Mason and Jackson again.
Right.
So this is pretty much it for this chapter with cover mainly every feature with learn in the Go language
so that you can see that it's pretty simple so we went through our own custom types constructors methods
are passing channels go routines select statements range for us many stuff.
So I think from now on you are able to start writing your own applications.
You have all the tools you need.
So I think that's it.
Happy Hacking.