WEBVTT 00:00.210 --> 00:00.700 Hi. 00:00.780 --> 00:02.790 This is the last video of the section. 00:02.790 --> 00:04.700 Pipeline design pattern. 00:04.740 --> 00:09.750 Previously we learned about future design pattern in this video we're going to take a look at the concept 00:09.840 --> 00:13.260 working and implementation of the pipeline design pattern. 00:13.320 --> 00:17.000 The third and final pattern we'll see in this section is the pipeline pattern. 00:17.250 --> 00:19.740 You'll see this pattern heavily in your concurrent structures. 00:19.740 --> 00:22.180 And we can consider it one of the most useful too. 00:22.320 --> 00:24.510 We already know what a pipeline is. 00:24.510 --> 00:29.040 Every time that we write any function that performs some logic we're writing a pipeline. 00:29.070 --> 00:35.310 If This Then That or else something else pipeline patents can be made more complex by using a few functions 00:35.310 --> 00:39.390 that call to each other they can even get looped in their out execution. 00:39.830 --> 00:45.060 The pipeline patterning go works in a similar fashion but each step in the pipeline will be in a different 00:45.060 --> 00:48.910 go routine and communication and synchronizing will be done using channels. 00:48.990 --> 00:51.450 What will we gain by creating a pipeline. 00:51.450 --> 00:53.720 We're mainly looking for these benefits. 00:53.850 --> 00:59.310 We can create a concurrent structure of a multi-step algorithm we can exploit the parallelism of multiple 00:59.310 --> 01:02.910 machines by decomposing an algorithm in different go routines. 01:02.910 --> 01:07.830 However just because we can decompose an algorithm in different go routines doesn't necessarily mean 01:07.830 --> 01:09.210 the execute the fastest. 01:09.600 --> 01:12.060 We're constantly talking about CPE use. 01:12.090 --> 01:17.000 So ideally the algorithm must be CPO intensive to take advantage of a concurrent structure. 01:17.100 --> 01:20.880 The overhead of creating go routines and channels could make an algorithm smaller. 01:21.120 --> 01:24.000 Here we'll create a concurrent multi operation. 01:24.000 --> 01:26.340 We're going to do some math for our example. 01:26.460 --> 01:31.860 We're going to generate a list of numbers starting with 1 and ending at some arbitrary number and then 01:31.860 --> 01:33.060 we'll take each number. 01:33.180 --> 01:37.020 Power it to 2 and sum the resulting numbers to unique result. 01:37.020 --> 01:41.750 So if n equals 3 our list will be 1 2 3. 01:41.940 --> 01:46.000 After powering them to 2 our list becomes 1 4 9. 01:46.080 --> 01:49.000 If we sum the resulting list the resulting value is 14. 01:49.500 --> 01:51.590 Let's look at the acceptance criteria. 01:51.660 --> 01:56.460 Functionally speaking our pipeline pattern needs to raise to the power of to every number and then some 01:56.510 --> 01:57.360 more. 01:57.450 --> 02:04.680 It will be divided into a number generator and two operations so one generates a list from one to N 02:04.950 --> 02:09.180 where N can be any integer number to take each number. 02:09.180 --> 02:14.850 This generated list and raise it to the power of 2 3 some each resulting number into a final result 02:15.060 --> 02:16.470 and return it. 02:16.470 --> 02:21.400 Let's begin with tests will create only one function that will manage everything. 02:21.480 --> 02:24.900 We'll call this function launch pipeline to simplify things. 02:24.900 --> 02:31.050 It will take an integer as an argument which will be our N number the number of items in our list. 02:31.050 --> 02:33.520 Let's start create two documents. 02:33.600 --> 02:38.670 Pipeline Dok Go and pipeline test or go 02:42.460 --> 02:50.430 at the function launch pipeline in our test file we'll create a table of tests by using a slice of slices. 02:50.530 --> 02:58.010 This is our table when passing 3 it must return 14 when passing five it must return 55. 02:58.080 --> 03:02.520 Then we have to iterate over the table and pass the first index of each right to the launch pipeline 03:02.520 --> 03:03.240 function. 03:03.270 --> 03:09.780 Let's do that using range we get every row in the matrix each row is contained in a temporary variable 03:09.780 --> 03:10.080 called. 03:10.080 --> 03:14.190 Test test 0 represents n and test 1. 03:14.190 --> 03:15.470 The expected result. 03:15.600 --> 03:20.250 We compare the expected result with the returning value of the launch pipeline function. 03:20.370 --> 03:21.260 If they aren't the same. 03:21.300 --> 03:24.510 The test files say this file and test in the terminal 03:27.620 --> 03:29.240 time for implementation. 03:29.390 --> 03:33.920 The key for implementation is to separate every operation in different go routines and connect them 03:33.920 --> 03:35.070 with channels. 03:35.330 --> 03:39.830 The launch pipeline function is the one that orchestrates them all as shown in the diagram. 03:39.830 --> 03:46.310 The operation consists of three steps generate a list of numbers raise them to the power of two and 03:46.400 --> 03:48.760 add the resulting numbers. 03:48.760 --> 03:54.940 Now we'll see the structure each step in this pipeline pattern will have. 03:55.030 --> 03:59.820 This is the syntax let's look at the flow chart of the process. 03:59.820 --> 04:01.760 This function represents a common step. 04:02.100 --> 04:06.140 Let's dissect it in the same order that go scheduler will probably take to execute it. 04:06.270 --> 04:11.940 The function name function will commonly receive a channel to take values from we call it the end function 04:12.270 --> 04:14.040 as in the word incoming. 04:14.040 --> 04:17.060 We cannot same value through it within the scope of this function. 04:17.070 --> 04:22.560 That's why the arrow points out of the keyword Chan the function name function returns a channel that 04:22.560 --> 04:27.150 the function caller will only be allowed to take values from again represented by the arrow pointing 04:27.180 --> 04:28.710 out of the keyword channel. 04:28.710 --> 04:33.060 This also means that any value that goes through that channel must be generated within the scope of 04:33.060 --> 04:37.860 the function in the first line of the function we create a channel called out that will be the return 04:37.860 --> 04:38.560 of the function. 04:38.580 --> 04:40.590 Then we'll launch a new go routine. 04:40.710 --> 04:43.400 Its scope will enter into play after returning this function. 04:43.410 --> 04:44.160 So let's continue. 04:44.640 --> 04:47.430 We return the previously created out channel. 04:47.430 --> 04:52.230 Eventually after finishing the execution of the function and returning the channel out the go routine 04:52.230 --> 04:56.030 executes it will take values from the inn channel until it's closed. 04:56.130 --> 04:59.130 So the core of this function is responsible for closing this channel. 04:59.130 --> 05:04.560 Otherwise the go routine will never end when the in channel is closed the for loop finishes and we closed 05:04.560 --> 05:09.510 the out channel any go routine making use of this channel will not receive any new values since the 05:09.510 --> 05:14.730 last century the only step that doesn't completely fit this approach is the first step that receives 05:14.730 --> 05:19.980 a number representing the upper threshold on the list instead of a channel of incoming values. 05:19.980 --> 05:22.040 Although the idea is exactly the same. 05:22.170 --> 05:26.670 Now we can see that it's the function launch pipeline that is the one that's going to be receiving channels 05:26.730 --> 05:29.600 and sending them back to the next step in the pipeline. 05:29.670 --> 05:33.930 Using this diagram we can clearly see the flow of the pipeline creation by following the numbers of 05:33.930 --> 05:41.380 the arrows a solid arrow represents a function call and the dashed arrow a channel let's look a little 05:41.380 --> 05:42.850 more closely at the code. 05:43.150 --> 05:46.410 Open pipeline don't go first. 05:46.500 --> 05:52.070 We have the list generator the first step in the operation is this generation. 05:52.180 --> 05:56.830 The list starts at 1 and will receive an integer representing the higher threshold. 05:57.040 --> 05:59.410 We have to pass each number in the list to the next step. 06:00.910 --> 06:05.130 So here you can see that done in the for loop as we mentioned earlier. 06:05.160 --> 06:10.200 This is the pattern that we will follow in each step created channel launched the go routine that will 06:10.200 --> 06:15.700 send the data through the channel and immediately return the channel this go routine will iterate from 06:15.700 --> 06:20.380 one to the max argument which is the higher threshold for our list and said each number through the 06:20.380 --> 06:21.140 channel. 06:21.280 --> 06:25.600 After sending every number the channel is closed so that no more data can be sent through it but the 06:25.600 --> 06:28.110 data already buffered can be retrieved. 06:28.240 --> 06:30.940 Next we'll raise numbers to the power of two. 06:30.940 --> 06:35.380 This step will take every incoming number from the first steps channel that is taken from the arguments 06:35.860 --> 06:37.730 and raise it to the power of two. 06:38.200 --> 06:42.010 Every result must be sent to the third step using a new channel. 06:42.010 --> 06:43.540 This is the code for that. 06:43.990 --> 06:45.460 We use the same pattern again. 06:45.460 --> 06:51.540 Create a channel and launch the go routine while we return the create a channel the full range loop 06:51.570 --> 06:57.720 keeps taking values from a channel indefinitely until the channel is closed the next part is final. 06:57.710 --> 06:59.160 Reduce operation. 06:59.250 --> 07:03.530 Third and final step receives every number from the second step and keeps adding them to a local value 07:03.560 --> 07:05.860 until the connection channel is closed. 07:05.870 --> 07:12.230 This is the function some the functions some also takes a channel as an argument. 07:12.230 --> 07:14.190 The one return from step 2. 07:14.320 --> 07:19.450 It also follows the same pattern of creating channel launching the go routine and returning a channel 07:19.930 --> 07:26.110 go routine keeps adding values to a variable called Sum until the in channel is closed when the channel 07:26.110 --> 07:26.720 is closed. 07:26.720 --> 07:30.790 The value of some is sent to the out channel and it's immediately closed. 07:30.790 --> 07:32.560 Now we launch the pipeline pattern. 07:32.560 --> 07:35.950 Finally we can implement the launch pipeline function. 07:36.070 --> 07:42.800 For that we need to add the function launch pipeline the function generator first returns the channel 07:42.810 --> 07:44.730 that is part of the power function. 07:44.730 --> 07:48.240 The power function returns a second channel as part of the sum function. 07:48.240 --> 07:53.430 The function sum finally returns the first channel that receives unique value the result. 07:53.450 --> 08:01.350 Let's try to test is now save the file open terminal and test it awesome. 08:01.380 --> 08:05.400 It's worth mentioning that the launch pipelines function doesn't need to allocate every channel and 08:05.400 --> 08:06.920 can be rewritten like this. 08:08.200 --> 08:12.520 The result of the generator function is passed directly to the power function and the result of power 08:12.520 --> 08:17.350 to some functions with the pipeline patterns we can create really complex concurrent workflows in a 08:17.350 --> 08:18.530 very easy way. 08:18.760 --> 08:25.000 In our case we created a linear workflow but it could also have conditionals pools and fan in and fan 08:25.000 --> 08:26.000 out behavior. 08:26.140 --> 08:28.740 In this video we worked with the pipeline design pattern. 08:28.840 --> 08:30.220 Great. 08:30.320 --> 08:34.640 This brings us to the end of this section and course this section cover the three important concurrent 08:34.640 --> 08:39.590 patterns in go concurrency barrier future and pipeline design patterns. 08:39.590 --> 08:42.290 We use them in different applications according to the need. 08:42.290 --> 08:44.160 With this we conclude the course. 08:44.300 --> 08:46.860 I hope you're now confident to use go concurrently. 08:47.090 --> 08:48.170 Keep practicing. 08:48.170 --> 08:48.740 Keep learning.