In this lesson you'll learn how to use the most important features of the said Command said stands for
stream Ed and its use for well editing streams.
You might not be familiar with the term stream but if you performed in the IO redirection or piping
you've been using streams you can think of a stream as the data that travels from one process to another
through a pipe or from one file to another as a redirect or from one device to another so you can think
of standard input as the standard input stream standard output as the standard output stream and standard
error as the standard error stream.
By the way these data streams are typically textual data the set command is used to perform basic text
transformations on an input stream.
For example you can use said to substitute some text for other text remove lines append text after given
lines and insert text before certain lines said differs from other editors such as vim emacs and nano
in that it's used programmatically.
Of course you can substitute some text for other tax remove lines append texts after given lines and
insert text before certain lines using them but that requires interaction from a human.
Sure you can write macros to perform these functions in vim but you have to have someone to start them
and then execute the macros which said you can programmatically perform these edits without the need
of interaction.
And this makes said ideal to be used in shell scripts for example.
The most common use of said is to act as a command line version of Find and Replace.
I'm going to use this particular application of said to teach you the basics of the said command.
Notice that I said said Command said is a standalone utility and not a shell built in.
We can prove this by using the type shell built in
so you can see that when you type said in press enter the user been said program will be executed of
course for standalone commands you can use man to get help and documentation on how to use that command.
So let's quickly look at Said's man page as we've already mentioned said is a stream editor the man
page says it's used for filtering and transforming text.
If we look at the synopsis section of the man page we can see that we can execute said with zero or
more options followed by a said script and optionally followed by one or more input files.
So let's exit out of this man page by typing Q and get to our first said command.
Let's create a file to work with real quick.
Let's do this
make sure our contents made it in the file I'm sure did.
But let's just prove it here by carrying that file.
So sure enough it says Dwight is the assistant regional manager and that's the text that's stored in
manager Dot T X T.
Now as I said a minute ago the most common use of said is to act as a command line version of finding
replace let's say you want to replace the text of assistant with the text of assistant to the to do
that you would use the substitute command and said the substitute command is represented by the character
s.
So let me show you how that works.
We'll start out with her said command.
Next we'll put our asset script in single quotes.
We'll tell said to perform a substitution by specifying the letter S after the s we use a forward slash
followed by the text we want to substitute and then a another forward slash.
In this case that text is assistant.
By the way the forward slash is acting as a delimiter and the text between the forward slashes is called
the search pattern.
This search pattern is actually a regular expression so you can make very advanced searches if you want
to.
In this case we're keeping things simple to demonstrate the concept of finding and replacing some text
using said.
Next we'll type the text that is going to replace the previously specified text.
We want to change assistant to assistant to the so we'll type this
next we'll close our substitution with another forward slash.
We'll call this text the replacement string that is the end of our set script so we'll supply the closing
single quote.
Finally we'll tell said what file to use as input and this example that file is Manager dot t t when
we execute the command you can see that the text of assistant has been replaced with Assistant to the
original text is Dwight is the assistant regional manager the text displayed after being transformed
by our set command is Dwight is the assistant to the regional manager.
Notice that I used the word displayed I used it very intentionally here in this example said is not
altering the contents of the file it's just sending its output via standard out which is our terminal
we can prove that the original file is unaltered by looking at its contents said can alter the contents
of the specified file and we'll get to that shortly but first let's look at another example of using
said to find and replace text.
Let's use this text to work with
let's change the text of my wife to said
in this example we didn't reuse any of the text that we searched for.
Instead we completely replaced it with brand new text.
It should probably go without saying that said is case sensitive by default.
So here's an example of that
notice that the lower case text on my wife was not substituted.
That's because it wasn't an exact match.
If nothing matches said doesn't perform any alterations and therefore the text is returned without alteration.
If you want to perform a case insensitive search you'll need to supply a flag.
So really the format of the said s command is this
you have s a forward slash and then a search pattern.
Another forward slash to terminate the search pattern in the replacement string.
Another forward slash and then any flags you want to use.
So I'm just going to type control C here because I don't want to execute that command I just want to
quit that command line there without running it.
So we're going to supply the ie flag to the command which you can think of as standing for insensitive.
Now the text gets replaced because you told said to ignore case by the way and it strikes me as a little
funny.
You can also use a capital I as the flag to do this in sensitive case matching.
So this command works as well
let's add some more lines to our loved text file and see how said handles that
notice that I've used to greater than signs and that causes texts to be appended to the file if I just
use one.
It would have truncated the file but I was careful here and used too.
So we can look at our contents here and sure enough there are two lines in that file.
So let's go ahead and add a third line.
Okay.
There it is.
Love dot text with three lines.
Now let's try the replacement command again.
What happens is that said reads one line from the file and executes the set commands in quotes against
that line.
Next it moves on to the next line and does the same thing until it reaches the end of the file.
You'll see that my wife was replaced by the words said on lines 1 and 3.
Line 2 was unchanged because our search pattern of my wife was not found on that line.
Let's add yet another line to this file to demonstrate another important concept.
Now let's run the replacement command again
notice that on the last line only the first occurrence of my wife was replaced.
It says I love said and my wife loves me.
Also my wife loves the cat by default said just replaces the first occurrence of the search pattern
on a line to override this behavior.
We'll need to use the G flag.
You can think of G as standing for global as in a global.
Replace
now the last line says that I love said and said loves me.
Also said loves the cat.
If you want to replace the second occurrence of the search pattern use the number two as a flag.
If you want to replace the third occurrence use three.
The fourth use for etc. etc..
So let's do this
this time.
None of the first occurrences of the text my wife were replaced but only the second occurrences were
on the line where only one occurrence of my wife existed.
That line was unaltered because there was no second occurrence on the last line where my wife appears
three times only the second one was changed by the way.
When I'm saying things like change I don't mean that the original file has changed again.
The contents of the file remain the same sometimes you don't want to just see the output displayed to
your terminal but instead you want to save that output.
One way to do that is to redirect the output of the set command to a file.
Of course this isn't anything unique was said.
It's just how Linux works with any command.
So let's create a new file called my new love.
Don t x t
again the original file is left unaltered.
So there you have it the original file with the original text and the new file with a redirected and
altered text.
If you want said to alter the file use the Dash I option to said which you can think of as in place
editing if you want said to make a backup copy of the file before it alters it supply the same text
right after the dash eye and said we'll append that text to the copy of the file.
So now we have our original file named Love dot t x Tepe and another one named Love dot t t dot B A
K said has performed an in-place edit of the original file so let's look at that now.
So there you can see where said has replaced the text of my wife where appropriate given the command
we ran.
If we look at the backup file we'll see that it has the original text in it it's important that you
don't use a space after the dash I option if you do you'll get an error.
So let me demonstrate that here.
So this is what not to do.
Notice I'm using a space.
This is going to cause an error.
So sure enough we get an error.
That is one little gotcha.
You can avoid.
So just do not use this space after Dash I.
If you only want to save the lines where matches were made you can use the w flag followed by a file
name.
So let's do this let's change love to like we'll do a global replace so that no matter how many times
love appears on the line it will get replaced by like and then we'll use the w flag and a file name
after it so we'll use like dot t x t and then the file we're going to use as input is love t x t in
this example said displays the entire contents of the loved text file to the screen with any replacements
it made and it created a new file named like dot t t that only contains the lines where the replacements
were performed.
So let's look at that like t text file now while we're talking about input and output.
I would like to point out that said can be used in a pipeline instead of specifying a file to work on.
The simplest example would be to cat a file and pipe that to said like this.
That is the exact same thing as doing this.
This is a very common pattern with Linux commands where they can be given a file to operate on or they
can use the data sent through a pipe to operate on.
We've seen this with other commands such as cut OK sort unique and others command a pipeline thing is
very powerful because you string as many commands together as needed to make the data look the way you
want it to.
I'm going to generate some text to work on with an echo command
let's say we want to change a forward slash home forward slash Jason to be forward slash export forward
slash users forward slice Jason C.
The challenging thing about this is that the strings we want to manipulate have Ford slashes in them
and we've been using Ford slashes as a delimiter with said one way to get around this is to escape the
Ford slashes.
I'm going to show you a better way in just a second but let's see how we would do it with escaping first.
Now to escape a character simply precede it with a backslash
so we're starting our search pattern with the forward slash.
But the next Ford slash isn't a delimiter so it needs to be escaped.
So if we're going to use a backslash to escape this forward slash same thing again here.
So now we end our search pattern with a forward slash and again R next pattern starts with a Ford slash
we have to escape it here again we're escaping a Ford slash and yet again.
And then finally we're closing out our command here.
That feels like a lot to keep straight and it's easy to make a typing mistake in this situation.
A nice feature was said is that you can use any character as the delimiter.
So let's use a pound sign instead.
So the first character that follows the S will be treated as the delimiter no matter what that character
is.
So let me give you an example here.
I'm going to use a pound sign and now said we'll treat that pound sign as the delimiter.
So now I can just go ahead and specify the text I want to use here and my search pattern which is Ford
slash home.
Jason close that search pattern and now the replacement string is Ford slash export Ford slash users
Ford's less Jason C and then we'll close out the replacement text there with a pound sign and the single
quote finishes are said script command there.
So let me hit enter.
And so as you can see that works as well.
Just to prove that any character works as a delimiter.
Let's use another character how bout a colon this time
OK that works as well.
So you get the idea if you need to use Ford slashes and your search pattern or replacement text choose
a different delimiter other than the forward slash.
All right.
So enough with actually performing substitutions.
Let's take a second here to talk about when you would actually want to perform them.
One idea is to use templates or template files.
For example if you are constantly deploying new Web sites and use the same configuration except for
the Web site name then it would be a good idea to create a template file that contains all the standard
configuration and a place holder for the Web site name.
Then you could use said to simply replace all the place holders with the actual Web site name.
When you're ready to deploy it.
Another example of when you could use said substitution feature is when you are migrating from one server
to another or when you are using a restore of one server to create another new server.
In this example you would need to find and replace the old hostname with a new one for all the files
in the Etsy directory files.
This is Etsy hosts and maybe Etsy hostname and there could be others depending on that systems configuration.
Yet another example would be when you are copying configuration for a given service from one host to
another.
You might find yourself doing this especially if you are working on clusters.
You would just copy the configuration from one host to the new host that you are adding to the cluster.
Typically you'll need to change the hostname in the configuration file and you could use is said to
do this for you said would be especially helpful if that hostname appeared many times in that configuration.
Okay so that's pretty much it.
With the said substitution command let's say we want to remove or delete some lines with said well how
you do that is with the D command.
Let's review the contents of our love dot text file
let's say we want to remove the line that says this is lying to us you have to do is come up with a
search pattern that will match that line and not the other ones.
So we could use a search pattern of this because the word this only appears on the line.
We want to delete our search pattern is this.
And the command is D
so to be clear here the syntax is a delimiter.
And by default we use the Ford slash as the delimiter followed by the search pattern and a closing delimiter.
Again this is a Ford slash and finally a D which tells said to delete the line that matches the provided
pattern.
We can see that this worked as our desired line is gone.
Let's say we want to remove all the lines that contain the search pattern of love.
Here's how we would do that.
So now all the lines that match love are gone.
Sometimes when I'm working with a configuration file with many lines I'd like to strip out the comments
and blank lines to compact the output.
That way I can see just the configuration and nothing else it makes it easier to read at least it does
for me.
I'm going to create an example configuration file to demonstrate this concept with
so this sample made up configuration file is just a few lines long but you can easily imagine one with
many many more lines.
First let's remove the comments.
Remember that the search pattern is actually a regular expression in regular expressions of the carat
symbol matches the beginning of a line it matches a position and not a character.
So if we want to match all the lines that start with a pound sign we use carat pound sign.
This makes sure we don't accidentally delete lines that have actual configuration at the beginning of
the line but have a comment later on that same line.
So now we have our configuration file without the comment lines but we still have the blank lines to
delete a blank line we'll use another regular expression
again the carat symbol matches the beginning of the line the dollar sign matches the end of the line.
So carat dollar sign matches a line if the beginning of the line is immediately followed by the end
of the line set another way carat dollar sign matches blank lines.
So we know how to do the two things we want to do.
But now we need to do them at the same time we need to combine them to use multiple said commands or
said expressions we can separate them with semicolons
for each line said we'll perform the first command which deletes lines that start with a pound sign
and then the second command which deletes blank lines to show you that you can combine different types
of commands let's perform a substitution.
Let's change Apache to h t TPD
so that particular commanded three things are deleted lines that started with a pound sign the deleted
blank lines and it also changed Apache to h t TPD just to be thorough.
I want to share with you another way to execute multiple said commands.
Mainly I want to show you this just in case you happen to see it in a script or in someone else's work.
That way you'll know exactly what is going on.
So the other way to do this is to use multiple dash e options to said one for each said command to execute
said also allows you to specify a file containing the said commands.
So let's demonstrate this now.
I'm going to create a file using the echo command here of course you can use an editor if you want to
so what we've done here is put one said command per line.
Now we'll use the dash f option and supply the path to the file that has the said commands in it so
we can do this said dash F.
And then the path to our file and then the path to the file that we want to use as input.
So here again we get the exact same result before we wrap up this lesson on said.
Let's take a look at using addresses an address determines on what lines the said command will be executed
on if no address is given.
The command is performed on all lines an address is specified before the said command.
The simplest of addresses is a line number.
Here are the contents of our sample configuration file to refresh our memories here's an example that
will only execute against line 2 of the file
as you can see the search pattern of Apache was replaced by H T TPD only on the second line.
There's also a match on the last line of the file but that was left alone because it didn't match the
address of line 2.
By the way some people do not use spaces after the address.
So this is the same command without that extra space.
You can also use a regular expression as an address to match lines.
Let's say you want to replace Apache with H TPD but only on lines that contain the word group
as you can see Apache was changed to HDTV D.
Only on the line that also contained the word group.
You can also specify a range by separating to address specifications with a comma.
For example if we wanted to change the word run to execute but only on lines 1 through 3 we would do
this.
There was only one occurrence of run from lines 1 through 3 and it was changed the occurrence online
4 was not.
So let's extend the range to include line 4
again.
You can use regular expressions instead of line numbers.
Let's say we want to change run to execute.
Starting with the line that matches pound sign user an ending at the next blank line.
So here's how we would do that.
So there you can see that every instance of run that was in between the line that started with LB user
and the next blank line was exchanged with a text execute.
Of course there was only one match so only one substitution was made but you get the point.
You could specify a large portion of the file if you wanted to.
One starting point with a another starting point as your address and then change everything in that
section while leaving everything else alone.
It's very powerful to use addresses was said in this lesson you learned the most common use case of
said which is to perform text substitutions.
You learned how to replace specific occurrences as well as how to replace all occurrences of the search
pattern.
In addition to finding and replacing texts you learned how to delete text with the D command.
From there you learned how to save the alterations performed by said as well as how to make backups
of the original file so that your data is safe.
Next you learned three different ways to execute multiple said commands on the same set of data.
Finally you learned how to use addresses to work on very specific sections of input.