Some SED basics

One of my favorite tools for Sysadmin work is the Stream EDitor utility or just SED. SED is useful for many things, and is a stepping stone along the way to making variable based shell scripts as well. Don’t want to have to edit the nameservers on a million zone files? SED it. Need to do certain things to a million files at once? SED it. In conjunction with cat, find, and grep SED is devastatingly effective in finding and eliminating administrator headaches. Lets start out with something extremely basic.

sed -i ‘s/ns1.domain1.com/ns1.domain.com/’ /var/named/*.db

What does this do? It goes through and changes the instances of ns1.domain1.com to ns1.domain.com in DNS zone files. Please note I advise grepping any thing out that you are changing because if there are multiple instances of this in the file it will only do the first instance.  If you have this issue, you can always repeat the command and check again until all instances are found. the s indicates a spelling correction, the -i puts it back into the original file. If we just wanted to print to the TTY we would use the -e augment.

Well, that’s pretty cool, but what about some other situations that come up? Lets say we’re migrating a cpanel box. There are a ton of scripts out there, but we have some special needs. Say we want to run it with –skip-homedir because this is going to be a pseudo-manual migration and we’ll sync the homedir over later. All we have to do is make a copy of /etc/users and then do the following:

sed -i 's/$/\/scripts\/pkgacct /' users

sed -i 's/^/  \-\-skip\-homedir\' users

chmod +x users

./users

Yes this is a few commands, but we want to progressively look over what’s happening here. The $ means that /scripts/pkgacct is prepended to the beginning of each line. Notice that there is a space at the end of the command so that the user name doesn’t become part of the command we are trying to run and error out. Also notice the \es. These are an escape character that is used in order to allow the use of a special character such as /  – . or other characters that may otherwise be taken as part of the command. The second line is similar to the first in the fact that it will add to every line in a file as well as the use of escape characters, however ^ will add to the end of every line. The last thing we are doing is making our script executeable with the +x command (you can chmod 755 if you want and get similar results) and then runs it. If we were smart we would probably put a she-bang at the top (#!/bin/bash) so that it is run with BASH.

Our input file would look like this:

user1

user2

user3

user4

and the output would look like

/scripts/pkgacct user1 --skip-homedir

/scripts/pkgacct user2 --skip-homedir

/scripts/pkgacct user3 --skip-homedir

/scripts/pkgacct user4 --skip-homedir

There are a ton of uses for this, I love cleaning spam in Exim’s mail queue with this if you’re not allowed to BOFH the system and delete the “clean” email with the spam as well. That will be a later episode however.

Leave a Reply

Your email address will not be published. Required fields are marked *