Sudo Make Install

This code is proof of concept, if you want to use it in a production environment I suggest you go over it heavily. For a person fairly new to perl there is a lot going on here that you may find useful. The overall idea is to convert IPs from dotted quad decimal numbers into binary then store them in a database. Because IPs can’t be duplicated on machines or it will cause a conflict, it is in general going to be a good value to have as a primary key. Feel free to use and adapt this code as you see fit. The end result should be something like:

mysql> select * from IPs;
+———————————-+———————————-+————————–+
| ip_address                       | netmask                          | computer_name            |
+———————————-+———————————-+————————–+
| 11000000101010000000001000000101 | 11111111111111111111111100000000 | control.frontandback.net |
+———————————-+———————————-+————————–+
1 row in set (0.00 sec)

#/usr/bin/perl

#IP2DB 0.1.0 (C) Febuary 2011 Howard A Underwood II
#Free for use and modification under the Creative Commons 1.0 License. If you want to give me a shout out try aunderwoodii#at#gmail.com
#The purpose of this code is to convert an IP address and netmask pair into Binary to make it easily stored in the database in a processable manner. This is only for IPV4 atm and is just a proof of concept, I’d love to see your adaptations to real world applications. Feel free to give me your feedback at the above address.

#This requires DBI and DBD::MySQL. Use CPAN or your package manager of choice to get them.
use DBI;
use DBD::mysql;

#info to connect to the DB server. This assumes that your table is pre-created. If you need to create a database do the following:
#create database ips;
#CREATE TABLE IPs (ip_address BINARY(32) PRIMARY KEY, netmask BINARY(32), computer_name char(200));

$hostname=localhost;
$db=”ips”;
$port=”3306″;
$user=”dbuser”;
$password=”wouldn’tyouliketoknow”;

#info to put into the DB. There’s the IP here, netmask and the computer name. These variables and the ones above are going to be what you need to use to adapt the script to your needs.
$ip=”192.168.2.5″;
$netmask=”255.255.255.0″;
$compname=”control.frontandback.net”;

#Getting down to business. This first line takes the netmask and breaks it into 4 ocets.
my @netmask = split (/\./, $netmask);
#Now that we have 4 ocets, we process each one into binary. Future modifications include cleaning this code up so that it’s a loop rather than 4 instances.
$ocetnm0= unpack(“B*”, pack(“C”, $netmask[0]));
$ocetnm1= unpack(“B*”, pack(“C”, $netmask[1]));
$ocetnm2= unpack(“B*”, pack(“C”, $netmask[2]));
$ocetnm3= unpack(“B*”, pack(“C”, $netmask[3]));
#We recombine everything into 1 Binary number after this.
$totalnm= $ocetnm0.$ocetnm1.$ocetnm2.$ocetnm3;
#Just printing the post process # on the TTY for human verification
print “$totalnm\n”;

#Now we repeat the process for the IP its self. This will probably get condensed into one instance along with the above code eventually. Once again, not the most efficient way to do it but rather straight forward.
my @ip = split (/\./, $ip);
$ocet0= unpack(“B*”, pack(“C”, $ip[0]));
$ocet1= unpack(“B*”, pack(“C”, $ip[1]));
$ocet2= unpack(“B*”, pack(“C”, $ip[2]));
$ocet3= unpack(“B*”, pack(“C”, $ip[3]));
$total= $ocet0.$ocet1.$ocet2.$ocet3;
print “$total\n”;

#Basic DBI connection code. We are using the DBI script to connect to the databse
$dsn = “DBI:mysql:database=$db;host=$hostname;port=$port”;
$DBIconnect = DBI->connect($dsn, $user, $password)
#If we don’t like what we see bail out because we can’t connect.
or die “Connection denied to database $db \n;”;
#Add the entry to the table. Please note that if you use the above table it will probably not let you run this more than once for any given IP.
eval { $DBIconnect->do(“INSERT INTO IPs (ip_address,netmask,computer_name) VALUES (‘$total’,'$totalnm’,'$compname’);”) };
print “Data not added to the database: $@\n” if $@;

The great thing about shell scripts is that they are a great way to solve complex problems that can cost you a lot of time to do manually. To this end, I had a client that needed some videos encoded on his server that didn’t encode properly. For an experienced script writer this would take about 5 minutes to write. It also makes it so that if the client wants to use it they can. The configuration was nice because the input and output file name was the same, just the extension was different. This is not very polished, if it were I would

A)run it as the same user

B)Put it in the user’s homedir

C)Make it so that it was password protected and executable via PHP script so the user wouldn’t require any bash experience at all but could upload a list via FTP and just run it.

#!/bin/bash

for video in `cat /root/list.txt` #We will run a loop where each line in list.txt is run as a variable $video.
do
mv /home/user/public_html/media/videos/flv/$video.flv /home/user/public_html/media/videos/flv/$video.flv.old #back up old files
ffmpeg -y -b 1500 -r 25 -i  /home/gogreenc/public_html/media/videos/vid/$video.* -f flv -s 640×480 -deinterlace -ac 1 -ar 41400 /home/user/public_html/media/videos/flv/$video.flv #encode new file, 640X480 out, FLV format deinterlaced.
chown user:user /home/user/public_html/media/videos/flv/$video.flv #chown to the right user. Not required if running as the right user.
done

I’ve seen my fair share of clients that need basic MySQL backups but have no control panel or don’t want to bother with Control panel based backups. This is a really simple setup that lets you do DB backups and put them in a local directory of the server. It would likely be easily modified to rsync to another server as well if you wanted to. There are a ton of options that could be added to this, your imagination (and shell scripting capacity) are the only limitations. Some suggestions I have would be

-Mail on success or failure and on old file deletion

-Connect to a remote DB

-Monitor the overall size

Well enough with the abstract, on to the shell!

#!/bin/bash
date=`date +%Y%m%d`
mysqldump –all-databases > /mysqlbackups/mysql-$date.sql
find /mysqlbackups/ -atime +30 -delete

If you notice, this takes up all of 4 lines. The first one is the she-bang, the second is establishing the date time stamp, the third dumps the databases and the last one purges any old backups. The only real variable you have to change here is the “+30″ so that it is the number of days you want to retain the backups for minus one.

At work I had a client with a Debian system that needed a bunch of IPs added to it. Since it doesn’t really support ranges (at least that I can find) I came up with the following script.

#/bin/bash
j=42
for i in  {186..190}
do
j=$(expr $j + 1)
echo auto eth0:$j >> interfaces; echo iface eth0:$j inet static >> interfaces; echo address 192.168.41.$i >> interfaces; echo netmask 255.255.255.248 >> interfaces;
done

How it works is that j is the last IP in the ranges currently set in the interfaces file. The address is defined in the script, and the range is defined in the i= section. Just change the numbers to match what you want, put this into /etc/networking, run it and restart networking. This is only for five IPs but you could do hundreds or thousands this way if it was the desired affect. Or you can use a distro that supports ranges :>

This is just a quickie I hijacked off another web site but it came in really handy since a client couldn’t access their wordpress admin panel and we don’t really support it. Just log in to SSH or PHPMyAdmin and run the query that follows:

wp_options SET option_value = ” WHERE option_name = ‘active_plugins’;

*BOOM* no more WP plugins. Then if you’re troubleshooting it go through and reactivate till it breaks again. If not tell em they are on their own.

If you are doing this via SSH you will need to go into the MySQL shell. That can usually be done by typing “mysql” or “mysql -u username -p” and entering the password for the user. If you need this you can grab it from wp-config. After that you need to select the db to use. You do that by going:

mysql>use dbname_here;

This will put you into the appropriate db. Then you run the query above. This is of course assuming you’re using the default schema for naming the tables. If  you are not wp_options needs to be changed to just “options” or schema_options as necessecary. If you don’t know what your schema is you can do:

mysql>show tables;

Welcome to the first edition of Cool One Liners. This will be a collection of one line commands you can use via BASH or another shell/scripting language to do something useful. Creativity will definitely be a big merit. Todays one liner is:

cat /var/log/secure | grep Failed | grep sshd | grep root | awk ‘{print $11}’ | sort | uniq -c | sort -n

What does it do? This takes the secure log, sorts out failed login attempts and then makes it so that the IPs are sorted based on the number attempts. Handy to try and track down brute force attempts on an box running SSH. As an example, I generated a few failed logins.

[root@DNS01 log]: cat /var/log/secure | grep Fail

May  9 03:31:58 DNS01 sshd[10706]: Failed password for root from 127.0.0.1 port 34900 ssh2
May  9 03:32:00 DNS01 sshd[10706]: Failed password for root from 127.0.0.1 port 34900 ssh2
May  9 03:32:04 DNS01 sshd[10706]: Failed password for root from 127.0.0.1 port 34900 ssh2

After this I ran the command given. Notice how the IPs have the number to the left of them. If this were a list the number with the most logins is going to be at the bottom.

[root@DNS01 log]: cat /var/log/secure | grep Failed | grep sshd | grep root | awk '{print $11}' | sort | uniq -c | sort -n

3 127.0.0.1

This command also serves an additional interesting use. Lets say someone is probing your machine, and they happen to be attempting to brute force some nonstandard account names in the hope of coming up with something on the system that is there and has a weak password. This script will also list any invalid users that attempt to log in as well. An example would be if I attempted to log in with the user root1. The output would look like:

[root@DNS01 log]: cat /var/log/secure | grep Failed | grep sshd | grep root | awk '{print $11}' | sort | uniq -c | sort -n

3 127.0.0.1
3 root1

In another blog we will likely take this command, convert it into a shell script, and make it so it will run as a cron job and email us periodic digests.

I have gone over more than a few Linux based commands at this point, so I want to introduce a new way of using them; Pipes. Pipes are really cool because they will let you take the command and put its output into another command. There are a nearly infinite amount of ways to use them as well. Enough with the introductions, lets get on to the commands. Lets say that I want to determine the number of connections currently open at the moment on port 80. We can do the command

netstat -anp

And it will give us a distinct number of entries. We are left looking for traffic that is on port 80, and we have a bunch of lines we don’t care to bother with quite frankly. This is where the pipe comes in.

netstat -anp | grep :80

This will show any traffic that is to port 80, or any traffic that is going to port 80 on a remote server. Lets say we have a huge amount of traffic though, and want to just get a count. If you have a high use server counting by hand is a nuisance at best and virtually impossible at worse. At this point we would just throw the results of our grep into another pipe like this:

netstat -anp | grep :80 | wc -l

And then we have a raw number of accounts used on port 80. Pretty neat. Because pipes are an infinitely versatile tool we can use them for dealing with static files or dealing with the server in real time using utilities like tail. Want to know more about tail? Check out my next blog, I’m going to show some of the tricks on server side troubleshooting.

I’ve always been a fan of collecting fonts, as I go across the net I find random font files and save them for later use. Seems like I can never find the fonts I want because they are scattered all over and unsorted. Got tired of having unorganized font files laying around, so I wrote this to organize them.

#!/bin/bash
#Edit path to the location of where you want your fonts organized
#create a folder called "Unsorted" and place all the files into that.
path="Fonts/"
cd "${path}Unsorted/"
for mFile in *
do
  #Check rather its a file or folder, if its a folder skip it!
  if ! [[ -f $mFile ]]; then continue; fi

  #Grab the first letter of the filename and set it to $mFirstChar/
  mFirstChar="${mFile:0:1}"

  #Convert all lowercase fist letters to upper case.
  mFL=$(tr "[:lower:]" "[:upper:]"<<<"$mFirstChar")

  #If the filename contains any chars such as "!@#$%^&*()" at the beginning
  # set the $mFl var to "MSC".
  if [[ $mFL != [[:alpha:]] ]]; then mFL="MSC"; fi

  #Make directories, prompt for overwrite and create dir if it does not exists.
  mkdir -ip "../${mFL}"

  #Move files to there new home.
  mv -v "${mFile}" "../${mFL}/${mFile}"
done

One of my favorite tools for Sysadmin work is the Stream EDitor utility or just SED. SED is useful for many things, and is a stepping stone along the way to making variable based shell scripts as well. Don’t want to have to edit the nameservers on a million zone files? SED it. Need to do certain things to a million files at once? SED it. In conjunction with cat, find, and grep SED is devastatingly effective in finding and eliminating administrator headaches. Lets start out with something extremely basic.

sed -i ‘s/ns1.domain1.com/ns1.domain.com/’ /var/named/*.db

What does this do? It goes through and changes the instances of ns1.domain1.com to ns1.domain.com in DNS zone files. Please note I advise grepping any thing out that you are changing because if there are multiple instances of this in the file it will only do the first instance.  If you have this issue, you can always repeat the command and check again until all instances are found. the s indicates a spelling correction, the -i puts it back into the original file. If we just wanted to print to the TTY we would use the -e augment.

Well, that’s pretty cool, but what about some other situations that come up? Lets say we’re migrating a cpanel box. There are a ton of scripts out there, but we have some special needs. Say we want to run it with –skip-homedir because this is going to be a pseudo-manual migration and we’ll sync the homedir over later. All we have to do is make a copy of /etc/users and then do the following:

sed -i 's/$/\/scripts\/pkgacct /' users

sed -i 's/^/  \-\-skip\-homedir\' users

chmod +x users

./users

Yes this is a few commands, but we want to progressively look over what’s happening here. The $ means that /scripts/pkgacct is prepended to the beginning of each line. Notice that there is a space at the end of the command so that the user name doesn’t become part of the command we are trying to run and error out. Also notice the \es. These are an escape character that is used in order to allow the use of a special character such as /  – . or other characters that may otherwise be taken as part of the command. The second line is similar to the first in the fact that it will add to every line in a file as well as the use of escape characters, however ^ will add to the end of every line. The last thing we are doing is making our script executeable with the +x command (you can chmod 755 if you want and get similar results) and then runs it. If we were smart we would probably put a she-bang at the top (#!/bin/bash) so that it is run with BASH.

Our input file would look like this:

user1

user2

user3

user4

and the output would look like

/scripts/pkgacct user1 --skip-homedir

/scripts/pkgacct user2 --skip-homedir

/scripts/pkgacct user3 --skip-homedir

/scripts/pkgacct user4 --skip-homedir

There are a ton of uses for this, I love cleaning spam in Exim’s mail queue with this if you’re not allowed to BOFH the system and delete the “clean” email with the spam as well. That will be a later episode however.

A common problem with PHP is that people write code that uses whats called an  “undefined index”,  basically this means they are calling for an unassigned variable.  In PHP, the error-suppression operator is an “@” sign.    It’s best to give you an example as where to use it.

Improper code:

$colors .= "purple,";
$colors .= "red,green,blue";
echo $colors;

This will return “Notice: Undefined variable: colors”

To suppress this error since you know about it:

@$colors .= "purple,";
$colors .= "red,green,blue";
echo $colors;

This will return “purple,red,green,blue” without a Notice warning.   Using @ is also useful for PHP-GD Image functions for capturing image sizes and resizing.