Juniper SRX Flow vs Packet Mode

The Juniper SRX as it comes forwards IP traffic based on flows between security zones.  It can be configured to forward traffic based on packets (no fancy security features).  In packet mode an SRX acts just like a router or layer 3 switch. This is useful for labs and learning.

Run the following command to get an idea of how your SRX is forwarding traffic.
> show security flow status

By default Inet (IPv4) traffic is the only traffic that is configured to forward traffic in flow mode.

To disable this simply delete all of the configuration under the security hierarchy.
# delete security
# commit
# run request system reboot

To enable other traffic types use the following commands

IPv6
# set security fowarding-options family inet6 mode packet-based

MPLS
# set security fowarding-options family mpls mode packet-based

ISO
# set security fowarding-options family iso mode packet-based

You must now commit the configuration and reboot the device.

There is another method to do this that allows you to use both flow and packet mode on the same family which requires firewall rule.  I will go over that in another post.

Juniper SA (Junos Pulse) Multi User Authentication

We’ve had a Juniper SA700 for around 5 years now and it has proven to be an absolutely brilliant bit of hardware, however we’ve always had this issue where it would bump our connection if we connected to the same user multiple times.

To get around this you can enable “Multiple User Sessions” which translates to multiple sessions per user.

On your main window click “Authentication > Signing In” and check the box for “Enable Multiple User Sessions”

juniper_signing_in

Once selected hit save changes.   Now Navigate to “Users > User Realms > [Users, Or other Realm name] > Authentication Policy > Limits”

Change this value to some sane number, you don’t want your system being tied up with dead connections.  We’ve opted for 5.

juniper_limits

 

Now click save and enjoy multiple connections!

The Sword of SEO part II

Well, it’s been a long time since I posted the first article on this. My time or lack thereof got the best of me. To counter this attack is actually very very easy. The first thing you do is you find out who is the referrer. This is simply done by tailing the logs. If you have a single domain, this can be fairly easy. Otherwise my preferred method involves using “watch ls -l” and seeing which log grows the fastest. This tends to be the one getting hit, or a likely suspect. I will probably write a perl script later to check this and tell me which log grows the most in say 10 seconds eventually. After this, you can use tail in the manner of:

tail -f /etc/httpd/domlogs/domain.log

When you do this, you will see what IPs are querying the page and the source they are being referred from. Look for any thing that doesn’t look like a search engine. To actually block them after they are identified what you do is you block the attack based on a referrer in the .htaccess. See the convenient rewrite code I jacked off another web site (about the same I did when I really saw the attack.)

RewriteEngine on
# Options +FollowSymlinks
RewriteCond %{HTTP_REFERER} attacker\.com [NC]
RewriteRule .* – [F]

So, why does this work you may ask? In the case of the scenario I saw the person was attacking a “high value” target. This means a page that hits the database and has dynamically generated content with no caching. Server side configuration CAN make these sort of attacks a lot harder to perpetrate as well. Anything that you can do to increase the robustness of a server will help with a DoS. When you add a rule like this where it denies access to the referrer basically what happens is you pull up static content instead. Static content uses virtually no resources compared to something PHP based and backed by a databse. It’s a good idea to know about this sort of attack, as I could see it being bigger in the future. Black hat SEO is very common these days, and if you have the SEO part down the resources to do the rest of this attack are virtually nothing compared to what it does. It could also be plausible we will see this attack combined with “conventional, network level” type DoSing to increase its effectiveness.

The sword of SEO

I was on a client server getting attacked, the DoS was heavily distributed. Since he’d mentioned something about someone linking to his web site, I was poking through the Apache logs. I noticed that one site was generating a huge amount of referrals. Investigating deeper, Ifound this on the referral site:

<iframe src=”http://www.domain.com” width=”1″ height=”1″ ></iframe>
0<br><iframe src=”http://www.domain.com” width=”1″ height=”1″ ></iframe>
1<br><iframe src=”http://www.domain.com” width=”1″ height=”1″ ></iframe>
2<br><iframe src=”http://www.domain.com” width=”1″ height=”1″ ></iframe>
3<br><iframe src=”http://www.domain.com” width=”1″ height=”1″ ></iframe>

…….

30<br><iframe src=”http://www.domain.com” width=”1″ height=”1″ ></iframe>

This is one of the slicker DoSes I’ve seen in a while. Because of the way it was set up it would be very difficult if not impossible to block on a network level and not traceable back to any particular IP on a network level (read:iptables, RTG or hardware firewall.) Within a few assumptions here this is what I believe to happen:

-Person sets up a web site with just a park page etc. on it.
-Person directs traffic to this using SEO. (back links, etc) to gain it status on search engines
-Person puts up the attack page similar to the above
-Every time a person from a search engine clicks the link, they load a few dozen copies of the page
-The iframe points to a “high value” target that generates a lot of load on the server, such as a forum or other dynamic content.

I personally saw this attack decimate a late model server with 16GB of RAM with enough IP distribution that it was not plausible to block it. It is viciously effective when planned out and done properly. It can also be done with virtually NO resources using a free shared hosting account. The person who loads it probably never realized they just made an attack on a server either. The plus side is that if you track it you can limit the damage done very easily provided you know what you are looking for. That will be my next blog.

What Chattr can do for you

Chattr is one of those great utilities in a security tool box. Since log files are a prime target of removal when root kits and other scripts are used on a system after the compromise, it’s an ideal way to help prevent destruction of the logs. Besides this, if you don’t want certain files tampered with you can use chattr to make them non writeable by anyone including root. This is done at a file system (inode) level regardless of what is there. My favorite options are:

+a. This sets a file append only. This is great for logs, and will catch a lot of script kiddies off guard. If overwriting of data is attempted it will deny the operation. as an example:

[root@DNS02 ~]# chattr +a ./new.file
[root@DNS02 ~]# echo “aghaklsjdfhadlwadjhad” >> new.file
[root@DNS02 ~]# echo “aghaklsjdfhadlwadjhad” > new.file
-bash: new.file: Operation not permitted

Since the first echo adds it to the end it will work. The second echo attempts to overwrite the file contents so it gives a permission error. Please note that this is even though the user has full permissions to access the file. to reverse this we use chattr -a new.file

The +i function is a little different, it will make the file totally unchangeable. If we have binaries that we consider sensitive this may be an option for it. I have had systems I needed to get back up with rootkits that would change files on the boot, this would take care of bringing them back up without allowing whatever was on the system to modify it.

The +u option allows for undeleting of files, this is another thing that may be good for sensitive information such as logs or key backup archives. Would be tempted to use it with +a as well for logs.

The +c option gives the files on disk compression. Never really played with it much however I would think it could be detrimental with large archives on RAM and CPU bound systems.

How To: Config Server & Firewall Installation Guide

I’ve written this guide to help you you through installing and configuring CSF – Config Server Firewall, a popular alternative to APF. CSF comes with LFD and works with or without cPanel.

CSF Install Guide and How To

CSF – Config Server Firewall is a stateful packet inspection firewall, login/intrusion detection and security application for Linux servers. What does that mean in English? Simple – it’s a program that can greatly improve your dedicated server or VPS’s security.

It’s a firewall – so it can block/restrict ports you don’t want open, and prevents someone from using any port they want if they did break in.
It has intrusion detection – so it will scan the log files and monitor failed login attempts, such as FTP password guessing and block the IP.
Those are the 2 big things I like about CSF – and it has a nice interface for the non techie person, on cPanel servers.

http://www.configserver.com/cp/csf.html

If you have another firewall installed, like APF, CSF can help you automatically remove the existing firewall and install theirs instead. Keep in mind it won’t migrate over your configuration.

Installation

Installation is quite straightforward:

Login as the root user to SSH and run the following commands.

rm -fv csf.tgz
wget http://www.configserver.com/free/csf.tgz
tar  -xzf csf.tgz
cd csf
sh install.sh

If you would like to disable APF+BFD (which you will need to do if you have
them installed otherwise they will conflict horribly):

sh disable_apf_bfd.sh

That’s it. You can then configure csf and lfd in WHM, or edit the files
directly in /etc/csf/*

Installation Completed

Don’t forget to:

1. Configure the TCP_IN, TCP_OUT, UDP_IN and UDP_OUT options in the csf configuration to suite your server

2. Restart csf and lfd

3. Set TESTING to 0 once you’re happy with the firewall

csf is preconfigured to work on a cPanel server with all the standard cPanel
ports open. It also auto-configures your SSH port if it’s non-standard on
installation.

You should ensure that kernel logging daemon (klogd) is enabled. Typically, VPS
servers have this disabled and you should check /etc/init.d/syslog and make
sure that any klogd lines are not commented out. If you change the file,
remember to restart syslog.

Now – login to your cPanel server’s WHM as root and go to the bottom left menu. If already logged in then reload the page. In Plugins – you will see:  ConfigServer Security&Firewall

The firewall is STOPPED by default – it is not running. We need to configured it, and then take it out of Test Mode.

Click on Firewall Configuration

ETH_DEVICE =: Set this to eth+

TCP_IN/TCP_OUT/UDP_IN/UDP_OUT = : These are the ports you want to leave open for your server to operate. If you change the default SSH port make sure to add it here. Also add any other services you might have running such as Shoutcast or game servers. By default most of the ports used should already be configured.

MONOLITHIC_KERNEL = : 0 Only change this to 1 if your firewall will not start – otherwise leave it as it.

LF_DSHIELD = 0: Change this option to 86400. This is an automatic updated list of known attacking IPs. Enabling this will stop them from being able to connect to your server.

Spam Protection Alerts

If you want to add some spam protection, CSF can help. Look in the configuraiton for the following:

LF_SCRIPT_ALERT = 0 change this to 1. This will send an email alert to the system administrator when the limit configured below is reached within an hour.

LF_SCRIPT_LIMIT = 100 change this to 250. This will alert you when any scripts sends out 250 email messages in an hour.

Configuration Complete – Almost

Scroll down to the bottom and click on Change to save the settings. Then click Restart csf+lfd

You should see a big page of ACCEPT and near the bottom you should see:

csf: TESTING mode is enabled – don’t forget to disable it in the configuration

Starting lfd:[  OK  ]

Click on Return

Now TEST all your services to make sure everything is working – SSH, FTP, http. After you do a few quick tests go back into the Firewall Configuration page.

TESTING = 1 change this to 0 and click Change at the bottom. Then Restart csf+lfd

That’s it, the firewall is successfully installed and running!!
Firewall Status: Running – you should see this on the main CSF page in WHM.

Uninstallation

==============
Removing csf and lfd is even more simple:

cd /etc/csf
sh uninstall.sh

How To: VNCServer on Fedora

Virtual Network Computing (VNC) is a desktop protocol to remotely control another computer. It transmits the keyboard presses and mouse clicks from one computer to another relaying the screen updates back in the other direction, over a network.” -WikiPedia-

This article describes in brief how to configure VNC server instances for one or multiple users on a remote machine, how to use VNC to start graphical applications on boot and finally how to enhance security by connecting to the server through encrypted SSH tunnels.

Prerequisites

A user account should exist on the remote machine.
The RPM packages xorg-x11-xauth, vnc-server and vnc should be installed on the remote machine and your workstation respectively.

yum install -y xorg-x11-xauth vnc-server vnc xsetroot xterm 

Setting up the server

I assume that we have setup a remote user account, named “Tyler” and we want to start an X session through VNC for this user.

In Red Hat based distros in general, all we have to do is define the VNC server instances in /etc/sysconfig/vncservers. These will be started by the vncserver initscript. This has to be done as root. Edit this file so that it contains the following:

VNCSERVERS="1:tyler"
VNCSERVERARGS[1]="-geometry 1024x768 -depth 16"

With these we define that a vnc server instance should be started as user Tyler on display 1 and we also set some options for this server such as resolution and color depth. Each VNC server instance listens on port 5900 plus the display number on which the server runs. In our case, Tylers’s vnc server would listen on port 5901.

For multiple vnc instances /etc/sysconfig/vncservers would look like this:

VNCSERVERS="1:tyler 2:albatros 3:leopard"
VNCSERVERARGS[1]="-geometry 1024x768 -depth 16"
VNCSERVERARGS[2]="-geometry 800x600 -depth 8"
VNCSERVERARGS[3]="-geometry 1024x768 -depth 16"

These would listen on ports 5901, 5902, 5903 respectively.

User Configuration

There is one more thing that needs to be done on the remote machine. User Tyler’s vnc password needs to be set. So, as user Tyler give the command:

# su -l Tyler
# vncpasswd
# exit

We are prompted for a password. This is the password that we will use when we connect to Tyler’s vnc server instance. This password is saved in /home/Tyler/.vnc/passwd.

Start the VNC server

After the initial configuration is done we restart the vnc service. As root:

# service vncserver restart

To make VNC server to start on boot:

# chkconfig vncserver on

More User Configuration

After the VNC service is started, some new files are created in /home/Tyler/.vnc/ directory. These include Tyler’s vnc server log file, pid file and an X startup script. As user Tyler we edit the script in order to customize some settings. The default /home/Tyler/.vnc/xstartup script contains some commands that are executed when the VNC server is started. These include:

xsetroot -solid grey
vncconfig -iconic &
xterm -geometry 80x24+10+10 -ls -title "$VNCDESKTOP Desktop" &
twm &

xsetroot in this case sets the background color.
vncconfig is a supplementary program that can be used to control the vnc server. Apart from this, when run without arguments it acts as a helper application and its main purpose is to provide support for clipboard transfers between the client (vncviewer) and the vnc server.
xterm starts an xterm terminal.
twm starts the X server’s default window manager. We probably want to change that to a more user friendly window manager, eg fluxbox.

The VNC server, apart from letting us control a remote machine using a graphical interface, it serves as a way to start graphical applications on boot. For example, I want my favourite p2p program, amule, to start on boot. So, I add this to the /home/Tyler/.vnc/xstartup script. This is how my xstartup file looks like:

xsetroot -solid grey
vncconfig -iconic &
xterm -geometry 80x24+10+10 -ls -title "$VNCDESKTOP Desktop" -e ./menu &
amule &
fluxbox &

menu is a script of mine that is executed when xterm is started.
Remember to put the “&” symbol after each command, so that it goes to the background and the xstartup script continues on.

Restart the VNC service for the changes to take effect. As root:

# service vncserver restart

Connect to the VNC server

In our example, Tyler’s vnc server listens for connections on port 5903. So, open this port in the remote machine’s firewall.

We connect to the remote machine using a vnc viewer. Having installed the vnc package, connect to to the server with the following command:

# vncviewer 192.168.0.1:5903:3

The general usage is :

vncviewer [Server's IP]:[Port]:[Display]

We are prompted for the password and eventually connect to the server. Closing the vncviewer’s window, does not affect the server or the programs we run on it. If we reconnect everything will be there.

Special Note: There is no need, actually it’s pointless and could give you some trouble, to logoff from your remote X session. If this happens, generally you need to restart the VNC service on the remote machine to get your remote desktop back. If you want to stop working on your remote desktop, just close the vncviewer’s window and you are done.

Security

The VNC protocol is not a secure communication protocol. The use of a vnc password provides security at the level of server access (it’s vulnerable to brute-force attacks though), but the whole VNC session is transmitted in the clear, without encryption. The easiest, but most effective, way to secure our connection to the VNC server is to connect through an encrypted SSH tunnel. This way the whole session will be encrypted.

The rest assume that you have the SSH server up and running on your remote machine (server.example.com) and you know what SSH tunnels are.

So, what we are going to do is to create an encrypted tunnel, and connect to our VNC server through it. We also want this tunnel to be automatically closed as soon as we shut down vncviewer. All this is done with the following command:

# ssh -f -L 25903:127.0.0.1:5903 Tyler@server.example.com sleep 10; vncviewer 127.0.0.1:25903:3

This is what it does:

  • -L 25903:127.0.0.1:5903 forwards our local port 25903 to port 5903 on the remote machine. In other words, it creates the tunnel.
  • -f forks the SSH session to the background, while sleep is being executed on the remote machine. This ssh option is needed because we want to execute the following command (vncviewer) in the same local machine’s terminal.
  • vncviewer connects to the forwarded local port 25903 in order to connect to the VNC server through the encrypted tunnel.

The sleep command is of major importance in the above line as it keeps the encrypted tunnel open for 10 seconds. If no application uses it during this period of time, then it’s closed. Contrariwise, if an application uses it during the 10 sec period, then the tunnel remains open until this application is shut down. This way the tunnel is automatically closed at the time we close vncviewer’s window, without leaving any SSH processes running on our workstation. This is pure convenience! More information can be found at the Auto-closing SSH Tunnels article.

Using SSH tunnels to conect to your VNC server has two advantages:

  1. The whole session is encrypted.
  2. Keeping port 5903 open on your remote machine is no longer needed, since all take place through the SSH tunnel. So, noone will know that you run a VNC server on the remote machine.

Further Reading

I recommend that you read the man pages. Everything is in there:

# man vncserver
# man Xvnc
# man vncconfig
# man vncviewer
# man ssh

Basic MySQL backups

Backing data up in MySQL is super important. The site content is often replaceable, however your board posts, ecommerce orders, etc. are far far harder to replace, if at all possible. The other nice thing is that with a lot of database driven web sites you retain most content even if you only back up the databases. You can still lose things like images, but by and large your posts will still be there even in event of a crash. Since bad code, etc. happens there are a few reasons I would recommend doing backups of the databases.

  • Making server side changes, including configuration tweaks that can influence system stability.
  • Making software changes, even adding plugins etc. to your site. I have seen odd things in the past caused by plugins and they can interact together with not always predictable results.
  • Periodical backups as part of a data protection plan. I would do more than less, by and large most databases are not huge for the data they contain. I have seen forums with about 100,000 posts have about 2GB of database; not really that much if you think about it.

There are a few ways to do backups, but the most common for people who don’t use command line interface is probably going to be PHPMyAdmin. Quite frankly, I do not like using it for database backup and restorations. There are two reasons for this pertaining to reliability

  • If the session timeout of PHP or Apache is too low, you will get an incomplete dump. This means that backup you may have of your databases is all of a sudden found to be truncated at a few megabytes. This is not a big deal for a lot of people with small DBs however if you’re rocking out with 50+ megabyte DBs this can be a huge issue.
  • Restoration can be dependent on if the person has large uploads and large timeouts enabled. Since most people have slower upload than down, it can be a real nuisance to get the DB back up to the server and restored. Not only that but If you have a server with low upload size limits and timeouts, it may be impossible to get that DB back up and restored to the system before you hit one of these limits.

The way I honestly prefer to back my databases up is mysqldump; this allows one to have a lot of control over what gets dumped, do a full backup without worrying about doing it individually etc. It also allows us to make a nice easy cron job to do these backups too if we want.

The way we back a particular DB is up is:

mysqldump dbname > backup.sql

This will put the database named dbname into backup.sql. I believe you can separate multiple DBs with a space as well, however I don’t normally use that functionality. Another thing you can do is you can back up everything with:

mysqldump --all-databases > backup.sql

and this will do everything on the server. Handy for migrations, etc. Definitely way easier than trying to do this via a web interface and having 1,000 DBs.

How we restore these is fairly simple as well, we just run the command as follows:

mysql dbname < backup.sql

If you have a full DB dump you wouldn’t specify anything for the name when restoring. Notice how the carrat points in the opposing direction though. Don’t screw this up otherwise you will likely end up with a blank file and have to re-dump or re-move the file. I usually make 2 copies on the destination server especially when it’s not easy to get a second copy. There are other ways of moving the DBs over, such as copying the raw MYI files but that will be another blog.

Mod_limitipconn

I have had a few clients with chronic security issues on machines, DoSes are by far the most common attack. It’s easy enough for anyone of average (or below average, up for debate) intelligence to carry out. A few zombie machines and some basic software and all of a sudden you can take down a decently massive web site.

There are a few caveats to mod_limitipconn, the biggest being if you run a site with a very large amount of images. Images are loaded last on a page, and also are loaded at once. This can be a perilous position because of the fact that mod_limitipconn can cause issues with dead images if not done properly. This is why you have to tune it. In general there is no CPanel integration so you’re going to be doing this by hand. For you people out there with other panels, your milage may vary.

First thing is installing it. This is a relatively simple process, be sure you have APXS installed so you can do this. Download the software from http://dominia.org/djao/limitipconn2.html and install it using their commands. You can also probably get packages from a place like Dag if that’s your thing. Certain distros such as Debian also come with a ready to rock version. After we get it installed, we need to go into httpd.conf and make some alterations.

MaxConnPerIP 10
NoIPLimit images/*

There are a few things to note here. For people that have static content on their systems, you would set an instance of “images” for each static content directory that you were using. This includes things like Javascript, CSS, and conventional images. The reason for this is if using mod_limitipconn for system hardening an attacker will usually not target an image. This can happen, but it’s also not as “profitable” as a DoS using a page because you don’t pull the images and other content on the page (increasing bandwidth usage) and static content tends to be super efficient compared to PHP or other server side scripting so it tends to be far more robust.

You can also do other cool tricks such as format based setting of the connection limits.


# local per-directory settings here
MaxConnPerIP 1
OnlyIPLimit audio/mpeg video

By putting this in with the main confiugration above, you can set up additional subdirectories and limit connections based on media type. This can slow a leecher down significantly because they can’t pull everything at once.

Since we have the basics covered, just a few words on tuning the configuration. The biggest thing you need to do is view the sites you use this on. Sites with mod_limitipconn can behave erratic behavior if an initial setup is done but left untuned. This can include random broken images and other content not showing up. There are two things we are concerned with to tune; the first is that the most static content possible is set to very loose or no limitations, the second being that you get the base connections as low as sanely possible. If you can get it around 5 that would likely be the most pages someone would “sanely” load of a typical site without having issues. The other thing is that if you have embedded pages in your site you will have to increase this number significantly. Just go slowly and when the page fully loads add say 10-20% to that amount. If you get reports of broken content it would likely be safe to up this number some at that point to “dial in” that perfect configuration.