Author: Alex Underwood

OK, in part one I went over the objectives that I wanted to achieve. In part II I’m going to go over the resources at my disposal. There are aspects of this that are compromise, but at the same time I decided to make priority what I wanted to. This is an interim environment that will carry us over to our 10G roll out. It is designed with a certain level of fault tolerance in mind, although it’s not as high as I would have liked.

Physically I am starting out with 2 Dell 6248s and 2 3750G-24P switches connected to a set of 2 3750G-48P switches acting as the cores. Each switch has a single 1G aggregate in LACP. I have a single uplink to the main network.

Due to having 2 stack cables, I made the decision that I would stack the 3750Gs that were previously acting as standalone switches. The reasoning is that I only have a single uplink so having that level of redundancy at the switch level is not really going to do much of anything. I am running 4 links from the 3750G-48Ps which will be used for the ESXi cluster to the 3750G-24Ps. Due to the stack cables, each of these switches acts as a single larger switch. This means that because I have 2 uplinks from each of the 48 port switches to the 24 port switches I can lose links or have the stacking cable fail on either side and remain up. It also means that I am getting 8GBPS of bandwidth to the core switches, which is going to be more important when I roll out 10G networking through those cores. This means I am going to only use the 3750G-48Ps as the ESXi cluster switches, and eliminate the Dell 6248s in the present configuration entirely. They will be re-used at some later point.

Logically, I’m really starting from scratch overall. There were some attempts at separating VLANs but that’s mangeld overall along with the vSwitch config in ESXi. I’ve decided to split the following items into separate VLANs, which is a very conservative plan but I think will give room to grow.

Non Routable:
-Internal /20 for people to configure networks on. I will probably allocate chunks to users on an IP plan of some form.
-/24 for managed power strips. Way overkill but who cares? Easier than running out of IPs
-/24 for switch management
-/24 for IPMI devices
-/24 for VMotion
-/24 for VMware Management.

Routable:
I have 3 /24s that are on the intranet. One of my goals with this networking function is to try and reclaim as much of that space as plausible just because I’m not sure what my difficulty in getting more space is. I would rather have this space be efficiently used no matter the challenge, so pushing things like IPMI over to non-routable IPs is a priority to me.

The last thing I will mention is I am documenting the physical configuration, port utilization and the addressing (VLANs and IPs.) This is probably the most critical step of doing this sort of reconfiguration. It’s way easier for the next guy (who may be you) to troubleshoot a configuration that it’s known where stuff is. They also tend to scale a lot better.

Next time, I’ll cover what I’m doing in the ESXi configuration and on the switch side. This is the stuff with a big learning curve!

Long time no post! After I became a field tech for EMC, my posting went by the wayside. Amazing how life goes, isn’t it? Well, just in time to not be a new years resolution I’m going to start posting some more stuff up at SMI. I have a ton of new stuff due to becoming Manager of the Lisle Solution Center. The LSC is a lab located at the Lisle EMC Office we actively demo from. It has a myriad of infrastructure from VPLEX to VMAX to XtremIO and software such as ViPR and VMware Site Recovery Manager.  The initial architecture is something that has fallen victim to atrophy that happens when we continuously build and adapt an environment. My goals when designing a network are really as follows:

Cleanliness-This is a biggie. Few like to work on a network that’s a rats nest and poorly organized, fewer still want to have anything to do with rebuilding it. This can be a big issue when service comes into play. Lack of standardization can be a huge issue. We’re actually going to be re-racking this environment at some point into a rack with better cable management (need power for the new rack first) so getting the base line in place is a big priority.

Standardization-There will always be some exceptions to standardization, but having it start to finish is a great thing. This means that when I’m done everything from the order the ports on the servers to the way the vSwitches are configured will have a method to them. This is also procedural standardization which means start to end certain things will be done with host adds.

Documentaion-Documentation is one of the critical steps of a build out that’s often overlooked. Ultimately I want the whole environment documented, but with networking being foundational it’s a very good place to start. Since I’m in charge, it will also make life easier on anyone who works in the LSC.

Lean-Less can easily be more. Some of the key purposes of this network restructure are to reduce switch count that’s needless and eliminate hundreds of feet of cabling. I know that there are many, many environments that could do the same very easily.

Reliable-Although redundancy of design is key, reliability to me is a few aspects. Reliability encompasses the aspects above as well. If you don’t know what goes where and can’t troubleshoot easily then it becomes far easier for human error to come into play.

Next time I will go over the initial environment, the resources at my disposal and the changes I’m looking to make. Stay Tuned!

OK, so we are migrating to a new server and we have a lot of accounts to migrate. Here’s a quick and dirty one liner to make it happen:

grep -v “*: nobody” /etc/userdomains |cut -d: -f2 | sort | uniq | xargs /scripts/pkgacct {} \;

This will remove the generic (nobody) user from the list, separate the user from their respective domain, and execute the pkgacct command.

I had always wanted to have a dumb terminal on my Linux workstation. I always liked the idea of having a text only environment just because I’m not a fan of clicking through a thousand windows to get what I want set up in regards to consoles. That being said, I haven’t had any luck finding any dumb terminals. Mainframe being out of style dumb terminals are getting progressively harder and harder to come by. Thin Clients are actually easy to get via Ebay or other sources, come in a variety of shapes, sizes and capabilities. I ended up buying a random terminal likely without doing adequate research into it, but for the price of $25 shipped I figured it was a minimal investment.

The terminal its self came with a copy of embedded Windows that was trying to connect to a Citrix Server. Considering that is far from what I wanted, I needed to figure out how to get Linux on it. There are a few things you need to know about these terminals. They have an AMD Geode processor, 128 Megabytes of RAM, but more importantly IMO they only have 32 Megabytes of flash. This ultimately is the biggest limitation.

This means you’re really having to chop things down to even get a base OS and some sort of SSH client on them. Make sure to use a USB 3 copier and copy any essential files before starting the boot process. There are really 2 vectors for getting another OS on these clients, the first is USB. Considering this is a USB1 terminal, it is horridly slow to do this way. It does work, but a basic flash drive takes forever to boot.  The other way, the way that I ultimately settled on is PXE booting. There are a few advantages of this IMO, among them being:

-Faster. 100 megs versus 12 megs. Even with a basic initrd it’s obvious to me which wins.

-Ease of migration. You can use this with pretty much any desktop, laptop or terminal that will PXE boot. That’s very convenient if you want to step up to something a bit hotter, or use a random laptop as a spare console.

-Remote connectivity. If I wanted to use this in my garage remotely, all I would have to do is run an Ethernet cable. Makes it really easy to get a VT.  No OS installations to worry about, it “just works.”

-Expandable. We can set this up to connect to a remote X11 server later if we want to.

To actually start the install, we need to change the BIOS. The password for these systems is “Fireport” which makes it easy for us to log in. There’s not a ton of options here, so we will change the boot order and exit.

Linux selection is a mixed bag on these, the fact the Geode is an I586 architecture limits the options pretty significantly. I decided on Slackware as I’ve always liked it, you can use the latest version with an I586 system and it’s fairly easy to “chop.”

On to actually getting this thing booted up in a (non-windows) environment. First things we need to do are set up services. On SuSE this can be a bit more trying than Red Hat due to having to open some ports, but the process is essentially the same. I have a separate NIC as well due to running DHCP on my main network and not wanting to cause conflicts with that. The idea is that the main server will be 192.168.1.1, the terminal will be set up as 192.168.1.2.

/etc/dhcpd.conf:

ddns-update-style none;
default-lease-time 14400;
filename “pxelinux.0″;

# IP address of the dhcp server nothing but this machine.
next-server 192.168.1.1;
subnet 192.168.1.0 netmask 255.255.255.0 {
# ip distribution range between 192.168.1.1 to 192.168.1.100
range 192.168.1.2 192.168.1.100;
default-lease-time 10;
max-lease-time 10;
}

I also edited /etc/sysconfig/dhcpd to set up dhcpd to listen on eth1:

DHCPD_INTERFACE=”eth1”

The next thing I did was install the TFTP server. There’s not much to that, it’s an Xinetd service. Be sure that ports are open, if at all possible I like to try and nmap the server to make sure everything is open and running. After this, we need to add a few things to the config. The first is the pxelinux.0 file which goes in /tftproot/. After this, a pxelinux.cfg directory needs to get created. Add a file to this called default. Since I started with the hugesmp kernel (I would just use the regular huge kernel since this is unicore and uniproc) I set it up the following way:

default hugesmp.s
prompt 1
timeout 1
display message.txt
F1 message.txt
F2 f2.tx
label hugesmp.s
kernel kernels/hugesmp.s/bzImage
append initrd=initrd.img load_ramdisk=1 prompt_ramdisk=0 rw SLACK_KERNEL=hugesmp.s

I copied the initrd.img off the DVD as well as the “kernels” directory into /tftpboot in their entirety. You should be able to actually boot the Slackware initrd at this point, and run any of the setup aps you want. We however, are going to do a lot more with it. This will come in part II where we will tweak it to our means. The cool thing about this initrd is that it has an SSH server as well as an SSH client built into it.