How to manage ceph.conf

Managing ceph is a little bit interesting because of how the data is referenced vs how it is actually used. One of the main headaches  with ceph is making changes to osds while they are already running.   I will go into detail on that later once I’ve had time to properly document it.

Heres how My cluster is configured

  • ceph0-mon0 (Mon, Management)
  • ceph0-mon1 (Mon)
  • ceph0-mon2 (Mon)
  • ceph0-n0de0 (OSD)
  • ceph0-n0de1 (OSD)
  • ceph0-n0de2 (OSD)

In /home/ceph on ceph0-mon0 you will have ceph.conf, this was generated by ceph-deploy originally.  Edit this file to adjust your configuration and then run this to distribute that configuration to the monitors and nodes.   I am using the increment naming convention.  (http://ceph.com/docs/master/rados/deployment/ceph-deploy-admin/)

ceph-deploy –overwrite-conf config push ceph0-mon{0,1,2} ceph0-node{0,1,2}

 

Perforce P4D init.d script (CentOS)

Basic init script to control p4 / p4d for perforce.

Uses /var/p4 as the working directory and p4service as the user.

#!/bin/sh
#
#
# Startup/shutdown script for Perforce
#

# Source function library. this is where ‘daemon’ comes from
. /etc/init.d/functions

prog=Perforce Server

p4d_bin=/usr/local/bin/p4d
p4_bin=/usr/local/bin/p4
p4user=p4service
p4authserver=p4authserver:1667
p4root=/var/p4/root
p4journal=/var/p4/journal
p4port=1818
p4log=/var/p4/log
p4loglevel=3

start () {
echo -n $”Starting $prog: ”

# start

#If you wish to use a perforce auth server add this into the below command line.
# -a $p4authserver

 

#Start the daemon as the p4user.

/bin/su $p4user -c “$p4d_bin -r $p4root -J $p4journal -p $p4port -L $p4log -v server=$p4loglevel -d” &>/dev/null
}

stop () {
# stop
echo -n $”Stopping $prog: ”
$p4_bin -p $p4port admin stop
}

restart() {
stop
start
}

case $1 in
start)
start
;;
stop)
stop
;;
restart)
restart
;;
*)

echo $”Usage: $prog {start|stop|restart}”
exit 3
esac

exit $RETVAL

Quick and Dirty Ceph Deployment

Replace the disk names and ssd device name.   This will build a ceph cluster with 2 object redundancy in about 5 minutes.

ceph-deploy purge ceph0-mon0 ceph0-mon1 ceph0-mon2 ceph0-node0 ceph0-node1
ceph-deploy purgedata ceph0-mon0 ceph0-mon1 ceph0-mon2 ceph0-node0 ceph0-node1
ceph-deploy forgetkeys


ceph-deploy new ceph0-mon0 ceph0-mon1 ceph0-mon2

echo "osd pool default size = 2" >> ~/ceph.conf
echo "public network = 10.1.8.0/22" >> ~/ceph.conf
echo "cluster network = 10.1.12.0/22" >> ~/ceph.conf
echo "osd journal size = 12000" >> ~/ceph.conf

ceph-deploy install ceph0-mon0 ceph0-mon1 ceph0-mon2 ceph0-node0 ceph0-node1
ceph-deploy mon create-initial

ceph-deploy admin ceph0-mon0 ceph0-mon1 ceph0-mon2 ceph0-node0 ceph0-node1

sudo chmod +r /etc/ceph/ceph.client.admin.keyring

ceph-deploy disk zap ceph0-node0:/dev/oczpcie_4_0_ssd
ceph-deploy disk zap ceph0-node0:/dev/sdb
ceph-deploy disk zap ceph0-node0:/dev/sdc
ceph-deploy disk zap ceph0-node0:/dev/sdd
ceph-deploy disk zap ceph0-node0:/dev/sde
ceph-deploy disk zap ceph0-node0:/dev/sdf
ceph-deploy disk zap ceph0-node0:/dev/sdg
ceph-deploy disk zap ceph0-node0:/dev/sdh
ceph-deploy disk zap ceph0-node0:/dev/sdi
ceph-deploy disk zap ceph0-node0:/dev/sdj
ceph-deploy disk zap ceph0-node0:/dev/sdk
ceph-deploy disk zap ceph0-node0:/dev/sdl
ceph-deploy disk zap ceph0-node0:/dev/sdm

ceph-deploy disk zap ceph0-node1:/dev/oczpcie_4_0_ssd
ceph-deploy disk zap ceph0-node1:/dev/sdb
ceph-deploy disk zap ceph0-node1:/dev/sdc
ceph-deploy disk zap ceph0-node1:/dev/sdd
ceph-deploy disk zap ceph0-node1:/dev/sde
ceph-deploy disk zap ceph0-node1:/dev/sdf
ceph-deploy disk zap ceph0-node1:/dev/sdg
ceph-deploy disk zap ceph0-node1:/dev/sdh
ceph-deploy disk zap ceph0-node1:/dev/sdi
ceph-deploy disk zap ceph0-node1:/dev/sdj
ceph-deploy disk zap ceph0-node1:/dev/sdk
ceph-deploy disk zap ceph0-node1:/dev/sdl
ceph-deploy disk zap ceph0-node1:/dev/sdm

ceph-deploy osd prepare ceph0-node0:/dev/sdb:/dev/oczpcie_4_0_ssd
ceph-deploy osd prepare ceph0-node1:/dev/sdb:/dev/oczpcie_4_0_ssd

ceph-deploy osd prepare ceph0-node0:/dev/sdc:/dev/oczpcie_4_0_ssd
ceph-deploy osd prepare ceph0-node1:/dev/sdc:/dev/oczpcie_4_0_ssd

ceph-deploy osd prepare ceph0-node0:/dev/sdd:/dev/oczpcie_4_0_ssd
ceph-deploy osd prepare ceph0-node1:/dev/sdd:/dev/oczpcie_4_0_ssd

ceph-deploy osd prepare ceph0-node0:/dev/sde:/dev/oczpcie_4_0_ssd
ceph-deploy osd prepare ceph0-node1:/dev/sde:/dev/oczpcie_4_0_ssd

ceph-deploy osd prepare ceph0-node0:/dev/sdf:/dev/oczpcie_4_0_ssd
ceph-deploy osd prepare ceph0-node1:/dev/sdf:/dev/oczpcie_4_0_ssd

ceph-deploy osd prepare ceph0-node0:/dev/sdg:/dev/oczpcie_4_0_ssd
ceph-deploy osd prepare ceph0-node1:/dev/sdg:/dev/oczpcie_4_0_ssd

ceph-deploy osd prepare ceph0-node0:/dev/sdh:/dev/oczpcie_4_0_ssd
ceph-deploy osd prepare ceph0-node1:/dev/sdh:/dev/oczpcie_4_0_ssd

ceph-deploy osd prepare ceph0-node0:/dev/sdi:/dev/oczpcie_4_0_ssd
ceph-deploy osd prepare ceph0-node1:/dev/sdi:/dev/oczpcie_4_0_ssd

ceph-deploy osd prepare ceph0-node0:/dev/sdj:/dev/oczpcie_4_0_ssd
ceph-deploy osd prepare ceph0-node1:/dev/sdj:/dev/oczpcie_4_0_ssd

ceph-deploy osd prepare ceph0-node0:/dev/sdk:/dev/oczpcie_4_0_ssd
ceph-deploy osd prepare ceph0-node1:/dev/sdk:/dev/oczpcie_4_0_ssd

ceph-deploy osd prepare ceph0-node0:/dev/sdl:/dev/oczpcie_4_0_ssd
ceph-deploy osd prepare ceph0-node1:/dev/sdl:/dev/oczpcie_4_0_ssd

ceph-deploy osd prepare ceph0-node0:/dev/sdm:/dev/oczpcie_4_0_ssd
ceph-deploy osd prepare ceph0-node1:/dev/sdm:/dev/oczpcie_4_0_ssd

CentOS 7 / Fedora systemctl driven kexec reboots

Simply create these files:

File: /usr/bin/kexec-load

#!/usr/bin/env bash

GRUBBY_FILE="/var/log/grubby"
TMP=$(mktemp)

#  Command "grubby --default-kernel" has a bug/feature that fsyncs
#  after writting each line to a debug log file, making it slow (several seconds).
#  Workaround is to write to /dev/null instead.
if [ -e $GRUBBY_FILE ]
        then rm -f $GRUBBY_FILE
fi
ln -s /dev/null $GRUBBY_FILE
KERNEL_IMG=$(grubby --default-kernel)
unlink $GRUBBY_FILE

#  Get the detailed information of the default kernel (as seen by grub)
#  This will create a temporary file in /tmp
grubby --info=$KERNEL_IMG | grep -v title > $TMP
source $TMP
rm $TMP

#  Simple log to see if this script gets executed
date --rfc-3339=seconds >> /var/log/kexec

#  Load (prepare) the kernel for execution
kexec -l $kernel --initrd=$initrd --command-line="root=$root $args"

File: /etc/systemd/system/kexec-load.service

[Unit]
Description=loads the kernel
Documentation=man:kexec(8)
DefaultDependencies=no
Before=shutdown.target umount.target final.target

[Service]
Type=oneshot
ExecStart=/usr/bin/kexec-load

[Install]
WantedBy=kexec.target 

Then to enable it and reboot:

$ chmod +x /usr/bin/kexec-load
$ systemctl enable kexec-load.service
$ systemctl kexec

 

Linux SYN Attacks

Block it fast with CSF:

SYNFLOOD = “1”
SYNFLOOD_RATE = “30/s”
SYNFLOOD_BURST = “5”

Good Articles

  • http://www.cyberciti.biz/faq/check-network-connection-linux/
  • http://www.cyberciti.biz/tips/howto-limit-linux-syn-attacks.html

 

Checkout TCPTrack: http://pkgs.repoforge.org/tcptrack/tcptrack-1.4.0-1.el6.rf.x86_64.rpm