raid0 for varnish

To speed up varnish i choosen to cache data to a mdadm raid0 partition on half the drives (rest is raid1).

Create raid0 partitions (done twice)

(parted) print
Model: ATA ST2000NC001-1DY1 (scsi)
Disk /dev/sda: 2000GB
Sector size (logical/physical): 512B/4096B
Partition Table: msdos
Disk Flags:
Number Start End Size Type File system Flags
1 1049kB 1001MB 1000MB primary boot, raid
2 1001MB 1009GB 1008GB primary raid

(parted) mkpart
Partition type? primary/extended? primary
File system type? [ext2]? xfs
Start? 1100GB
End? 1900GB

(parted) set 3 raid on
(parted) print
Model: ATA ST2000NC001-1DY1 (scsi)
Disk /dev/sda: 2000GB
Sector size (logical/physical): 512B/4096B
Partition Table: msdos
Disk Flags:

Number Start End Size Type File system Flags
1 1049kB 1001MB 1000MB primary boot, raid
2 1001MB 1009GB 1008GB primary raid
3 1100GB 1900GB 800GB primary raid

mdadm raid0

mdadm -C /dev/md0 -l raid0 -n 2 /dev/sd[a-b]3
mkfs.xfs /dev/md0
mdadm --detail --scan >> /etc/mdadm.conf

fstab

To make sure this server can start evan if the raid0 partition fails, add nofail. noatime is added since atime is not needed for this drive.

/dev/md0  /var/lib/varnish/                     xfs     defaults,nofail,noatime 0 0

If it fails

If the drive fails varnish will crash, at a reboot varnish will not start (this is because the file varnish want to use dont fit within the root partition. to fix the raid0 partition, recreate all steps it and mount. dont forget to update /etc/mdadm.conf.

Deploy keys Github

Deploy keys gives you read access by default to a single repository. Deploy keys can be given write access when created in github.

In CentOS 7.

yum install rh-git29-git -y

Get public key to import to github.

cat ~/.ssh/id_rsa.pub

If you dont have a private/public key create one witht he following command

ssh-keygen -t rsa -b 4096 -C "you@email.com"

To test validation

ssh -T git@github.com

Error unpacking rpm package chkconfig

Trying to update some CentOS servers to in this case CentOS 7.4 i stumbled onto a problem there it was not possible to perform a update on chkconfig.

[root@moln.dalesjo.net ~]# yum update -y
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirror.zetup.net
 * epel: mirror.zetup.net
 * extras: mirror.zetup.net
 * updates: mirror.zetup.net
Resolving Dependencies
--> Running transaction check
---> Package chkconfig.x86_64 0:1.7.2-1.el7 will be updated
---> Package chkconfig.x86_64 0:1.7.4-1.el7 will be an update
--> Finished Dependency Resolution

Dependencies Resolved

====================================================================================================================================================================================================================================================
 Package Arch Version Repository Size
====================================================================================================================================================================================================================================================
Updating:
 chkconfig x86_64 1.7.4-1.el7 base 181 k

Transaction Summary
====================================================================================================================================================================================================================================================
Upgrade 1 Package

Total download size: 181 k
Downloading packages:
Delta RPMs disabled because /usr/bin/applydeltarpm not installed.
chkconfig-1.7.4-1.el7.x86_64.rpm | 181 kB 00:00:00
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
 Updating : chkconfig-1.7.4-1.el7.x86_64 1/2
Error unpacking rpm package chkconfig-1.7.4-1.el7.x86_64
error: unpacking of archive failed on file /etc/init.d: cpio: rename
chkconfig-1.7.2-1.el7.x86_64 was supposed to be removed but is not!
 Verifying : chkconfig-1.7.2-1.el7.x86_64 1/2
 Verifying : chkconfig-1.7.4-1.el7.x86_64 2/2

Failed:
 chkconfig.x86_64 0:1.7.2-1.el7 chkconfig.x86_64 0:1.7.4-1.el7

Complete!

The problem occured because /etc/init.d was a directory instead of a softlink to /etc/rc.d/init.d/. To fix it, simply do

mv /etc/init.d/* /etc/rc.d/init.d/
rm /etc/init.d -r
ln -s /etc/rc.d/init.d/ /etc/init.d

Extended permissions on backup directory

Using rsnapshot to do backups I want to enable some users the availability to easily restore a single file from the snapshot. Problem is that rsnapshot also is keeping the original permissions for each file. To solve this i will create a usergroup called dalesjo-backup and give this group read access to all files in the backup using an Access Control list.

Enable ACL

First you need to enable ACL on your zfs pool in this case zfs-pool-2

zfs set acltype=posixacl zfs-pool-2

If you dont do this setfacl will return the error below.

setfacl: .: Operation not supported

Set Filepermissions

Below I’m setting a default acl giving dalesjo-backup read/execute permissions on all new files. And after that changing all currently existing files to give read/execute access to the same group.

cd /zfs-pool-2/backup
setfacl -Rdm "g:dalesjo-backup:rx" .
setfacl -Rm "g:dalesjo-backup:rx" .
getfacl .

Source: Serverfault

Extending snmpd with sensor data from ipmitool

Create script

Create a script that returns the data you want to monitor through snmp. the output should be cleaned from whitespaces.

My examples monitor sensor data from ipmi by creating cron script that outputs ipmitool sensor data to  a temp file every two minutes. And then parsing it with snmp_ipmi from github. Cron script below.

# .--------------------------- minute (0 - 59)
# | .------------- hour (0 - 23)
# | | .---------- day of month (1 - 31)
# | | | .------- month (1 - 12) OR jan,feb,mar,apr ...
# | | | | .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat
# | | | | |
# * * * * * user-name command to be executed
 */2 * * * * root ipmitool sensor list > /tmp/ipmi_sensor_list

Add script to snmpd

Edit /etc/snmp/snmpd.conf and add your scripts

rouser OP5View

extend "System Fan 1" /usr/lib64/nagios/plugins/custom/ipmi/snmp_ipmi -f /tmp/ipmi_sensor_list -u "System Fan 1"
extend "System Fan 3" /usr/lib64/nagios/plugins/custom/ipmi/snmp_ipmi -f /tmp/ipmi_sensor_list -u "System Fan 3"
extend "System Fan 4" /usr/lib64/nagios/plugins/custom/ipmi/snmp_ipmi -f /tmp/ipmi_sensor_list -u "System Fan 4"

Test

restart snmpd and run the following command to see your snmp data. example output below.

# snmpwalk -u OP5View -A password -a SHA -X encryptionkey -x AES -l authPriv -v3 localhost .1.3.6.1.4.1.8072.1.3
NET-SNMP-EXTEND-MIB::nsExtendNumEntries.0 = INTEGER: 4
NET-SNMP-EXTEND-MIB::nsExtendCommand."test" = STRING: /bin/echo
NET-SNMP-EXTEND-MIB::nsExtendCommand."System Fan 1" = STRING: /usr/lib64/nagios/plugins/custom/ipmi/snmp_ipmi
NET-SNMP-EXTEND-MIB::nsExtendCommand."System Fan 3" = STRING: /usr/lib64/nagios/plugins/custom/ipmi/snmp_ipmi
NET-SNMP-EXTEND-MIB::nsExtendCommand."System Fan 4" = STRING: /usr/lib64/nagios/plugins/custom/ipmi/snmp_ipmi
NET-SNMP-EXTEND-MIB::nsExtendArgs."test" = STRING: hello
NET-SNMP-EXTEND-MIB::nsExtendArgs."System Fan 1" = STRING: -f /tmp/ipmi_sensor_list -u \"System Fan 1\"
NET-SNMP-EXTEND-MIB::nsExtendArgs."System Fan 3" = STRING: -f /tmp/ipmi_sensor_list -u \"System Fan 3\"
NET-SNMP-EXTEND-MIB::nsExtendArgs."System Fan 4" = STRING: -f /tmp/ipmi_sensor_list -u \"System Fan 4\"
NET-SNMP-EXTEND-MIB::nsExtendInput."test" = STRING:
NET-SNMP-EXTEND-MIB::nsExtendInput."System Fan 1" = STRING:
NET-SNMP-EXTEND-MIB::nsExtendInput."System Fan 3" = STRING:
NET-SNMP-EXTEND-MIB::nsExtendInput."System Fan 4" = STRING:
NET-SNMP-EXTEND-MIB::nsExtendCacheTime."test" = INTEGER: 5
NET-SNMP-EXTEND-MIB::nsExtendCacheTime."System Fan 1" = INTEGER: 5
NET-SNMP-EXTEND-MIB::nsExtendCacheTime."System Fan 3" = INTEGER: 5
NET-SNMP-EXTEND-MIB::nsExtendCacheTime."System Fan 4" = INTEGER: 5
NET-SNMP-EXTEND-MIB::nsExtendExecType."test" = INTEGER: exec(1)
NET-SNMP-EXTEND-MIB::nsExtendExecType."System Fan 1" = INTEGER: exec(1)
NET-SNMP-EXTEND-MIB::nsExtendExecType."System Fan 3" = INTEGER: exec(1)
NET-SNMP-EXTEND-MIB::nsExtendExecType."System Fan 4" = INTEGER: exec(1)
NET-SNMP-EXTEND-MIB::nsExtendRunType."test" = INTEGER: run-on-read(1)
NET-SNMP-EXTEND-MIB::nsExtendRunType."System Fan 1" = INTEGER: run-on-read(1)
NET-SNMP-EXTEND-MIB::nsExtendRunType."System Fan 3" = INTEGER: run-on-read(1)
NET-SNMP-EXTEND-MIB::nsExtendRunType."System Fan 4" = INTEGER: run-on-read(1)
NET-SNMP-EXTEND-MIB::nsExtendStorage."test" = INTEGER: permanent(4)
NET-SNMP-EXTEND-MIB::nsExtendStorage."System Fan 1" = INTEGER: permanent(4)
NET-SNMP-EXTEND-MIB::nsExtendStorage."System Fan 3" = INTEGER: permanent(4)
NET-SNMP-EXTEND-MIB::nsExtendStorage."System Fan 4" = INTEGER: permanent(4)
NET-SNMP-EXTEND-MIB::nsExtendStatus."test" = INTEGER: active(1)
NET-SNMP-EXTEND-MIB::nsExtendStatus."System Fan 1" = INTEGER: active(1)
NET-SNMP-EXTEND-MIB::nsExtendStatus."System Fan 3" = INTEGER: active(1)
NET-SNMP-EXTEND-MIB::nsExtendStatus."System Fan 4" = INTEGER: active(1)
NET-SNMP-EXTEND-MIB::nsExtendOutput1Line."test" = STRING: hello
NET-SNMP-EXTEND-MIB::nsExtendOutput1Line."System Fan 1" = STRING: 1274.000
NET-SNMP-EXTEND-MIB::nsExtendOutput1Line."System Fan 3" = STRING: 1372.000
NET-SNMP-EXTEND-MIB::nsExtendOutput1Line."System Fan 4" = STRING: 1372.000
NET-SNMP-EXTEND-MIB::nsExtendOutputFull."test" = STRING: hello
NET-SNMP-EXTEND-MIB::nsExtendOutputFull."System Fan 1" = STRING: 1274.000
NET-SNMP-EXTEND-MIB::nsExtendOutputFull."System Fan 3" = STRING: 1372.000
NET-SNMP-EXTEND-MIB::nsExtendOutputFull."System Fan 4" = STRING: 1372.000
NET-SNMP-EXTEND-MIB::nsExtendOutNumLines."test" = INTEGER: 1
NET-SNMP-EXTEND-MIB::nsExtendOutNumLines."System Fan 1" = INTEGER: 1
NET-SNMP-EXTEND-MIB::nsExtendOutNumLines."System Fan 3" = INTEGER: 1
NET-SNMP-EXTEND-MIB::nsExtendOutNumLines."System Fan 4" = INTEGER: 1
NET-SNMP-EXTEND-MIB::nsExtendResult."test" = INTEGER: 0
NET-SNMP-EXTEND-MIB::nsExtendResult."System Fan 1" = INTEGER: 0
NET-SNMP-EXTEND-MIB::nsExtendResult."System Fan 3" = INTEGER: 0
NET-SNMP-EXTEND-MIB::nsExtendResult."System Fan 4" = INTEGER: 0
NET-SNMP-EXTEND-MIB::nsExtendOutLine."test".1 = STRING: hello
NET-SNMP-EXTEND-MIB::nsExtendOutLine."System Fan 1".1 = STRING: 1274.000
NET-SNMP-EXTEND-MIB::nsExtendOutLine."System Fan 3".1 = STRING: 1372.000
NET-SNMP-EXTEND-MIB::nsExtendOutLine."System Fan 4".1 = STRING: 1372.000

Get OID

Get you oid so you can search from another computer without MIB

snmptranslate -On 'NET-SNMP-EXTEND-MIB::nsExtendOutput1Line."System Fan 1"'

Nagios / OP5 Test

Setup a snmp test in Nagios/OP5 using check_snmp. because the output from the script is strings check_snmp has to do string conversion to integer, which it can if you configure it correctly. The example below does one check_snmp request it returns the value for all 3 fans we configured above. if any of the 3 Fans RPM value is outside of 1300-1500 rpm a warning is generated, if the rpm value is outside of 1000-2000 a critical is generated.

_USER1_/check_snmp -H "192.168.80.11" -o "".1.3.6.1.4.1.8072.1.3.2.3.1.1.12.83.121.115.116.101.109.32.70.97.110.32.51,.1.3.6.1.4.1.8072.1.3.2.3.1.1.12.83.121.115.116.101.109.32.70.97.110.32.49,.1.3.6.1.4.1.8072.1.3.2.3.1.1.12.83.121.115.116.101.109.32.70.97.110.32.52"" -P 3 -U "OP5View " -a sha -A "password " -x aes -X "encryptionkey " -L authPriv -w 1300:1500,1300:1500,1300:1500 -c 1000:2000,1000:2000,1000:2000 --label=FAN1,FAN3,FAN4
Result code: OK
SNMP OK - FAN1 1453 FAN3 1453 FAN4 1453 | FAN1=1453 FAN3=1453 FAN4=1453

How it looks in OP5

Enable IPMI from Linux

Install ipmitool
yum install ipmitool -y
Show Firmware
ipmitool mc info
Show Network
ipmitool lan print 1
Configure first network card (usally default) with static iP-number
ipmitool lan set 1 ipsrc static
ipmitool lan set 1 ipaddr 192.168.61.119
ipmitool lan set 1 netmask 255.255.255.0
ipmitool lan set 1 defgw ipaddr 192.168.61.1
Configure second network with DHCP
ipmitool lan set 2 ipsrc dhcp
Set SNMP community
ipmitool lan set 1 snmp dalesjo
List users
ipmitool user list 1
Change password on second user in list
ipmitool user set password 2
List Sensors
ipmitool sdr list
Restart IPMI
ipmitool mc reset
Sources

http://www.openfusion.net/linux/ipmi_on_centos

https://discuss.pivotal.io/hc/en-us/articles/206396927-How-to-work-on-IPMI-and-IPMITOOL

Backup of CIFS/Samba share

By mounting a smb share on your linux machine you can make an rsnapshot of the entire share without installing cwrsync. example below

  1. Download ds_smbmount
  2. Create a script that mounts all samba shares you want to snapshot.  the script should exit with an exitcode > 0 if mounting fails.  Script is later refered to as /root/smb/radion-mount
    #!/bin/bash
    
    /root/bin/ds_smbmount -d "/mnt/share/radion/k2/foreningar" -s "//192.168.60.202/föreningar" -u backup -p mypassword -U 0 -G 1003
    if [ $? -ne "0" ]; then
     echo "Failed to mount /mnt/share/radion/k2/foreningar"
     exit 1;
    fi;
  3. Create a script that umount all samba shares after your rsnapshot is finished. script is later refered to as /root/smb/radion-umount
    #!/bin/bash
    
    umount /mnt/share/radion/k2/foreningar
  4. Configure rsnapshot to run an script before and after snapshot.
    #################################################
    # rsnapshot.conf - rsnapshot configuration file #
    #################################################
    
    #######################
    # CONFIG FILE VERSION #
    #######################
    
    config_version 1.2
    
    ###########################
    # SNAPSHOT ROOT DIRECTORY #
    ###########################
    
    # All snapshots will be stored under this root directory.
    snapshot_root /zfs-pool-2/backup/radion
    no_create_root 0
    
    #################################
    # EXTERNAL PROGRAM DEPENDENCIES #
    #################################
    
    cmd_cp /bin/cp
    cmd_rm /bin/rm
    cmd_rsync /usr/bin/rsync
    cmd_ssh /usr/bin/ssh
    cmd_logger /usr/bin/logger
    
    cmd_preexec /root/smb/radion-mount
    cmd_postexec /root/smb/radion-umount
    
    #########################################
    # BACKUP INTERVALS #
    #########################################
    
    retain daily 30
    
    ############################################
    # GLOBAL OPTIONS #
    ############################################
    
    verbose 2
    loglevel 3
    logfile /var/log/rsnapshot-radion.log
    lockfile /var/run/rsnapshot-radion.pid
    
    # Bandwith limited to 30000KB/s =~ 240Mb/s
    # default --relative removed to keep simple folder structure.
    rsync_long_args --bwlimit=30000 --delete --numeric-ids --delete-excluded
    
    
    ###############################
    ### BACKUP POINTS / SCRIPTS ###
    ###############################
    
    backup /mnt/share/radion/k2/foreningar     k2/foreningar

 

CIFS/Smbclient speed

By default the linux smbclient mounts smb shares using smb protocol version 1.0 (now deprecated). resulting in slow transfer speed against Windows 10 shares. by adding vers=3.0 to you mount options you are using smb protocol version 3.0 with greater speed.

mount -t cifs //192.168.60.202/föreningar "/mnt/share/radion/k2/foreningar" --verbose -o username=backup,password=mypassword,vers=3.0

More info in this stackexchange thread about cifs randomly losing connection to windows share

FFmpeg Reference file

Records video from an ddshow device and records in to a file with current datestamp in filename. restart after 8 seconds on error or if user user presses Q.

echo off
:loop
SET tid=%TIME: =0%
FFmpeg\bin\ffmpeg -strict -2 -f dshow -video_size 1280x720 -framerate 25 -pixel_format yuv420p -i video="Osprey-700 HD Video Device 1":audio="SDI Input 1 (Osprey-700 HD 1)" -filter:v fps=25 -c:v libx264 -preset ultrafast -tune fastdecode -crf 16 -r 25 -g 25 -x264opts "keyint=50:min-keyint=50:no-scenecut" -c:a aac -ab 224000 -ar 48000 -ac 2 c:\referens\referens_720p_%date:~0,4%%date:~-5,2%%date:~-2,2%_%tid:~0,2%%tid:~3,2%%tid:~6,2%%tid:~-2,2%.mp4
timeout 8
goto loop

Set ip-number from commandline

Set Static IP-number

netsh interface ipv4 set address "Extern" static x.x.x.x y.y.y.y z.z.z.z
netsh interface ipv4 add dnsserver name="Extern" address=208.67.220.220 index=1
netsh interface ipv4 add dnsserver name="Extern" address=208.67.222.222 index=2

Extern is the name of your network device
x.x.x.x is your ip-number
y.y.y.y is your netmask.
z.z.z.z is your default gateway.

Enable DHCP

netsh interface ipv4 set address "Extern" source=dhcp
netsh interface ipv4 set dnsserver "Extern" source=dhcp