Category: Linux

Start NRPE after openvpn tunnel is connected

NRPE will not start if its server address is a openvpn ip and the tunnel is not yet established when NRPE tries to start. to solve this. create a new systemd file

systemctl -all | grep ovpn
cp /usr/lib/systemd/system/nrpe.service /etc/systemd/system/nrpe.service

Add your tun device to Requires and After. Note you need the systemD name of your tune device. in this case. OpenVPN was configured to use tun ovpn-gwSamuel check systemctl for its correct name. The result should look something like this. Notice the esacped dash sign in the name.

[Unit]
Description=Nagios Remote Program Executor
Documentation=http://www.nagios.org/documentation
Conflicts=nrpe.socket
Requires=network.target sys-devices-virtual-net-ovpn\x2dgwSamuel.device
After=network-online.target sys-devices-virtual-net-ovpn\x2dgwSamuel.device

[Install]
WantedBy=multi-user.target

[Service]
Type=forking
User=nrpe
Group=nrpe
EnvironmentFile=/etc/sysconfig/nrpe
ExecStart=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -d $NRPE_SSL_OPT

Error unpacking rpm package chkconfig

Trying to update some CentOS servers to in this case CentOS 7.4 i stumbled onto a problem there it was not possible to perform a update on chkconfig.

[root@moln.dalesjo.net ~]# yum update -y
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirror.zetup.net
 * epel: mirror.zetup.net
 * extras: mirror.zetup.net
 * updates: mirror.zetup.net
Resolving Dependencies
--> Running transaction check
---> Package chkconfig.x86_64 0:1.7.2-1.el7 will be updated
---> Package chkconfig.x86_64 0:1.7.4-1.el7 will be an update
--> Finished Dependency Resolution

Dependencies Resolved

====================================================================================================================================================================================================================================================
 Package Arch Version Repository Size
====================================================================================================================================================================================================================================================
Updating:
 chkconfig x86_64 1.7.4-1.el7 base 181 k

Transaction Summary
====================================================================================================================================================================================================================================================
Upgrade 1 Package

Total download size: 181 k
Downloading packages:
Delta RPMs disabled because /usr/bin/applydeltarpm not installed.
chkconfig-1.7.4-1.el7.x86_64.rpm | 181 kB 00:00:00
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
 Updating : chkconfig-1.7.4-1.el7.x86_64 1/2
Error unpacking rpm package chkconfig-1.7.4-1.el7.x86_64
error: unpacking of archive failed on file /etc/init.d: cpio: rename
chkconfig-1.7.2-1.el7.x86_64 was supposed to be removed but is not!
 Verifying : chkconfig-1.7.2-1.el7.x86_64 1/2
 Verifying : chkconfig-1.7.4-1.el7.x86_64 2/2

Failed:
 chkconfig.x86_64 0:1.7.2-1.el7 chkconfig.x86_64 0:1.7.4-1.el7

Complete!

The problem occured because /etc/init.d was a directory instead of a softlink to /etc/rc.d/init.d/. To fix it, simply do

mv /etc/init.d/* /etc/rc.d/init.d/
rm /etc/init.d -r
ln -s /etc/rc.d/init.d/ /etc/init.d

Extended permissions on backup directory

Using rsnapshot to do backups I want to enable some users the availability to easily restore a single file from the snapshot. Problem is that rsnapshot also is keeping the original permissions for each file. To solve this i will create a usergroup called dalesjo-backup and give this group read access to all files in the backup using an Access Control list.

Enable ACL

First you need to enable ACL on your zfs pool in this case zfs-pool-2

zfs set acltype=posixacl zfs-pool-2

If you dont do this setfacl will return the error below.

setfacl: .: Operation not supported

Set Filepermissions

Below I’m setting a default acl giving dalesjo-backup read/execute permissions on all new files. And after that changing all currently existing files to give read/execute access to the same group.

cd /zfs-pool-2/backup
setfacl -Rdm "g:dalesjo-backup:rx" .
setfacl -Rm "g:dalesjo-backup:rx" .
getfacl .

Source: Serverfault

Extending snmpd with sensor data from ipmitool

Create script

Create a script that returns the data you want to monitor through snmp. the output should be cleaned from whitespaces.

My examples monitor sensor data from ipmi by creating cron script that outputs ipmitool sensor data to  a temp file every two minutes. And then parsing it with snmp_ipmi from github. Cron script below.

# .--------------------------- minute (0 - 59)
# | .------------- hour (0 - 23)
# | | .---------- day of month (1 - 31)
# | | | .------- month (1 - 12) OR jan,feb,mar,apr ...
# | | | | .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat
# | | | | |
# * * * * * user-name command to be executed
 */2 * * * * root ipmitool sensor list > /tmp/ipmi_sensor_list

Add script to snmpd

Edit /etc/snmp/snmpd.conf and add your scripts

rouser OP5View

extend "System Fan 1" /usr/lib64/nagios/plugins/custom/ipmi/snmp_ipmi -f /tmp/ipmi_sensor_list -u "System Fan 1"
extend "System Fan 3" /usr/lib64/nagios/plugins/custom/ipmi/snmp_ipmi -f /tmp/ipmi_sensor_list -u "System Fan 3"
extend "System Fan 4" /usr/lib64/nagios/plugins/custom/ipmi/snmp_ipmi -f /tmp/ipmi_sensor_list -u "System Fan 4"

Test

restart snmpd and run the following command to see your snmp data. example output below.

# snmpwalk -u OP5View -A password -a SHA -X encryptionkey -x AES -l authPriv -v3 localhost .1.3.6.1.4.1.8072.1.3
NET-SNMP-EXTEND-MIB::nsExtendNumEntries.0 = INTEGER: 4
NET-SNMP-EXTEND-MIB::nsExtendCommand."test" = STRING: /bin/echo
NET-SNMP-EXTEND-MIB::nsExtendCommand."System Fan 1" = STRING: /usr/lib64/nagios/plugins/custom/ipmi/snmp_ipmi
NET-SNMP-EXTEND-MIB::nsExtendCommand."System Fan 3" = STRING: /usr/lib64/nagios/plugins/custom/ipmi/snmp_ipmi
NET-SNMP-EXTEND-MIB::nsExtendCommand."System Fan 4" = STRING: /usr/lib64/nagios/plugins/custom/ipmi/snmp_ipmi
NET-SNMP-EXTEND-MIB::nsExtendArgs."test" = STRING: hello
NET-SNMP-EXTEND-MIB::nsExtendArgs."System Fan 1" = STRING: -f /tmp/ipmi_sensor_list -u \"System Fan 1\"
NET-SNMP-EXTEND-MIB::nsExtendArgs."System Fan 3" = STRING: -f /tmp/ipmi_sensor_list -u \"System Fan 3\"
NET-SNMP-EXTEND-MIB::nsExtendArgs."System Fan 4" = STRING: -f /tmp/ipmi_sensor_list -u \"System Fan 4\"
NET-SNMP-EXTEND-MIB::nsExtendInput."test" = STRING:
NET-SNMP-EXTEND-MIB::nsExtendInput."System Fan 1" = STRING:
NET-SNMP-EXTEND-MIB::nsExtendInput."System Fan 3" = STRING:
NET-SNMP-EXTEND-MIB::nsExtendInput."System Fan 4" = STRING:
NET-SNMP-EXTEND-MIB::nsExtendCacheTime."test" = INTEGER: 5
NET-SNMP-EXTEND-MIB::nsExtendCacheTime."System Fan 1" = INTEGER: 5
NET-SNMP-EXTEND-MIB::nsExtendCacheTime."System Fan 3" = INTEGER: 5
NET-SNMP-EXTEND-MIB::nsExtendCacheTime."System Fan 4" = INTEGER: 5
NET-SNMP-EXTEND-MIB::nsExtendExecType."test" = INTEGER: exec(1)
NET-SNMP-EXTEND-MIB::nsExtendExecType."System Fan 1" = INTEGER: exec(1)
NET-SNMP-EXTEND-MIB::nsExtendExecType."System Fan 3" = INTEGER: exec(1)
NET-SNMP-EXTEND-MIB::nsExtendExecType."System Fan 4" = INTEGER: exec(1)
NET-SNMP-EXTEND-MIB::nsExtendRunType."test" = INTEGER: run-on-read(1)
NET-SNMP-EXTEND-MIB::nsExtendRunType."System Fan 1" = INTEGER: run-on-read(1)
NET-SNMP-EXTEND-MIB::nsExtendRunType."System Fan 3" = INTEGER: run-on-read(1)
NET-SNMP-EXTEND-MIB::nsExtendRunType."System Fan 4" = INTEGER: run-on-read(1)
NET-SNMP-EXTEND-MIB::nsExtendStorage."test" = INTEGER: permanent(4)
NET-SNMP-EXTEND-MIB::nsExtendStorage."System Fan 1" = INTEGER: permanent(4)
NET-SNMP-EXTEND-MIB::nsExtendStorage."System Fan 3" = INTEGER: permanent(4)
NET-SNMP-EXTEND-MIB::nsExtendStorage."System Fan 4" = INTEGER: permanent(4)
NET-SNMP-EXTEND-MIB::nsExtendStatus."test" = INTEGER: active(1)
NET-SNMP-EXTEND-MIB::nsExtendStatus."System Fan 1" = INTEGER: active(1)
NET-SNMP-EXTEND-MIB::nsExtendStatus."System Fan 3" = INTEGER: active(1)
NET-SNMP-EXTEND-MIB::nsExtendStatus."System Fan 4" = INTEGER: active(1)
NET-SNMP-EXTEND-MIB::nsExtendOutput1Line."test" = STRING: hello
NET-SNMP-EXTEND-MIB::nsExtendOutput1Line."System Fan 1" = STRING: 1274.000
NET-SNMP-EXTEND-MIB::nsExtendOutput1Line."System Fan 3" = STRING: 1372.000
NET-SNMP-EXTEND-MIB::nsExtendOutput1Line."System Fan 4" = STRING: 1372.000
NET-SNMP-EXTEND-MIB::nsExtendOutputFull."test" = STRING: hello
NET-SNMP-EXTEND-MIB::nsExtendOutputFull."System Fan 1" = STRING: 1274.000
NET-SNMP-EXTEND-MIB::nsExtendOutputFull."System Fan 3" = STRING: 1372.000
NET-SNMP-EXTEND-MIB::nsExtendOutputFull."System Fan 4" = STRING: 1372.000
NET-SNMP-EXTEND-MIB::nsExtendOutNumLines."test" = INTEGER: 1
NET-SNMP-EXTEND-MIB::nsExtendOutNumLines."System Fan 1" = INTEGER: 1
NET-SNMP-EXTEND-MIB::nsExtendOutNumLines."System Fan 3" = INTEGER: 1
NET-SNMP-EXTEND-MIB::nsExtendOutNumLines."System Fan 4" = INTEGER: 1
NET-SNMP-EXTEND-MIB::nsExtendResult."test" = INTEGER: 0
NET-SNMP-EXTEND-MIB::nsExtendResult."System Fan 1" = INTEGER: 0
NET-SNMP-EXTEND-MIB::nsExtendResult."System Fan 3" = INTEGER: 0
NET-SNMP-EXTEND-MIB::nsExtendResult."System Fan 4" = INTEGER: 0
NET-SNMP-EXTEND-MIB::nsExtendOutLine."test".1 = STRING: hello
NET-SNMP-EXTEND-MIB::nsExtendOutLine."System Fan 1".1 = STRING: 1274.000
NET-SNMP-EXTEND-MIB::nsExtendOutLine."System Fan 3".1 = STRING: 1372.000
NET-SNMP-EXTEND-MIB::nsExtendOutLine."System Fan 4".1 = STRING: 1372.000

Get OID

Get you oid so you can search from another computer without MIB

snmptranslate -On 'NET-SNMP-EXTEND-MIB::nsExtendOutput1Line."System Fan 1"'

Nagios / OP5 Test

Setup a snmp test in Nagios/OP5 using check_snmp. because the output from the script is strings check_snmp has to do string conversion to integer, which it can if you configure it correctly. The example below does one check_snmp request it returns the value for all 3 fans we configured above. if any of the 3 Fans RPM value is outside of 1300-1500 rpm a warning is generated, if the rpm value is outside of 1000-2000 a critical is generated.

_USER1_/check_snmp -H "192.168.80.11" -o "".1.3.6.1.4.1.8072.1.3.2.3.1.1.12.83.121.115.116.101.109.32.70.97.110.32.51,.1.3.6.1.4.1.8072.1.3.2.3.1.1.12.83.121.115.116.101.109.32.70.97.110.32.49,.1.3.6.1.4.1.8072.1.3.2.3.1.1.12.83.121.115.116.101.109.32.70.97.110.32.52"" -P 3 -U "OP5View " -a sha -A "password " -x aes -X "encryptionkey " -L authPriv -w 1300:1500,1300:1500,1300:1500 -c 1000:2000,1000:2000,1000:2000 --label=FAN1,FAN3,FAN4
Result code: OK
SNMP OK - FAN1 1453 FAN3 1453 FAN4 1453 | FAN1=1453 FAN3=1453 FAN4=1453

How it looks in OP5

Run commands over SSH

Run one or multiple commands on a remote server

ssh root@host <<'ENDSSH'
#commands to run on remote host
ENDSSH

Run a local script on a remote server. (none interactive, cant open nano). the script does not need to start with #/!bin/bash

ssh root@host < dalesjo-install-postfix.sh

Run a command interactive, ex start nano on remote server.

ssh -t root@cdn2.dalesjo.com 'nano /etc/postfix/main.cf'

Uninstall compiled software

When uninstalling software you compiled yourself you can run make -n install to see what the make install command did. For Nginx you only had to remove /sbin/nginx before installning NginX from YUM.

# make -n install
make -f objs/Makefile install
make[1]: Entering directory `/root/install/cache/nginx-1.9.9'
test -d '/' || mkdir -p '/'
test -d '/' || mkdir -p '/'
test ! -f '/sbin' || mv '/sbin' '/sbin.old'
cp objs/nginx '/sbin'
test -d '/etc/nginx' || mkdir -p '/etc/nginx'
cp conf/koi-win '/etc/nginx'
cp conf/koi-utf '/etc/nginx'
cp conf/win-utf '/etc/nginx'
test -f '/etc/nginx/mime.types' || cp conf/mime.types '/etc/nginx'
cp conf/mime.types '/etc/nginx/mime.types.default'
test -f '/etc/nginx/fastcgi_params' || cp conf/fastcgi_params '/etc/nginx'
cp conf/fastcgi_params '/etc/nginx/fastcgi_params.default'
test -f '/etc/nginx/fastcgi.conf' || cp conf/fastcgi.conf '/etc/nginx'
cp conf/fastcgi.conf '/etc/nginx/fastcgi.conf.default'
test -f '/etc/nginx/uwsgi_params' || cp conf/uwsgi_params '/etc/nginx'
cp conf/uwsgi_params '/etc/nginx/uwsgi_params.default'
test -f '/etc/nginx/scgi_params' || cp conf/scgi_params '/etc/nginx'
cp conf/scgi_params '/etc/nginx/scgi_params.default'
test -f '/etc/nginx/nginx.conf' || cp conf/nginx.conf '/etc/nginx/nginx.conf'
cp conf/nginx.conf '/etc/nginx/nginx.conf.default'
test -d '/var/run/nginx' || mkdir -p '/var/run/nginx'
test -d '/var/log/nginx/access' || mkdir -p '/var/log/nginx/access'
test -d '//html' || cp -R html '/'
test -d '/var/log/nginx/errors' || mkdir -p '/var/log/nginx/errors'
make[1]: Leaving directory `/root/install/cache/nginx-1.9.9'

Source

Javier Rivera on askubuntu

Add a second network card

To add a second ip-number to an network interface create a new ifcfg file ex /etc/sysconfig/network-scripts/ifcfg-enp1s0f0:0

TYPE="Ethernet"
BOOTPROTO="static"
DEVICE="enp1s0f0:1"
ONBOOT="no"
ZONE=public
IPADDR=80.67.x.x
PREFIX=31
GATEWAY=80.67.x.x

The default configuration for that interface might have parameters such as UUID, NAME those are not needed. change ONBOOT to yes after testing the configuration. To test run.

systemctl restart network.service

SSH Filtering

The goal is to allow root to login with username/password from local IP-addresses. But only allow root to login with public key authentication from internet.

First change the following settings in /etc/ssh/sshd_config. this will allow normal users to login with username/password but root users must use public key authentication.

PasswordAuthentication yes
PermitRootLogin without-password

Second, add the following to the end of /etc/ssh/sshd_config. Its important that you are in the end of the file because how sshd_config is read by the daemon.

Match address 192.168.0.0/16
 PermitRootLogin yes