Category: Network

Test Unix socket

Post author By samuel
Post date June 17, 2020

This is a small test/benchmark to se if the use of a unix domain socket in Nginx would affect performance in any significant way for an api

The api is hosted by .NET Core and containerized. Test is done by using apache benchmark. doing 100 concurrent request 50 times. The request that is done has a payload of 1,7 KB.

ab -n 5000 -c 100 "http://localhost/v3/api/"

Result

This is the average request per second after 10 tests against each configuration.

	Requests per second
HTTP	100
HTTP (Keepalive enabled)	118
Unix Socket	98
Unix Socket (Keepalive enabled)	120

Keepalive

Nginx option keepalive has an impact on both http and unix sockets. It looks to be far more important than the actual connection type.

upstream backend {
        keepalive 100;
        server 10.265.44.3:8080;
}

Tags .NET Core, Nginx

Linux Network

Start NRPE after openvpn tunnel is connected

Post author By samuel
Post date August 7, 2018

NRPE will not start if its server address is a openvpn ip and the tunnel is not yet established when NRPE tries to start. to solve this. create a new systemd file

systemctl -all | grep ovpn
cp /usr/lib/systemd/system/nrpe.service /etc/systemd/system/nrpe.service

Add your tun device to Requires and After. Note you need the systemD name of your tune device. in this case. OpenVPN was configured to use tun ovpn-gwSamuel check systemctl for its correct name. The result should look something like this. Notice the esacped dash sign in the name.

[Unit]
Description=Nagios Remote Program Executor
Documentation=http://www.nagios.org/documentation
Conflicts=nrpe.socket
Requires=network.target sys-devices-virtual-net-ovpn\x2dgwSamuel.device
After=network-online.target sys-devices-virtual-net-ovpn\x2dgwSamuel.device

[Install]
WantedBy=multi-user.target

[Service]
Type=forking
User=nrpe
Group=nrpe
EnvironmentFile=/etc/sysconfig/nrpe
ExecStart=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -d $NRPE_SSL_OPT

Tags CentOS 7, icinga, nrpe

Linux Network

Extending snmpd with sensor data from ipmitool

Post author By samuel
Post date March 15, 2018

Create script

Create a script that returns the data you want to monitor through snmp. the output should be cleaned from whitespaces.

My examples monitor sensor data from ipmi by creating cron script that outputs ipmitool sensor data to a temp file every two minutes. And then parsing it with snmp_ipmi from github. Cron script below.

# .--------------------------- minute (0 - 59)
# | .------------- hour (0 - 23)
# | | .---------- day of month (1 - 31)
# | | | .------- month (1 - 12) OR jan,feb,mar,apr ...
# | | | | .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat
# | | | | |
# * * * * * user-name command to be executed
 */2 * * * * root ipmitool sensor list > /tmp/ipmi_sensor_list

Add script to snmpd

Edit /etc/snmp/snmpd.conf and add your scripts

rouser OP5View

extend "System Fan 1" /usr/lib64/nagios/plugins/custom/ipmi/snmp_ipmi -f /tmp/ipmi_sensor_list -u "System Fan 1"
extend "System Fan 3" /usr/lib64/nagios/plugins/custom/ipmi/snmp_ipmi -f /tmp/ipmi_sensor_list -u "System Fan 3"
extend "System Fan 4" /usr/lib64/nagios/plugins/custom/ipmi/snmp_ipmi -f /tmp/ipmi_sensor_list -u "System Fan 4"

Test

restart snmpd and run the following command to see your snmp data. example output below.

# snmpwalk -u OP5View -A password -a SHA -X encryptionkey -x AES -l authPriv -v3 localhost .1.3.6.1.4.1.8072.1.3
NET-SNMP-EXTEND-MIB::nsExtendNumEntries.0 = INTEGER: 4
NET-SNMP-EXTEND-MIB::nsExtendCommand."test" = STRING: /bin/echo
NET-SNMP-EXTEND-MIB::nsExtendCommand."System Fan 1" = STRING: /usr/lib64/nagios/plugins/custom/ipmi/snmp_ipmi
NET-SNMP-EXTEND-MIB::nsExtendCommand."System Fan 3" = STRING: /usr/lib64/nagios/plugins/custom/ipmi/snmp_ipmi
NET-SNMP-EXTEND-MIB::nsExtendCommand."System Fan 4" = STRING: /usr/lib64/nagios/plugins/custom/ipmi/snmp_ipmi
NET-SNMP-EXTEND-MIB::nsExtendArgs."test" = STRING: hello
NET-SNMP-EXTEND-MIB::nsExtendArgs."System Fan 1" = STRING: -f /tmp/ipmi_sensor_list -u \"System Fan 1\"
NET-SNMP-EXTEND-MIB::nsExtendArgs."System Fan 3" = STRING: -f /tmp/ipmi_sensor_list -u \"System Fan 3\"
NET-SNMP-EXTEND-MIB::nsExtendArgs."System Fan 4" = STRING: -f /tmp/ipmi_sensor_list -u \"System Fan 4\"
NET-SNMP-EXTEND-MIB::nsExtendInput."test" = STRING:
NET-SNMP-EXTEND-MIB::nsExtendInput."System Fan 1" = STRING:
NET-SNMP-EXTEND-MIB::nsExtendInput."System Fan 3" = STRING:
NET-SNMP-EXTEND-MIB::nsExtendInput."System Fan 4" = STRING:
NET-SNMP-EXTEND-MIB::nsExtendCacheTime."test" = INTEGER: 5
NET-SNMP-EXTEND-MIB::nsExtendCacheTime."System Fan 1" = INTEGER: 5
NET-SNMP-EXTEND-MIB::nsExtendCacheTime."System Fan 3" = INTEGER: 5
NET-SNMP-EXTEND-MIB::nsExtendCacheTime."System Fan 4" = INTEGER: 5
NET-SNMP-EXTEND-MIB::nsExtendExecType."test" = INTEGER: exec(1)
NET-SNMP-EXTEND-MIB::nsExtendExecType."System Fan 1" = INTEGER: exec(1)
NET-SNMP-EXTEND-MIB::nsExtendExecType."System Fan 3" = INTEGER: exec(1)
NET-SNMP-EXTEND-MIB::nsExtendExecType."System Fan 4" = INTEGER: exec(1)
NET-SNMP-EXTEND-MIB::nsExtendRunType."test" = INTEGER: run-on-read(1)
NET-SNMP-EXTEND-MIB::nsExtendRunType."System Fan 1" = INTEGER: run-on-read(1)
NET-SNMP-EXTEND-MIB::nsExtendRunType."System Fan 3" = INTEGER: run-on-read(1)
NET-SNMP-EXTEND-MIB::nsExtendRunType."System Fan 4" = INTEGER: run-on-read(1)
NET-SNMP-EXTEND-MIB::nsExtendStorage."test" = INTEGER: permanent(4)
NET-SNMP-EXTEND-MIB::nsExtendStorage."System Fan 1" = INTEGER: permanent(4)
NET-SNMP-EXTEND-MIB::nsExtendStorage."System Fan 3" = INTEGER: permanent(4)
NET-SNMP-EXTEND-MIB::nsExtendStorage."System Fan 4" = INTEGER: permanent(4)
NET-SNMP-EXTEND-MIB::nsExtendStatus."test" = INTEGER: active(1)
NET-SNMP-EXTEND-MIB::nsExtendStatus."System Fan 1" = INTEGER: active(1)
NET-SNMP-EXTEND-MIB::nsExtendStatus."System Fan 3" = INTEGER: active(1)
NET-SNMP-EXTEND-MIB::nsExtendStatus."System Fan 4" = INTEGER: active(1)
NET-SNMP-EXTEND-MIB::nsExtendOutput1Line."test" = STRING: hello
NET-SNMP-EXTEND-MIB::nsExtendOutput1Line."System Fan 1" = STRING: 1274.000
NET-SNMP-EXTEND-MIB::nsExtendOutput1Line."System Fan 3" = STRING: 1372.000
NET-SNMP-EXTEND-MIB::nsExtendOutput1Line."System Fan 4" = STRING: 1372.000
NET-SNMP-EXTEND-MIB::nsExtendOutputFull."test" = STRING: hello
NET-SNMP-EXTEND-MIB::nsExtendOutputFull."System Fan 1" = STRING: 1274.000
NET-SNMP-EXTEND-MIB::nsExtendOutputFull."System Fan 3" = STRING: 1372.000
NET-SNMP-EXTEND-MIB::nsExtendOutputFull."System Fan 4" = STRING: 1372.000
NET-SNMP-EXTEND-MIB::nsExtendOutNumLines."test" = INTEGER: 1
NET-SNMP-EXTEND-MIB::nsExtendOutNumLines."System Fan 1" = INTEGER: 1
NET-SNMP-EXTEND-MIB::nsExtendOutNumLines."System Fan 3" = INTEGER: 1
NET-SNMP-EXTEND-MIB::nsExtendOutNumLines."System Fan 4" = INTEGER: 1
NET-SNMP-EXTEND-MIB::nsExtendResult."test" = INTEGER: 0
NET-SNMP-EXTEND-MIB::nsExtendResult."System Fan 1" = INTEGER: 0
NET-SNMP-EXTEND-MIB::nsExtendResult."System Fan 3" = INTEGER: 0
NET-SNMP-EXTEND-MIB::nsExtendResult."System Fan 4" = INTEGER: 0
NET-SNMP-EXTEND-MIB::nsExtendOutLine."test".1 = STRING: hello
NET-SNMP-EXTEND-MIB::nsExtendOutLine."System Fan 1".1 = STRING: 1274.000
NET-SNMP-EXTEND-MIB::nsExtendOutLine."System Fan 3".1 = STRING: 1372.000
NET-SNMP-EXTEND-MIB::nsExtendOutLine."System Fan 4".1 = STRING: 1372.000

Get OID

Get you oid so you can search from another computer without MIB

snmptranslate -On 'NET-SNMP-EXTEND-MIB::nsExtendOutput1Line."System Fan 1"'

Nagios / OP5 Test

Setup a snmp test in Nagios/OP5 using check_snmp. because the output from the script is strings check_snmp has to do string conversion to integer, which it can if you configure it correctly. The example below does one check_snmp request it returns the value for all 3 fans we configured above. if any of the 3 Fans RPM value is outside of 1300-1500 rpm a warning is generated, if the rpm value is outside of 1000-2000 a critical is generated.

_USER1_/check_snmp -H "192.168.80.11" -o "".1.3.6.1.4.1.8072.1.3.2.3.1.1.12.83.121.115.116.101.109.32.70.97.110.32.51,.1.3.6.1.4.1.8072.1.3.2.3.1.1.12.83.121.115.116.101.109.32.70.97.110.32.49,.1.3.6.1.4.1.8072.1.3.2.3.1.1.12.83.121.115.116.101.109.32.70.97.110.32.52"" -P 3 -U "OP5View " -a sha -A "password " -x aes -X "encryptionkey " -L authPriv -w 1300:1500,1300:1500,1300:1500 -c 1000:2000,1000:2000,1000:2000 --label=FAN1,FAN3,FAN4
Result code: OK
SNMP OK - FAN1 1453 FAN3 1453 FAN4 1453 | FAN1=1453 FAN3=1453 FAN4=1453

How it looks in OP5

Tags CentOS 7, IPMI, Linux, OP5, SNMP

Network

Enable IPMI from Linux

Post author By samuel
Post date March 11, 2018

Install ipmitool

yum install ipmitool -y

Show Firmware

ipmitool mc info

Show Network

ipmitool lan print 1

Configure first network card (usally default) with static iP-number

ipmitool lan set 1 ipsrc static
ipmitool lan set 1 ipaddr 192.168.61.119
ipmitool lan set 1 netmask 255.255.255.0
ipmitool lan set 1 defgw ipaddr 192.168.61.1

Configure second network with DHCP

ipmitool lan set 2 ipsrc dhcp

Set SNMP community

ipmitool lan set 1 snmp dalesjo

List users

ipmitool user list 1

Change password on second user in list

ipmitool user set password 2

List Sensors

ipmitool sdr list

Restart IPMI

ipmitool mc reset

Sources

http://www.openfusion.net/linux/ipmi_on_centos

https://discuss.pivotal.io/hc/en-us/articles/206396927-How-to-work-on-IPMI-and-IPMITOOL

Tags CentOS 7, IPMI, Linux, SNMP, Supermicro

Network

Run letsencrypt as a normal user

Post author By samuel
Post date February 8, 2018

Install and change permission to work with your normal user account.

yum install yum-utils -y
yum-config-manager --enable rhui-REGION-rhel-server-extras rhui-REGION-rhel-server-optional
yum install certbot -y

useradd certmaster
chown certmaster: /etc/letsencrypt/ -R
chown certmaster: /var/log/letsencrypt/ -R
chown certmaster: /var/lib/letsencrypt/ -R

You might want to allow your certmaster account to reload nginx or another service then a new certificate exists. To allow certmaster to reload nginx configuration, create sudoers file /etc/sudoers.d/certmaster with this content.

certmaster ALL=NOPASSWD: /usr/bin/systemctl reload nginx.service
certmaster ALL=NOPASSWD: /usr/sbin/nginx -t -c /etc/nginx/nginx.conf

Example of cron script to run.

# .---------------- minute (0 - 59)
# | .------------- hour (0 - 23)
# | | .---------- day of month (1 - 31)
# | | | .------- month (1 - 12) OR jan,feb,mar,apr ...
# | | | | .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat
# | | | | |
# * * * * * user-name command to be executed
# Updates all certificates on saturday morning, checks nginx config and reloads
15 2 * * * certmaster chronic /usr/bin/certbot --no-self-upgrade renew && sudo /usr/sbin/nginx -t -c /etc/nginx/nginx.conf && sudo /usr/bin/systemctl reload nginx.service

Tags CentOS 7, letsencrypt

Network

Encrypt private key OpenVPN

Post author By samuel
Post date February 6, 2018

You can password protect your key file so that ed an extra password is needed to connect with openvpn. It will also work with embedded certificates in .ovpn

Tags OpenVPN, pfSense, Windows 10

Network

Round-Robin webservers with letsencrypt

Post author By samuel
Post date February 5, 2018

Nginx on Edge servers

Configure your edgeservers to redirect all http challanges to your main webserver.

Tags letsencrypt, Nginx

Network

Embedding certificates in .ovpn

Post author By samuel
Post date February 5, 2018

You can simplify OpenVPN distribution by only use one file for both config and certificates. A normal .ovpn file with separate .key and .crt files looks like this.

client dev tun 
dev-node Dalesjo VPN 
proto tcp 
remote my-server 443 
resolv-retry infinite 
nobind 
persist-key 
persist-tun 
remote-cert-tls server 
verb 3 
comp-lzo yes

ca DALESJO-OpenVPN.crt
cert DALESJO-SADAL.crt
key DALESJO-SADAL.key
tls-auth DALESJO-Server.tls 1

Tags OpenVPN

Network

Varnish reload

Post author By samuel
Post date February 5, 2018

A reload of the varnish server does not destroy your cache but reloads your vcl. Check your vcl file for errors before trying a reload.

varnishd -C -f /etc/varnish/default.vcl && systemctl reload varnish.service && systemctl status varnish.service varnishncsa.service

Tags Varnish

Linux Network

Add a second network card

Post author By samuel
Post date February 1, 2018

To add a second ip-number to an network interface create a new ifcfg file ex /etc/sysconfig/network-scripts/ifcfg-enp1s0f0:0

TYPE="Ethernet"
BOOTPROTO="static"
DEVICE="enp1s0f0:1"
ONBOOT="no"
ZONE=public
IPADDR=80.67.x.x
PREFIX=31
GATEWAY=80.67.x.x

The default configuration for that interface might have parameters such as UUID, NAME those are not needed. change ONBOOT to yes after testing the configuration. To test run.

systemctl restart network.service

Tags CentOS 7