Categories
Linux

SPF and DKIM with Postfix

SPF (Sender Policy Framework) record specifies which hosts or IP addresses are allowed to send emails on behalf of a domain. You should allow only your own email server or your ISP’s server to send emails for your domain.

DKIM (DomainKeys Identified Mail) uses a private key to add a signature to emails sent from your domain. Receiving SMTP servers verify the signature by using the corresponding public key, which is published in your DNS manager.

Create SPF record in DNS zone

n your DNS management interface, create a new TXT record like below.

TXT  @   v=spf1 mx ~all

Some DNS managers require you to wrap the SPF record with quotes like below.

TXT  @   "v=spf1 mx ~all"

Keep in mind that it can take up to an hour for the new record to be available.

Configure Postfix for SPF

First, install required packages:

sudo apt install postfix-policyd-spf-python

Edit the Postfix master process configuration file located at /etc/postfix/master.cf. Add these lines to the end:

policyd-spf  unix  -       n       n       -       0       spawn
    user=policyd-spf argv=/usr/bin/policyd-spf

Now open up the configuration file at /etc/postfix/main.cf. Add these lines to the end of the file:

policyd-spf_time_limit = 3600
smtpd_recipient_restrictions =
   permit_mynetworks,
   permit_sasl_authenticated,
   reject_unauth_destination,
   check_policy_service unix:private/policyd-spf

Now restart postfix

sudo systemctl restart postfix

Configure DKIM

sudo apt install opendkim opendkim-tools

Add the Postfix user to the OpenDKIM group

sudo gpasswd -a postfix opendkim

Now open the configuration of OpenDKIM and enable or add these lines:

Canonicalization   simple
Mode               sv
SubDomains         no
AutoRestart         yes
AutoRestartRate     10/1M
Background          yes
DNSTimeout          5
SignatureAlgorithm  rsa-sha256

Go to the end of the file and add these lines:

#OpenDKIM user
# Remember to add user postfix to group opendkim
UserID             opendkim

# Map domains in From addresses to keys used to sign messages
KeyTable           refile:/etc/opendkim/key.table
SigningTable       refile:/etc/opendkim/signing.table

# Hosts to ignore when verifying signatures
ExternalIgnoreList  /etc/opendkim/trusted.hosts

# A set of internal hosts whose mail should be signed
InternalHosts       /etc/opendkim/trusted.hosts

We will need to create the signing table, key table and the trusted hosts file.

sudo mkdir /etc/opendkim
sudo mkdir /etc/opendkim/keys
sudo chown -R opendkim:opendkim /etc/opendkim
sudo chmod go-rw /etc/opendkim/keys

Now create the signing table, using your domain. Open the file and add the second line in it:

sudo nano /etc/opendkim/signing.table
*@bontekoe.technology    default._domainkey.bontekoe.technology

Now create the key table

sudo nano /etc/opendkim/key.table
default._domainkey.bontekoe.technology     bontekoe.technology:default:/etc/opendkim/keys/bontekoe.technology/default.private

Now create the trusted hosts file:

sudo nano /etc/opendkim/trusted.hosts
127.0.0.1
localhost

*.bontekoe.technology

Generating DKIM Keypair

Create a separate folder for the domain.

sudo mkdir /etc/opendkim/keys/bontekoe.technology

Generate keys using opendkim-genkey tool.

sudo opendkim-genkey -b 2048 -d bontekoe.technology -D /etc/opendkim/keys/bontekoe.technology -s default -v
sudo chown opendkim:opendkim /etc/opendkim/keys/bontekoe.technology/default.private

Display the public key that was generated:

sudo cat /etc/opendkim/keys/bontekoe.technology/default.txt

This file contains the entire DNS record that should be published. Copy everything, startking with the v=DKIM1 and in your DNS record. After 15 minutes, test is the record has been successfully published:

sudo opendkim-testkey -d bontekoe.technology -s default -vvv

Result:

opendkim-testkey: using default configfile /etc/opendkim.conf
opendkim-testkey: checking key 'default._domainkey.bontekoe.technology'
opendkim-testkey: key secure
opendkim-testkey: key OK

Connecting Postfix to OpenDKIM

sudo mkdir /var/spool/postfix/opendkim
sudo chown opendkim:postfix /var/spool/postfix/opendkim

Open the configuration file at /etc/opendkim.conf, replace the socket (if defined, or add it):

Socket    local:/var/spool/postfix/opendkim/opendkim.sock

Open /etc/postfix/main.cf and add the following to the end:

# Milter configuration
milter_default_action = accept
milter_protocol = 6
smtpd_milters = local:opendkim/opendkim.sock
non_smtpd_milters = $smtpd_milters

Now restart Postfix and OpenDKIM:

sudo systemctl restart opendkim postfix

Categories
Linux

Apply Database Partitions to a live Zabbix database – without downtime

Due to the growth of our database (> 1TB), the 'housekeeper' no longer worked properly. The best solution to this problem is to apply Database Partitioning, however with a database of this size this takes a lot of time if you want to keep the data. We tried this action in several ways, the one below was the only way we were able to implement partitioning without downtime.

The example below must be repeated for each table and takes several hours per table.

# Create temporary partition 
CREATE TABLE `history_log_tmp` LIKE `history_log`;
# Apply partitioning
CALL partition_maintenance('zabbix', 'history_log_tmp', 30, 24, 3);

# Rename tables so the new empty table will be used by Zabbix. Leaving the old one as backup
BEGIN;
RENAME TABLE history_log TO history_backup_log;
RENAME TABLE history_log_tmp TO history_log;
COMMIT;

# Output all data from backup table to file
SELECT * INTO OUTFILE '/var/lib/mysql-files/history_backup_log.sql' FROM history_backup_log;

# Open MySQL Shell and start import
mysqlsh
shell.connect('localhost:3306')
util.importTable("/var/lib/mysql-files/history_backup_log.sql", {schema: "zabbix", table: "history_log", columns: ["itemid","clock","value","ns"], dialect: "default", skipRows: 0, showProgress: true, fieldsOptionallyEnclosed: false, linesTerminatedBy: "\n",threads: 2, bytesPerChunk: "50M", maxRate: "10M"})
Categories
Linux

How to Get the Size of all tables in a MySQL Database

SELECT
  TABLE_NAME AS `Table`,
  ROUND((DATA_LENGTH + INDEX_LENGTH) / 1024 / 1024) AS `Size (MB)`
FROM
  information_schema.TABLES
WHERE
  TABLE_SCHEMA = "zabbix"
ORDER BY
  (DATA_LENGTH + INDEX_LENGTH)
DESC;

Categories
Linux

Mysql Clear Diskspace

When you are running out of diskspace you can purge the MySQL binary logs to free up some space

mysql> PURGE BINARY LOGS BEFORE 'yyyy-mm-dd hh:mm:ss';

Sometimes you are already on 99% disk space and need more drastic methods. This requires manually removing the logfiles.

systemctl stop mysql
cd /var/llog/mysql && a=`ls |grep -v relay |grep bin.index` && b=`wc -l <$a` ; c=`echo $(($b/2))` |xargs -l rm ; echo $c | head -n $b $a |cut -d "/" -f2 && sed 1,$c\d $a -i
systemctl start mysql
Categories
Linux

Check nvme health and temperature – nvme-cli

Make sure nvme-cli is installed:

$ sudo apt install nvme-cli

Check for availible nvme disks:

$ sudo nvme list
Node             SN                   Model                                    Namespace Usage                      Format           FW Rev  
---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- --------
/dev/nvme0n1     S4EVNFXXXXXXXX9972H      Samsung SSD 970 EVO Plus 500GB           1          26,60  GB / 500,11  GB    512   B +  0 B   2B2XXXXXM7

With nvme-cli you can now check the internal temperature, disk usage, power cycles, and much more:

$ sudo nvme smart-log /dev/nvme0
Smart Log for NVME device:nvme0 namespace-id:ffffffff
critical_warning                    : 0
temperature                         : 40 C
available_spare                     : 100%
available_spare_threshold           : 10%
percentage_used                     : 0%
data_units_read                     : 90935
data_units_written                  : 119679
host_read_commands                  : 4491381
host_write_commands                 : 2370351
controller_busy_time                : 8
power_cycles                        : 34
power_on_hours                      : 9
unsafe_shutdowns                    : 1
media_errors                        : 0
num_err_log_entries                 : 0
Warning Temperature Time            : 0
Critical Composite Temperature Time : 0
Temperature Sensor 1                : 40 C
Temperature Sensor 2                : 38 C
Thermal Management T1 Trans Count   : 0
Thermal Management T2 Trans Count   : 0
Thermal Management T1 Total Time    : 0
Thermal Management T2 Total Time    : 0
Categories
Linux Networking Security

Ubuntu 18.04 – OpenVPN Server in less then 5 minutes

OpenVPN provides flexible VPN solutions to secure your data communications, whether it's for Internet privacy, remote access for employees, securing IoT, or for networking Cloud data centers. Our VPN Server software solution can be deployed on-premises using standard servers or virtual appliances, or on the cloud.

Prepare your system

Make sure all latests packages and updates have been installed:

$ sudo apt update
$ sudo apt upgrade
$ sudo apt dist-upgrade

Download and run installation script

$ wget https://git.io/vpn -O openvpn-install.sh
$ sudo chmod +x openvpn-install.sh
$ sudo ./openvpn-install.sh 

The script will ask you some questions for it's basic configuration.
- When your IP address is asked, choose your WAN (public) address
- When protocol is asked, i recommend default UDP
- Port can be anything you want, i normally keep default
- When asked, pick 1.1.1.1 as your DNS server as this is one of the fastest currently online.

After this the installation will go ahead and inform you when it's done. You can verify if OpenVPN is running or not:

$ sudo systemctl status openvpn@server # <--- get server status

You can start or stop OpenVPN with the following commands:

$ sudo systemctl stop openvpn@server # <--- stop server
$ sudo systemctl start openvpn@server # <--- start server

Client configuration

At the end of the installation you whould see a message like this:

Your client configuration is available at: /root/bontekoe.ovpn

As i am using Linux (Ubuntu) on my laptop, i can simply copy that ovpn file to my computer using scp.

$ sudo scp root@88.99.189.27:/root/bontekoe.ovpn /etc/openvpn/client.conf

This should be enough to connect! Check if everything is working by running:

$ sudo openvpn --client --config /etc/openvpn/client.conf

Now, by opening another terminal you should be able to ping 10.8.0.1 (the VPN host).

If you are running windows, there is a client here.

Categories
Linux

Ubuntu 18.04 – Laggy bluetooth

After installing this version my mouse became laggy and also my headphones. Here is the fix:

# HANDLE="$(hcitool con | grep '<Bluetooth Mouse mac address>' | awk '{print $5}')"  # get the device handle
# hcitool lecup --handle $HANDLE --latency 0 --min 6 --max 8
Categories
Linux

Benchmarking SSDs with fio

Fio which stands for Flexible I/O Tester is a free and open source disk I/O tool used both for benchmark and stress/hardware verification that i mainly use for benchmarking ceph or specific ssd harware.

When using an SSD make sure it's pre-warmed. This can be done using the dd command:

dd if=/dev/zero of=/dev/xvdb bs=100M &

After this you can start performance measurement with fio. My advice is to run this test for 6 to 8 hours in order to get real data out of it.

fio --filename=/dev/nvmeXnXpX --direct=1 --rw=randwrite --refill_buffers --norandommap --randrepeat=0 --ioengine=libaio --bs=128k --iodepth=16 --numjobs=1 --time_based --runtime=86400 --group_reporting –-name=benchtest

This command will run for 24 hours and perform write-only workload of 128k blocks on a single process.

Random Read test

sudo fio --name=randread --ioengine=libaio --iodepth=16 --rw=randread --bs=4k --direct=0 --size=512M --numjobs=4 --runtime=240 --group_reporting

This will use 4 processes, run for 2 minutes and only perform read iops.

Random Write test

sudo fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=random_read_write.fio --bs=4k --iodepth=64 --size=4G --readwrite=randrw --rwmixread=75

This will to a read/write test on a 4 GB file.

Categories
Ansible Linux

Ansible through Ubuntu (WSL) on Windows 10

Windows Subsystem for Linux (WSL) allows you to run Linux straight from your Windows Desktop. I use this on a daily basis for running Ansible scripts without having to install VM's. Make sure you installed al latest updates.

Enable WSL feature

Open up a Powershell box as Administrator (search powershell, right click and run as Administrator).

Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Windows-Subsystem-Linux

This will initiate the installation and once finished ask if you would like to reboot your system. Go ahead and do that. When the reboot is done search for 'bash' and open that, it will first require a few anwsers. Simply fill out all the questions and once that is done you will have Ubuntu up and running.

Install Ansible

Now you are basicly in a Linux environment so you can install Ansible the typical way. Again, in the 'bash' window of course, use these instructions:

sudo apt-get -y install python-pip python-dev libffi-dev libssl-dev
sudo pip install ansible

Should you get any permission errors (i did not have this time, but given the nature of how WSL works that could happen) install pip with the --user flag. This will cause it to install ansible in the users home dir, not globally.

You are done. Using the following command you can check what ansible version is now installed:

ansible --version

If you need the most recent version check out my other post here.

Categories
Ansible Linux

Install latest version ansible on Ubuntu 16.04 / 18.04

Ubuntu doesn't ship with the newest version of ansible out of the box, sadly. You have to manually configure the PPA on your system in order to upgrade to the stable version. Follow these commands to install the PPA:

$ sudo apt update
$ sudo apt upgrade
$ sudo apt install software-properties-common
$ sudo apt-add-repository ppa:ansible/ansible

Hit Enter when asked, and once the process is done update your apt repos:

$ sudo apt update

Now you can either upgrade or simply install ansible:

$ sudo apt install ansible

This should be all, use the following to verify the ansible version:

$ ansible --version