Split Tunneling a VPN on an Ubuntu Server
I decided I needed a single bare metal setup rather than using LXC/LXD containers to separate certain apps on my server. This required a split tunnel VPN. This enables the server to route certain traffic through a VPN connection, while everything else will have direct access, bypassing the VPN. Network traffic will be securely separated.
Install OpenVPN from the OpenVPN repository. I’ll use the actual OpenVPN repository just to install the latest version.
wget https://swupdate.openvpn.net/repos/repo-public.gpg -O - | sudo apt-key add - echo "deb http://build.openvpn.net/debian/openvpn/stable xenial main" | sudo tee -a /etc/apt/sources.list.d/openvpn.list sudo apt-get update && sudo apt-get install openvpn -
I create the systemd service that will start the required OpenVPN configuration on system start. This script will also restart the OpenVPN service if the service was terminated for some reason. Logs will be located in syslog (/var/log/syslog). For debugging purposes I’ll also add them to the OpenVPN config file later.
[Unit] Description=OpenVPN connection to %i Documentation=man:openvpn(8) Documentation=https://community.openvpn.net/openvpn/wiki/Openvpn23ManPage Documentation=https://community.openvpn.net/openvpn/wiki/HOWTO After=network.target [Service] RuntimeDirectory=openvpn PrivateTmp=true KillMode=mixed Type=forking ExecStart=/usr/sbin/openvpn --daemon ovpn-%i --status /run/openvpn/%i.status 10 --cd /etc/openvpn --script-security 2 --config /etc/openvpn/%i.conf --writepid /run/openvpn/%i.pid PIDFile=/run/openvpn/%i.pid ExecReload=/bin/kill -HUP $MAINPID WorkingDirectory=/etc/openvpn Restart=on-failure RestartSec=3 ProtectSystem=yes LimitNPROC=10 DeviceAllow=/dev/null rw DeviceAllow=/dev/net/tun rw [Install] WantedBy=multi-user.target
Enable the openvpn@openvpn.service to start on boot.
sudo systemctl enable openvpn@openvpn.service
Create PIA (Private Internet Access)Configuration File for Split Tunneling
I use Private Internet Access as my VPN, but this should work with any VPN. The next step is to modify the configuration file provided by PIA to adjust it for the Split Tunneling. In this guide we will use the Sweden VPN server, but you can use any of the available servers. For best VPN performance (especially for torrents) I strongly recommend using UDP protocol, and not TCP. I’m going to also copy over the necessary certificates for PIA’s servers.
Get the Required Certificates for PIA
cd /tmp sudo wget https://www.privateinternetaccess.com/openvpn/openvpn.zipsudo unzip openvpn.zip sudo cp crl.rsa.2048.pem ca.rsa.2048.crt /etc/openvpn/
Create Modified PIA Configuration File for Split Tunneling
We make the following changes to the default PIA configuration file. Add route-noexec to prevent the server from push “redirect-gateway” and make the client send all traffic over VPN by default. Add auth-nocache to prevent caching passwords in memory. Add the call for login.txt file with the username and password to make automatic login possible. Add script-security 2 to allow client to call externals scripts (up and down).
Call the first script, iptables.sh, to mark packets for the VPN user (OpenVPN allows only one up call). Add down script, update-resolv-conf, to restore DNS servers when disconnecting from VPN. You can change the hostname highlighted in red to the server of your choice. For the list of available locations and hostnames look at PIA’s site and select the hostname of your choice.
We are editing the openvpn.conf file which is launched by the systemd service we created earlier (openvpn@openvpn.service). This way we ensure the up and down scripts we made are also executed with the OpenVPN systemd service is restarted.
Create the OpenVPN configuration file
sudo nano /etc/openvpn/openvpn.conf
Copy and paste the following, adjust the country you want to use by replacing sweden.privateinternetaccess.com
client dev tun proto udp remote sweden.privateinternetaccess.com 1198 resolv-retry infinite nobind persist-key persist-tun cipher aes-128-cbc auth sha1 tls-client remote-cert-tls server auth-user-pass /etc/openvpn/login.txt auth-nocache comp-lzo verb 1 reneg-sec 0 crl-verify /etc/openvpn/crl.rsa.2048.pem ca /etc/openvpn/ca.rsa.2048.crt disable-occ script-security 2 route-noexec #up and down scripts to be executed when VPN starts or stops up /etc/openvpn/iptables.sh down /etc/openvpn/update-resolv-conf #debugging log /var/log/openvpn.log pull-filter ignore "auth-token"
Make OpenVPN Auto Login on Service Start
The username and password for PIA will be stored in a login.txt file, this way OpenVPN can auto connect on service start. Create the txt file
sudo nano /etc/openvpn/login.txt
Enter your PIA username and password
USERNAME PASSWORD
Configure VPN DNS Servers to Stop DNS Leaks
DNS Leaks are often the main reason your real identity gets exposed even if using VPN. The update-resolv-conf script that comes with OpenVPN will automatically apply the preferred DNS servers when OpenVPN connects.
This script will make sure that when using OpenVPN you are not subject to DNS leaks. We will use PIA’s DNS Servers (209.222.18.222 and 209.222.18.218) and Google’s (8.8.8.8) as a third option. You are free to use the DNS servers you trust and prefer. It is advised to change the local DNS to a public even if you are not using VPN. If you are behind a router (and you probably are), it is also a good practice to configure public DNS address on the router too.Note: make sure you are using a static IP on your machine or reserved DHCP also known as static DHCP. Do not configure the static IP on your server, as resolvconf will not work then. You should set the static IP from your router.
Open the update-resolv-conf file
sudo nano /etc/openvpn/update-resolv-conf
Add the following under the example DNS settings:
foreign_option_1='dhcp-option DNS 209.222.18.222'<br>foreign_option_2='dhcp-option DNS 209.222.18.218'<br>foreign_option_3='dhcp-option DNS 8.8.8.8'
Split Tunneling with IPtables and Routing Tables
We will use iptables to mark packets from a user (in our case a “vpn” user), and then use routing tables to route these marked packets through the OpenVPN interface, while allowing unmarked packets direct access to the Internet. Create the user vpn. All of the applications you want tunneled over VPN will run as this user. Create vpn user with no login option for security reasons as there is no need to log in to the system as the vpn user.
sudo adduser --disabled-login vpn
We are going to use the vpn user to run certain services, it is recommended to add your regular user to the vpn group and vpn user to your regular user’s group to avoid any permission issues.
Replace username with the user you would like to add to the vpn group
sudo usermod -aG vpn username
Replace group with the group name of your regular user that you would like to add the vpn user to
sudo usermod -aG group vpn
Get Routing Information for the iptables Script
We need the local IP and the name of the network interface. Again, make sure you are using a static IP on your machine or reserved DHCP also known as static DHCP, but configured on your router.
ip route list
The output will be similar to this
default via 192.168.1.1 dev enp0s25<br>192.168.1.0/24 dev enp0s25 proto kernel scope link src 192.168.1.150
enp0s25 is the network interface (NETIF), and 192.168.1.150 is the local IP address (LOCALIP). You will need to enter these two into the following script, which we are going to create now.
Flush current iptables rules
sudo iptables -F
Add the following rule, which will block vpn user’s access to Internet (except the loopback device).
sudo iptables -A OUTPUT ! -o lo -m owner --uid-owner vpn -j DROP
Now install iptables-persistent to save this single rule that will be always applied on each system start.
sudo apt-get install iptables-persistent -y
During the install, iptables-persistent will ask you to save current iptables rules to /etc/iptables/rules.v4 if you choose YES at the prompt.
Now when system starts, vpn user is not able to access Internet. If the OpenVPN service is started successfully, then this rule is flushed (only until the next system restart), and the Split Tunnel rules are applied.
Scripts for managing iptables and routing.
The first script will mark the packets for vpn user, the second script will take care of proper routing. Create the iptables script
sudo nano /etc/openvpn/iptables.sh
Copy the following to the iptables.sh script, and make sure you enter the network interface and the local IP we identified earlier. You can see the comments for each section about the function of the given part of the script. If you are interested in more details about iptables, a good starting point is the Official Ubuntu Documentation. Remember, this script will flush your existing iptables rules (UFW included), therefore you need to append your own rules into this script if you need any additional firewall rules.
#! /bin/bash export INTERFACE="tun0" export VPNUSER="vpn" export LOCALIP="192.168.1.0/24" export NETIF="enp0s25" # flushes all the iptables rules, if you have other rules to use then add them into the script iptables -F -t nat iptables -F -t mangle iptables -F -t filter # mark packets from $VPNUSER iptables -t mangle -A OUTPUT -j CONNMARK --restore-mark iptables -t mangle -A OUTPUT ! --dest $LOCALIP -m owner --uid-owner $VPNUSER -j MARK --set-mark 0x1 iptables -t mangle -A OUTPUT --dest $LOCALIP -p udp --dport 53 -m owner --uid-owner $VPNUSER -j MARK --set-mark 0x1 iptables -t mangle -A OUTPUT --dest $LOCALIP -p tcp --dport 53 -m owner --uid-owner $VPNUSER -j MARK --set-mark 0x1 iptables -t mangle -A OUTPUT ! --src $LOCALIP -j MARK --set-mark 0x1 iptables -t mangle -A OUTPUT -j CONNMARK --save-mark # allow responses iptables -A INPUT -i $INTERFACE -m conntrack --ctstate ESTABLISHED -j ACCEPT # block everything incoming on $INTERFACE to prevent accidental exposing of ports iptables -A INPUT -i $INTERFACE -j REJECT # let $VPNUSER access lo and $INTERFACE iptables -A OUTPUT -o lo -m owner --uid-owner $VPNUSER -j ACCEPT iptables -A OUTPUT -o $INTERFACE -m owner --uid-owner $VPNUSER -j ACCEPT # all packets on $INTERFACE needs to be masqueraded iptables -t nat -A POSTROUTING -o $INTERFACE -j MASQUERADE # reject connections from predator IP going over $NETIF iptables -A OUTPUT ! --src $LOCALIP -o $NETIF -j REJECT # Start routing script /etc/openvpn/routing.sh exit 0
Make the iptables script executable
sudo chmod +x /etc/openvpn/iptables.sh
With the routing rules we configure the route for the packets we just marked with the first script.
sudo nano /etc/openvpn/routing.sh
Paste the following script which makes the default route after the VPN the loopback interface, effectively nullifying the traffic if the VPN connection goes down.
#! /bin/bash
VPNIF="tun0"
VPNUSER="vpn"
GATEWAYIP=$(ifconfig $VPNIF | egrep -o '([0-9]{1,3}\.){3}[0-9]{1,3}' | egrep -v '255|(127\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3})' | tail -n1)
if [[ `ip rule list | grep -c 0x1` == 0 ]]; then
ip rule add from all fwmark 0x1 lookup $VPNUSER
fi
ip route replace default via $GATEWAYIP table $VPNUSER
ip route append default via 127.0.0.1 dev lo table $VPNUSER
ip route flush cache
# run update-resolv-conf script to set VPN DNS
/etc/openvpn/update-resolv-conf
exit 0
Finally, make the script executable
sudo chmod +x /etc/openvpn/routing.sh
We will need a routing table by adding the table name to the rt_tables file (in our case it is vpn). Open rt_tables
sudo nano /etc/iproute2/rt_tables
Add the vpn user table at the bottom of the file
200 vpn
It should look similar to this:
# # reserved values # 255 local 254 main 253 default 0 unspec # # local # # 1 inr.ruhep 200 vpn
Finally, we need to change the default level of reverse path filtering to ensure the kernel routes the traffic correctly. By default it is set to value of 1 that is “strict mode”. It is not necessary to disable reverse path filtering completely (setting to “0”), but we need to set it to level 2, “loose mode”.
Create a reverse path filter file for the vpn user
sudo nano /etc/sysctl.d/9999-vpn.conf
Copy the following, make sure you use the correct network interface name in the third line (remember the ip route list command from before and the output, in my case it was enp0s25)
net.ipv4.conf.all.rp_filter = 2 net.ipv4.conf.default.rp_filter = 2 net.ipv4.conf.enp0s25.rp_filter = 2
To apply new sysctl rules run:
sudo sysctl --system
Testing the VPN Split Tunnel
I recommend a system restart, and if everything was configured properly, you should have a running OpenVPN service enabled for the vpn user and all the other users on your server should have direct access to Internet. Now lets check if everything is correct. Login as your regular user over SSH, and check OpenVPN service status
sudo systemctl status openvpn@openvpn.service
This should return something like this:
openvpn@openvpn.service - OpenVPN connection to client Loaded: loaded (/etc/systemd/system/openvpn@openvpn.service; enabled; vendor preset: enabled) Active: active (running) since Mon 2016-09-05 11:25:18 CEST; 1 day 4h ago Docs: man:openvpn(8) https://community.openvpn.net/openvpn/wiki/Openvpn23ManPage https://community.openvpn.net/openvpn/wiki/HOWTO Process: 3223 ExecStart=/usr/sbin/openvpn --daemon ovpn-%i --status /run/openvpn/%i.status 10 --cd /etc/openvpn Main PID: 3266 (openvpn) CGroup: /system.slice/system-openvpn.slice/openvpn@openvpn.service └─3266 /usr/sbin/openvpn --daemon ovpn-client --status /run/openvpn/client.status 10 --cd /etc/openvpn Sep 05 11:25:21 server ovpn-client[3266]: OPTIONS IMPORT: route options modified Sep 05 11:25:21 server ovpn-client[3266]: OPTIONS IMPORT: --ip-win32 and/or --dhcp-option options modified
If the service is not running you can check if there is a log of the error in /var/log/syslog . For troubleshooting you can set output verbosity in the openvpn.conf file to a higher level. Set it to 3 and check the syslog again. Remember to set verbosity level back to 1 if you don’t need more detailed logs anymore.
Check IP address
Using the SSH session for the regular user, check the IP address
curl ipinfo.io
It will return your IP and depending on how much information is provided, the country should be listed in each case. Obviously, it should be your ISP now and your location.
Now check the IP address of the vpn user with
sudo -u vpn -i -- curl ipinfo.io
If everything went fine, it should return the IP address and the country of the VPN server you selected. If you used Sweden server, then the country should be “SE”. It is very important that the IP address for user vpn should be different then your regular user’s IP.
In my case for user vpn and using Sweden PIA server I have the following output
{
"ip": "X.XXX.XXX.XX",
"hostname": "No Hostname",
"city": "",
"region": "",
"country": "SE",
"loc": "59.3294,18.0686",
"org": "AS57858 Inter Connects Inc"
Obviously, the “x.xxx.xxx.xx” part is my assigned VPN IP address which is different then my public IP, and you can see the country as SE which is Sweden.
Check DNS Server
Finally, check if the DNS for VPN is properly configured, type
sudo -u vpn -i -- cat /etc/resolv.conf
The output should be
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8) # DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN nameserver 209.222.18.222 nameserver 209.222.18.218 nameserver 8.8.8.8
If you see the above DNS servers then your DNS for VPN is configured correctly.