X.509 IPsec on EdgeOS

Configuring X.509 based IPsec on an Edgerouter running EdgeOS.

Preparation

Create a directory in /config for persistent storage of the certificates. Eg:

mkdir /config/ipsec
cd /config/ipsec

Generation of Certificates

For IPsec to work reliably, you need three certificate files:

CA-certificate: ca.crt, obtain this from your CA.
Server certificate: server.crt
Server private key: server.key

Running a CA is outside of the scope of this document, though a script (/usr/lib/ssl/misc/CA.sh) is provided to make it very easy. There are other blog posts documenting this. Personally, I prefer to use the openssl commands directly. I also advise to run the following as root (sudo -i).

First, generate a certificate signing request (CSR) for the CA to sign, replace the subject with your own values:

openssl req -new -newkey 4096 -nodes -out server.csr \
  -keyout server.pem \
  -subj "/C=NL/ST=/L=/O=SLXH/CN=ipsec.slxh.nl"

This private key needs to be converted to a RSA private key for the IPsec daemon on EdgeOS:

openssl rsa -in server.pem -out server.key

Change the permissions on the files:

chmod u=r,go= server.pem server.key
chown root: server.pem server.key

This CSR can now be signed by your CA. After you get the signed certificate, place in next to the private keys.

IPsec on EdgeOS

Follow the instructions provided by Ubiquiti or the ones found on their forum, with the following exceptions:

set vpn l2tp remote-access ipsec-settings authentication mode x509
set vpn l2tp remote-access ipsec-settings authentication x509 ca-cert-file /config/ipsec/ca.crt
set vpn l2tp remote-access ipsec-settings authentication x509 server-cert-file /config/ipsec/server.crt
set vpn l2tp remote-access ipsec-settings authentication x509 server-key-file /config/ipsec/server.key

Adding a certificate revocation list (CRL) is optional, but advised:

set vpn l2tp remote-access ipsec-settings authentication x509 crl-file /config/ipsec/ca.crl

Commit and save:

commit
save

Firewall

Again, follow the instructions provided by Ubiquiti. It can also be configured in the CLI (check the rule numbers first):

edit firewall name WAN_LOCAL rule 23
set action accept
set description "L2TP/IPsec"
set destination port isakmp,l2f,4500
set protocol udp
exit
edit firewall name WAN_LOCAL rule 24
set action accept
set description "ESP (IPsec)"
set protocol esp
commit
save

IPsec should now be working.