Home Page

VPN & Cryptography


Email & Spam

Security Terminology


VPN Terminology

VPN Tutorial Guide



Aggressive Mode

Authentication Header

Asymmetric Encryption


Certification Authority

Data Integrity



Digital Certificate

Dynamic IP addresses





IPSec Quick Mode


Main Mode







Pre-Shared Key

Remote Access User


Security Association


Site to Site VPN


Transform Sets

Tunnel mode and Transport mode

VPN client tunneling option

VPN Topologies

VPN Tunnel


IPsec VPN Traffic Tutorial



IPsec works at the network layer of the OSI model and is a framework consisting of protocols and algorithms for protecting data through an un-trusted network such as the internet. IPsec provides data security in various ways such as encrypting and authenticating data, protection against masquerading and manipulation. IPsec is a complex framework consisting of many settings, which is why it provides a powerful and flexible set of security features that can be used.

IPsec is a collection of different protocols or algorithms and can be configured using over 30 different settings. IPsec is used to secure traffic between site-to-site VPN gateway devices or between remote access users and VPN gateway devices. As the world of IT and network security is constantly changing, this very much fits in well with IPsec, simply because IPsec is a framework, which allows you to add new and better algorithms as they are developed and released, and keep up to the pace in line with other IT evolvements and security standards.

When a VPN tunnel is to be created between two IPsec VPN gateway devices, the devices negotiate on various settings and parameters and must make an agreement on the parameters used. For example the type of authentication and encryption that will be used within the VPN tunnel, and both sides must use the exact same algorithms, otherwise it doesn't work. This is generally called VPN negotiation.

IPsec typically uses the following algorithms as detailed below:

- Encryption: 3DES, AES 128, AES 192, AES 256 for encryption of data,

- Authentication: MD5, Sha1, Sha26, Sha384, Sha512 are common authentication algorithms used

- Peer Authentication or Internet Key Exchange algorithms:

-- RSA is one common algorithm used for internet key exchange used during the peer authentication phase, to ensure the other side is authentic and who they say they are.

-- Diffie-Hellman is another commonly used algorithm, and the higher the Diffie-Hellman group, the more secure it is, but also has an impact on performance. Some VPN devices provide the option to support a wide range of groups such as the ones detailed below.

Diffie-Hellman Group, 1, 2, 5, 14, 15, 16, 17, 18, 19, 20, 21, 27, 28, 29, 30, 31, 32


Four key functions or services of IPsec are as follows:

1 Confidentiality – by encrypting data, this scrambling of data to make it unreadable.

2 Data Integrity – to ensure data has not been changed, whilst in transit.

3 Data Authentication – to ensure both sides trust the end other of the VPN tunnel, to prove both sender & receiver are who they say they are.

4 Anti-replay – to verify each packet is unique, and has not been duplicated or intercepted.


There are five phases of IPsec negotiation as detailed below:

1 Definition interesting traffic - the IP subnets that have been identified that is to be encrypted within the tunnel

2 IKE phase 1 – this is the IPsec key exchange phase

3 IKE phase 2 – IPsec policy and transform sets are processed and agreed

4 Transfer data – After the tunnel has been established, data can be transferred between the hosts defined within the interesting traffic

5 Tear down the tunnel - after the transfer of data, the tunnel is removed (unless its a permanent tunnel)


IPsec uses two different protocols to encapsulate the data over a VPN tunnel:

Encapsulation Security Payload (ESP): IP Protocol 50

Authentication Header (AH): IP Protocol 51

ESP is more secure as it provides data encryption. AH provides authentication only.

Further Reading

Wikipedia's guide to IPSec