Publication Date: September 11, 2000

Preliminary Test Plan

 

v. 1.02 Copyright © 2000 Network Test Inc. and CMP Media Inc.  Vendors are encouraged to comment on this document and any other aspect of test methodology. However, Network Test and CMP Media reserve the right to change the parameters of this test at any time.

 

This document describes the procedures to be used in comparative testing of virtual private network (VPN) gateways. Most tests described here assume a gateway-to-gateway configuration, as is typically used between corporate offices. One prominent exception is the concurrent connection test, which demonstrates a gateway’s ability to support a large number of dial-in clients.

 

There are three main areas in these tests: security, performance, and management. It is assumed that all gateways use IPSec for authentication and encryption and IKE (Internet key exchange) for key management.

 

1. The Test Bed

The following diagram shows the test bed layout. Both VPN gateways under test attach to a Summit48 switch from Extreme Networks Inc., which has proven in past tests to be capable of forwarding all traffic on all ports at line speed.

 

All interfaces on the test bed will be forced to use 100-Mbit/s rates and full duplex, with autonegotiation disabled.

 

 

2. Security Tests

To evaluate device security, the lab will use Internet Scanner 6.0 from Internet Security Systems Inc. (Atlanta) to probe the VPN gateway under test for various vulnerabilities, such as source porting, teardrop attacks, IP address spoofing, SYN flood attacks, and many other potential weaknesses.

 

A complete description of attacks in Internet Scanner is available here:

http://xforce.iss.net

In addition, Network Test may employ various other vulnerability assessment tools to determine that devices are not prone to various well-known attacks.

Vulnerability scans will be run against both the unprotected (public) and protected (private) interfaces of the gateway.

 

3. Performance Tests

The test bed is shown in Section 1. It comprises four subnets:

 

·        10.1.0.0/16  - Private subnet at corporate headquarters

·        10.2.0.0/16 - Private subnet at corporate branch office

·        10.3.0.0/16 - Public Internet

·        10.4.0.0/16 - Public Internet

 

Note that a Radius server is supplied on the test bed, but its use is optional if the DUT incorporates its own user directory and authentication server.

 

Vendors may supply configurations that use load-balancing or clustering of multiple devices. In all cases, the test bed will employ the IP addressing scheme given above.

 

The VPN performance evaluation has two components. First, throughput tests will determine the maximum forwarding rate of packets sent through a single tunnel (i.e., large number of bits, low number of tunnels), using both UDP/IP and TCP/IP flows. Second, a concurrent connection test will determine the maximum number of concurrent tunnels and TCP sessions the device under test (DUT) can maintain (i.e., relative low number of bits, large number of tunnels).

 

For all performance testing, vendor's representatives will configure all devices under test (DUTs) to meet the following criteria:

 

--Compression disabled

--Network address translation disabled

--Triple DES encryption enabled

--Message authentication enabled using SHA-1 or MD5

--Tunneling enabled using ESP (encapsulating security payload)

 

Time permitting, vendors may choose to conduct retests with compression and/or network address translation enabled to determine the performance costs of these features.

 

3.1 UDP and TCP Throughput

In the first test, a Netcom Systems Smartbits traffic generator/analyzer offers a unidirectional stream of 64-byte UDP/IP packets from the headquarters office destined to another Smartbits at the branch office. This exercise is repeated with 1,400-byte UDP/IP packets. The lab will record throughput and minimum, maximum, and average latency.

 

Traffic will be measured at two places: on the local (protected) side of the branch office, by the receiving Netcom Systems Smartbits interface; and on the public (unprotected) interface of the branch office, using a Network Associates Sniffer Pro protocol analyzer and port mirroring configured on the Extreme switch. Past tests have shown that the software-based Sniffer cannot capture all packets at 100-Mbit/s line speeds; however, it is capable of counting packets accurately at line rate.

 

These tests are repeated using TCP/IP flows generated by Chariot, an application-layer traffic generator from Ganymede Software. The lab will use Chariot’s Filesndl script, which simulates a bulk file transfer over TCP. Chariot reports on throughput and response time, which is measured on the local (protected) side of the branch-office gateway.

 

3.2 Maximum concurrent connections

To determine maximum concurrent connections, the lab will use FTPConnLoad, a Windows NT-based testing tool developed by NSTL Inc. (Conshohocken, Pa.).

 

FTPConnLoad seeks to open as many FTP (file transfer protocol) sessions as possible. Once the connections are established, the tool verifies that each is capable of delivering traffic. To do so, FTPConnLoad generates the command “cd ..” once per second on each connection.

 

In previous tests, the lab has been able to generate up to 80,000 concurrent connections using this tool on 4-CPU machines, each running Windows NT and equipped with 256 Mbytes of RAM.

 

For devices that prove capable of maintaining more than 65,534 connections, we will reconfigure the test bed’s addressing scheme with /8 prefixes.

 

4. Management Tests

 

Vendors will supply the software and hardware necessary for managing their gateways. The lab will evaluate VPN management by completing a series of scenario-based tasks.

 

For each task, the lab will grade each device, on a 1 to 5 scale (where 1 = poor, 2 = fair, 3 = good, 4 = very good, and 5 = excellent), on ease of accomplishing each of the following tasks.

 

1. The headquarters site uses Entrust/PKI, running on a Windows NT platform, for certificate management. The VPN gateways will interact with the Entrust/PKI server to obtain certificates used in tunnel establishment.

 

The lab will determine whether the VPN gateway under test will then change keys at a regularly scheduled interval, and whether it will change keys upon notification of certificate revocation by the Entrust server

 

2. The company's security policy formerly allowed only access only to a subnet in the engineering group, but the company wishes to allow access to a subnet in the marketing group. Further, the network manager is making the change from outside the corporate network's security perimeter.

 

a. The lab will verify that remote configuration traffic is actually encrypted.

b. The lab will determine whether new rules take effect without causing the dropping of current connections. Dropping of current connections will force reauthentication.

c. The lab will determine whether a rule change requires a system reboot. If a reboot is required, lab personnel will measure how long the DUT is unavailable.

d. The lab will determine whether configuration changes can be made for groups encompassing multiple VPN gateways, or whether each gateway has to be individually updated.

 

3. The company has users at headquarters and at various branch sites. It has already established user accounts residing on different platforms (various OSs, email directory services, RADIUS servers, etc.).

 

a. The lab will determine whether the VPN device can leverage these user accounts from these various platforms without having to recreate each user individually

b. The lab will determine whether user accounts can be remotely administered

c.  The lab will determine whether users into can be organized into logical groups (e.g., engineering, marketing, East Coast, West Coast, etc.)

d.  The lab will determine whether the VPN device provides resource accounting by individuals

e. The lab will determine whether the VPN device provides resource accounting by user groups