Light Reading Lab Test: SSL-Based VPN Gateways
Scheduled for Publication in Fall 2003
Test Methodology

v. 1.4 Copyright Ó 2003 by Network Test. Vendors are encouraged to comment on this document and any other aspect of test

methodology. Network Test reserves the right to change test parameters at any time.

1 Executive Summary

This document describes a series of tests to be conducted on SSL-based VPN gateways. We plan to publish test results, along with a features comparison, in Light Reading. The tests described here include:

¨ SSL setup/teardown rates

¨ Maximum concurrent users

¨ Outlook Web Access performance

¨ Outlook Web Access performance while under attack

¨ HTTP goodput

¨ Security assessment

1.1 Organization of this document

This document is organized as follows. This section introduces the project. Section 2 describes the test bed. Section 3 describes test procedures. Section 4 logs changes to this document.

2 The Test Bed

The figure below shows the basic test bed configuration. The device under test (DUT) bed should have at least 2 gigabit Ethernet interfaces, preferably copper (1000Base-T), although we can accommodate 1000Base-SX (multimode fiber) interfaces provided your DUT allows for manual speed and duplex configuration. Devices may have 100Base-T interfaces, although lower rates are possible in goodput tests.

External (public) IP addresses are part of the unsecured network. Internal (private) IP addresses should be isolated from the external side. We may perform leak tests during setup to verify that unsecured traffic does not reach the private network.

We use a Foundry FastIron 12GCF switch to connect both external and internal sides of the test bed, assigning different untagged VLANs to each side. We have verified the Foundry switch does not leak traffic between VLANs.

The primary test instruments for this project are:

Spirent Avalanche 2200 and Reflector 2200: These appliances generate and analyze very high volumes of layer 4-7 traffic. For this project we use SSL, HTTP, and Outlook Web Access sessions. More information about Avalanche and Reflector are available here:

http://www.spirentcom.com/analysis/product.cfm?WS=65&PR=95&wt=1

http://www.spirentcom.com/analysis/product.cfm?WS=65&PR=357&wt=1

Nessus: Nessus is an open-source vulnerability assessment scanner. We run version 2.0.7 on Linux for the server, and use Windows XP machines for clients. More information about Nessus is available here:

http://nessus.org

2.1 Test considerations

2.1.1 IP configuration

Vendors should configure the DUT’s external IP address as 3.1.0.1/16 with a default gateway of 3.1.0.254. The internal IP address should be 2.1.0.1/16 with a default gateway of 2.1.0.254. We do not require DNS for this project.

Spirent's Reflector 2200 will simulate two HTTP servers on two different interfaces. IP addresses are 2.1.1.1 (on interface if0) and 2.1.1.2 (on interface if2). Spirent's Avalanche 2200 will simulate traffic from many clients (each with a unique IP address), each trying to connect to each of the Reflector's emulated servers via the SSL VPN Appliance. The number of IP addresses we use depends upon the test. IP addresses on Avalanche interface if0 start at 1.1.1.1 and may run through 1.1.127.255. . IP address on Avalanche interface if2 start at 1.1.128.1 and may run through 1.1.255.254.

To reduce the number of ARP entries the SSL-based VPN gateway must track, we configure with Avalanche with virtual routers in front of the client networks. The virtual router addresses are 3.1.0.2/17 for if0 and 3.1.0.3/17 for if2.

The Nessus device uses an address of 3.1.0.254 on the external network and 2.1.0.254 on the internal network. Note that the Nessus IP addresses match those of the default gateways of the DUT. When not running security tests, we run tcpdump to verify ARPs and check whether the DUT sends traffic to nonlocal addresses.

2.1.2 Username and Passwords

Each SSL session requires a unique username and password. The total number of unique username/password accounts will be the maximum supported by the DUT. We use the following username/password combinations:

Username	Password
user0001	pass0001
user0002	pass0002
…	…
userNNNN	passNNNN

Where “NNNN” is the maximum number of concurrent users supported by the DUT.

A hint for vendors: It is very desirable to automate account and password creation. The authors tend to get grumpy after typing in thousands of names and passwords.

3 Test Procedures

3.1 SSL setup/teardown rates

3.1.1 Objective

To determine maximum rate at which users can log in to the DUT (authenticate), complete a transaction, and log out.

3.1.2 Configuration

We configure the Avalanche and Reflector using the following parameters:

- User think time = 0

- HTTP 1.1

- Accept-Encoding: gzip, deflate, compress

- SSL Version: SSLv2, SSLv3, or TLSv1

- Cipher is RC4-MD5

- SSL session ID reuse of 4

- Object size = 1024 bytes

- Username/password pool = 2500

- IP address pool = 2500

- Reflector latency = 0

The Avalanche load profile will be in simusers (a Spirent term for “simulated user”). Avalanche will ramp up from 0 to N simusers/sec in 30 seconds, where N is the desired rate of users per second. We disable Phase II ramp-up. Avalanche will generate a sustained load from N active simusers/sec for 120 seconds.

3.1.3 Procedure

We begin by verifying that the DUT authenticates only authorized users.

Then, using a binary search algorithm, we vary the number of simusers/second to determine the highest rate at which we can login and tear down sessions with zero failed transactions.

A successful SSL setup and teardown involves 4 URLs:

1. HTTP GET of the welcome screen. This is the screen where users enter name and password.

2. HTTP POST of the username/password.

3. HTTP GET a 1024 byte object from Reflector (not the DUT).

4. HTTP POST to log out the user.

During the final 60 seconds of the sustained period, we average the rate at which the DUT forward objects from the Reflector to the Avalanche. We record this rate as login/teardowns per second.

For DUTs that support data compression, we run this test twice: Once with compression disabled, and again with compression enabled.

3.1.4 Metrics

- SSL logins/teardowns per second

3.2 Maximum concurrent users

3.2.1 Objective

To determine the maximum number of concurrent users the DUT can support without dropping a user.

3.2.2 Configuration

We configure the Avalanche and Reflector using the following parameters:

- User think time = 0

- HTTP 1.1

- Accept-Encoding: gzip, deflate, compress

- SSL Version: SSLv2, SSLv3, or TLSv1

- Cipher is RC4-MD5

- SSL session ID reuse of 4

- Object size = 1024 bytes

- username/password pool = bulk imported into DUT

- IP address pool = 2500

- Reflector latency = 0 or 60 seconds, depending on object

The Avalanche load profile will be in simusers (a Spirent term for “simulated user”). Avalanche will ramp up from 0 to 80% login rate for 30 seconds. We disable Phase II ramp-up. Avalanche will generate a sustained load of 80% login rate for 300 seconds. If at the end of the test, it was determined that the DUT did not reach the maximum number of logins, we rerun the test with a longer sustained load time.

3.2.3 Procedure

A successful session involves 13 URLs:

1. HTTP GET of the welcome screen. This is the screen where users enter name and password.

2. HTTP POST of the username/password.

3. HTTP GET a 1024-byte object from Reflector (not the DUT).

4. HTTP GET a different 1024-byte object from Reflector. The transaction will have the same byte content as the previous, but will have 60-second latency.

5. We repeat step 4 10 times to ensure the DUT does not "time-out" the session.

At the end of the test, we record the total number of successful returned objects from step 3 (object with no latency) as the maximum number of concurrent users.

Using a binary search algorithm, we vary the number of simusers to find the highest number of logins/GETs the DUT can handle with 0 failures.

For DUTs that support data compression, we run this test twice: Once with compression disabled, and again with compression enabled.

3.2.4 Metrics

- Number of configured users

- Maximum number of concurrent users

3.3 Outlook Web Access (OWA) Performance

3.3.1 Objective

To determine transaction and response times through the DUT when handling simulated Microsoft Outlook Web Access (OWA) sessions.

3.3.2 Configuration

We configure the Avalanche and Reflector using the following parameters:

- User think time = 0

- HTTP 1.1

- Accept-Encoding: gzip, deflate, compress

- SSL Version: SSLv2, SSLv3, or TLSv1

- Cipher is RC4-MD5

- SSL session ID reuse = 79

- username/password pool = 200

- IP address pool = 200

The Avalanche load profile will be in simusers (a Spirent term for “simulated user”). Avalanche will ramp up from 0 to N simusers/sec in 30 seconds, where N is the highest number of users with 0 failed transactions. We disable Phase II ramp up. We configure Avalanche to generate a sustained load from N active users for 120 seconds.

3.3.3 Procedure

The simulated OWA session on Avalanche and Reflector emulates a user signing in, being authenticated, and seeing the Outlook inbox. The entire session requires 82 URLs:

1 HTTP GET of the welcome screen. This is the screen where users enter name and password.

2 HTTP POST of the username/password.

3 HTTP GET a replicated Microsoft Outlook Web Access inbox. This consists of 79 GETs of small objects, with an average object size of 976 bytes. Appendix A shows the URL order.

4 HTTP POST to logout the user.

We run a binary search algorithm to determine the highest number of users with 0 failed transaction.

During the final 60 seconds of sustained load, we average transaction rate from the Reflector. We record this figure as transactions per second.

For DUTs that support data compression, we run this test twice: Once with compression disabled, and again with compression enabled.

3.3.4 Metrics

Transactions per second

Average URL Response Time

Average Page Response Time

3.4 Outlook Web Access performance while under DDOS attack (optional)

3.4.1 Objective

To determine the of effect DDOS attacks on Outlook Web Access performance

3.4.2 Configuration

We configure Avalanche and Reflector with the following parameters:

- User think time = 0

- HTTP 1.1

- Accept-Encoding: gzip, deflate, compress

- SSL Version: SSLv2, SSLv3, or TLSv1

- Cipher is RC4-MD5

- SSL session ID reuse of 79

- username/password pool = 200

- IP address pool = 200

- DDOS: Smurf, SynFlood, UDPFlood, XmasTree

We use the following DDOS global attack variables:

- Starting SourceMAC Address: 00:80:51:00:00:01

- Starting DestMAC Address: (MAC address of DUT external interface)

- Starting SourceIP Address: 192.168.10.1

- Starting DestIP Address: 3.1.0.1

- Smurf Repeat Count: 1

- Smurf Packets To Generate: 42,000

- Smurf Packet Rate: 350

- SynFlood Repeat Count: 1

- SynFlood Packets To Generate: 42,000

- SynFlood Packet Rate: 350

- SynFlood TCP Source Port 1024

- SynFlood TCP DestPort 443

- UDPFlood Repeat Count: 1

- UDPFlood Packets To Generate: 42,000

- UDPFlood Packet Rate: 350

- UDPFlood UDP Source Port 1024

- UDPFlood UDP DestPort 512

- XmasTree Repeat Count: 1

- XmasTree Packets To Generate: 42,000

- XmasTree Packet Rate: 350

We use an attack rate of 350 frames/second because it is the median rate of attacks observed on Internet links as reported by CAIDA. More details of CAIDA’s observations are available here:

http://www.caida.org/outreach/papers/2001/BackScatter/usenixsecurity01.pdf

The Avalanche load profile will be in simusers (a Spirent term for “simulated user”). Avalanche will ramp up from 0 to N simusers/sec in 30 seconds, where N is the maximum number of users with 0 failed transactions, as determined in section 3.3. We disable Phase II ramp up. We configure Avalanche to generate a sustained load from N active users for 120 seconds. We source all attacks through virtual router 3.1.0.2/17 (Avalanche interface if0).

3.4.3 Procedure

The simulated OWA session on Avalanche and Reflector emulates a user signing in, being authenticated, and seeing the Outlook inbox. The entire session requires 82 URLs:

1 HTTP GET of the welcome screen. This is the screen where users enter name and password.

2 HTTP POST of the username/password.

3 HTTP GET a replicated Microsoft Outlook Web Access inbox. This consists of 79 GETs of small objects, with an average object size of 976 bytes. Appendix A shows the URL order.

4 HTTP POST to logout the user.

During the final 60 seconds of sustained load, we average transaction rate from the Reflector. We record this figure as transactions per second.

For DUTs that support data compression, we run this test twice: Once with compression disabled, and again with compression enabled.

3.4.4 Test Metrics

Transactions per second

Average URL Response Time

Average Page Response Time

3.5 HTTP Goodput

3.5.1 Objective

To determine the maximum goodput of the DUT as defined in RFC 2647.

3.5.2 Configuration

We configure the Avalanche and Reflector using the following parameters:

- User think time = 0

- HTTP 1.1

- Accept-Encoding: gzip, deflate, compress

- SSL Version: SSLv2, SSLv3, or TLSv1

- Cipher is RC4-MD5

- SSL session ID reuse of 4

- Transaction is 1 million byte object (Body size = 1000000).

- username/password pool = 200

- IP address pool = 2

The Avalanche load profile will be in simusers (a Spirent term for “simulated user”). Avalanche will ramp up from 0 to N Simusers in 30 seconds, where N is the maximum number of users with 0 failed transactions. We disable Phase II ramp-up. We configure Avalanche to generate a sustained load from N active simusers for 120 seconds.

3.5.3 Procedure

A successful session involves 23 URLs:

1. HTTP GET of the welcome screen. This is the screen where users enter name and password.

2. HTTP POST of the username/password.

3. HTTP GET a 1 million byte object from one HTTP server. We configure the Reflector with 0 latency. We configure each interface on the Avalanche to request objects from a different HTTP server.

4. HTTP GET a 1 million byte object from the other HTTP server.

5. Repeat steps 3 and 4 another 9 times. This equates to each user pulling a total of 20 Mbytes for each login.

6. HTTP POST to logout the user.

We use a binary search algorithm to determine the maximum number of users with 0 failed transaction.

During the final 60 seconds of the sustained period, we average the incoming traffic rate from the reflector. and record this as maximum goodput.

For DUTs that support data compression, we run this test twice: Once with compression disabled, and again with compression enabled.

3.5.4 Metrics

Maximum goodput (in bits per second)

3.6 Security Assessment

3.6.1 Objective

To determine the vulnerability of the DUT to known forms of compromise

3.6.2 Configuration

- We uses Nessus 2.0.7 (see section 2 for more details). We run the nessus-update-plugins script on the first day of production testing for any updates. We will use the same set of plugins for all vendors, even if nessus adds to its vulnerability library during testing.

3.6.3 Procedure

We configure Nessus to run a full scan the external interface of the DUT.

We do not necessarily take the Nessus report at face value, as the tool can generate false positives. Wherever possible, we will attempt to disprove false positive reports. We also ask vendors to comment on all vulnerabilities reported by Nessus.