2.1        Test Bed Topology

Figure 1 below shows the basic configuration of equipment to be used in all tests.

Figure 1: The Internet Core Router Test Bed

Each vendor must supply four routers, each equipped with six OC-48c POS interfaces. Each router must support BGP4 routing. We define a “router” as a device that supports BGP4 and contains a unique L2 forwarding database and L3 routing table for each 6 POS interfaces.

Support for MPLS is preferable but not mandatory. As noted in the invitation letter mailed to prospective participants, Light Reading plans to give separate awards for “best BGP device,” “best MPLS device,” and “best overall device.” Nonparticipation in the MPLS tests does not preclude winning awards in the other events.

In addition, vendors must supply any cabling necessary to interconnect their core interfaces. Network Test and/or Spirent Communications will supply cabling to link the tester devices to the routers.

2.2        Device Configuration

2.2.1       Layer-1 Considerations

Vendors must configure edge interfaces as follows:

Framing: Sonet (not SDH)

Rate: OC-48c

Clocking: Router A is the clock source; all other devices use loop timing from Router A

Path signal label: 0xCF

FCS: 16-bit CRC

2.2.2       Layer-2 Considerations

Vendors must configure edge interfaces as follows:

Encapsulation: PPP with HDLC framing (RFC 1662; header = FF 03 00 21)

MRU: 1500 bytes

Maximum configurations: 10

Maximum failures: 5

Maximum terminations: 2

Magic number: 2

Restart timer: 3

Retry count: 1

LCP enabled

2.2.3       Layer-3 Considerations

This test bed models part of the core network for a single service provider; addressing inter-provider connectivity issues is beyond the scope of this project. Accordingly, 1 autonomous system (AS) will be used in the entire test bed.

For all tests except maximum BGP table capacity (section 3.3), vendors should set BGP hold timers to zero (infinite, no keepalives exchanged) to avoid conflicts with offered test traffic.

Vendors must configure the edge interfaces with the following IP addresses:

The prefixes and test traffic to be offered are described in a separate document. This document, an Excel spreadsheet, is available for anonymous FTP here:

ftp://public.networktest.com/LR_router_00/LR_router_prefixes.zip

2.3        Test Equipment

2.3.1       Test Hardware

2.3.1.1       Spirent SmartBits

The principal test instrument for this project is the SmartBits traffic generator/analyzer manufactured by Spirent Communications Inc. (Chatsworth, Calif.). Spirent’s SmartBits 6000 chassis will be equipped with the company’s new POS-6505A SmartModules. The  SmartModule offers line-rate traffic generation and analysis over OC-48c interfaces.

Three OC-48c SmartBits interfaces will be attached to each of four routers under test. Thus, a total of 12 OC-48c SmartBits interfaces will be attached to the system under test.

2.3.1.2       Adtech AX-4000

In addition to the SmartBits, an Adtech AX/4000 Broadband Test System equipped with OC-48c interfaces will be used for troubleshooting and protocol capture and decode purposes.

2.3.1.3       Physical Cables

All interfaces will be connected using single-mode fiber-optic cabling and SC (rectangular) connectors.

3          Test Procedures

For each routine in this section, this document describes:

·        the test objective;

• the configuration to be used;
• the procedure to be used;
• the test metrics to be recorded.

3.1        Baseline IP forwarding, latency, and jitter measurements

3.1.1       Objective

To determine baseline packet loss, latency, and jitter for routed IP traffic

3.1.2       Configuration

--Test bed topology shown in section 2.1

--Total BGP prefixes advertised to the SUT: 201,165 (approximately 2.3 times the 88,500 prefixes shown in core routers on 21 August 2000, according to Telstra)

--No overlapping prefixes

--Prefix lengths will range from /13 to /26, with distribution following Mae-East and Mae-West statistics taken from http://www.merit.edu/ipma/routing_table/#prefix_length     

3.1.3       Procedure

Using the Spirent SmartBits to offer incrementally heavier traffic loads, Network Test will determine the maximum forwarding rate each system under test can sustain with zero packet loss.

Two iterations will be run: One consisting exclusively of 40-byte IP packets[1], and one with a quad-modal distribution of packet sizes.

In the quad-modal distribution, the lengths and distribution of the IP packets are as follows:

The packet length distribution is a composite of 22 samples taken from Merit, a consortium of Michigan-based ISPs, between 28 August 2000 and 13 September 2000. The raw data is available from http://moat.nlanr.net/Datacube. The packet lengths shown here are the four highest medians of all IP packet lengths between 20 and 1,500 bytes.

3.1.4       Test Metrics

Maximum forwarding rate with zero loss (percent of media rate)

Average latency (microseconds)

Average jitter (microseconds)

3.2        Longest-match lookup times

3.2.1       Objective

To determine the impact of route lookups for mixed-length prefixes on device forwarding rates, latency, and jitter

3.2.2       Configuration

--Test bed topology shown in section 2.1

--Total BGP prefixes advertised to the SUT: 251,165 (includes the 201,165 prefixes from the baseline tests (section 3.1), plus 50,000 additional, shorter prefixes to force longest-match lookups

--Overlap of 50,000 prefixes, about 20 percent of total prefixes

3.2.3       Procedure

Network Test will offer the same maximum offered load as in the baseline test (section 3.1).

Network Test will compare results of this test from results from the baseline test and determine any differences in forwarding rates, latency, and jitter.

3.2.4       Test Metrics

Maximum forwarding rate with zero loss (percent of media rate)

Average latency (microseconds)

Average jitter (microseconds)

3.3        Maximum BGP4 table capacity

3.3.1       Objective

This test will determine the maximum number of BGP4 prefixes one router will learn and propagate.

3.3.2       Configuration

--Test bed topology shown in section 2.1, although only routers A and B are used

--Vendors should set BGP hold timers to 30 seconds for this test only

3.3.3       Procedure

The Spirent SmartBits tester will be attached to one interface of routers A and B. The routing tables of the devices under test will be cleared. Then 80,000 /22 prefixes will be advertised by the tester to Router A. The correct learning of all prefixes will be verified on the BGP4 instance running on the SmartBits tester attached to Router B. 

The number of prefixes to be injected will begin at 80,000 and increase in increments of 40,000 until the system under test fails to propagate one or more prefixes. The routing table of the devices under test will be cleared between iterations.

3.3.4       Test Metrics

Maximum prefixes learned

3.4        Performance during route flapping on stable and unstable paths

3.4.1       Objective

This test will determine the effect, if any, on forwarding rate during route flapping.

3.4.2       Configuration

Test bed topology shown in section 2.1.

All devices under test will be configured to run BGP4. The Spirent SmartBits tester will be attached to one interface of routers A and B to generate route flapping and to verify table entries at every flap event.

3.4.3       Procedure

The Spirent SmartBits 6000 will offer 40-byte packets to all POS interfaces in a partially meshed pattern at line rate. Then this procedure will be followed:

Step 1. Advertise 201,165 prefixes to Router A. Each prefix has primary, secondary, and tertiary routes.

Step 2. Withdraw 50,291 primary routes from the routing table. (This is 25 percent of all prefixes.)

Step 3. Verify that traffic is routed over 50,291 secondary routes.

Step 4. Verify that traffic is routed on all other available routes.

Step 5. Re-advertise the original 50,291 primary routes after an interval of 30 seconds.

Step 6. Verify that all traffic is routed over the primary paths.

Step 7. Remove and re-advertise 50,291 routes from the routing table continuously in 30-second intervals for a period of 120 seconds.

The routes removed in each iteration will be noncontiguous entries in the routing table.

3.4.4       Test Metrics

Forwarding rate of stable paths (packets per second)

Forwarding rate of unstable paths (packets per second)

3.5        Baseline MPLS forwarding, latency, and jitter

3.5.1       Objective

To determine baseline packet loss, latency, and jitter for label-switched traffic

3.5.2       Configuration

Test bed topology shown in section 2.1.

The SmartBits tester will initiate nine label-switched paths (LSPs) at each edge interface – one label apiece for each of the nine possible destination interfaces. The nine labels concatenate all the IP prefixes described in section 3.1.3. The total number of LSPs for this test is 108 (12 edge interfaces, each with 9 LSPs).

3.5.3       Procedure

Using the Spirent SmartBits to offer incrementally heavier traffic loads, Network Test will determine the maximum forwarding rate each system under test can sustain with zero packet loss.

This test will be run exclusively with 40-byte IP packets (with packet lengths increased by MPLS labels, of course).

3.5.4       Test Metrics

Maximum forwarding rate with zero loss (percent of media rate)

Average latency (microseconds)

Average jitter (microseconds)

3.6        Maximum LSP capacity

3.6.1       Objective

This test will determine the maximum number of label-switched paths one router can maintain.

3.6.2       Configuration

Test bed topology shown in section 2.1, although only routers A and B are used

3.6.3       Procedure

The Spirent SmartBits tester will be attached to one interface of routers A and B. The forwarding tables of the devices under test will be cleared. Then the SmartBits attached to router A, acting as a label edge router (LER), will offer one label to Router A, establishing a label-switched path (LSP) to a destination interface on Router B. The SmartBits will create primary paths using RSVP-TE (resource reservation protocol-traffic engineering). The SmartBits will continue to initiate LSPs at a low rate – approximately 10 per second – until traffic fails to reach the destination interface on Router B.

3.6.4       Test Metrics

Maximum LSPs maintained