Network World Lab Test: Wireless
LAN Switches
Scheduled for Publication in September 2003
Test Methodology
v.
2.0 Copyright Ó 2003 by Network Test.
Vendors are encouraged to comment on this document and any other aspect of test
methodology. Network Test reserves the right to change the parameters of this
test at any time
Please
send comments to David Newman of Network Test (dnewman at networktest.com)
This
document describes the test procedures we use to assess the emerging class of
wireless LAN switches.
We plan tests of device capabilities on four areas:
For the provisioning and management tests, we ask vendors to complete a “mini-RFP.” Written responses and a walk-through of system features will help determining scoring in this area.
This document is organized as follows. This section introduces the project. Section 2 describes product requirements. Section 3 describes test procedures. Section 4 logs changes to this document.
Participating vendors must supply the following:
One wireless LAN switch
At least four 802.11b access points
Provisioning and management software, if necessary
We ask each participating vendor to prepare a written response to the following mini-RFP. We also ask each vendor to walk us through the salient features of its proposal.
While we do use subjective scoring in this section, we also gather a large amount of data to compare system feature sets. Preparing a product taxonomy is a major goal of this project.
Ohshutup.com is a leader in cell-phone jamming technology with more than 20,000 employees around the world. The company is opening a small branch location in an office park in Westlake Village, CA. This 1,200-square-foot office suite will accommodate members of the company’s research, marketing, and finance teams.
Each of the three groups requires aggregate bandwidth of at least 5 Mbit/s. For security policy reasons, Ohshutup.com requires that different groups’ traffic run on physically diverse networks wherever possible. Separate access point(s) for each group may be desirable.
This policy of separation of workgroups extends to the company’s wired infrastructure as well. Branch sites (as well as departments or floors at headquarters) use layer-2 workgroup switches with 802.1q VLAN tagging to identify workgroups. The VLAN tags in turn are mapped to different IP subnets on aggregation switch/routers, with different IP subnets for each division.
The company currently uses internal firewalls between its various departments. If possible, traffic from wireless users should be subject to the same firewall rules as other users.
The company also uses IPSec gateways to authenticate and encrypt traffic from telecommuters and road warriors. IPSec VPN access for wireless users is highly desirable, if available. In addition, the company currently uses a Windows 2000 Radius server for layer-2 authentication of users, and this server can easily be upgraded to allow for 802.1x PEAP authentication.
Whether these security features are implemented with a new or existing firewall and/or VPN gateway is unimportant.
Unauthorized access is another security concern. Ohshutup.com’s employees may attempt to deploy unauthorized access points, and other companies in the office park may have their own WLAN equipment. For both reasons, rogue access point detection and adaptation are both highly desirable.
Although the company uses its network only for data traffic today, it is seriously considering adaptation of voice-over-IP telephony. The WLAN infrastructure should demonstrate the ability to prioritize voice traffic and, if possible, enforce latency and jitter boundaries.
Ohshutup.com relies on Openview to manage virtually any device that supports SNMP. It expects to be able to do the same with all WLAN infrastructure, including clients, access points, and switches.
Ohshutup.com’s current fleet of notebook computers and PDAs are equipped with 802.11b radios. The company is very interested in migrating to 802.11a and/or 802.11g within one to two years. However, depreciation schedules for existing equipment dictate that 802.11b connectivity is a must for the near term.
Please demonstrate how your product will address each of these requirements and provide an itemized cost breakout for the total solution.
We assess performance with measurements of forwarding, delay, jitter, and failover delay.
We use Netwarrior, a Linux-based traffic generator/analyzer from QOSmetrix, to send traffic from one server on a wired segment to client(s) on a wireless segment.
The Netwarrior machines are equipped with GPS and special software to ensure precise measurements of delay and jitter. Timestamp resolution is accurate to within 40 nanoseconds.
The Netwarrior systems run Red Hat Linux (2.4.20 kernel). They are equipped with Netgear MA311 cards and the linux-wlan-ng driver.
We test performance in four configurations:
We run four tests: UDP unidirectional and bidirectional, and TCP unidirectional and bidirectional. In all cases, the test duration is 300 seconds and the IP packet length is 40 bytes. We report forwarding rates for the client station.
We repeat the same tests from the single AP scenario, this time using 3 APs in relatively close proximity. The APs should use different channels (e.g., 1, 6, and 11).
We repeat the same tests from the single AP scenario but this time using 4 APs in relatively close proximity. The first two APs use different channels but the third and fourth APs will have to share a channel. We report forwarding rates for all stations. (Note: All 4 APs will operate for at least 300 seconds before we offer data traffic to give the systems under test a chance to adjust signal levels.)
We repeat the same tests from the single AP scenario, this time using 3 authorized APs in relatively close proximity. The authorized APs should use different channels (e.g., 1, 6, and 11), but one of them must contend for bandwidth with a rogue AP. We report forwarding rates for all stations.
(Note: All 4 APs will operate for at least 300 seconds before we offer data traffic to give the systems under test a chance to adjust signal levels.)
Using the Airopeek NX and RF Grabber analyzers from Wildpackets Inc., we measure authentication and association times for a single client using 802.1x PEAP authentication. A single CA/Radius server running Windows 2000 Advanced Server authenticates users. This machine also acts as a DHCP server.
To isolate the processing time of the WLAN switch itself, we measure only downstream delays. This ensures any processing time for authentication on the server is not charged to the system under test.
A client associates to one of three APs attached to a switch. (In this scenario, the system under test must be configured to allow clients to roam from one AP to another). We use Netperf to offer a unidirectional stream of UDP packets to the client’s IP address.
Once traffic flow begins, we physically disconnect the AP to which the client is associated from the switch. The client should re-associate with a different AP after some interval. We derive the length of the interval from packet loss statistics. We also determine what the SUT’s management tools say about user roaming and AP status.
To assess system security, we use FakeAP, an open-source denial-of-service attack tool that mimics tens of thousands of rogue access points. More information about FakeAP is available here:
http://www.blackalchemy.to/project/fakeap/
Note that FakeAP requires Linux, the hostap driver, and a card with a Prism 2, 2.5, or 3.0 chipset. For this test we use SMC’s SMC2532W-B card, which is an OEM’d version of the Zcomax XI-325.
Version: 2.0
Date: 27 July 2003
Added reference to QOSmetrix Netwarrior
Deleted Netperf as traffic generator
Version: 1.1
Date: 30 June 2003
Initial vendor release
Section 3.4: Added card and driver info for hostap
Version: 1.0
Date: 16 June 2003
Initial public release to Network World
