Benchmarking Methodology WG (bmwg) MONDAY, July 15 at 1930-2200 ============================= CHAIR: Kevin Dubray kdubray@juniper.net The meeting was chaired by Marianne Lepp, marianane@lepp.com, who presented Kevin Dubray's apologies. (note taker Elywin Davies) The agenda was examined and no changes were made. AGENDA: 0. Overall Status 2 drafts were in last call: - Firewall draft Authors replied to some comments and last call ended 10 July 2002. Discussion will continue on the list. - BGP control plane convergence Terminology draft No comments received so far. A number of other drafts were on the table for consideration - Diffserv terminology and methodology Jerry Perser presented an update, discussed below. The OSPF drafts , initial , initial There was no action at this meeting. - FIB Methodology Comment was solicited, none received. This draft is on its way to last call. - Resource reservation - Multicast methodology draft Although this draft has expired, and has been around for a long time, the authors would like more comments on it. In particular to consider whether the assumptions made when it was originally written (1996) and now are still valid - This will be taken to the mailing list. Presentations: 1. Benchmarking Network-layer Traffic Control Mechanisms - status update. (Jerry Perser) http://www.ietf.org/internet-drafts/draft-ietf-bmwg-dsmterm-03.txt Changes since last time presented by Jerry Perser. Particular attention to the changes in the definitions of delay vectors, jitter vectors and congestion detection definition. The largest change was in the Sequence Tracking Section where packets could now be classified as 'In Sequence/Out of Order/Duplicate' Option for 'Out of Sequence' is no longer needed. Questions: There was some discussion of the definition of congestion measurement. This involves monitoring the egress stream over a period of time to determine the proportion of input packets that appear. If this period is set too short congestion can be concealed in the DUT. This had been extensively discussed by the authors. JP pointed out that WG had agreed time period of 30 seconds for congestion measurements was suitable to determine how the DUT was behaving whilst not being excessive when multiple test runs were required. A member of the audience thought a longer time period might be needed but after some discussion it was eventually agreed that the definition was appropriate and the time period (not actually in this draft) was appropriate. Jerry noted that when testing devices he normally runs a small number of tests for both 10s and 60s and checks for differences to check if device is concealing congestion on short periods. The meeting agreed that it was ready for WG last call. 2. Rationale and Goals for a proposed, new BMWG work item: "SONET/SDH APS Performance Benchmarking." (Takumi Kimura) TK presented some slides describing how the automatic protection system of a SONET/SDH transport network employed to interconnect two routers could interact with an IP packet flow which it was carrying between the routers when a failure occurred in the ring. Background is that there are various means of using transport nodes such as SONET/SDH rings and RPR, and the recovery timing in the IP layer is different from lower layers (e.g. buffering effects in IP routers and hysteresis on I/F up and down switching). Hence there is a need to compare implementations. Goals for work would be: - Focus on SDH performance - Consider IP layer aspects - Consider defined topologies (pt to pt, Ring, Mesh) - Consider different protection types ( 1+1, 1:n, etc) The proposed work would benchmark a simple arrangement of two routers and two optical add/drop multiplexers with a two leg ring (effectively main and backup paths) interconnecting the OADMs. The intention was to demonstrate the qualitative (eg packet dropping or misordering) and quantitative (timing) effects when the APS was stimulated by breaking one of the fibre connections between the OADMs. The document would specify testing terminology and methodology, concentrating on the IP layer. Questions: JP commented that this was a useful piece of work provided that the effects were measured at the IP layer in the routers connected to the ring. JP did not expect that effects other than packet loss would be seen with the simple test setup but they could occur in principle. Marianne Lepp pointed out that timers are an added consideration, since the outage will be reflected both in the Sonet/SDH and the historesis timer in the Layer 3. It was agreed that the work was potentially useful and TK was asked to refine his proposal, possibly incorporating other types of optical connection, such as RPR, and present at the next WG meeting. 3. Disposition of Benchmarking Terminology for Routers Supporting Resource Reservation. Last spring a WG Last call was issued on the above (expired) draft. NO commentary, pro or con, was issued from this or other related WGs on the I-D. Does this WG really desire this work to die a quiet death? Please retrieve an archived copy of the I-D, read it, and bring a recommendation to Yokohama (or the list.) A cached copy of the document can be found here: http://www.watersprings.org/pub/id/draft-ietf-bmwg-benchres-term-01.txt Jerry Perser expressed enthusiasm that this should move forward. There were no other comments. There was no hard opinion on the future disposition of the document, and this will have to be considered on the list. The meeting closed at approximately 19.50.