Date: Fri, 17 Apr 1998 19:14:27 -0400 From: Joel Apisdorf To: hwb@nlanr.net Cc: apisdorf@mci.net, coral-dev@caida.org Subject: OC3MON interface to FORE PCA200e cards from host FORE's "forethought partners adaptor documentation" Governs downloading of code to the card, getting messages from the card, and some of initialization. Also see headers fore/pca200e.h and those it includes for data structures as visible from the host PC. Differences: Normal firmware maintains many queues: commands, buffers to transmit, and 2 each for large and small buffers to receive. For buffers, the host doesn't simply give a pointer to the card -- instead it gives a pointer to a data structure that describes the host-usable (logical, paged) and card-usable (physical) addresses of many packet buffers, and the card is expected to suck that data structure out of host memory using DMA. In order to use the PCI bus most efficiently, OC3MON firmware retains only the command queue, and instead manages blocks with many contiguous cell buffers in them. The host gives the card the physical addresses of pointers to 2 buffer blocks for receive. The number of bytes per buffer block is fixed at initialization time. After a buffer block is filled, the card starts using the second block pointer, sets the 2nd pointer to NULL, and interrupts the host. If the second pointer is NULL, the card drops cells until the host fills it in. Depending upon which firmware you download to the card, OC3MON receive will keep 1, 2, 3, or all cells at the beginning of each packet, and/or the last cell unconditionally. There is also a firmware that copies the ATM header only (not the payload) for all cells. For each cell, there will always be a 16-bit FIFO depth count, followed by a 16-bit timestamp, followed by a 32-bit software expansion of the timestamp. Depending upon initialization-time options, the software expansion may always be zero. When more than one cell for a packet is copied into the host, be aware that they may be interspersed with cells for other VP/VC's, as they were on the physical link being monitored. OC3MON can also transmit, using pointer and length fields for each of 2 buffer blocks. Length is measured in 32-bit words. Each buffer in a block contains a repeat count, ATM header, and ATM payload. Cells in a block are transmitted back-to-back, so the host has explicit control over when idle cells are transmitted and how cells on the various VP/VC's are interspersed. When the current buffer block has been sent, the card starts using the 2nd buffer block, sets the 2nd pointer to NULL, and interrupts the host. If the 2nd pointer was already NULL, the card sends idle cells until the host fills it in. Transmit is presently only implemented in the 1-cell-per-packet firmware. OC3MON firmware normally sets the PMC/Sierra S/UNI-Lite SONET framer chip to do facility loopback, a.k.a. line loopback, so that all cells received by the card will also be transmitted, so that you can inserty an OC3MON into a link without an optical splitter if you don't care about the link dying when the cards are not initialized or powered off. Facility loopback needs to be disabled if you are going to transmit (I think), so a field in the initialization control structure specifies what sort of loopback should be performed. Normal firmware can only handle about 1000 VC's only on VP 0, but OC3MON firmware can handle 2^20 (when only the first cell is copied to the host, or 2^19 if first 2 or 3 cells are copied, or infinite if all cells are copied) VP/VC combinations. It does this by keeping less state (1 or 2 bits) per VP/VC in the same 128 KB of memory. Fields in the initialization block control how many bits of the 20 will come from VCI. If you choose to use less than 16 (all of them), then the least significant bits of VCI will be used. The rest of the 20 always come from the least significant bits of VPI, so if you want all 8 bits of VPI to be usable, you can only allow 12 bits of VCI to be used. Cells which do not match a user-specified (in the initialization block again) value for the unused bits of VCI and VPI, will be dropped. This lets you safely operate an OC3MON which has been told not to use VPI bit 7 (counting lsb as bit 0) on a link with VP's >128 in use, by telling it to assume bit 7 of VPI is either 1 or 0, depending which set of VP's you are interested in. Normal firmware doesn't timestamp arriving cells, but since OC3MON needs to, and the host will not process cells until much later, OC3MON firmware timestamps using a count of 25 MHz ticks since the card was cleared with a (new to OC3MON) command. Note that since there is a 2048-cell FIFO between the SONET framer and the on-card i960 CPU, the timestamp attached may represent the processing time instead of the arrival time, and hence have a coarser granularity, unless the FIFO is empty. ATM fills the SONET payload completely with cells, back-to-back, using idle cells when no user cells are available, so the finiest theoretical granularity of any timestamp attached to an OC3 link is 2.83 microseconds.