Current IP Multicast Development in the Computer Industry

William Sloan

CS193, Internship in Industry

15 December 1998

    I am currently a senior at UCSB majoring in Computer Science and I also have been working part-time as a developer in the local computer industry for three years.  During that time I have worked within the datacommunications and telecommunications fields, first at Stratacom, then cisco Systems, and now for Cadence Design Systems.  I have been with Cadence for a little over a year.  Within Cadence I work for the Broadband Services Business Unit, a group that specializes in design services of data/tele-communications applications and products.
    The Trout project got started because the customer hired us to port Mrouted so they would be able to have a multicast capable router.  When Trout started I was not actively on a project so I was assigned as one of the developers to Trout.  After Trout, I suggested a port of the Trout software be added to the Dorado project and since it was my suggestion, it became my assignment.

    For the past several weeks I have worked on a project code named Dorado.  The project is a continuation of a customer project and involves porting DVMRP, a IP Multicast Routing Protocol, onto a single board computer that is based on a  Motorola MPC860.  The original source code used was UNIX Mrouted 3.8 which was downloaded from the Xerox PARC internet site.  There were several issues to consider in moving the original sources from a UNIX based host to a router platform that contained no file system.  After removing the UNIX dependencies, the software had to be grafted into our customer's router. Instrumentation had to be added to conform with the new SNMP MIBs that the customer required for remote management, which are not supported by Mrouted.  The final stage of this project (Dorado) involved porting Mrouted from the customer's router to the internal develop system, using what was learned in the original port.

BACKGROUND:  This Dorado project grew out of another project that I worked on called Trout.  The goal of Trout was to port the UNIX Mrouted to a customer router based on the Intel i960 using VxWorks 5.1.

    When the project was started, it was discovered that the customer was just using VxWorks as a scheduler and that they had their own implementation of the various protocol stacks.  This created two main issues that had to be overcome in the port.  Other issues which arose were the need for additional configuration and management functionality, and the removal of UNIX-centric programming practices in the Mrouted code.
    First, since the customer's router had no file system, the way that Mrouted expected to be started and get its configuration had to be changed.  Instead of retrieving its' configuration from a file, Mrouted was modified to use a set of function calls that could query the customer's router to get its configuration.
    The new platform had its own networking code, which did not include socket calls, get and set socket options, or ioctl functionality. Mrouted  had to be changed to call other existing library functions,  or ones that were implemented along with the port, to perform kernel related functions.
    There has been substantial work done on MIB's for DVMRP.  The customer insisted that the port complied with these new MIBs.  This meant that a new chunk of instrumentation in the way of SNMP sets and gets had to be added.  There was additional instrumentation that had to be added to allow for remote administration of Mrouted, to include being able to dynamically turn on and off the IP Multicast routing on a per port basis.
    One of the biggest problems in porting the code turned out to be the way memory was allocated.  The original Mrouted allocated all static data structures on the stack and only used heap memory for dynamic memory.  While this is perfectly valid for software running on a workstation it creates problems inside of an embedded world.  Inside of the of embedded OS that the customer had each task had to declare what it stack space would be.  Knowing that there was significant processing going on inside of Mrouted, a stack space of 4KB was allocated to ensure that the stack would not be overrun.  This would normally be plenty of room except for the fact that all these static data structures were allocated.  The two that really stand out was the static transmit and receive buffers.  Each of these buffers were 8KB long and in global space.  Just these two buffers comprised four times our allocated stack space.  This was not noticed until some random behavior was observed, which is a tell-tale sign of a stack overrun, and the resulting investigation uncovered these data structures.  After moving the data structures off of the stack and onto the heap, the software no longer behaved randomly.  This just illustrates one of the complication of programming in an embedded environment.

DORADO:  After the completion of the Trout project, it was decided by management that IP Multicast functionality would be something that customers in the future would want and that the new Mrouted code would be a good starting point

    It was decided that the Trout software would be ported onto our development platform. and in this effort an abstraction layer would be added to allow this next port to be as independent of the operating system as possible.  There is also an internal standard that the code had to comply with.  This internal standard included using SystemV style mblk's as the buffer abstraction and that all protocol stacks would use a stream interface (for a discussion on streams, see
    The internal standard came in handy during this project because these were two places that were very dependent on the customer's code during the Trout project.  It is impossible to make functionality like Mrouted completely independent of the underlying operating system, but since the Dorado port was moving onto a system with VxWorks 5.3 running, it was handy because VxWorks also uses mblks as the abstraction for its buffers.  This allowed the software to fit nicely into VxWorks and also allow it to conform to the internal standard and thus be mostly OS independent.
    The other piece of the Trout code that had to be changed was the configuration management.  There were several places in which the code expected to have access to data structures that our customer had, and that do not exists in the scope of Dorado.  Most of these data structures included Task Control Blocks, Queue Control Blocks, and Port Control Blocks.  Luckily this turns out not to be much of a problem because most of the configuration routines defined an API that the customer's software used to tell the code what to do.  After replacing the customer's data structures with VxWorks data structures, the configuration routines have most of the data that they need and now can be accessed through the VxWorks shell.  This will have to be the extent of the interface to the Dorado Mrouted until Dorado has a user interface made for it.

DOCUMENTATION:  The documentation for Dorado consists of a design specification and a test plan.  Both of these documents were based on the Trout documentation.  The design specification for Trout was mostly a reiteration of the RFCs that covered DVMRP, IGMP, and the required MIB support.  However, there was a major portion of the design specification that related how Mrouted was going to be integrated into the customers router product.
    There were three issues in the design specification that had to be changed when the code was moved from Trout to Dorado.  First was how the code was going to get packets from the network.  Second was how the code was going to send packets onto the network, and lastly was how the code was going to fit into the operating system.  Of course these three items where also the only places in the Trout design specification were it was not just a reiteration of the relevant RFC's.  These three areas of the Dorado design specification now read that Mrouted will interface with the network via a streams interface, as required, and that the code will get its configuration from someone calling functions from the VxWorks shell prompt.  The test plan was much easier to write since the basic Mrouted functionality has not changed and we were able to use the original Trout test plan.

DEVELOPMENT ENVIRONMENT:  The development environment during the Trout project is worth noting.  In accordance with our customer's wishes, we choose Linux as the development environment.  This mirrors how the customer does development and had a known good tool chain for Linux.

    The Trout team commendeered a PC from one of the labs and replaced Windows with Linux.  This PC became the file server, compile server, and the NFS server.  The Linux PC kept all of the customer's supporting source code along with all of the code that was written/ported for Mrouted with RCS being used for source code control management.  Since there was only a Linux tool chain, all compiling during the development cycle was handled by this PC.  Eventually a Solaris version of the tool chain was delivered by the customer, but it was never installed or tested.  The customer's router was able to download a runable image in RAM to execute via NFS.  Once again the Linux PC came through by pulling triple duty as the NFS server for the three routers that the customer had provided.  At one point in time there was five developers working full time on this PC and it kept a sustained load average about 0.2 - 0.5 and at its maximum was up for 40 days before a power outage forced it to shutdown.  Since Dorado is using a MPC860 and not an i960, the development environment has moved back to Solaris since that is where the tool chain for the MPC860 lives.  The source code control has also migrated from RCS to Clearcase, which is made by Atria Software.

CONCLUSION:  Being involved with Trout and Dorado, I learned several things about being a software engineer in the current computer industry.  On the technical side I learned how the MBONE works and how DVMRP is used to route IP multicast packets across the internet.

    In conjunction with DVMRP, I received good exposure to IGMP version 1 and 2.  This was my first experience writing a software design specification which was at first a daunting task but in retrospect it was not that difficult.  Also this was my first experience in porting software of any notable size.  While it was tempting for me to rewrite parts of Mrouted that I felt were not done correctly there simply was not enough time in the schedule to allow that to happen.  In the end this probably was a better approach because the less that was changed, the greater the chance that it would correctly inter operate with other Mrouted running on UNIX workstations.  After spending the last 18 weeks working with this technology, I have come to the realization that IP multicast has a big part to play in the evolution of the internet and I am glad I have gotten this exposure so early in my career.