This note summarizes the results of maintenance across the entire ExoGENI Testbed in April 2014.
- Minor fixes added to BEN to better support inter-domain topologies
- UDP performance issue addressed. See below for more details.
- FOAM updated across the racks
- Floodlight updated across the racks
- most of the connectivity caveats from the previous note still apply.
- UvA rack is currently not reachable. We suspect a problem with the configuration in SURFnet, which we will address.
- This behavior appears to have resolved itself by 05/06 without our intervention. Please report any further problems.
The details: UDP performance
Some of you have observed very poor performance for UDP transfers – extremely high packet losses and very low transfer rates. This issue was traced to three separate causes:
- Poor implementation of “learning switch” functionality in the version of Floodlight OpenFlow controller we were using. It resulted in sudden losses of packets after a period of time, particularly between bare-metal nodes. To resolve this issue we upgraded Floodlight to version 0.9 and replaced the “learning switch” module in it with a better-behaved “forwarding” module.
- Insufficient configuration of the QEMU interface to the guest VM, which resulted in very high packet losses. We updated the Quantum agent to support the proper options.
- Sensitivity of UDP transfers to host- and guest-side transmit and receive buffers. We tuned the host-side buffers on worker nodes, however the tuning guest-side must be accomplished by the experimenter.
To explain further how to get the best performance out of UDP on ExoGENI we will publish a separate blog entry in the immediate future.