High Speed Networking: Open Sourcing our Kernel Bypass Work

Published: 24 April 2018

Andrew Bonney (BEng)
Senior R&D Engineer

Over the past few years, the IP Studio project has been designing and developing solutions to test out the principle of producing broadcast quality content on Internet Protocol (IP) networks. This work has resulted in contributions to industry standards and specifications, but has also accrued a wealth of software which can be used in whole or in part to solve problems in this area. As standards mature and 'off the shelf' implementations become more common, we're publishing some of our software as open source contributions to further aid adoption.

One aspect of our work has covered the challenges of using software to transmit and receive streams of high data rate media over IP networks. This is all the more relevant as the broadcast industry starts to which implement the . The unique challenge here involves handling IP packets (around 1500 bytes each) at data rates of between 1 and 8 Gigabits per second for typical uncompressed High Definition and Ultra High Definition formats. As we discovered, one of the greatest bottlenecks when handling this volume of data in software is the computational expense of copying these packets between different areas of memory, something which has to happen multiple times when using the Linux kernel's standard sockets library. Back in 2015 we discussed our use of on , which is one of several possible tools which allow the transmission and reception of much higher data rates by using 'kernel bypass' techniques. As the name suggests, this mechanism bypasses the operating system's default path for processing packets, providing a more direct route between applications and the network hardware.

Whilst Netmap helps us to reduce the number of copy operations required, we can further enhance performance by interacting directly with the network interface card (NIC) hardware, allowing multiple packets to be passed to the NIC at one time. This could be achieved by interacting directly with any vendor's NIC driver from our applications, but given a desire to prototype with a range of network cards we didn't want to have to write code to interact with each vendor's driver directly. Thankfully, Netmap provides an abstraction layer to enable this. The Netmap codebase includes patches for several vendors' drivers which allow an application to interact with Netmap's API, but under the hood directly pass data to and from the NIC hardware.

Sign up for the IP Studio Insider Newsletter:
Join our mailing list and receive news and updates from our IP Studio team every quarter.
First Name:
Last Name:
Company:
Email:
or Unsubscribe

At the time we started this work we used 10Gb/s NICs which Netmap already had good support for, however given our desire to work with multiple HD or UHD streams we rapidly outgrew the capacity of a single 10Gb/s link. With the advent of 25 and 100Gb/s networking we wanted to see what performance we could achieve on those cards too.

With very few 25 and 100Gb/s network cards available at the time we opted to prototype with the Mellanox ConnectX-4 range. One of the major advantages to this was the availability of an open source network driver, which is bundled as part of the Linux kernel. This permissive open source licence allowed us to patch the driver source directly. Through the development of a patched driver for Netmap we were able to send and receive data rates exceeding .

As our work moves on to other afforded by IP production we wanted to make the results of our efforts available to the wider community, enhancing Netmap's capabilities to operate with a greater range of manufacturers' network cards, and allowing implementers a straightforward path to implement high data rate networking in software without being locked to a single hardware vendor.

These are now available from the , and we're hoping to release further software and patches in the future covering other IP broadcasting requirements.

Thank you to for their support throughout this project, and to the Netmap project for their support in integrating the patches.

大象传媒

Accessibility links

Andrew Bonney (BEng)

Rebuild Page

Useful links

Theme toggler

大象传媒