Discover a remarkable new way to Benchmark SDN Controllers

Enterprises and Service Providers are planning transition towards SDN based architecture which enables them to create agile, responsive and easily manageable network infrastructure. With SDN architecture, the network control has been decoupled and centralized in the SDN controller for meeting the above needs. However, the key question everyone has in their mind is whether the SDN network guarantees the performance as that of a legacy network.
One of the key factors to guarantee the SDN network performance is ensuring SDN controller’s performance. But there is a lack of tools to comprehensively benchmark the SDN controller.

Following are some of the key performance parameters to be bench-marked for a SDN controller.

• How many flow requests a controller can process per second and how long it takes to process a flow request ?
• What is the impact on controller’s performance for various applications traffic ?
• What is the optimum network configuration that the controller performs better ?
• How much time does a controller take to provision an end-end flow ?
• Ability of the controller to adapt towards network disturbances ?

Cbench is one among the open-source tool widely being used to measure the performance of SDN controller. However, to ensure reliable deployment of SDN controller, will a tool like Cbench be able to address these requirements?
This post analyzes Cbench’s shortcomings and limitations in meeting some of the above requirements and proposes an alternate tool called PktBlaster SDN Controller Test, which will help in meeting the above requirements.

We conducted some performance tests in our lab with OpenMuL controller using both Cbench and PktBlaster SDN Controller Test. We have provided below a sample performance test result for the test conducted.

PktBlaster SDN Controller Test – Result

One of the shortcomings that we observed with Cbench is the accuracy of performance test results. On analyzing their logs, we found that Cbench is reporting the aggregate value of all OpenFlow responses received instead of packet-out responses.

Secondly, when a performance problem is detected, it is difficult to narrow down whether the controller had not processed the received flow requests or whether Cbench had not sufficiently generated sufficient flow requests.

Thirdly, Controller’s performance is heavily influenced by the nature of traffic and their payload size. But Cbench supports only fixed traffic type (IP) and fixed size (64 bytes).

With Cbench, it is difficult to correlate the controller’s performance across various test configurations to determine the optimal configuration for which the controller performs best. This is now possible with the Comprehensive Reporting and Analysis feature of PktBlaster SDN Controller Test.

Realizing the need for a comprehensive controller testing solution, Veryx has developed PktBlaster SDN Controller Test Lite – the Free Version for performance measurement and PktBlaster SDN Controller Test Premium – the Commercial Version for Performance testing and SDN Network emulation that would help the industry to accelerate their testing and deployment cycles.