Experiments were performed to investigate the performance of SCTP in our transatlantic gigabit network, to examine the viability of SCTP as an alternative to TCP. We studied the performance of the only two Linux implementations of SCTP that were publicly available at the time this study was performed: LKSCTP and OpenSS7. Our work on OpenSS7 is still ongoing, so only our results with LKSCTP are presented here.
Linux Kernel SCTP (LKSCTP)
The LKSCTP implementation of SCTP runs in kernel space. For our tests, we used Linux kernel 2.5.65 and lksctp-2_5_65-0_6_8.
Detailed information on LKSCTP can be found on the Web site of the LKSCTP project. According to Randall Stewart (co-author of SCTP), the version of LKSCTP that we tested is not completely compliant with RFC 2960 and the current Implementer's Guide, and is not optimized for performance. The latter was confirmed by Jon Grimm, from the LKSCTP project.
Loading the SCTP modules is done with: /sbin/modprobe -a sctp
Test tools
A version of iperf-1.65 was modified to support the LKSCTP implementation of SCTP. An SCTP option "-z" has been added to allow SCTP to be set as the transport protocol, the usual TCP and UDP support are provided in the standard way. This new version of iperf was then used to carry out the benchmarking of SCTP. Here is some sample output from the newly modified Iperf:
Server:
[asim@w02chi iperf-1.6.5-hb-lksctp2]$ ./iperf -s -w 32M -z
sctp selected
In SCTP mode
------------------------------------------------------------
Server listening on SCTP port 5001
TCP window size: 61.8 MByte (WARNING: requested 32.0 MByte)
------------------------------------------------------------
[ 6] local 0.0.0.0 port 5001 connected with 192.91.239.2 port 128
[ ID] Interval
Transfer Bandwidth
[ 6] 0.0-60.0 sec 383 MBytes 52.3 Mbits/sec
Client:
[asim@w02gva iperf-1.6.5-hb-lksctp2]$ ./iperf -c 192.91.236.2 -w 32M -t 60
-i 10 -z
sctp selected
In SCTP mode
------------------------------------------------------------
Client connecting to 192.91.236.2, SCTP port 5001
TCP window size: 61.8 MByte (WARNING: requested 32.0 MByte)
------------------------------------------------------------
[ 3] local 0.0.0.0 port 32768 connected with 0.0.0.0 port 32768
[ ID] Interval Transfer
Bandwidth
[ 3] 0.0-10.1 sec 64.0 MBytes 53.2 Mbits/sec
[ 3] 10.1-20.0 sec 64.2 MBytes 54.1 Mbits/sec
[ 3] 20.0-30.0 sec 63.6 MBytes 53.5 Mbits/sec
[ 3] 30.0-40.0 sec 63.8 MBytes 53.5 Mbits/sec
[ 3] 40.0-50.0 sec 63.7 MBytes 53.3 Mbits/sec
[ 3] 50.0-60.0 sec 63.6 MBytes 53.3 Mbits/sec
[ 3] 0.0-60.3 sec 383 MBytes 53.3 Mbits/sec
As can be seen, iperf gives all the relevant information needed for performance tests, i.e. bandwidth and data transferred. Its output is easy to read.
Equipment
The DataTAG transatlantic gigabit network extends from Geneva to Chicago, with a capacity of 2 Gbit/s at the time our measurements were carried out. On both sides of the link are two dual processor Intel Pentium 4 machines, which are the testbed machines used to make the data transfers. The transatlantic link is composed of two gigabit channels, we used one of these channels for the tests, giving a wire speed of 1 Gbit/s.
Before performing our tests we had to set the txqueuelen with: /sbin/ifconfig eth0 txqueuelen 10000
Before all new connections we had to perform a "flush": sysctl -w net.ipv4.route.flush=1
Results
Performance tests were first done for SCTP and then TCP to be able to make a fair comparison between the two protocols. We took results for short, medium and long time intervals (10 seconds to 3600 seconds).
| LKSCTP | SCTP | TCP | ||
| Time | Transfer (Mbytes) | Bandwidth (Mbit/s) | Transfer (Mbytes) | Bandwidth (Mbit/s) |
| 10 | 63.90 |
52.30 |
921.00 | 777.00 |
| 20 | 127.00 | 53.00 | 2060.00 | 882.00 |
| 30 | 138.00 | 38.60 | 2480.00 | 705.00 |
| 40 | 177.00 | 37.00 | 4190.00 | 899.00 |
| 50 | 308.00 | 51.30 | 5340.00 | 918.00 |
| 60 | 383.00 | 53.30 | 6420.00 | 917.00 |
| 70 | 351.00 | 42.00 | 7520.00 | 922.00 |
| 80 | 511.00 | 53.40 | 8610.00 | 925.00 |
| 90 | 552.00 | 51.50 | 9650.00 | 921.00 |
| 100 | 640.00 | 53.50 | 10800.00 | 930.00 |
| 200 | 863.00 | 36.20 | 21700.00 | 930.00 |
| 300 | 1490.00 | 42.60 | 32500.00 | 801.00 |
| 400 | 2420.00 | 52.00 | 37100.00 | 796.00 |
| 500 | 2460.00 | 42.20 | 49000.00 | 842.00 |
| 600 | 2990.00 | 42.80 | 47200.00 | 676.00 |
| 1200 | 6810.00 | 48.80 | 57000.00 | 408.00 |
| 1800 | 8960.00 | 42.70 | 63600.00 | 303.00 |
| 2400 | 12500.00 | 44.60 | 90500.00 | 324.00 |
| 3000 | 15700.00 | 45.00 | 125000.00 | 357.00 |
| 3600 | 18740.00 | 44.25 | 152000.00 | 363.00 |
Below we are some graphs of the above data
Graph 1.0 - SCTP & TCP Bandwidth (Short Time Intervals)
TCP outperforms SCTP significantly.
Graph 1.1 - SCTP & TCP Bandwidth (Long Time Intervals)
Graph 1.2 - SCTP & TCP Data Transfer (Short Time Intervals)
Graph 1.3 - SCTP & TCP Data Transfer (Long Time Intervals)
Graph 1.4 - Bandwidth over Entire Time Period
Graph 1.5 - Data Transfer over Entire Time Period
The TCP bandwidth drops for long time intervals, however still maintaining a sizeable gap over SCTP.
Data Analysis
It is evident from these results that TCP provides a significantly higher bandwidth than SCTP, which at times only reaches about 5% of the TCP bandwidth. The tests show that this performance gap exists across a range of various time intervals.
I suppose there are a few ways we can explain this
behaviour:
Verification of Results
Jon Grimm (from the LKSCTP
project) verified that my results were roughly of the correct order of
magnitude. Their
initial performance tests were done on different
equipment, so not directly comparable, but good enough to check I was not way off
the mark. Their tests were performed on 100 Mbit/s networks with short round trip
times (RTTs). My results were gathered with a 1 Gbit/s network and an RTT of 120
ms.
The LKSCTP project holds great potential and it is probably just a matter of time before the performance gap is closed and SCTP becomes a real contender with TCP. The results I obtained are not the theoretical limit of SCTP: they simply represent a snapshot of its current progress.
My performance tests of SCTP show that it is not yet ready to replace TCP. Consequently, the deployment of SCTP in the DataTAG testbed seems a bit premature at this stage.
Future work in this area could include a variety of options:
 | Port a BSD implementation of SCTP to Linux: for instance, some performance work was carried out in KAME; this version of SCTP was created by Randall Stewart and includes HighSpeed SCTP, a port of Sally Floyd's HighSpeed TCP. |
| Test newer versions of LKSCTP: The LKSCTP project regularly releases new versions of their SCTP code; it would be interesting to track their progress. |
Disclaimer: The results and opinions given here are based on the results obtained during Asim's experiments. These experiments are only trial tests and therefore subject to error.