TCP multi-flows probe -- a tcp_probe extension

I modified the tcp_probe kernel module to help study multiple flows' some TCP stack status. I called it "tcp_m_probe". This code was originally used to help research on TCP loss synchronization over buffer-overflowed bottleneck of high-speed networks. Since it is a kernel module when the kernel is compiled with kernel probes (called kprobes) that provide hooks to kernel functions, the performance is really good. I have tested my version under 1Gbps and 10Gbps networks, see my small benchmark bellow.

I think it is generally helpful for anyone who is interested in Linux network stack of TCP or other layers. It is free under "GNU General Public License". 

 This link contains my "tcp_probe_ccui.c" and the Makefile. Additionally, I also attach a shell script "runIperf.sh" which uses Iperf as a traffic generator to trigger the function of tcp_probe_ccui. The format of dumped data file (${Protocol}${Counts}flows${Destination}.$index.tsv) is similar to the original tcp_probe. The data file is a database which contains the source port number as the key for each records.

Any questions, please leave a comment and I will respond as fast as I can. Have fun!

Figure of 6 TCP-CUBIC flows' congestion window behaviors as a function of time through a 1Gbps bottleneck link, MTU==1500 bytes.

Sniffing 10 TCP-CUBIC flows: CPU and storage cost benchmark between tcpdump and tcp_probe_ccui: tcpdump only records 100 bytes of header in each packet, tcp_probe_ccui only records when cwnd changes.

some interesting but true understanding about TCP

It's about TCP congestion control.
http://blog.leith.ie/some-ways-to-break-tcp/

try this link:

How to Enable Oracle Java in Your Web Browsers on Ubuntu Linux

case study: irqbalance is not useful for AMD SMP

The original usage of IRQBALANCE is for Intel SMP (symmetric multi-processing CPU) that distributes attendant interrupts (IRQ) over the processors and cores. It is designed for finding a balance between power savings and optimal performance. Obviously, it is developed for Intel SMP at the moment of 2006 (you can find the article here), a time when Intel quad-core CPU has double group of two-cores sharing a single level 2 (L2) cache.

However, time always changes and IT industry flies faster. When AMD released its third generation  SMP CPU in 2007, take quad-core Opteron™ for example, there is no shared L2 cache. All L1 and L2 caches are dedicated to reduce cache pollution, while they are sharing a large L3 cache for buffering main memory.

So IRQBALANCE is obsolete. Its latest release code is irqbalance-0.56, which is published at the same period in 2006. At old time, system may get some benefits from balancing IRQ distributed among groups of cache domains. Now, with dedicated L2 cache in AMD SMP, IRQ balancing will dampen performance, especially for Linux network stack of large steaming packets, where threading performance is much more critical than load balancing. Because each L2 cache is a separated one, switching IRQ between cores even between multi-way CPU sockets will increase cache missing rates. As a result, performance for handling SFB IRQ is decreased. On the other hand, if the same CPU handles the same IRQ almost all the time, the IRQ handler function will unlikely leave the CPU cache, boosting the kernel performance.

There are some complains about IRQBALANCE. Here, a Chinese post shows an example: <irqbalance服务可能拖累双核、多核cpu>. Also another example is here: 
<Improving desktop performance in Fedora 14 by disabling irqbalance>.

To disable IRQBALANCE is quite easy. In Ubuntu, it is as follows:
ubuntu: set ENABLED=0 in /etc/default/irqbalance or sudo apt-get purge irqbalance


In Fedora, it is as follows:
fedora: service irqbalance stop or chkconfig irqbalance off or yum remove -y irqbalance

Ubuntu & Debian Ramdisk for JVM

Ubuntu 12.04 – install sun jdk 6-7

Ubuntu 12.04 – install sun jdk 6-7 A very helpful artical. 

Ubuntu 12.04: install Grub2 on a separated partition, not MBR

I am not talking about repairing Grub2 inside Ubuntu 12.04, I am talking about install Grub2 on a separated partition, not on Master Boot Record (MBR). The command line is simple, but not like what I used before:


option 1, not work:

testuser@pc:~$ sudo grub-install /dev/sda1
/usr/sbin/grub-setup: warn: Attempting to install GRUB to a partitionless disk or to a partition.  This is a BAD idea..
/usr/sbin/grub-setup: warn: Embedding is not possible.  GRUB can only be installed in this setup by using blocklists.  However, blocklists are UNRELIABLE and their use is discouraged..
/usr/sbin/grub-setup: error: will not proceed with blocklists.


option 2, not work:

testuser@pc:~$ sudo mount /dev/sda1 /mnt
testuser@pc:~$ sudo grub-install --boot-directory=/mnt/boot /dev/sda1
/usr/sbin/grub-setup: warn: Attempting to install GRUB to a partitionless disk or to a partition. This is a BAD idea..
/usr/sbin/grub-setup: warn: Embedding is not possible. GRUB can only be installed in this setup by using blocklists. However, blocklists are UNRELIABLE and their use is discouraged..
/usr/sbin/grub-setup: error: will not proceed with blocklists.


option 3, not work:

testuser@pc:~$ sudo mount /dev/sda1 /mnt
testuser@pc:~$ sudo grub-install --force --boot-directory=/mnt/boot /dev/sda1
/usr/sbin/grub-setup: warn: Attempting to install GRUB to a partitionless disk or to a partition. This is a BAD idea..
/usr/sbin/grub-setup: warn: Embedding is not possible. GRUB can only be installed in this setup by using blocklists. However, blocklists are UNRELIABLE and their use is discouraged..
/usr/sbin/grub-setup: error: cannot read `/mnt/boot/grub/core.img' correctly.

option 4, works!

testuser@pc:~$ sudo grub-install --force /dev/sda1
/usr/sbin/grub-setup: warn: Attempting to install GRUB to a partitionless disk or to a partition. This is a BAD idea..
/usr/sbin/grub-setup: warn: Embedding is not possible. GRUB can only be installed in this setup by using blocklists. However, blocklists are UNRELIABLE and their use is discouraged..
Installation finished. No error reported.