Evaluation of the HTB hysteresis patches on a real-life production system.

For context see netdev email discussion, titled: "HTB scheduler HTB_HYSTERESIS modifications".
[PATCH 1/2] HTB scheduler, change default hysteresis mode to off.
[PATCH 2/2] Change HTB_HYSTERESIS to a runtime parameter htb_hysteresis.

We saw an increased number of interrupts on the test system, when enabling/disabling htb_hysteresis.
This didn't show up on the production system.

# Enable:
echo 1 > /sys/module/sch_htb/parameters/htb_hysteresis

# Disable:
echo 0 > /sys/module/sch_htb/parameters/htb_hysteresis

I believe that the increased intr (on test system) is an artifact of the TCP download got less bursty and more smooth.

Kernel on Production host

A new kernel 2.6.25.4 with my HTB hysteresis patches, and default htb_hysteresis off, were booted (tuesdag 10/6-2008) on the production host (internal name BCU24). Previous kernel 2.6.15. The production host is power full machine, a HP DL380 dual Xeon (3Ghz, 2Mb Cache, ht).

Interrupts

There is no real change in the interrupt load, see interrupts image:

The machine has a simple HTB setup on eth0 and eth1 each with 610 HTB class leafs under the root node.
It also has quite a lot of iptables rules (11405 rules and 3015 chains).

When changing htb_hysteresis runtime, there where not measurable changes in interrupts.

CPU usage

Surprisingly the CPU usage decreased significanly, when upgrading to kernel 2.6.25.4 (from kernel 2.6.15).
I have not Oprofiled, but from previous experiences, I believe that its primarily caused by Patrick McHardys RCU patches to netfilter conntrack, Thanks!!! :-).

Only looking at the system and softirq load:

When changing htb_hysteresis runtime, there where not measurable changes in CPU usage.

Traffic Load

The traffic and packets/sec load didn't changed significantly over the period.

Route Cache

The route cache behavior is also much better in kernel 2.6.25.
Actually I believe most of it got corrected in kernel 2.6.24, by Eric Dumazet, Thanks!

Author of this document

http://people.netfilter.org/hawk/patches/htb_nohyst/production_test_results.html

Name: Jesper Dangaard Brouer
Email: hawk (at) diku (dot) dk
Author of http://adsl-optimizer.dk
LinkedIn:
http://www.linkedin.com/in/brouer