Sneaky refcount

Sat, 05 Aug 2006 02:00:00 CEST

I've finally got qemu working as I want so I can start to test my RCU patches without having to reboot my laptop all the time. I'm having a weird refcount problem with the L3/L4 RCU patch, I sometimes get a conntrack entry that has an elevated refcount which never drops down to zero. Very annoying.

Steps to reproduce: ping the testmachine and run while true; do rmmod nf_conntrack_ipv4 ; modprobe nf_conntrack_ipv4; done on the testmachine

This results in the rmmod not returning and an entry with use=1 in /proc/net/nf_conntrack_deleted. All entries to be deleted as a result of the forced kill are added to this linked-list after beeing removed from the hashtable, and we wait until they are all dead since they contain pointers to the L3/L4 protocol handlers that we are about to unload. They die and get removed from this deleted linked-list when the refcount, use, drops to zero. But this never happens. And the counters of the entry always say that there's only been traffic in the ORIG direction, and since it was an icmp echo-request and that 'ping' reports that it has received all reponses we know that the packet must have passed through the stack properly which should have decreased the refcount when the packet was kfree_skb()'d. I've seen it with tcp packets as well, but it's easier to verify that all packets got through with icmp.

We have three diffrent "users" of refcounts. The timer for the conntrack entry holds one reference. Each packet passing through the conntrack infrastructure holds one reference. And we sometimed manually increase and decrease the refcount when we want to force an entry to stick around for an extended period of time (some of these forced refcount increases/decreases might go away with the use of RCU for the actual entries later, but that's another story).

Testing is performed on an UP kernel without preemption and without the -rt patches which means that there's no preemption of the softirq going on. Everything should be serialized and pretty but something somewhere increases the refcount without decreasing it, the L3/L4 patch decreases the refcount when it forcibly kills the timer for the entry so that should be ok.

Maybe I'll see more clearly after a beer...