= Netfilter's list of ideas for students participanting in GSoC 2019 = This document provides an introduction to the students that are willing to apply to this round of Netfilter's GSoC 2019. Please, take the time to read before you make questions. == Introduction == In this edition, we propose that the students focus again on the nftables [1] project, the successor of the popular iptables [2] firewalling tool. There is a huge ongoing development effort to get nftables into production state. We believe that GSoC students will help to boost this development, more specifically on tasks that are relatively simple but time consuming and that really need to be done. == Prerequisites == General requirements for students to participate are: * must know C, writing code fluently. * computer networking at a good level, more specifically, layer 2/3/4 of TCP/IP stack. == Proposed tasks == We propose several tasks for GSoC students in the next sections, this year we provide mostly tasks that range from average to hard in terms of difficulty. All these tasks also involve helping in bug hunting and fixing. We added a tag to the classify proposed tasks following the description available in GSoC's "Defining a Project (Ideas List)" [6]. == Task 1: Implement missing features in nftables == * Description: As of Linux kernel 4.20, nftables provides around 90% of the iptables feature-set [5]. We believe that this is fundamental to help users to migrate to nftables. We describe a list of possible subtasks for reference. * Tag: Core development. * Tasks: Help by implementing missing features available in iptables as matches/targets. * Level of difficulty: Average. There is already code that you can use as reference for this task. * Mentors: Pablo Neira Ayuso / Arturo Borrero / Florian Westphal === Subtask 1.1: native "-m time" support in nftables == The first and also most simple feature that could be implemented in native nft is the --datestart / --datestop option of -m time, i.e. to match when the packet arrives between two fixed, absolute points in time. For instance "match between Feb 14th 2019 and March 30th 2019". For instance "match between Feb 14th 2019 and March 30th 2019". To implemend this one would first have to add support in the kernel, i.e. add a new key to the "meta" expression, e.g.: NFT_META_TSTAMP_NS. Then, extend net/netfilter/nft_meta.c to handle NFT_META_TSTAMP_NS in the evaluation function: case NFT_META_TSTAMP_NS: if (skb->tstamp == 0) __net_timestamp((struct sk_buff *)skb); *reg64 = ktime_to_ns(skb->tstamp); break; in other words, exposes the 64bit nanosecond timestamp of the packets arrival time in a register. You can refer to: https://git.kernel.org/pub/scm/linux/kernel/git/pablo/nf-next.git/commit/?id=0fb4d21956f4a9af225594a46857ccf29bd747bc to see an example of adding new meta functionality. This is enough to allow matching a fixed time interval from a kernel point of view. To make this useful, next step is to extend nft and libnftnl to be able to make use of this new functionality. nft userspace needs to be extended to handle a new 'meta timestamp' keyword. You will need to extend the scanner (flex) and/or parser (bison) to teach the frontend how to interpret a new keyword. Adding a new keyword for meta is simple, for example http://git.netfilter.org/nftables/commit/?id=512795a673f999fb04b84dbbbe41174e9c581430 shows all that is needed to add a new meta keyword on the nftables side. Once nft parser understands 'meta timestamp'. === Subtask 1.2: complete osf support == The existing OSF support for nftables does not support for matching the OS version type. You have to add a new NFTA_OSF_FLAGS attribute: enum nft_osf_flags { NFT_OSF_F_VERSION = (1 << 0), }; By default, if there is no NFTA_OSF_FLAGS, flag is set to zero, hence no version matching needs to be done. You have to extend nf_osf_find() to pass a structure as parameter: struct nf_osf_data { const char *os_name; const char *version; }; Then, pass this structure to nf_osf_find() that is called from packet path: if (nf_osf_find(skb, ..., &data)) { switch (priv->type) { case NFT_OSF_GENRE: strncpy((char *)dest, os_name, ...); break; case NFT_OSF_VERSION: ... break; } } else { strncpy((char *)dest, "unknown", NFT_OSF_MAXNAMELEN); } === Subtask 1.3: Extending stateful object infrastructure === Extend the stateful object [1] support to allow to runtime update, eg. upgrade a 'quota' object without losing stateful values, such as rising or lowering the quota limit of an existing object. This also would allow us to update timeout policies for conntrack in runtime. Support for this needs to integrate into de transaction infrastructure. Moreover, allowing for internal toggles for stateful objects, such as connection tracking helpers is also desiderable, e.g, we will not need a global 'loose' switch in the FTP helper anymore, instead this you be explicitly set on from the nftables ruleset policy file. [1] https://wiki.nftables.org/wiki-nftables/index.php/Stateful_objects === Subtask 1.4: allow deletion of set elements from ruleset == iptables' SET target has a "--del-set" option that will remove the matching address from the given set. in nftables, its possible to add and update, but not delete a set element. Example ruleset: table ip test { set testset { type ipv4_addr size 65535 flags timeout } chain testchain { type filter hook input priority filter; policy accept; ip saddr @testset counter update @testset { ip saddr timeout 1m } # add @testset { ip saddr timeout 1h } } } The difference is that 'add' will not add the element with timeout of 2h if it already exists, whereas 'update' will change the timeout even if the entry already exists. This is useful when elements get also added via nft update element ip test testset { 127.1.2.3 timeout 2h} . In this case, a packet sent to that address (matching ip saddr) would change timeout to 1 minute. If only the 'add @testset' rule would be used, the timeout would remain at 2 hours, whereas a different source address would be added with a 1 hour timeout. Its currently not possible to early-remove elements from the set. One can add a rule that sets a very low timeout, e.g. ... update @testset { ip saddr timeout 1ms } but it would be better to add ''proper'' support for a delete feature, e.g. delete @testset { ip saddr } === Subtask 1.5: performance optimizations in netfilter === CONFIG_RETPOLINE (cpu bug workaround ...) makes indirect calls expensive. One way to improve performance is to remove such indirections where possible. The set infrastructure would be one field where indirection removal might make sense. This task involves demodularization of nf_tables_set (make it a boolean, not a module), a size analysis of the nf_tables module (absolute size increase, percentage increase) after this change. Then, because functions can be called directly, the lookup indirections can be removed. https://git.breakpoint.cc/cgit/fw/nf-next.git/commit/?h=nft_set_rework_01 contains a untested hack that illustrates the second part (demodularization is missing, so this won't work with NF_TABLES_SET=m config). DO NOT BOTHER investigating the NAT engine, work on this is almost complete already at this point, but not integrated into the main development tree yet). === Subtask 1.6: rework netfilter logging === netfilter supports two logging backends, one is prink-based (logs decoded packet information like ip addresses, port numbers and so on to dmesg buffer), one is nfnetlink based (copy of packet gets sent to userspace listener for processing). Its not possible to use both at the same time: sysctl -a |grep nf_log_ipv4 net.netfilter.nf_log.2 = nfnetlink_log When nfnetlink_log is enabled, the dmesg support disappears, and users need to run sysctl net.netfilter.nf_log.2=nf_log_ipv4 to re-enable dmesg logging, but that turns off the nfnetlink backend. Because only two packet logger types exist (printk-based and nfnetlink one) it could be worth to investigate if we can support both being active at the same time and implement this. == Task 2: Improving automated test infrastructure == * Description: Test infrastructure is fundamental to catch regressions. This project already comes with a nice test infrastructure, but we always consider good to have more coverage. * Tag: Infrastructure/Automation. * Tasks: Help by extending the existing infrastructure to support more tests based on recent fixes and new features that got merged upstream. Add a new 'make tests' target to iptables and nftables code base to auto-run all test cases (python and shell based) that exist. Add a new build test for iptables/nftables that makes sure the projects can be compiled when various 'configure' switches get enabled or disabled. There was a recent build breakage in nftables because developer ran a build test without json support enabled and failed to notice that nftables would fail to compile for users that tried to do "./configure --with-json && make". Add more netfilter functionality tests to the kernel kselftest infrastructure, see tools/testing/selftests/netfilter in the linux kernel directory. For example, one could add a test that makes sure connection tracking events (conntrack -E) work, that conntrack packet/byte accounting works, and so on. The existing iptables test infrastructure cover per-netns control plane path. This is missing in nftables, we need an extension for nft-test.py and shell/run-tests.py to extend coverage to the network namespace path. Basically, in order to make sure resources are properly cleaned up after netnamespace removal. * Level of difficulty: Easy. There is already code in place that can be used as reference. * Mentors: Pablo Neira Ayuso / Arturo Borrero / Florian Westphal == Task 3: nftlb: nftables load balancer == * Description: We have userspace daemon to configure load balancing scenario using the nftables framework that provides a REST API. * Tag: Fun/Peripheral. * Tasks: Improve the existing codebase, add new features, such as: ** hardening of the REST API json parser. ** replication of sets and maps between nftlb instances. ** implement least connection schedulers. ** support of conntrack events management. ** add compatibility with kubernetes and docker rules. ** add support of tproxy. * Level of difficulty: Easy/Average. this is userspace codebase after all, but this is the kind of software that utilizes the nftables framework that people find useful :) * Mentors: Pablo Neira Ayuso / Arturo Borrero / Florian Westphal = More information on nftables = The next Netfilter workshop in June-July 2019 [3] will surely focus on nftables ongoing and future development discussions. The kernel components were already merged into mainstream Linux kernel 3.13. Nonetheless, implementation works are still far from complete. All existing code is available under git.netfilter.org. More specifically: * libnftnl: low-level userspace library for nftables (for libmnl) iptables. * which already includes the iptables compatibility layer working over nftables. * nft: the new user-space command line tool, with a new syntax different from iptables. The Linux kernel tree containing the nftables modules is currently available in a different repository [4]. = Contact us / Make us questions = If you are a student willing to participate in GSoC 2019 and you're interested in any of our tasks, please subscribe to this mailing list: https://lists.netfilter.org/mailman/listinfo/gsoc2013 Subscribing to this mailing list requires approval from the administrator, so please be patient, we'll accept it asap. You can use this mailing list to ask your questions regarding Netfilter's task during the GSoC 2019. You can also drop a line to arturo@netfilter.org, please make sure you Cc gsoc2013@lists.netfilter.org in your questions since most likely what you ask and the reply you get will help others in the community too. = Applying to netfilter's GSoC = If you want to be selected, go start getting familiarized with the nftables software asap. Patches for the userspace library libnftnl, the command line utility nft and kernel patches will make you rank higher in the student selection process. No patches at all mean little chances to be selected. = References = [1] http://en.wikipedia.org/wiki/Nftables [2] http://www.netfilter.org/projects/iptables/index.html [3] http://workshop.netfilter.org/ [4] http://git.kernel.org/cgit/linux/kernel/git/pablo/nftables.git [5] http://wiki.nftables.org [6] https://google.github.io/gsocguides/mentor/defining-a-project-ideas-list Author: Pablo Neira Ayuso et al. Last update: 10:31:44 +01:00 16/Feb/2019 -EOF-