aboutsummaryrefslogtreecommitdiffstats
path: root/sys/netinet/tcp_usrreq.c
diff options
context:
space:
mode:
authorAndrew Gallatin <gallatin@FreeBSD.org>2020-12-19 22:04:46 +0000
committerAndrew Gallatin <gallatin@FreeBSD.org>2020-12-19 22:04:46 +0000
commita034518ac8793059220af22e6ab25f84c5a6ddb8 (patch)
treecdeacb29f82ad8c1a7d63f8f6da2c77827ec1525 /sys/netinet/tcp_usrreq.c
parent02bc3865aadf1bfd907bbf555e26719744ebf3c9 (diff)
downloadsrc-a034518ac8793059220af22e6ab25f84c5a6ddb8.tar.gz
src-a034518ac8793059220af22e6ab25f84c5a6ddb8.zip
Filter TCP connections to SO_REUSEPORT_LB listen sockets by NUMA domain
In order to efficiently serve web traffic on a NUMA machine, one must avoid as many NUMA domain crossings as possible. With SO_REUSEPORT_LB, a number of workers can share a listen socket. However, even if a worker sets affinity to a core or set of cores on a NUMA domain, it will receive connections associated with all NUMA domains in the system. This will lead to cross-domain traffic when the server writes to the socket or calls sendfile(), and memory is allocated on the server's local NUMA node, but transmitted on the NUMA node associated with the TCP connection. Similarly, when the server reads from the socket, he will likely be reading memory allocated on the NUMA domain associated with the TCP connection. This change provides a new socket ioctl, TCP_REUSPORT_LB_NUMA. A server can now tell the kernel to filter traffic so that only incoming connections associated with the desired NUMA domain are given to the server. (Of course, in the case where there are no servers sharing the listen socket on some domain, then as a fallback, traffic will be hashed as normal to all servers sharing the listen socket regardless of domain). This allows a server to deal only with traffic that is local to its NUMA domain, and avoids cross-domain traffic in most cases. This patch, and a corresponding small patch to nginx to use TCP_REUSPORT_LB_NUMA allows us to serve 190Gb/s of kTLS encrypted https media content from dual-socket Xeons with only 13% (as measured by pcm.x) cross domain traffic on the memory controller. Reviewed by: jhb, bz (earlier version), bcr (man page) Tested by: gonzo Sponsored by: Netfix Differential Revision: https://reviews.freebsd.org/D21636
Notes
Notes: svn path=/head/; revision=368819
Diffstat (limited to 'sys/netinet/tcp_usrreq.c')
-rw-r--r--sys/netinet/tcp_usrreq.c10
1 files changed, 10 insertions, 0 deletions
diff --git a/sys/netinet/tcp_usrreq.c b/sys/netinet/tcp_usrreq.c
index ffee58b9043f..bfa96ce093b5 100644
--- a/sys/netinet/tcp_usrreq.c
+++ b/sys/netinet/tcp_usrreq.c
@@ -2143,6 +2143,16 @@ unlock_and_done:
INP_WUNLOCK(inp);
break;
+ case TCP_REUSPORT_LB_NUMA:
+ INP_WUNLOCK(inp);
+ error = sooptcopyin(sopt, &optval, sizeof(optval),
+ sizeof(optval));
+ INP_WLOCK_RECHECK(inp);
+ if (!error)
+ error = in_pcblbgroup_numa(inp, optval);
+ INP_WUNLOCK(inp);
+ break;
+
#ifdef KERN_TLS
case TCP_TXTLS_ENABLE:
INP_WUNLOCK(inp);