Abstract:
packetdrillIt is a very useful tool for testing network protocol stack
1. introduction
packetdrillIt is a very useful tool for testing network protocol stackGoogleDevelopment, it is often used to regression test the network protocol stack to ensure that the new function will not affect the original function. It supportsLinux, FreeBSD, OpenBSDAndNetBSDKernel. It uses scripted language to write test statements and predict protocol stack output. The official also provides many test script examples.
2. principle
packetdrillThe overall framework of is shown in the figure below
packetdrillThe application internally simulates a connectedRemote terminalandLocal terminal。 amongRemote terminalAs a remote channel to send messages to the local machine, we canpacketdrillApplied introversiontunDevice writeIPThe message comes from the kernel protocol stack, which is equivalent to receiving this message from the remote endIPThe message, after routing, will be sent to the protocol stack. Conversely, the direction of the kernel protocol stackRemote terminalThe message will be sent through thistunDevice backpacketdrillAt this time, we can verify the correctness of the function of the protocol stack by comparing its input.
The script file is.pktFile with suffix,packetdrillRead the file after startup,Script parserResolve each line of script statement to runtimeevent，Script runnerExecute each in turnevent。
3. installation
packetdrillDependentpackage: gcc、python、flex、bison
Download the source code from the official GitHub and compile it
> ./configure
> make4. entry
Execute a test script
> ./packetdrill tests/linux/fast_retransmit/fr-4pkt-sack-linux.pkt
> If there is no output, it means that the script test has passed:), otherwise, it will prompt which line of script does not meet the expectation and the reason for the error respectively
For example, an error occurred when executing the following script on my machine (kernel version 4.4.0):
> ./packetdrill tests/linux/listen/listen-incoming-ack.pkt
tests/linux/listen/listen-incoming-ack.pkt:17: error handling packet: bad value outbound TCP option 3
script packet: 0.200000 S. 0:0(0) ack 1 <mss 1460,nop,nop,sackOK,nop,wscale 6>
actual packet: 0.201014 S. 0:0(0) ack 1 win 29200 <mss 1460,nop,nop,sackOK,nop,wscale 7>
>It indicates that the17There was an error on line, in the scriptRemote terminalExpected receiptSYNACKIn the messagewscale=6, but in the message actually receivedwscale=7。
The reason for this error is that the protocol stack implementation of the kernel version suitable for the script is inconsistent with that of my native version! The kernel version is inconsistent, and some implementations of the protocol stack are inconsistent! In this case, we can simply modify the script to adapt to the kernel version we use.
5. Script language
packetdrillInstead of using a ready-made scripting language, its scripts have sometcpdumpShadow, some moresocketTrace of programming
// Test behavior when a listener gets an incoming packet that has
// the ACK bit set but not the SYN bit set.
0.000 socket(..., SOCK_STREAM, IPPROTO_TCP) = 3
0.000 setsockopt(3, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
0.000 bind(3, ..., ...) = 0
0.000 listen(3, 1) = 0
0.100 < . 0:0(0) win 32792 <mss 1000,sackOK,nop,nop,nop,wscale 7>
0.100 > R 0:0(0) win 0
// Now make sure that when a valid SYN arrives shortly thereafter
// (with the same address 4-tuple) we can still successfully establish
// a connection.
0.200 < S 0:0(0) win 32792 <mss 1000,sackOK,nop,nop,nop,wscale 7>
0.200 > S. 0:0(0) ack 1 <mss 1460,nop,nop,sackOK,nop,wscale 6>
0.300 < . 1:1(0) ack 1 win 320
0.300 accept(3, ..., ...) = 4I think the best way to learn this kind of script is to learn the official example. In the example, you can build the script you need by following the rules! Really have doubt still can turn over code slightly!
time stamp
Every line of a script isTimestamp + statementThe form. The time stamp indicates the execution time of the statement,packetdrillSupportAbsolute timeandRelative timeTwo formats
//Relative time, 0.1 seconds after the last script
+.1 setsockopt(3, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
//Absolute time, 0.2 seconds after the script starts running
0.200 setsockopt(3, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0Inject message into protocol stack
Time stamp followed<The statement representation of the symbol fromRemote terminalInject message into protocol stack, followed by defaultTCPThe content of the message (of course, it can also be connected to other protocols, but the complexity of the protocol stack mostly lies inTCP)
//Inject a syn message (s for SYN), the start and end serial numbers are 0, the data length is 0, the notification window size is 32792, and the options of MSS, sack and wscale are carried.
0.200 < S 0:0(0) win 32792 <mss 1000,sackOK,nop,nop,nop,wscale 7>Receive message from protocol stack
Time stamp followed>Statement representation of symbolsRemote terminalThe message is expected to be received from the protocol stack. The expected receiving time here is a range[ts-tolerance,ts+tolerance], tolerance timetoleranceThe default is4MS (can be changed by running parameters)
//It is expected to receive a sync message (. Indicates ACK). The ACK sequence number is 1, which carries the options of MSS, sack and wscale
0.200 > S. 0:0(0) ack 1 <mss 1460,nop,nop,sackOK,nop,wscale 6>system call
The message statement above is standing atRemote terminalFrom the point of view, system call is standingLocal terminalYes,packetdrillThe following system calls are supported
struct system_call_entry system_call_table[] = {
{"socket", syscall_socket},
{"bind", syscall_bind},
{"listen", syscall_listen},
{"accept", syscall_accept},
{"connect", syscall_connect},
{"read", syscall_read},
{"readv", syscall_readv},
{"recv", syscall_recv},
{"recvfrom", syscall_recvfrom},
{"recvmsg", syscall_recvmsg},
{"write", syscall_write},
{"writev", syscall_writev},
{"send", syscall_send},
{"sendto", syscall_sendto},
{"sendmsg", syscall_sendmsg},
{"fcntl", syscall_fcntl},
{"ioctl", syscall_ioctl},
{"close", syscall_close},
{"shutdown", syscall_shutdown},
{"getsockopt", syscall_getsockopt},
{"setsockopt", syscall_setsockopt},
{"poll", syscall_poll},
{"cap_set", syscall_cap_set},
{"open", syscall_open},
{"sendfile", syscall_sendfile},
{"epoll_create", syscall_epoll_create},
{"epoll_ctl", syscall_epoll_ctl},
{"epoll_wait", syscall_epoll_wait},
{"pipe", syscall_pipe},
{"splice", syscall_splice},
};What’s different from the system call we are used to is that,packetdrillSome parameters in the system call in cannot be changed. We need to fill in...(script running opportunity will help us to fill in), in addition, we need to set its return value.
//The socket system call returns FD = 3, where the three are only valid in the script scope. The descriptor value returned at runtime is maintained by the framework, and the framework will maintain their corresponding relationship
0.000 socket(..., SOCK_STREAM, IPPROTO_TCP) = 3
//Setsockopt system call, the third parameter [1] represents a pointer to the value 1
0.000 setsockopt(3, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
//Listen system call, the last two parameters are determined by the framework
0.000 bind(3, ..., ...) = 0
0.000 listen(3, 1) = 0assert
Sometimes we need to peepTCPMore information about the runtime, such asMSSWhat is the current window sizecwndWhat is the slow start thresholdssthreshHow much is it? Now we can useassertStatement to expect its state
//Expected status information at this time
0.300 %{
assert tcpi_reordering == 3
assert tcpi_unacked == 10
assert tcpi_sacked == 1
}%packetdrillSupport expectationsTCPThe information is as follows:
/* packetdrill/gtests/net/packetdrill/tcp.h */
struct _tcp_info {
__u8 tcpi_state;
__u8 tcpi_ca_state;
__u8 tcpi_retransmits;
__u8 tcpi_probes;
__u8 tcpi_backoff;
__u8 tcpi_options;
__u8 tcpi_snd_wscale:4, tcpi_rcv_wscale:4;
__u8 tcpi_delivery_rate_app_limited:1;
__u32 tcpi_rto;
__u32 tcpi_ato;
__u32 tcpi_snd_mss;
__u32 tcpi_rcv_mss;
__u32 tcpi_unacked;
__u32 tcpi_sacked;
__u32 tcpi_lost;
__u32 tcpi_retrans;
__u32 tcpi_fackets;
/* Times. */
__u32 tcpi_last_data_sent;
__u32 tcpi_last_ack_sent; /* Not remembered, sorry. */
__u32 tcpi_last_data_recv;
__u32 tcpi_last_ack_recv;
/* Metrics. */
__u32 tcpi_pmtu;
__u32 tcpi_rcv_ssthresh;
__u32 tcpi_rtt;
__u32 tcpi_rttvar;
__u32 tcpi_snd_ssthresh;
__u32 tcpi_snd_cwnd;
__u32 tcpi_advmss;
__u32 tcpi_reordering;
__u32 tcpi_rcv_rtt;
__u32 tcpi_rcv_space;
__u32 tcpi_total_retrans;
__u64 tcpi_pacing_rate;
__u64 tcpi_max_pacing_rate;
__u64 tcpi_bytes_acked; /* RFC4898 tcpEStatsAppHCThruOctetsAcked */
__u64 tcpi_bytes_received; /* RFC4898 tcpEStatsAppHCThruOctetsReceived */
__u32 tcpi_segs_out; /* RFC4898 tcpEStatsPerfSegsOut */
__u32 tcpi_segs_in; /* RFC4898 tcpEStatsPerfSegsIn */
__u32 tcpi_notsent_bytes;
__u32 tcpi_min_rtt;
__u32 tcpi_data_segs_in; /* RFC4898 tcpEStatsDataSegsIn */
__u32 tcpi_data_segs_out; /* RFC4898 tcpEStatsDataSegsOut */
__u64 tcpi_delivery_rate;
__u64 tcpi_busy_time; /* Time (usec) busy sending data */
__u64 tcpi_rwnd_limited; /* Time (usec) limited by receive window */
__u64 tcpi_sndbuf_limited; /* Time (usec) limited by send buffer */
};
Special attention is needed when usingassertWe need to make surestruct _tcp_infoStructure inpacketdrillIt is consistent with the definition in the current kernel. Otherwise, an error will be reported!
(end)
