What is the maximum packet size for a TCP connection?

This question is also somewhat ambiguous, but for a different reason: the size of IP packets that a TCP stack uses to send data is chosen by the stack itself, and the user or application writing to the socket has very little control over packet-sizes. TCP stacks are free to:

  • Amalgamate multiple small chunks of data written separately to the socket into a single IP packet; and
  • Segment a single large block of data written once to the socket into multiple IP packet.

The user can set the TCP_NODELAY option on a socket as a hint to the stack that it should avoid the first behaviour and send every write to the socket immediately in a separate packet, but there is no way to disable the latter behaviour. In particular, the TCP stack will do its very best to avoid fragmentation, and sending IP packets that exceed the MTU will result in fragmentation.

Generally TCP will do an excellent job of using the largest packet size possible to minimize the overheads of IP and TCP headers, while avoiding fragmentation. TCP implementations typically start with the MTU of the interface as the limit to respect to avoid fragmentation; however, a link on the route the receiver may have a smaller MTU. To detect this, TCP generally sets the DF (don’t fragment) flag in the IP header which should result in the device with the smaller MTU sending back an ICMP message to this effect (specifically a Destination Unreachable message with code 4). In response to this, the sender will reduce its maximum segment size to avoid the need for fragmentation.

Note that it is possible for this mechanism to run into problems: some devices are configured not to send ICMP messages, and some firewalls will block all ICMP traffic. In this case, the TCP connection will get established, since the packets used in the three-way handshake are (almost certainly) no bigger than the path MTU; any small application control packets will likely be successfully transferred, but as soon as bulk data starts to flow, the sender will use segments as large as allowed by the local MTU. Once these hit the link with the smaller MTU, the DF flag means they will not be forwarded, but the lack of ICMP messages means the sender gets no indication of this, and the packets disappear into a black hole.

In short, the maximum packet-size that TCP will use is limited by the path MTU.


Blog: Cloudy with a Chance of TCP Drops