/bugzilla3/ Bug 495 – Port-forwarding in Dom0 causes corrupt TCP checksums in DomU
Bug 495 - Port-forwarding in Dom0 causes corrupt TCP checksums in DomU
: Port-forwarding in Dom0 causes corrupt TCP checksums in DomU
Status: RESOLVED DUPLICATE of bug 447
Product: Xen
Unspecified
: 3.0.0
: x86 Linux
: P2 normal
Assigned To: Xen Bug List
:
:
:
  Show dependency treegraph
 
Reported: 2006-01-27 18:53 CST by Kevin Lai
Modified: 2006-02-28 21:47 CST (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Kevin Lai 2006-01-27 18:53:10 CST
I encountered a very odd problem which I eventually found a workaround
for, but I wanted to make sure the developers are aware of it. The
basic idea is that I created a NAT-ed domU using the standard
scripts and kernels from the 2.6.12.6-xen3_7.1_fc4 RPM. I can make
outgoing connections from DomU and it is otherwise fine. I then
forwarded an external port to the domU: 

iptables -t nat -A PREROUTING -p tcp --dst 15.4.89.26 --dport 11014 \
  -j DNAT --to 10.202.107.174:22 

15.4.89.26 and 11014 are the external IP address and external port,
respectively and 10.202.107.174 and 22 are the internal IP address and
port, respectively. I then did 

> ssh -p 11014 15.4.89.26

from another machine. This hung. Upon closer inspection, I saw that
DomU was sending some of its TCP packets with a corrupt TCP checksum
(?!):

[root@klai-tycoon ~]# tcpdump -i eth0 -nvvvvv
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
17:34:34.202033 IP (tos 0x10, ttl  63, id 1746, offset 0, flags [DF], proto 6,
length: 60) 15.4.89.35.47694 > 10.202.107.174.ssh: S [tcp sum ok]
845312449:845312449(0) win 5840 <mss 1460,sackOK,timestamp 257914643 0,nop,wscale 2>
17:34:34.225891 IP (tos 0x0, ttl  64, id 0, offset 0, flags [DF], proto 6,
length: 60) 10.202.107.174.ssh > 15.4.89.35.47694: S [tcp sum ok]
3227641594:3227641594(0) ack 845312450 win 5792 <mss 1460,sackOK,timestamp
4294947271 257914643,nop,wscale 2>
17:34:34.202227 IP (tos 0x10, ttl  63, id 1748, offset 0, flags [DF], proto 6,
length: 52) 15.4.89.35.47694 > 10.202.107.174.ssh: . [tcp sum ok] 1:1(0) ack 1
win 1460 <nop,nop,timestamp 257914643 4294947271>
17:34:34.236769 IP (tos 0x0, ttl  64, id 22332, offset 0, flags [DF], proto 6,
length: 72) 10.202.107.174.ssh > 15.4.89.35.47694: P [bad tcp cksum ded9
(->9f8f)!] 1:21(20) ack 1 win 1448 <nop,nop,timestamp 4294947275 257914643>
17:34:34.446430 IP (tos 0x0, ttl  64, id 22334, offset 0, flags [DF], proto 6,
length: 72) 10.202.107.174.ssh > 15.4.89.35.47694: P [bad tcp cksum ded9
(->9f7a)!] 1:21(20) ack 1 win 1448 <nop,nop,timestamp 4294947296 257914643>
17:34:34.866394 IP (tos 0x0, ttl  64, id 22336, offset 0, flags [DF], proto 6,
length: 72) 10.202.107.174.ssh > 15.4.89.35.47694: P [bad tcp cksum ded9
(->9f50)!] 1:21(20) ack 1 win 1448 <nop,nop,timestamp 4294947338 257914643>
17:34:35.706410 IP (tos 0x0, ttl  64, id 22338, offset 0, flags [DF], proto 6,
length: 72) 10.202.107.174.ssh > 15.4.89.35.47694: P [bad tcp cksum ded9
(->9efc)!] 1:21(20) ack 1 win 1448 <nop,nop,timestamp 4294947422 257914643>

This only happens for traffic forwarded through Dom0. I have no idea
why this happens. After much debugging, I found that executing

> iptables -t nat --list

on DomU fixes this problem. In particular, having the iptable_nat
kernel module loaded in DomU causes the correct checksums to be used.
Comment 1 Richard W.M. Jones 2006-02-08 15:13:12 CST
I have this bug too, with a Xen unstable nightly tarball downloaded on
2006/02/02.

Here is a tcpdump of a simple TCP connection made from a domU.  Note that the
initial 3 way handshake goes OK, but thereafter all TCP packets have a bad 
checksum.

tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
15:18:47.955933 IP (tos 0x10, ttl  63, id 55208, offset 0, flags [DF], length:
60) 192.168.2.250.4361 > 80.68.91.176.80: S [tcp sum ok] 87812993:87812993(0)
win 5840 <mss 1460,sackOK,timestamp 146473 0,nop,wscale 2>
15:18:47.989850 IP (tos 0x0, ttl  55, id 0, offset 0, flags [DF], length: 60)
80.68.91.176.80 > 192.168.2.250.4361: S [tcp sum ok] 2338351:2338351(0) ack
87812994 win 5792 <mss 1460,sackOK,timestamp 164554683 146473,nop,wscale 0>
15:18:47.990004 IP (tos 0x10, ttl  63, id 55209, offset 0, flags [DF], length:
52) 192.168.2.250.4361 > 80.68.91.176.80: . [tcp sum ok] 1:1(0) ack 1 win 1460
<nop,nop,timestamp 146482 164554683>
15:18:51.900292 IP (tos 0x10, ttl  63, id 55210, offset 0, flags [DF], length:
68) 192.168.2.250.4361 > 80.68.91.176.80: P [bad tcp cksum fe8b (->4dca)!]
1:17(16) ack 1 win 1460 <nop,nop,timestamp 147460 164554683>
15:18:52.134882 IP (tos 0x10, ttl  63, id 55211, offset 0, flags [DF], length:
68) 192.168.2.250.4361 > 80.68.91.176.80: P [bad tcp cksum fe8b (->4d8f)!]
1:17(16) ack 1 win 1460 <nop,nop,timestamp 147519 164554683>
15:18:52.606902 IP (tos 0x10, ttl  63, id 55212, offset 0, flags [DF], length:
68) 192.168.2.250.4361 > 80.68.91.176.80: P [bad tcp cksum fe8b (->4d19)!]
1:17(16) ack 1 win 1460 <nop,nop,timestamp 147637 164554683>
15:18:53.550959 IP (tos 0x10, ttl  63, id 55213, offset 0, flags [DF], length:
68) 192.168.2.250.4361 > 80.68.91.176.80: P [bad tcp cksum fe8b (->4c2d)!]
1:17(16) ack 1 win 1460 <nop,nop,timestamp 147873 164554683>
15:18:55.567116 IP (tos 0x10, ttl  63, id 55214, offset 0, flags [DF], length:
68) 192.168.2.250.4361 > 80.68.91.176.80: P [bad tcp cksum fe8b (->4a35)!]
1:17(16) ack 1 win 1460 <nop,nop,timestamp 148377 164554683>
15:18:59.583385 IP (tos 0x10, ttl  63, id 55215, offset 0, flags [DF], length:
68) 192.168.2.250.4361 > 80.68.91.176.80: P [bad tcp cksum fe8b (->4649)!]
1:17(16) ack 1 win 1460 <nop,nop,timestamp 149381 164554683>
15:19:07.135834 IP (tos 0x10, ttl  63, id 55216, offset 0, flags [DF], length:
68) 192.168.2.250.4361 > 80.68.91.176.80: P [bad tcp cksum fe8b (->3ee9)!]
1:17(16) ack 1 win 1460 <nop,nop,timestamp 151269 164554683>
15:19:22.240792 IP (tos 0x10, ttl  63, id 55217, offset 0, flags [DF], length:
68) 192.168.2.250.4361 > 80.68.91.176.80: P [bad tcp cksum fe8b (->3029)!]
1:17(16) ack 1 win 1460 <nop,nop,timestamp 155045 164554683>

I am using the standard NAT scripts.

Rich (rich at annexia dot org)
Comment 2 Richard W.M. Jones 2006-02-08 15:55:51 CST
And I can also confirm that simply loading the iptable_nat module
_in the domU_ fixes this bug - I have no idea why.

Rich.
Comment 3 Martin Schaller 2006-02-28 21:47:56 CST

*** This bug has been marked as a duplicate of 447 ***