Huge thanks to our Platinum Members Endace and LiveAction,
and our Silver Member Veeam, for supporting the Wireshark Foundation and project.

Wireshark-users: [Wireshark-users] SSL, Retransmits, Windows 7, Linux LVS==FUN!

From: Chris Chen <chchen@xxxxxxx>
Date: Fri, 04 Jun 2010 23:49:39 -0700
So here's my dilemma--

I've got Windows 7 clients that are having a hell of a time doing SSL/TLSv1 Handshakes with SMTP and IMAP servers that live behind a Linux Virtual Server.

All other clients work great. The Windows 7 clients report timeouts connecting to IMAPS and SMTPS. So, I ran a trace, and...

Here's a simple tshark dump:

1 0.000000 WIN7_IP -> LINUX_IP TCP 49168 > urd [SYN] Seq=0 Win=8192 Len=0 MSS=1380 WS=2 8192 52 2 0.000010 LINUX_IP -> WIN7_IP TCP urd > 49168 [SYN, ACK] Seq=0 Ack=1 Win=5840 Len=0 MSS=1460 WS=2 5840 52 3 0.000680 WIN7_IP -> LINUX_IP TCP 49168 > urd [ACK] Seq=1 Ack=1 Win=66240 Len=0 66240 40
  4   0.000990 WIN7_IP -> LINUX_IP SSL Client Hello 66240 209
5 0.001005 LINUX_IP -> WIN7_IP TCP urd > 49168 [ACK] Seq=1 Ack=170 Win=6912 Len=0 6912 40
  6   0.009298 LINUX_IP -> WIN7_IP TLSv1 Server Hello,  6912 1420
7 0.009313 LINUX_IP -> WIN7_IP TLSv1 Certificate, Server Key Exchange, Server Hello Done 6912 1356 8 0.010187 WIN7_IP -> LINUX_IP TCP 49168 > urd [ACK] Seq=170 Ack=2697 Win=66240 Len=0 66240 40 9 0.024244 WIN7_IP -> LINUX_IP TLSv1 Client Key Exchange, Change Cipher Spec, Encrypted Handshake Message 66240 174 10 0.025339 LINUX_IP -> WIN7_IP TLSv1 Change Cipher Spec, Encrypted Handshake Message 7984 99 11 0.225653 LINUX_IP -> WIN7_IP TLSv1 [TCP Retransmission] Change Cipher Spec, Encrypted Handshake Message 7984 99 12 0.226324 WIN7_IP -> LINUX_IP TCP 49168 > urd [ACK] Seq=304 Ack=2756 Win=66180 Len=0 SLE=1597629342 SRE=1597629401 66180 52 13 0.627556 LINUX_IP -> WIN7_IP TLSv1 [TCP Retransmission] Change Cipher Spec, Encrypted Handshake Message 7984 99 14 0.628474 WIN7_IP -> LINUX_IP TCP [TCP Dup ACK 12#1] 49168 > urd [ACK] Seq=304 Ack=2756 Win=66180 Len=0 SLE=1597629342 SRE=1597629401 66180 52 15 1.431361 LINUX_IP -> WIN7_IP TLSv1 [TCP Retransmission] Change Cipher Spec, Encrypted Handshake Message 7984 99 16 1.432062 WIN7_IP -> LINUX_IP TCP [TCP Dup ACK 12#2] 49168 > urd [ACK] Seq=304 Ack=2756 Win=66180 Len=0 SLE=1597629342 SRE=1597629401 66180 52 17 3.039985 LINUX_IP -> WIN7_IP TLSv1 [TCP Retransmission] Change Cipher Spec, Encrypted Handshake Message 7984 99 18 3.040671 WIN7_IP -> LINUX_IP TCP [TCP Dup ACK 12#3] 49168 > urd [ACK] Seq=304 Ack=2756 Win=66180 Len=0 SLE=1597629342 SRE=1597629401 66180 52 19 6.255200 LINUX_IP -> WIN7_IP TLSv1 [TCP Retransmission] Change Cipher Spec, Encrypted Handshake Message 7984 99 20 6.255869 WIN7_IP -> LINUX_IP TCP [TCP Dup ACK 12#4] 49168 > urd [ACK] Seq=304 Ack=2756 Win=66180 Len=0 SLE=1597629342 SRE=1597629401 66180 52 21 12.686644 LINUX_IP -> WIN7_IP TLSv1 [TCP Retransmission] Change Cipher Spec, Encrypted Handshake Message 7984 99 22 12.687330 WIN7_IP -> LINUX_IP TCP [TCP Dup ACK 12#5] 49168 > urd [ACK] Seq=304 Ack=2756 Win=66180 Len=0 SLE=1597629342 SRE=1597629401 66180 52 23 25.548536 LINUX_IP -> WIN7_IP TLSv1 [TCP Retransmission] Change Cipher Spec, Encrypted Handshake Message 7984 99 24 25.549373 WIN7_IP -> LINUX_IP TCP [TCP Dup ACK 12#6] 49168 > urd [ACK] Seq=304 Ack=2756 Win=66180 Len=0 SLE=1597629342 SRE=1597629401 66180 52
 25  29.280023 WIN7_IP -> LINUX_IP TLSv1 Encrypted Alert 66180 77
 26  29.280040 LINUX_IP -> WIN7_IP TLSv1 Application Data 7984 157
27 29.280109 WIN7_IP -> LINUX_IP TCP 49168 > urd [FIN, ACK] Seq=341 Ack=2756 Win=66180 Len=0 66180 40 28 29.280147 LINUX_IP -> WIN7_IP TCP urd > 49168 [FIN, ACK] Seq=2873 Ack=342 Win=7984 Len=0 7984 40 29 29.280582 WIN7_IP -> LINUX_IP TCP 49168 > urd [RST, ACK] Seq=342 Ack=2873 Win=0 Len=0 0 40

So it looks like the handshake goes great up until line 9. The Linux host retransmits, then the ACK arrives, but the two hosts keep doing this dance until someone gives up (looks like the Windows 7 host, who sends the disconnect).

The crazy thing is I got openssl for windows installed and ran the s_client tool to see what was going on--it made it through the entire handshake, but
stalled. Here's the sample output from a successful run:

---
No client certificate CA names sent
---
SSL handshake has read 2756 bytes and written 247 bytes
---
New, TLSv1/SSLv3, Cipher is DHE-RSA-AES256-SHA
Server public key is 1024 bit
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : TLSv1
    Cipher    : DHE-RSA-AES256-SHA
Session-ID: 21534131F1D5D5A7CCEE2D20A6CA23BA023D93B2F85F07ABF23C45666DDC63D6
    Session-ID-ctx:
Master-Key: AEF4CF4E7F85DA24345902E54B7815E5235876EDE334614AA99A04C86BD4F361699A7C3BEB0338E80162EFB617F186CA
    Key-Arg   : None
    Krb5 Principal: None
    PSK identity: None
    PSK identity hint: None
    Start Time: 1275719887
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
---
220 mailhost.domain blah blah blah (This is the MTA's greeting line)

However, when the retransmit nonsense happens, there is not 220 line--the output stops at ---.

The crazy thing is, when I press enter, an additional packet is sent, and the connection resumes, and the 220 line appears.

Has anyone seen this behavior before?

cc

--
Chris Chen <chchen@xxxxxxx>
UNIX Systems Administrator
Office of Information Technologies
Portland State University