Wireshark-users: Re: [Wireshark-users] Mysterious packet loss during capture

From: Guy Harris <guy@xxxxxxxxxxxx>
Date: Fri, 9 Oct 2009 15:52:20 -0700

On Oct 9, 2009, at 2:27 PM, Stuart Kendrick wrote:

I've read about Gulp (http://staff.washington.edu/corey/gulp/)

It's now

	http://corey.elsewhere.org/gulp/

as he no longer works at the U of Washington, as per his home page:

	http://corey.elsewhere.org/

Note: when he says

3. I think "struct pcap_pkthdr" in pcap.h should be re-defined to be independent of sizeof(long). In pcap files, a struct pcap_pkthdr precedes every packet. Unfortunately, the size of struct pcap_pkthdr (which contains a struct timeval) depends upon sizeof(long). This makes pcap files from 64-bit linux systems incompatible with those from 32-bit systems. Apparently as a workaround, some 64-bit linux distributions are providing tcpdump and wireshark binaries which read/ write 32-bit compatible pcap files (which makes Gulp's pcap output appear to be corrupt).

(To build Gulp on 64-bit linux systems so that it reads/writes 32-bit compatible pcap files, try installing the 32-bit (i386) "libpcap- devel" package and making Gulp with "-m32" added to CFLAGS.)

"32-bit compatible pcap files" can also just be called "pcap files" - Ethereal/Wireshark, and all programs using libpcap, including tcpdump, expect 32-bit time stamps. (And, no, "struct pcap_pkthdr" shouldn't be redefined to be independent of sizeof(long); programs that write pcap files without using libpcap should not write a "struct pcap_pkthdr" before each packet, it should write a structure that has a 32-bit seconds value, a 32-bit microseconds value, a 32-bit captured length, and a 32-bit on-the-wire length before each packet. The libpcap man pages say that a "struct pcap_pkthdr" starts with a "struct timeval", and a "struct timeval", on many platforms, depends on sizeof(long), so changing "struct pcap_pkthdr" will break source and binary compatibility.

So you should apply the attached patch to Gulp before building. I'll send that patch to Corey.

*** Makefile.dist	2008-03-14 12:50:10.000000000 -0700
--- Makefile	2009-10-09 15:49:58.000000000 -0700
***************
*** 1,13 ****
  CFLAGS = -O
  #CFLAGS = -O -DRHEL3
  #CFLAGS = -O -DJUSTCOPY
- #CFLAGS = -O -DPCAP32_KLUDGE
- #CFLAGS = -O -m32
  
  FILES=license.txt NOTICE gulp.c gulp.1 gulpman.html gulpman.pdf Makefile conv.c gulp.html check64bit.c changelog
  
! gulp:	gulp.c check64bit
! 	./check64bit; rm -f check64bit
  	cc -g $(CFLAGS) gulp.c -o gulp -lpthread -lpcap
  
  gulpman.html:	gulp.1
--- 1,10 ----
  CFLAGS = -O
  #CFLAGS = -O -DRHEL3
  #CFLAGS = -O -DJUSTCOPY
  
  FILES=license.txt NOTICE gulp.c gulp.1 gulpman.html gulpman.pdf Makefile conv.c gulp.html check64bit.c changelog
  
! gulp:	gulp.c
  	cc -g $(CFLAGS) gulp.c -o gulp -lpthread -lpcap
  
  gulpman.html:	gulp.1

*** gulp.c.dist	2008-03-18 14:12:32.000000000 -0700
--- gulp.c	2009-10-09 15:49:34.000000000 -0700
***************
*** 224,245 ****
  	ph.caplen -= gre_hdrlen;
  	ph.len -= gre_hdrlen;
  
- #ifdef PCAP32_KLUDGE
  	/*
! 	 * Because struct timeval is bigger on 64-bit linux than 32-bit
! 	 * linux and struct pcap_pkthdr has a struct timeval in it, pcap
! 	 * files generated by Gulp on 64-bit linux may be incompatible
! 	 * with programs expecting pcap files from 32-bit systems.  If
! 	 * you want Gulp to generate 32-bit compatible pcap files, first
! 	 * try compiling it after adding -m32 to CFLAGS in the Makefile
! 	 * (you may also need to install a 32-bit libpcap-devel package).
! 	 *
! 	 * If that doesn't work, try instead adding -DPCAP32_KLUDGE but
! 	 * note that the PCAP32_KLUDGE is not a complete solution
! 	 * because it will cause gulp to write files which it can't
! 	 * itself read (because reading is done by the pcap library code
! 	 * which will still be expecting 64-bit longs in struct timevals
! 	 * in packet headers).
  	 */
  	if (sizeof(long) > sizeof(int) && sizeof(int) > sizeof(short)) {
  	    struct timeval_32 {
--- 224,237 ----
  	ph.caplen -= gre_hdrlen;
  	ph.len -= gre_hdrlen;
  
  	/*
! 	 * So that a capture file's layout is independent of the
! 	 * word size of the machine on which it's written, a packet
! 	 * record in a pcap file doesn't have a struct pcap_pkthdr
! 	 * at the beginning; it has a structure defined internally
! 	 * to libpcap, struct pcap_sf_pkthdr, which begins with another
! 	 * structure define internally to libpcap, struct pcap_timeval,
! 	 * with 32-bit tv_sec and tv_usec values.
  	 */
  	if (sizeof(long) > sizeof(int) && sizeof(int) > sizeof(short)) {
  	    struct timeval_32 {
***************
*** 253,259 ****
  		   sizeof(struct pcap_pkthdr) - sizeof(struct timeval), 0);
  	    }
  	else 
- #endif /* PCAP32_KLUDGE */
  	    append((char *)&ph, sizeof(struct pcap_pkthdr), 0);
  	append((char *)packet+gre_hdrlen, ph.caplen, 1);
  	}
--- 245,250 ----