Wireshark-dev: Re: [Wireshark-dev] wireshark utilities unable to read large files
From: Guy Harris <[email protected]>
Date: Sat, 28 Jun 2008 10:24:40 -0700
Romel Khan wrote:
Any wireshark utilities such as tshark & editcap, are unable to read large files. When using wireshark package to monitor live package, it may be necessary to read files that are many giga bytes. Upon investigating the code, I find that a eth_open function call is made::
            wiretap/file_access.c:          wth->fd = eth_open(filename, 
O_RDONLY|O_BINARY, 0000 /* no creation so don't matter */);
With mindset on linux system,
Wireshark runs on systems that aren't Linux, so one shouldn't have a 
Linux mindset when developing code for Wireshark, one should have a 
multi-platform mindset.
Not all systems have O_LARGEFILE; on 4.4-Lite-flavored systems 
({Free,Net,Open,DragonFly}BSD, Mac OS X), it's not needed, as off_t is 
64 bits even on 32-bit systems, and large file support comes 
automatically.  On Windows, large file support, if it's even possible 
with stdio routines, probably requires some other mechanism.
this function would call the open(..) function. It seems that including O_LARGEFILE added as an ‘or’ in the flags argument (ie the 2^nd argument in the eth_open(..)) would help::::
                                                            wth->fd = 
eth_open(filename, O_RDONLY|O_BINARY| O_LARGEFILE, 0000)
It would, however, not be sufficient.

The reads are done with the fh member of the wtap structure, not the fd member. The fh member comes from the filed_open() call; filed_open is #defined to be fdopen if you don't have libz - and gzdopen if you do. As of when I last looked at it, libz's code for reading files did *NOT* support large files - and we won't be removing the ability to read gzipped files, as that would be a significant regression.
Alternative, option is to rewrite this code with a call to fopen(filename,”r”) instead of using open(..).
Note that "this code" also uses ws_dup(), for the case where it's 
reading from the standard input.
Tcpdump using libpcap which calls fopen(..) has no issue dealing with large files.
That's only in the top-of-tree CVS version, and that's because the 
configure script for libpcap uses AC_SYS_LARGEFILE.