Wireshark-dev: Re: [Wireshark-dev] Wireshark memory handling
From: Guy Harris <[email protected]>
Date: Fri, 9 Oct 2009 18:48:29 -0700
On Oct 8, 2009, at 1:15 PM, Erlend Hamberg wrote:

The "problem", as we have understood it, and as we have seen it to be, is that Wireshark keeps its internal representation (from reading a capture file) in
memory.
The data Wireshark currently keeps in its address space that could  
grow in size as the capture file grows are:
	the frame_data structure (epan/frame_data.h) - one structure instance  
per packet;
	the text for some or all of the columns in all of the rows of the  
packet list (all, in current releases of Wireshark; some, in the  
development branch);
	various per-packet private data attached to some frames by dissectors;

	various per-dissector private data structures;

	the results of reassembly.

The data from the frames in the capture file are not kept in Wireshark's address space - they are read in as necessary, into a small number of buffers (one for the main window, and one for each packet window opened). *HOWEVER*, if data from a frame is reassembled into a higher-level multiple-frame packet, the result of the reassembly is, as noted, kept in Wireshark's address space.
I write "problem" in quotes, because in most use cases I guess that
this is not a problem at all,
People complain about it enough that, while in *most* cases it might  
not be a problem, we frequently get mail from people who have to split  
up capture files to read them - I'd call it enough of a problem that  
we should work on it (ideally, by reducing the amount of address space  
required by the aforementioned data items).
So, on OS X (and possibly other modern OS-es), as long as you have available harddisk space, a process will not run out of memory, ever? (A process can
have address space of ~18 exabytes on 64-bit OS X. [1])
On modern OSes that support 64-bit userland, and that can dynamically  
grow the swap space, running on a machine with enough disk space, I  
would expect that a process running 64-bit would not run out of  
address space unless there are some in-kernel data structures that  
scale with address space size and the kernel's maximum address space  
size limits the size of those data structures enough to stop address  
space growth before you hit another limit.  32-bit programs aren't  
going to be so lucky.