-----Original Message-----
From: wireshark-dev-bounces@xxxxxxxxxxxxx
[mailto:wireshark-dev-bounces@xxxxxxxxxxxxx] On Behalf Of Bill Meier
Sent: den 6 oktober 2009 00:49
To: Developer support list for Wireshark
Subject: Re: [Wireshark-dev] New packet list - out of memory?
Bill Meier wrote:
>> A data point: A few days ago while working on bug #2375 I noticed
that
>> on Windows that if I defined the env logicals to not use se_alloc &
>> etc that *much* less memory was required to load a capture. (20% of
>> that used with se_alloc & etc ? I don't quite remember).
>>
>
>Details:
>
>Env: Windows Vista
>
>Actions:
>
>1. Load Wireshark
> Determine "Private Working Set", "Working Set", "Commit Size"
>
>2. Load a 40 Meg capture file.
> Determine .... again
>
>The above was done for:
>
>a. Wireshark with new_packet_list:
>
>b. Wireshark w/new_packet_list:
> set WIRESHARK_DEBUG_EP_NO_CHUNKS=T
> set WIRESHARK_DEBUG_SE_NO_CHUNKS=T
>
>c. Wireshark without new packet list
>
>d. Wireshark without new packet list
> set WIRESHARK_DEBUG_EP_NO_CHUNKS=T
> set WIRESHARK_DEBUG_SE_NO_CHUNKS=T
>
>Measurements
>
>Note: The Working Set was always about 20 Meg larger than the
> Private Working Set(PWS) so only the PWS is shown.
>
>(All values in Megs)
> EP/SE Chunks No Chunks
> PWS Commit PWS Commit
> ---- ------- ----- -------
>Wireshark: current_packet_list
> After startup: 42 62 42 52
> After 40M CF load: 180 399 195 205
>
>
>Wireshark: new_packet_list:
> After startup: 42 62 42 52
> After 40M CF load: 136 564 165 177
>
>
>A conclusion:
>
>Commit Size (which I believe is the amount of VM address space used
> by the process and which is presumably the limiting factor for
> memory usage).
> - Best for new_packet_list/no_chunks;
> - worst (quite) for new_packet_list/chunks;
>
>Comments welcome...
>
>(I did these tests somewhat quickly so I could have missed
something...)
>
>Bill
Hi,
I expanded on Bills measurements using the file that caused out of
memory for the new packet list. (Thanks Bill I didn't know about the
commit size option in the task manager).
146 MB 661417 pkts (TCP reassembly)
Version 1.2.2 (SVN Rev 29910)
PWS Commit
42 016 61 876
704 656 1 639 256
Loading time ~57s
Old packet list SVN 30353
PWS Commit
43 060 63 384
492 456 1 427 480
Loading time 47.752 - 48.485 (2 Loading time measures)
Old packet list SVN 30353
set WIRESHARK_DEBUG_SE_NO_CHUNKS=T
PWS Commit
43 060 63 384
559 744 579 968
Loading time 48.797
Old packet list SVN 30353
set WIRESHARK_DEBUG_EP_NO_CHUNKS=T
set WIRESHARK_DEBUG_SE_NO_CHUNKS=T
43 208 53 312
559 612 569 632
Loading time 55.162
Old packet list SVN 30353
#define EMEM_ALLOCS_PER_CHUNK (EMEM_PACKET_CHUNK_SIZE / 64)
PWS Commit
43 288 64 188
492 104 567 060
Loading time 47.736 - 48.391 (2 Loading time measures)
(This file can't be loaded using 512)
New packet list SVN 30353
#define EMEM_ALLOCS_PER_CHUNK (EMEM_PACKET_CHUNK_SIZE / 64)
PWS Commit
43 492 64 512
332 400 471 264
Loading time 55.895 - 57.018 (4 Loading time measures)
New packet list SVN 30353
#define EMEM_ALLOCS_PER_CHUNK (EMEM_PACKET_CHUNK_SIZE / 64)
set WIRESHARK_DEBUG_SE_NO_CHUNKS=T
PWS Commit
43 660 64 736
468 496 493 660
Loading time 53.666
New packet list SVN 30353
#define EMEM_ALLOCS_PER_CHUNK (EMEM_PACKET_CHUNK_SIZE / 64)
set WIRESHARK_DEBUG_EP_NO_CHUNKS=T
set WIRESHARK_DEBUG_SE_NO_CHUNKS=T
PWS Commit
43 660 64 736
468 428 481 516
Loading time 1:01.215 - 1:02.682 (2 Loading time measures)
Compared with 1.2.2 memory usage has improved quite a bit. Loading
time
has improved with the old packet list but not so much with the new
packet list!?
Conclusions:
- se alloced memory is a problem as the allocation "waists" memory.
This is not a big problem for ep memory as it's given back to the pool
after each packet.
- Using the old packet list the file loads faster !?
This was not the case some way along the line...
- Surprisingly enough the file loads faster not using chunks for
se_alloc ???
Whereas it is faster using ep memory.
Increasing EMEM_ALLOCS_PER_CHUNK looks like a good thing - drawbacks?
Should the use of se_alloc be discouraged/removed?
Regards
Anders
________________________________________________________________________
___
Sent via: Wireshark-dev mailing list <wireshark-dev@xxxxxxxxxxxxx>
Archives: http://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev
mailto:wireshark-dev-request@xxxxxxxxxxxxx?subject=unsubscribe
