Doug,
I was thinking of going that route, but I really
only need the current days worth of traffic. I changed the ring buffer limit
and re-complied Wireshark. I now have the capture running and it got past the
10000 limitation and currently is moving towards the 20000 mark now. For
someone like me that had no idea on how to compile code the change was very
easy and didn’t require me to re-invent the back-end processes that we use to
find sequence number of the market data that we are capturing. I built a whole
web console on a linux machine that searches multiple sniffer machines and uses
tshark to read the capture files and parse out the sequence number that we are
looking for. The last piece of this puzzle was get the data captured for the
whole entire day, instead of 4 hours at a time. The next thing I’m looking to
figure out is a dissector for the market data that we capture, ie: NASDAQ,
NYSE, ARCA, CME, etc…
Thanks
Joe
From:
wireshark-users-bounces@xxxxxxxxxxxxx
[mailto:wireshark-users-bounces@xxxxxxxxxxxxx] On Behalf Of Douglas Ross
Sent: Thursday, May 20, 2010 6:26 PM
To: Community support list for Wireshark
Subject: Re: [Wireshark-users] tshark or dumpcap ring buffer limitations
Have
you considered compressing the capture files?
If
two of your concerns are the huge amount of disk space and consequent
network traffic you need to manage this data?
I'm
very new to Wireshark, but have used Ethereal in the past,
and periodically compressed capture files and deleted the originals.
(eg.
WinRar achieves a compression ratio of about 10/1)
Anyway,
I made a script to automatically compress daily files into two archives per
month. This not only reduced disk usage, but also dramatically reduced the
number of files/folders.
If
1000 ring files is the hard coded limit, then auto compress and delete after
every 500 or 800.
If
"0" ring files implies no limit, then auto compress after
whatever number is most convenient.
I
used the capture file dts (date.time stamp) to determine which half-month
archive it should be put in. eg:
...20100501...
to ...20100515... I'd put in archive ...201051
...20100516...
to ...20100531... I'd put in archive ...201052
(undoubtedly
you'd make a finer split, perhaps into one or two archives per day ..)
(capture
file name includes start of capture dts; system dts is at
close of file (= start of next))
Hopefully,
that will help solve the problems of volume of files, and waste of disk space
and network capacity.
Similarly,
I used scripts to help decompress whichever file I needed, based on dts.
Hope
this helps for a relatively quick fix, at least to give you some ideas.
PS.
while you're scripting this (if you go down that route) you could consider
doing first pass analysis, and filter out the stuff you're not interested
in, and/or split the capture into known good and useful stuff; definite
rubbish(discard); and possible trouble...
From: Jeff Morriss
<jeff.morriss.ws@xxxxxxxxx>
To: Community support list for Wireshark
<wireshark-users@xxxxxxxxxxxxx>
Sent: Fri, 21 May, 2010 3:34:54 AM
Subject: Re: [Wireshark-users] tshark or dumpcap ring buffer limitations
Joseph Laibach wrote:
> All,
>
> I’m running a
continuous capture of data. I’m trying to
> use a ring buffer of 25000 files with an 8mb file size. The problem is
> that the ring buffer starts overwriting after 10000 files. I’ve tried it
> with dumpcap and tshark. The command is using the –b files:25000 –b
> filesize:8192. Is there a limitation to the size of the ring buffer for
> dumpcap and/or tshark?
Turns out that if you specify the number of files as 0 then
dumpcap/*shark will create an unlimited number of files. I don't know
if that's acceptable or if you really need it to roll over at 25,000,
but it's an option.
___________________________________________________________________________
Sent via: Wireshark-users mailing list <wireshark-users@xxxxxxxxxxxxx>
Archives: http://www.wireshark.org/lists/wireshark-users
Unsubscribe: https://wireshark.org/mailman/options/wireshark-users
mailto:wireshark-users-request@xxxxxxxxxxxxx?subject=unsubscribe
This communication is for informational purposes only. It is not intended as an offer or solicitation or as an official confirmation. Market prices and other information are not guaranteed as to completeness or accuracy and are subject to change without notice. Schonfeld Group reserves the right to monitor and review the content of all messages sent to or from this e-mail address.
|
