Wireshark · Wireshark-dev: Re: [Wireshark-dev] Packet Loss due to Disk Contention with Running Dumpcap in a high packet rate en

[John Powell <jrp999@xxxxxxxxx>]

Hi Ronnie,

I am capturing a 250 MB file every few seconds.  My ATOP reports:

MDD |          md2 | busy      0% | read       1  | write  15442 | KiB/r      4 | KiB/w      4 | MBr/s   0.00 | MBw/s  60.32  | avq     0.00 | avio 0.00 ms |

DSK |          sda | busy    107% | read       1  | write    205 | KiB/r      4 | KiB/w    506 | MBr/s   0.00 | MBw/s 101.33  | avq    93.88 | avio 4.51 ms |
DSK |          sdb | busy     92% | read       0  | write    191 | KiB/r      0 | KiB/w    511 | MBr/s   0.00 | MBw/s  95.50  | avq    86.84 | avio 4.20 ms |

I need the resulting files to be searchable by TSHARK and be able to create a PCAP extraction based on the search.

The dumpcap command being used is:

usr/local/bin/dumpcap -B 16 -i 4 -f vlan and (not vrrp and not udp port 1985 and not ether host 01:00:0c:cc:cc:cc) -g -b filesize:250000 -b duration:900 -w /data/eth2.cap

I am looking at using a SSD for my OS and my Capture volume which may help out with the Disk IO issue but eliminating the copy from the /TMP would definitely be an asset.

Thanks in advance.

-John

On Wed, Dec 12, 2012 at 1:52 PM, ronnie sahlberg <ronniesahlberg@xxxxxxxxx> wrote:

How high bandwidths are we talking about here ?


Without going to expensive higher end storage, the fastest is probably
if you can use a dedicated disk and stream directly to the raw disk.
This should avoid any of the seeks that a filesystem would induce when
writing to both file, journal and the fs metadata.
And should allow you to capture at near the peak sequential write
speed of your spindle.


regards
ronnie sahlberg

On Wed, Dec 12, 2012 at 11:33 AM, John Powell <jrp999@xxxxxxxxx> wrote:
> Hi Everyone,
>
> I am using DUMPCAP to capture packets in a high packet rate environment.
>
> My operating system is: CENTOS 6.3
>
> I am experience this problem on source compiled versions:  wireshark-1.6.12
> and wireshark-1.8.4.
>
> In order to allow DUMPCAP to be run as a NON-ROOT user I am using the
> following:
>
> setcap 'CAP_NET_RAW+eip CAP_NET_ADMIN+eip' /usr/local/bin/dumpcap -v
>
> The issue is that I am experiencing packet loss to apparent disk contention
> when writing the packets to the disk - see attached file:
> packet-loss-atop.txt
>
> To help alleviate the problem I have tried the following:
>
> Disabled SELINUX
> Disabled AUDIT
> RAID 0 (striped disks) to load share the writing out of the data
>
> ARRAY /dev/md2 level=raid0 num-devices=2
>    devices=/dev/sda4,/dev/sdb4
>
> Turn off journals on ext4
>
> tune2fs -o journal_data_writeback /dev/md2
> tune2fs -O ^has_journal /dev/md2
> change fstab to:
>
> UUID=.. /data   ext4    defaults,data=""         0 0
>
> Use -B option on Dumpcap to buffer the data
>
> root      /usr/local/bin/dumpcap -B 16 -i 2 -f vlan and (not vrrp and not
> udp port 1985 and not ether host 01:00:0c:cc:cc:cc) -g -b filesize:250000 -b
> duration:900 -w /data/eth1.cap
>
> These changes have increased the throughput but I still experience packet
> loss - see attached IO Graph: packet-loss-io-graph.jpg
>
> The Vendor solutions we have looked at will not decode UNISTIM signalling
> properly which is requirement for this tool.
>
> Any suggestions on how to better configure either the operating system or
> wireshark to increase packet capture throughput will be greatly appreciated.
>
> Thanks in advance for your assistance.
>
> -John
>

> ___________________________________________________________________________
> Sent via:    Wireshark-dev mailing list <wireshark-dev@xxxxxxxxxxxxx>
> Archives:    http://www.wireshark.org/lists/wireshark-dev
> Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev
>              mailto:wireshark-dev-request@xxxxxxxxxxxxx?subject=unsubscribe
___________________________________________________________________________
Sent via:    Wireshark-dev mailing list <wireshark-dev@xxxxxxxxxxxxx>
Archives:    http://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev
             mailto:wireshark-dev-request@xxxxxxxxxxxxx?subject=unsubscribe