Huge thanks to our Platinum Members Endace and LiveAction,
and our Silver Member Veeam, for supporting the Wireshark Foundation and project.

Wireshark-dev: Re: [Wireshark-dev] pcapng / interface names / OPT_IDB_NAME

From: Guy Harris <gharris@xxxxxxxxx>
Date: Sat, 24 Oct 2020 13:23:57 -0700
On Oct 24, 2020, at 10:16 AM, Harald Welte <laforge@xxxxxxxxxxxx> wrote:

> On Fri, Oct 23, 2020 at 10:21:36AM -0700, Guy Harris wrote:
>> On Oct 17, 2020, at 7:25 AM, Harald Welte <laforge@xxxxxxxxxxxx> wrote:
>> 
>>> I'm currently facing a problem where I need to create pcap files
>> 
>> (For the third of those, Michael Richardson has been talking about other changes to support faster writing of capture files, including support for asynchronous I/O.)
> 
> sidenote: I've just been doing some development work with io_uring
> (using liburing) on modern Linux systems, and it's amazing in terms of
> performance of asynchronous I/O.  Might be worth investigating.

Might be, although we'd need to either:

1) figure out a way to do that, while hiding the platform-dependent details, on *all* (currently living) platforms supported by libpcap:

	macOS;
	the *BSDs;
	Solaris;
	HP-UX;
	AIX;
	Windows;
	Linux;

which don't all have the same asynchronous I/O mechanisms (POSIX aio on most if not all of the UN*Xes, "overlapped I/O" or whatever Microsoft calls it on Windows)

or

2) arrange that the user may, but need not, provide their own low-level writing code that the new writing APIs call, so they can either use a platform-independent mechanism supplied by libpcap or write their own code.  I think Michael Richardson has been thinking of something such as that.

>>> Furthermore, when starting a cooked Linux capture on the Linux 'any' device,
>>> it also appears wireshark is not displaying the information about which netdevice
>>> the message was captured.
>> 
>> Yes.  Fixing that would be part of the "get additional information when capturing, to put into pcapng files" item above.
> 
> Now I'm a bit confused.  You indicated libpcap doesn't have those APIs
> yet, but dumpcap uses libwirestap.  Now you say that wireshark (which,
> AFAIK, is using dumpcap for captures) suffers from that constraint.

dumpcap uses libpcap for *capturing* and libwritecap (not libwiretap) for *writing capture files*.

>> Currently, dumpcap doesn't work around the absence of support for that in libpcap; it probably *could* do so.
> 
> So dumpcap uses libwiretap, and libwiretap uses libpcap? I should go and
> read the code.

No - see previous comment.

What *might* be possible to do, in the absence of new libpcap capture APIs, would be to have dumpcap, when capturing from the "any" device on Linux and writing to a pcapng file:

	when the capture starts, write out Interface Description Blocks (IDBs) for all the currently-known interfaces on the system, and make a table mapping from the kernel's interface indices (ifIndexes, in SNMP terms) to interface IDs for those IDBs;

	when a packet arrives, look up its interface index of the packet, and:

		if it's found, write the packet out with that interface index;

		if it's *not* found, write out an IDB for the new interface, add it to the table, and write the packet out with that interface index.

>> It can, although you can also do memory-mapped capture, which is what current versions of libpcap do.  The information in question is provided by memory-mapped captures as well.
> 
> Thanks for confirming the latter.  I was aware of mmap'ed AF_PCAP,

(PF_PACKET/AF_PACKET)

> just not sure libpcap is actually using it.

Yes.  The master branch of libpcap *only* uses it - it dropped support for non-memory-mapped capture on Linux.