Hi Guy,
On Fri, Oct 23, 2020 at 10:21:36AM -0700, Guy Harris wrote:
> On Oct 17, 2020, at 7:25 AM, Harald Welte <laforge@xxxxxxxxxxxx> wrote:
>
> > I'm currently facing a problem where I need to create pcap files
>
> Or, at least, *some* form of capture file....
correct.
> Yes, those were some of the goals of pcapng:
excellent.
> We (the libpcap developers) would like to provide APIs to:
Thanks for sharing.
> (For the third of those, Michael Richardson has been talking about other changes to support faster writing of capture files, including support for asynchronous I/O.)
sidenote: I've just been doing some development work with io_uring
(using liburing) on modern Linux systems, and it's amazing in terms of
performance of asynchronous I/O. Might be worth investigating.
> libwiretap, which is a general library for reading and writing many capture file formats, including pcap, pcapng, and *other* formats;
Thanks, I was familiar with that one a high level.
> libwritecap, which is a much smaller and simpler library, used by dumpcap, to write pcap and pcapng files.
Didn't know that yet.
> > Furthermore, when starting a cooked Linux capture on the Linux 'any' device,
> > it also appears wireshark is not displaying the information about which netdevice
> > the message was captured.
>
> Yes. Fixing that would be part of the "get additional information when capturing, to put into pcapng files" item above.
Now I'm a bit confused. You indicated libpcap doesn't have those APIs
yet, but dumpcap uses libwirestap. Now you say that wireshark (which,
AFAIK, is using dumpcap for captures) suffers from that constraint.
> Currently, dumpcap doesn't work around the absence of support for that in libpcap; it probably *could* do so.
So dumpcap uses libwiretap, and libwiretap uses libpcap? I should go and
read the code.
> (This is, by the way, one of the reasons why the pcapng spec does *not* require that all Interface Description Blocks appear at the beginning of the file - if you capture on the "any" device, new devices may appear while the capture is in progress, and, at some point before the first packet on the new interface is written to a pcapng file, an IDB for that interface would have to be written, if we're writing separate IDBs for each interface rather than just writing a single IDB for the "any" interface.)
Sure.
> > As far as I know, on AF_PACKET sockets one can do recvmsg() and will then get
> > a sockaddr_ll structure alongside the actual packet, which contains the ifindex
> > of the underlying network deivce. Together with the usual sockopt or netlink
> > based method that can be trnaslated to a device name.
>
> It can, although you can also do memory-mapped capture, which is what current versions of libpcap do. The information in question is provided by memory-mapped captures as well.
Thanks for confirming the latter. I was aware of mmap'ed AF_PCAP, just
not sure libpcap is actually using it.
> > Am I missing something? Is there a specific reason why this information is not
> > obtained/displayed or written when writing an output file, even in pcap-ng mode?
>
> Yes. The reason is "nobody's written the code to do so yet". :-)
Thanks. I'm not trying to imply there's anything bad about that. It
just somehow amazes me in what "lacking" state some of the most
fundamental tools are. I also think the entire pcap-ng development came
way too late. The entire internet community is using pcap for decades,
and the need for a more modern/capable file format than the good old
pcap is hardly something that only became apparent recently.
Once again, this is not a complaint, it is just me noticing something I
am quite surprised about. Wireshark has been around a long time, as has
packet capturing. Probably millions of users out there, a very active
community, ... - and yet something seemingly mundane as recording the
id/name of the interface is still missing in 2020.
I know those kind of problems typically from much small and much more
niche FOSS projects, without much commercial/professional user base. All of
which clearly does not apply to the wireshark community, or the wider
community of users and developers of pcap related software and systems.
In any case, thanks for your time.
Regards,
Harald
--
- Harald Welte <laforge@xxxxxxxxxxxx> http://laforge.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
(ETSI EN 300 175-7 Ch. A6)
