Wireshark-dev: Re: [Wireshark-dev] pcap-ng support
From: Ulf Lamping <[email protected]>
Date: Tue, 22 Jan 2008 00:20:17 +0100
Stephen Donnelly schrieb:
On Mon, 2008-01-21 at 22:00 +0100, Ulf Lamping wrote:
Gianluca Varenni schrieb:
I think the description of timestamp formats is quite bad in the specs.
The timestamps are represented as a 64bit quantity split into high and low 32 bits, that represent the number of microseconds/nanoseconds/??? from 1/1/1970 (that's the meaning of in "in standard unix format i.e. since 1/1/1970"). The reason behind using a single 64bit quantity instead of seconds/subseconds is twofold: 1. if we use seconds and subseconds, 32bits don't allow to go under the nanosecond. 2. several hardware-based capture cards represent timestamps as nanoseconds/microseconds as a single 64bit quantity i.e. they don't split them into seconds and subseconds.
BTW, there was a discussion on the timestamp format on the ntar-workers 
mailing list, here
http://www.winpcap.org/pipermail/ntar-workers/2006-March/000122.html
Yes, the timestamp spec of the EPB (and PB) is *very misleading* here and definitely needs a clarification! The structure - and the descriptive text - looks far too much "libpcap like" to get an idea that it's actually different.
Reading the text a few times now, I think it's even not very consistent 
in itself ...
    
I believe part of the idea behind allowing the timestamp precision to be
specified as a binary fraction (2^-X) is to allow the use of Endace ERF
timestamps in pcapng/ntar natively.
Well, my complain in the first place is that the specification text is pretty easy to misunderstand (just as I did), which is usually a very bad thing for a specification.
Timestamps in network traces are typically 64-bit, comprising a 32-bit
'seconds' part with a UNIX epoch, and a 32-bit 'subsecond' field
counting milli/micro/nanoseconds withing the second. The most
significant few bits of the subsecond are not used as the conuter
saturates before using all the available bits.
I'm pretty much aware of that - I'm the author of http://wiki.wireshark.org/Development/LibpcapFileFormat and I've implemented the Wireshark switch from millisecond to nanosecond internal resolution :-)
ERF timestamps are also 64-bits, and can be considered to be a 64-bit
value in units of 2^-32 seconds, with a UNIX epoch. Alternatively it can
be considered a fixed point count of seconds from the UNIX epoch.
Conveniently the 32 MSB are equivalent to the conventional 'seconds'
count, while the 32 LSB can be considered a binary fraction of seconds.
The lowest few LSB may not be used by specific hardware depending on the
available clock resolution.
Thanks for explaining it. But it just doesn't really help to explain it here, it must be explained in the spec! Otherwise the next one that graps the spec and implements stuff will be trapped.
Advantages include the ability to perform a single 64-bit integer
subtraction to find timestamp differences, and the ability to support
different hardware clock resolutions without changing timestamp format
or interpretation.

Using ERF timestamps directly in pcapng avoids the requirement to
convert the timestamp formats on the fly at capture time. This can be a
significant load at extreme packet rates.
Full ACK.

But as a drawback, now the former libpcap timestamp format is not directly supported and needs conversion "on the fly" while writing packet data.
It does suggest that timestamp
accessor functions should be available so that the any timestamp
(record/block) can be requested in any format.
That reminds me of the big/little endian problem, which is solved by writing stuff in the native format and being able to detect and then read (and probably convert) whatever comes in.
For timestamps this means: write libpcap, ERF, or whatever comes in and 
being able to detect and then read (and probably convert) the timestamp 
format.
I'm pretty much aware that this adds complexity to the programs reading 
the format, but as you've said writing performance is the key.
Regards, ULFL