Wireshark · Wireshark-dev: Re: [Wireshark-dev] Anyone working on a Syncthing dissector?

[Antoine d'Otreppe <a.dotreppe@xxxxxxxxxx>]

Thanks for the additional input, github links and discussions. I took some time to explore the possible roads.

(long-ish mail ahead, feel free to skip down to the conclusion)

I tried making a very simple dissector in lua, reusing the existing protobuf dissector from wireshark.

The protobuf dissector works just fine, but the output is too generic to be useful.
It shows the fields, their respective numeric tag, length and hex value.
But I'm really interested in 3 pieces of info: ID, addresses and instance ID.

Added to that, the syncthing protocols are not pure protobuf. They mix fixed width data with protobuf in various ways. That causes a few side effects that make using the protobuf dissector not ideal for display.

I also had a look at the merge requests pointed out by Anders, but if I'm correct they haven't been merged, are about 1 year old each and now conflicted. I'm not sure how much work would be required to fix them.
That, combined with the fact that there's already support for protobuf and grpc in wireshark, makes me think they're just too much trouble for their value.

After looking through some of the code from wireshark, I found the following:

- packet-protobuf.c, which seems to be the protobuf dissector I used from the lua script
- tvbuff.c contains some functions to parse protobuff data, like tvb_get_varint()

I'm pretty sure I can reuse the functions available in tvbuff.
From packet-protobuf, on the other hand, I don't think I can readily reuse anything because:

1. not much is accessible from the header file, and is probably better kept private anyway
2. most of the functions will manipulate and add data to the proto_tree, which probably means refactoring would be needed to reuse the code.

However it's going to be a pretty good source of information to build the syncthing dissector.

Finally, I tried generating code with the protoc-c-compiler, but this results in a dependency to protobuf headers, which I couldn't find bundled in wireshark, so I'm assuming they're not available. Since a protobuf dissector exists, but the protobuf headers are not in the sources, I'm guessing this is a choice made in the past that we should respect. (Or should we?) Actually there is at least 1 other dissector using protobuf at the moment: packet-riemann.c

Ok this email is long enough, sorry about that. So here's the plan:

1. write the dissector in c, possibly in epan/dissectors/packet-syncthing.c
2. use the functions available in tvbuff to parse the protobuf in the syncthing packets, instead of adding a dependency to protobuf-c headers.
3. do not modify any existing code. I don't know much about the wireshark codebase, so I could easily create nasty side effects.

Does that sound sensible? Am I missing something, or would you see a better solution?

Cheers,
Antoine

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐

On Friday, November 9, 2018 10:23 AM, Martin Mathieson via Wireshark-dev  wrote:

Hi,

I have a simple dissector (private) which just calls protoc --decode, reads the output from a pipe and shows the output using "data-text-lines".  In my environment, I have quite a few protobuf protocols that change often.

I basically have a table whose columns are:

- .UDP port number

-  proto file name

- top-level protobuf message name

And I have a preference that points to the folder that contains the .protof files and protoc.

I automatically update the list of UDP ports the dissector listens on in the handoff function.  The dissector looks up by port number and calls protoc with the appropriate arguments.

I am guessing I am unusual in having multiple 'unstable' protobuf-based protocols to support on not well-known ports?  Mine is a different use-case from having a public, stable protocol on a well-known port, but I still want to be able to see the details of the decode.

Regards,

Martin

On Fri, Nov 9, 2018 at 9:03 AM Maciej Krüger  wrote:

Hi,

I have written a still WIP (but mostly abandoned) dissector for libp2p

which also uses protobuf.

https://github.com/mkg20001/libp2p-dissector

This might give you some inspiration. Especially the CMakeLists.txt

could be useful

https://github.com/mkg20001/libp2p-dissector/blob/master/CMakeLists.txt#L49-L80

I also am using a patched version of protobuf-c which allows getting the

offsets for each of fields so they can be highlighted in the UI easily:

https://github.com/mkg20001/libp2p-dissector/blob/master/packet-secio.c#L309-L315

Maciej

Am 09.11.18 um 09:52 schrieb Antoine d'Otreppe:

> Hi Peter, hi Anders,

>

> Thanks for the pointers! I'll look into it and report back here when I have more information on the topic.

>

>

> Cheers,

> Antoine

>

>

>

>

> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐

> On Friday, November 9, 2018 9:32 AM, Anders Broman  wrote:

>

>>

>> -----Original Message-----

>>

>>> From: Wireshark-dev wireshark-dev-bounces@xxxxxxxxxxxxx On Behalf Of Peter

>>> Wu

>>> Sent: den 9 november 2018 00:22

>>> To: Antoine d'Otreppe a.dotreppe@xxxxxxxxxx; Developer support list for

>>> Wireshark wireshark-dev@xxxxxxxxxxxxx

>>> Subject: Re: [Wireshark-dev] Anyone working on a Syncthing dissector?

>>> Hi Antoine!

>>> Based on the specifications for Syncthing, it appears that it uses Protobuf

>>> for defining its messages:

>>> https://docs.syncthing.net/specs/

>>> I am not sure how well protobuf is currently supported on Wireshark, you

>>> could scan the issue tracker and code review site to see if there is any

>>> current work in that area.

>>> Kind regards,

>>> Peter

>>> https://lekensteyn.nl

>>> (pardon my brevity, top-posting and formatting, sent from my phone)

>> Hi,

>> I think these pending commits are relevant:

>> https://code.wireshark.org/review/#/c/22892/

>> https://code.wireshark.org/review/#/c/23988/

>>

>> Regards

>> Anders

>>

>> On November 8, 2018 9:32:50 PM GMT+01:00, Antoine d'Otreppe

>> a.dotreppe@xxxxxxxxxx wrote:

>>

>>> Hi there,

>>> I'm interested in learning more about wireshark in general, and in

>>> particular learning how to make my own dissectors.

>>> I just happened to find a protocol that doesn't seem to have any

>>> dissector for it yet: syncthing. https://syncthing.net/ The local

>>> discovery protocol looks easy enough to begin with, as it is only UDP

>>> broadcasts.

>>> Your developer guide recommends to send a mail before starting

>>> development to check if anyone else would be working on a similar

>>> topic. That sounds reasonable :)

>>> Anybody working on that protocol yet?

>>> Regards,

>>> Antoine d'Otreppe

>> Sent via: Wireshark-dev mailing list wireshark-dev@xxxxxxxxxxxxx

>> Archives: [https://www.wireshark.org/lists/wireshark-dev](<a href=)">https://www.wireshark.org/lists/wireshark-dev";>[https://www.wireshark.org/lists/wireshark-dev](<a href=)">https://www.wireshark.org/lists/wireshark-dev";>[https://www.wireshark.org/lists/wireshark-dev](<a href=)">https://www.wireshark.org/lists/wireshark-dev";>[https://www.wireshark.org/lists/wireshark-dev](<a href=)">https://www.wireshark.org/lists/wireshark-dev

>> Unsubscribe: [https://www.wireshark.org/mailman/options/wireshark-dev](<a href=)">https://www.wireshark.org/mailman/options/wireshark-dev";>[https://www.wireshark.org/mailman/options/wireshark-dev](<a href=)">https://www.wireshark.org/mailman/options/wireshark-dev";>[https://www.wireshark.org/mailman/options/wireshark-dev](<a href=)">https://www.wireshark.org/mailman/options/wireshark-dev";>[https://www.wireshark.org/mailman/options/wireshark-dev](<a href=)">https://www.wireshark.org/mailman/options/wireshark-dev

>> mailto:wireshark-dev-request@xxxxxxxxxxxxx?subject=unsubscribe

>>

>> Sent via: Wireshark-dev mailing list wireshark-dev@xxxxxxxxxxxxx

>> Archives: [https://www.wireshark.org/lists/wireshark-dev](<a href=)">https://www.wireshark.org/lists/wireshark-dev";>[https://www.wireshark.org/lists/wireshark-dev](<a href=)">https://www.wireshark.org/lists/wireshark-dev";>[https://www.wireshark.org/lists/wireshark-dev](<a href=)">https://www.wireshark.org/lists/wireshark-dev";>[https://www.wireshark.org/lists/wireshark-dev](<a href=)">https://www.wireshark.org/lists/wireshark-dev

>> Unsubscribe: [https://www.wireshark.org/mailman/options/wireshark-dev](<a href=)">https://www.wireshark.org/mailman/options/wireshark-dev";>[https://www.wireshark.org/mailman/options/wireshark-dev](<a href=)">https://www.wireshark.org/mailman/options/wireshark-dev";>[https://www.wireshark.org/mailman/options/wireshark-dev](<a href=)">https://www.wireshark.org/mailman/options/wireshark-dev";>[https://www.wireshark.org/mailman/options/wireshark-dev](<a href=)">https://www.wireshark.org/mailman/options/wireshark-dev

>> mailto:wireshark-dev-request@xxxxxxxxxxxxx?subject=unsubscribe

>

> ___________________________________________________________________________

> Sent via:    Wireshark-dev mailing list

> Archives:    [https://www.wireshark.org/lists/wireshark-dev](<a href=)">https://www.wireshark.org/lists/wireshark-dev";>[https://www.wireshark.org/lists/wireshark-dev](<a href=)">https://www.wireshark.org/lists/wireshark-dev";>[https://www.wireshark.org/lists/wireshark-dev](<a href=)">https://www.wireshark.org/lists/wireshark-dev";>[https://www.wireshark.org/lists/wireshark-dev](<a href=)">https://www.wireshark.org/lists/wireshark-dev

> Unsubscribe: [https://www.wireshark.org/mailman/options/wireshark-dev](<a href=)">https://www.wireshark.org/mailman/options/wireshark-dev";>[https://www.wireshark.org/mailman/options/wireshark-dev](<a href=)">https://www.wireshark.org/mailman/options/wireshark-dev";>[https://www.wireshark.org/mailman/options/wireshark-dev](<a href=)">https://www.wireshark.org/mailman/options/wireshark-dev";>[https://www.wireshark.org/mailman/options/wireshark-dev](<a href=)">https://www.wireshark.org/mailman/options/wireshark-dev

>              mailto:wireshark-dev-request@xxxxxxxxxxxxx?subject=unsubscribe

___________________________________________________________________________

Sent via:    Wireshark-dev mailing list

Archives:    [https://www.wireshark.org/lists/wireshark-dev](<a href=)">https://www.wireshark.org/lists/wireshark-dev";>[https://www.wireshark.org/lists/wireshark-dev](<a href=)">https://www.wireshark.org/lists/wireshark-dev";>[https://www.wireshark.org/lists/wireshark-dev](<a href=)">https://www.wireshark.org/lists/wireshark-dev";>[https://www.wireshark.org/lists/wireshark-dev](<a href=)">https://www.wireshark.org/lists/wireshark-dev

Unsubscribe: [https://www.wireshark.org/mailman/options/wireshark-dev](<a href=)">https://www.wireshark.org/mailman/options/wireshark-dev";>[https://www.wireshark.org/mailman/options/wireshark-dev](<a href=)">https://www.wireshark.org/mailman/options/wireshark-dev";>[https://www.wireshark.org/mailman/options/wireshark-dev](<a href=)">https://www.wireshark.org/mailman/options/wireshark-dev";>[https://www.wireshark.org/mailman/options/wireshark-dev](<a href=)">https://www.wireshark.org/mailman/options/wireshark-dev

             mailto:wireshark-dev-request@xxxxxxxxxxxxx?subject=unsubscribe