[pcap-ng-format] Proposal for new "custom" option codes

[Jasper Bongertz]

Wednesday, July 22, 2015, 9:13:49 PM, Guy Harris wrote:

> On Jul 22, 2015, at 11:37 AM, Jasper Bongertz <jasper at packet-foo.com> wrote:

>> Yeah, that's where the headache starts, because you need to find out
>> which interfaces you have in the input file and which ones of those
>> you really need in the output file. A file should not contain IDBs
>> that are not referenced by at least one output packet.

> That requires two passes through the input files when merging or filtering files.

> It also requires that any program that's capturing, even if it's
> only on one interface, not write out IDBs until the first packet is seen for the interface.

> And it means that, if you were capturing on interfaces A, B, and C,
> but didn't happen to see any packets on interface C (because they
> didn't pass the filter, or whatever), the fact that you were
> capturing on interface C isn't recorded anywhere in the file.

You're right, I didn't think of that. And having IDBs at the beginning
of the file has advantages as they can be read without scanning the
whole file first. Which may still be required in some situations where
the IDBs are spread across the file, of course.

> So I wouldn't support "A file should not contain IDBs that are not
> referenced by at least one output packet." as a requirement or even
> a "nice to have".  I'd say "a file should not have IDBs that refer
> to an interface that wasn't used in any of the capture processes
> from which the data in this capture file originally came".

Agreed.

>> Also, you may want to merge identical IDBs in multiple input files
>> into a single IDB in the output file, which is easy for Windows
>> captures because interface names are based on GUIDs and a game of
>> chance for all other captures.

> That's a problem with IDBs, but, even if you have an XYZZY
> Description Block that mentions the existence of an XYZZY, giving
> each XYZZY a UUID so as making duplicate recognition easy, and an
> XYZZY option for packet description that refers back to an XYZZY
> associated with the packet by an ordinal ID similar to interface
> IDs, merging files would *still* require plugins to handle the XYZZY
> Description Block and XYZZY option to renumber the ordinal IDs.

> That might be an argument *against* ordinal IDs and *for* assigning
> UUIDs *and* using the UUID in the option.

Ordinal IDs are still useful since it's much more space efficient to
reference them in the packet blocks.

I guess merging could allow the users to specify if and how IDBs are
to be merged. Worst case it's always possible to simply write all IDBs
to file and not bothering with merging them.

Cheers,
Jasper
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3681 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://www.winpcap.org/pipermail/pcap-ng-format/attachments/20150722/f8ad348a/attachment.bin>