[ntar-workers] Seekable file layouts etc

[Alexander Dupuy]

Gianluca Varenni writes:
> Well, in any case if a trace file is truncated (for any reason, download
> error, dumper app crashing....), it's not a valid pcap-ng file. If you
> concatenate such a file with another (even correct) one, you end having
> garbage.

Unfortunately, truncated trace files are quite common, generally as a result of 
leaving packet capture running until the filesystem fills up...  And, 
inevitably, those trace files get concatenated.

I actually modified libpcap's savefile support to allow it to resynchronize in 
cases like this; basically, when a packet header seemed to be garbage (capture 
length > packet length, non-zero timezone offset field [never used by any 
version of libpcap/tcpdump], or some other failed sanity check) it would look 
backwards in the (saved) previous packet data searching for a file header.  (I 
also modified it to handle the special case of concatenated non-truncated 
capture files, where a file header appeared in place of a packet header, but it 
sounds like NTAR already deals well with this.)

This did not require a seek, or even special stdio buffering, it just used the 
library buffers for packet header and data, so that the capture data from the 
last packet was still available after reading the next packet header.  This 
approach was not 100% reliable, but did allow recovery from most cases of 
concatenated trace files.

It would be very nice if there was some similar sanity checking that could be 
performed on NTAR packet block headers (I suspect the best bet would be a valid 
option field - either four bytes of zeros or a complete chain ending in EOOB). 
  With the special magic for SPB headers, resynchronization in this case should 
not be too difficult.

@alex
-- 
mailto:dupuy at counterstorm.com