[ntar-workers] On Markers and Bookmarks

[Gianluca Varenni]

----- Original Message ----- 
From: "Stephen Donnelly" <stephen at endace.com>
To: <ntar-workers at winpcap.org>
Sent: Thursday, June 30, 2005 2:34 PM
Subject: Re: [ntar-workers] On Markers and Bookmarks


> Christian Kreibich wrote:
>> On Thu, 2005-06-30 at 13:32 -0700, Jose M. Gonzalez wrote:
>>
>>>   /-----------\  /-\  /--\  /-\  /-\  /-\  /--\  /-\  /->
>>> |             ||   ||    ||   ||   ||   ||    ||   ||
>>> +|------------+v|--+v|---+v|--+v|--+|---+|----+|---+|---+----
>>> | PacketBlock | PB | REW | FF | PB | PB | REW | FF | PB | ...
>>> +-------------+----+-----+^|--+----+----+-----+^|--+----+----
>>>                           ||                   ||
>>>           ---------------/  \-----------------/  \---->
>>
>>
>> Damn Chema, stop competing with my ASCII art! ;)
>>
>> (The FWs/REWs are good stuff btw!)
>>
>>
>>>Note: this is not indexing at all. I agree with Christian that indexing 
>>>is better done offline, in a separate file.
>>
>>
>> No no, I want no separate files. I just want a dedicated type of block
>> for indexing, essentially leaving it up the the application how indexing
>> is done, and store such blocks in the SEB.
>
> I agree, separate files are too ease to lose. Better to keep everything in 
> one place.
>
> Allowing the indexing data to be added to the capture file in a post 
> processing step (if there is no time to do it at capture time) would seem 
> a better approach.
>
> This could either be via fields that can be left blank/undefined at 
> capture time (still wastes space/bandwidth), or by inserting index records 
> later (requires recomputing and updating offsets in existing navigation 
> fields).

In general, I like the idea of having some indexing records to the file. And 
maybe they could be put at the very end of a section, in order to avoid 
recomputing the offsets). My question is: what is the best/most useful 
indexing scheme we should use? I think it heavily depends on the application 
processing the trace file. Maybe the best approach is to put some very 
simple markers (as several folks here on the mailing list proposed) at 
*reasonable* places (optional, of course), leaving the application to build 
its own tree/db/whatever data structure to fast access the data. I think 
that the problem of indexing such data is similar to the indexing problems 
in databases... anyone knowing something about it? (my knowledge about DBs 
is close to zero).

GV

>
> Stephen.
> -- 
> -----------------------------------------------------------------------
>     Stephen Donnelly BCMS PhD           email: sfd at endace.com
>     Endace Technology Ltd           phone: +64 7 839 0540
>     Hamilton, New Zealand               cell:  +64 21 1104378
> -----------------------------------------------------------------------
> _______________________________________________
> ntar-workers mailing list
> ntar-workers at winpcap.org
> https://www.winpcap.org/mailman/listinfo/ntar-workers