[Winpcap-users] Capturing from a 'tap' type deviceusing2networkcards - and how to order the packets

[Gianluca Varenni]

100% CPU usage (and packet drops) means that your application is not fast 
enough at processing packets.

Some things to remember
- Wireshark usually loses packets at high packet rates because
  1. it dumps the packets to disk. Disks are SLOW.
  2. if you have enabled real-time update of the packets in the wireshark 
GUI, that takes time and memory, and can affect performance
- in one of your previous e-mails, I seem to remember that you were dumping 
some packet contents (or fields) in a database. Unless you are using
  an embedded DB in memory, databases are SLOW. They are slow because 
normally their backing store is a disk. Hard disks are slow.

Again, I would probably profile your application and try to locate where all 
the CPU load is going.

Have a nice day
GV

----- Original Message ----- 
From: "Maria de Fatima Requena" <MariaF.Requena at a-e.es>
To: <winpcap-users at winpcap.org>
Sent: Thursday, May 29, 2008 4:10 AM
Subject: RE: [Winpcap-users] Capturing from a 'tap' type 
deviceusing2networkcards - and how to order the packets


> Sorry, i have forgotten to mention that I'm also having a 100%CPU usage 
> for both my app and Wireshark
>
> Thanks
>
>
> María de Fátima Requena Cabot (2488)
> +34 91 787 23 00 alhambra-eidos.es
>
>
>
> -----Mensaje original-----
> De: winpcap-users-bounces at winpcap.org 
> [mailto:winpcap-users-bounces at winpcap.org] En nombre de Maria de Fatima 
> Requena
> Enviado el: jueves, 29 de mayo de 2008 12:09
> Para: winpcap-users at winpcap.org
> Asunto: RE: [Winpcap-users] Capturing from a 'tap' type device 
> using2networkcards - and how to order the packets
>
> I have similar problems. I posted some time ago about missing packets 
> (packets that Wireshark captured but my application didn't) but now I am 
> in a different machine with high traffic and both Wireshark and my app 
> miss them.
>
> Any ideas?
>
>
> María de Fátima Requena Cabot (2488)
> +34 91 787 23 00 alhambra-eidos.es
>
>
> -----Mensaje original-----
> De: winpcap-users-bounces at winpcap.org 
> [mailto:winpcap-users-bounces at winpcap.org] En nombre de Gianluca Varenni
> Enviado el: miércoles, 28 de mayo de 2008 19:55
> Para: winpcap-users at winpcap.org
> Asunto: Re: [Winpcap-users] Capturing from a 'tap' type device using 
> 2networkcards - and how to order the packets
>
>
> ----- Original Message ----- 
> From: "Michael Shiels" <MaSSoft at massoftware.com>
> To: <winpcap-users at winpcap.org>
> Sent: Wednesday, May 28, 2008 7:06 AM
> Subject: [Winpcap-users] Capturing from a 'tap' type device using 2
> networkcards - and how to order the packets
>
>
>> This problem has had a couple attempts and each has failed to get the
>> packets ALWAYS into the proper order.
>>
>> It seems some packets hang around for many seconds somewhere in the pcap
>> world, but I can't say for sure.
>>
>> The code is doing everything I could find for high speed/high volume
>> capturing.
>>
>> Specifically
>>
>> Timeout of '1' on pcap_open, with flags PCAP_OPENFLAG_PROMISCUOUS
>>
>> Pcap_setbuff( 5*1024*1024 )
>>
>> Pcap_setmintocopy(0)
>
> Putting mintocopy to 0 and setting the timeout to 1ms does *not* help
> getting better performance. It increases the responsiveness of the
> application, at the expense of a higher CPU load (because of the larger
> number of syscalls/second used to bring packets to user level).
>
>
>>
>> We then have a loop of code that gets the pcap events and does a wait 
>> with
>> 1
>> second timeout, then polls both pcap handles till we find no data ON BOTH
>> HANDLES.  That should retrieve all packets is my understanding. We then
>> delay the packets in some queues for a period of time, that should allow
>> all
>> 'hardware/software' buffers to empty I expected.  Then the packets are
>> merged by timestamp.
>>
>> BUT under load it seems that we still some packets stuck somewhere, for
>> multiple seconds.  WE started by waiting for packets to be older than 5
>> seconds to merge, and it had problems.  Bumping the process to wait for 
>> 30
>> seconds before merging packets, seems to work, but it's a supreme
>> hack/probably not something 100% reliable.
>>
>
> WinPcap does not starve the packets in the driver for all that time (5
> seconds or so), if you are actively reading data from the devices. How do
> you "poll both the pcap handler till we find no data"? Do you use two
> threads to do that?
> Also, what is the traffic load we are talking about?
>
> Have a nice day
> GV
>
>> ANY IDEAS?? HELP!!!!!!
>>
>> _______________________________________________
>> Winpcap-users mailing list
>> Winpcap-users at winpcap.org
>> https://www.winpcap.org/mailman/listinfo/winpcap-users
>
> _______________________________________________
> Winpcap-users mailing list
> Winpcap-users at winpcap.org
> https://www.winpcap.org/mailman/listinfo/winpcap-users
>
> _______________________________________________
> Winpcap-users mailing list
> Winpcap-users at winpcap.org
> https://www.winpcap.org/mailman/listinfo/winpcap-users
>
> _______________________________________________
> Winpcap-users mailing list
> Winpcap-users at winpcap.org
> https://www.winpcap.org/mailman/listinfo/winpcap-users