[Winpcap-users] How to ensure all packets are captured

Gianluca Varenni gianluca.varenni at cacetech.com
Tue Aug 11 16:55:12 PDT 2009


Replies quoted with --GV--

Have a nice day
GV


----- Original Message ----- 
From: Denis Kourktchan
To: winpcap-users at winpcap.org
Sent: Tuesday, August 11, 2009 11:27 AM
Subject: Re: [Winpcap-users] How to ensure all packets are captured


Dear Mr. Varenni,

Thanks for your reply. I just wanted to clarify a few points you have made. 
(Note: I am using a multicore machine so the multithread is a natural choice 
and my processing code is definitely not as optimized as your pcap code, but 
this is a solo project so I don't need to satisfy anyone else)
--GV--
Using multiple threads, even if you have an SMP machine, isn't always the 
best choice. It makes sense if the CPU needed for receiving packets + CPU 
needed for processing packets is greater than 100% (where by 100% i mean 1 
single core fully loaded, if you have 4 cores the total processing time is 
400%), and the cost of pushing packets to a queue/linked-list is negligible 
versus the processing. So for example you have 60%CPU for receiving packets 
and 5% to push the packets on a queue (on one CPU) and another 60%CPU for 
processing packets and 5% to pull the packets out of the queue.
--GV--


1.       You mention, that I should pre-allocate and reuse buffer space 
where to store packets and not allocate new buffer. Do you mean something 
like:
PCAPDATANODE myPacketBuffer[amountOfNeededPackets];   // store in a specific 
memory location predetermined
Is PCAPDATANODE packetNode = new PCAPDATANODE(); not a good way to go?

--GV--
Allocating your storage (PCAPDATANODE) whenever a packet arrives is a bad 
strategy because 1. memory allocation is expensive and 2. you will keep 
allocating and freeing chunks of memory, which is kinda useless in your 
scenario.
--GV--

2.       To prevent overloading, I need to pause capture (not extraction) 
for a some amounts time according to SYSTEM time settings, is there a 
command that can facilitate that? Or should I simply stop the adapter, 
release it, and then reacquire when the time comes to capture (also would 
this automatically flush the buffer)? At the moment, I merely stopped the 
extraction but as you might have predicted the capture continued and I 
simply continued processing where I last left it off.

--GV--
I don't understand why you want to pause the capture. Either you just drop 
drop the packets when let's say your queue is full, or you can stop the 
capture (i.e. close the pcap_t handle_ and restart it (calling 
pcap_open_live again). There is no API to pause reception. Another 
possibility to "virtually" pause reception is to just set a filter that 
drops almost all the packets.

--GV--

Thanks again for you time,

-          Dennis

From: Gianluca Varenni [mailto:gianluca.varenni at cacetech.com]
Sent: Monday, August 10, 2009 2:50 PM
To: dennis.kg at gmail.com; winpcap-users at winpcap.org
Subject: Re: [Winpcap-users] How to ensure all packets are captured

Moving the processing code in a separate thread may or might not help. If 
you move the processing on a different thread and use a FIFO/linked list to 
pass the packets among the threads, you will need to copy the packets in 
your own allocated buffer. Copying a packet is extremely fast, but it makes 
sense only if your packet processing code is much more CPU intensive than 
copying the packet itself. Also, using a different thread makes sense if you 
run on a multicore machine, where basically you can use both the CPUs.
If you go this way, remember to preallocate and reuse the buffers in which 
you copy the packets, do not allocate a new buffer whenever you receive a 
packet.

Hope this helps
GV
----- Original Message ----- 
From: Denis Kourktchan
To: winpcap-users at winpcap.org
Sent: Sunday, August 09, 2009 1:05 PM
Subject: [Winpcap-users] How to ensure all packets are captured

Hi Everybody,

I am writing an application that is intercepting packets from a network and 
after reading a whole lot of posts regarding lost messages I figured the 
fault is usually in application taking too long to process a batch of 
packets which causes the overfilled buffer to replace oldest messages. So I 
decided that the following setup is the best possible way to ensure that my 
messages are not lost.

1.       Put capture (using pcap_next_ex()) into a different thread then the 
processing.
2.       Store packets immediately upon extraction into a custom written 
linked list, adding to the end and processing from the front (in different 
threads) ensuring no mutex controls are necessary and no idle time.

Now here is the problem for which I need your expert advice, since the 
pcap_next_ex() returns pointers to data, I still need to extract them (I use 
for loop) before I can store which still preoccupies the capture thread and 
slows down extraction process. So is there a better way to extract packets 
without this delay?

I would appreciate any and all advice regarding the optimal solution 
architecture wise.

Regards,

-          Dennis


















_______________________________________________
Winpcap-users mailing list
Winpcap-users at winpcap.org
https://www.winpcap.org/mailman/listinfo/winpcap-users



_______________________________________________
Winpcap-users mailing list
Winpcap-users at winpcap.org
https://www.winpcap.org/mailman/listinfo/winpcap-users 



More information about the Winpcap-users mailing list