[Winpcap-users] How to ensure all packets are captured
Gianluca Varenni
gianluca.varenni at cacetech.com
Tue Aug 11 16:55:12 PDT 2009
Replies quoted with --GV--
Have a nice day
GV
----- Original Message -----
From: Denis Kourktchan
To: winpcap-users at winpcap.org
Sent: Tuesday, August 11, 2009 11:27 AM
Subject: Re: [Winpcap-users] How to ensure all packets are captured
Dear Mr. Varenni,
Thanks for your reply. I just wanted to clarify a few points you have made.
(Note: I am using a multicore machine so the multithread is a natural choice
and my processing code is definitely not as optimized as your pcap code, but
this is a solo project so I don't need to satisfy anyone else)
--GV--
Using multiple threads, even if you have an SMP machine, isn't always the
best choice. It makes sense if the CPU needed for receiving packets + CPU
needed for processing packets is greater than 100% (where by 100% i mean 1
single core fully loaded, if you have 4 cores the total processing time is
400%), and the cost of pushing packets to a queue/linked-list is negligible
versus the processing. So for example you have 60%CPU for receiving packets
and 5% to push the packets on a queue (on one CPU) and another 60%CPU for
processing packets and 5% to pull the packets out of the queue.
--GV--
1. You mention, that I should pre-allocate and reuse buffer space
where to store packets and not allocate new buffer. Do you mean something
like:
PCAPDATANODE myPacketBuffer[amountOfNeededPackets]; // store in a specific
memory location predetermined
Is PCAPDATANODE packetNode = new PCAPDATANODE(); not a good way to go?
--GV--
Allocating your storage (PCAPDATANODE) whenever a packet arrives is a bad
strategy because 1. memory allocation is expensive and 2. you will keep
allocating and freeing chunks of memory, which is kinda useless in your
scenario.
--GV--
2. To prevent overloading, I need to pause capture (not extraction)
for a some amounts time according to SYSTEM time settings, is there a
command that can facilitate that? Or should I simply stop the adapter,
release it, and then reacquire when the time comes to capture (also would
this automatically flush the buffer)? At the moment, I merely stopped the
extraction but as you might have predicted the capture continued and I
simply continued processing where I last left it off.
--GV--
I don't understand why you want to pause the capture. Either you just drop
drop the packets when let's say your queue is full, or you can stop the
capture (i.e. close the pcap_t handle_ and restart it (calling
pcap_open_live again). There is no API to pause reception. Another
possibility to "virtually" pause reception is to just set a filter that
drops almost all the packets.
--GV--
Thanks again for you time,
- Dennis
From: Gianluca Varenni [mailto:gianluca.varenni at cacetech.com]
Sent: Monday, August 10, 2009 2:50 PM
To: dennis.kg at gmail.com; winpcap-users at winpcap.org
Subject: Re: [Winpcap-users] How to ensure all packets are captured
Moving the processing code in a separate thread may or might not help. If
you move the processing on a different thread and use a FIFO/linked list to
pass the packets among the threads, you will need to copy the packets in
your own allocated buffer. Copying a packet is extremely fast, but it makes
sense only if your packet processing code is much more CPU intensive than
copying the packet itself. Also, using a different thread makes sense if you
run on a multicore machine, where basically you can use both the CPUs.
If you go this way, remember to preallocate and reuse the buffers in which
you copy the packets, do not allocate a new buffer whenever you receive a
packet.
Hope this helps
GV
----- Original Message -----
From: Denis Kourktchan
To: winpcap-users at winpcap.org
Sent: Sunday, August 09, 2009 1:05 PM
Subject: [Winpcap-users] How to ensure all packets are captured
Hi Everybody,
I am writing an application that is intercepting packets from a network and
after reading a whole lot of posts regarding lost messages I figured the
fault is usually in application taking too long to process a batch of
packets which causes the overfilled buffer to replace oldest messages. So I
decided that the following setup is the best possible way to ensure that my
messages are not lost.
1. Put capture (using pcap_next_ex()) into a different thread then the
processing.
2. Store packets immediately upon extraction into a custom written
linked list, adding to the end and processing from the front (in different
threads) ensuring no mutex controls are necessary and no idle time.
Now here is the problem for which I need your expert advice, since the
pcap_next_ex() returns pointers to data, I still need to extract them (I use
for loop) before I can store which still preoccupies the capture thread and
slows down extraction process. So is there a better way to extract packets
without this delay?
I would appreciate any and all advice regarding the optimal solution
architecture wise.
Regards,
- Dennis
_______________________________________________
Winpcap-users mailing list
Winpcap-users at winpcap.org
https://www.winpcap.org/mailman/listinfo/winpcap-users
_______________________________________________
Winpcap-users mailing list
Winpcap-users at winpcap.org
https://www.winpcap.org/mailman/listinfo/winpcap-users
More information about the Winpcap-users
mailing list