[Winpcap-users] Multithreaded programming with WinPCap

[Loris Degioanni]

Prachyabrued Mores wrote:

> Hello!,
> 
> I found out today that setting read timeout to -1 value allow my multithread 
> program to surpass the single thread version. Now the achievable throughput 
> is about 55 Mbps (out of 100Mbps). A large improvement from a previous 10 
> Mbps.
> 
> Surprisingly, -1 value (no wait, return immediately) is not mentioned in 
> current version of WinPCap help file. I have to browse through the archives.

For pcap functions, we use the libpcap documentation from tcpdump.org, 
which is not very complete. I've always had in my mind to improve that 
documentation, maybe switching to doxygen, but of course I've not had 
time for that yet.

Note that setting the timeout to -1 essentially switches your 
architecture to polling, which is efficient but wastes a lot of CPU. You 
may try to set a very small timeout and see if things are still good, 
this would make you program much more efficient.

> Btw, i still need an answer to this questions,
> 
> 1. Can i access the same descriptor in different thread without 
> synchronization?

Accessing any kind of handle or descriptor from different threads 
without solid synchronization is evil.

> 2. Since i am implementing (application level) router queue using WinPCap, i 
> wonder if there is any underlying system queue used by WinPCap? Is it 
> possible for packet to be dropped by WinPCap before reaching my app?

WinPcap has a kernel buffer, whose size can be changed with the 
pcap_setbuff() function.
if with "Is it possible for packet to be dropped" you mean "does winpcap 
have a filtering system", the answer is yes, give a look at 
http://www.winpcap.org/docs/docs31/html/group__wpcap__tut5.html

> 3. If there is someone who has enough patience to read my first question. A 
> suggestion or comment to improve my program will be very welcome.!

Decrease mintocopy. This benefits pure performance at the cost of higher 
CPU load, like decreasing the timeout.

Loris

> Thank you,
> Mark
> 
> On Fri, 28 Apr 2006 02:36:19 -0500, Prachyabrued Mores wrote
>> Hello!,
>>
>> I am experimenting with a queue management algorithm (BLUE) for a 
>> router. I use a PC with two network interface cards to simulate a 
>> router, write a program using WinPCap to capture packets from one 
>> interface and forward to the other interface.
>>
>> I tried first with a single thread version to test WinPCap features 
>> (i am new to WinPCap). I tested with ftp and got about 50 Mbps using 
>> my simulated router on 100 Mbps network. The code was something like;
>>
>> (set read timeout to 1ms in pcap_open_live)
>> for(;;)
>>    if (pcap_next_ex(nic1,,packet) == 1) {
>>       // if packet available at nic1, forward to nic2
>>       pcap_sendpacket(nic2,packet,); 
>>    }
>>    if (pcap_next_ex(nic2,,packet) == 1) {
>>       // if packet available at nic2, forward to nic1
>>       pcap_sendpacket(nic1,packet,);
>>    }
>> }
>>
>> At first, i didn't care about performance and go on to implement 
>> multithread version to experiment with queue algorithm. The code is 
>> something like
>>
>> (again, set read timeout to 1ms in pcap_open_live)
>>
>> Thread1: 
>> for(;;)
>>    if (pcap_next_ex(nic1, packet)==1) {
>>       queue1.enqueue(packet);   
>>    }
>>    packet = queue2.dequeue();
>>    if (packet != NULL) {
>>       pcap_sendpacket(nic1,packet,);
>>    }
>> }
>>
>> Thread2: 
>> for(;;)
>>    if (pcap_next_ex(nic2, packet)==1) {
>>       queue2.enqueue(packet2);   
>>    }
>>    packet = queue1.dequeue();
>>    if (packet != NULL) {
>>       pcap_sendpacket(nic2,packe,);
>>    }
>> }
>>
>> (queues are already synchronized by semaphore, packet is local var)
>>
>> The idea is each thread reads/writes on its corresponding nic and 
>> communicate with each other via queues. Surprisingly, i get only 
>> about 9 Mbps using this multithread programming. Now, i seriously 
>> seek how to improve the performance of my implementation.
>>
>> I try to browse the WinPCap archive and found many related topics. 
>> But admittedly, i couldn't understand them completely and decide to 
>> ask a direct questions here.
>>
>> One of the thing i can grab from the archive is about thread sleep,
>>  read timeout of WinPCap and maybe more. I have 3 questions to ask here:
>>
>> 1. According to my implementation, i may have to wait for 1ms(read 
>> timeout) before i can dequeue. This maybe the cause?? Will it help 
>> if i have 4 threads? (read nic1, write nic1, read nic2, write nic2)
>>  I think that this will decouple dequeue operation from read 
>> timeout. In other words, dequeue doesn't have to wait for pcap_next_ex.
>>
>> 2. If i have 4 threads, can i read&write to the same descriptor at 
>> the same time? Do i need to protect it with some synchronization construct?
>>
>> 3. Could you suggest a way to improve my multithreaded 
>> implementation?? I am sure that my implementation is very naive and 
>> can be improved a lot. Alas, i am not good with WinPCap and 
>> multithread programming and would like to seek some sage advice here.
>>
>> Thank you for bearing with me and hope to get some response soon,
>> Mark
>> --
>>
>> _______________________________________________
>> Winpcap-users mailing list
>> Winpcap-users at winpcap.org
>> https://www.winpcap.org/mailman/listinfo/winpcap-users
> 
> 
> --
> 
> _______________________________________________
> Winpcap-users mailing list
> Winpcap-users at winpcap.org
> https://www.winpcap.org/mailman/listinfo/winpcap-users
>