[Winpcap-users] Multithreaded programming with WinPCap

[Prachyabrued Mores]

Hello!,

I am experimenting with a queue management algorithm (BLUE) for a router. I 
use a PC with two network interface cards to simulate a router, write a 
program using WinPCap to capture packets from one interface and forward to 
the other interface.

I tried first with a single thread version to test WinPCap features (i am 
new to WinPCap). I tested with ftp and got about 50 Mbps using my simulated 
router on 100 Mbps network. The code was something like;

(set read timeout to 1ms in pcap_open_live)
for(;;)
   if (pcap_next_ex(nic1,,packet) == 1) {
      // if packet available at nic1, forward to nic2
      pcap_sendpacket(nic2,packet,); 
   }
   if (pcap_next_ex(nic2,,packet) == 1) {
      // if packet available at nic2, forward to nic1
      pcap_sendpacket(nic1,packet,);
   }
}

At first, i didn't care about performance and go on to implement multithread 
version to experiment with queue algorithm. The code is something like

(again, set read timeout to 1ms in pcap_open_live)

Thread1: 
for(;;)
   if (pcap_next_ex(nic1, packet)==1) {
      queue1.enqueue(packet);   
   }
   packet = queue2.dequeue();
   if (packet != NULL) {
      pcap_sendpacket(nic1,packet,);
   }
}

Thread2: 
for(;;)
   if (pcap_next_ex(nic2, packet)==1) {
      queue2.enqueue(packet2);   
   }
   packet = queue1.dequeue();
   if (packet != NULL) {
      pcap_sendpacket(nic2,packe,);
   }
}

(queues are already synchronized by semaphore, packet is local var)

The idea is each thread reads/writes on its corresponding nic and 
communicate with each other via queues. Surprisingly, i get only about 9 
Mbps using this multithread programming. Now, i seriously seek how to 
improve the performance of my implementation. 

I try to browse the WinPCap archive and found many related topics. But 
admittedly, i couldn't understand them completely and decide to ask a direct 
questions here. 

One of the thing i can grab from the archive is about thread sleep, read 
timeout of WinPCap and maybe more. I have 3 questions to ask here:

1. According to my implementation, i may have to wait for 1ms(read timeout) 
before i can dequeue. This maybe the cause?? Will it help if i have 4 
threads? (read nic1, write nic1, read nic2, write nic2) I think that this 
will decouple dequeue operation from read timeout. In other words, dequeue 
doesn't have to wait for pcap_next_ex.

2. If i have 4 threads, can i read&write to the same descriptor at the same 
time? Do i need to protect it with some synchronization construct?

3. Could you suggest a way to improve my multithreaded implementation?? I am 
sure that my implementation is very naive and can be improved a lot. Alas, i 
am not good with WinPCap and multithread programming and would like to seek 
some sage advice here.

Thank you for bearing with me and hope to get some response soon,
Mark
--