[Winpcap-users] Trying to handle a huge sniffed RTP packets trafficwith winpcap

Thu Aug 19 11:08:45 PDT 2010

Damian,

a) can solve the problem in the short term, but you still don't know why you are dropping the packets. And given the fact that wireshark doesn't drop packets (and wireshark does dump packets to disk when capturing), I think you are some non negligible bottleneck in your code.
b) technically there is no limit. However I would definitely avoid opening 25 (or worse, 200) capture instances at the same time, one per RTP stream.
c) Again, I would definitely avoid that. The kernel buffer memory is kernel non paged memory, which is a scarce resource, expecially on a 32bit system (I think the limit on a 32bit system is a total of some hundreds MB of non paged pool). I would definitely go with one single capture instance and demultiplexing the packets in your application
d) Definitely. I think you should concentrate in analyzing where the bottleneck is. Usually the bottleneck is in the marshalling of the packet buffer into a managed object. What I would do is create a minimal C# application that just receives the packets with your wrapper and counts them. Do the same with a C application based on the native WinPcap APIs. And then compare the packet drop rate *and* the CPU load in both the cases. You said "we are just creating C# objects". I would be quite worried about that "just". Suppose you receive 10k packets per second. You will create 10k objects every second. It's true, memory allocation in a .NET application is extremely fast. But eventually the GC will kick in to release those objects, maybe causing some delays. Even if you are working with managed code, I would probably use a preallocated pool of "packet objects".

Just my two cents
GV

From: DamioX 
Sent: Wednesday, August 18, 2010 8:03 AM
To: winpcap-users at winpcap.org 
Subject: [Winpcap-users] Trying to handle a huge sniffed RTP packets trafficwith winpcap

Hello Guys,

  We are using winpcap (4.1.0.1753) to handle a huge RTP packets traffic being sniffed from an Audio Gateways of my company. Our application has the ability to persist specific RTP packets to Hard-Disk (we get the IP Addresses and Ports information to filter these data from our application).

  Our tests were executed in a Windows 2003 Standard Edition x64 (64 Bits) server, Intel Xeon E5520 2.27 Ghz & 6.00 GB of RAM.

  We have noticied that we are getting peaks of RTP packets traffic corresponding to 25 concurrent calls (G.711 codec) from this Audio Gateway. Based on our notes, the current traffic being spanned is quite lower than 1 MByte per second.

  Our application starts and opens the interface which is getting the mentioned traffic and sets the interface's filter as "udp". Later, we are filtering (from our application) the IP Addresses and Ports needed to be persisted. In our current test, we are just saving one call in the Hard-Disk (only two sniffed RTP packets flow) with a different Thread-Pool. Thus, we are receiving peaks of 25 concurrent calls, but a different application thread is saving just 1 call to Hard-Disk.

  The problem is that we are experiencing several packet losses of several seconds each in this persisted call. We checked that Wireshark is not loosing so many packets. Thus, we tried to increase our winpcap kernel buffer to 4 Mbytes, 10 Mbytes, 500 Mbytes. In all cases we detected a performance improvement, but we are still having packet losses (2 seconds more or less sporadically with a kernel buffer of 500 Mbytes) :(.

  Our application is developed in C# .NET 3.5 and wrapping the Winpcap APIs. Our application's piece we are putting our eyes to is when Winpcap invokes the wrapped C# Callback. From that point we are creating a couple of C# objects in order to turn the unmanaged winpcap structures into C# managed objects. After that, we are enqueueing these objects in an own circular buffer and then another thread-pool is unbuffering this information. We have figured out that our circular buffer is not discarding any packets.

  Our big concern is that currently we are experiencing Winpcap issues in our application which has peaks of 25 concurrent calls, and if we think that we might get peaks of 200 concurrent calls in the future, I'm not sure that some application's performance improvement will make a big difference than today's performance. We think that maybe a better approach would be to open several times the sniffed interface and, for each one, sets the respective IPAddresses and Ports' filter. 

  Based on what I've told you, we want to know your opinion:

  a) Do you think the fact of opening several times the same interface and setting better filters will be a good approach? 
  b) How many concurrent winpcap handlers (for one interfaces) we may have opened at the same time?
  c) We think that if we open the interface several times there will be several winpcap kernel ring buffers. What size do you think would be the best for each of these kernel ring buffers? If we think in having Kernel ring buffers of 5 MBytes, and 200 concurrent calls, we would need 1 GByte of memory dedicated to Kernel Ring Buffer. This memory requirement is not a problem anyways.
  d) Do you think that we should concentrate our effort in trying to improve our application's performance? We are just creating C# objects... That shouldn't consume much time.

Thanks in advance for your help.
Damian Nardelli - C/C++/.NET Software Developer.

--------------------------------------------------------------------------------

_______________________________________________
Winpcap-users mailing list
Winpcap-users at winpcap.org
https://www.winpcap.org/mailman/listinfo/winpcap-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.winpcap.org/pipermail/winpcap-users/attachments/20100819/35dd6941/attachment-0001.htm 