[Winpcap-users] RE: [Winpcap-team] High CPU Use Tracked to DPC Time[Scanned]

Mon May 8 15:11:08 GMT 2006

Hi Gianluca

Yes you right it is a single hyperthreaded single core CPU and there are two forms of presentation:

Form 1: DPCs are all scheduled on the same logical CPU and the figures are:

Total: 100%

CPU 0:  100%

CPU 1: ~0%

Form 2: DPCs are split between CPUs:

Total: 100%

CPU 0:  47%

CPU 1:  53%

Form 2 draws a nice symmetrical pattern in perfmon around the 50% mark.  The DPC Totals for either are reported as 50% however.  I presume this is because the box has two logical processors and so it adds up the amounts on each processor and divides it by the number of processors.

I should have emailed the group earlier also (sorry!!!) as we have no managed to send into this state (crashed for want of a better word) a number of NIC cards by bombarding them with UDP data.  However all the ones we have crashed appear to have crashed around this rollover value and are Marvell Yukon based chipsets.  Interestingly enough we have tried some alternatives (Broadcom, netgear, DLink, intel) which use Intel or Realtek chipsets and these *do NOT* crash and indeed we have left the Broadcom and intel versions running.

Additonally this morning we have setup two PCs from scratch and *not installed winpcap* and the results persist although instead of the crash occurring after a determined 6/7 hrs (based on the packet rate and looking at a possible packet count reap at 32 bits) the XP machine I tried this morning crashed twice within 1 hr, the second time it crashed was actually within about 12 minutes of kicking the test off after reboot.  That said, I have had a 2K and the same XP machine without winpcap installed running now for over 2 hours.

This does NOT look like a winpcap issue

Still I shall post the outcome shortly.  I should like to state for the record that I don’t believe at this point this is a Marvell issue necessarily as Marvell themselves have released updated drivers as recently as this year.  The NIC manufacturer has not however and so we are in talks with them to see if they can update their driver.  At present we have successfully shoe-horned the Marvell driver onto our problem NIC and we are going to see if it exhibits this behaviour.   If it does we have some diagnostics from Marvell that may help us identify a suspect counter wrap…

Thanks to all the winpcap community and Giancula for looking at this and I’m sorry to involve winpcap when it now appears it cannot be anything to do with this issue.  The topic may however be of interest to anyone capturing such huge amounts of UDP data as we as a company are.

Kind Regards

Ian Hawley

   _____  

From: Gianluca Varenni [mailto:gianluca.varenni at cacetech.com] 
Sent: 08 May 2006 15:43
To: Ian Hawley; winpcap-team at winpcap.org; winpcap-users at winpcap.org
Subject: Re: [Winpcap-team] High CPU Use Tracked to DPC Time[Scanned]

Ian,

I haven't tried to replicate the problem on my machines, but it seems quite strange to me that the DPC time goes to 50% (probably a HT/multicore/multiprocessor machine and 1 CPU is 100%!?!). 

Even if there's a wrap-around bug in WinPcap, I would expect a blue screen, no packets captured or packets completely dropped, but not a polling at DPC level (as you seem to be experiencing). As someone pointed out on the ddk newsgroup, it's possible that a buggy nic driver causes this effect by not properly acknowledging an interrupt and continuously scheduling DPCs on the system.

In any case, I'll try to reproduce the bug on one of my machines here.

Have a nice day

GV

----- Original Message ----- 

From: HYPERLINK "mailto:i.hawley at synectics.co.uk"Ian Hawley 

To: HYPERLINK "mailto:winpcap-team at winpcap.org"winpcap-team at winpcap.org ; HYPERLINK "mailto:winpcap-users at winpcap.org"winpcap-users at winpcap.org 

Sent: Friday, May 05, 2006 9:27 AM

Subject: [Winpcap-team] High CPU Use Tracked to DPC Time

Hello Everyone, apologies if you receive this email twice!

The company I work for has been using winpcap for some time now and we recently noticed an issue which we thought might be NIC related but could of course be WinPCap related as well and I thought I’d float it by the mailing list and see if anyone was aware of an issue.

Basically we recording MPEG2 via a UDP packet stream from proprietary hardware and after a period of time the machine enters a state where the CPU usage is around 50+% and 50% if that usage is spent servicing DPC requests according to perfmon.exe.  We initially felt this might be the NIC we are using but we have since recreated this by bombarding a different NIC with UDP data through a test application and this has presented on that NIC also.  At present I am wondering whether it is an OS issue (we are running Windows 2000 Service Pack 4 on many boxes) but I am also concerned it might be our IBM Boxes and of course, our capture mechanism uses WinPCap and not Winsock.

The curious thing about this collapse of CPU is that it appears to manifest around about the time when the host machine/application has received 2^32 packets.  Knowing the data rate we are sending to the box we are able to predict quite accurately as to when the machine will enter this state and it appears to suggest that some driver or piece of hardware or windows itself is wrapping a 32bit counter and not handling the wrap correctly.

We are presently trying various other network cards, Windows XP and different PCs to see if we can get it to manifest but I wanted to ask the WinPCap community if there was something that might go pop with such a large volume of data?  By accident we are using an oldish beta of WinPCap 3.0 but one of our engineers hasn’t seen any evidence that the release version of 3.0 nor a subsequent version might fix this.  We are however going to set our test apps running on a similar box using WinPCap 3.1.

Note again that we have detached our proprietary hardware from the PC and managed to generate the problem without using that hardware so we are very confident that this has nothing to do with the issue.  The curious thing is that once the CPU has jumped into this state it does not matter whether the data continues to be fed to the NIC; if you stop the application that is bombarding the NIC with data, the CPU continues to be stoically at the 50% mark.  Curiously the number DPCs queued/second appears to drop from what appears normal at circa 2000/second to around 40/second (this is while data is going into it, I have no metrics for when there is no data atm).  As strange is that if you unplug the NIC cable, then the CPU drops back to ostensibly zero and the PC is happy again.  Plug the cable back in however and the CPU shoots back to 50%.  Unplug the cable from the PC and plug it into a hub with nothing else attached and it is still at 50%.

I would like to emphasize that the position we are in presently means that we do not know if it is the IBM PCs we are using, the 3COM NIC, some BIOS setting, the Operating System (Old systems are running 2K and we are trying to see if it presents on XP) or some setting we have incorrect somewhere, as unlikely as that might seem.  I am not saying it is definitely a WinPCap issue, but it would be very interesting if any of the winpcap team could think if a way that it might be.

Any help would be greatly appreciated.

Kind Regards

Ian Hawley

--
No virus found in this outgoing message.
Checked by AVG Free Edition.
Version: 7.1.392 / Virus Database: 268.5.4/332 - Release Date: 04/05/2006

   _____  

_______________________________________________
Winpcap-team mailing list
Winpcap-team at winpcap.org
https://www.winpcap.org/mailman/listinfo/winpcap-team

--
No virus found in this incoming message.
Checked by AVG Free Edition.
Version: 7.1.392 / Virus Database: 268.5.5/333 - Release Date: 05/05/2006

-- 
No virus found in this outgoing message.
Checked by AVG Free Edition.
Version: 7.1.392 / Virus Database: 268.5.5/333 - Release Date: 05/05/2006

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.winpcap.org/pipermail/winpcap-users/attachments/20060508/12c65d7f/attachment-0001.htm