<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=Content-Type content="text/html; charset=utf-8">
<meta name=Generator content="Microsoft Word 12 (filtered medium)">
<!--[if !mso]>
<style>
v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style>
<![endif]-->
<style>
<!--
/* Font Definitions */
@font-face
{font-family:SimSun;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
@font-face
{font-family:Verdana;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:"Comic Sans MS";
panose-1:3 15 7 2 3 3 2 2 2 4;}
@font-face
{font-family:"\@SimSun";
panose-1:2 1 6 0 3 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0in;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:Consolas;}
span.apple-style-span
{mso-style-name:apple-style-span;}
span.EmailStyle21
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle23
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
-->
</style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body bgcolor=white lang=EN-US link=blue vlink=purple id=MailContainerBody
name="Compose message area">
<div class=WordSection1>
<p class=MsoNormal><span style='font-size:10.5pt;font-family:"Calibri","sans-serif";
color:#1F497D'>#1: I wholeheartedly agree.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.5pt;font-family:"Calibri","sans-serif";
color:#1F497D'>#2: Oops! Yes, you are right again. I suggested it since I knew
without it enabled the host CPU must then calculate the checksum which would likely
introduce a noticeable delay at higher traffic volumes, but forgot that such
delays only impact the TCP/IP stack and not packet capturing. My bad. Thanks
for correcting me on that.<o:p></o:p></span></p>
<div>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span
style='font-size:7.5pt;font-family:"Arial","sans-serif";color:black'>-- </span><span
style='color:#1F497D'><br>
</span><span style='font-size:13.5pt;font-family:"Arial","sans-serif";
color:black'> "</span><b><span style='font-family:"Arial","sans-serif";
color:green'>Fish</span></b><span style='font-size:13.5pt;font-family:"Arial","sans-serif";
color:black'>"</span><span style='font-size:10.0pt;font-family:"Arial","sans-serif";
color:black'> (</span><span style='font-size:10.0pt;font-family:"Comic Sans MS";
color:purple'>David B. Trout</span><span style='font-size:10.0pt;font-family:
"Arial","sans-serif";color:#1F497D'>)</span><span style='color:#1F497D'> <br>
</span><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";
color:#1F497D'> </span><span style='color:#1F497D'> </span><u><span
style='font-size:10.0pt;font-family:"Arial","sans-serif";color:blue'>fish@softdevlabs.com</span></u><span
style='color:#1F497D'><o:p></o:p></span></p>
</div>
<p class=MsoNormal><span style='font-size:10.5pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<div style='border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt'>
<div>
<div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in'>
<p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span
style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> winpcap-users-bounces@winpcap.org
[mailto:winpcap-users-bounces@winpcap.org] <b>On Behalf Of </b>Gianluca Varenni<br>
<b>Sent:</b> Monday, September 20, 2010 4:46 PM<br>
<b>To:</b> winpcap-users@winpcap.org<br>
<b>Subject:</b> Re: [Winpcap-users] About the packets loss , what is the
bottleneck ?<br>
<b>Importance:</b> High<o:p></o:p></span></p>
</div>
</div>
<p class=MsoNormal><o:p> </o:p></p>
<div>
<p class=MsoNormal><span style='font-family:"Calibri","sans-serif"'>Two notes: </span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal> <o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-family:"Calibri","sans-serif"'>1. if the
objective is dumping the packets to disk, then the bottleneck is mostly the
disk. </span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-family:"Calibri","sans-serif"'>2. checksum
offloading does not play any role in packet capture. It plays a role when
packets have to be processed by the TCP/IP stack (because the validation is
done in hardware rather software). But when you do packet capture, you don't
care about that. In fact I suggested to totally disable TCP/IP from the NIC
where you capture the packets from.</span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal> <o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><span style='font-family:"Calibri","sans-serif"'>GV</span><o:p></o:p></p>
</div>
<div>
<div>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'><o:p> </o:p></span></p>
</div>
<div>
<div>
<p class=MsoNormal style='background:whitesmoke'><b><span style='font-size:
10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span
style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> <a
href="mailto:fish@infidels.org" title="fish@infidels.org">"Fish"
(David B. Trout)</a> <o:p></o:p></span></p>
</div>
<div>
<p class=MsoNormal style='background:whitesmoke'><b><span style='font-size:
10.0pt;font-family:"Tahoma","sans-serif"'>Sent:</span></b><span
style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> Sunday, September
19, 2010 4:44 AM<o:p></o:p></span></p>
</div>
<div>
<p class=MsoNormal style='background:whitesmoke'><b><span style='font-size:
10.0pt;font-family:"Tahoma","sans-serif"'>To:</span></b><span style='font-size:
10.0pt;font-family:"Tahoma","sans-serif"'> <a
href="mailto:winpcap-users@winpcap.org" title="winpcap-users@winpcap.org">winpcap-users@winpcap.org</a>
<o:p></o:p></span></p>
</div>
<div>
<p class=MsoNormal style='background:whitesmoke'><b><span style='font-size:
10.0pt;font-family:"Tahoma","sans-serif"'>Subject:</span></b><span
style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> Re: [Winpcap-users]
About the packets loss ,what is the bottleneck ?<o:p></o:p></span></p>
</div>
</div>
</div>
<div>
<p class=MsoNormal><o:p> </o:p></p>
</div>
<p class=MsoNormal><span style='font-size:10.5pt;font-family:"Calibri","sans-serif";
color:#1F497D'>There are other factors as well:<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.5pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoListParagraph style='text-indent:-.25in'><span style='font-size:
10.5pt;font-family:"Calibri","sans-serif";color:#1F497D'>1.</span><span
style='font-size:7.0pt;color:#1F497D'> </span><span
style='font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1F497D'>Number
of processors and processor speed.<o:p></o:p></span></p>
<p class=MsoListParagraph style='text-indent:-.25in'><span style='font-size:
10.5pt;font-family:"Calibri","sans-serif";color:#1F497D'>2.</span><span
style='font-size:7.0pt;color:#1F497D'> </span><span
style='font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1F497D'>Amount
of RAM<o:p></o:p></span></p>
<p class=MsoListParagraph style='text-indent:-.25in'><span style='font-size:
10.5pt;font-family:"Calibri","sans-serif";color:#1F497D'>3.</span><span
style='font-size:7.0pt;color:#1F497D'> </span><span
style='font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1F497D'>Operating
system workload<o:p></o:p></span></p>
<p class=MsoListParagraph style='text-indent:-.25in'><span style='font-size:
10.5pt;font-family:"Calibri","sans-serif";color:#1F497D'>4.</span><span
style='font-size:7.0pt;color:#1F497D'> </span><span
style='font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1F497D'>Network
Adapter Task Offloading capability (esp. checksum offloading)<o:p></o:p></span></p>
<p class=MsoListParagraph style='text-indent:-.25in'><span style='font-size:
10.5pt;font-family:"Calibri","sans-serif";color:#1F497D'>5.</span><span
style='font-size:7.0pt;color:#1F497D'> </span><span
style='font-size:10.5pt;font-family:"Calibri","sans-serif";color:#1F497D'>Speed
of disk subsystem<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.5pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.5pt;font-family:"Calibri","sans-serif";
color:#1F497D'>On an modern fast quad-core system that is relatively idle (has
no other work to do) with lots of RAM and a high performance multi-disk RAID-0
array and with checksum offloading enabled in the adapter, it should be
entirely possible to capture a gigabit speed without packet loss.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.5pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.5pt;font-family:"Calibri","sans-serif";
color:#1F497D'>If your system is an overtaxed (busy) single processor system
with only an average amount of RAM and an average speed non-RAID disk subsystem
however, then you’re very likely going to experience significant packet loss
beyond about 480 Mbps.<o:p></o:p></span></p>
<div>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span
style='font-size:7.5pt;font-family:"Arial","sans-serif";color:black'>-- </span><span
style='color:#1F497D'><br>
</span><span style='font-size:13.5pt;font-family:"Arial","sans-serif";
color:black'> "</span><b><span style='font-family:"Arial","sans-serif";
color:green'>Fish</span></b><span style='font-size:13.5pt;font-family:"Arial","sans-serif";
color:black'>"</span><span style='font-size:10.0pt;font-family:"Arial","sans-serif";
color:black'> (</span><span style='font-size:10.0pt;font-family:"Comic Sans MS";
color:purple'>David B. Trout</span><span style='font-size:10.0pt;font-family:
"Arial","sans-serif";color:#1F497D'>)</span><span style='color:#1F497D'> <br>
</span><span style='font-size:10.0pt;font-family:"Verdana","sans-serif";
color:#1F497D'> </span><span style='color:#1F497D'> </span><u><span
style='font-size:10.0pt;font-family:"Arial","sans-serif";color:blue'>fish@softdevlabs.com</span></u><span
style='color:#1F497D'><o:p></o:p></span></p>
</div>
<p class=MsoNormal><span style='font-size:10.5pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<div style='border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt'>
<div>
<div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in'>
<p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span
style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>
winpcap-users-bounces@winpcap.org [mailto:winpcap-users-bounces@winpcap.org] <b>On
Behalf Of </b>yulou liu<br>
<b>Sent:</b> Sunday, September 19, 2010 12:50 AM<br>
<b>To:</b> winpcap-users@winpcap.org<br>
<b>Subject:</b> [Winpcap-users] About the packets loss , what is the bottleneck
?<br>
<b>Importance:</b> High<o:p></o:p></span></p>
</div>
</div>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal style='margin-bottom:12.0pt'><span style='font-size:11.5pt'><br>
</span><span class=apple-style-span><span style='font-size:10.0pt;font-family:
"Calibri","sans-serif"'>There is still the question about packets loss.</span></span><span
style='font-size:11.5pt'><br>
</span><span class=apple-style-span><span style='font-size:10.0pt;font-family:
"Calibri","sans-serif"'> </span></span><span class=apple-style-span><span
style='font-size:11.5pt'> </span></span><span style='font-size:11.5pt'><br>
</span><span class=apple-style-span><span style='font-size:10.0pt;font-family:
"Calibri","sans-serif"'>According to the essay ' </span></span><span
class=apple-style-span><span style='font-size:10.0pt;font-family:SimSun'>Profiling
and Optimization of Software-Based Network-Analysis Applications</span></span><span
class=apple-style-span><span style='font-size:10.0pt;font-family:"Calibri","sans-serif"'>'
, every packet is copied twice in the main memory before
reaching the user. In order to reduce the cost of CPU and the bus
occupying of the SDRAM of pc, is it possible to copy data directly
from the kernel buffer to the final buffer , which I want the date
kept in ? </span></span><span style='font-size:11.5pt'><br>
</span><span class=apple-style-span><span style='font-size:10.0pt;font-family:
"Calibri","sans-serif"'> </span></span><span class=apple-style-span><span
style='font-size:11.5pt'> </span></span><span style='font-size:11.5pt'><br>
</span><span class=apple-style-span><span style='font-size:10.0pt;font-family:
"Calibri","sans-serif"'>Here is another idea --- </span></span><span
class=apple-style-span><span style='font-size:10.0pt;font-family:SimSun'> allocate
several different user buffers , once </span></span><span
class=apple-style-span><span style='font-size:10.0pt;font-family:"Calibri","sans-serif"'>a </span></span><span
class=apple-style-span><span style='font-size:10.0pt;font-family:SimSun'>user
buffer is fulled , then let the next user buffer to save the </span></span><span
class=apple-style-span><span style='font-size:10.0pt;font-family:"Calibri","sans-serif"'>new</span></span><span
class=apple-style-span><span style='font-size:10.0pt;font-family:SimSun'> datas
from kernel buffer. Meanwhile copy datas from the first user buffer to
disk (assume that the hard disk write rate is fast enough).</span></span><span
class=apple-style-span><span style='font-size:10.0pt;font-family:"Calibri","sans-serif"'>
Is this idea work with the winpcap ?</span></span><span class=apple-style-span><span
style='font-size:11.5pt'> </span></span><span style='font-size:11.5pt'><br>
</span><span class=apple-style-span><span style='font-size:10.0pt;font-family:
"Calibri","sans-serif"'> </span></span><span class=apple-style-span><span
style='font-size:11.5pt'> </span></span><span style='font-size:11.5pt'><br>
</span><span class=apple-style-span><span style='font-size:10.0pt;font-family:
"Calibri","sans-serif"'> </span></span><span class=apple-style-span><span
style='font-size:11.5pt'> </span></span><span style='font-size:11.5pt'><br>
</span><span class=apple-style-span><span style='font-size:10.0pt;font-family:
"Calibri","sans-serif"'>Thank you!</span></span><span class=apple-style-span><span
style='font-size:11.5pt'> </span></span><span style='font-size:11.5pt'><br>
</span><span class=apple-style-span><span style='font-size:10.0pt;font-family:
"Calibri","sans-serif"'> </span></span><span class=apple-style-span><span
style='font-size:11.5pt'> </span></span><span style='font-size:11.5pt'><br>
</span><span class=apple-style-span><span style='font-size:10.0pt;font-family:
"Calibri","sans-serif"'>==============================================================</span></span><span
class=apple-style-span><span style='font-size:11.5pt'> </span></span><span
style='font-size:11.5pt'><br>
</span><span class=apple-style-span><span style='font-size:10.0pt;font-family:
"Calibri","sans-serif"'> </span></span><span class=apple-style-span><span
style='font-size:11.5pt'> </span></span><span style='font-size:11.5pt'><br>
</span><span class=apple-style-span><span style='font-size:10.0pt;font-family:
SimSun'>Q1: tough question to answer, as it depends on a number of factors.
What is the average packet size? Are you just counting the packets, or dumping
them to disk/DB/...? Do you see packet drops in the pcap_stats?</span></span><span
style='font-size:10.0pt;font-family:SimSun'><br>
<span class=apple-style-span>Q2: I will need to run some tests on this. Do you
have some minimal sample code that shows the issue?</span><br>
<span class=apple-style-span>Q3: Yes, *if* the NIC (and NIC driver) are not
dropping packets themselves. </span><br>
<br>
<span class=apple-style-span>Did you disable any protocol bound to the NIC
where you receive all these packets?</span><br>
<br>
<span class=apple-style-span>Have a nice day</span><br>
<span class=apple-style-span>GV</span><br>
<br>
<br>
<br>
<br>
<span class=apple-style-span>From: yulou liu </span><br>
<span class=apple-style-span>Sent: Wednesday, August 25, 2010 11:08 AM</span><br>
<span class=apple-style-span>To: winpcap </span><br>
<span class=apple-style-span>Subject: [Winpcap-users] About the packets loss ,
what is the bottleneck ?</span><br>
<br>
<br>
<span class=apple-style-span>I'm using winpcap to capture datas from FPGA board
via 1G Ethernet connection(directly connected).</span><br>
<span class=apple-style-span>the FPGA is configed to send data at a special
rate. With the fpga sending rate increases,especially above 500Mbps, it
sometimes loss packets. </span><br>
<br>
<span class=apple-style-span>Q1: Is it possible to totally avoid packets
loss by optimism the code? I want to collect datas at speed 614Mbps
without packet loss (collecting all datas </span><br>
<br>
<span class=apple-style-span>last about 1 minutes). My workstations features
with 2 Xeon CPU (each has 4core), DDR3 SDRAM, 1 G onboard netcard.
Which part is the most probably bottleneck ?</span><br>
<br>
<span class=apple-style-span>Q2: As a test , I found the pcap_next_ex()
can't get any packets when the user buffer is set over 64 MB, what
does it happen ?</span><br>
<br>
<span class=apple-style-span>Q3: if the Kernel buffer's size is 16MB , then the
first 16Mb packets from fpga won't be lost, so , if I set the
kernel buffer as 128MB , then at least </span><br>
<br>
<span class=apple-style-span>the first coming 128MB data from FPGA won't be
lost either ? but I found When I set the kernel buffer bigger than 100
MB, the packets drops is getting </span><br>
<br>
<span class=apple-style-span>worse. </span><br>
<br>
<br>
<span class=apple-style-span>Thanks a lot!</span></span><span
class=apple-style-span><span style='font-size:11.5pt'><o:p></o:p></span></span></p>
<pre><o:p> </o:p></pre></div>
<div class=MsoNormal align=center style='text-align:center'>
<hr size=2 width="100%" align=center>
</div>
<p class=MsoNormal>_______________________________________________<br>
Winpcap-users mailing list<br>
Winpcap-users@winpcap.org<br>
https://www.winpcap.org/mailman/listinfo/winpcap-users<o:p></o:p></p>
</div>
</div>
</body>
</html>