[Winpcap-users] is winpcap appropriate for my gigabit ethernetdevice?

[Zafer SAVAS]

Hi Max,

I have been working on a similar project like yours. My aim was to transfer LARGE AMOUNT of data (like ~110MBytes/sec) from a FPGA based hardware to a standard PC. My FPGA is a Virtex-5 xilinx FPGA and it has gigabit ethernet core in it. To maximize the speed I am avoiding any header and sending raw ethernet frames (Layer-2) with format
-6 bytes dest MAC
-6 bytes source MAC
-2 bytes type/length
-x bytes data
By the way there is a cross link with the PC and the FPGA and no hubs/routers etc...


Here are my answers for your questions:

Q: 
So my first question is, "does winpcap do what I need?".
A:
Yes, it is DEFINITELY possible. Thats something what winpcap is capable of doing. It can capture/send raw ethernet packets.

Q: 
I hope winpcap can simply "give my application the packet"  and do no [additional] processing whatsoever.
A:
Without any processing whole packet can be passed to the user side.

And here are some advices for your application:
Winpcap can nearly capture gigabit ethernet packets at line speed (which is about 120MBytes/sec) but it is very dependent on your hardware and your software. You can capture packets but if you process and try saving the packets to harddisk with a standard PC YOU WILL DEFINITELY LOSE PACKETS. 

To avoid these problems, here are some advices for you:
- Use Multi-CPU PC
- Allocate a large amount of kernel and user RAM in your applciation. (For example RAM for 1G packets which requires a large amount of RAM)
- Copy group of packets.
- Do not process them after you capture
- Use RAID disks to speed up your write to HD.

As I said I achieved ~110MBytes/sec of data and in my application. There are two threads for achieving this;
- 1 thread for capturing packets and writing them to user RAM 
- and the other one for copying the packets to Hard-drive.

God bless winpcap and its creators :)

Good luck
zafer.

Zafer SAVAS
Senior Expert Test Engineer
Aselsan Inc.



-----Original Message-----
From: winpcap-users-bounces at winpcap.org on behalf of max reason
Sent: Thu 04.09.2008 12:52
To: winpcap-users at winpcap.org
Subject: [Winpcap-users] is winpcap appropriate for my gigabit ethernetdevice?
 

 This is my first post, since I just learned about winpcap.
 I suspect my application for winpcap is unusual, so I'd like
 to ask others with winpcap experience whether winpcap will
 do what I need.  So, here goes.

 I designed an inexpensive but high-speed, high-resolution
 digital video camera with 5 megapixel [and 8 megapixel] CCDs.
 To keep the per-unit cost low, I had to find some way to get
 151,165,440 bytes per second from camera to PC - which equals
 roughly 1.2 gigabits per second.  My solution was to base the
 camera circuit on a cyclone3 FPGA and gigabit ethernet PHY.
 Plus I had to concoct a lossless compression system that is
 both brain-dead simple, and simple and fast enough to compress
 100 million 16-bit pixels per second enough to reduce the
 bandwidth far enough below 1 gigabit per second to squeeze
 the data through the PHY along with ethernet packet overhead.

 The bottom line is the following.  For this to work reliably,
 the packets need to be captured, then stored and/or displayed
 with as little overhead as possible.  Thet's probably obvious!

 To make this work, the ethernet cable from the camera must plug
 directly into the RJ45 connector on the PC - no routers/hubs/etc.
 The connection must be direct connect, or no way.

 My plan has been to send the data in a simple ethernet packet:
   6-byte target-MAC-address
   6-byte source-MAC-address
   2-byte length-field
   data (2592 pixels)
   CRC32

 I didn't learn until recently that PC operating systems do not
 typically receive or send such fundamental ethernet packets,
 and support might be non-trivial.  I wrote some prototype code
 on Linux with PF_PACKET that appears to provide me a way to
 process these fundamental packets efficiently, and avoid ALL
 subsequent [higher-level protocol] processing by the OS.

 But so far, I have not found a similar mechanism in win32.
 However, if I understand the nature of winpcap correctly,
 winpcap appears to provide similar capabilities.

 So my first question is, "does winpcap do what I need?".

 I think "what I need" is fairly obvious, but I'll rephrase
 it this way.  On the "PC receives data" side, I need a way to
 receive the packet - at least the data payload, but also the
 header if feasible.  I prefer to receive the CRC32 at the end
 of the packet (after the data), but can live without it.

 On the "PC sends data" side, I need to send short, simple
 command packets fairly rarely - no more often than one per
 image == 1944 ~ 2448 received packets.  And that packet
 can and should be sent when no data is being received
 (during a short pause between images).  These packets
 should also be plain, ordinary ethernet packets with
 the same fields shown above.

 I very much prefer to ***avoid*** IP, UDP, TCP/IP packets!
 It is difficult enough to make the FPGA control and receive
 data from four CCDs simultaneously (?did I mention that?),
 plus perform realtime lossless data compression, plus send
 the compressed data through the gigabit ethernet PHY also
 in realtime (albeit one scanline later == pipelined).

 To waste space, bandwidth and complexity in the packet header
 on totally useless NOTHINGNESS (to my application anyway),
 is just stupid.  To create those larger packets just tempts
 the OS to perform some packet processing, which just consumes
 CPU bandwidth that my application need for other work (say,
 saving the data to disk, or blasting it into a texture-map
 in a GPU card, or etc).  It also throws a complexity-wrench
 into my plans for computing the CRC32 in the FPGA.

 I hope winpcap can simply "give my application the packet"
 and do no [additional] processing whatsoever.  It is also
 very important that the OS not waste its time trying to
 process the packet too.  I mean, what on earth for?

 And no, I can't just slow down the frame rate a little, to
 whatever bandwidth is "feasible".  Some of the applications
 of this device require "super-cinema HDTV" media output,
 which is 2592 x 1080 x 24 FPS --- and the frame rate is
 fixed by convention (and the equipment that processes it).
 And that's not the only application that requires this
 little device crank out data out at nearly 1 gigabit.
 PS:  I suspect the actual data-payload rate can be in
 the neighborhood of 900Mbps (maybe even 850Mbps), but
 it must not "studder or skip" on frames that contain
 "complexity" the lossless compression cannot compress
 as much as "easy" frames.  That's an absolute no-can-do.

 I thank whoever decides to answer my question.  I am happy
 to dig-in, read and learn how to hook winpcap into my code.
 I just want to make sure I don't do all that work just to
 discover winpcap can't do what I need.  Thanks, Max.

_______________________________________________
Winpcap-users mailing list
Winpcap-users at winpcap.org
https://www.winpcap.org/mailman/listinfo/winpcap-users

######################################################################
Dikkat:

Bu elektronik posta mesaji kisisel ve ozeldir. Eger size 
gonderilmediyse lutfen gondericiyi bilgilendirip mesaji siliniz. 
Firmamiza gelen ve giden mesajlar virus taramasindan gecirilmekte, 
guvenlik nedeni ile kontrol edilerek saklanmaktadir. Mesajdaki 
gorusler ve bakis acisi gondericiye ait olup Aselsan A.S. resmi 
gorusu olmak zorunda degildir.

######################################################################
Attention: 

This e-mail message is privileged and confidential. If you are 
not the intended recipient please delete the message and notify 
the sender. E-mails to and from the company are monitored for 
operational reasons and in accordance with lawful business practices. 
Any views or opinions presented are solely those of the author and 
do not necessarily represent the views of the company.

######################################################################