Re: F1 TDC Meeting

[Graham Heyes <heyes@jlab.org>]

Ok, sorry folks, can't sit still for this one. If sounds like a really nice
idea, but...

1) It throws out any hope of having something which conforms to a recognized
standard. Each board sits on a network and uses some "to be determined"
protocol to move data. You can't, at a later date, buy a commercial product
to plug into the system. Everything has to be custom.

2) It looks nice that each board has it's own 10MByte/S line into the switch
but what sits on the other side of the switch? There are two options. Either
you gather your 24 data streams into a single CPU then forward the merged
streams to a central point for event building, or you take the 24xN
individual streams from all the boards in the system to a single point. The
first case is a single CPU per crate and isn't a cost saving. The second
scheme requires hundreds of individual network connections handled by a
single machine which may be a serous bottleneck.

3) Reliability. Network based systems require more "care and feeding" than
bus based systems. You have to deal with identification of a data source on
the network. Transfer protocol. Retry mechanisms. Connection failure modes,
etc. The bulk of the "problem code" with CODA was making the network data
transfer robust at high data rates for connections live over hours/days and
we were only dealing with a few tens of data sources.

4) CPU resources. Is it clear that once can put a cheap enough CPU on each
board which can handle the network transfer? Is it clear what the data
destination is and how it will perform?

5) Ok, it's not all bad. This idea has actually been done several times
before for smaller experiments. A few years back I was on the review panel
for an experiment at RIHC, the name Phobos springs to mind. Anyway they
proposed something similar. Their data fell into a FIFO on each board and
was serialized via a TAXI onto an optical fiber. The boards, and the whole
front end, had no CPUs at all. The data was in free fall all the way from
the digitizer through the front end FIFO, down the fiber and into a FIFO on
a "receiver board" several front end modules were serviced by one receiver.
By free fall I mean that there was no feedback or flow control so the
protocol was very simple. These receiver boards were standard 6U VME
containing a big FIFO and were read out by a CPU in the VME crate (running a
modified version of a CODA readout controller). The CPU in VME performed
partial event building and the partial events from several VME crates were
shipped over a "real network" to a workstation where the event building was
completed.

At CERN in about 1986 we had a similar design using a copper based network
instead of optical. Anyway the main idea is to get around my objections
listed in items 2-4 by not using a commercial network like Ethernet but
using some simple hardware driven protocol. The cabling for these systems is
simpler too. Again the problem here is that objection 1 still holds since
the system is not standards based and is hard to maintain for the long haul.

    This started out as my 10c worth but looks like at least $1.50 ;^)

    Regards as always,

                     Graham
----------------------------------------------------------------------------
Beware of Geeks bearing GIFS. http://www.jlab.org/coda Jlab DAQ group



on 1/16/01 11:08 AM, Elton Smith at elton@jlab.org wrote:

> 
> 
> ---------- Forwarded message ----------
> Date: Tue, 16 Jan 2001 11:07:16 -0500 (EST)
> From: Paul Smith <paul@xanadu.physics.indiana.edu>
> To: elton@jlab.org, barbosa@jlab.org, jastrzem@jlab.org
> Cc: doughty@jlab.org, doughty@pcs.cnu.edu, crsteffe@indiana.edu
> Subject: Re: F1 TDC Meeting (fwd)
> 
> I've been thinking about the VME vs VMEP crates for Hall D.  It seems to me
> the 
> model of a crate with a backplane and a single board computer per crate is
> obsolete.
> 
> Instead, we should think about having a CPU on each front end card with a
> 100baseT interface.  A switch with 24 100baseT ports can be purchased for
> around 
> $500, which is much less expensive than a VME64 backplane and plenty fast
> enough.
> 
> We should schedule some time at the collaboration meeting to talk about DAQ,
> but 
> in the meantime, please poke holes in this idea.
> 
> Paul Smith
> 
> 
>