Pentek Tutorial

MicroTCA/AMC Solutions for Real-Time Data Acquisition

Originally designed for high-availability and cost-effective telecom systems, the ATCA (Advanced Telecommunications Computing Architecture) using AMC (Advanced Mezzanine Card) modules has evolved to MicroTCA, an evolving standard now capturing design wins for an increasing share of embedded real-time applications. Based on a well-defined gigabit serial backplane and switch topology, the MicroTCA, or ìTCA, platform provides a fast, flexible infrastructure for plug-in AMC modules, now available with functions well beyond traditional telecom. High-speed A/Ds, D/As and the latest FPGAs on new AMC modules take advantage of fast PCIe links to deliver data rates for demanding realtime data acquisition systems while lowering the cost over alternative architectures.

Real-Time Data Acquisition

In the purest sense, real-time data acquisition simply means acquiring data at a specified rate with the guarantee of no data loss. However, diverse markets are driving these rates to exponentially higher levels.

In most cases, data acquisition includes digitization of sensor signals followed by transmission, buffering, storing and processing. Communication systems for commercial and military applications struggle to meet user demands for more information. These applications include high-definition imaging, video and audio programming, streaming internet content, email traffic, large data files, and cloud storage for databases. Radar systems operating with new wideband waveforms, not only detect speed, range and direction of travel, but also capture complex information for target classification, and identification. Likewise, sonar, medical imaging, and security-scanning systems are migrating to higher resolution and higher frame rates.

These factors boost the required bandwidth for each signal channel. But because many of the sensors now have multiple elements, the number of channels is growing as well. As a result, demands on data acquisition hardware and system infrastructure have outstripped older openarchitecture embedded systems due to data transfer bottlenecks into and across the backplane. Not only must sensor signals be digitized, but the data must also be delivered to useful system destinations such as shared memory, communication links or storage disks.

As an example, a four-channel 200 MHz 16-bit A/D converter is a relatively popular configuration for embedded data acquisition in many of the above applications. Operating at full speed, one such module generates data samples at a rate of 1600 MB/sec.

Parallel-bus architectures such as VMEbus or CompactPCI, with peak data transfer capacities across the backplane between 160 and 800 MB/sec, are completely overwhelmed by just a single module. This shortfall spurred the development of gigabit serial backplanes and motherboards including standards such as PCI Express, OpenVPX, CompactPCI Serial, ATCA, MicroTCA and others.

Highlights of MicroTCA and AMC

In 2005, ATCA vendors announced a replacement for the older parallel-bus PMC modules found on the first ATCA boards. These new mezzanine modules (AMCs) are daughter cards that add various analog and digital I/O functions to ATCA system boards, including A/D and D/A converters, DSPs, FPGAs, CPUs, network interfaces for copper and optical links, graphics engines and storage interfaces. Offered in six different sizes, they support several different chassis and carrier board shapes. Figure 1 on the next page, shows the Pentek Model 56660 4-Channel 200 MHz A/D AMC, a single-width, full-height module based on the AMC.1 Specification that defines PCIe as the backplane fabric interface. It consists of a carrier board housing an XMC module, with its fast PCIe Gen. 2 x4 interface delivered to the backplane connector.

AMC modules offer many advantages that are directly suitable to real-time data acquisition. They support hot-swap capability and IPMI (Intelligent Platform Management Interface) features for high availability, built-in testing, and performance monitoring for mission critical applications.

AMCs offer a rich collection of gigabit serial interfaces including GbE (Gigabit Ethernet) to system controllers, SATA (Serial ATA), SAS (Serial-Attached SCSI) or FC (Fibre Channel) links to storage peripherals, and XAUI (10 GbE Attachment Unit Interface), PCIe or SRIO (Serial RapidIO) links for high-speed data transfer through the system connector.

As AMC modules rapidly gained popularity, developers sought a way to retarget them as independent plug-in modules for a simple backplane system architecture instead of as daughter cards for ATCA carrier boards. This effort evolved to the MicroTCA architecture. Many of the concepts from ATCA were pulled forward into this new MicroTCA mechanical chassis configuration to take advantage of the wealth of system management, protocol, and industry infrastructure already in place for ATCA.

The MicroTCA Carrier is an essential aspect of this new architecture. Shown in Figure 2, the Carrier incorporates all elements of a complete system and typically accepts twelve AMC modules. These include a card cage or "shelf" to house the plug-in AMC modules and a backplane that engages with all of the power and signal pins on the AMC connectors. Also connected to the backplane are one or two MCHs (MicroTCA Carrier Hubs), one to four PMs (Power Modules), and one or two CUs (Cooling Units).

The MCH includes fabric and Ethernet switches for at least one GbE port plus four lanes of XAUI, PCIe or SRIO to each of the twelve AMC modules. In this way, all of the AMC modules can communicate over Ethernet and send fabric data to each other.

The MCH also provides a fabric channel uplink and Ethernet ports to other systems. The MCMC (MicroTCA Hub Management Controller) performs management services for twelve AMCs, four Power Modules and two Cooling Units. Other MCMC functions include shelf management, clock distribution, and alarms.

MicroTCA System Example

Because of the fast gigabit serial fabric links, MicroTCA offers an attractive platform for high-speed data acquisition systems. As an example, the Pentek Model 56660 AMC module in Figure 3 is quite suitable as the front end of a real-time recording system. The four A/D converters digitize front- panel analog inputs, each producing 200 MSamples/ sec. With two bytes per sample, this means 400 MB/sec per channel, or 1600 MB/sec for the 4-channel AMC module. This rather demanding traffic load can be accommodated through the PCIe Gen 2 x4 backplane fabric interface, which supports a peak transfer rate of 2000 MB/sec.

Figure 4 on the next page shows a simplified block diagram of a complete MicroTCA recording system, showing connectivity of the important PCIe fabric links. An SBC (Single Board Computer) AMC module hosts the operating system and provides system memory accessible through its PCIe link to the MCH switch. A RAID Controller AMC module, also connected via PCIe through the MCH fabric switch, offers eight SATA-III ports to eight solid-state drives, each capable of read/write speeds of over 300 MB/sec.

Pentek's SystemFlow® recording software runs on the SBC to orchestrate real-time data transfers among these three AMC modules by managing hardware DMA (Direct Memory Access) controllers using the PCIe fabric links. Parameters are sent to these linkedlist DMA engines to specify the size and destination of data blocks to be moved to or from system memory on the SBC. Once a DMA block transfer is completed, the next DMA operation starts automatically, and the CPU receives a notification interrupt so it can monitor progress.

Specifically the DMA controller on the AMC data acquisition module moves blocks of A/D data into circular buffers on system memory of the SBC. Then, the DMA controller on the RAID controller AMC moves data from completed system memory blocks to the RAID controller. Finally, the RAID controller "stripes" data by writing simultaneously across eight SSDs to achieve aggregate storage speeds to the RAID array of over 2000 MB/sec.

This scheme ensures that the CPU does not touch any data, so that Windows or Linux host operating systems impose no adverse effect on maintaining sustained real-time recordings. Also, the data on the RAID array is stored in NTFS format so it is immediately available for analysis, display or processing applications running on the host CPU.

MicroTCA: Ready for Real-Time

Its well-defined, fast and straightforward architecture makes MicroTCA a serious contender for high-end, real-time embedded systems. Using popular gigabit serial fabrics like XAUI, PCIe and SRIO, it delivers substantial backplane bandwidth, more than adequate for many applications. It also provides a rich system infrastructure that includes various form factors, power and cooling strategies, system management facilities, high availability and redundancy features, all in a highly modular design.

Because of the wealth of MicroTCA products available for the cost-sensitive telecom community, systems are often 20% to 40% less expensive than comparable systems using more traditional embedded card cage architectures. Extensions to the base specification now support ruggedized MicroTCA systems with both air- and conduction-cooled AMC modules, including specifications for shock, altitude and vibration required for many government and military applications.

System level software written for PCIe or SRIO systems is highly portable to MicroTCA because of the standard fabric interconnects and the operating systems supported by available SBC AMC modules. All in all, MicroTCA is definitely worth considering for your next data acquisition system.

Pentek Tutorial