Многоядерная микроархитектура Cell/BE: История и применение.

Multi core microprocessor microarchitecture

Cell is a 64 bit multi core microprocessor microarchitecture that combines a general purpose PowerPC core of modest performance with streamlined coprocessing elements which greatly accelerate multimedia and vector processing applications, as well as many other forms of dedicated computation. Cell is shorthand for Cell Broadband Engine Architecture, commonly abbreviated CBEA in full or Cell BE in part. The first major commercial application of Cell was in Sony's PlayStation 3 game console, released in 2006. In May 2008, the Cell based IBM Roadrunner supercomputer became the first TOP500 LINPACK sustained 1.0 petaflops system. Mercury Computer Systems also developed designs based on the Cell. The Cell architecture includes a memory coherence architecture that emphasizes power efficiency, prioritizes bandwidth over low latency, and favors peak computational throughput over the simplicity of program code. For these reasons, Cell is widely regarded as a challenging environment for software development. IBM provides a Linux based development platform to help developers program for Cell chips.

In mid 2000, Sony Computer Entertainment, Toshiba Corporation, and IBM formed an alliance known as "STI" to design and manufacture the processor. The STI Design Center opened in March 2001. The Cell was designed over a period of four years, using enhanced versions of the design tools for the POWER4 processor. Over 400 engineers from the three companies worked together in Austin, with critical support from eleven of IBM's design centers. with Bandai Namco Entertainment using the Cell/BE processor for their 357 arcade board as well as the subsequent 369. In February 2008, IBM announced that it would begin to fabricate Cell processors with the 45 nm process. In May 2008, IBM introduced the high performance double precision floating point version of the Cell processor, the PowerXCell 8i, at the 65 nm feature size. In May 2008, an Opteron and PowerXCell 8i based supercomputer, the IBM Roadrunner system, became the world's first system to achieve one petaFLOPS, and was the fastest computer in the world until third quarter 2009. The world's three most energy efficient supercomputers, as represented by the Green500 list, are similarly based on the PowerXCell 8i. In August 2009 the 45 nm Cell processor was introduced in concert with Sony's PlayStation 3 Slim. By November 2009, IBM had discontinued the development of a Cell processor with 32 APUs but was still developing other Cell products.

On May 17, 2005, Sony Computer Entertainment confirmed some specifications of the Cell processor that would be shipping in the then forthcoming PlayStation 3 console. This Cell configuration has one PPE on the core, with eight physical SPEs in silicon. Mercury has since then released blades, conventional rack servers and PCI Express accelerator boards with Cell processors. Sony's high performance media computing server ZEGO uses a 3.2 GHz Cell/B. E processor.

The Cell Broadband Engine, or Cell as it is more commonly known, is a microprocessor intended as a hybrid of conventional desktop processors (such as the Athlon 64, and Core 2 families) and more specialized high performance processors, such as the NVIDIA and ATI graphics processors (GPUs). The longer name indicates its intended use, namely as a component in current and future online distribution systems; as such it may be utilized in high definition displays and recording equipment, as well as HDTV systems. Additionally the processor may be suited to digital imaging systems (medical, scientific, etc.) and physical simulation (e. g., scientific and structural engineering modeling). As used in the PlayStation 3 it has 250 million transistors. In a simple analysis, the Cell processor can be split into four components: external input and output structures, the main processor called the Power Processing Element (PPE) (a two way simultaneous multithreaded PowerPC 2.02 core), eight fully functional co processors called the Synergistic Processing Elements, or SPEs, and a specialized high bandwidth circular data bus connecting the PPE, input/output elements and the SPEs, called the Element Interconnect Bus or EIB. To achieve the high performance needed for mathematically intensive tasks, such as decoding/encoding MPEG streams, generating or transforming three dimensional data, or undertaking Fourier analysis of data, the Cell processor marries the SPEs and the PPE via EIB to give access, via fully cache coherent DMA (direct memory access), to both main memory and to other external data storage. To make the best of EIB, and to overlap computation and data transfer, each of the nine processing elements (PPE and SPEs) is equipped with a DMA engine. Since the SPE's load/store instructions can only access its own local scratchpad memory, each SPE entirely depends on DMAs to transfer data to and from the main memory and other SPEs' local memories. A DMA operation can transfer either a single block area of size up to 16KB, or a list of 2 to 2048 such blocks. One of the major design decisions in the architecture of Cell is the use of DMAs as a central means of intra chip data transfer, with a view to enabling maximal asynchrony and concurrency in data processing inside a chip. The PPE, which is capable of running a conventional operating system, has control over the SPEs and can start, stop, interrupt, and schedule processes running on the SPEs. To this end, the PPE has additional instructions relating to the control of the SPEs. Unlike SPEs, the PPE can read and write the main memory and the local memories of SPEs through the standard load/store instructions. Despite having Turing complete architectures, the SPEs are not fully autonomous and require the PPE to prime them before they can do any useful work. As most of the "horsepower" of the system comes from the synergistic processing elements, the use of DMA as a method of data transfer and the limited local memory footprint of each SPE pose a major challenge to software developers who wish to make the most of this horsepower, demanding careful hand tuning of programs to extract maximal performance from this CPU. The PPE and bus architecture includes various modes of operation giving different levels of memory protection, allowing areas of memory to be protected from access by specific processes running on the SPEs or the PPE. Both the PPE and SPE are RISC architectures with a fixed width 32 bit instruction format. The PPE contains a 64 bit general purpose register set (GPR), a 64 bit floating point register set (FPR), and a 128 bit Altivec register set. The SPE contains 128 bit registers only. These can be used for scalar data types ranging from 8 bits to 64 bits in size or for SIMD computations on a variety of integer and floating point formats. System memory addresses for both the PPE and SPE are expressed as 64 bit values for a theoretic address range of 264 bytes (16 exabytes or 16,777,216 terabytes). In practice, not all of these bits are implemented in hardware. Local store addresses internal to the SPU (Synergistic Processor Unit) processor are expressed as a 32 bit word. In documentation relating to Cell a word is always taken to mean 32 bits, a doubleword means 64 bits, and a quadword means 128 bits.

In 2008, IBM announced a revised variant of the Cell called the PowerXCell 8i, which is available in QS22 Blade Servers from IBM. The PowerXCell is manufactured on a 65 nm process, and adds support for up to 32 GB of slotted DDR2 memory, as well as dramatically improving double precision floating point performance on the SPEs from a peak of about 12.8 GFLOPS to 102.4 GFLOPS total for eight SPEs, which, coincidentally, is the same peak performance as the NEC SX 9 vector processor released around the same time. The IBM Roadrunner supercomputer, the world's fastest during 2008–2009, consisted of 12,240 PowerXCell 8i processors, along with 6,562 AMD Opteron processors. The PowerXCell 8i powered super computers also dominated all of the top 6 "greenest" systems in the Green500 list, with highest MFLOPS/Watt ratio supercomputers in the world. Beside the QS22 and supercomputers, the PowerXCell processor is also available as an accelerator on a PCI Express card and is used as the core processor in the QPACE project. Since the PowerXCell 8i removed the RAMBUS memory interface, and added significantly larger DDR2 interfaces and enhanced SPEs, the chip layout had to be reworked, which resulted in both larger chip die and packaging.

While the Cell chip can have a number of different configurations, the basic configuration is a multi core chip composed of one "Power Processor Element" ("PPE") (sometimes called "Processing Element", or "PE"), and multiple "Synergistic Processing Elements" ("SPE"). The PPE and SPEs are linked together by an internal high speed bus dubbed "Element Interconnect Bus" ("EIB").

The PPE is the PowerPC based, dual issue in order two way simultaneous multithreaded CPU core with a 23 stage pipeline acting as the controller for the eight SPEs, which handle most of the computational workload. PPE has limited out of order execution capabilities; it can perform loads out of order and has delayed execution pipelines. The PPE will work with conventional operating systems due to its similarity to other 64 bit PowerPC processors, while the SPEs are designed for vectorized floating point code execution. The PPE contains a 32 KiB level 1 instruction cache, a 32 KiB level 1 data cache, and a 512 KiB level 2 cache. The size of a cache line is 128 bytes in all caches. which is fully pipelined for single precision floating point (Altivec 1 does not support double precision floating point vectors. ), 32 bit Fixed Point Unit (FXU) with 64 bit register file per thread, Load and Store Unit (LSU), 64 bit Floating Point Unit (FPU), Branch Unit (BRU) and Branch Execution Unit(BXU).

The PPE was designed specifically for the Cell processor but during development, Microsoft approached IBM wanting a high performance processor core for its Xbox 360. IBM complied and made the tri core Xenon processor, based on a slightly modified version of the PPE with added VMX128 extensions.

Each SPE is a dual issue in order processor composed of a "Synergistic Processing Unit", SPU, and a "Memory Flow Controller", MFC (DMA, MMU, and bus interface). SPEs do not have any branch prediction hardware (hence there is a heavy burden on the compiler). Each SPE has 6 execution units divided among odd and even pipelines on each SPE : The SPU runs a specially developed instruction set (ISA) with 128 bit SIMD organization for single and double precision instructions. With the current generation of the Cell, each SPE contains a 256 KiB embedded SRAM for instruction and data, called "Local Storage" (not to be mistaken for "Local Memory" in Sony's documents that refer to the VRAM) which is visible to the PPE and can be addressed directly by software. Each SPE can support up to 4 GiB of local store memory. The local store does not operate like a conventional CPU cache since it is neither transparent to software nor does it contain hardware structures that predict which data to load. The SPEs contain a 128 bit, 128 entry register file and measures 14.5 mm2 on a 90 nm process. An SPE can operate on sixteen 8 bit integers, eight 16 bit integers, four 32 bit integers, or four single precision floating point numbers in a single clock cycle, as well as a memory operation. Note that the SPU cannot directly access system memory; the 64 bit virtual memory addresses formed by the SPU must be passed from the SPU to the SPE memory flow controller (MFC) to set up a DMA operation within the system address space. In one typical usage scenario, the system will load the SPEs with small programs (similar to threads), chaining the SPEs together to handle each step in a complex operation. For instance, a set top box might load programs for reading a DVD, video and audio decoding, and display and the data would be passed off from SPE to SPE until finally ending up on the TV. Another possibility is to partition the input data set and have several SPEs performing the same kind of operation in parallel. At 3.2 GHz, each SPE gives a theoretical 25.6 GFLOPS of single precision performance. Compared to its personal computer contemporaries, the relatively high overall floating point performance of a Cell processor seemingly dwarfs the abilities of the SIMD unit in CPUs like the Pentium 4 and the Athlon 64. However, comparing only floating point abilities of a system is a one dimensional and application specific metric. Unlike a Cell processor, such desktop CPUs are more suited to the general purpose software usually run on personal computers. In addition to executing multiple instructions per clock, processors from Intel and AMD feature branch predictors. The Cell is designed to compensate for this with compiler assistance, in which prepare to branch instructions are created. For double precision floating point operations, as sometimes used in personal computers and often used in scientific computing, Cell performance drops by an order of magnitude, but still reaches 20.8 GFLOPS (1.8 GFLOPS per SPE, 6.4 GFLOPS per PPE). The PowerXCell 8i variant, which was specifically designed for double precision, reaches 102.4 GFLOPS in double precision calculations. Tests by IBM show that the SPEs can reach 98% of their theoretical peak performance running optimized parallel matrix multiplication. In total, the SPEs have 2 MB of local memory.

On August 29, 2007, IBM announced the BladeCenter QS21. Generating a measured 1.05 giga–floating point operations per second (gigaFLOPS) per watt, with peak performance of approximately 460 GFLOPS it is one of the most power efficient computing platforms to date. A single BladeCenter chassis can achieve 6.4 tera–floating point operations per second (teraFLOPS) and over 25.8 teraFLOPS in a standard 42U rack. On May 13, 2008, IBM announced the BladeCenter QS22. The QS22 introduces the PowerXCell 8i processor with five times the double precision floating point performance of the QS21, and the capacity for up to 32 GB of DDR2 memory on blade. IBM has discontinued the Blade server line based on Cell processors as of January 12, 2012.

Several companies provide PCI e boards utilising the IBM PowerXCell 8i. The performance is reported as 179.2 GFlops (SP), 89.6 GFlops (DP) at 2.8 GHz.

Sony's PlayStation 3 video game console was the first production application of the Cell processor, clocked at 3.2 GHz and containing seven out of eight operational SPEs, to allow Sony to increase the yield on the processor manufacture. Only six of the seven SPEs are accessible to developers as one is reserved by the OS.

Toshiba has produced HDTVs using Cell. They presented a system to decode 48 standard definition MPEG 2 streams simultaneously on a 1920×1080 screen. This can enable a viewer to choose a channel based on dozens of thumbnail videos displayed simultaneously on the screen.

IBM's supercomputer, IBM Roadrunner, was a hybrid of General Purpose x86 64 Opteron as well as Cell processors. This system assumed the #1 spot on the June 2008 Top 500 list as the first supercomputer to run at petaFLOPS speeds, having gained a sustained 1.026 petaFLOPS speed using the standard LINPACK benchmark. IBM Roadrunner used the PowerXCell 8i version of the Cell processor, manufactured using 65 nm technology and enhanced SPUs that can handle double precision calculations in the 128 bit registers, reaching double precision 102 GFLOPs per chip.

Clusters of PlayStation 3 consoles are an attractive alternative to high end systems based on Cell blades. Innovative Computing Laboratory, a group led by Jack Dongarra, in the Computer Science Department at the University of Tennessee, investigated such an application in depth. Terrasoft Solutions is selling 8 node and 32 node PS3 clusters with Yellow Dog Linux pre installed, an implementation of Dongarra's research. As first reported by Wired on October 17, 2007, an interesting application of using PlayStation 3 in a cluster configuration was implemented by Astrophysicist Gaurav Khanna, from the Physics department of University of Massachusetts Dartmouth, who replaced time used on supercomputers with a cluster of eight PlayStation 3s. Subsequently, the next generation of this machine, now called the PlayStation 3 Gravity Grid, uses a network of 16 machines, and exploits the Cell processor for the intended application which is binary black hole coalescence using perturbation theory. In particular, the cluster performs astrophysical simulations of large supermassive black holes capturing smaller compact objects and has generated numerical data that has been published multiple times in the relevant scientific research literature. The Cell processor version used by the PlayStation 3 has a main CPU and 6 SPEs available to the user, giving the Gravity Grid machine a net of 16 general purpose processors and 96 vector processors. The machine has a one time cost of $9,000 to build and is adequate for black hole simulations which would otherwise cost $6,000 per run on a conventional supercomputer. The black hole calculations are not memory intensive and are highly localizable, and so are well suited to this architecture. Khanna claims that the cluster's performance exceeds that of a 100+ Intel Xeon core based traditional Linux cluster on his simulations. The PS3 Gravity Grid gathered significant media attention through 2007, 2008, 2009, and 2010. The computational Biochemistry and Biophysics lab at the Universitat Pompeu Fabra, in Barcelona, deployed in 2007 a BOINC system called PS3GRID for collaborative computing based on the CellMD software, the first one designed specifically for the Cell processor. The United States Air Force Research Laboratory has deployed a PlayStation 3 cluster of over 1700 units, nicknamed the "Condor Cluster", for analyzing high resolution satellite imagery. The Air Force claims the Condor Cluster would be the 33rd largest supercomputer in the world in terms of capacity. The lab has opened up the supercomputer for use by universities for research.

With the help of the computing power of over half a million PlayStation 3 consoles, the distributed computing project Folding@home has been recognized by Guinness World Records as the most powerful distributed network in the world. The first record was achieved on September 16, 2007, as the project surpassed one petaFLOPS, which had never previously been attained by a distributed computing network. Additionally, the collective efforts enabled PS3 alone to reach the petaFLOPS mark on September 23, 2007. In comparison, the world's second most powerful supercomputer at the time, IBM's Blue Gene/L, performed at around 478.2 teraFLOPS, which means Folding@home's computing power is approximately twice Blue Gene/L's (although the CPU interconnect in Blue Gene/L is more than one million times faster than the mean network speed in Folding@home). As of May 7, 2011, Folding@home runs at about 9.3 x86 petaFLOPS, with 1.6 petaFLOPS generated by 26,000 active PS3s alone.

IBM announced on April 25, 2007, that it would begin integrating its Cell Broadband Engine Architecture microprocessors into the company's System z line of mainframes. This has led to a gameframe.

The architecture of the processor makes it better suited to hardware assisted cryptographic brute force attack applications than conventional processors.

The PPE maintains a job queue, schedules jobs in SPEs, and monitors progress. Each SPE runs a "mini kernel" whose role is to fetch a job, execute it, and synchronize with the PPE.

The mini kernel and scheduling is distributed across the SPEs. Tasks are synchronized using mutexes or semaphores as in a conventional operating system. Ready to run tasks wait in a queue for an SPE to execute them. The SPEs use shared memory for all tasks in this configuration.

Each SPE runs a distinct program. Data comes from an input stream and is sent to SPEs. When an SPE has terminated the processing, the output data is sent to an output stream. This provides a flexible and powerful architecture for stream processing, and allows explicit scheduling for each SPE separately. Other processors are also able to perform streaming tasks but are limited by the kernel loaded.

In 2005, patches enabling Cell support in the Linux kernel were submitted for inclusion by IBM developers. Arnd Bergmann (one of the developers of the aforementioned patches) also described the Linux based Cell architecture at LinuxTag 2005. As of release 2.6.16 (March 20, 2006), the Linux kernel officially supports the Cell processor. Both PPE and SPEs are programmable in C/C++ using a common API provided by libraries. Fixstars Solutions provides Yellow Dog Linux for IBM and Mercury Cell based systems, as well as for the PlayStation 3. Terra Soft strategically partnered with Mercury to provide a Linux Board Support Package for Cell, and support and development of software applications on various other Cell platforms, including the IBM BladeCenter JS21 and Cell QS20, and Mercury Cell based solutions. Terra Soft also maintains the Y HPC (High Performance Computing) Cluster Construction and Management Suite and Y Bio gene sequencing tools. Y Bio is built upon the RPM Linux standard for package management, and offers tools which help bioinformatics researchers conduct their work with greater efficiency. IBM has developed a pseudo filesystem for Linux coined "Spufs" that simplifies access to and use of the SPE resources. IBM is currently maintaining a Linux kernel and GDB ports, while Sony maintains the GNU toolchain (GCC, binutils). In November 2005, IBM released a "Cell Broadband Engine (CBE) Software Development Kit Version 1.0", consisting of a simulator and assorted tools, to its web site. Development versions of the latest kernel and tools for Fedora Core 4 are maintained at the Barcelona Supercomputing Center website. In August 2007, Mercury Computer Systems released a Software Development Kit for PlayStation 3 for High Performance Computing. In November 2007, Fixstars Corporation released the new "CVCell" module aiming to accelerate several important OpenCV APIs for Cell. In a series of software calculation tests, they recorded execution times on a 3.2 GHz Cell processor that were between 6x and 27x faster compared with the same software on a 2.4 GHz Intel Core 2 Duo. In October 2009, IBM released an OpenCL driver for POWER6 and CBE. This allows programs written in the cross platform API to be easily run on Cell PSE.

Illustrations of the different generations of Cell/B. E. processors and the PowerXCell 8i. The images are not to scale; All Cell/B. E. packages measures 42.5×42.5 mm and the PowerXCell 8i measures 47.5×47.5 mm.