Бейімделген бейне кодегі: H.264 стандартының шолуы

Most widely used standard for video compression

Advanced Video Coding (AVC), also referred to as H.264 or MPEG 4 Part 10, is a video compression standard based on block oriented, motion compensated coding. It is by far the most commonly used format for the recording, compression, and distribution of video content, used by 91% of video industry developers as of September 2019. It supports a maximum resolution of 8K UHD. The commercial use of patented H.264 technologies requires the payment of royalties to Via and other patent owners. MPEG LA has allowed the free use of H.264 technologies for streaming Internet video that is free to end users, and Cisco Systems paid royalties to MPEG LA on behalf of the users of binaries for its open source H.264 encoder openH264.

The H.264 name follows the ITU T naming convention, where Recommendations are given a letter corresponding to their series and a recommendation number within the series. H.264 is part of "H Series Recommendations: Audiovisual and multimedia systems". H.264 is further categorized into "H.200 H.499: Infrastructure of audiovisual services" and "H.260 H.279: Coding of moving video". The MPEG 4 AVC name relates to the naming convention in ISO/IEC MPEG, where the standard is part 10 of ISO/IEC 14496, which is the suite of standards known as MPEG 4. The standard was developed jointly in a partnership of VCEG and MPEG, after earlier development work in the ITU T as a VCEG project called H.26L. It is thus common to refer to the standard with names such as H.264/AVC, AVC/H.264, H.264/MPEG 4 AVC, or MPEG 4/H.264 AVC, to emphasize the common heritage. Occasionally, it is also referred to as "the JVT codec", in reference to the Joint Video Team (JVT) organization that developed it. (Such partnership and multiple naming is not uncommon. For example, the video compression standard known as MPEG 2 also arose from the partnership between MPEG and the ITU T, where MPEG 2 video is known to the ITU T community as H.262.) Some software programs (such as VLC media player) internally identify this standard as AVC1.

In early 1998, the Video Coding Experts Group (VCEG – ITU T SG16 Q.6) issued a call for proposals on a project called H.26L, with the target to double the coding efficiency (which means halving the bit rate necessary for a given level of fidelity) in comparison to any other existing video coding standards for a broad variety of applications. VCEG was chaired by Gary Sullivan (Microsoft, formerly PictureTel, U. S.). The first draft design for that new standard was adopted in August 1999. In 2000, Thomas Wiegand (Heinrich Hertz Institute, Germany) became VCEG co chair. In December 2001, VCEG and the Moving Picture Experts Group (MPEG – ISO/IEC JTC 1/SC 29/WG 11) formed a Joint Video Team (JVT), with the charter to finalize the video coding standard. Formal approval of the specification came in March 2003. The JVT was (is) chaired by Gary Sullivan, Thomas Wiegand, and Ajay Luthra (Motorola, U. S.: later Arris, U. S.). In July 2004, the Fidelity Range Extensions (FRExt) project was finalized. From January 2005 to November 2007, the JVT was working on an extension of H.264/AVC towards scalability by an Annex (G) called Scalable Video Coding (SVC). The JVT management team was extended by Jens Rainer Ohm (RWTH Aachen University, Germany). From July 2006 to November 2009, the JVT worked on Multiview Video Coding (MVC), an extension of H.264/AVC towards 3D television and limited range free viewpoint television. That work included the development of two new profiles of the standard: the Multiview High Profile and the Stereo High Profile. Throughout the development of the standard, additional messages for containing supplemental enhancement information (SEI) have been developed. SEI messages can contain various types of data that indicate the timing of the video pictures or describe various properties of the coded video or how it can be used or enhanced. SEI messages are also defined that can contain arbitrary user defined data. SEI messages do not affect the core decoding process, but can indicate how the video is recommended to be post processed or displayed. Some other high level properties of the video content are conveyed in video usability information (VUI), such as the indication of the color space for interpretation of the video content. As new color spaces have been developed, such as for high dynamic range and wide color gamut video, additional VUI identifiers have been added to indicate them.

The standardization of the first version of H.264/AVC was completed in May 2003. In the first project to extend the original standard, the JVT then developed what was called the Fidelity Range Extensions (FRExt). These extensions enabled higher quality video coding by supporting increased sample bit depth precision and higher resolution color information, including the sampling structures known as Y′CBCR 4:2:2 (a. k. a. YUV 4:2:2) and 4:4:4. Several other features were also included in the FRExt project, such as adding an 8×8 integer discrete cosine transform (integer DCT) with adaptive switching between the 4×4 and 8×8 transforms, encoder specified perceptual based quantization weighting matrices, efficient inter picture lossless coding, and support of additional color spaces. The design work on the FRExt project was completed in July 2004, and the drafting work on them was completed in September 2004. Five other new profiles (see version 7 below) intended primarily for professional applications were then developed, adding extended gamut color space support, defining additional aspect ratio indicators, defining two additional types of "supplemental enhancement information" (post filter hint and tone mapping), and deprecating one of the prior FRExt profiles (the High 4:4:4 profile) that industry feedback indicated should have been designed differently.

The next major feature added to the standard was Scalable Video Coding (SVC). Specified in Annex G of H.264/AVC, SVC allows the construction of bitstreams that contain layers of sub bitstreams that also conform to the standard, including one such bitstream known as the "base layer" that can be decoded by a H.264/AVC codec that does not support SVC. For temporal bitstream scalability (i. e., the presence of a sub bitstream with a smaller temporal sampling rate than the main bitstream), complete access units are removed from the bitstream when deriving the sub bitstream. In this case, high level syntax and inter prediction reference pictures in the bitstream are constructed accordingly. On the other hand, for spatial and quality bitstream scalability (i. e. the presence of a sub bitstream with lower spatial resolution/quality than the main bitstream), the NAL (Network Abstraction Layer) is removed from the bitstream when deriving the sub bitstream. In this case, inter layer prediction (i. e., the prediction of the higher spatial resolution/quality signal from the data of the lower spatial resolution/quality signal) is typically used for efficient coding. The Scalable Video Coding extensions were completed in November 2007.

The next major feature added to the standard was Multiview Video Coding (MVC). Specified in Annex H of H.264/AVC, MVC enables the construction of bitstreams that represent more than one view of a video scene. An important example of this functionality is stereoscopic 3D video coding. Two profiles were developed in the MVC work: Multiview High profile supports an arbitrary number of views, and Stereo High profile is designed specifically for two view stereoscopic video. The Multiview Video Coding extensions were completed in November 2009.

Additional extensions were later developed that included 3D video coding with joint coding of depth maps and texture (termed 3D AVC), multi resolution frame compatible (MFC) stereoscopic and 3D MFC coding, various additional combinations of features, and higher frame sizes and frame rates.

Versions of the H.264/AVC standard include the following completed revisions, corrigenda, and amendments (dates are final approval dates in ITU T, while final "International Standard" approval dates in ISO/IEC are somewhat different and slightly later in most cases). Each version represents changes relative to the next lower version that is integrated into the text. Version 1 (Edition 1): (May 30, 2003) First approved version of H.264/AVC containing Baseline, Main, and Extended profiles. Version 2 (Edition 1.1): (May 7, 2004) Corrigendum containing various minor corrections. Version 3 (Edition 2): (March 1, 2005) Major addition containing the first amendment, establishing the Fidelity Range Extensions (FRExt). This version added the High, High 10, High 4:2:2, and High 4:4:4 profiles. After a few years, the High profile became the most commonly used profile of the standard. Version 4 (Edition 2.1): (September 13, 2005) Corrigendum containing various minor corrections and adding three aspect ratio indicators. Version 5 (Edition 2.2): (June 13, 2006) Amendment consisting of removal of prior High 4:4:4 profile (processed as a corrigendum in ISO/IEC). Version 6 (Edition 2.2): (June 13, 2006) Amendment consisting of minor extensions like extended gamut color space support (bundled with above mentioned aspect ratio indicators in ISO/IEC). Version 8 (Edition 3): (November 22, 2007) Major addition to H.264/AVC containing the amendment for Scalable Video Coding (SVC) containing Scalable Baseline, Scalable High, and Scalable High Intra profiles. Version 9 (Edition 3.1): (January 13, 2009) Corrigendum containing minor corrections. Version 10 (Edition 4): (March 16, 2009) Amendment containing definition of a new profile (the Constrained Baseline profile) with only the common subset of capabilities supported in various previously specified profiles. Version 11 (Edition 4): (March 16, 2009) Major addition to H.264/AVC containing the amendment for Multiview Video Coding (MVC) extension, including the Multiview High profile. Version 13 (Edition 5): (March 9, 2010) Corrigendum containing minor corrections. Version 15 (Edition 6): (June 29, 2011) Corrigendum containing minor corrections. Version 17 (Edition 8): (April 13, 2013) Amendment with additional SEI message indicators. Version 18 (Edition 8): (April 13, 2013) Amendment to specify the coding of depth map data for 3D stereoscopic video, including a Multiview Depth High profile. Version 22 (Edition 9): (February 13, 2014) Amendment to specify the multi resolution frame compatible (MFC) enhancement for 3D stereoscopic video, the MFC High profile, and minor corrections. Version 24 (Edition 11): (October 14, 2016) Amendment to specify additional levels of decoder capability supporting larger picture sizes (Levels 6, 6.1, and 6.2), the green metadata SEI message, the alternative depth information SEI message, and additional color related VUI codepoint identifiers. Version 25 (Edition 12): (April 13, 2017) Amendment to specify the Progressive High 10 profile, hybrid log–gamma (HLG), and additional color related VUI code points and SEI messages. Version 26 (Edition 13): (June 13, 2019) Amendment to specify additional SEI messages for ambient viewing environment, content light level information, content color volume, equirectangular projection, cubemap projection, sphere rotation, region wise packing, omnidirectional viewport, SEI manifest, and SEI prefix. Version 27 (Edition 14): (August 22, 2021) Amendment to specify additional SEI messages for annotated regions and shutter interval information, and miscellaneous minor corrections and clarifications.

The H.264 video format has a very broad application range that covers all forms of digital compressed video from low bit rate Internet streaming applications to HDTV broadcast and Digital Cinema applications with nearly lossless coding. With the use of H.264, bit rate savings of 50% or more compared to MPEG 2 Part 2 are reported. For example, H.264 has been reported to give the same Digital Satellite TV quality as current MPEG 2 implementations with less than half the bitrate, with current MPEG 2 implementations working at around 3.5 Mbit/s and H.264 at only 1.5 Mbit/s. Sony claims that 9 Mbit/s AVC recording mode is equivalent to the image quality of the HDV format, which uses approximately 18–25 Mbit/s. To ensure compatibility and problem free adoption of H.264/AVC, many standards bodies have amended or added to their video related standards so that users of these standards can employ H.264/AVC. Both the Blu ray Disc format and the now discontinued HD DVD format include the H.264/AVC High Profile as one of three mandatory video compression formats. The Digital Video Broadcast project (DVB) approved the use of H.264/AVC for broadcast television in late 2004. The Advanced Television Systems Committee (ATSC) standards body in the United States approved the use of H.264/AVC for broadcast television in July 2008, although the standard is not yet used for fixed ATSC broadcasts within the United States. It has also been approved for use with the more recent ATSC M/H (Mobile/Handheld) standard, using the AVC and SVC portions of H.264. The CCTV (Closed Circuit TV) and Video Surveillance markets have included the technology in many products. Many common DSLRs use H.264 video wrapped in QuickTime MOV containers as the native recording format.

AVCHD is a high definition recording format designed by Sony and Panasonic that uses H.264 (conforming to H.264 while adding additional application specific features and constraints). AVC Intra is an intraframe only compression format, developed by Panasonic. XAVC is a recording format designed by Sony that uses level 5.2 of H.264/MPEG 4 AVC, which is the highest level supported by that video standard. XAVC can support 4K resolution (4096 × 2160 and 3840 × 2160) at up to 60 frames per second (fps). The Sony PMW F55 can record XAVC with 4K resolution at 30 fps at 300 Mbit/s and 2K resolution at 30 fps at 100 Mbit/s. XAVC can record 4K resolution at 60 fps with 4:2:2 chroma sampling at 600 Mbit/s.

Feature CBP BP XP MP ProHiP HiP Hi10P Hi422P Hi444PP I and P slices Bit depth (per sample) Chroma formats Flexible macroblock ordering (FMO) Arbitrary slice ordering (ASO) Redundant slices (RS) Data Partitioning SI and SP slices Interlaced coding (PicAFF, MBAFF) B slices Multiple reference frames In loop deblocking filter CAVLC entropy coding CABAC entropy coding 4:0:0 (Monochrome) 8×8 vs. 4×4 transform adaptivity Quantization scaling matrices Separate CB and CR QP control Separate color plane coding Predictive lossless coding

As the term is used in the standard, a "level" is a specified set of constraints that indicate a degree of required decoder performance for a profile. For example, a level of support within a profile specifies the maximum picture resolution, frame rate, and bit rate that a decoder may use. A decoder that conforms to a given level must be able to decode all bitstreams encoded for that level and all lower levels. + Levels with maximum property values Microsoft, with the release of Internet Explorer 9, has added support for HTML 5 video encoded using H.264. At the Gartner Symposium/ITXpo in November 2010, Microsoft CEO Steve Ballmer answered the question "HTML 5 or Silverlight?" by saying "If you want to do something that is universal, there is no question the world is going HTML5." In January 2011, Google announced that they were pulling support for H.264 from their Chrome browser and supporting both Theora and WebM/VP8 to use only open formats. On March 18, 2012, Mozilla announced support for H.264 in Firefox on mobile devices, due to prevalence of H.264 encoded video and the increased power efficiency of using dedicated H.264 decoder hardware common on such devices. On February 20, 2013, Mozilla implemented support in Firefox for decoding H.264 on Windows 7 and above. This feature relies on Windows' built in decoding libraries. Firefox 35.0, released on January 13, 2015, supports H.264 on OS X 10.6 and higher. On October 30, 2013, Rowan Trollope from Cisco Systems announced that Cisco would release both binaries and source code of an H.264 video codec called OpenH264 under the Simplified BSD license, and pay all royalties for its use to MPEG LA for any software projects that use Cisco's precompiled binaries, thus making Cisco's OpenH264 binaries free to use. However, any software projects that use Cisco's source code instead of its binaries would be legally responsible for paying all royalties to MPEG LA. Target CPU architectures include x86 and ARM, and target operating systems include Linux, Windows XP and later, Mac OS X, and Android; iOS was notably absent from this list, because it doesn't allow applications to fetch and install binary modules from the Internet. Also on October 30, 2013, Brendan Eich from Mozilla wrote that it would use Cisco's binaries in future versions of Firefox to add support for H.264 to Firefox where platform codecs are not available. Cisco published the source code to OpenH264 on December 9, 2013. Although iOS was not supported by the 2013 Cisco software release, Apple updated its Video Toolbox Framework with iOS 8 (released in September 2014) to provide direct access to hardware based H.264/AVC video encoding and decoding.4:2:04:2:24:4:4   || || || || || || || Largest sample depth (bit) Predictive lossless coding

Because H.264 encoding and decoding requires significant computing power in specific types of arithmetic operations, software implementations that run on general purpose CPUs are typically less power efficient. However, the latest quad core general purpose x86 CPUs have sufficient computation power to perform real time SD and HD encoding. Compression efficiency depends on video algorithmic implementations, not on whether hardware or software implementation is used. Therefore, the difference between hardware and software based implementation is more on power efficiency, flexibility and cost. To improve the power efficiency and reduce hardware form factor, special purpose hardware may be employed, either for the complete encoding or decoding process, or for acceleration assistance within a CPU controlled environment. CPU based solutions are known to be much more flexible, particularly when encoding must be done concurrently in multiple formats, multiple bit rates and resolutions (multi screen video), and possibly with additional features on container format support, advanced integrated advertising features, etc. CPU based software solution generally makes it much easier to load balance multiple concurrent encoding sessions within the same CPU. The 2nd generation Intel "Sandy Bridge" Core i3/i5/i7 processors introduced at the January 2011 CES (Consumer Electronics Show) offer an on chip hardware full HD H.264 encoder, known as Intel Quick Sync Video. A hardware H.264 encoder can be an ASIC or an FPGA. ASIC encoders with H.264 encoder functionality are available from many different semiconductor companies, but the core design used in the ASIC is typically licensed from one of a few companies such as Chips&Media, Allegro DVT, On2 (formerly Hantro, acquired by Google), Imagination Technologies, NGCodec. Some companies have both FPGA and ASIC product offerings. Texas Instruments manufactures a line of ARM + DSP cores that perform DSP H.264 BP encoding 1080p at 30fps. This permits flexibility with respect to codecs (which are implemented as highly optimized DSP code) while being more efficient than software on a generic CPU.