Meridian, a 2016 Netflix release, is available for uncompressed download for engineers, production artists, or anyone else who wants to practice their media processing skills on studio production quality material.  Many thanks are due to Netflix for releasing this title under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License

The film is described as “cerebral, full of suspense, chilling, ominous.” I don’t know about the story, but those words certainly describe the process of obtaining and using this film’s downloadable media assets.  It is released in multiple color renderings on two different sites, some versions in Interoperable Media Format (IMF) and other encodings wrapped in mp4 packages.

You know with a name like “Interoperable Media Format” that the IMF package is likely to present some barriers to actually access the elementary audio and video media.  IMF adds a layer of packaging on top Material eXchange Format (MXF), which by itself is always a challenge to wrap and unwrap.

In this post we’ll enter the Meridian cave armed with only a flashlight and ffmpeg.  When we emerge, we’ll have elementary stream files in hand and know what color rendering they belong to.

IMF tools and converters

Chris Fetner’s LinkedIn post does a great job explaining why IMF is important for storing the masters from which distribution media files are encoded.  He also talks about a number of tools, all open source with links (great!), for accessing the IMF package contents:

• imf-conversion-utility, to convert to Digital Production Partnership (DPP) format
• imf-conversion-utility, to convert to iTunes Package format
• IMFTool, to edit the composition playlist (CPL) and make other tweaks to the IMF package.

However none of these tools will perform a simple demultiplex of elementary stream files.  Don’t worry, ffmpeg can do it.  Command line details below.

Multiple color renderings

Meridian is released in standard dynamic range (SDR) and two different high dynamic range (HDR) formats, on two different sites.

The files on https://media.xiph.org/video/derf/meridian/MERIDIAN_SHR_C_EN-XX_US-NR_51_LTRT_UHD_20160909_OV/ are the SDR rendering in an IMF package.  They were released in September, 2016.

The files on http://download.opencontent.netflix.com/?prefix=TechblogAssets/ are explained in the readme.txt file and include SDR, HDR10, and HDR Dolby Vision (HEVC).  There is an IMF package for Dolby Vision HDR rendering, and encodings in three color renderings in mp4 packages.  They were released in May, 2018.

These links were provided in a Netflix Tech Blog post on May 11, 2018.

Demultiplexing

ffmpeg is our tool of choice for  media file operations.  This is the command line utility that wraps the avcodec, avformat, avutil and other libraries to perform many kinds of media processing functions.

In general you can know what is inside a media file package just by providing the input file with no other options.  For example,

ffmpeg -i Meridian_3840x2160_5994fps_SDR.mp4

will tell you that this package contains AVC/H.264 video elementary stream with resolution 3840×2160 at 59.94 fps, chroma format Y’CbCr 4:2:0 (yuv_420p), and AAC audio with 48 kHz sample rate.

The version of ffmpeg used in the following examples is 2.8.14 as packaged for Ubuntu 16.04 LTS, except as noted for audio MXF demultiplexing below.

Video elementary streams

The video elementary stream in the IMF package is wrapped in an MXF file which ffmpeg is able to demultiplex.  The video elementary stream is JPEG2000, 4:2:2 chroma format, with 10 bit samples at 3840x2160p59.94 resolution.  You will likely transcode this file as in this example:

ffmpeg -i MERIDIAN_SHR_C_EN-XX_US-NR_51_LTRT_UHD_20160909_OV_01.mxf -c:v libx265 -pix_fmt yuv420p10 -crf 25 meridian_hevc10.mp4

• -c:v libx265 means to transcode video frames using the x265 encoder
• -pix_fmt yuv420p10 means convert chroma format to Y’CbCr 4:2:0 chroma format with 10 bit samples
• -crf 25 means to use “constant rate factor” rate control which is variable bit rate with generally constant quantizer adjusted for picture and block coding type

Netflix also provides some encodes, some in AVC/H.264 and others in HEVC/H.265, wrapped in MP4 packages.  If you want to extract the raw elementary video stream without transcoding:

ffmpeg -i Meridian_3840x2160_5994fps_HDR10.mp4 -format hevc -c:v copy -bsf hevc_mp4toannexb Meridian_3840x2160_5994fps_HDR10.hevc

• -format hevc specifies the output format
• -c:v copy means copy frames from input package to output without transcoding, which only works if the input video is already in the format specified for output
• -bsf hevc_mp4toannexb adds a bit stream filter to restore Annex B NAL unit start code prefixes, 0x000001 or 0x00000001, which are not carried in the MP4 packaged video elementary stream data.

Note that without the -bsf hevc_mp4toannexb bit stream filter, the resulting output file would not be usable because every decoder or utility that can take a raw HEVC elementary video stream as input expects to see the byte stream format specified in Annex B of the HEVC standard.  The corresponding bit stream filter for AVC/H.264 video in an MP4 package is -bsf h264_mp4toannexb.

Audio elementary streams

The audio elementary streams in the IMF package are also wrapped in MXF files.  However, ffmpeg is not able to demultiplex them due to a bug.   I had already isolated the problem and written a patch to bypass the malfunction, but then noticed that a real fix was checked in last week.  If you want to have the original, uncompressed PCM audio data extracted from these MXF files, your best option is to clone the ffmpeg git repository and build ffmpeg yourself to get a version that includes the fix to this bug.  Then do

ffmpeg -i MERIDIAN_SHR_C_EN-XX_US-NR_51_LTRT_UHD_20160909_OV_01_EN_20_B.mxf -c:a copy meridian_pcm_20_s24le.wav

or

ffmpeg  -i MERIDIAN_SHR_C_EN-XX_US-NR_51_LTRT_UHD_20160909_OV_01_EN_20_B.mxf -c:a pcm_s16le meridian_pcm_20_s16le.wav

• -c:a copy means copy audio data without transcoding from the MXF package, which contains PCM 24 bit (little endian) samples.
• -c:a pcm_s16le means convert the audio samples to PCM 16 bit (little endian)

Note: if you see the error message

 mxf @ 0x1069400] OPAtom misinterpreted as OP1a?KLV for edit unit 0 extending into next edit unit is not implemented. Update your FFmpeg version to the newest one from Git.

it means your version of ffmpeg does not include the fix to the bug linked above.

Alternatively, you can extract the audio from any of the encoded MP4 packages.  However it will be subject to some fidelity loss due to being compressed in the MP4 package.

ffmpeg -i Meridian_3840x2160_5994fps_HDR10.mp4 -vn -c:a pcm_s16le out.wav

• -vn means don’t include video in the output file
• -c:a pcm_s16le means convert audio to PCM 16 bit (little endian)

Putting audio and video together

The following command line is an example of assembling separate audio and video files into a new media package, ready for playback on your device:

ffmpeg -i meridian_hevc10.mp4 -i meridian_pcm_20_s24le.wav -c:v copy -c:a ac3 meridian.mp4

• -c:v copy means copy the video stream without transcoding
• -c:a ac3 means encode audio using AC3

Playing Meridian

Now that you have the Meridian movie in exactly the format you want, where can you play it?  My dual quad-core Xeon compute server can only play this movie at 25% real time with VLC player due to the HEVC encoding and UHD resolution.  However a LePotato single board computer running CoreELEC software handles it just fine, because it has 4kp60 HEVC hardware decoding support in its video processing unit.

Much of the benefit of the Meridian open source movie is that the release includes both Standard Dynamic Range (SDR) and High Dynamic Range (HDR) renderings.  Playing an HDR coded media file and seeing the correct colors on a 4K UHD HDR television is a great goal and hopefully will be the subject of a future post.