FFmpeg encoder

The %ffmpeg encoder should support all the options for ffmpeg’s muxers and encoders, including private configuration options. Configuration value are passed as key/values, with values being of types: string, int, or float. If an option is not recognized (or: unused), it will raise an error during the instantiation of the encoder. Here are some configuration examples:

Interleaved muxing

FFmpeg provides two different APIs for muxing data, interleaved or not. The interleaved API buffers packets waiting to be outputted to make sure that all streams, e.g. audio and video, have their packets as close to each other as possible. This ensures that for instance, the stream does not start with a long chunk of audio data without any video content. However, this can come with some increased memory usage due to buffering.

On the other hand, the non-interleaved API allows to send encoded packets directly to the output without intermediate buffering. This can sometimes result in better latency and lower memory usage.

The %ffmpeg encoder can use either API. By default, it uses the interleaved API when encoding more than one stream. You can also specify the interleaving mode by passing the interleaved parameter: %ffmpeg(interleaved=<true|false|"default">, ...).

You might also want to take this into consideration when setting your encoder’s parameters. Some video encoders can buffer frames for a while before outputting the first encoded frame, which can also create issues even with the interleaved API enabled (the interleaving buffer has a max size too!). Typically, with libx264, you can set tune = "zerolatency" to make sure that the encoder starts outputting data right away.

Encoding examples

  • AAC encoding at 22050kHz using fdk-aac encoder and mpegts muxer
%ffmpeg(format="mpegts",
  %audio(
    codec="libfdk_aac",
    samplerate=22050,
    b="32k",
    afterburner=1,
    profile="aac_he_v2"
  )
)
  • Mp3 encoding using libshine at 48000kHz
%ffmpeg(format="mp3", %audio(codec="libshine", samplerate=48000))
  • AC3 audio and H264 video encapsulated in an MPEG-TS stream
%ffmpeg(
  format="mpegts",
  %audio(codec="ac3", channel_coupling=0),
  %video(
    codec="libx264",
    b="2600k",
    "x264-params"="scenecut=0:open_gop=0:min-keyint=150:keyint=150",
    preset="ultrafast"
  )
)
  • AC3 audio and H264 video encapsulated in an MPEG-TS stream using ffmpeg raw frames
%ffmpeg(
  format="mpegts",
  %audio.raw(codec="ac3", channel_coupling=0),
  %video.raw(
    codec="libx264",
    b="2600k",
    "x264-params"="scenecut=0:open_gop=0:min-keyint=150:keyint=150",
    preset="ultrafast"
  )
)
  • Mp3 encoding using libmp3lame and video copy
%ffmpeg(
  format="mp3",
  %audio(codec="libmp3lame"),
  %video.copy
)

The full syntax is as follows:

%ffmpeg(
  format=<format>,
  # Audio section
  %audio(codec=<codec>, <option_name>=<option_value>, ...),
  # Or:
  %audio.raw(codec=<codec>, <option_name>=<option_value>, ...),
  # Or:
  %audio.copy(<option>),
  # Video section
  %video(codec=<codec>, <option_name>=<option_value>, ...),
  # Or:
  %video.raw(codec=<codec>, <option_name>=<option_value>, ...),
  # Or:
  %video.copy(<option>),
  # Generic options
  <option_name>=<option_value>, ...
)

Where:

  • <format> is either a string value (e.g. "mpegts"), as returned by the ffmpeg -formats command or none. When set to none or simply no specified, the encoder will try to auto-detect it.
  • <codec> is a string value (e.g. "libmp3lame"), as returned by the ffmpeg -codecs command.
  • <option_name> can be any syntactically valid variable name or string. Strings are typically used when the option name is of the form: foo-bar.
  • %audio(...) is for options specific to the audio codec. Unused options will raise an exception. Any option supported by ffmpeg can be passed here. Streams encoded using %audio are using liquidsoap internal frame format and are fully handled on the liquidsoap side.
  • %audio.raw(...) behaves like %audio except that the audio data is kept as ffmpeg’s internal format. This can avoid data copy and is also the format required to use ffmpeg filters.
  • %audio.copy copies data without decoding or encoding it. This is great to avoid using the CPU, but in this case, the data cannot be processed with operators that modify it, such as fade.{in,out} or smart_cross. Also, all streams must agree on the same data format.
  • %video(...) is for options specific to the video codec. Unused options will raise an exception. Any option supported by ffmpeg can be passed here.
  • %video.raw and %video.copy have the same meaning as their %audio counterpart.
  • Generic options are passed to audio, video and format (container) setup. Unused options will raise an exception. Any option supported by ffmpeg can be passed here.

HLS output

The %ffmpeg encoder is the prime encoder for HLS output as it is the only one of our collection of encoder which can produce Mpeg-ts muxed data, which is required by most HLS clients.

File output

Some encoding formats, for instance mp4, require to rewind their stream and write a header after the fact, when encoding of the current track has finished. For historical reasons, such formats cannot be used with output.file. To remedy that, we have introduced the output.url operator. When using this operator, the encoder is fully in charge of the output file and can thus write headers after the fact. The %ffmpeg encoder is one such encoder that can be used with this operator.

Copy options

The %audio.copy and %video.copy encoders have two mutually exclusive options to handle keyframes:

  • %audio.copy(wait_for_keyframe) and %video.copy(wait_for_keyframe): Wait until at least one keyframe has been passed to start passing encoded packets from a new stream.
  • %audio.copy(ignore_keyframe) and %video.copy(ignore_keyframe): Ignore all keyframes.

These options are useful when switching from one encoded stream to the next.

With option wait_for_keyframe, the encoder discards any new packet at the beginning of a stream until a keyframe is passed. This means that playback will be paused until it can be resumed properly with no decoding glitches. This option is implemented globally when possible, i.e. in case of a video track with keyframes and an audio track with no keyframes, the audio track will discard packets until a video keyframe has been passed. This is the default option.

With option ignore_keyframe, the encoder starts passing encoded data right away. Content is immediately added but playback might get stuck until a new keyframe is passed.

It is worth noting that some audio encoders may also have keyframes.

Hardware acceleration

The %ffmpeg encoder supports multiple hardware acceleration provided by ffmpeg.

If you are lucky and the encoder you are using provides support for hardware acceleration without any specific configuration, all you might have to do is select codec="..." (for instance on macOS, codec="h264_videotoolbox") and it should work immediately.

The type of hardware acceleration provided by ffmpeg are:

  1. Internal hardware acceleration that works without any specific configuration. This is the happy path described above!
  2. Device-based hardware acceleration that works with a specific device.
  3. Frame-based hardware acceleration that work with a specific pixel format.

The type of hardware acceleration to use for a given stream can be specified using the hwaccel option. Its value is one of: "auto", "none", "internal", "device" or "frame".

For device-based hardware acceleration, the device to use can be specified using hwaccel_device. For frame-based hardware acceleration, the pixel format can be specified using hwaccel_pixel_format. In most cases, liquidsoap should be able to guess these values from the codec.

Here’s an example:

enc = %ffmpeg(
  format="mpegts",
  %video(
    hwaccel="device",
    hwaccel_devic="/dev/...",
    ...
  )
)

Hardware acceleration support is, of course, very hardware dependent, so we might not have tested all possible combinations. If you are having issues setting it up, do not hesitate to get in touch with us to see if your use-case is properly covered.