Liquidsoap’s scripting language

The following is adapted from the Liquidsoap book. The reader is avised to check out the whole chapter in the book for more details about the liquidsoap language

General features

Liquidsoap is a novel language which was designed from scratch to handle media stream. It takes some inspiration from functional languages such as OCaml but features a syntax that is more intuitive to the general purpose programmer, similar to Ruby or Javascript.

Typing

One of the main features of the language is that it is typed. This means that every expression belongs to some type which indicates what it is. For instance, "hello" is a string whereas 23 is an integer, and, when presenting a construction of the language, we will always indicate the associated type. Liquidsoap implements a typechecking algorithm which ensures that whenever a string is expected a string will actually be given, and similarly for other types. This is done without running the program, so that it does not depend on some dynamic tests, but is rather enforced by theoretical considerations. Another distinguishing feature of this algorithm is that it also performs type inference: you never actually have to write a type, those are guessed automatically by Liquidsoap. This makes the language very safe, while remaining very easy to use.

Functional programming

The language is functional, which means that you can very easily define functions, and that functions can be passed as arguments of other functions. This might look like a crazy thing at first, but it is actually quite common in some language communities (such as OCaml). It also might look quite useless: why should we need such functions when describing webradios? You will soon discover that it happens to be quite convenient in many places: for handlers (we can specify the function which describes what to do when some event occurs such as when a DJ connects to the radio), for transitions (we pass a function which describes the shape we want for the transition) and so on.

Streams

The unique feature of Liquidsoap is that it allows the manipulation of sources which are functions which will generate streams. These streams typically consist of stereo audio data, but we do restrict to this: they can contain audio with arbitrary number of channels, they can also contain an arbitrary number of video channels, and also MIDI channels (there is limited support for sound synthesis).

Standard library

Although the core of Liquidsoap is written in OCaml, many of the functions of Liquidsoap are written in the Liquidsoap language itself. Those are defined in the stdlib.liq script, which is loaded by default and includes all the libraries. You should not be frightened to have a look at the standard library, it is often useful to better grasp the language, learn design patterns and tricks, and add functionalities. Its location on your system is indicated in the variable configure.libdir and can be obtained by typing

Basic values

Integers and floats

The integers, such as 3 or 42, are of type int. Depending on the current architecture of the computer on which we are executing the script (32 or 64 bits, the latter being the most common nowadays) they are stored on 31 or 63 bits. The minimal (resp. maximal) representable integer can be obtained as the constant min_int (resp. max_int); typically, on a 64 bits architecture, they range from -4611686018427387904 to 4611686018427387903.

The floating point numbers, such as 2.45, are of type float, and are in double precision, meaning that they are always stored on 64 bits. We always write a decimal point in them, so that 3 and 3. are not the same thing: the former is an integer and the latter is a float. This is a source of errors for beginners, but is necessary for typing to work well.

Strings

Strings are written between double or single quotes, e.g. "hello!" or 'hello!', and are of type string.

In order to write the character “"” in a string, one cannot simply type “"” since this is already used to indicate the boundaries of a string: this character should be escaped, which means that the character “\” should be typed first so that

print("My name is \"Sam\"!")

will actually display “My name is "Sam"!”. Other commonly used escaped characters are “\\” for backslash and “\n” for new line. Alternatively, one can use the single quote notation, so that previous example can also be written as

print('My name is "Sam"!')

This is most often used when testing JSON data which can contain many quotes or for command line arguments when calling external scripts. The character “\” can also be used at the end of the string to break long strings in scripts without actually inserting newlines in the strings. For instance, the script

print("His name is \
       Romain.")

will actually print

His name is Romain.

Note that there is no line change between “is” and “Romain”, and the indentation before “Romain” is not shown either.

The concatenation of two strings is achieved by the infix operator “^”, as in

user = "dj"
print("Current user is " ^ user)

Instead of using concatenation, it is often rather convenient to use string interpolation: in a string, #{e} is replaced by the string representation of the result of the evaluation of the expression e:

user = "admin"
print("The user #{user} has just logged.")

will print The user admin has just logged. or

print("The number #{random.float()} is random.")

will print The number 0.663455738438 is random. (at least it did last time I tried).

Escaping strings

Liquidsoap strings follow the most common lexical conventions from C and javascript and JSON, in particular, string.unescape recognizes the same escape sequences as C (except for UTF-16 characters) and javascript.

The following sequences are recognized:

Escape sequence Hex value in ASCII Character represented
\a \x07 Alert (Beep, Bell)
\b \x08 Backspace
\e \x1B Escape character
\f \x0C Formfeed, Page Break
\n \x0A Newline (Line Feed)
\r \x0D Carriage Return
\t \x09 Horizontal Tab
\v \x0B Vertical Tab
\\ \x5C Backslash
\/ \x2f Forward slash
\' \x27 Apostrophe or single quotation mark
\" \x22 Double quotation mark
\? \x3F Question mark (used to avoid Digraphs and trigraphs)
\nnn any The byte whose numerical value is given by nnn interpreted as an octal number
\xhh any The byte whose numerical value is given by hh interpreted as a hexadecimal number
\uhhhh none UTF8-8 code point given by hhhh interpreted as an hexadecimal number

This convention has been decided to follow the most common practices. In particular, \nnn is an octal escape sequence in most languages including C, Ruby, Javascript, Python and more. This differs from OCaml where \nnn is considered a digital escape sequence.

These lexical conventions are used in the default string.escape and string.unescape.

Here’s an example of an escaped string:

# "\" \t \045 \x2f \u4f32";;
- : string = "\" \t % / 2"

The function string.quote returns JSON-compatible strings.

Regular expressions

This feature was introduced in liquidsoap version 2.1.0

Regular expressions can be created using the regexp operator or the syntactic sugar: r/.../<flags>. For instance:

# Using the regexp operator:
r = regexp(flags=["g","i"], "foo([\\w])+bar")

# Using the r/../ syntactic sugar:
r = r/foo([\w])bar/gi

Using the r/../ syntactic sugar makes it possible to write regular expressions without having to escape \ characters, which makes them more easily readable.

Regular expression flags are:

  • i: perform case-insensitive match
  • g: substitute all matched sub-strings, not just the first one
  • s: match all characters, including \n when using the . pattern
  • m: ^ and $ match before/after newlines, not just at the beginning/end of a string

Regular expressions have the following methods:

  • replace(fn, s): replace matched substrings of s using function fn. If the g flag is not passed, only the first match is replaced otherwise, all matches are replaced
  • split(s): split the given string on all substrings matching the regular expression.
  • test(s): returns true if the given string matches the regular expression.
  • exec(s): execute the regular expression and return a of list matches of the form: [(<match index>, <match>), ..]. Named matches are also supported and returned as property groups of type [string * string]:
r/(foo)(?<gno>gni)?/g.exec("foogni")
- : [int * string].{groups : [string * string]} =
[
    (2, "gni"),
    (1, "foo"),
    (0, "foogni")
].{
    groups = [
      ("gno", "gni")
    ]
}

Booleans

The booleans are either true or false and are of type bool. They can be combined using the usual boolean operations

  • and: conjunction,
  • or: disjunction,
  • not: negation.

Booleans typically originate from comparison operators, which take two values and return booleans:

  • ==: compares for equality,
  • !=: compares for inequality,
  • <=: compares for inequality,

and so on (<, >=, >). For instance, the following is a boolean expression:

(n < 3) and not (s == "hello")

Conditional branchings execute code depending on whether a condition is true or not. For instance, the code

if (1 <= x and x <= 12) or (not 10h-15h) then
  print("The condition is satisfied")
else
  print("The condition ain't satisfied")
end

will print that the condition is satisfied when either x is between 1 and 12 or the current time is not between 10h and 15h. A conditional branching might return a value, which is the last computed value in the chosen branch. For instance,

y = if x < 3 then "A" else "B" end

will assign "A" or "B" to y depending on whether x is below 3 or not. The two branches of a conditional should always have the same return type:

x = if 1 == 2 then "A" else 5 end

will result in

At line 1, char 19-21:
Error 5: this value has type string
but it should be a subtype of int

meaning that "A" is a string but is expected to be an integer because the second branch returns an integer, and the two should be of same nature. The else branch is optional, in which case the then branch should be of type unit:

if x == "admin" then print("Welcome admin") end

In the case where you want to perform a conditional branching in the else branch, the elsif keyword should be used, as in the following example, which assigns 0, 1, 2 or 3 to s depending on whether x is "a", "b", "c" or something else:

s = if x == "a" then 0
    elsif x == "b" then 1
    elsif x == "c" then 2
    else 3 end

This is equivalent (but shorter to write) to the following sequence of imbricated conditional branchings:

s = if x == "a" then 0
    else
      if x == "b" then 1
      else
        if x == "c" then 2
        else 3 end
      end
    end

Finally, we should mention that the notation c?a:b is also available as a shorthand for if c then a else b end, so that the expression

y = if x < 3 then "A" else "B" end

can be shortened to

y = (x<3)?"A":"B"

(and people will think that you are a cool guy).

Time predicates

Time predicates are special boolean values such as {0h-7h}. These values are true or false depending on the current time. Some examples of time predicates are

{11h15-13h} between 11h15 and 13h {12h} between 12h00 and 12h59 {12h00} at 12h00 {00m} on the first minute of every hour {00m-09m} on the first 10 minutes of every hour {2w} on Tuesday {6w-7w} on weekends

Above, w stands for weekday: 1 is Monday, 2 is Tuesday, and so on. Sunday is both 0 and 7.

Time predicate can also be parsed at runtime, for instance if you want to create them dynamically. The syntax is:

# f = time.predicate("00m-30m");;
f : () -> bool = <fun>

Be aware that, if parsing fails, it will raise error.string:

# f = time.predicate("foo")
Error 14: Uncaught runtime error:
type: string, message: "Failed to parse foo as time predicate"

Unit

Some functions, such as print, do not return a meaningful value: we are interested in what they are doing (e.g. printing on the standard output) and not in their result. However, since typing requires that everything returns something of some type, there is a particular type for the return of such functions: unit. Just as there are only two values in the booleans (true and false), there is only one value in the unit type, which is written (). This value can be thought of as the result of the expression saying “I’m done”.

Lists

Some more elaborate values can be constructed by combining the previous ones. A first kind is lists which are finite sequences of values, being all of the same type. They are constructed by square bracketing the sequence whose elements are separated by commas. For instance, the list

[1, 4, 5]

is a list of three integers (1, 4 and 5), and its type is [int], and the type of ["A", "B"] would obviously be [string]. Note that a list can be empty: [].

You can extract list elements through splats such as

l = [1, 5, 7, 8, 9]
let [x, _, z, ...t] = l

In this example, the value of x is 1, the value of z is 7 and the value of t is [8, 9].

You can also combine lists in a similar way

x = [1, ...[2, 3, 4], 5, ...[6, 7]]

In this example, the value of x is [1, 2, 3, 4, 5, 6 ,7]

Tuples

Another construction present in Liquidsoap is tuples of values, which are finite sequences of values which, contrarily to lists, might have different types. For instance,

(3, 4.2, "hello")

is a triple (a tuple with three elements) of type

int * float * string

which indicate that the first element is an integer, the second a float and the third a string.

Similarly to lists, there is a special syntax in order to access tuple elements. For instance, if t is the above tuple (3, 4.2, "hello"), we can write

let (n, x, s) = t

which will assign the first element to the variable n, the second element to the variable x and the third element to the variable s.

Programming primitives

Variables

We have already seen many examples of uses of variables: we use

x = e

in order to assign the result of evaluating an expression e to a variable x, which can later on be referred to as x. Variables can be masked: we can define two variables with the same name, and at any point in the program the last defined value for the variable is used:

n = 3
print(n)
n = n + 2
print(n)

will print 3 and 5. Contrarily to most languages, the value for a variable cannot be changed (unless we explicitly require this by using references, see below), so the above program does not modify the value of n, it is simply that a new n is defined.

There is an alternative syntax for declaring variables which is

def x =
  e
end

It has the advantage that the expression e can spread over multiple lines and thus consist of multiple expressions, in which case the value of the last one will be assigned to x. This is particularly useful to use local variables when defining a value.

References

As indicated above, by default, the value of a variable cannot be changed. However, one can use a reference in order to be able to do this. Those can be seen as memory cells, containing values of a given fixed type, which can be modified during the execution of the program. They are created with the ref keyword, with the initial value of the cell as argument. For instance,

r = ref(5)

declares that r is a reference which contains 5 as initial value. Since 5 is an integer (of type int), the type of the reference r will be

ref(int)

meaning that its a memory cell containing integers. On such a reference, two operations are available.

  • One can obtain the value of the reference by applying the reference to (), so that r() denotes the value contained in the reference r, for instance

    x = r() + 4

    declares the variable x as being 9 (which is 5+4).

  • One can change the value of the reference by using the := keyword, e.g.

    r := 2

    will assign the value 2 to r. Internally, this is done by calling the set method of the reference, so that the above is equivalent to writing

    r.set(2)

    which used to be the syntax for some reference manipulations.

Loops

The usual looping constructions are available in Liquidsoap. The for loop repeatedly executes a portion of code with an integer variable varying between two bounds, being increased by one each time. For instance, the following code will print the integers 1, 2, 3, 4 and 5, which are the values successively taken by the variable i:

for i = 1 to 5 do
  print(i)
end

In practice, such loops could be used to add a bunch of numbered files (e.g. music1.mp3, music2.mp3, music3.mp3, etc.) in a request queue for instance.

The while loop repeatedly executes a portion of code, as long a condition is satisfied. For instance, the following code doubles the contents of the reference n as long as its value is below 10:

n = ref(1)
while n() < 10 do
  n := n() * 2
end
print(n())

The variable n will thus successively take the values 1, 2, 4, 8 and 16, at which point the looping condition n() < 10 is not satisfied anymore and the loop is exited. The printed value is thus 16.

Functions

Liquidsoap is built around the notion of function: most operations are performed by those. For some reason, we sometimes call operators the functions acting on sources. Liquidsoap includes a standard library which consists of functions defined in the Liquidsoap language, including fairly complex operators such as playlist which plays a playlist or crossfade which takes care of fading between songs.

Basics

A function is a construction which takes a bunch of arguments and produces a result. For instance, we can define a function f taking two float arguments, prints the first and returns the result of adding twice the first to the second:

def f(x, y)
  print(x)
  2*x+y
end

This function can also be written on one line if we use semicolons (;) to separate the instructions instead of changing line:

def f(x, y) = print(x); 2*x+y end

The type of this function is

(int, int) -> int

The arrow -> means that it is a function, on the left are the types of the arguments (here, two arguments of type int) and on the right is the type of the returned value of the function (here, int). In order to use this function, we have to apply it to arguments, as in

f(3, 4)

This will trigger the evaluation of the function, where the argument x (resp. y) is replaced by 3 (resp. 4), i.e., it will print 3 and return the evaluation of 2*3+4, which is 10.

Anonymous functions

For concision in scripts, it is possible define a function without giving it a name, using the syntax

fun (x) -> ...

This is called an anonymous function, and it is typically used in order to specify short handlers in arguments.

Anonymous function with no arguments

You will see that it is quite common to use anonymous functions with no arguments. For this reason, we have introduced a special convenient syntax for those and allow writing

{...}

instead of

fun () -> ...

Labeled arguments

A function can have an arbitrary number of arguments, and when there are many of them it becomes difficult to keep track of their order and their order matter! For instance, the following function computes the sample rate given a number of samples in a given period of time:

def samplerate(samples, duration) = samples / duration end

which is of type

(float, float) -> float

For instance, if you have 110250 samples over 2.5 seconds the samplerate will be samplerate(110250., 2.5) which is 44100. However, if you mix the order of the arguments and type samplerate(2.5, 110250.), you will get quite a different result and this will not be detected by the typing system because both arguments have the same type. Fortunately, we can give labels to arguments in order to prevent this, which forces explicitly naming the arguments. This is indicated by prefixing the arguments with a tilde “~”:

def samplerate(~samples, ~duration) = samples / duration end

The labels will be indicated as follows in the type:

(samples : float, duration : float) -> float

Namely, in the above type, we read that the argument labeled samples is a float and similarly for the one labeled duration. For those arguments, we have to give the name of the argument when calling the function:

samplerate(samples=110250., duration=2.5)

The nice byproduct is that the order of the arguments does not matter anymore, the following will give the same result:

samplerate(duration=2.5, samples=110250.)

Of course, a function can have both labeled and non-labeled arguments.

Optional arguments

Another useful feature is that we can give default values to arguments, which thus become optional: if, when calling the function, a value is not specified for such arguments, the default value will be used. For instance, if for some reason we tend to generally measure samples over a period of 2.5 seconds, we can make this become the value for the duration parameter:

def samplerate(~samples, ~duration=2.5) =
  samples / duration
end

In this way, if we do not specify a value for the duration, its value will implicitly be assumed to be 2.5, so that the expression:

samplerate(samples=110250.)

will still evaluate to 44100. Of course, if we want to use another value for the duration, we can still specify it, in which case the default value will be ignored:

samplerate(samples=132300., duration=3.)

The presence of an optional argument is indicated in the type by prefixing the corresponding label with “?”, so that the type of the above function is

(samples : float, ?duration : float) -> float

Getters

We often want to be able to dynamically modify some parameters in a script. For instance, consider the operator amplify, which takes a float and an audio source and returns the audio amplified by the given volume factor: we can expect its type to be

(float, source('a)) -> source('a)

so that we can use it to have a radio consisting of a microphone input amplified by a factor 1.2 by

mic   = input.alsa()
radio = amplify(1.2, mic)

In the above example, the volume 1.2 was supposedly chosen because the sound delivered by the microphone is not loud enough, but this loudness can vary from time to time, depending on the speaker for instance, and we would like to be able to dynamically update it. The problem with the current operator is that the volume is of type float and a float cannot change over time: it has a fixed value.

In order for the volume to have the possibility to vary over time, instead of having a float argument for amplify, we have decided to have instead an argument of type

() -> float

This is a function which takes no argument and returns a float (remember that a function can take an arbitrary number of arguments, which includes zero arguments). It is very close to a float excepting that each time it is called the returned value can change: we now have the possibility of having something like a float which varies over time. We like to call such a function a float getter, since it can be seen as some kind of object on which the only operation we can perform is get the value. For instance, we can define a float getter by

n = ref(0.)
def f ()
  n := n() + 1.
  n()
end

Each time we call f, by writing f() in our script, the resulting float will be increased by one compared to the previous one: if we try it in an interactive session, we obtain

# f();;
- : float = 1.0
# f();;
- : float = 2.0
# f();;
- : float = 3.0

Since defining such arguments often involves expressions of the form

fun () -> e

which is somewhat heavy, we have introduced the alternative syntax

{e}

for it.

Finally, in order to simplify things a bit, you will see that the type of amplify is actually

({float}, source('a)) -> source('a)

where the type {float} means that both float and () -> float are accepted, so that you can still write constant floats where float getters are expected. What we actually call a getter is generally an element of such a type, which is either a constant or a function with no argument.

In order to work with such types, the standard library often uses the following functions:

  • getter, of type ({'a}) -> {'a}, creates a getter,
  • getter.get, of type ({'a}) -> 'a, retrieves the current value of a getter,
  • getter.function, of type ({'a}) -> () -> 'a, creates a function from a getter.

Recursive functions

Liquidsoap supports functions which are recursive, i.e., that can call themselves. For instance, in mathematics, the factorial function on natural numbers is defined as fact(n)=1×2×3×…×n, but it can also be defined recursively as the function such that fact(0)=1 and fact(n)=n×fact(n-1) when n>0: you can easily check by hand that the two functions agree on small values of n (and prove that they agree on all values of n). This last formulation has the advantage of immediately translating to the following implementation of factorial:

def rec fact(n) =
  if n == 0 then 1
  else n * fact(n-1) end
end

for which you can check that fact(5) gives 120, the expected result. As another example, the list.length function, which computes the length of a list, can be programmed in the following way in Liquidsoap:

def rec length(l)
  if l == [] then 0
  else 1 + length(list.tl(l)) end
end

We do not detail much further this trait since it is unlikely to be used for radios, but you can see a few occurrences of it in the standard library.

Records and modules

Records

Suppose that we want to store and manipulate structured data. For instance, a list of songs together with their duration and tempo. One way to store each song is as a tuple of type string * float * float, but there is a risk of confusion between the duration and the length which are both floats, and the situation would of course be worse if there were more fields. In order to overcome this, one can use a record which is basically the same as a tuple, excepting that fields are named. In our case, we can store a song as

song = { filename = "song.mp3", duration = 257., bpm = 132. }

which is a record with three fields respectively named filename, duration and bpm. The type of such a record is

{filename : string, duration : float, bpm : float}

which indicates the fields and their respective type. In order to access a field of a record, we can use the syntax record.field. For instance, we can print the duration with

print("The duration of the song is #{song.duration} seconds")

Records can be re-used using spreads:

song = { filename = "song.mp3", duration = 257., bpm = 132. }

# This is a fresh value with all the fields from `song` and
# a new `id` field:
song_with_id = { id = 1234, ...song }

Alternatively, you can also extend a record using the explicit v.{...} syntax:

song = { filename = "song.mp3", duration = 257., bpm = 132. }

# This is a fresh value with all the fields from `song` and
# a new `id` field:
song_with_id = song.{id = 1234}

Modules

Records are heavily used in Liquidsoap in order to structure the functions of the standard library. We tend to call module a record with only functions, but this is really the same as a record. For instance, all the functions related to lists are in the list module and functions such as list.hd are fields of this record. For this reason, the def construction allows adding fields in record. For instance, the definition

def list.last(l)
  list.nth(l, list.length(l)-1)
end

adds, in the module list, a new field named last, which is a function which computes the last element of a list. Another shorter syntax to perform definitions consists in using the let keyword which allows assigning a value to a field, so that the previous example can be rewritten as

let list.last = fun(l) -> list.nth(l, list.length(l)-1)

If you often use the functions of a specific module, the open keyword allows using its fields without having to prefix them by the module name. For instance, in the following example

l = [1,2,3]
open list
x = nth(l, length(l)-1)

the open list directive allows directly using the functions in this module: we can simply write nth and length instead of list.nth and list.length.

Values with fields

A unique feature of the Liquidsoap language is that it allows adding fields to any value. We also call them methods by analogy with object-oriented programming. For instance, we can write

song = "test.mp3".{duration = 123., bpm = 120.}

which defines a string ("test.mp3") with two methods (duration and bpm). This value has type

string.{duration : float, bpm : float}

and behaves like a string, e.g. we can concatenate it with other strings:

print("the song is " ^ song)

but we can also invoke its methods like a record or a module:

print("the duration is #{song.duration}")

The construction def replaces allows changing the main value while keeping the methods unchanged, so that

def replaces song = "newfile.mp3" end
print(song)

will print

"newfile.mp3".{duration = 123., bpm = 120.}

(note that the string is modified but not the fields duration and bpm).

Optional fields

During the execution of your script, it can be useful to allow functions to receive records that may or may not have a specific field. This can be used, for instance, to model optional arguments.

This can be achieved in two ways:

  1. Using the x.foo ?? default syntax

Here’s an example:

# This functions adds 1 to x unless options has a
# add field in which case it adds this value
def f(x, options) =
  x + (options.add ?? 1)
end

The type of this function is:

f : (int, 'a.{add? : int}) -> int = <fun>

which denotes that the options argument can be any value that may or may not have a add field. However, if this field is present, it must be of type int.

  1. Using the x?.foo syntax

Given a variable x, x?.foo returns the field value foo, if present, or null otherwise.

The ?. syntax can be chained and works with functions, which make it a very convenient way to drill deep inside nested records:

x?.fn(123, "aabb")?.field

Patterns

As explained earlier, you can use several constructions to extract data from structured values such as let [x, y] = l and etc. These constructions are called patterns.

Patterns allows to quickly access values nested deeply inside structured data in a way that remains pretty intuitive when reading the code.

Patterns are constructed using variable placeholders, which are either a variable name such as: x, foo, etc. or the special symbol _ for any ignored value.

Tuple patterns

Tuple patterns are pretty straight forward and consist of any sequence of variable captures:

let (x, y, _, z) = (123, "aabbcc", true, 3.14)
# x = 1, y = "aabbcc", z = 3.14

List patterns

List patterns are composed of variable placeholders, etc. and spreads of the form: ...<variable placeholder> such as: ...z. The spread ..._ can be simply written .... See below for an example.

You can use any combination of:

  • Forward variable names: these capture the first elements of the list.
  • One spread: this captures any remaining element as a list.
  • Backward variable names: these capture the last elements of a the list.

Here are some examples:

# Forward capture:
let [x, y, z] = [1, 2, 3]
# x = 1, y = 2, z = 3

# Forward capture with spread:
let [x, y, ...z] = [1, 2, 3, 4]
# x = 1, y = 2, z = [3, 4]

# Forward capture with ignored values:
let [_, x, ...z] = [1, 2, 3, 4]
# x = 2, z = [3, 4]

# Full capture:
let [x, y, ...z, t, u, v] = [1, 2, 3, 4, 5, 6, 7, 8, 9]
# x = 1, y = 2, z = [3, 4, 5, 6, 7], t = 7, u = 8, v = 9

# Backward capture only.
let [..., t, u, v] = [1, 2, 3, 4, 5]
# t = 3, u = 4, v = 5

Record and module patterns

Record and module patterns consist of either variable names (not variable capture!), which capture method values or variable names with an associated pattern.

Record patterns are of the form: {<captured methods>} while module patterns are of the form: <variable capture>.{<captured methods>}

Here are some examples:

# Record capture
let {foo, bar} = {foo = 123, bar = "baz", gni = true}
# foo = 123, bar = "baz"

# Record capture with spread
let {foo, bar, ...x} = {foo = 123, bar = "baz", gni = true}
# foo = 123, bar = "baz", x = {gni = true}

# Module capture
let v.{foo, bar} = "aabbcc".{foo = 123, bar = "baz", gni = true}
# v = "aabbcc", foo = 123, bar = "baz"

# Module capture with ignored value
let _.{foo, bar} = "aabbcc".{foo = 123, bar = "baz", gni = true}
# foo = 123, bar = "baz"

# Record capture with sub-patterns. Same works for module!
let {foo = [x, y, z], gni} = {foo = [1, 2, 3], gni = "baz"}
# foo = [1, 2, 3], x = 1, y = 2, z = 3, gni = "baz"

# Record capture with optional methods:
let { foo? } = ()
# foo = null()

let { foo? } = { foo = 123 }
# foo = 123

Combining patterns

As seen with record and modules, patterns can be combined at will, for instance, these are all valid patterns:

let [{foo}, {gni}, ..., {baz}] = l

let (_.{ bla = [..., z] }, t, _, u) = x

Advanced values

In this section, we detail some more advanced values than the ones presented in. You are not expected to be understanding those in details for basic uses of Liquidsoap.

Errors

In the case where a function does not have a sensible result to return, it can raise an error. Typically, if we try to take the head of the empty list without specifying a default value (with the optional parameter default), an error will be raised. By default, this error will stop the script, which is usually not a desirable behavior. For instance, if you try to run a script containing

list.hd([])

the program will exit printing

Error 14: Uncaught runtime error:
type: not_found, message: "no default value for list.hd"

This means that the error named “not_found” was raised, with a message explaining that the function did not have a reasonable default value of the head to provide.

In order to avoid this, one can catch exceptions with the syntax

try
  code
catch err do
  handler
end

This will execute the instructions code: if an error is raised at some point during this, the code handler is executed, with err being the error. For instance, instead of writing

l = []
x = list.hd(default=0, l)

we could equivalently write

l = []
x =
  try
    list.hd(l)
  catch err do
    0
  end

The name and message associated to an error can respectively be retrieved using the error kind and message attributes, e.g. we can write

try
  ...
catch err do
  print("the error #{err.kind} was raised")
  print("the error message is #{err.message}")
end

Typically, when reading from or writing to a file, errors will be raised when a problem occurs (such as reading from a non-existent file or writing a file in a non-existent directory) and one should always check for those and log the corresponding message:

data = "bla"
try
  file.write(data=data, "/non/existent/path")
catch err do
  log.important("Could not write to file: #{error.message(err)}")
end

Specific errors can be caught with the syntax

try
  ...
catch err : l do
  ...
end

where l is a list of error names that we want to handle here.

Errors can be raised from Liquidsoap with the function error.raise, which takes as arguments the error to raise and the error message. For instance:

error.raise(error.not_found, "we could not find your result")

We should also mention that all the errors should be declared in advance with the function error.register, which takes as argument the name of the new error to register:

myerr = error.register("my_error")
error.raise(myerr, "testing my own error")

Lastly, if you need to make sure that a certain piece of code is executed whether or not there is an exception raised, you can use finally:

# Without a catch block
try
  ...
finally
  ...
end

# With a catch block
try
  ...
catch ... do
  ...
finally
  ...
end

This is roughly equivalent to:

finally_called = ref(false)
def finally() = ... end
try
  let ret = ...
  finally_called := true
  finally()
  ret
# If specified:
catch ... do
  let ret = ...
  if not finally_called() then finally() end
  ret
end

The biggest different is that finally is called on all errors, including internal errors that cannot be caught by the runtime code.

Errors raised in a finally block do override any previously raised errors.

Nullable values

It is sometimes useful to have a default value for a type. In Liquidsoap, there is a special value for this, which is called null. Given a type t, we write t? for the type of values which can be either of type t or be null: such a value is said to be nullable. For instance, we could redefine the list.hd function in order to return null (instead of raising an error) when the list is empty:

def list.hd(l)
  if l == [] then null() else list.hd(l) end
end

whose type would be

(['a]) -> 'a?

since it takes as argument a list whose elements are of type 'a and returns a list whose elements are 'a or null. As it can be observed above, the null value is created with null().

In order to use a nullable value, one typically uses the construction x ?? d which is the value x excepting when it is null, in which case it is the default value d. For instance, with the above head function:

x = list.hd(l)
print("the head is " ^ (x ?? "not defined"))

Some other useful functions include

  • null.defined: test whether a value is null or not,
  • null.get: obtain the value of a nullable value supposed to be distinct from null,
  • null.case: execute a function or another, depending on whether a value is null or not.

Runtime evaluation of scripting values

Similarly to how JSON is parsed, you can evaluate string into values at runtime using the eval decorator. As with JSON, too, the recommended way to use it is by adding an explicit type annotation:

let eval (x: {foo: int, bla: string}) = "{foo = 123, bla = \"gni\"}"
print("x.foo = #{x.foo}, x.bla = #{x.bla}")

Including other files

It is often useful to split your script over multiple files, either because it has become quite large, or because you want to be able to reuse common functions between different scripts. You can include a file file.liq in a script by writing

%include "file.liq"

which will be evaluated as if you had pasted the contents of the file in place of the command.

For instance, this is useful in order to store passwords out of the main file, in order to avoid risking leaking those when handing the script to some other people. Typically, one would have a file passwords.liq defining the passwords in variables, e.g.

radio_pass = "secretpassword"

and would then use it by including it:

%include "passwords.liq"

radio = ...
output.icecast(%mp3, host="localhost", port=8000,
               password=radio_pass, mount="my-radio.mp3", radio)

so that passwords are not shown in the main script.

Code comments

Comments can be added to your code in two ways:

Multi-line comments are comments that can span multiple lines. They are delimitated by the sequence of characters #< at the beginning and ># at the end. Anything in between those two sequences is considered code comment.

Here are some examples:

Simple multiline comments:

#< This is a comment >#

Multiline comments can be nested:

#<
This is a top-level comment

  # This is also a comment

  #<
    This is a nested code comment
  >#
>#

Fancy looking multiline comment

#<------- BEGIN CODE COMMENT ----#
Comments can also look like this
#--------- END CODE COMMENT ----->#

Single-line comments are comments that are limited to the current line. Such comments are started with the character # without a following <. Anything after the initial # character and until the end of the line is considered code comment:

def f(x) = # This is a single line comment.
  123
end

Caching

Type-checking scripts can take a lot of time and consume memory. To optimize things, this step can be cached.

During the first execution, the script is parsed, type checked and evaluated. On second and any following execution, a cache of the script is used, reducing the typechecking phase, sometimes by a 100x factor!

Here’s a log without caching on a M3 macbook pro:

2024/07/03 14:31:41 [startup:3] main script hash computation: 0.03s
2024/07/03 14:31:41 [startup:3] main script cache retrieval: 0.03s
2024/07/03 14:31:41 [startup:3] stdlib hash computation: 0.03s
2024/07/03 14:31:41 [startup:3] stdlib cache retrieval: 0.03s
2024/07/03 14:31:41 [startup:3] Typechecking stdlib: 3.37s
2024/07/03 14:31:41 [startup:3] Typechecking main script: 0.00s

And the same log after caching:

2024/07/03 14:32:59 [startup:3] main script hash computation: 0.02s
2024/07/03 14:32:59 [startup:3] Loading main script from cache!
2024/07/03 14:32:59 [startup:3] main script cache retrieval: 0.05s

Scripts can be cached ahead of time without executing them, for instance while compiling a docker image, using --cache-only. Caching can also be disabled using --no-cache.

Caching happens at two different time:

  • First the standard library is cached
  • Then the script itself is cached

Caching the standard library makes it possible to run the type-checker faster on new scripts. Here’s an example of a log from running a new script with a cached standard library:

2024/07/03 14:33:27 [startup:3] main script hash computation: 0.02s
2024/07/03 14:33:27 [startup:3] main script cache retrieval: 0.02s
2024/07/03 14:33:27 [startup:3] stdlib hash computation: 0.03s
2024/07/03 14:33:27 [startup:3] Loading stdlib from cache!
2024/07/03 14:33:27 [startup:3] stdlib cache retrieval: 0.10s
2024/07/03 14:33:27 [startup:3] Typechecking main script: 0.00s

Caching can be disabled by setting LIQ_CACHE to anything else than "true".

Cache locations

Cache files can accumulate and also take up disk space so it is important to know where they are located!

There are two type of cache locations:

  • System cache for cached files that should be shared with all liquidsoap scripts. This is where the standard library cache is located. This location is a system-wide path on unix system such as /var/cache/liquidsoap.
  • User cache for cached files that are specific to the user running liquidsoap scripts. On unix systems, this location is at $HOME/.cache/liquidsoap.

On windows, the default cache directory for both type of cache locations is in the same directory as the binary.

At runtime, liquidsoap.cache(mode=<mode>) returns the cache directory. mode should be one of: "user" or "system".

Cache maintenance

There is a cache maintenance routine which deletes unused cache files after 10 days and keeps the cache to a maximum of 200 files.

You can run the cache maintenance routing by calling liquidsoap.cache.maintenance(mode=<mode>) manually. Here, too, mode should be one of: "user" or "system".

Cache security

Please be aware that the cache does not encrypt its values. As such, user cache files should be considered sensitive as they may contain password and other runtime secrets that are available through your scripts. We recommend to:

  • Use environment variables as much as possible when passing secrets
  • Secure your user script and cache files.

The default creation permissions for user cache files is: 0o600 so only the user creating them should be able to read them. You should make sure that your script permissions are also similarly restricted.

Cache and memory usage

One side-benefit from loading a script from cache is that the entire typechecking process is skipped.

This can result is significant reduction in the initial memory consumption, typically down from about 375MB to about 80MB!

If memory consumption is a concern but you are not sure you can cache your script, you can also set the environment variable settings.init.compact_before_start to true:

settings.init.compact_before_start := true

This will run the OCaml memory compaction algorithm after typechecking your script but before running it. This will result in a similar memory footprint when running the script but will delay its initial startup time.

Cache environment variables

The following environment variables control the cache behavior:

  • LIQ_CACHE: disable the cache when set to anything else than 1 or true
  • LIQ_CACHE_SYSTEM_DIR: set the cache system directory
  • LIQ_CACHE_SYSTEM_DIR_PERMS: set the permission used when creating cache system directory (and its parents when needed). Default: 0o755
  • LIQ_CACHE_SYSTEM_FILE_PERMS: set the permissions used when creating a system cache file. Default: 0o644
  • LIQ_CACHE_USER_DIR: set the cache user directory
  • LIQ_CACHE_USER_DIR_PERMS: set the permission used when creating cache user directory (and its parents when needed). Default: 0o700.
  • LIQ_CACHE_USER_FILE_PERMS: set the permissions used when creating a user cache file. Default: 0o600
  • LIQ_CACHE_MAX_DAYS: set the maximum days a cache file can be stored before it is eligible to be deleted during the next cache maintenance pass.
  • LIQ_CACHE_MAX_FILES: set the maximum number of files in each cache directory. Older files are removed first.