Build FFmpeg WebAssembly version (= ffmpeg.js): Part.3 ffmpeg.js v0.1.0 — Transcoding avi to mp4

Previous story: Build FFmpeg WebAssembly version (= ffmpeg.js): Part.2 Compile with Emscripten

In this party you will learn:

  1. Build a library version of FFmpeg with optimized arguments.
  2. Manage Emscripten File System.
  3. Develop ffmpeg.js v0.1.0 with transcoding feature.

Build a library version of FFmpeg with optimized arguments.

In Part.3, our goal is to create a basic ffmpeg.js v0.1.0 to transcode avi to mp4, but in Part.2, we only create a “bare metal” version of FFmpeg and we need to further optimized with few arguments.

  1. -Oz : optimize code and reduce code size (from 30 MB to 15 MB)
  2. -o javascript/ffmpeg-core.js : output js and wasm file to javascript folder.
    (From here we call it ffmpeg-core.js as we will create a ffmpeg.js library to wrap the ffmpeg-core.js and provide user friendly APIs.)
  3. -s MODULARIZE=1 : create a library instead of a command line tool (need to modify source code, more details below)
  4. -s EXPORTED_FUNCTIONS="[_ffmpeg]" : export “ffmpeg” C function to JavaScript world
  5. -s EXTRA_EXPORTED_RUNTIME_METHODS="[cwrap, FS, getValue, setValue]" : extra functions for manipulating functions, file system and pointers, check Interacting with code to know details.
  6. -s ALLOW_MEMORY_GROWTH=1 : Allow memory to grow when running out of memory
  7. -lpthread : removed as we want to create our owned version of worker. (details in Part.4)
For more details about these arguments, you can check src/settings.js in emscripten github repository.

As we add -s MODULARIZE=1, it is necessary to update the source code to fit the requirement of modularization (basically remove the main() function). We only have to modify 3 lines of code to make it work.

  1. fftools/ffmpeg.c : rename main to ffmpeg
- int main(int argc, char **argv)
+ int ffmpeg(int argc, char **argv)

2. fftools/ffmpeg.h : add ffmpeg in the end of the file to export ffmpeg function

+ int ffmpeg(int argc, char** argv);
#endif /* FFTOOLS_FFMPEG_H */

3. fftools/cmdutils.c : comment exit(ret) as we don’t want our library to exit the runtime for us. (we will enhance this part in the future)

void exit_program(int ret){
if (program_exit)
program_exit(ret);
-   exit(ret);
+ // exit(ret);
}

Our new version of build script:

Now we have our ffmpeg-core.js ready!

If you are familiar with ffmpeg, you must know the common usage of ffmpeg is through command line like this:

$ ffmpeg -i input.avi output.mp4

As we reuse main function to be our ffmpeg function, below is how we will call our function:

const args = ['./ffmpeg', '-i', 'input.avi', 'output.mp4'];
ffmpeg(args.length, args);

Of course it is not that simple, we need to do some pre-processing to bridge the world of JavaScript and C, let’s start from the emscripten File System.

Manage Emscripten File System.

In emscripten, there is a virtual file system to support standard file read/write in C, as our ffmpeg-core.js reuses command line version of FFmpeg which reads and writes video files to file system, it is important to understand the fundamental concepts.

Find more details in File System API.

To use file system, you need to export the FS API from emscripten first, this is achieved by one of the arguments above:

-s EXTRA_EXPORTED_RUNTIME_METHODS="[cwrap, FS, getValue, setValue]"

To save a file to file system, you need to prepare an array in Uint8Array format, in Node.js environment, you can do something like this:

const fs = require('fs');
const data = new Uint8Array(fs.readFileSync('./input.avi'));

And save to emscripten file system by FS.writeFile():

require('./ffmpeg-core.js)()
.then(Module => {
Module.FS.writeFile('input.avi', data);
});

To load file from emscripten file system is similar:

require('./ffmpeg-core.js)()
.then(Module => {
const data = Module.FS.readFile('output.mp4');
});

With this concept in mind, now let’s move on to the development of ffmpeg.js which hides complex concepts and provide user friendly APIs.

Develop ffmpeg.js v0.1.0 with transcoding feature.

The development of ffmpeg.js is not easy as you need to switch between JavaScript and C context, but if you are familiar with pointers, it will be much easier for you to understand what we are doing most of the time.

In this part, our goal is to develop a ffmpeg.js library like this:

To begin with, we need to load our ffmpeg-core.js, the loading process of a library built by emscripten is asynchronous as we don’t want it to block our main thread.

A typical way of loading can be:

You might feel strange that we use Promise to wrap another Promise, it is because FFmpegCore() is not a real promise, just a function with similar API.

Module is the initialized library for us to perform all the operations, here we use a Singleton utility to store it and get later.

Next step we will use Module to get our ffmpeg function, here we need to use cwrap :

// int ffmpeg(int argc, char **argv)
const ffmpeg = Module.cwrap('ffmpeg', 'number', ['number', 'number']);

The 1st argument of cwrap is the name of the function (must be in EXPORTED_FUNCTIONS with underscore prefix), the 2nd argument is the type of return value, the 3rd argument is the type of function arguments (int argc and char **argv).

It is obvious that type ofargc is number, but why type of argv is also number? As argv is actually a pointer, and what pointer stores is memory address (like 0xfffffff), so the type of pointer is a 32 bit unsigned int in WebAssembly. That’s why we use number as type of argv.

To call ffmpeg(), the 1st argument is easy as we can use native number in JavaScript without any modification, but the 2nd argument is not easy as we need to create a pointer points to a 2D array of type char (Uint8 in JavaScript).

Let break this problem into 2 sub-problems:

  1. How can we create a pointer points to an array of char?
  2. How can we create a pointer points to an array of pointers?

For the first problem, let’s create an utility function called str2ptr :

Module._malloc() is similar to malloc() in C, it allocates a portion of memory in heap. Module.setValue() set the actual value in the specific pointer.

Don’t forget to add an 0 in the end of char array, or you might have unexpected behaviors.

Now we have solved first problem, let’s create strList2ptr to solve the later problem:

The key here is to understand that pointer is a Uint32 value inside JavaScript, so listPtr is a Uint32 array pointer stores pointers to Uint8 array.

Put it all together, we have our first version of ffmepg.transcode() :

That’s it! Now we have our ffmpeg.js v0.1.0 to transcode avi to mp4.

If you would like to try ffmpeg.js v0.1.0 directly, you can now install it with:

$ npm install @ffmpeg/ffmpeg@0.1.0

The usage is very straight forward:

Please note currently it is a Node.js only version, but we will develop browser version in Part.4 with web worker (and child_process in Node.js)

Look forward to seeing you in Part.4. 😃

Repositories: