haraldwolff
/
collabora-online
mirror of https://github.com/CollaboraOnline/online.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
collabora-online/wasm
History
Michael Meeks 4423770c52 cowasm: disable werror for now for dlopen detection: 
otherwise generated:

| int
| main ()
| {
| return dlopen ();

wasm-ld: warning: function signature mismatch: dlopen
>>> defined as () -> i32 in /tmp/emscripten_temp_6nz4f0ww/conftest_0.o
>>> defined as (i32, i32) -> i32 in /home/builder/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/libc.a(dynlink.o)
em++: error: running limited binaryen optimizations because DWARF info requested (or indirectly required) [-Wlimited-postlink-optimizations] [-Werror]

and erroneously claims that dlopen is not present, and
configure fails.

Change-Id: Ie54db7c2099787a0b63b76b0b6e3e44750c9046e
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
 		2023-10-29 11:05:18 -04:00
..
Makefile.am WASM: fix wasm build errors 2023-10-03 10:18:46 +02:00
README cowasm: disable werror for now for dlopen detection: 2023-10-29 11:05:18 -04:00
README.no-container.md wasm: support serving wasm files 2023-10-24 21:33:02 -04:00
base64.hpp Make the temporary "WASM app" work 2023-01-28 22:04:39 +02:00
poco-1.12.4-emscripten.patch Drop POCO_HAVE_FD_EPOLL for Emscripten 2023-02-22 09:00:39 +01:00
poco-devel-emscripten.patch WASM document how to build dependencies in wasm/README 2023-01-09 15:21:37 +02:00
poco-no-special-expat-sauce.diff Add patch that prevents expat in Poco from causing trouble with WASM 2023-01-16 18:43:21 +02:00
wasmapp.cpp Bin superfluous logging of what we are about to send to the JS bits 2023-02-10 18:37:16 +01:00
wasmapp.hpp More WASM hacking 2023-01-26 22:01:14 +02:00

README 

NOTE: For instructions on how to build and run it without the
Allotropia-supplied container, see wasm/README.no-container.md.

= WASM with emscripten =

Building for WASM with emscripten is still a bit immature and many projects
need patching to work.

Note that there are wrapper tools like "emconfigure", "emmake", "emcmake" that
tend to set up environment variables like CC properly, although some project's
build system unfortunately override that.

Linking WASM executable uses a lot of memory; without optimisations or with -O1
in LDFLAGS, /usr/bin/time prints "1660068maxresident" which should work
anywhere, but with -O2 in LDFLAGS (which is the default) some LTO happens that
uses about 8x as much RAM, "12261016maxresident" or ~12 GiB... so append
CXXFLAGS='-g -O1' or similar to the configure invocation if out of memory.

For convenience it's recommended to build with a docker container that has the
required build tools.

    podman pull public.ecr.aws/allotropia/libo-builders/wasm

== Build LO core ==

First you need to build LibreOffice core with emscripten.
Currently best to use "feature/wasm" branch.

It can be good to create an environment file you can source in new shells eg.

export GITREPO=$HOME/lo/feature_wasm
export EXTSOURCES=$HOME/lo/ext_sources
export BUILDDIR=/data/lo/build_feature_wasm
export ONLINEGITREPO=$HOME/lo/online
export ONLINEBUILDDIR=/data/lo/build_online_wasm

These examples assume that the git repo is in $GITREPO, the build
directory in $BUILDDIR and the tarball download cache in
$EXTSOURCES - adjust these to your preferences.

    podman run -v $GITREPO:$GITREPO:ro -v $BUILDDIR:$BUILDDIR -v $EXTSOURCES:/ext_sources:rw --security-opt=label=disable -ti public.ecr.aws/allotropia/libo-builders/wasm /bin/bash -c "source /home/builder/emsdk/emsdk_env.sh && cd $BUILDDIR/ && $GITREPO/autogen.sh --with-external-tar=/ext_sources --with-distro=LibreOfficeWASM32"

    podman run -v $GITREPO:$GITREPO:ro -v $BUILDDIR:$BUILDDIR -v $EXTSOURCES:/ext_sources:rw --security-opt=label=disable -ti public.ecr.aws/allotropia/libo-builders/wasm /bin/bash -c "source /home/builder/emsdk/emsdk_env.sh && cd $BUILDDIR/ && make -rj8"

== Build Online using dependencies from container ==

The container contains prebuilt libzstd and POCO for convenience.

Note that here the LO core (and builddir, if separate) must be mapped to the
same directory in the container as it was when building core, due to files
containing absolute paths.

    podman run -v $GITREPO:$GITREPO:ro -v $BUILDDIR:$BUILDDIR:ro -v $ONLINEGITREPO:$ONLINEGITREPO:rw -v $ONLINEBUILDDIR:$ONLINEBUILDDIR:rw --security-opt=label=disable -ti public.ecr.aws/allotropia/libo-builders/wasm /bin/bash -c "source /home/builder/emsdk/emsdk_env.sh && cd $ONLINEBUILDDIR/ && $ONLINEGITREPO/autogen.sh && emconfigure $ONLINEGITREPO/configure --disable-werror --with-lokit-path=$GITREPO/include --with-lo-path=$BUILDDIR/instdir --with-lo-builddir=$BUILDDIR --with-zstd-includes=/usr/local/include --with-zstd-libs=/usr/local/lib --with-poco-includes=/usr/local/include --with-poco-libs=/usr/local/lib --host=wasm32-local-emscripten"
    podman run -v $GITREPO:$GITREPO:ro -v $BUILDDIR:$BUILDDIR:ro -v $ONLINEGITREPO:$ONLINEGITREPO:rw -v $ONLINEBUILDDIR:$ONLINEBUILDDIR:rw --security-opt=label=disable -ti public.ecr.aws/allotropia/libo-builders/wasm /bin/bash -c "source /home/builder/emsdk/emsdk_env.sh && cd $ONLINEBUILDDIR/ && emmake make -rj8"

== Build Online dependencies manually ==

Then build libzstd:
* with assembly code disabled
* using the Makefile (didn't try its other build systems)

    tar -xzvf ~/Downloads/zstd-1.5.2.tar.gz
    cd zstd-1.5.2/
    podman run -v /data/lo/zstd-1.5.2:/data/lo/zstd-1.5.2:rw --security-opt=label=disable -ti public.ecr.aws/allotropia/libo-builders/wasm /bin/bash -c 'source /home/builder/emsdk/emsdk_env.sh && cd /data/lo/zstd-1.5.2/ && emmake make -j8 lib-nomt ZSTD_NO_ASM=1'

Then build POCO:
* this requires two patches (currently exists patch for 1.12.4 and untested "devel"
  branch) plus renaming one source file (which actually is from expat but Poco includes)
* note that the header Poco/Platform.h maps EMSCRIPTEN to POCO_OS_LINUX
* it has both Makefiles and CMake but i couldn't get CMake to use CXXFLAGS to
  disable some Linux-specific code; i tried variations on:
    emcmake cmake -S .. -DENABLE_MONGODB=off -DENABLE_DATA_SQLITE=off -DENABLE_REDIS=off -DENABLE_PROMETHEUS=off -DENABLE_PDF=off -DENABLE_SEVENZIP=off -DENABLE_ZIP=off -DENABLE_CPPPARSER=off -DENABLE_POCODOC=off -DENABLE_PAGECOMPILER=off -DENABLE_PAGECOMPILER_FILE2PAGE=off -DENABLE_ACTIVERECORD=off -DENABLE_ACTIVERECORD_COMPILER=off -DENABLE_DATA=off -DENABLE_DATA_ODBC=off -DENABLE_DATA_POSTGRESQL=off -DENABLE_DATA_MYSQL=off -DENABLE_APACHECONNECTOR=off -DENABLE_JWT=off CXXFLAGS=-DPOCO_NO_LINUX_IF_PACKET_H
* Online requires a single include directory so "make install" must be used

    tar -xjvf ~/Downloads/poco-1.12.4-all.tar.bz2
    cd poco-1.12.4-all
    patch -p1 < $ONLINEGITREPO/wasm/poco-1.12.4-emscripten.patch
    mv XML/src/xmlparse.cpp XML/src/xmlparse.c
    patch -p0 < $ONLINEGITREPO/wasm/poco-no-special-expat-sauce.diff
    podman run -v /data/lo/poco-1.12.4-all:/data/lo/poco-1.12.4-all:rw --security-opt=label=disable -ti public.ecr.aws/allotropia/libo-builders/wasm /bin/bash -c 'source /home/builder/emsdk/emsdk_env.sh && cd /data/lo/poco-1.12.4-all && emconfigure ./configure --static --no-samples --no-tests --omit=Crypto,NetSSL_OpenSSL,JWT,Data,Data/SQLite,Data/ODBC,Data/MySQL,Data/PostgreSQL,Zip,PageCompiler,PageCompiler/File2Page,MongoDB,Redis,ActiveRecord,ActiveRecord/Compiler,Prometheus && emmake make -j8 CC=/home/builder/emsdk/upstream/emscripten/emcc CXX=/home/builder/emsdk/upstream/emscripten/em++ LD=/home/builder/emsdk/upstream/emscripten/em++ CXXFLAGS="-DPOCO_NO_LINUX_IF_PACKET_H -DPOCO_NO_INOTIFY -pthread -s USE_PTHREADS=1 -s DISABLE_EXCEPTION_CATCHING=0" && make -j8 install INSTALLDIR=/data/lo/poco-1.12.4-all/install'

== Build Online with manually built dependencies ==

Then build Online itself:

Note that here the LO core (and builddir, if separate) must be mapped to the
same directory in the container as it was when building core, due to files
containing absolute paths.

    podman run -v /data/lo/zstd-1.5.2:/data/lo/zstd-1.5.2:ro -v /data/lo/poco-1.12.4-all:/data/lo/poco-1.12.4-all:ro -v $GITREPO:$GITREPO:ro -v $BUILDDIR:$BUILDDIR:ro -v $ONLINEGITREPO:$ONLINEGITREPO:rw -v $ONLINEBUILDDIR:$ONLINEBUILDDIR:rw --security-opt=label=disable -ti public.ecr.aws/allotropia/libo-builders/wasm /bin/bash -c "source /home/builder/emsdk/emsdk_env.sh && cd $ONLINEBUILDDIR/ && $ONLINEGITREPO/autogen.sh && emconfigure $ONLINEGITREPO/configure --with-lokit-path=$GITREPO/include --with-lo-path=$BUILDDIR/instdir --with-lo-builddir=$BUILDDIR --with-zstd-includes=/data/lo/zstd-1.5.2/lib --with-zstd-libs=/data/lo/zstd-1.5.2/lib --with-poco-includes=/data/lo/poco-1.12.4-all/install/include --with-poco-libs=/data/lo/poco-1.12.4-all/install/lib --host=wasm32-local-emscripten"
    podman run -v /data/lo/zstd-1.5.2:/data/lo/zstd-1.5.2:ro -v /data/lo/poco-1.12.4-all:/data/lo/poco-1.12.4-all:ro -v $GITREPO:$GITREPO:ro -v $BUILDDIR:$BUILDDIR:ro -v $ONLINEGITREPO:$ONLINEGITREPO:rw -v $ONLINEBUILDDIR:$ONLINEBUILDDIR:rw --security-opt=label=disable -ti public.ecr.aws/allotropia/libo-builders/wasm /bin/bash -c "source /home/builder/emsdk/emsdk_env.sh && cd $ONLINEBUILDDIR/ && emmake make"