This is how I imagine that this stuff should work:
- In one tree you build with --host=wasm32-local-emscripten. This
results in only the WASM binary being built, containing C++ code
from Collabora Office core and C++ code from Online (to a similar
extent as in the iOS, Android, and gtk apps). No HTML and JS for the
client's browser is produced in this tree. The embedded filesystem
from the corresponding WASM core build could be copied into this
tree for use as below. (Or, alternatively the build in the other
tree could look into the config.status file in this tree to figure
out where to pick up the embedded fs data file.)
- In another tree you build with --with-wasm-fallback=<path to the
above WASM tree>. Note that the C++ code will be built normally in
this case, for Linux. This build results in an otherwise normal COOL
build, with the added feature that the COOL server sends the whole
document file also to the COOL client whenever it is saved. The HTML
and JS browser code will have code that enables receiving the
document in this case, plus incremental updates to the document (if
possible), and code to detect that the connection to the COOL server
is broken and then fall back to using the local WASM in the
browser. Obviously the WASM binary and related JS and other data
(like the embedded file system from core) will have to be downloaded
into the client right at the start, to be present if the connection
breaks, but the WASM will likely not need to be started until
needed.
Signed-off-by: Tor Lillqvist <tml@collabora.com>
Change-Id: I945c93451d3f5f0b1bc6ad8550da64e8e6453b6b
Copy the list of .cpp files from the Android project, assuming this will
be similar in scope.
Signed-off-by: Michael Stahl <michael.stahl@allotropia.de>
Change-Id: I57c7ad2f10d1867307ff4fcea3d0c650726d18d8
add zotero UI only if user is logged in
when zotero citation edit button is clicked,
fetch the library with zotero API.
Let user select what to insert
at this stage its not a complete solution but just first stage
Signed-off-by: Pranam Lashkari <lpranam@collabora.com>
Change-Id: Id67d8409bcd6416994713acd9ace495b24823fb0
The client connection management had nothing to
do with the WopiTestServer and it was there
for convenience. The issue was that it was
not available to tests that do not need
the custom WopiTestServer and depending
on it would make them unnecessarily
complicated.
This adds a new intermediary class to manage
client connections in tests. The reason
this logic didn't move to UnitWSD or even
UnitBase is simply because the client
connection logic depends on the helpers
namespace and other test-specific code
that isn't available in COOLWSD. In short,
UnitBase and UnitWSD are primarily an
interface that COOLWSD and co depend on
and cannot contain test-specific logic
that isn't linked to coolwsd.
We also move the UnitWebSocket helper class.
Change-Id: I79567774164e137349dc162482529578f150353c
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
zstd provides for much faster compression server-side, as well
as better decompression.
zstd allows us to de-compress a keyframe and several deltas in a
single call in JS, so it is necessar to add a terminator to the
delta stream so that we can detect when to flush the buffer we
are working on - so the next delta applies to the correct data.
Change-Id: I0e292e3a697b4902d6488b7c04deaba2d1485e94
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
This is a full round-trip http fuzzer.
It can achieve >1000 iterations per second
on a single 2 Ghz core, even while going
through the network loopback layer.
The advantage is that more networking code
is fuzzed this way, including not just
the http code, but also the sockets.
Change-Id: I75d21bd0e25221ee6621097a2605d62c4bb2ae4d
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
This adds support for code-coverage HTML reporting.
To achieve this, we must use file-linking in jails
so that we can update the coverage data (.gcda files)
from the jails. This means that creating jails is
slower than with bind-mounting and we need to
account for that in our timeouts.
We also can't kill child processes with SIGKILL,
which is un-catchable. Instead, we use SIGTERM
and dump the profile data before exiting.
Change-Id: I16fa534f6ed42f7133014d841bb024423315e0a4
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
coolwsd_fuzzer predates actual fuzzing
with libfuzzer and is currently unsused.
Change-Id: Id095b165943ba14dbf525ddc607ad329f5d952d4
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
This reduces the build time significantly while also
minimizing the disk-space requirements for builds.
When configured with --enable-fuzzers the fuzzing
target binaries are built.
Change-Id: Ia8560d876f548d04ac085503e55a3a5dca90f590
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
Apparently this breaks the build on at least
CentOS. Reverting for now.
This reverts commit 23f02b8929.
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
Change-Id: Ie8f00bbc53a63098b473d625846c0b6fd4e074bd
There are no deprecated API currently in use,
but that is a moving target as we upgrade openssl.
This prevents any accidental use of deprecated API
and will reveal any deprecated API used in any
particular build.
Change-Id: I969f7f7c4efe163d4704dea9aab45c09550baa40
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
An initial set of seven of these, easy to add more as/when needed.
Change-Id: I6c65e052d00f9eaa10adee3c9464043e4c594848
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
Build-time unit-test output is now only
displayed when the test fails. This is to
reduce noise while building when not helpful.
Change-Id: I273d97dae192a24e9a1ae9f662b0fcd7ff555b75
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
Refactor UnitWOPIDocumentConflict.cpp into
WOPIUploadConflictCommon.hpp and reuse for both
DocumentConflict and SaveOnExit tests.
Change-Id: I54ec1e37e5e9c6298b12a2b2e596363683fb2e34
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
number of versions to maintain per file can be specified in loolwsd.xml
on exceeding specified quarantine size oldest file(s) is deleted
Signed-off-by: Pranam Lashkari <lpranam@collabora.com>
Change-Id: I3ca55b9ab29a82988f19fe0acd43e0fae2c2a423
E: loolwsd: binary-or-shlib-defines-rpath usr/bin/loolconvert /snap/loolwsd/current/usr/lib
E: loolwsd: binary-or-shlib-defines-rpath usr/bin/loolconvert /opt/poco/lib
These RPATHs are not needed when we statically link poco, as we do with production
packages.
Signed-off-by: Andras Timar <andras.timar@collabora.com>
Change-Id: I5050fe1f1925937388793d443020fdc6a14ec97d
... in order to avoid confusion. Binaries that Collabora compile, and
distribute are under a EULA. Source code is not.
Signed-off-by: Andras Timar <andras.timar@collabora.com>
Change-Id: I34c7e089a100b43e6b9c69ee566736a9ae0cb93c
Some of the folders we setup from the system are:
dr-xr-xr-x 1 michael users 2824 Jun 9 21:05 systemplate/usr/share/fonts/
before adjustment, and resist removal.
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
Change-Id: I8e5a96264b98d8091b205f7469bc46c401f47ab1
Move the generic dummy implementation of
TraceEvent::emitOneRecording() to a source file of its own. (That is
the one which is used in test and tool executables.)
Signed-off-by: Tor Lillqvist <tml@collabora.com>
Change-Id: I81cab07e5a6852b42d278a5446c13c3825cf546e
Modelled on how it is done in core. ProfileZone is derived from
NamedEvent which is derived from TraceEvent. Here we don't keep any
separate ProfileZone.hpp, though.
This was needed to introduce generation of "instant" events here, too.
Signed-off-by: Tor Lillqvist <tml@collabora.com>
Change-Id: I6583134e96001641c50339deb4197fca6ab7d5d5
npm installation is taking too long,
it is preferable to process in parallel
while compiling some c++ source files.
Change-Id: Ie04726805723d94227806fb8b30c39909b84cb0f
Signed-off-by: Henry Castro <hcastro@collabora.com>
That is our convention, and this also avoids a global variable in the
kit process.
Change-Id: I37d2d53aa7eb24f9848fa8ef98bc57d75db90d13
Signed-off-by: Tor Lillqvist <tml@collabora.com>
We don't have permission for /var/log/loolwsd.log typically.
Change-Id: I37c04bfba25c4c8241b2d92d77d93e5a3b662d82
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
It enables turning Trace Event recording on (and off again). The
option is passed down to the client through loleaflet.html, and to the
KIT processes. If the option is not used, the new JS functions that
send trace events to the server turn into no-ops to avoid wasting
bandwidth.
It is always on in a "make run".
Change-Id: Iafe1919ccba7c376137d3e0568b857e20780bbc8
Signed-off-by: Tor Lillqvist <tml@collabora.com>
Compiled but not yet used.
Signed-off-by: Tor Lillqvist <tml@collabora.com>
Change-Id: I3b85696ca6076e42d16e710b49bfd37bac342ec8
Signed-off-by: Tor Lillqvist <tml@collabora.com>
HttpResponse covers http::Response::readData(), HttpStatus covered
http::StatusLine::parse(). The first calls the second, so remove the
second.
Signed-off-by: Miklos Vajna <vmiklos@collabora.com>
Change-Id: I163819ca470b766a7bc4694a9c6cfe4919e17963
And remove the httpheader one, which is not useful, since it uses Poco
for the actual parsing, it did not find anything. (If we switch away
from Poco there in the future, it's easy enough to restore it.)
Also fix some problems found by the fuzzer.
Signed-off-by: Miklos Vajna <vmiklos@collabora.com>
Change-Id: I254247c46ecc78c9c3e75aac4f10c441b0e10fb3
And fix an unhandled std::length_error it found.
Signed-off-by: Miklos Vajna <vmiklos@collabora.com>
Change-Id: I571cdd71caeda84820f2c64088966936637ce2bf
Run the actual fuzzer like this:
./httpheader_fuzzer -max_len=16384 fuzzer/httpheader-data/
Signed-off-by: Miklos Vajna <vmiklos@collabora.com>
Change-Id: I91afe44a632826cc15bd1c338bcc5234582e9674
The next version of Collabora Online will be version 2021.
Development of version 2021 will be carried on master branch
for the time being.
The corresponding core branch is distro/collabora/co-2021
Signed-off-by: Andras Timar <andras.timar@collabora.com>
Change-Id: I9c97aaac711c9e1f1e48ed25066b169ea7e26e84
A broken loolwsd may be lurking around, which will
then get used during the next `make` invocation to
cleanup the jails. If it fails, it will break the
build altogether. This is not ideal. In this case,
we delete the loolwsd binary so that we force
building it anew. And in any case it was useless
for make, if it failed to do the only thing that
make needs it for: cleaning up the jails.
The new loolwsd will subsequently get used to
clean any left over jails before running the
tests, so that should be fine.
Change-Id: I76c16b5fc7c6f08308c9fb2e619228f8e0266b74
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
This implements HTTP/1.1 per RFC 7230, partially.
Unit-tests are provided with documentation on usage.
This is desgined to serve as the http implementation
throughout loolwsd, for both synchronous and
asynchronous requests.
Change-Id: Iaf1b8c5fcb8cec032445e27c9f70d2fb807aa4dc
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
This reverts commit f6bf6f49ed. I did not
consider that this makes it harder to do a local developer setup with
reverse proxy + make run; also that if you don't read the doc to set up
a reverse proxy, then the default now serves http content and advertises
https, which is not helping sysadmins.
This needs more thinking how to improve one scenario without hurting
others.
Signed-off-by: Miklos Vajna <vmiklos@collabora.com>
Change-Id: Ic789faa2dc5bb19a79d651dc81d4eaaf0b48607a
Move the connect function into the NetUtil
translation unit to aid using it for the
upcoming async socket logic.
The NetUtil should also come in handy for
the miscellaneous network helpers we have.
Change-Id: I2ee0c6e3e1769fd87572d7407d3b4979b59ffe6a
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
The intention is to have defaults which are close to how people
typically use Online in production.
However, keep using ssl for 'make run', so that the https environment in
the browser is unchanged.
Signed-off-by: Miklos Vajna <vmiklos@collabora.com>
Change-Id: I7fd725a83b0e9ca1012f2c0e0c3bf038e5fa0059
In some cases we cannot do a fast bind-mount of the files we want
in our jail since we don't have cap_sys_admin for loolmount inside
eg. docker.
Thus we need to fallback to hard-linking, however various security
systems namespace parts of our tree, such that link() fails with
EXDEV even across the (apparently) same file-system.
As such we need to assemble a copy of what we want to hard-link
close to our jails. However, this needs to be owned by root / the
system to avoid having writable files shared between jails. Hence
we need cap_chown in addition to cap_fowner, to get ownership right
and then hard-link.
Change-Id: Iba0ef46ddbc1c03f3dc7177bc1ec1755624135db
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
This moves the test log macros into a new home,
test/testlog.hpp, to avoid cycling dependencies.
Change-Id: Iacb80e813a64ff830fa18f63ec4de2535ee702b7
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
For large transfers eg. image previews, particularly with SSL's
protocol limit of 16k byte blocks, we see lots of inefficiency
repeatedly copying a 20Mb image and shuffling it down a
std::vector as we write data out.
Change-Id: I620568cad2e6f41684c35289b0ee77cf7f59c077
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
This adds the infrastructure to be able to pass the info which elements
like the statusbar / ruler / sidebar are supposed to be shown or hidden
on startup of the editor.
Change-Id: I188264dec6961074444934ff5fd7088e23b170d4
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/103169
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Mert Tumer <mert.tumer@collabora.com>
We now gracefully fallback to copying when/if systemplate
is readonly.
The bulk of the change is to support proper cleanup in
both cases.
First, we had to move as much of the jail bootstrapping
into the loolwsd-systemplate-setup script, so systemplate
will be as complete as possible before it is locked down.
Next, we needed to update the jail with graceful fallback
to linking/copying upon failure. For that, the jail setup
logic in Kit.cpp has been reworked to support not just
update failures, but also more comprehensive mounting
failures as well.
Finally, jail cleanup now is seamless. To support proper
cleanup when we had mounting enabled but had to fallback,
we mark jails that aren't mounted so we can 'rm -rf' the
contents safely and without fear or causing undue damage
(as unlikely as that is, technically we wouldn't want to
rm systemplate files, if mounting read-only had failed).
There are a few minor refactorings of JailUtil to make
it cleaner and more robust.
Change-Id: Iac34869cb84f45acf64fbbc46d46898367b496d2
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/101260
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Andras Timar <andras.timar@collabora.com>
libfuzzer only produces fuzzer binaries, not a loolwsd binary, so don't
expect it in the libfuzzer case.
Change-Id: Ib818667031665aa60a447ddd5edd3a09bca76e18
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/98910
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Miklos Vajna <vmiklos@collabora.com>
Since this file is only created manually,
it shouldn't be removed automatically.
Change-Id: I8d26b7bfc7f7cd899318b2edd3e5ef9bd462cc99
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/98184
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
A small re-factoring to help planned re-plumbing of the iOS app.
Change-Id: I21f09216a7c5adf965179765a75f5a0d521cd7f3
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/97771
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tor Lillqvist <tml@collabora.com>
Leaving behind jails with bind-mount
entries makes build-workspace removal
complicated, and jenkins builds start failing.
The cleanup stage is integrated in Makefiles
and should be transparent.
In the event that manual cleanup is necessary,
'loolwsd --cleanup' can be invoked.
Change-Id: Ia4b99b0c66e56dfa2d50e79b0ba98f714cf32886
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/97470
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
loolmount now works and supports mounting and
unmounting, plus numerous improvements,
refactoring, logging, etc.. When enabled,
binding improves the jail setup time by anywhere
from 2x to orders of magnitude (in docker, f.e.).
A new config entry mount_jail_tree controls
whether mounting is used or the old method of
linking/copying of jail contents. It is set to
true by default and falls back to linking/copying.
A test mount is done when the setting is enabled,
and if mounting fails, it's disabled to avoid noise.
Temporarily disabled for unit-tests until we can
cleanup lingering mounts after Jenkins aborts our
build job. In a future patch we will have mount/jail
cleanup as part of make.
The network/system files in /etc that need frequent
refreshing are now updated in systemplate to make
their most recent version available in the jails.
These files can change during the course of loolwsd
lifetime, and are unlikely to be updated in
systemplate after installation at all. We link to
them in the systemplate/etc directory, and if that
fails, we copy them before forking each kit
instance to have the latest.
This reworks the approach used to bind-mount the
jails and the templates such that the total is
now down to only three mounts: systemplate, lo, tmp.
As now systemplate and lotemplate are shared, they
must be mounted as readonly, this means that user/
must now be moved into tmp/user/ which is writable.
The mount-points must be recursive, because we mount
lo/ within the mount-point of systemplate (which is
the root of the jail). But because we (re)bind
recursively, and because both systemplate and
lotemplate are mounted for each jails, we need to
make them unbindable, so they wouldn't multiply the
mount-points for each jails (an explosive growth!)
Contrarywise, we don't want the mount-points to
be shared, because we don't expect to add/remove
mounts after a jail is created.
The random temp directory is now created and set
correctly, plus many logging and other improvements.
Change-Id: Iae3fda5e876cf47d2cae6669a87b5b826a8748df
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92829
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Seems to not cause any serious regressions in the iOS app or in "make
run", but of course I am not able to run a comprehensive check of all
functionality.
Change-Id: I44a0e8d60bdbc0a885db88475961575c5e95ce88
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/93037
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tor Lillqvist <tml@collabora.com>
Share it with various other places requiring similar data.
Change-Id: I873f56798f5a34dcf7440456bd649b68f6d3df98
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/94069
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Also adds ServiceRoot handling for clipboard.
Change-Id: I7bc6591130fcc7d693e59ab8561fb9e99f4e93d5
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/93578
Tested-by: Michael Meeks <michael.meeks@collabora.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tor Lillqvist
Michael Stahl
Andras Timar
Pranam Lashkari
Ashod Nakashian
Michael Meeks
Henry Castro
Rash419
Aron Budea
Jan Holesovsky
Mert Tumer
Miklos Vajna
Yunusemre Şentürk
Jan Dageförde