We only run one task in each thread when run() is called, then complete
the remainder of tasks in the main thread while the other threads wait.
https: //github.com/CollaboraOnline/online/issues/7374
Signed-off-by: Caolán McNamara <caolan.mcnamara@collabora.com>
Change-Id: Id4fc0a620d98b0bb55310b495eeff7411a4544b5
std::make_shared and std::make_unique
are superior to explict smart pointer
construction. Where we have private
constructors, we can't use them.
Change-Id: I492122f58e958113c6e6d31445c6614ad98c89aa
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
but they don't go through the delta mechanism, so move the assert
to the delta path, the other path is ok wrt gt 256x256
Signed-off-by: Caolán McNamara <caolan.mcnamara@collabora.com>
Change-Id: I2ba4d8affb7645349540f3a5de31d1802f04c53d
We want to keep pre-loaded, and invalidated tiles around so that
we have something to render in lots of cases, particularly for
other parts, and slides. However - we need to know that it is
invalid so we can on-demand fetch something better.
It is very important that this invalid state doesn't race with
tile arrivals ie. in a banal way:
invalidate, req, invalidate, req, tile, invalidate, req, tile
it is unclear whether the latest tile is still invalid; similarly,
if we request(req) a tile and there is no change to it we get no
response.
To disambiguate this, use the monotonic wid to check we are in-sync.
Re-factor:
setup debug tile layer more pro-active.y
rename HasContent to NeedsFetch to be more explicit.
Remove:
previous _invalidCount that was perhaps a similar idea.
previous .loaded concept.
previous _pruneTiles concept - now it seems we can be
confident in cached tiles from other slides and sheets.
Change-Id: Ibc6d3b6050e756ace91a9bd53d3e4efe12b023cf
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
Added a debug option (triggerable using the about debug menu) that
allows you to dump tiles as they come in
Signed-off-by: Paris Oplopoios <paris.oplopoios@collabora.com>
Change-Id: If5939b816c1e3598e50c2db4971710524f855909
avoid an allocation and some pointer chasing
Signed-off-by: Noel Grandin <noel.grandin@collabora.co.uk>
Change-Id: I29711075d4bf840c2ef992341a6dadeeb87c6edd
The default would be std:🧵:hardware_concurrency() which can be
ridiculously high for the WASM case.
Signed-off-by: Tor Lillqvist <tml@collabora.com>
Change-Id: I77902a58b96248d7afe638a039886712516fe905
This avoids problems with watermarking cross-talking betwene view.
Change-Id: Ifecf098423451bf7de3827dfdf9fdc078a06c5c9
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
Needs a run-time check to avoid breaking concurrency assumption
during compression.
Change-Id: Icc959693e19b3506497eb1aa198477445085aeb8
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
Squashed from feature/deltas-expanded.
TileCache changes:
+ add montonic sequence (wid) numbers to TileData
+ account for sizes of TileData with multiple blobs
+ simplify saving and notifying of tiles
Sends updates (via appendChanges) based on the sequence the
right mix of keyframes and/or deltas required as a single
message, and parse and apply those on the JS side.
We continue to use PNG for slide previews and dialogs,
but remove PngCache - used by document tiles only.
Annotates delta: properly as a binary package for the websocket.
Distinguishes between deltas and keyframes we get from
the Kit based on an initial un-compressed prefix
character which we then discard.
kit can be forced to render a keyframe by oldWid=0
Track invalidity on tiles themselves - to keep the keyframe around.
We need to be able to track that a tile is invalid, and so subscribe
to the updated version as/when it is ready - but we also want to
store the keyframe underneath any deltas.
force rendering of a keyframe for an empty slot in the TileCache.
force tile sequence to be zero for combinedtiles - so the client can
always request standalone tiles with explicit combinedtiles, or tile
requests.
move Blob to Common.hpp
use zero size for un-changed tiles.
remove obsolete render-id, and color deltas in debug mode.
cleanup unit tests for non-png tile results.
Change-Id: I987f84ac4e58004758a243c233b19a6f0d60f8c2
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
We want to always generate a delta vs. the last state we have so we
move linearly forward into the future.
Change-Id: I730d1dfb125a19e2c48b8f84ad5563664d196ab0
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
In 'debug' log-level we expect a detailed, but
still readable output. Having one area with
disproportionately large number of logs reduces
the overall utility of the log output.
This patch reduces a number of redundant log
entries, including errors that are already
logged. It also reduces the level of some
others from 'information' to 'debug' and
from 'debug' to 'trace'.
The goal is to make 'debug' level as useful as
possible to read the progress and be able to
understand what was going on, such that one is
able to decide which area to dig deeper into.
Then, trace level could be used to get more
insight into that area, if necessary. For
example, when investigating a test failure,
one first enables 'debug' logs and reads through.
Once a section between two debug entries is
identified as being of interest, enabling 'trace'
level logs becomes more productive as it's
now possible to easily reach the first DBG
entry and read through until the second one.
It's unfortunate that we don't have per-area
control for enabling/disabling logs, so it
is common to see more and more 'debug' log
entries added all around, making logs
less and less readable.
It is also a limitation of the levels we have
that we really only have 3 usable levels:
one, two, many. That is, 'information' for
the most important events, 'debug' for
technical details needed to investigate issues,
and 'trace' for everything else. ('warning'
and 'error' aren't really 'levels'; they have
semantics that makes them special-cases.)
So we have to avoid degrading one into the
other, or have differences without distinction.
If any of these entries are needed to be
displayed more frequently, changing them
back to 'debug' or even 'information' should
be done. Though for me they seem special
cases that don't benefit most log readings.
Change-Id: Id2c6a9dc027483b81a066b0b4b50a298c5eff449
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
We don't want leaking tiles to take up valuable space on the device.
Not even just while a document is open.
Signed-off-by: Tor Lillqvist <tml@collabora.com>
Change-Id: I761c2bb2ab121ee76b5535e96bd5702d0fce6856
Normally (ideally), tiles (.bmp files) are removed as soon as the JS
has displayed them. But occasionally something goes wrong and they are
left behind. (For instance, it seems to happen if the user closes the
document immediately when it shows up.)
Do not leave them on disk until the app starts the next time.
Change-Id: I0c764280a69a16ad3b7b67c329832fd5331c2e1e
Signed-off-by: Tor Lillqvist <tml@collabora.com>
The worker threads can be running and have not yet responded
to the last _cond.wait() by the time we start compressing again
- at least under valgrind:
loolforkit-nocaps: ./common/RenderTiles.hpp:304: void ThreadPool::run(): Assertion `_working == 0' failed.
==240379== Process terminating with default action of signal 6 (SIGABRT): dumping core
==240379== at 0x5505322: raise (raise.c:50)
==240379== by 0x54EE863: abort (abort.c:79)
==240379== by 0x54EE748: __assert_fail_base.cold (assert.c:92)
==240379== by 0x54FD9D5: __assert_fail (assert.c:101)
==240379== by 0x5886BB: ThreadPool::run() (RenderTiles.hpp:304)
==240379== by 0x56DC02: RenderTiles::doRender(std::shared_ptr<lok::Document>, TileCombined&, PngCache&, ThreadPool&, bool, std::function<void (unsigned char*, int, int, unsigned long, unsigned long, int, int, LibreOfficeKitTileMode)> const&, std::function<void (char const*, unsigned long)> const&) (RenderTiles.hpp:711)
==240379== by 0x5A0104: Document::renderTiles(TileCombined&, bool) (Kit.cpp:762)
==240379== by 0x59CF1A: Document::renderCombinedTiles(StringVector const&) (Kit.cpp:719)
==240379== by 0x59AEC8: Document::drainQueue(std::chrono::time_point<std::chrono::_V2::steady_clock, std::chrono::duration<long, std::ratio<1l, 1000000000l> > > const&) (Kit.cpp:1570)
That means our queue starts processing work as we do the unlocked
queue push - causing some potential badness.
Change-Id: Ib0578dac009376c0676da73a8c1d8960304dc072
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
Remove the app's copy of the document after core has saved it and we
have stored the edited version at its real location.
Remove possible leftover tile bitmaps when the app starts.
Signed-off-by: Tor Lillqvist <tml@collabora.com>
Change-Id: Ibc41be38c2cfb689c532640d148116bc06a248ab
Add `dumpState` methods to `TileQueue`, `PngCache`, `ThreadPool` classes
and call them from Document::dumpState method within kit.Kit.cpp.
Because `dumpState` method of `_websocketHandler` is protected,
currently we can't dump its internals.
Also dump the other members of the `Document` class. Note that as
`_lastMemStatsTime` is never used, we don't dump it. It seems we could
simply delete it.
Signed-off-by: Gökhan Karabulut <gokhanettin@gmail.com>
Change-Id: Ia89b7fff41eaf475c7a09a644c0eb523b72cf97d
It caused a crash when opening some PDF files, for instance.
Change-Id: I85515e2e14ffac8928714d218cd2353df228ff4b
Signed-off-by: Tor Lillqvist <tml@collabora.com>
Also, makes the logging of units much less error prone.
The overloaded streaming operators are temporary as
they are provided in C++20. The ones here (though
incomplete) are fashioned after the C++20 specs.
Change-Id: Ieb499282ccb6e63fa939ba07bed3e5a4fbef1bd0
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
chrono::system_clock can go back in time.
For time interval measurements, where we don't
care about the local time, a monotonic clock
should be used.
This avoids the server uptime jumping around
with daylight saving (or indeed by regular
synchronization with an atomic clock), among
other cases.
Change-Id: I09f9b24c82d19439348a2e66cad9e9de7d755208
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
When we get a wid match, this helps WSD to cleanup its tile
subscriber list effectively.
Change-Id: I6517039fb3d8c9ad8f53aef549b8adbb79961ce1
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/100348
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Avoids the need for PNG encoding (takes significant amount of CPU
time) and Base64 encoding in the app process, transfer to JavaScript
(running in a WebKit process of its own), and corresponding decoding
(in the WebKit process). Instead simply pass the URL of each tile file
to the JavaScript. Remove each BMP file once it has been loaded.
Change-Id: I6e7b9450691679c64813979976c59f1763ec104c
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/98710
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
This is for the benefit of a next-gen iOS app (without FakeSockets and
much of the current Online plumbing).
This is not supposed to cause any functional changes in normal Online
even if code is organised a bit differently.
Change-Id: Ib09a84ff5d3ba858cf3f50553d76757966af7ad2
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/98655
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tor Lillqvist <tml@collabora.com>
It is good practice, and those are needed when it is used in another
context.
Change-Id: I7811cbc6e072ed88ef2f0061994e7a18e797fd0d
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/98588
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tor Lillqvist <tml@collabora.com>
"Disable for now - pushed in error" says the comment added in
e11794da25.
Change-Id: Ia2b72bfe20f8ff16d74d1966d511c74eab3e4417
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/98587
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tor Lillqvist <tml@collabora.com>
Michael Meeks
Caolán McNamara
Ashod Nakashian
Paris Oplopoios
Noel Grandin
Tor Lillqvist
Szymon Kłos
Henry Castro
Gökhan Karabulut
Andras Timar