If we have all default values of imgSize, wid, old-wid then we
should simply not serialize these out to save space and improve
readability.
Unfortunately, this means that we need to catch the mutation of
the TileCombined's content in two places; now just one. Building
the queue of rendered tiles to send back is now moved and
wrapped into a TileCombinedBuilder class - this also lets us
stop serialize taking a random vectors of TileDescs. However
the DocumentBroker::handleTileCombinedRequest method needs to
set the force-keyframe oldWid=0 flag on individual tiles and
then update the TileCombined.
Ideally we would have a nice visitor API and drop the non-const
getTiles() method that returns a reference to our internal vector.
Update tests, to not have redundant attributes, and add more tests
to ensure redundant attributes are removed.
Change-Id: Id76d3ae14d459d73bbae8112d840dea27f66588b
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
This stops a huge amount of parsing, re-parsing and conversion
from strings and back, making our few N^2 loops over the tiles
much more efficient.
Retain queue tests with the same strings via using a helper
to convert back to something expected.
Replace TileDesc::getId() with isPreview().
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
Change-Id: If40fc45f1fb474d37371e4b949da5fdfc594fdc8
Parsing a string to a TileDesc and then comparing is inefficient
enough, without doing that in an N^2 loop for the SenderQueue.
Still N^2 - and of dubious value in the world of deltas; but in
almost all cases a simple string + integer compare.
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
Change-Id: Ibe8230679e27b98cfa95567fea700e2f7d5ac09c
make -C cypress_test check-desktop
asserts seen in cypress_test/cypress/wsd_logs/coolwsd_output.log of:
coolwsd: wsd/DocumentBroker.cpp:3134: void DocumentBroker::sendTileCombine(const TileCombined&): Assertion `!newTileCombined.hasDuplicates()' failed.
If we check for, and don't reuse, an old request with a different
NormalizedViewId then we could end up with multiple requests with
different NormalizedViewIds that end up in the same final tilecombine.
similarly there was no check for different modes ending up in the
same tilecombine.
just split out the logic we have to see if two tiles have the same
properties that appear as a shared set of properties for tilecombine
and use that in the two relevant places.
Signed-off-by: Caolán McNamara <caolan.mcnamara@collabora.com>
Change-Id: Ieb2ee0e85f124dd57c6b050e5b669dd808cf6bbf
perf reported 1% of time in collaborative multi user test
on 2023-09-14 was spent in TileCombined::parse.
generic-ize and reuse the TileDesc::parse approach for
TileCombined::parse to avoid need for std::unordered_map
Signed-off-by: Caolán McNamara <caolan.mcnamara@collabora.com>
Change-Id: Iadfc2001e298d8f4d46200c8488f0eb4cd8734c2
apparently since:
commit b0a7532b08
Date: Sat Nov 14 19:43:02 2020 +0300
Turn off broadcast on presentation preview tiles
so drop this field from TileDesc and protocol.txt
Signed-off-by: Caolán McNamara <caolan.mcnamara@collabora.com>
Change-Id: Ib4b1eca0d30911e13c245551cb3e3261afc99dd2
getTokenString(const StringVector& tokens...) loops over each
token and calls
getTokenString(const std::string&...) with token.getParam() on each
token. We already have a loop over each token, so we can merge those
loops here.
perf reports TileDesc::parse taking 3.68% of the time in a hour
of an interactive writer session with multiple participants
Though looking further, broadcast seems unused by anything now.
To be followed up on that topic.
Signed-off-by: Caolán McNamara <caolan.mcnamara@collabora.com>
Change-Id: I244637043c58d90562001ec58fac494da54e55a8
perf reported 6.58% of time in collaborative multi user test
on 2023-08-24 was spent in TileDesc::parse and much of that
in std::unordered_map.
There are only 12 arguments in the map we care about here so
we can just used a sorted array, look by name on write, and
read by index.
Signed-off-by: Caolán McNamara <caolan.mcnamara@collabora.com>
Change-Id: Iadfc2001e298d8f4d46200c8488f0eb4cd8734c2
Needs a run-time check to avoid breaking concurrency assumption
during compression.
Change-Id: Icc959693e19b3506497eb1aa198477445085aeb8
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
Squashed from feature/deltas-expanded.
TileCache changes:
+ add montonic sequence (wid) numbers to TileData
+ account for sizes of TileData with multiple blobs
+ simplify saving and notifying of tiles
Sends updates (via appendChanges) based on the sequence the
right mix of keyframes and/or deltas required as a single
message, and parse and apply those on the JS side.
We continue to use PNG for slide previews and dialogs,
but remove PngCache - used by document tiles only.
Annotates delta: properly as a binary package for the websocket.
Distinguishes between deltas and keyframes we get from
the Kit based on an initial un-compressed prefix
character which we then discard.
kit can be forced to render a keyframe by oldWid=0
Track invalidity on tiles themselves - to keep the keyframe around.
We need to be able to track that a tile is invalid, and so subscribe
to the updated version as/when it is ready - but we also want to
store the keyframe underneath any deltas.
force rendering of a keyframe for an empty slot in the TileCache.
force tile sequence to be zero for combinedtiles - so the client can
always request standalone tiles with explicit combinedtiles, or tile
requests.
move Blob to Common.hpp
use zero size for un-changed tiles.
remove obsolete render-id, and color deltas in debug mode.
cleanup unit tests for non-png tile results.
Change-Id: I987f84ac4e58004758a243c233b19a6f0d60f8c2
Signed-off-by: Michael Meeks <michael.meeks@collabora.com>
In an attempt to reduce the size of Util.{c,h}pp
which has grown to contain all sorts of unrelated
helpers, we move StringVector helpers into
the StringVector.{c,h}pp files.
This makes the code better organized.
Change-Id: I152f341606807ae66253415b951bc9f89b09df57
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
... without copying the token.
And use it in TileDesc::parse(), which is known to be a hot path.
Signed-off-by: Miklos Vajna <vmiklos@collabora.com>
Change-Id: I0dcf2eb26c93254cdc6a1c11f9708daf213a825d
And use it in TileDesc::parse(), which is known to be a hot path.
Signed-off-by: Miklos Vajna <vmiklos@collabora.com>
Change-Id: I20375d7a1c31f61662446979e4d6799fd45b49d3
size_t in C and in C++ are not necessarily the same
type. The C++ size_t is in the std namespace. Since
we do include many C headers, and indeed some C++
runtime headers do define size_t for backwards
compatibility, it's easy to mix and match the two
types.
Also, 'using std::size_t;' isn't a great practice,
so removed.
This is not exhaustive, just some low-hanging cases.
Change-Id: I85a36b6fd1acd204274b1869de9bcb94c8b3cf13
Signed-off-by: Ashod Nakashian <ashod.nakashian@collabora.co.uk>
They are especially efficient for small lookups.
Change-Id: Ia005f40127cf222debe185515fc45cd92b8ae752
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/100413
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Tested-by: Jenkins
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Stale tiles were still being counted, unhelpfully. Avoid doing lots
of ::now() calls, and yet detect this.
Change-Id: Ib1e4b2f1968c1994849bb23ec54e28f6706230ee
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/100347
Tested-by: Jenkins
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
* Excised TileCacheDesc to improve performance and simplify code.
* clang-tidy suggestions and auto-rewrite fixes.
* Const-correctness.
* Inlined and improved a couple of trivial functions (that are called
often).
* Reduced some logs from INF to DBG as they are only meaningful to devs.
Change-Id: I1c4eb8c63da49aa061afbf3eb68cae23d4d5e7f3
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/98661
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Tested-by: Jenkins
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
The tokenizer(s) are more generic than the protocol
logic, and are used from contexts that don't involve
the protocol as such.
Change-Id: Ie8c256bf11a91e466bff794021f41603c9596a7f
More readable and typically more efficient.
Change-Id: I9bd5bfc91f4ac255bb8ae0987708fb8b56b398f8
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/95285
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Jenkins
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
LibreOffice core uses that, too, and we support an even more
restricted set of compilers.
Change-Id: I0d0e2c8608e323eb5ef0f35ee8c46d02ab49a745
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/92467
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Tor Lillqvist <tml@collabora.com>
This is meant to reduce lots of small allocations and instead have
pointers into the single string for the various tokens instead.
This has a few requirements, though:
1) It's no longer OK to modify the tokens, changing their length would
invalidate the start/length of other tokens. Rework
DocumentBroker::load() to avoid such mutation.
2) The iterators no longer expose zero-terminated strings, so
Poco::cat() doesn't work anymore: add an own cat() instead and use that
in e.g. ChildSession. The own cat() has the benefit that it won't read
past the end of the array if the begin index is out of bounds to add
more safety.
(This nicely works towards killing Poco usage in general.)
3) If zero-terminated strings for all individual tokens is needed, a
copy has to be made, as done in spawnProcess().
(For all of these requirements, the build fails if there are problems.)
Change-Id: Iea40e4400e630b2d669f5c72aea85cb40edf9a2c
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/89711
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
The bulk of this commit just changes std::vector<std::string> to
StringVector when we deal with tokens from a websocket message.
The less boring part of it is the new StringVector class, which is a
wrapper around std::vector<std::string>, and provides the same API,
except that operator[] returns a string, not a string&, and this allows
returning an empty string in case that prevents reading past the end of
the underlying array.
This means in case client code forgets to check size() before invoking
operator[], we don't crash. (See the ~3 previous commits which fixed
such crashes.)
Later the ctor could be changed to take a single underlying string to
avoid lots of tiny allocations, that's not yet done in this commit.
Change-Id: I8a6082143a8ac0b65824f574b32104d7889c184f
Reviewed-on: https://gerrit.libreoffice.org/c/online/+/89687
Tested-by: Jenkins CollaboraOffice <jenkinscollaboraoffice@gmail.com>
Reviewed-by: Miklos Vajna <vmiklos@collabora.com>
removed use of Poco::StringTokenizer from the wsd directory using LOOLProtocol::tokenize and std::vecor<std::string>
Change-Id: Ic50b4d4d71d4ffd005aacf6aef0ed2bfde66d40d
Reviewed-on: https://gerrit.libreoffice.org/82569
Reviewed-by: Jan Holesovsky <kendy@collabora.com>
Tested-by: Jan Holesovsky <kendy@collabora.com>
We currently combine only horizontally, but ctrl-right arrow in
calc can throw us to the other side of the sheet, creating a very
large area to re-render.
Change-Id: I7125ab815e3de1296b3af32632626005eeee0ec9
Moves appending tokens into the serializer and
avoids making extra copies of itself.
Change-Id: I62d374e69d9c4a55643ea20cb5f8c2b9c75c88c5
Reviewed-on: https://gerrit.libreoffice.org/71022
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
Remove redundant _id member from TileCombined, add constructor
from TileDesc, and use it to shrink the code.
Change-Id: Idc0ded63166ed350ab81b07e191b7a60d4407cd4
Saves lots of string construction and storage, simplifies lots of code.
Store the more exotic font-caching bits etc. in a separate store: we
should also pre-render these really and share them.
Change-Id: Icaeff8fd72f52d7215c06a687b1e39cfb7631503
Changes protocol to use 'wid' instead of 'hash' everywhere. Wire-ids
are monotonically increasing integers that can be mapped to hash
values for all of the hash values and tiles we cache internally.
Change-Id: Ibcb25817bab0f453e93d52a6f99d3ff65059e47d
The server tells the client the hash of each tile it sends (calculated
from the contents of the tile, not its PNG encoding). When the client
asks for a tile to be refreshed, it tells the server what the hash of
the existing tile is. If the server notices that the tile contents
hasn't actually changed, it doesn't PNG encode it and doesn't send it
to the client.
The intent is that this will reduce load on the server and also avoid
unnecessary tile traffic.
Change-Id: Ia06ca68655ea984ed4319f24f4470afda322eccf
Previously tilecombine had its own version, which is
nonesensical, since it's not really a tile.
Now it passes the version to the tiles when
parsing and serializes version per-tile.
Change-Id: I5db8d94880431e3d2a40b6787c6fe51a05771305
Reviewed-on: https://gerrit.libreoffice.org/32633
Reviewed-by: Ashod Nakashian <ashnakash@gmail.com>
Tested-by: Ashod Nakashian <ashnakash@gmail.com>
Michael Meeks
Caolán McNamara
Ashod Nakashian
Szymon Kłos
Andras Timar
Miklos Vajna
Tor Lillqvist
Pranam Lashkari
mert
Miklos Vajna
Jan Holesovsky