Commit graph

80563 commits

Author SHA1 Message Date
Peter Maydell 76dd0f8497 qapi/migration.json: Fix indentation
Commits 6a9ad15420 and 9004db48c0 added some
new text to qapi/migration.json which doesn't fit the stricter
indentation requirements imposed by the rST documentation generator.
Reindent those lines to the new standard.

Reviewed-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20200810195019.25427-2-peter.maydell@linaro.org>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2020-09-07 17:12:39 +02:00
Markus Armbruster dcdc07a97c qapi: Make section headings start a new doc comment block
Our current QAPI doc-comment markup allows section headers (introduced
with a leading '=' or '==') anywhere in a free-form documentation
comment.  This works for Texinfo because the generator simply prints a
Texinfo section command at that point in the output stream.  For rST
generation, since we're assembling a tree of docutils nodes, this is
awkward because a new section implies starting a new section node at
the top level of the tree and generating text into there.

Make section headers start a new free-form documentation block, so the
future rST document generator doesn't have to look at every line in
free-form blocks and handle headings in odd places.

This change makes no difference to the generated Texinfo.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20200320091805.5585-3-armbru@redhat.com>
2020-09-07 16:35:16 +02:00
Markus Armbruster d98884b75d qapi: Reject section markup in definition documentation
Section markup in definition documentation makes no sense and can
produce invalid Texinfo.  Reject.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20200320091805.5585-2-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-09-07 16:35:16 +02:00
Peter Maydell 9d5589bb3f Add ppoll_time64() and pselect6_time64()
Some fixes for for elfload, fcntl, termbits and btrfs
 -----BEGIN PGP SIGNATURE-----
 
 iQJGBAABCAAwFiEEzS913cjjpNwuT1Fz8ww4vT8vvjwFAl9V+qQSHGxhdXJlbnRA
 dml2aWVyLmV1AAoJEPMMOL0/L748fSwQAJJFV7a9zFDGA2wmiu4mLffFN4FUwkE6
 Ov4/pzE8O6By5AnaBN8JiuoIYLKD5vdIjz/vuMwxxVUbwqBeqLGir+LmrGMhGNWO
 Ghu/SyDhGiot+WU3rAvn3cr2ceZgn+eZtDHdYO3pFcthe18JP1wJIYtYISL7I4Tm
 ulsp0JiL0+UqBQeJHwA+ttmL1kNDqnvwJqDLLX2cZ4QaMhmg/c825CXmvaP7gXUO
 FOhBBdSrCl8sPr/A4Lem8lvCJpZSPt3cZvmpRm/7H9Oe2FRMhaizIDuPhrJoWqui
 NH1UwOig0Sy9vBTAzpNCNjKOwXYFTePOB3QQcmaQSqWjSSReUHdcD/Fx2Dw9BYot
 9mv8QWZukyYOaO6Py4LmtFOc2uupDIEqkOnD0mVlGYjMlCd2LI6AG3eI1VDZNtiL
 mc4Mv6nZfn+/c7jXMg9ZrhR5Hm7lbBEtSSLs7hczgMoE2o9RyD4mILLdUKm+VZYn
 DfnwlrLtDAQvP3ANYemdSuMODQnFzcbz348m6MQr/6p98+9q9yxehE1SoDIBLSmk
 3xfSiTTv1nolTYdBG6/5h3pI7r3b76iHICKJVU0XTCd+XFs9o+MfRrtQ95f7T5hJ
 BRkyDHi/HDD8XFYbmCd3Zdy+/SlyL4TYsNLflSP9wwcFOX4odL7L9RN4T2bpcS1a
 t4qgCgCJxrNH
 =edT0
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/vivier2/tags/linux-user-for-5.2-pull-request' into staging

Add ppoll_time64() and pselect6_time64()
Some fixes for for elfload, fcntl, termbits and btrfs

# gpg: Signature made Mon 07 Sep 2020 10:17:24 BST
# gpg:                using RSA key CD2F75DDC8E3A4DC2E4F5173F30C38BD3F2FBE3C
# gpg:                issuer "laurent@vivier.eu"
# gpg: Good signature from "Laurent Vivier <lvivier@redhat.com>" [full]
# gpg:                 aka "Laurent Vivier <laurent@vivier.eu>" [full]
# gpg:                 aka "Laurent Vivier (Red Hat) <lvivier@redhat.com>" [full]
# Primary key fingerprint: CD2F 75DD C8E3 A4DC 2E4F  5173 F30C 38BD 3F2F BE3C

* remotes/vivier2/tags/linux-user-for-5.2-pull-request:
  linux-user: Protect btrfs ioctl target definitions
  linux-user: fix ppc/termbits.h
  linux-user: Map signal numbers in fcntl
  linux-user: Correctly start brk after executable
  linux-user: Add support for ppoll_time64() and pselect6_time64()

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-09-07 15:31:04 +01:00
Peter Maydell 62f9256052 vga: fixes for cirrus and virtio-gpu.
-----BEGIN PGP SIGNATURE-----
 Version: GnuPG v2.0.22 (GNU/Linux)
 
 iQIcBAABCgAGBQJfUiRsAAoJEEy22O7T6HE4Tv4QAIhkJ8uyt4Z6OZYZclAs86Ny
 3z96+WCsuzCmI+qP8JAnLnegSxdqJgKWpBOQ6I/BdST6hsLJ5+56q+SYkOt27ry9
 CJF7l+xZgYuutl8dF4GXR6M8sDjWlKJtMwjFTZvo9/RQ9jm3DHsfFz0zi87oOyhr
 3XJM36aR/OMXSouxaPGhrw8tmi+AU0ditOa95NKK8qu872v4fHn2s/L0GKi61Qh9
 YoE3N141gFFmkPuWjalSFERc6ozQEf0DzOLK3efTSMHmjBRnzI8SmKUoXBd77+tj
 enLvRQW0vrGMFziV2sxCs4jJXvvY1QbW+7IOvry6hnRr7sIU+GsRcrBxWlKgCHrR
 1Gn2qY+9ALJTBbiGSWXiCfhoAqjZnv4MYWS4bZTuih3SlWgk8An+vIanM5YkVZg0
 1JMmEuSwfSFhet12fmR8SOStIGwXHACYTkVr6ITGvThaoAEaBG76wd80kv920ruW
 F892hq7Al6rm7rZ/hH5FZwXHOqEn8bomOZtl6DN2q5w5waZfzk6FnAVS86b79/18
 LvR1pf8xYAhUcckkJWKvk2kDghOjVsNun0o6FY7L3+8MBDHRGcSgJaZ2feAkH2S9
 ywUrYSItn0lvmVTvl9WWApRp8uR42f8R2Luf9kSWCzetBn1XAfGHjOiT20WO7jBB
 tx1oz5kE7sO5FwKXZJE0
 =inDV
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/kraxel/tags/vga-20200904-pull-request' into staging

vga: fixes for cirrus and virtio-gpu.

# gpg: Signature made Fri 04 Sep 2020 12:26:36 BST
# gpg:                using RSA key 4CB6D8EED3E87138
# gpg: Good signature from "Gerd Hoffmann (work) <kraxel@redhat.com>" [full]
# gpg:                 aka "Gerd Hoffmann <gerd@kraxel.org>" [full]
# gpg:                 aka "Gerd Hoffmann (private) <kraxel@gmail.com>" [full]
# Primary key fingerprint: A032 8CFF B93A 17A7 9901  FE7D 4CB6 D8EE D3E8 7138

* remotes/kraxel/tags/vga-20200904-pull-request:
  cirrus: handle wraparound in cirrus_invalidate_region
  virtio-gpu: fix unmap the already mapped items

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-09-07 13:27:20 +01:00
Richard Henderson 88e74b6122 target/microblaze: Collected fixes for env->iflags
There are several problems here that can result in soft lockup,
depending on exactly where an interrupt or exception is delivered:

Include BIMM_FLAG in IFLAGS_TB_MASK, since it needs to follow D_FLAG.
Ensure that iflags is 0 when entering an interrupt/exception handler.
Add mb_cpu_synchronize_from_tb to restore iflags from tb->flags.
The change to t_sync_flags is cosmetic, but makes the code clearer.

This fixes the reported regression in acceptance/replay_kernel.py.

Fixes: 683a247ed7 ("target/microblaze: Store "current" iflags in insn_start")
Tested-by: Thomas Huth <thuth@redhat.com>
Reported-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20200904190842.2282109-2-richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-09-07 12:34:17 +02:00
Paolo Bonzini cc9962d8ea tests/qtest: do not list ahci-test twice
This is caught by "meson test", which complains about two tests with the
same name.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20200904120342.11370-1-pbonzini@redhat.com>
Fixes: a2ce7dbd91 ("meson: convert tests/qtest to meson")
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-09-07 12:34:17 +02:00
Daniel P. Berrangé affcc09c2d gitlab: expand test coverage for crypto builds
Most jobs test the latest nettle library. This adds explicit coverage
for latest gcrypt using Fedora, and old gcrypt and nettle using
CentOS-7. The latter does a minimal tools-only build, as we only need to
validate that the crypto code builds and unit tests pass. Finally a job
disabling both nettle and gcrypt is provided to validate that gnutls
still works.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20200901133050.381844-3-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-09-07 12:34:17 +02:00
Daniel P. Berrangé 46859d9398 crypto: fix build with gcrypt enabled
If nettle is disabled and gcrypt enabled, the compiler and linker flags
needed for gcrypt are not passed.

Gnutls was also not added as a dependancy when gcrypt is enabled.

Attempting to add the library dependencies at the same time as the
source dependencies is error prone, as there are alot of different
rules for picking which sources to use, and some of the source files
use code level conditionals intead. It is thus clearer to add the
library dependencies separately.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20200901133050.381844-2-berrange@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-09-07 12:34:17 +02:00
Thomas Huth 6bcb5fc0f7 gitlab-ci: Add cross-compiling build tests
Now that we can use all our QEMU test containers in the gitlab-CI, we can
easily add some jobs that test cross-compilation for various architectures.
There is just only small ugliness: Since the shared runners on gitlab.com
are single-threaded, we have to split each compilation job into two parts
(--disable-user and --disable-system), and exclude some additional targets,
to avoid that the jobs are running too long and hitting the timeout of 1 h.

Message-Id: <20200823111757.72002-8-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-09-07 12:34:17 +02:00
Thomas Huth 8c2787629e stubs: Move qemu_fd_register stub to util/main-loop.c
The linker of MinGW sometimes runs into the following problem:

libqemuutil.a(util_main-loop.c.obj): In function `qemu_fd_register':
/builds/huth/qemu/build/../util/main-loop.c:331: multiple definition of
 `qemu_fd_register'
libqemuutil.a(stubs_fd-register.c.obj):/builds/huth/qemu/stubs/fd-register.c:5:
 first defined here
collect2: error: ld returned 1 exit status
/builds/huth/qemu/rules.mak:88: recipe for target 'tests/test-timed-average.exe'
 failed

qemu_fd_register() is defined in util/main-loop.c for WIN32, so let's simply
move the stub also there in the #else part of the corresponding #ifndef
to fix this problem.

Message-Id: <20200903054503.425435-1-thuth@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-09-07 12:34:17 +02:00
Thomas Huth b0476d6602 stubs: Move qemu_timer_notify_cb() and remove qemu_notify_event() stub
When cross-compiling with MinGW, there are sometimes some weird linker
errors like:

ibqemuutil.a(util_main-loop.c.obj): In function `qemu_notify_event':
/builds/huth/qemu/build/../util/main-loop.c:139: multiple definition of
 `qemu_notify_event'
libqemuutil.a(stubs_notify-event.c.obj):/builds/huth/qemu/stubs/notify-event.c:5:
 first defined here
collect2: error: ld returned 1 exit status
/builds/huth/qemu/rules.mak:88: recipe for target 'tests/test-timed-average.exe'
 failed

It seems like it works better when the qemu_timer_notify_cb() stub (which
calls qemu_notify_event()) is in a separate file - then we can also even
remove the qemu_notify_event() stub now.

This patch is based on ideas from the patch "stubs: Remove qemu_notify_event()"
by Philippe Mathieu-Daudé and the patch "cpu-timers, icount: new modules" from
Claudio Fontana.

Message-Id: <20200902102433.304737-1-thuth@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-09-07 12:34:17 +02:00
Thomas Huth e7a222aeb8 configure: Allow automatic WHPX detection
The whpx variable is currently initialized to "no" which causes the WHPX
check to skip the detection unless the user specified --enable-whpx.
Since the detection code should be able to figure it out correctly, let's
initialized the variable to "" on MinGW-builds for proper auto-detection
instead.

Message-Id: <20200804170055.2851-11-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Stefan Weil <sw@weilnetz.de>
Message-Id: <20200823111757.72002-7-thuth@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-09-07 12:34:17 +02:00
Thomas Huth d3dd34a1e5 dockerfiles/debian-win64-cross: Download WHPX MinGW headers
To compile-test the WHPX accelerator, we need to download these system
headers first (they are unfortunately not part of any released and
packaged MinGW toolchain yet).

Idea taken from another patch by Stefan Weil.

Message-Id: <20200804170055.2851-12-thuth@redhat.com>
Message-Id: <20200823111757.72002-6-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-09-07 12:34:17 +02:00
Thomas Huth ce4e510ac7 tests/Makefile: test-image-locking needs CONFIG_POSIX
test-image-locking.c uses the qemu_lock_fd_test() function which is
only available on POSIX-like systems.

Message-Id: <20200804170055.2851-4-thuth@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-Id: <20200823111757.72002-4-thuth@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-09-07 12:34:17 +02:00
Yonggang Luo 0fdc1f2f5e tests: Fixes building test-util-filemonitor.c on msys2/mingw
Fixes the following compiling error:
../tests/test-util-filemonitor.c: In function 'test_file_monitor_events':
../tests/test-util-filemonitor.c:620:17: error: too many arguments to function 'mkdir'
  620 |             if (mkdir(pathsrc, 0700) < 0) {
      |                 ^~~~~
In file included from C:/CI-Tools/msys64/mingw64/x86_64-w64-mingw32/include/unistd.h:10,
                 from C:/work/xemu/qemu/include/qemu/osdep.h:93,
                 from ../tests/test-util-filemonitor.c:21:
C:/CI-Tools/msys64/mingw64/x86_64-w64-mingw32/include/io.h:282:15: note: declared here
  282 |   int __cdecl mkdir (const char *) __MINGW_ATTRIB_DEPRECATED_MSVC2005;
      |               ^~~~~

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Message-Id: <20200905203425.1470-1-luoyonggang@gmail.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-09-07 12:34:17 +02:00
Yonggang Luo 85c93c57f1 tests: fixes test-vmstate.c compile error on msys2
../tests/test-vmstate.c: In function 'int_cmp':
../tests/test-vmstate.c:884:5: error: unknown type name 'uint'; did you mean 'uInt'?
  884 |     uint ua = GPOINTER_TO_UINT(a);
      |     ^~~~
      |     uInt
../tests/test-vmstate.c:885:5: error: unknown type name 'uint'; did you mean 'uInt'?
  885 |     uint ub = GPOINTER_TO_UINT(b);
      |     ^~~~
      |     uInt
make: *** [Makefile.ninja:5461:tests/test-vmstate.exe.p/test-vmstate.c.obj] 错误 1

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Message-Id: <20200905063813.1875-1-luoyonggang@gmail.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-09-07 12:34:17 +02:00
Max Reitz c682bf18c9 iotests: Let complete_and_wait() work with commit
complete_and_wait() and wait_ready() currently only work for mirror
jobs.  Let them work for active commit jobs, too.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
2020-09-07 12:31:31 +02:00
Max Reitz c365625b0a iotests: Test that qcow2's data-file is flushed
Flushing a qcow2 node must lead to the data-file node being flushed as
well.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
2020-09-07 12:31:31 +02:00
Max Reitz 0b877d09df block: Leave BDS.backing_{file,format} constant
Parts of the block layer treat BDS.backing_file as if it were whatever
the image header says (i.e., if it is a relative path, it is relative to
the overlay), other parts treat it like a cache for
bs->backing->bs->filename (relative paths are relative to the CWD).
Considering bs->backing->bs->filename exists, let us make it mean the
former.

Among other things, this now allows the user to specify a base when
using qemu-img to commit an image file in a directory that is not the
CWD (assuming, everything uses relative filenames).

Before this patch:

$ ./qemu-img create -f qcow2 foo/bot.qcow2 1M
$ ./qemu-img create -f qcow2 -b bot.qcow2 foo/mid.qcow2
$ ./qemu-img create -f qcow2 -b mid.qcow2 foo/top.qcow2
$ ./qemu-img commit -b mid.qcow2 foo/top.qcow2
qemu-img: Did not find 'mid.qcow2' in the backing chain of 'foo/top.qcow2'
$ ./qemu-img commit -b foo/mid.qcow2 foo/top.qcow2
qemu-img: Did not find 'foo/mid.qcow2' in the backing chain of 'foo/top.qcow2'
$ ./qemu-img commit -b $PWD/foo/mid.qcow2 foo/top.qcow2
qemu-img: Did not find '[...]/foo/mid.qcow2' in the backing chain of 'foo/top.qcow2'

After this patch:

$ ./qemu-img commit -b mid.qcow2 foo/top.qcow2
Image committed.
$ ./qemu-img commit -b foo/mid.qcow2 foo/top.qcow2
qemu-img: Did not find 'foo/mid.qcow2' in the backing chain of 'foo/top.qcow2'
$ ./qemu-img commit -b $PWD/foo/mid.qcow2 foo/top.qcow2
Image committed.

With this change, bdrv_find_backing_image() must look at whether the
user has overridden a BDS's backing file.  If so, it can no longer use
bs->backing_file, but must instead compare the given filename against
the backing node's filename directly.

Note that this changes the QAPI output for a node's backing_file.  We
had very inconsistent output there (sometimes what the image header
said, sometimes the actual filename of the backing image).  This
inconsistent output was effectively useless, so we have to decide one
way or the other.  Considering that bs->backing_file usually at runtime
contained the path to the image relative to qemu's CWD (or absolute),
this patch changes QAPI's backing_file to always report the
bs->backing->bs->filename from now on.  If you want to receive the image
header information, you have to refer to full-backing-filename.

This necessitates a change to iotest 228.  The interesting information
it really wanted is the image header, and it can get that now, but it
has to use full-backing-filename instead of backing_file.  Because of
this patch's changes to bs->backing_file's behavior, we also need some
reference output changes.

Along with the changes to bs->backing_file, stop updating
BDS.backing_format in bdrv_backing_attach() as well.  This way,
ImageInfo's backing-filename and backing-filename-format fields will
represent what the image header says and nothing else.

iotest 245 changes in behavior: With the backing node no longer
overriding the parent node's backing_file string, you can now omit the
@backing option when reopening a node with neither a default nor a
current backing file even if it used to have a backing node at some
point.

273 also changes: The base image is opened without a format layer, so
ImageInfo.backing-filename-format used to report "file" for the base
image's overlay after blockdev-snapshot.  However, the image header
never says "file" anywhere, so it now reports $IMGFMT.

Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-09-07 12:31:31 +02:00
Max Reitz 549ec0d978 block: Inline bdrv_co_block_status_from_*()
With bdrv_filter_bs(), we can easily handle this default filter behavior
in bdrv_co_block_status().

blkdebug wants to have an additional assertion, so it keeps its own
implementation, except bdrv_co_block_status_from_file() needs to be
inlined there.

Suggested-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
2020-09-07 12:31:31 +02:00
Max Reitz 05ea385afd blockdev: Fix active commit choice
We have to perform an active commit whenever the top node has a parent
that has taken the WRITE permission on it.

This means that block-commit's @backing-file parameter is no longer
allowed for such nodes, and that users will have to issue a
block-job-complete command.  Neither should pose a problem in practice,
because this case was basically just broken until now.

(Since this commit already touches block-commit's documentation, it also
moves up the chunk explaining general block-commit behavior that for
some reason was situated under @backing-file.)

Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-09-07 12:31:31 +02:00
Max Reitz f1a7f18f07 block: Drop backing_bs()
We want to make it explicit where bs->backing is used, and we have done
so.  The old role of backing_bs() is now effectively taken by
bdrv_cow_bs().

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
2020-09-07 12:31:31 +02:00
Max Reitz 4a2061e66d qemu-img: Use child access functions
This changes iotest 204's output, because blkdebug on top of a COW node
used to make qemu-img map disregard the rest of the backing chain (the
backing chain was broken by the filter).  With this patch, the
allocation in the base image is reported correctly.

Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-09-07 12:31:31 +02:00
Max Reitz ee2f94ca27 nbd: Use CAF when looking for dirty bitmap
When looking for a dirty bitmap to share, we should handle filters by
just including them in the search (so they do not break backing chains).

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
2020-09-07 12:31:31 +02:00
Max Reitz 9a71b9de3f commit: Deal with filters
This includes some permission limiting (for example, we only need to
take the RESIZE permission if the base is smaller than the top).

Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-09-07 12:31:31 +02:00
Max Reitz 2b088c60bb backup: Deal with filters
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
2020-09-07 12:31:31 +02:00
Max Reitz 3f072a7fb7 mirror: Deal with filters
This includes some permission limiting (for example, we only need to
take the RESIZE permission for active commits where the base is smaller
than the top).

base_overlay is introduced so we can query bdrv_is_allocated_above() on
it - we cannot do that with base itself, because a filter's block_status
is the same as its child node, so if there are filters on base,
bdrv_is_allocated_above() on base would return information including
base.

Use this opportunity to rename qmp_drive_mirror()'s "source" BDS to
"target_backing_bs", because that is what it really refers to.

Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-09-07 12:31:31 +02:00
Max Reitz c6f6d8462c block-copy: Use CAF to find sync=top base
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
2020-09-07 12:31:31 +02:00
Max Reitz 0a7585dbba block: Use child access functions for QAPI queries
query-block, query-named-block-nodes, and query-blockstats now return
any filtered child under "backing", not just bs->backing or COW
children.  This is so that filters do not interrupt the reported backing
chain.  This changes the output for iotest 184, as the throttled node
now appears as a backing child.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
2020-09-07 12:31:31 +02:00
Max Reitz 3f26191c73 block: Report data child for query-blockstats
It makes no sense to report the block stats of a purely metadata-storing
child in query-blockstats.  So if the primary child does not have any
data, try to find a unique data-storing child.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
2020-09-07 12:31:31 +02:00
Max Reitz 7cc734a9ea blockdev: Use CAF in external_snapshot_prepare()
This allows us to differentiate between filters and nodes with COW
backing files: Filters cannot be used as overlays at all (for this
function).

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
2020-09-07 12:31:31 +02:00
Max Reitz 07cd7b659a block/null: Implement bdrv_get_allocated_file_size
It is trivial, so we might as well do it.

Remove _filter_actual_image_size from iotest 184, so we get to see the
result in its reference output.

Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-09-07 12:31:31 +02:00
Max Reitz 081e465026 block: Improve get_allocated_file_size's default
There are two practical problems with bdrv_get_allocated_file_size()'s
default right now:
(1) For drivers with children, we should generally sum all their sizes
    instead of just passing the request through to bs->file.  The latter
    is good for filters, but not so much for format drivers.

(2) Filters need not have bs->file, so we should actually go to the
    filtered child instead of hard-coding bs->file.

Fix this by splitting the default implementation into three branches:
(1) For filter drivers: Return the size of the filtered child
(2) For protocol drivers: Return -ENOTSUP, because the default
    implementation cannot make a guess
(3) For other drivers: Sum all data-bearing children's sizes

Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-09-07 12:31:31 +02:00
Max Reitz f706a92f24 block: Use CAFs for debug breakpoints
When looking for a blkdebug node (which implements debug breakpoints),
use bdrv_primary_bs() to iterate through the graph, because that is
where a blkdebug node would be.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
2020-09-07 12:31:31 +02:00
Max Reitz c8af87573f block/snapshot: Fix fallback
If the top node's driver does not provide snapshot functionality and we
want to fall back to a node down the chain, we need to snapshot all
non-COW children.  For simplicity's sake, just do not fall back if there
is more than one such child.  Furthermore, we really only can fall back
to bs->file and bs->backing, because bdrv_snapshot_goto() has to modify
the child link (notably, set it to NULL).

Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
2020-09-07 12:31:31 +02:00
Max Reitz c4db2e25df block: Use CAF in bdrv_co_rw_vmstate()
If a node whose driver does not provide VM state functions has a
metadata child, the VM state should probably go there; if it is a
filter, the VM state should probably go there.  It follows that we
should generally go down to the primary child.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
2020-09-07 12:31:31 +02:00
Max Reitz 52f72d6fb6 block: Use CAFs in bdrv_refresh_filename()
bdrv_refresh_filename() and the kind of related bdrv_dirname() should
look to the primary child when they wish to copy the underlying file's
filename.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
2020-09-07 12:31:31 +02:00
Max Reitz 66b129ac5e block: Iterate over children in refresh_limits
Instead of looking at just bs->file and bs->backing, we should look at
all children that could end up receiving forwarded requests.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
2020-09-07 12:31:31 +02:00
Max Reitz fb787f02a6 vmdk: Drop vmdk_co_flush()
Before HEAD^, we needed this because bdrv_co_flush() by itself would
only flush bs->file.  With HEAD^, bdrv_co_flush() will flush all
children on which a WRITE or WRITE_UNCHANGED permission has been taken.
Thus, vmdk no longer needs to do it itself.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
2020-09-07 12:31:31 +02:00
Max Reitz 883833e29c block: Flush all children in generic code
If the driver does not support .bdrv_co_flush() so bdrv_co_flush()
itself has to flush the children of the given node, it should not flush
just bs->file->bs, but in fact all children that might have been written
to (judging from the permissions taken on them).

This is a bug fix for qcow2 images with an external data file, as they
so far did not flush that data_file node.

In any case, the BLKDBG_EVENT() should be emitted on the primary child,
because that is where a blkdebug node would be if there is any.

Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
2020-09-07 12:31:31 +02:00
Max Reitz 1d42f48c3a block: Re-evaluate backing file handling in reopen
Reopening a node's backing child needs a bit of special handling because
the "backing" child has different defaults than all other children
(among other things).  Adding filter support here is a bit more
difficult than just using the child access functions.  In fact, we often
have to directly use bs->backing because these functions are about the
"backing" child (which may or may not be the COW backing file).

Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-09-07 12:31:31 +02:00
Max Reitz 23b93525a2 block: Use bdrv_cow_child() in bdrv_co_truncate()
The condition modified here is not about potentially filtered children,
but only about COW sources (i.e. traditional backing files).

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
2020-09-07 12:31:31 +02:00
Max Reitz dcf3f9b268 block: Use CAFs when working with backing chains
Use child access functions when iterating through backing chains so
filters do not break the chain.

In addition, bdrv_find_overlay() will now always return the actual
overlay; that is, it will never return a filter node but only one with a
COW backing file (there may be filter nodes between that node and @bs).

Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-09-07 12:31:31 +02:00
Max Reitz 67acfd2188 stream: Deal with filters
Because of the (not so recent anymore) changes that make the stream job
independent of the base node and instead track the node above it, we
have to split that "bottom" node into two cases: The bottom COW node,
and the node directly above the base node (which may be an R/W filter
or the bottom COW node).

Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-09-07 12:31:31 +02:00
Max Reitz cb8503159a block: Use CAFs in block status functions
Use the child access functions in the block status inquiry functions as
appropriate.

Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-09-07 12:31:31 +02:00
Max Reitz 93393e698c block: Use bdrv_filter_(bs|child) where obvious
Places that use patterns like

    if (bs->drv->is_filter && bs->file) {
        ... something about bs->file->bs ...
    }

should be

    BlockDriverState *filtered = bdrv_filter_bs(bs);
    if (filtered) {
        ... something about @filtered ...
    }

instead.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
2020-09-07 12:31:31 +02:00
Max Reitz 4935e8be22 copy-on-read: Support compressed writes
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
2020-09-07 12:31:31 +02:00
Max Reitz e7e754aec3 throttle: Support compressed writes
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
2020-09-07 12:31:31 +02:00
Max Reitz ae23f78646 block: Add bdrv_supports_compressed_writes()
Filters cannot compress data themselves but they have to implement
.bdrv_co_pwritev_compressed() still (or they cannot forward compressed
writes).  Therefore, checking whether
bs->drv->bdrv_co_pwritev_compressed is non-NULL is not sufficient to
know whether the node can actually handle compressed writes.  This
function looks down the filter chain to see whether there is a
non-filter that can actually convert the compressed writes into
compressed data (and thus normal writes).

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
2020-09-07 12:31:31 +02:00