Commit graph

7924 commits

Author SHA1 Message Date
Cédric Le Goater 4c5fc0c5fc tests/acceptance: Test powernv machines
Fetch the OpenPOWER images to boot the powernv8 and powernv9 machines
with a simple PCI layout.

Cc: Cleber Rosa <crosa@redhat.com>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Cc: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: Cédric Le Goater <clg@kaod.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210817093036.1288791-1-clg@kaod.org>
2021-09-27 19:21:37 +02:00
Cleber Rosa 3d2ec56550 tests/acceptance/ppc_prep_40p.py: clean up unused import
Just a removal of an unused imported symbol.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210924185506.2542588-16-crosa@redhat.com>
2021-09-27 19:06:47 +02:00
Cleber Rosa 089f25877f tests/acceptance/ppc_prep_40p.py: NetBSD 7.1.2 location update
The NetBSD-7.1.2-prep.iso is no longer available on the CDN, but it's
still available in the archive.

Let's update its location so that users without the file on cache can
still fetch it and run the test.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210924185506.2542588-15-crosa@redhat.com>
2021-09-27 19:06:47 +02:00
Cleber Rosa 6f1f86cfa7 tests/acceptance/boot_xen.py: removed unused import
Just a clean up for an unused import.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210924185506.2542588-13-crosa@redhat.com>
2021-09-27 19:06:47 +02:00
Cleber Rosa 0f981d8792 acceptance/tests/vnc.py: use explicit syntax for enabling passwords
This matches the command line on 82a17d1d67, where the "on" or "off"
should be explicitly given.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210924185506.2542588-9-crosa@redhat.com>
2021-09-27 19:06:47 +02:00
Cleber Rosa 181e1ab2ad Acceptance Tests: improve check-acceptance description
The "check-acceptance" make rule won't necessarily run *all* available
tests, because it employs a filter based on the currently configured
targets.  This change in the description of the rule makes that
behavior extra clear.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210924185506.2542588-3-crosa@redhat.com>
2021-09-27 19:06:47 +02:00
David Gibson c5f0a81650 qemu: Split machine_ppc.py acceptance tests
machine_ppc.py contains tests for 3 different ppc based machine types.  It
is listed in MAINTAINERS along with the PPC TCG cpu code.  That's not
really accurate though, since it's really more about testing those machines
than the CPUs.

Therefore, split it up into separate files for the separate machine types,
and list those along with their machine types in MAINTAINERS.

Suggested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210927044808.73391-2-david@gibson.dropbear.id.au>
2021-09-27 19:06:47 +02:00
Willian Rampazzo 94c714620b tests/Makefile: add AVOCADO_TESTS option to make check-acceptance
Add the possibility of running all the tests from a single file, or
multiple files, running a single test within a file or multiple tests
within multiple files using `make check-acceptance` and the
AVOCADO_TESTS environment variable.

Suggested-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210923161141.232208-4-willianr@redhat.com>
2021-09-27 19:06:47 +02:00
Willian Rampazzo 23022794de tests/Makefile: allow control over tags during check-acceptance
Although it is possible to run a specific test using the avocado
command-line, a user may want to use a specific tag while running the
``make check-acceptance`` during the development or debugging.

This allows using the AVOCADO_TAGS environment variable where the user
takes total control of which tests should run based on the tags defined.

This also makes the check-acceptance command flexible to restrict tests
based on tags while running on CI.

e.g.:

AVOCADO_TAGS="foo bar baz" make check-acceptance

Signed-off-by: Willian Rampazzo <willianr@redhat.com>
Tested-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210923161141.232208-2-willianr@redhat.com>
2021-09-27 19:06:47 +02:00
Willian Rampazzo e519df437a avocado_qemu: fix inheritance order on LinuxTest class
Class hierarchy on Python is defined from right to left. Although the
current code is not harmful, let's fix it to avoid problems in the future.

Signed-off-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210920204932.94132-7-willianr@redhat.com>
2021-09-27 19:06:42 +02:00
Willian Rampazzo 6c58af2e17 avocado_qemu: explicitly return None to avoid R1710
The linter is complaining the `pick_default_qemu_bin` is not explicitly
returning None. Fix it to explicitly return None and avoid R1710
inconsistent-return-statements.

Signed-off-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210920204932.94132-6-willianr@redhat.com>
2021-09-27 19:06:36 +02:00
Willian Rampazzo 22e82e0982 avocado_qemu: tweak ssh connect method
The current implementation will crash if the connection fails as the
`time` module is not imported. Fix the import problem. While here,
tweaks the connection to wait progressively when the connection fails.

Signed-off-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
[PMD: Reworded description]
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210920204932.94132-5-willianr@redhat.com>
2021-09-27 19:06:30 +02:00
Willian Rampazzo e410bac070 avocado_qemu: fix import module based on isort
Signed-off-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210920204932.94132-4-willianr@redhat.com>
2021-09-27 19:06:25 +02:00
Willian Rampazzo 20bf915418 avocado_qemu: standardize super() call following PEP3135
PEP3135 states when calling super(), there is no need to use arguments.
This changes the calls on avocado_qemu to standardize according to
PEP3135 and avoid warnings from linters.

Signed-off-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210920204932.94132-3-willianr@redhat.com>
2021-09-27 19:06:15 +02:00
Cleber Rosa 1f1fcf0c57 Acceptance Tests: add standard clean up at test tearDown()
The avocado.Test class, used as the basis of the avocado_qemu.Test
class, performs a clean of temporary directories up as part of its own
tearDown() implementation.

But the avocado_qemu.Test class is currently missing the same clean
up, as it implemented its own tearDown() method without resorting to
the upper class behavior.

This brings avocado_qemu.Test behavior in sync with the standard
avocado.Test behavior and prevents temporary directories from
cluttering the test results directory (unless instructed to do so with
Avocado's "--keep-tmp" option).

Reported-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
[willianr: respin to new Python super format]
Signed-off-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210920204932.94132-2-willianr@redhat.com>
2021-09-27 19:06:15 +02:00
Pavel Dovgalyuk 82184f4018 tests/acceptance: Linux boot test for record/replay
This patch adds a test for record/replay, which boots Linux
image from the disk and interacts with the network.
The idea and code of this test is borrowed from boot_linux.py
This test includes only x86_64 platform. Other platforms and
machines will be added later after testing and improving
record/replay to completely support them.

Each test consists of the following phases:
 - downloading the disk image
 - recording the execution
 - replaying the execution

Replay does not validates the output, but waits until QEMU
finishes the execution. This is reasonable, because
QEMU usually hangs when replay goes wrong.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <162737554047.1735673.13133593401566029378.stgit@pasha-ThinkPad-X280>
2021-09-27 18:55:41 +02:00
Pavel Dovgalyuk 7f7c382a4d tests/acceptance: add replay kernel test for alpha
This patch adds record/replay test which boots Linux
kernel on alpha platform. The test uses kernel binaries
taken from boot_linux_console test.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <162737553482.1735673.10021851966976933952.stgit@pasha-ThinkPad-X280>
2021-09-27 18:51:40 +02:00
Pavel Dovgalyuk ce9771f5a4 tests/acceptance: add replay kernel test for nios2
This patch adds record/replay test which boots Linux
kernel on nios2 platform. The test uses kernel binaries
taken from boot_linux_console test.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <162737552919.1735673.12493523185952280539.stgit@pasha-ThinkPad-X280>
2021-09-27 18:51:40 +02:00
Pavel Dovgalyuk 27f551135e tests/acceptance: add replay kernel test for openrisc
This patch adds record/replay test which boots Linux
kernel on openrisc platform. The test uses kernel binaries
taken from boot_linux_console test.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <162737552350.1735673.14603125561530143423.stgit@pasha-ThinkPad-X280>
2021-09-27 18:51:40 +02:00
Pavel Dovgalyuk f7ade77931 tests/acceptance: add replay kernel test for s390x
This patch adds record/replay test which boots Linux
kernel on s390x platform. The test uses kernel binaries
taken from boot_linux_console test.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Acked-by: Thomas Huth <thuth@redhat.com>
[PMD: Drop default '-smp 1' as suggested by Thomas]
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <162737551785.1735673.6775108576116333386.stgit@pasha-ThinkPad-X280>
2021-09-27 18:51:36 +02:00
Markus Armbruster f333681c6e tests/qapi-schema: Make test-qapi.py -u work when files are absent
test-qapi.py -u updates the expected files.  Since it fails when they
are absent, users have to create them manually before they can use
test-qapi.py to fill in the contents, say for a new test.  Silly.
Improve -u to create them.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210922125619.670673-3-armbru@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
2021-09-27 08:23:25 +02:00
Markus Armbruster 436911c2a1 tests/qapi-schema: Use Python OSError instead of outmoded IOError
https://docs.python.org/3.6/library/exceptions.html has

    Changed in version 3.3: EnvironmentError, IOError, WindowsError,
    socket.error, select.error and mmap.error have been merged into
    OSError, and the constructor may return a subclass.

and

    The following exceptions are kept for compatibility with previous
    versions; starting from Python 3.3, they are aliases of OSError.

    exception EnvironmentError

    exception IOError

    exception WindowsError

        Only available on Windows.

Switch to the preferred name.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210922125619.670673-2-armbru@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
[Details added to commit message]
2021-09-27 08:23:25 +02:00
Markus Armbruster f90ae4d104 test-clone-visitor: Correct an accidental rename
Commit b359f4b203 "tests: Rename UserDefNativeListUnion to
UserDefListUnion" renamed test_clone_native_list() to
test_clone_list_union().  The function has nothing to do with unions.
Rename it to test_clone_list().

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210917143134.412106-24-armbru@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-09-27 08:23:25 +02:00
Markus Armbruster 8b3b3a16df tests/qapi-schema: Rename flat-union-* test cases to union-*
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210917143134.412106-23-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com
2021-09-27 08:23:25 +02:00
Markus Armbruster 4e99f4b12c qapi: Drop simple unions
Simple unions predate flat unions.  Having both complicates the QAPI
schema language and the QAPI generator.  We haven't been using simple
unions in new code for a long time, because they are less flexible and
somewhat awkward on the wire.

The previous commits eliminated simple union from the tree.  Now drop
them from the QAPI schema language entirely, and update mentions of
"flat union" to just "union".

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210917143134.412106-22-armbru@redhat.com>
2021-09-27 08:23:25 +02:00
Markus Armbruster 76432d988b tests/qapi-schema: Purge simple unions from tests
Drop tests that are specifically about simple unions:

* SugaredUnion in doc-good: flat unions are covered by @Object.

* union-branch-case and union-clash-branches: branch naming for flat
  unions is enforced for the tag enum instead, which is covered by
  enum-member-case and enum-clash-member.

* union-empty: empty flat unions are covered by flat-union-empty.

Rewrite the remainder to use flat unions: args-union, bad-base,
flat-union-base-union, union-branch-invalid-dict, union-unknown.

Except drop union-optional-branch. because converting this one is not
worth the trouble; we don't explicitly check names beginning with '*'
in other places, either.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210917143134.412106-21-armbru@redhat.com>
2021-09-27 08:23:25 +02:00
Markus Armbruster bb5821dd81 tests/qapi-schema: Drop simple union __org.qemu_x-Union1
Replace simple union __org.qemu_x-Union1 with flat union
__org.qemu_x-Union2, except drop it from __org.qemu_x-command, because
there it's only used to pull it into QMP.  Now drop the unused
-Union1, and rename -Union2 to -Union.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210917143134.412106-20-armbru@redhat.com>
2021-09-27 08:23:25 +02:00
Markus Armbruster 7a22dc17ac test-clone-visitor: Wean off __org.qemu_x-Union1
test_clone_complex3() uses simple union __org.qemu_x-Union1 to cover
arrays.  Use UserDefOneList instead.  Unions are still covered by
test_clone_complex1().

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210917143134.412106-19-armbru@redhat.com>
2021-09-27 08:23:25 +02:00
Markus Armbruster 5450186337 tests/qapi-schema: Rewrite simple union TestIfUnion to be flat
Simple unions predate flat unions.  Having both complicates the QAPI
schema language and the QAPI generator.  We haven't been using simple
unions in new code for a long time, because they are less flexible and
somewhat awkward on the wire.

To prepare for their removal, rewrite TestIfUnion to be flat.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210917143134.412106-18-armbru@redhat.com>
2021-09-27 08:23:25 +02:00
Markus Armbruster def1d1f571 tests/qapi-schema: Simple union UserDefListUnion is now unused, drop
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210917143134.412106-17-armbru@redhat.com>
2021-09-27 08:23:25 +02:00
Markus Armbruster 1e65e16ca3 tests/qapi-schema: Wean off UserDefListUnion
Command boxed-union uses simple union UserDefListUnion to cover
unions.  Use UserDefFlatUnion instead.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210917143134.412106-16-armbru@redhat.com>
2021-09-27 08:23:25 +02:00
Markus Armbruster 16821fc85b test-clone-visitor: Wean off UserDefListUnion
test_clone_complex1() uses simple union UserDefListUnion to cover
unions.  Use UserDefFlatUnion instead.  Arrays are still covered by
test_clone_complex3().

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210917143134.412106-15-armbru@redhat.com>
2021-09-27 08:23:25 +02:00
Markus Armbruster 00e6832f41 test-qobject-output-visitor: Wean off UserDefListUnion
The test_visitor_out_list_union_FOO() use simple union
UserDefListUnion to cover lists of builtin types.  Rewrite as
test_visitor_out_list_struct(), using struct ArrayStruct and a lot
less code.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210917143134.412106-14-armbru@redhat.com>
2021-09-27 08:23:25 +02:00
Markus Armbruster e7a76fe25a test-qobject-input-visitor: Wean off UserDefListUnion
The test_visitor_in_list_union_FOO() use simple union UserDefListUnion
to cover lists of builtin types.  Rewrite as
test_visitor_in_list_struct(), using struct ArrayStruct and a lot less
code.

test_visitor_in_fail_union_list() uses UserDefListUnion to cover
"variant members don't match the discriminator value".  Cover that in
test_visitor_in_fail_union_flat() instead, and drop
test_visitor_in_fail_union_list().  Appropriating the former for this
purpose is okay, because it actually failed due to missing
discriminator, which is still covered by
test_visitor_in_fail_union_flat_no_discrim().

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210917143134.412106-13-armbru@redhat.com>
2021-09-27 08:23:25 +02:00
Markus Armbruster dffd8ff9e6 tests/qapi-schema: Prepare for simple union UserDefListUnion removal
Simple unions predate flat unions.  Having both complicates the QAPI
schema language and the QAPI generator.  We haven't been using simple
unions in new code for a long time, because they are less flexible and
somewhat awkward on the wire.

To prepare for their removal, simple union UserDefListUnion has to go.
It is used to cover arrays.  The next few commits will eliminate its
uses, and then it gets deleted.  As a first step, provide struct
ArrayStruct for the tests to be rewritten.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210917143134.412106-12-armbru@redhat.com>
2021-09-27 08:23:25 +02:00
Markus Armbruster 935a867c87 qapi: Convert simple union SocketAddressLegacy to flat one
Simple unions predate flat unions.  Having both complicates the QAPI
schema language and the QAPI generator.  We haven't been using simple
unions in new code for a long time, because they are less flexible and
somewhat awkward on the wire.

To prepare for their removal, convert simple union SocketAddressLegacy
to an equivalent flat one, with existing enum SocketAddressType
replacing implicit enum type SocketAddressLegacyKind.  Adds some
boilerplate to the schema, which is a bit ugly, but a lot easier to
maintain than the simple union feature.

Cc: "Daniel P. Berrangé" <berrange@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210917143134.412106-9-armbru@redhat.com>
2021-09-27 08:23:25 +02:00
Markus Armbruster 8ebc3120e1 qapi: Stop enforcing "type name should not end in 'Kind'
I'm about to convert simple unions to flat unions, then drop simple
union support.  The conversion involves making the implict enum types
explicit.  To reduce churn, I'd like to name them exactly like the
implicit types they replace.  However, these names are reserved for
the generator's use.  They won't be once simple unions are gone.  Stop
enforcing this naming rule now rather than then.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210917143134.412106-3-armbru@redhat.com>
2021-09-25 06:55:54 +02:00
Markus Armbruster 4cfd6537e4 qapi: Tidy up unusual line breaks
Break lines between members instead of within members.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20210917143134.412106-2-armbru@redhat.com>
2021-09-25 06:55:54 +02:00
Eric Blake 1899bf4737 qemu-img: Add -F shorthand to convert
Although we have long supported 'qemu-img convert -o
backing_file=foo,backing_fmt=bar', the fact that we have a shortcut -B
for backing_file but none for backing_fmt has made it more likely that
users accidentally run into:

qemu-img: warning: Deprecated use of backing file without explicit backing format

when using -B instead of -o.  For similarity with other qemu-img
commands, such as create and compare, add '-F $fmt' as the shorthand
for '-o backing_fmt=$fmt'.  Update iotest 122 for coverage of both
spellings.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210913131735.1948339-1-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2021-09-15 18:42:38 +02:00
Vladimir Sementsov-Ogievskiy ff812c5563 qcow2: handle_dependencies(): relax conflict detection
There is no conflict and no dependency if we have parallel writes to
different subclusters of one cluster when the cluster itself is already
allocated. So, relax extra dependency.

Measure performance:
First, prepare build/qemu-img-old and build/qemu-img-new images.

cd scripts/simplebench
./img_bench_templater.py

Paste the following to stdin of running script:

qemu_img=../../build/qemu-img-{old|new}
$qemu_img create -f qcow2 -o extended_l2=on /ssd/x.qcow2 1G
$qemu_img bench -c 100000 -d 8 [-s 2K|-s 2K -o 512|-s $((1024*2+512))] \
        -w -t none -n /ssd/x.qcow2

The result:

All results are in seconds

------------------  ---------  ---------
                    old        new
-s 2K               6.7 ± 15%  6.2 ± 12%
                                 -7%
-s 2K -o 512        13 ± 3%    11 ± 5%
                                 -16%
-s $((1024*2+512))  9.5 ± 4%   8.4
                                 -12%
------------------  ---------  ---------

So small writes are more independent now and that helps to keep deeper
io queue which improves performance.

271 iotest output becomes racy for three allocation in one cluster.
Second and third writes may finish in different order. Second and
third requests don't depend on each other any more. Still they both
depend on first request anyway. Filter out second and third write
offsets to cover both possible outputs.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20210824101517.59802-4-vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
[hreitz: s/ an / and /]
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2021-09-15 15:54:07 +02:00
Vladimir Sementsov-Ogievskiy a13de40a05 block: bdrv_inactivate_recurse(): check for permissions and fix crash
We must not inactivate child when parent has write permissions on
it.

Calling .bdrv_inactivate() doesn't help: actually only qcow2 has this
handler and it is used to flush caches, not for permission
manipulations.

So, let's simply check cumulative parent permissions before
inactivating the node.

This commit fixes a crash when we do migration during backup: prior to
the commit nothing prevents all nodes inactivation at migration finish
and following backup write to the target crashes on assertion
"assert(!(bs->open_flags & BDRV_O_INACTIVE));" in
bdrv_co_write_req_prepare().

After the commit, we rely on the fact that copy-before-write filter
keeps write permission on target node to be able to write to it. So
inactivation fails and migration fails as expected.

Corresponding test now passes, so, enable it.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20210911120027.8063-3-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2021-09-15 15:54:07 +02:00
Vladimir Sementsov-Ogievskiy 2f43482733 tests: add migrate-during-backup
Add a simple test which tries to run migration during backup.
bdrv_inactivate_all() should fail. But due to bug (see next commit with
fix) it doesn't, nodes are inactivated and continued backup crashes
on assertion "assert(!(bs->open_flags & BDRV_O_INACTIVE));" in
bdrv_co_write_req_prepare().

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20210911120027.8063-2-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2021-09-15 15:54:07 +02:00
Hanna Reitz 098d983ea5 iotests/297: Cover tests/
297 so far does not check the named tests, which reside in the tests/
directory (i.e. full path tests/qemu-iotests/tests).  Fix it.

Thanks to the previous two commits, all named tests pass its scrutiny,
so we do not have to add anything to SKIP_FILES.

Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20210902094017.32902-6-hreitz@redhat.com>
2021-09-15 15:54:07 +02:00
Hanna Reitz b90d7a18b6 mirror-top-perms: Fix AbnormalShutdown path
The AbnormalShutdown exception class is not in qemu.machine, but in
qemu.machine.machine.  (qemu.machine.AbnormalShutdown was enough for
Python to find it in order to run this test, but pylint complains about
it.)

Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20210902094017.32902-5-hreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2021-09-15 15:54:07 +02:00
Hanna Reitz d8c2e47dbe migrate-bitmaps-test: Fix pylint warnings
There are a couple of things pylint takes issue with:
- The "time" import is unused
- The import order (iotests should come last)
- get_bitmap_hash() doesn't use @self and so should be a function
- Semicolons at the end of some lines
- Parentheses after "if"
- Some lines are too long (80 characters instead of 79)
- inject_test_case()'s @name parameter shadows a top-level @name
  variable
- "lambda self: mc(self)" were equivalent to just "mc", but in
  inject_test_case(), it is not equivalent, so add a comment and disable
  the warning locally
- Always put two empty lines after a function
- f'exec: cat > /dev/null' does not need to be an f-string

Fix them.

Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20210902094017.32902-4-hreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2021-09-15 15:54:07 +02:00
Hanna Reitz e2ad17a62d migrate-bitmaps-postcopy-test: Fix pylint warnings
pylint complains that discards1_sha256 and all_discards_sha256 are first
set in non-__init__ methods.

These variables are not really class-variables anyway, so let them
instead be returned by start_postcopy(), thus silencing pylint.

Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20210902094017.32902-3-hreitz@redhat.com>
2021-09-15 15:54:07 +02:00
Hanna Reitz 26db7b23ce iotests/297: Drop 169 and 199 from the skip list
169 and 199 have been renamed and moved to tests/ (commit a44be0334be:
"iotests: rename and move 169 and 199 tests"), so we can drop them from
the skip list.

Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20210902094017.32902-2-hreitz@redhat.com>
2021-09-15 15:54:07 +02:00
Hanna Reitz cc16153f1f iotests: Fix use-{list,dict}-literal warnings
pylint proposes using `[]` instead of `list()` and `{}` instead of
`dict()`, because it is faster.  That seems simple enough, so heed its
advice.

Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20210824153540.177128-3-hreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2021-09-15 15:54:07 +02:00
Hanna Reitz 81dcb9ca1f iotests: Fix unspecified-encoding pylint warnings
As of recently, pylint complains when `open()` calls are missing an
`encoding=` specified.  Everything we have should be UTF-8 (and in fact,
everything should be UTF-8, period (exceptions apply)), so use that.

Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20210824153540.177128-2-hreitz@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
2021-09-15 15:54:07 +02:00
Shashi Mallela 0a93293eb2 tests/data/acpi/virt: Update IORT files for ITS
Updated expected IORT files applicable with latest GICv3
ITS changes.

Full diff of new file disassembly:

/*
 * Intel ACPI Component Architecture
 * AML/ASL+ Disassembler version 20180629 (64-bit version)
 * Copyright (c) 2000 - 2018 Intel Corporation
 *
 * Disassembly of tests/data/acpi/virt/IORT.pxb, Tue Jun 29 17:35:38 2021
 *
 * ACPI Data Table [IORT]
 *
 * Format: [HexOffset DecimalOffset ByteLength]  FieldName : FieldValue
 */

[000h 0000   4]                    Signature : "IORT"    [IO Remapping Table]
[004h 0004   4]                 Table Length : 0000007C
[008h 0008   1]                     Revision : 00
[009h 0009   1]                     Checksum : 07
[00Ah 0010   6]                       Oem ID : "BOCHS "
[010h 0016   8]                 Oem Table ID : "BXPC    "
[018h 0024   4]                 Oem Revision : 00000001
[01Ch 0028   4]              Asl Compiler ID : "BXPC"
[020h 0032   4]        Asl Compiler Revision : 00000001

[024h 0036   4]                   Node Count : 00000002
[028h 0040   4]                  Node Offset : 00000030
[02Ch 0044   4]                     Reserved : 00000000

[030h 0048   1]                         Type : 00
[031h 0049   2]                       Length : 0018
[033h 0051   1]                     Revision : 00
[034h 0052   4]                     Reserved : 00000000
[038h 0056   4]                Mapping Count : 00000000
[03Ch 0060   4]               Mapping Offset : 00000000

[040h 0064   4]                     ItsCount : 00000001
[044h 0068   4]                  Identifiers : 00000000

[048h 0072   1]                         Type : 02
[049h 0073   2]                       Length : 0034
[04Bh 0075   1]                     Revision : 00
[04Ch 0076   4]                     Reserved : 00000000
[050h 0080   4]                Mapping Count : 00000001
[054h 0084   4]               Mapping Offset : 00000020

[058h 0088   8]            Memory Properties : [IORT Memory Access Properties]
[058h 0088   4]              Cache Coherency : 00000001
[05Ch 0092   1]        Hints (decoded below) : 00
                                   Transient : 0
                              Write Allocate : 0
                               Read Allocate : 0
                                    Override : 0
[05Dh 0093   2]                     Reserved : 0000
[05Fh 0095   1] Memory Flags (decoded below) : 03
                                   Coherency : 1
                            Device Attribute : 1
[060h 0096   4]                ATS Attribute : 00000000
[064h 0100   4]           PCI Segment Number : 00000000
[068h 0104   1]            Memory Size Limit : 00
[069h 0105   3]                     Reserved : 000000

[068h 0104   4]                   Input base : 00000000
[06Ch 0108   4]                     ID Count : 0000FFFF
[070h 0112   4]                  Output Base : 00000000
[074h 0116   4]             Output Reference : 00000030
[078h 0120   4]        Flags (decoded below) : 00000000
                              Single Mapping : 0

Raw Table Data: Length 124 (0x7C)

    0000: 49 4F 52 54 7C 00 00 00 00 07 42 4F 43 48 53 20  // IORT|.....BOCHS
    0010: 42 58 50 43 20 20 20 20 01 00 00 00 42 58 50 43  // BXPC    ....BXPC
    0020: 01 00 00 00 02 00 00 00 30 00 00 00 00 00 00 00  // ........0.......
    0030: 00 18 00 00 00 00 00 00 00 00 00 00 00 00 00 00  // ................
    0040: 01 00 00 00 00 00 00 00 02 34 00 00 00 00 00 00  // .........4......
    0050: 01 00 00 00 20 00 00 00 01 00 00 00 00 00 00 03  // .... ...........
    0060: 00 00 00 00 00 00 00 00 00 00 00 00 FF FF 00 00  // ................
    0070: 00 00 00 00 30 00 00 00 00 00 00 00              // ....0.......

Signed-off-by: Shashi Mallela <shashi.mallela@linaro.org>
Acked-by: Igor Mammedov <imammedo@redhat.com>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-id: 20210910143951.92242-10-shashi.mallela@linaro.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-09-13 21:01:08 +01:00