Commit graph

68 commits

Author SHA1 Message Date
Alex Bennée d6db2a1cdf docker: add debian-buster-arm64-cross
We can't build QEMU with this but we can use this image to build newer
arm64 testcases which need more up to date tools.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-02-08 17:32:35 +00:00
Alex Bennée d406015b6a tests/docker: remove SID_AGE test hack
Now we are using "named" snapshots of debian-sid we can rely on the
existing checksum mechanism for detecting changes.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-01-14 14:52:30 +00:00
Stefan Weil e50a61219f tests: Fix typos in comments and help message (found by codespell)
Fix also a grammar issue.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20180713054755.23323-1-sw@weilnetz.de>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2018-10-26 17:17:32 +02:00
Philippe Mathieu-Daudé b28ef6b9eb docker: Clean dangling tarball files
When a container fails, it leaves a dangling tarball which name is
based on a timestamp. Further uses of make won't clean those files,
neither calling the 'docker-clean' target.

Use the .DELETE_ON_ERROR built-in target to let make remove those
temporary tarballs in case of failure.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20180818030337.22271-1-f4bug@amsat.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
2018-09-10 11:50:06 +08:00
Fam Zheng 73fb4f1de3 tests: Allow overriding archive path with SRC_ARCHIVE
In VM based tests, the source archive is created in host, we don't have
to run archive-source.sh again, as it complicates the Makefile and
scripts.

Signed-off-by: Fam Zheng <famz@redhat.com>
Message-Id: <20180712012829.20231-4-famz@redhat.com>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
2018-08-15 10:12:35 +08:00
Alex Bennée d19f5fc041 docker: report hint when docker.py check fails
When a check fails we currently just report why we failed. This is not
totally helpful to people who want to boot-strap a new image. Report a
hint as to why it failed.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Suggested-by: Fam Zheng <famz@redhat.com>
2018-07-24 11:45:25 +01:00
Alex Bennée 2796346069 docker: drop QEMU_TARGET check, fallback in EXECUTABLE not set
The addition of QEMU_TARGET was intended to ensure we fall back to
checking for the existence of an image if the build system was not
currently configured to build it. However this breaks the direct use
of the rule for building custom binfmt_misc images. We already check
for EXECUTABLE so let us just use that as a proxy for deciding if we
are just going to check the image exits.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-07-24 11:45:25 +01:00
Alex Bennée ff1a390296 docker: add expansion for docker-test-FOO to Makefile.include
This allows us to run a particular test on all docker images. For
example:

  make docker-test-unit

Will run the unit tests on every supported image. At the same time
rename docker-test to docker-all-tests to be clearer.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-07-24 11:45:25 +01:00
Alex Bennée 77b08f73c8 docker: Makefile.include don't include partial images
Rename DOCKER_INTERMEDIATE_IMAGES to DOCKER_PARTIAL_IMAGES and add the
incomplete cross compiler images that can build tests but can't build
QEMU itself. We also add debian, debian-bootstrap and the tricode
images to the list.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-07-24 11:45:25 +01:00
Alex Bennée 19c9a18f45 docker: add linux-user powered cross builder for QEMU
We can't use cross compilers in the current Debian stable and Debian
sid is sketchy as hell. So for powerpc fall back to dog-fooding our
own linux-user to do the build.

As we can only build the base image with a suitably configured
source tree we fall back to checking for its existence when we can't
build it from scratch. However this does mean you don't have to keep
a static powerpc-linux-user in your active configuration just to
update the cross build image.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-07-05 16:57:39 +01:00
Alex Bennée dc338fdb07 docker: add special rule for deboostrapped images
We might as well have a custom rule for this. For one thing the
dependencies are different. As the primary dependency for
docker-image-% could never be docker-image-debian-bootstrap we can
drop that test in the main rule as well.

Missing EXECUTABLE, DEB_ARCH and DEB_TYPE are treated as hard faults
now. We also error out if the EXECUTABLE file isn't there. We should
really do this with a dependency on any source rules but currently
subdir-FOO-linux-user isn't enough on a clean build.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-07-05 15:59:41 +01:00
Philippe Mathieu-Daudé 272e551b43 docker: Do not run tests in 'intermediate' images
We can still build the DOCKER_INTERMEDIATE_IMAGES images,
but they won't appear in 'make test*@$IMAGE'.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2018-07-05 15:59:41 +01:00
Alex Bennée 5499fa9e44 tests/docker/Makefile.include: only force SID to NOCACHE if old
Now we can check the age of a docker image we can be a little more
intelligent about re-building Sid images and only force NOCACHE if
it is "old".

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2018-06-20 22:13:39 +01:00
Alex Bennée cff5477330 tests/Makefile: call sub-makes with SKIP_DOCKER_BUILD=1
As we now ensure all the images we are going to use are built in the
top level make file lets not over complicate things by running the
full script again. We do run the check script just in case someone
deletes the docker image while we are running.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2018-06-20 22:13:39 +01:00
Alex Bennée 08f56d8c9c docker: move debian-powerpc-cross to sid based build
The original Jessie based cross builder hasn't worked for a while. The
state of the libraries is still perilous for cross-building QEMU but
we can use it for building TCG tests.

The debian-apt-fake.sh script can also be dropped as it is no longer
used.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée e100a96755 tests/tcg: enable building for RISCV64
As before, using Debian SID compilers.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée 33b2c4b50f tests/tcg: enable building for mips64
As before, using Debian SID compilers.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée cc6c7365b8 tests/tcg: enable building for sparc64
As before, using Debian SID compilers.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée a5ec3e36dc tests/tcg: enable building for sh4
As before, using Debian SID compilers. While the compiler can be
coerced into generating big-endian code it seems the linker can't deal
with it so we only enable the building for little endian SH4.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée 21d71c8c41 tests/tcg: enable building for m68k
As before, using Debian SID compilers.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée 3ae8a1000b tests/tcg: enable building for HPPA
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée 409c1c9cbd tests/tcg: enable building for Alpha
We can't use our normal Debian based compilers as Alpha isn't an
officially supported architecture. However it is available as a port
and fortunately cross compilers for all these targets are included in
Debian Sid, the perpetual rolling/unstable/testing version of Debian.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée 6bf77518dd tests/docker/Makefile.include: fix mipsel-cross dependancy
This got broken in commit 4319db7 but generally only shows up when you
try and do massive parallel builds on fresh machines.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Alex Bennée e27cae268f docker: Makefile.include introduce DOCKER_SCRIPT
Define this in one place to make it easy to re-use.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-06-20 20:22:34 +01:00
Philippe Mathieu-Daudé 36dc5fedab docker: add debian/tricore image
TriCore binutils is built from Bastian Koppelmann repository.

Note: There is no TriCore compiler in this image (only assembler/linker).

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
[AJB: base of Debian9, add to Makefile.include]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2018-06-05 16:25:43 +01:00
Alex Bennée 5609c5127d tests/docker/Makefile.include: handle empty TARGET_LIST
If the user doesn't specify a TARGET_LIST they get the current
configuration but with spaces and hilarity ensues. This adds some make
magic to turn the TARGET_LIST back into a comma separated list.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

Message-Id: <20180521103504.26432-1-alex.bennee@linaro.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
2018-06-04 14:39:18 +08:00
Philippe Mathieu-Daudé bcaf457786 docker: do not display deprecated images in 'make docker' help
the 'debian' base image is deprecated since 3e11974988

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2018-06-01 14:55:03 +01:00
Philippe Mathieu-Daudé 1a06f5c910 docker: sort images list displayed by 'make docker'
we can now directly see different version sort consecutively.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2018-06-01 14:55:03 +01:00
Philippe Mathieu-Daudé 9d5e546af0 docker: Fix trivial typo
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20180508144358.13530-1-f4bug@amsat.org>
Reviewed-by: Laurent Vivier <laurent@vivier.eu>
Signed-off-by: Fam Zheng <famz@redhat.com>
2018-05-18 16:52:53 +08:00
Philippe Mathieu-Daudé 6423795efc docker: correctly escape $BACKEND in the help output
In Makefiles the $ must be escaped as $$ in shell uses.

Since 8a2390a4f47:

 $ make docker
     [...]
     NETWORK=1            Enable virtual network interface with default backend.
     NETWORK=ACKEND     Enable virtual network interface with ACKEND.

Once escaped:

 $ make docker
     [...]
     NETWORK=1            Enable virtual network interface with default backend.
     NETWORK=$BACKEND     Enable virtual network interface with $BACKEND.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Fam Zheng <famz@redhat.com>
Message-Id: <20171108024719.8389-1-f4bug@amsat.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
2017-11-08 10:59:42 +08:00
Greg Kurz 1dc1700a60 docker: fix out-of-tree 'make docker-test-build@debian-powerpc-cross'
Without this patch, it fails with:

IOError: [Errno 2] No such file or directory:
 'tests/docker/dockerfiles/debian-apt-fake.sh'

Signed-off-by: Greg Kurz <groug@kaod.org>
Message-Id: <150841324224.31292.4012464539983692364.stgit@bahia.lan>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
2017-10-20 14:28:50 +08:00
Paolo Bonzini bc739b8e7d docker: allow running from srcdir != builddir build
The new script uses "git submodule", which is picky about being invoked
from the top of the git checkout.  Invoke the script from $(SRC_PATH)
to avoid git's wrath.

Fixes: b7f404201e
Cc: Fam Zheng <famz@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

Message-Id: <1508331989-142364-1-git-send-email-pbonzini@redhat.com>
Reviewed-by: Daniel P. Berrange <berrange@redhat.com>
Signed-off-by: Fam Zheng <famz@redhat.com>
2017-10-20 14:28:50 +08:00
Peter Xu 08c3f6d57f docker: cleanup temp directory after test
There are temp directories named "docker-src.*" after doing docker
tests.  I don't see much point in keeping that (it only contains the
qemu.tar which is exactly current tree, and the copied "run" file).
Let's remove it after test finished.

CC: "Alex Bennée" <alex.bennee@linaro.org>
CC: Fam Zheng <famz@redhat.com>
CC: "Philippe Mathieu-Daudé" <f4bug@amsat.org>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20171017071247.32355-1-peterx@redhat.com>
Reviewed-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Fam Zheng <famz@redhat.com>
2017-10-20 14:28:50 +08:00
Fam Zheng 4b2c6bc563 docker: Don't allocate tty unless DEBUG=1
The existence of tty in the container seems to urge gcc into colorizing
the errors, but the escape chars will clutter the report once turned
into email replies on patchew. Move -t to debug mode.

Reported-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Fam Zheng <famz@redhat.com>
Message-Id: <20171013011954.9975-1-famz@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
2017-10-20 14:28:50 +08:00
Fam Zheng 36ac78e65a docker: Don't mount ccache db if NOUSER=1
With NOUSER=1 the container runs code as root, which may create
privileged files that will not be be accssible next time. Skip ccache
dir mount in this case.

Signed-off-by: Fam Zheng <famz@redhat.com>
Message-Id: <20170925075458.18047-1-famz@redhat.com>
Acked-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
2017-09-29 11:14:15 +08:00
Fam Zheng b7f404201e docker: Use archive-source.py
Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
2017-09-22 14:51:43 +08:00
Fam Zheng 4470749186 docker: Use unconfined security profile
Some by default blocked syscalls are required to run tests for example
userfaultfd.

Reviewed-by: Kashyap Chamarthy <kchamart@redhat.com>
Signed-off-by: Fam Zheng <famz@redhat.com>
Message-Id: <20170905025614.579-4-famz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Based-on: 20170905021201.25684-1-famz@redhat.com
2017-09-22 10:20:34 +08:00
Alex Bennée 1fddbf7c5e docker: ensure NOUSER for travis images
While adding the current user is a useful default behaviour for
creating new images it is not appropriate for Travis which already has
a default user.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20170725133425.436-2-alex.bennee@linaro.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
2017-09-22 10:20:34 +08:00
Paolo Bonzini 7437866bfc docker: fix creation of archives
The pixman submodule does not exist anymore, and its removal broke
docker-based tests.  Fix it.

Cc: Fam Zheng <famz@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2017-09-19 16:20:49 +02:00
Philippe Mathieu-Daudé 3e11974988 docker: warn users to use newer debian8/debian9 base image
to stay backward incompatible.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-07-18 10:58:36 +01:00
Philippe Mathieu-Daudé 982e758af7 docker: add MXE (M cross environment) base image for MinGW-w64
see http://mxe.cc/

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-07-18 10:58:36 +01:00
Philippe Mathieu-Daudé fd7b284d79 docker: add debian/mips64el image
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-07-18 10:58:36 +01:00
Philippe Mathieu-Daudé 4319db7b13 docker: add debian/mips[eb] images
change image mips little -> big endian

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-07-18 10:58:35 +01:00
Alex Bennée 16dcece4d5 docker: add debian/powerpc based on Jessie
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
[AJB: split from merged patch]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-07-18 10:58:35 +01:00
Philippe Mathieu-Daudé 15288046e0 docker: add debian/amd64 based on Stretch
By itself this doesn't add much to our coverage. However later patches
will extend this image to include more bleeding edge libraries which
are not yet widely available in distros.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
[AJB: extend commit msg]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-07-18 10:54:30 +01:00
Philippe Mathieu-Daudé 2b1c10bdcf docker: add debian/ppc64el based on Stretch
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-07-18 10:54:30 +01:00
Philippe Mathieu-Daudé 31e205d1bf docker: add debian/armel based on Stretch
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-07-18 10:54:30 +01:00
Philippe Mathieu-Daudé 7d2bcf2363 docker: debian/s390x no more in unstable, now available in Stretch
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-07-18 10:54:30 +01:00
Philippe Mathieu-Daudé 24044d4825 docker: rename debian stable -> 9 (Stretch)
We'll also want to support some older Debian combinations for
architectures that didn't make the Debian 9 cut.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
[AJB: extend commit msg]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-07-18 10:54:30 +01:00
Daniel P. Berrange 8a2390a4f4 docker: Don't enable networking as a side-effect of DEBUG=1
When trying to debug problems with tests it is natural to set
DEBUG=1 when starting the docker environment. Unfortunately
this has a side-effect of enabling an eth0 network interface
in the container, which changes the operating environment of
the test suite. IOW tests with fail may suddenly start
working again if DEBUG=1 is set, due to changed network setup.

Add a separate NETWORK variable to allow enablement of
networking separately from DEBUG=1. This can be used in two
ways. To enable the default docker network backend

  make docker-test-build@fedora NETWORK=1

while to enable a specific network backend, eg join the network
associated with the container 'wibble':

  make docker-test-build@fedora NETWORK=container:wibble

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Message-Id: <20170713144352.2212-1-berrange@redhat.com>
[Drop the superfluous second $(subst ...). - Fam]
Signed-off-by: Fam Zheng <famz@redhat.com>
2017-07-17 11:34:20 +08:00