We have some special RAM memory regions (managed by virtio-mem), whereby
the guest agreed to only use selected memory ranges. "unused" parts are
discarded so they won't consume memory - to logically unplug these memory
ranges. Before the VM is allowed to use such logically unplugged memory
again, coordination with the hypervisor is required.
This results in "sparse" mmaps/RAMBlocks/memory regions, whereby only
coordinated parts are valid to be used/accessed by the VM.
In most cases, we don't care about that - e.g., in KVM, we simply have a
single KVM memory slot. However, in case of vfio, registering the
whole region with the kernel results in all pages getting pinned, and
therefore an unexpected high memory consumption - discarding of RAM in
that context is broken.
Let's introduce a way to coordinate discarding/populating memory within a
RAM memory region with such special consumers of RAM memory regions: they
can register as listeners and get updates on memory getting discarded and
populated. Using this machinery, vfio will be able to map only the
currently populated parts, resulting in discarded parts not getting pinned
and not consuming memory.
A RamDiscardManager has to be set for a memory region before it is getting
mapped, and cannot change while the memory region is mapped.
Note: At some point, we might want to let RAMBlock users (esp. vfio used
for nvme://) consume this interface as well. We'll need RAMBlock notifier
calls when a RAMBlock is getting mapped/unmapped (via the corresponding
memory region), so we can properly register a listener there as well.
Reviewed-by: Pankaj Gupta <pankaj.gupta@cloud.ionos.com>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: "Michael S. Tsirkin" <mst@redhat.com>
Cc: Alex Williamson <alex.williamson@redhat.com>
Cc: Dr. David Alan Gilbert <dgilbert@redhat.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Pankaj Gupta <pankaj.gupta.linux@gmail.com>
Cc: Peter Xu <peterx@redhat.com>
Cc: Auger Eric <eric.auger@redhat.com>
Cc: Wei Yang <richard.weiyang@linux.alibaba.com>
Cc: teawater <teawaterz@linux.alibaba.com>
Cc: Marek Kedzierski <mkedzier@redhat.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
Message-Id: <20210413095531.25603-2-david@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
It is not safe to pretend that emulated NVDIMM supports
persistence while backend actually failed to enable it
and used non-persistent mapping as fall back.
Instead of falling-back, QEMU should be more strict and
error out with clear message that it's not supported.
So if user asks for persistence (pmem=on), they should
store backing file on NVDIMM.
Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210111203332.740815-1-imammedo@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Having properties registered conditionally makes QOM type
introspection difficult. Instead of skipping registration of the
"instanceid" property, always register the property but validate
its value against the instance id required by the class.
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Acked-by: Maciej S. Szmigiero <maciej.szmigiero@oracle.com>
Message-Id: <20201009200701.1830060-1-ehabkost@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
* Generalize XSAVE area offset so that it matches AMD processors on KVM
* Improvements for -display and deprecation of -no-quit
* Enable SMP configuration as a compound machine property ("-M smp.cpus=...")
* Haiku compilation fix
* Add icon on Darwin
-----BEGIN PGP SIGNATURE-----
iQFIBAABCAAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAmDkB7sUHHBib256aW5p
QHJlZGhhdC5jb20ACgkQv/vSX3jHroOISgf+Nn5BiXQRY52DK/2PoG330F6UeOcp
kWFAE4k4qEktDiCcd5xKekiUd7h+TiRS8bLeycmRtiSXvbzXioE2eCelui0SZDQl
zpIb8wV2WaxrD/zUYPV7r5n+VFAaTCm9lUEzzqnwaThBG/Oat45gnossZEIWv85g
KtQMsSh3pc+KpTjWbIA8V01ohzwFE2q7cA9CB/pDgR3h8M5p4K0ZdaPoAO2auhvu
2sbu9oBl1JwqpIhPme9JR6Je5fMCILBRlXTvPgJ/0iaGdxcNmZxoflO/TZVFB1pl
tUiCu0GB0yEasMO1E6+cP7ezhm15Lz3vKqjr/boV5Y9osfU36k9xkLTvAg==
=itIm
-----END PGP SIGNATURE-----
Merge remote-tracking branch 'remotes/bonzini-gitlab/tags/for-upstream' into staging
* More Meson test conversions and configure cleanups
* Generalize XSAVE area offset so that it matches AMD processors on KVM
* Improvements for -display and deprecation of -no-quit
* Enable SMP configuration as a compound machine property ("-M smp.cpus=...")
* Haiku compilation fix
* Add icon on Darwin
# gpg: Signature made Tue 06 Jul 2021 08:35:23 BST
# gpg: using RSA key F13338574B662389866C7682BFFBD25F78C7AE83
# gpg: issuer "pbonzini@redhat.com"
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [full]
# gpg: aka "Paolo Bonzini <pbonzini@redhat.com>" [full]
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4 E2F7 7E15 100C CD36 69B1
# Subkey fingerprint: F133 3857 4B66 2389 866C 7682 BFFB D25F 78C7 AE83
* remotes/bonzini-gitlab/tags/for-upstream: (40 commits)
config-host.mak: remove unused compiler-related lines
Set icon for QEMU binary on Mac OS
qemu-option: remove now-dead code
machine: add smp compound property
vl: switch -M parsing to keyval
keyval: introduce keyval_parse_into
keyval: introduce keyval_merge
qom: export more functions for use with non-UserCreatable objects
configure: convert compiler tests to meson, part 6
configure: convert compiler tests to meson, part 5
configure: convert compiler tests to meson, part 4
configure: convert compiler tests to meson, part 3
configure: convert compiler tests to meson, part 2
configure: convert compiler tests to meson, part 1
configure: convert HAVE_BROKEN_SIZE_MAX to meson
configure, meson: move CONFIG_IVSHMEM to meson
meson: store dependency('threads') in a variable
meson: sort existing compiler tests
configure, meson: convert libxml2 detection to meson
configure, meson: convert liburing detection to meson
...
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Most of the build is not done via Makefiles, therefore the toolchain
variables are mostly unused. They are still used by tests/tcg
and pc-bios/roms, but most of them are not needed there.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Before switching the build system over to Meson, an icon was
added to the QEMU binary on Mac OS. This patch adds back that
feature; it piggybacks on the existing scripts/entitlement.sh,
which already does in-place changes to the executable on Darwin.
Signed-off-by: John Arbuckle <programmingkidx@gmail.com>
Message-Id: <20210705195328.36442-1-programmingkidx@gmail.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
-M was the sole user of qemu_opts_set and qemu_opts_set_defaults,
remove them and the arguments that they used.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Make -smp syntactic sugar for a compound property "-machine
smp.{cores,threads,cpu,...}". machine_smp_parse is replaced by the
setter for the property.
numa-test will now cover the new syntax, while other tests
still use -smp.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Switch from QemuOpts to keyval. This enables the introduction
of non-scalar machine properties, and JSON syntax in the future.
For JSON syntax to be supported right now, we would have to
consider what would happen if string-based dictionaries (produced by
-M key=val) were to be merged with strongly-typed dictionaries
(produced by -M {'key': 123}).
The simplest way out is to never enter the situation, and only allow one
-M option when JSON syntax is in use. However, we want options such as
-smp to become syntactic sugar for -M, and this is a problem; as soon
as -smp becomes a shortcut for -M, QEMU would forbid using -M '{....}'
together with -smp. Therefore, allowing JSON syntax right now for -M
would be a forward-compatibility nightmare and it would be impossible
anyway to introduce -M incrementally in tools.
Instead, support for JSON syntax is delayed until after the main
options are converted to QOM compound properties. These include -boot,
-acpitable, -smbios, -m, -semihosting-config, -rtc and -fw_cfg. Once JSON
syntax is introduced, these options will _also_ be forbidden together
with -M '{...}'.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Allow parsing multiple keyval sequences into the same dictionary.
This will be used to simplify the parsing of the -M command line
option, which is currently a .merge_lists = true QemuOpts group.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
This patch introduces a function that merges two keyval-produced
(or keyval-like) QDicts. It can be used to emulate the behavior of
.merge_lists = true QemuOpts groups, merging -readconfig sections and
command-line options in a single QDict, and also to implement -set.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Machines and accelerators are not user-creatable but they are going
to share similar command-line parsing machinery. Export functions
that will be used with -machine and -accel in softmmu/vl.c.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
And remove them from the summary, since now their outcome is verbosely
included in the meson output.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
This is a duplicate of CONFIG_EVENTFD, handle it directly in meson.build.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
The next patches will add more compiler tests. Sort and group the
existing tests, keeping similar cc.has_* tests together and sorting them
alphabetically by macro name. This should make it easier to look for
examples when adding new tests to meson.build.
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
All currently supported distros have vte 0.37 or newer, which is where the
ABI changed from 2.90 to 2.91. So drop support for the older ABI.
Suggested-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Given that TCG is now the only consumer of X86XSaveArea, move the
structure definition and associated offset declarations and checks to a
TCG specific header.
Signed-off-by: David Edmondson <david.edmondson@oracle.com>
Message-Id: <20210705104632.2902400-9-david.edmondson@oracle.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Rather than relying on the X86XSaveArea structure definition,
determine the offset of XSAVE state areas using CPUID leaf 0xd where
possible (KVM and HVF).
Signed-off-by: David Edmondson <david.edmondson@oracle.com>
Message-Id: <20210705104632.2902400-8-david.edmondson@oracle.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Rather than relying on the X86XSaveArea structure definition directly,
the routines that manipulate the XSAVE state area should observe the
offsets declared in the x86_ext_save_areas array.
Currently the offsets declared in the array are derived from the
structure definition, resulting in no functional change.
Signed-off-by: David Edmondson <david.edmondson@oracle.com>
Message-Id: <20210705104632.2902400-7-david.edmondson@oracle.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Provide visibility of the x86_ext_save_areas array and associated type
outside of cpu.c.
Signed-off-by: David Edmondson <david.edmondson@oracle.com>
Message-Id: <20210705104632.2902400-6-david.edmondson@oracle.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
In preparation for removing assumptions about XSAVE area offsets, pass
a buffer pointer and buffer length to the XSAVE helper functions.
Signed-off-by: David Edmondson <david.edmondson@oracle.com>
Message-Id: <20210705104632.2902400-5-david.edmondson@oracle.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Replace the hard-coded size of offsets or structure elements with
defined constants or sizeof().
Signed-off-by: David Edmondson <david.edmondson@oracle.com>
Message-Id: <20210705104632.2902400-4-david.edmondson@oracle.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Rather than having similar but different checks in cpu.h and kvm.c,
move them all to cpu.h.
Message-Id: <20210705104632.2902400-3-david.edmondson@oracle.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Declare and use manifest constants for the XSAVE state component
offsets.
Signed-off-by: David Edmondson <david.edmondson@oracle.com>
Message-Id: <20210705104632.2902400-2-david.edmondson@oracle.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
lm32 has been removed in commit 9d49bcf699 ("Drop the deprecated
lm32 target"), and unicore32 in 4369223902 ("Drop the deprecated
unicore32 target").
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20210619091342.3660495-2-f4bug@amsat.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
The sdl and gtk display options support more parameters than currently
documented. Also the "vnc" option got lost during a recent commit,
add it again.
Fixes: ddc717581c ("Add display suboptions to man pages")
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20210630163231.467987-5-thuth@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
It's just a wrapper around the -display ...,window-close=off parameter,
and the name "no-quit" is rather confusing compared to "window-close"
(since there are still other means to quit the emulator), so we should
rather tell our users to use the "window-close" parameter instead.
While we're at it, update the documentation to state that
"-no-quit" is available for GTK, too, not only for SDL.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20210630163231.467987-4-thuth@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
According to the QAPI schema, there is a "-" and not a "_" between
"window" and "close", and we're also talking about "window-close"
in the long parameter description in qemu-options.hx, so we should
make sure that we rather use the variant with the "-" by default
instead of only allowing the one with the "_" here. The old way
still stays enabled for compatibility, but we deprecate it, so that
we can switch to a QAPIfied parameter one day more easily.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20210630163231.467987-3-thuth@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
The frame parameter has been removed along with the support for
SDL 1.2.
Fixes: 09bd7ba9f5 ("Remove deprecated -no-frame option")
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20210630163231.467987-2-thuth@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
The wrapper should not be needed here (it's not the shebang line of
a shell script), and it is causing trouble on Haiku where "env"
resides in a different directory.
Reported-by: Richard Zak <richard.j.zak@gmail.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210705082542.936856-1-thuth@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Dropped Peter Xu's migration-test fix to reenable
most of the migration tests when uffd isn't available;
we're seeing at least one seg in github CI (on qemu-system-i386)
and Peter Maydell is reporting a hang on Openbsd.
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEERfXHG0oMt/uXep+pBRYzHrxb/ecFAmDi2H8ACgkQBRYzHrxb
/edgRw//cr5xtzaTP2G3zP/R4T06Hp8xVgGNz8lXuqXOctLJNxMgj2uaty1fq4AL
vaRHhQ/DFgh5aQU2FbnoMFeaOdsGLbAla+p7bKuDswMGbQnl1sy/qRgmeJyN7McS
2Zo/pc4xkCv3YkUpVJnvysn4vyA7xyQ6a4ndmmsI3EyqX5d3IWaTrK6vRbFpi/tI
en6QVH1x5zXkygmFQddyhBNyS1OQGane3AhaetZs5y8FSFUsBcuixsgmSZZIhN5Y
Y1Yk+qRTQfdXVCYaBjbxoWk7yIiw/X4xy0wwrZUDc6zScv64nZDu+V1olq0DBGsg
Zo5lFfNx/mzq5FwjD3eU27PSaa9zHOnBmkvIhToGU+od+Ct4sB2Xbq4e2FtVYgX1
cOhptfE2OewKbNsSHWvKL6UqgxJ4tT+6COclwrdO7bqnIuWpJWPORWYmR5fKvN42
xnPZYopTpFJkEv2jKZAVif2tWnD+l806ebNtkAlldFwbAudzq4+qMCbMeDopyBP2
ksrUTchmyFq8nm7cuRvEm2z9EHDLHP/RtwsUZmTiY1A+d1B5HUO2ndByWO/buoc0
N6Y+UwTldi7ZK2bWphEpNsEqtT2vZs5X5CB3nWLOqPGoPYWnl6H58T/3kh9W/fvG
ycVoxZ3iH5rr3lsilQcFDwQyG3nprrWj2CO0V7b+oI5BBCdpOnQ=
=PBi+
-----END PGP SIGNATURE-----
Merge remote-tracking branch 'remotes/dgilbert-gitlab/tags/pull-migration-20210705a' into staging
Migration and virtiofs pull 2021-07-01 v2
Dropped Peter Xu's migration-test fix to reenable
most of the migration tests when uffd isn't available;
we're seeing at least one seg in github CI (on qemu-system-i386)
and Peter Maydell is reporting a hang on Openbsd.
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
# gpg: Signature made Mon 05 Jul 2021 11:01:35 BST
# gpg: using RSA key 45F5C71B4A0CB7FB977A9FA90516331EBC5BFDE7
# gpg: Good signature from "Dr. David Alan Gilbert (RH2) <dgilbert@redhat.com>" [full]
# Primary key fingerprint: 45F5 C71B 4A0C B7FB 977A 9FA9 0516 331E BC5B FDE7
* remotes/dgilbert-gitlab/tags/pull-migration-20210705a:
migration/rdma: Use error_report to suppress errno message
tests/migration: fix "downtime_limit" type when "migrate-set-parameters"
tests/migration: parse the thread-id key of CpuInfoFast
virtiofsd: Add an option to enable/disable posix acls
virtiofsd: Switch creds, drop FSETID for system.posix_acl_access xattr
virtiofsd: Add capability to change/restore umask
virtiofsd: Add umask to seccom allow list
virtiofsd: Add support for extended setxattr
virtiofsd: Fix xattr operations overwriting errno
virtiofsd: Fix fuse setxattr() API change issue
virtiofsd: Don't allow file creation with FUSE_OPEN
docs: describe the security considerations with virtiofsd xattr mapping
virtiofsd: use GDateTime for formatting timestamp for debug messages
migration: failover: continue to wait card unplug on error
migration: move wait-unplug loop to its own function
migration: Allow reset of postcopy_recover_triggered when failed
migration: Move yank outside qemu_start_incoming_migration()
migration: fix the memory overwriting risk in add_to_iovec
tests: migration-test: Add dirty ring test
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
As with previous performance optimization on Treaddir handling;
reduce the overall latency, i.e. overall time spent on processing
a Twalk request by reducing the amount of thread hops between the
9p server's main thread and fs worker thread(s).
In fact this patch even reduces the thread hops for Twalk handling
to its theoritical minimum of exactly 2 thread hops:
main thread -> fs worker thread -> main thread
This is achieved by doing all the required fs driver tasks altogether
in a single v9fs_co_run_in_worker({ ... }); code block.
Signed-off-by: Christian Schoenebeck <qemu_oss@crudebyte.com>
Reviewed-by: Greg Kurz <groug@kaod.org>
Message-Id: <1a6701674afc4f08d40396e3aa2631e18a4dbb33.1622821729.git.qemu_oss@crudebyte.com>
There is no longer a user of root_qid, so drop it.
Signed-off-by: Christian Schoenebeck <qemu_oss@crudebyte.com>
Reviewed-by: Greg Kurz <groug@kaod.org>
Message-Id: <6896dd161d3257db6b0513842a14f87ca191fdf6.1622821729.git.qemu_oss@crudebyte.com>
As we are actually only comparing the filesystem ID (i.e. device number
and inode number pair) let's use the POSIX stat buffer instead of QIDs,
because resolving QIDs requires to be done on 9p server's main thread
only as it might mutate the server state if inode remapping is enabled.
Signed-off-by: Christian Schoenebeck <qemu_oss@crudebyte.com>
Reviewed-by: Greg Kurz <groug@kaod.org>
Message-Id: <26aa465ff9cc9c07e053331554a02fdae3994417.1622821729.git.qemu_oss@crudebyte.com>
There is only one user of fid_to_qid() which is v9fs_walk(). Let's
open-code fid_to_qid() directly within v9fs_walk(), because
fid_to_qid() hides the POSIX stat buffer which we are going to need
in the subsequent patch.
Signed-off-by: Christian Schoenebeck <qemu_oss@crudebyte.com>
Reviewed-by: Greg Kurz <groug@kaod.org>
Message-Id: <e9a4c9c7a0792ed4db6578d105a0823ea05bc324.1622821729.git.qemu_oss@crudebyte.com>