Closing the FD does not necessarily mean that it is unlocked. Fix this
by relinquishing all permission locks before qemu_close().
Reported-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
raw_apply_lock_bytes() takes a bit mask of "permissions that are NOT
shared".
Also, make the "perm" and "shared" variables uint64_t, because I do not
particularly like using ~ on signed integers (and other permission masks
are usually uint64_t, too).
Reported-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
This is a way to ensure that the log superblock is periodically
updated. Before, this was only done on flush requests, which may
not be enough if the VM exits abnormally, omitting the final flush.
The default interval is 4096 write requests.
Signed-off-by: Ari Sundholm <ari@tuxera.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Suggested by Kevin Wolf. May be useful when testing multiple batches
of writes or doing long-term testing involving restarts of the VM.
Signed-off-by: Ari Sundholm <ari@tuxera.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
This was a simple oversight when working on intermediate versions
of the original patch which introduced blklogwrites.
Signed-off-by: Ari Sundholm <ari@tuxera.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
If the user passes a too long node name string, we silently truncate it
to fit into BlockDriverState.node_name, i.e. to 31 characters. Apart
from surprising the user when the node has a different name than
requested, this also bypasses the check for duplicate names, so that the
same name can be assigned to multiple nodes.
Fix this by just making too long node names an error.
Reported-by: Peter Krempa <pkrempa@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Implements a block device write logging system, similar to Linux kernel
device mapper dm-log-writes. The write operations that are performed
on a block device are logged to a file or another block device. The
write log format is identical to the dm-log-writes format. Currently,
log markers are not supported.
This functionality can be used for crash consistency and fs consistency
testing. By implementing it in qemu, tests utilizing write logs can be
be used to test non-Linux drivers and older kernels.
The driver accepts an optional parameter to set the sector size used
for logging. This makes the driver require all requests to be aligned
to this sector size and also makes offsets and sizes of writes in the
log metadata to be expressed in terms of this value (the log format has
a granularity of one sector for offsets and sizes). This allows
accurate logging of writes to guest block devices that have unusual
sector sizes.
The implementation is based on the blkverify and blkdebug block
drivers.
Signed-off-by: Aapo Vienamo <aapo@tuxera.com>
Signed-off-by: Ari Sundholm <ari@tuxera.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
This allows using the two constants outside of block.c, which will
happen in a subsequent patch.
Signed-off-by: Ari Sundholm <ari@tuxera.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Do data compression in separate threads. This significantly improve
performance for qemu-img convert with -W (allow async writes) and -c
(compressed) options.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Make a separate function for compression to be parallelized later.
- use .avail_out field instead of .next_out to calculate size of
compressed data. It looks more natural and it allows to keep dest to
be void pointer
- set avail_out to be at least one byte less than input, to be sure
avoid inefficient compression earlier
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
No reason to forbid them, and they are needed to improve performance
with compress-threads in further patches.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Bitmap lock/unlock were added to bdrv_enable_dirty_bitmap in
8b1402ce80, but some places were not updated correspondingly, which
leads to trying to take this lock twice, which is dead-lock. Fix this.
Actually, iotest 199 (about dirty bitmap postcopy migration) is broken
now, and this fixes it.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id: 20180625165745.25259-3-vsementsov@virtuozzo.com
Signed-off-by: John Snow <jsnow@redhat.com>
Add _locked version of bdrv_enable_dirty_bitmap, to fix dirty bitmap
migration in the following patch.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id: 20180625165745.25259-2-vsementsov@virtuozzo.com
Signed-off-by: John Snow <jsnow@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-33-armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-32-armbru@redhat.com>
qmp_greeting() offers capabilities to the client, and
qmp_qmp_capabilities() accepts or denies capabilities requested by the
client. The two compute the set of available capabilities
independently. Not nice.
Clean this up as follows. Compute available capabilities just once in
monitor_qmp_caps_reset(), and store them in Monitor member
qmp.capab_offered[]. Have qmp_greeting() and qmp_qmp_capabilities()
use that. Both are now oblivious of capability details.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-31-armbru@redhat.com>
qobject_from_jsonf() aborts on error, unlike qobject_from_jsonv(),
which returns null. Since all remaining users of qobject_from_jsonf()
cope fine with null, change it to return null.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-30-armbru@redhat.com>
There's just one use of qobject_from_jsonf() to parse a JSON object
left: timestamp_put(). Switch it to qdict_from_jsonf_nofail().
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-29-armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-28-armbru@redhat.com>
monitor_qmp_respond() takes both a response object and an error
object. If an error object is non-null, the response object must be
null, and the response is built from the error object.
Of the two callers, one always passes a null response object, and one
a null error object. Move building the response object from the error
object to the latter, and drop the error object parameter.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-27-armbru@redhat.com>
get_qmp_greeting() returns a QDict * as QObject *. It's caller
converts it right back.
Return QDict * instead. While there, rename to qmp_greeting().
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-26-armbru@redhat.com>
monitor_json_emitter() and monitor_json_emitter_raw() are
unnecessarily general: they can send arbitrary JSON values, even
though we only ever use them for QMP, which may send only JSON
objects.
Specialize the argument from QObject * to QDict *, and rename to
qmp_queue_response(), qmp_send_response().
All callers but one lose an upcast. The lone exception gains a
downcast; the next commit will get rid of it.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-25-armbru@redhat.com>
By using the more specific type, we get fewer downcasts. The
downcasts are safe, but not obviously so, at least not locally.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-24-armbru@redhat.com>
All callers of qmp_build_error_object() duplicate the code to wrap it
in a response object. Replace it by qmp_error_response() that
captures the duplicated code, including error_free().
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-23-armbru@redhat.com>
Many uses of qobject_from_jsonf() convert JSON objects. Create new
convenience function qdict_from_jsonf_nofail() that includes the
conversion to QDict. The next few commits will put it to use.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-22-armbru@redhat.com>
Wrapping global variables in a struct without a use for the wrapper
struct buys us nothing but longer lines. Unwrap them.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-21-armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-20-armbru@redhat.com>
handle_qmp_command() reports JSON syntax errors right away. This is
wrong when OOB is enabled, because the errors can "jump the queue"
then.
The previous commit fixed the same bug for semantic errors, by
delaying the checking until dispatch. We can't delay the checking, so
delay the reporting.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-19-armbru@redhat.com>
handle_qmp_command() reports certain errors right away. This is wrong
when OOB is enabled, because the errors can "jump the queue" then, as
the previous commit demonstrates.
To fix, we need to delay errors until dispatch. Do that for semantic
errors, mostly by reverting ill-advised parts of commit cf869d5317
"qmp: support out-of-band (oob) execution". Bonus: doesn't run
qmp_dispatch_check_obj() twice, once in handle_qmp_command(), and
again in do_qmp_dispatch(). That's also due to commit cf869d5317.
The next commit will fix queue jumping for syntax errors.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-18-armbru@redhat.com>
When OOB is enabled, out-of-band commands are executed right away,
everything else is queued. This lets out-of-band commands "jump the
queue".
However, certain errors are always reported right away, and therefore
can jump the queue even when the erroneous input does not request
out-of-band execution. These errors are pretty unlikely to occur in
production, but it's wrong all the same. Mark FIXME.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180703085358.13941-17-armbru@redhat.com>
Change monitor_qmp_dispatch_one() to take its parameters unwrapped,
move monitor_resume() to the one caller that needs it, rename the
function to monitor_qmp_dispatch().
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-16-armbru@redhat.com>
monitor_qmp_dispatch_one() frees a QMPRequest manually, because it
needs to keep a reference to ->id. Premature optimization. Take an
additional reference so we can use qmp_request_free().
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-15-armbru@redhat.com>
Commit 71da4667db "monitor: separate QMP parser and dispatcher" moved
the handle_qmp_command tracepoint from handle_qmp_command() to
monitor_qmp_dispatch_one(). This delays tracing from enqueue time to
dequeue time. Revert that. Dequeue remains adequately visible via
tracepoint monitor_qmp_cmd_in_band.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-14-armbru@redhat.com>
Commit cf869d5317 "qmp: support out-of-band (oob) execution" added a
general mechanism for command-independent arguments just for an
out-of-band flag:
The "control" key is introduced to store this extra flag. "control"
field is used to store arguments that are shared by all the commands,
rather than command specific arguments. Let "run-oob" be the first.
However, it failed to reject unknown members of "control". For
instance, in QMP command
{"execute": "query-name", "id": 42, "control": {"crap": true}}
"crap" gets silently ignored.
Instead of fixing this, revert the general "control" mechanism
(because YAGNI), and do it the way I initially proposed, with key
"exec-oob". Simpler code, simpler interface.
An out-of-band command
{"execute": "migrate-pause", "id": 42, "control": {"run-oob": true}}
becomes
{"exec-oob": "migrate-pause", "id": 42}
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-13-armbru@redhat.com>
[Commit message typo fixed]
Commit cf869d5317 "qmp: support out-of-band (oob) execution"
accidentally made qemu-ga accept and ignore "control". Fix that.
Out-of-band execution in a monitor that doesn't support it now fails
with
{"error": {"class": "GenericError", "desc": "QMP input member 'control' is unexpected"}}
instead of
{"error": {"class": "GenericError", "desc": "Please enable out-of-band first for the session during capabilities negotiation"}}
The old description is suboptimal when out-of-band cannot not be
enabled, or the command doesn't support out-of-band execution.
The new description is a bit unspecific, but it'll do.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-12-armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-11-armbru@redhat.com>
Commit cf869d5317 "qmp: support out-of-band (oob) execution" changed
how we check "id":
Note that in the patch I exported qmp_dispatch_check_obj() to be
used to check the request earlier, and at the same time allowed
"id" field to be there since actually we always allow that.
The part after "and" is ill-advised: it makes qemu-ga accept and
ignore "id". Revert.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-10-armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-9-armbru@redhat.com>
Commit cf869d5317 "qmp: support out-of-band (oob) execution" made
"id" mandatory for all commands when the client accepted capability
"oob". This is rather onerous when you play with QMP by hand, and
unnecessarily so: only out-of-band commands need an ID for reliable
matching of response to command.
Revert that part of commit cf869d5317 for now, but have documentation
advise on the need to use "id" with out-of-band commands.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20180703085358.13941-8-armbru@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-7-armbru@redhat.com>
tests/qmp-test tests an out-of-band command overtaking a slow in-band
command. To do that, it needs:
1. An in-band command that *reliably* takes long enough to be
overtaken.
2. An out-of-band command to do the overtaking.
3. To avoid delays, a way to make the in-band command complete quickly
after it was overtaken.
To satisfy these needs, commit 469638f9cb provides the rather
peculiar oob-capable QMP command x-oob-test:
* With "lock": true, it waits for a global semaphore.
* With "lock": false, it signals the global semaphore.
To satisfy 1., the test runs x-oob-test in-band with "lock": true.
To satisfy 2. and 3., it runs x-oob-test out-of-band with "lock": false.
Note that waiting for a semaphore violates the rules for oob-capable
commands. Running x-oob-test with "lock": true hangs the monitor
until you run x-oob-test with "lock": false on another monitor (which
you might not have set up).
Having an externally visible QMP command that may hang the monitor is
not nice. Let's apply a little more ingenuity to the problem. Idea:
have an existing command block on reading a FIFO special file, unblock
it by opening the FIFO for writing.
For 1., use
{"execute": "blockdev-add", "id": ID1,
"arguments": {
"driver": "blkdebug", "node-name": ID1, "config": FIFO,
"image": { "driver": "null-co"}}}
where ID1 is an arbitrary string, and FIFO is the name of the FIFO.
For 2., use
{"execute": "migrate-pause", "id": ID2, "control": {"run-oob": true}}
where ID2 is a different arbitrary string. Since there's no migration
to pause, the command will fail, but that's fine; instant failure is
still a test of out-of-band responses overtaking in-band commands.
For 3., open FIFO for writing.
Drop QMP command x-oob-test.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-6-armbru@redhat.com>
[Error checking tweaked]
Events are broadcast to all monitors. If another monitor's client has
a command with the same ID in flight, the event will incorrectly claim
that command was dropped. This must be fixed before out-of-band
execution can graduate from "experimental".
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-5-armbru@redhat.com>
OOB documentation is spread over qmp-spec.txt sections 2.2.1
Capabilities and 2.3 Issuing Commands. The amount of detail is a bit
distracting there. Move the meat of the matter to new section 2.3.1
Out of band execution.
Throw in a few other improvements while there:
* 2.2 Server Greeting: Drop advice to search entire capabilities
array; should be obvious.
* 3. QMP Examples
- 3.1 Server Greeting: Update greeting to the one we expect for the
release. Now shows capability "oob". Update qmp-intro.txt
likewise.
- 3.2 Capabilities negotiation: Show client accepting capability
"oob".
- 3.7 Out-of-band execution: New.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180703085358.13941-4-armbru@redhat.com>
[Whitespace tidied up]