Commit graph

365 commits

Author SHA1 Message Date
Markus Armbruster 7b3bc9e28f qapi: Consistently put @features parameter right after @ifcond
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200317115459.31821-14-armbru@redhat.com>
2020-03-17 19:58:34 +01:00
Markus Armbruster 013b4efc9b qapi: Add feature flags to remaining definitions
In v4.1.0, we added feature flags just to struct types (commit
6a8c0b5102^..f3ed93d545), to satisfy an immediate need (commit
c9d4070991 "file-posix: Add dynamic-auto-read-only QAPI feature").  In
v4.2.0, we added them to commands (commit 23394b4c39 "qapi: Add
feature flags to commands") to satisfy another immediate need (commit
d76744e65e "qapi: Allow introspecting fix for savevm's cooperation
with blockdev").

Add them to the remaining definitions: enumeration types, union types,
alternate types, and events.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200317115459.31821-13-armbru@redhat.com>
2020-03-17 19:58:34 +01:00
Markus Armbruster e4405b3069 qapi/schema: Clean up around QAPISchemaEntity.connect_doc()
QAPISchemaEntity calls doc.connect_feature() in .check().  Improper
since commit ee1e6a1f6c split .connect_doc() off .check().  Move the
call.  Requires making the children call super().connect_doc() as they
should.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200317115459.31821-12-armbru@redhat.com>
2020-03-17 19:58:34 +01:00
Kevin Wolf 2af282ec51 qemu-storage-daemon: Add --monitor option
This adds and parses the --monitor option, so that a QMP monitor can be
used in the storage daemon. The monitor offers commands defined in the
QAPI schema at storage-daemon/qapi/qapi-schema.json.

The --monitor options currently allows to create multiple monitors with
the same ID. This part of the interface is considered unstable. We will
reject such configurations as soon as we have a design for the monitor
subsystem to perform these checks. (In the system emulator, we depend on
QemuOpts rejecting duplicate IDs.)

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20200224143008.13362-21-kwolf@redhat.com>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-03-06 17:21:28 +01:00
Markus Armbruster 8ec0e1a4e6 qapi: Brush off some (py)lint
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20200304155932.20452-5-armbru@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
2020-03-05 09:24:11 +01:00
Markus Armbruster 2cae67bcb5 qapi: Use super() now we have Python 3
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20200304155932.20452-4-armbru@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
2020-03-05 09:24:11 +01:00
Markus Armbruster ed39c03e2f qapi: Drop conditionals for Python 2
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200304155932.20452-3-armbru@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
2020-03-05 09:24:11 +01:00
Markus Armbruster baa310f1bb qapi: Inheriting from object is pointless with Python 3, drop
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200304155932.20452-2-armbru@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
2020-03-05 09:24:11 +01:00
Paolo Bonzini 423edd9a31 drop "from __future__ import print_function"
This is only needed for Python 2, which we do not support anymore.

Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20200204160604.19883-1-pbonzini@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:15:16 +01:00
Markus Armbruster 43d1455cf8 qapi: Fix code generation with Python 3.5
Recent commit 3e7fb5811b "qapi: Fix code generation for empty modules"
modules" switched QAPISchema.visit() from

    for entity in self._entity_list:

effectively to

    for mod in self._module_dict.values():
        for entity in mod._entity_list:

Visits in the same order as long as .values() is in insertion order.
That's the case only for Python 3.6 and later.  Before, it's in some
arbitrary order, which results in broken generated code.

Fix by making self._module_dict an OrderedDict rather than a dict.

Fixes: 3e7fb5811b
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Thomas Huth <thuth@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: BALATON Zoltan <balaton@eik.bme.hu>
Tested-by: Alex Bennée <alex.bennee@linaro.org>
Message-id: 20200116202558.31473-1-armbru@redhat.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-01-20 12:17:38 +00:00
Markus Armbruster 3bef3aaec9 qapi: Simplify QAPISchemaModularCVisitor
Since the previous commit, QAPISchemaVisitor.visit_module() is called
just once.  Simplify QAPISchemaModularCVisitor accordingly.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191120182551.23795-7-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-01-14 11:01:58 +01:00
Markus Armbruster 3e7fb5811b qapi: Fix code generation for empty modules
When a sub-module doesn't contain any definitions, we don't generate
code for it, but we do generate the #include.

We generate code only for modules that get visited.
QAPISchema.visit() visits only modules that have definitions.  It can
visit modules multiple times.

Clean this up as follows.  Collect entities in their QAPISchemaModule.
Have QAPISchema.visit() call QAPISchemaModule.visit() for each module.
Have QAPISchemaModule.visit() call .visit_module() for itself, and
QAPISchemaEntity.visit() for each of its entities.  This way, we visit
each module exactly once.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191120182551.23795-6-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-01-14 11:01:58 +01:00
Markus Armbruster a9f1dd7ee0 qapi: Proper intermediate representation for modules
Modules are represented only by their names so far.  Introduce class
QAPISchemaModule.  So far, it merely wraps the name.  The next patch
will put it to more interesting use.

Once again, arrays spice up the patch a bit.  For any other type,
@info points to the definition, which lets us map from @info to
module.  For arrays, there is no definition, and @info points to the
first use instead.  We have to use the element type's module instead,
which is only available after .check().

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191120182551.23795-5-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-01-14 11:01:58 +01:00
Markus Armbruster 00ca24ff9e qapi: Generate command registration stuff into separate files
Having to include qapi-commands.h just for qmp_init_marshal() is
suboptimal.  Generate it into separate files.  This lets
monitor/misc.c, qga/main.c, and the generated qapi-commands-FOO.h
include less.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191120182551.23795-4-armbru@redhat.com>
[Typos in docs/devel/qapi-code-gen.txt fixed]
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-01-14 11:01:58 +01:00
Markus Armbruster 7e9c1707e1 qapi: Tweak "command returns a nice type" check for clarity
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191120182551.23795-2-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-01-14 11:01:58 +01:00
Markus Armbruster e151941d1b qapi: Check feature documentation against the schema
Commit f3ed93d545 "qapi: Allow documentation for features" neglected
to check documentation against the schema.  Fix that: check them the
same way we check arguments.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-20-armbru@redhat.com>
2019-10-29 07:35:16 +01:00
Markus Armbruster e4def78755 qapi: Polish reporting of bogus member documentation
Improve error messages from

    the following documented members are not in the declaration: a
    the following documented members are not in the declaration: aa, bb

to the more concise

    documented member 'a' does not exist
    documented members 'aa', 'bb' do not exist

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-19-armbru@redhat.com>
2019-10-29 07:35:16 +01:00
Markus Armbruster b3cdff10e5 qapi: Lift features into QAPISchemaEntity
Commit 6a8c0b5102 "qapi: Add feature flags to struct types" added
features to QAPISchemaObjectType.  Commit a95daa5093 "qapi: Add
feature flags to commands in qapi" added them to QAPISchemaCommand,
duplicating the code.  Tolerable, but the duplication will only get
worse as we add features to more definitions.

To de-duplicate, lift features from QAPISchemaObjectType and
QAPISchemaCommand into QAPISchemaEntity.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-18-armbru@redhat.com>
2019-10-29 07:35:16 +01:00
Markus Armbruster ad1ecfc679 qapi: Fold normalize_enum() into check_enum()
check_features() is always called together with normalize_features().
Fold the latter into the former.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-17-armbru@redhat.com>
2019-10-29 07:35:16 +01:00
Markus Armbruster 2ce51ef68f qapi: Fold normalize_features() into check_features()
check_features() is always called together with normalize_features():
the former in check_struct() and check_command(), the latter in their
caller check_exprs().  Fold the latter into the former.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-16-armbru@redhat.com>
2019-10-29 07:35:16 +01:00
Markus Armbruster c145bfda18 qapi: Fold normalize_if() into check_if()
check_if() is always called together with normalize_if().  Fold the
latter into the former.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-15-armbru@redhat.com>
2019-10-29 07:35:16 +01:00
Markus Armbruster 1192a4862b qapi: Eliminate .check_doc() overrides
All sub-classes of QAPISchemaEntity now override .check_doc() the same
way, except for QAPISchemaType and and QAPISchemaArrayType.

Put the overrides' code in QAPISchemaEntity.check_doc(), and drop the
overrides.  QAPISchemaType doesn't care because it's abstract.
QAPISchemaArrayType doesn't care because its .doc is always None.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-14-armbru@redhat.com>
2019-10-29 07:35:16 +01:00
Markus Armbruster a710e1c8c3 qapi: Simplify ._make_implicit_object_type()
All callers now pass doc=None.  Drop the argument.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-13-armbru@redhat.com>
2019-10-29 07:35:16 +01:00
Markus Armbruster bf83f04e13 qapi: Fix doc comment checking for commands and events
When a command's 'data' is an object, its doc comment describes the
arguments defined there.  When 'data' names a type, the doc comment
does not describe arguments.  Instead, the doc generator inserts a
pointer to the named type.

An event's doc comment works the same.

We don't actually check doc comments for commands and events.
Instead, QAPISchema._def_command() forwards the doc comment to the
implicit argument type, where it gets checked.  Works because the
check only cares for the implicit argument type's members.

Not only is this needlessly hard to understand, it actually falls
apart in two cases:

* When 'data' is empty, there is nothing to forward to, and the doc
  comment remains unchecked.  Demonstrated by test doc-bad-event-arg.

* When 'data' names a type, we can't forward, as the type has its own
  doc comment.  The command or event's doc comment remains unchecked.
  Demonstrated by test doc-bad-boxed-command-arg.

The forwarding goes back to commit 069fb5b250 "qapi: Prepare for
requiring more complete documentation", put to use in commit
816a57cd6e "qapi: Fix detection of bogus member documentation".  That
fix was incomplete.

To fix this, make QAPISchemaCommand and QAPISchemaEvent check doc
comments, and drop the forwarding of doc comments to implicit argument
types.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-12-armbru@redhat.com>
2019-10-29 07:35:16 +01:00
Markus Armbruster 7faefad184 qapi: Clean up doc comment checking for implicit union base
An object type's doc comment describes the type's members, less the
ones defined in a named base type.  Cases:

* Struct: the members are defined in 'data' and inherited from 'base'.
  Since the base type cannot be implicit, the doc comment describes
  just 'data'.

* Simple union: the only member is the implicit tag member @type, and
  the doc comment describes it.

* Flat union with implicit base type: the members are defined in
  'base', and the doc comment describes it.

* Flat union with named base type: the members are inherited from
  'base'.  The doc comment describes no members.

Before we can check a doc comment with .check_doc(), we need
.connect_doc() connect each of its "argument sections" to the member
it documents.

For structs and simple unions, this is straightforward: the members in
question are in .local_members, and .connect_doc() connects them.

For flat unions with a named base type, it's trivial: .local_members
is empty, and .connect_doc() does nothing.

For flat unions with an implicit base type, it's tricky.  We have
QAPISchema._make_implicit_object_type() forward the union's doc
comment to the implicit base type, so that the base type's
.connect_doc() connects the members.  The union's .connect_doc() does
nothing, as .local_members is empty.

Dirt effect: we check the doc comment twice, once for the union type,
and once for the implicit base type.

This is needlessly brittle and hard to understand.  Clean up as
follows.  Make the union's .connect_doc() connect an implicit base's
members itself.  Do not forward the union's doc comment to its
implicit base type.

Requires extending .connect_doc() so it can work with a doc comment
other than self.doc.  Add an optional argument for that.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-11-armbru@redhat.com>
2019-10-29 07:35:16 +01:00
Markus Armbruster 36a43905ff qapi: Fix enum doc comment checking
Enumeration type documentation comments are not checked, as
demonstrated by test doc-bad-enum-member.  This is because we neglect
to call self.doc.check() for enumeration types.  Messed up in
816a57cd6e "qapi: Fix detection of bogus member documentation".  Fix
it.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-10-armbru@redhat.com>
2019-10-29 07:35:16 +01:00
Markus Armbruster ee1e6a1f6c qapi: Split .connect_doc(), .check_doc() off .check()
Splitting documentation checking off the .check() methods makes them a
bit more focused, which is welcome, as some of them are pretty big.
It also prepares the ground for the following commits.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-9-armbru@redhat.com>
2019-10-29 07:35:16 +01:00
Markus Armbruster a4bd91d3f3 qapi: De-duplicate entity documentation generation code
QAPISchemaGenDocVisitor.visit_command() duplicates texi_entity() for
its boxed arguments case.  The previous commit added another copy in
.visit_event().

Replace texi_entity() by texi_type() and texi_msg().  Use texi_msg()
for the boxed arguments case as well.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-8-armbru@redhat.com>
2019-10-29 07:35:16 +01:00
Markus Armbruster b621a26040 qapi: Implement boxed event argument documentation
Generate a reference "Arguments: the members of ...", just like we do
for commands since commit c2dd311cb7 "qapi2texi: Implement boxed
argument documentation".

No change to generated QMP documentation; we don't yet use boxed
events outside tests/.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-7-armbru@redhat.com>
2019-10-29 07:35:16 +01:00
Peter Krempa 23394b4c39 qapi: Add feature flags to commands
Similarly to features for struct types introduce the feature flags also
for commands. This will allow notifying management layers of fixes and
compatible changes in the behaviour of a command which may not be
detectable any other way.

The changes were heavily inspired by commit 6a8c0b5102.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191018081454.21369-3-armbru@redhat.com>
2019-10-22 13:54:13 +02:00
Markus Armbruster 02ac641a4d qapi: Clear scripts/qapi/doc.py executable bits again
Commit fbf09a2fa4 "qapi: add 'ifcond' to visitor methods" brought back
the executable bits.  Fix that.  Drop the #! line for good measure.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20191018074345.24034-8-armbru@redhat.com>
2019-10-22 13:54:13 +02:00
Markus Armbruster e6c42b96b9 qapi: Split up scripts/qapi/common.py
The QAPI code generator clocks in at some 3100 SLOC in 8 source files.
Almost 60% of the code is in qapi/common.py.  Split it into more
focused modules:

* Move QAPISchemaPragma and QAPISourceInfo to qapi/source.py.

* Move QAPIError and its sub-classes to qapi/error.py.

* Move QAPISchemaParser and QAPIDoc to parser.py.  Use the opportunity
  to put QAPISchemaParser first.

* Move check_expr() & friends to qapi/expr.py.  Use the opportunity to
  put the code into a more sensible order.

* Move QAPISchema & friends to qapi/schema.py

* Move QAPIGen and its sub-classes, ifcontext,
  QAPISchemaModularCVisitor, and QAPISchemaModularCVisitor to qapi/gen.py

* Delete camel_case(), it's unused since commit e98859a9b9 "qapi:
  Clean up after recent conversions to QAPISchemaVisitor"

A number of helper functions remain in qapi/common.py.  I considered
moving the code generator helpers to qapi/gen.py, but decided not to.
Perhaps we should rewrite them as methods of QAPIGen some day.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20191018074345.24034-7-armbru@redhat.com>
[Add "# -*- coding: utf-8 -*-" lines]
2019-10-22 13:53:55 +02:00
Markus Armbruster 61bfb2e1a4 qapi: Move gen_enum(), gen_enum_lookup() back to qapi/types.py
The next commit will split up qapi/common.py.  gen_enum() needs
QAPISchemaEnumMember, and that's in the way.  Move it to qapi/types.py
along with its buddy gen_enum_lookup().

Permit me a short a digression on history: how did gen_enum() end up
in qapi/common.py?  Commit 21cd70dfc1 "qapi script: add event support"
duplicated qapi-types.py's gen_enum() and gen_enum_lookup() in
qapi-event.py.  Simply importing them would have been cleaner, but
wasn't possible as qapi-types.py was a program, not a module.  Commit
efd2eaa6c2 "qapi: De-duplicate enum code generation" de-duplicated by
moving them to qapi.py, which was a module.

Since then, program qapi-types.py has morphed into module types.py.
It's where gen_enum() and gen_enum_lookup() started, and where they
belong.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20191018074345.24034-6-armbru@redhat.com>
2019-10-22 09:26:12 +02:00
Markus Armbruster 0002b557b5 qapi: Eliminate accidental global frontend state
The frontend can't be run more than once due to its global state.
A future commit will want to do that.

The only global frontend state remaining is accidental:
QAPISchemaParser.__init__()'s parameter previously_included=[].
Python evaluates the default once, at definition time.  Any
modifications to it are visible in subsequent calls.  Well-known
Python trap.  Change the default to None and replace it by the real
default in the function body.  Use the opportunity to convert
previously_included to a set.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20191018074345.24034-4-armbru@redhat.com>
2019-10-22 09:26:12 +02:00
Markus Armbruster 2a7bbedd77 qapi: Store pragma state in QAPISourceInfo, not global state
The frontend can't be run more than once due to its global state.
A future commit will want to do that.

Recent commit "qapi: Move context-sensitive checking to the proper
place" got rid of many global variables already, but pragma state is
still stored in global variables (that's why a pragma directive's
scope is the complete schema).

Move the pragma state to QAPISourceInfo.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20191018074345.24034-3-armbru@redhat.com>
2019-10-22 09:26:12 +02:00
Markus Armbruster f3d4aa5add qapi: Don't suppress doc generation without pragma doc-required
Commit bc52d03ff5 "qapi: Make doc comments optional where we don't
need them" made scripts/qapi2texi.py fail[*] unless the schema had
pragma 'doc-required': true.  The stated reason was inability to cope
with incomplete documentation.

When commit fb0bc835e5 "qapi-gen: New common driver for code and doc
generators" folded scripts/qapi2texi.py into scripts/qapi-gen.py, it
turned the failure into silent suppression.

The doc generator can cope with incomplete documentation now.  I don't
know since when, or what the problem was, or even whether it ever
existed.

Drop the silent suppression.

[*] The fail part was broken, fixed in commit e8ba07ea9a.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20191018074345.24034-2-armbru@redhat.com>
2019-10-22 09:26:12 +02:00
Markus Armbruster c615550df3 qapi: Improve source file read error handling
qapi-gen.py crashes when it can't open the main schema file, and when
it can't read from any schema file.  Lazy.

Change QAPISchema.__init__() to take a file name instead of a file
object.  Move the open code from _include() to __init__(), so it's
used for the main schema file, too.

Move the read into the try for good measure, and rephrase the error
message.

Reporting open or read failure for the main schema file needs a
QAPISourceInfo representing "no source".  Make QAPISourceInfo cope
with fname=None.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-27-armbru@redhat.com>
2019-09-28 17:17:48 +02:00
Markus Armbruster 56d2df5e65 qapi: Improve reporting of redefinition
Point to the previous definition, unless it's a built-in.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-26-armbru@redhat.com>
2019-09-28 17:17:48 +02:00
Markus Armbruster f63326985a qapi: Improve reporting of missing documentation comment
Have check_exprs() check this later, so the error message gains an "in
definition line".  Tweak the error message.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-25-armbru@redhat.com>
2019-09-28 17:17:48 +02:00
Markus Armbruster 13b3997f14 qapi: Eliminate check_keys(), rename check_known_keys()
check_keys() has become a trivial wrapper for check_known_keys().
Eliminate it.

This makes its name available.  Rename check_known_keys().

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-24-armbru@redhat.com>
2019-09-28 17:17:48 +02:00
Markus Armbruster fab12376d0 qapi: Improve reporting of invalid 'if' further
check_if()'s errors don't point to the offending part of the
expression.  For instance:

    tests/qapi-schema/alternate-branch-if-invalid.json:2: 'if' condition ' ' makes no sense

Other check_FOO() do, with the help of a @source argument.  Make
check_if() do that, too.  The example above improves to:

    tests/qapi-schema/alternate-branch-if-invalid.json:2: 'if' condition ' ' of 'data' member 'branch' makes no sense

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190927134639.4284-23-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-09-28 17:17:45 +02:00
Markus Armbruster eeb57c85da qapi: Avoid redundant definition references in error messages
Many error messages refer to the offending definition even though
they're preceded by an "in definition" line.  Rephrase them.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190927134639.4284-22-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-09-28 17:17:32 +02:00
Markus Armbruster 3f58cc29a8 qapi: Improve reporting of missing / unknown definition keys
Have check_exprs() call check_keys() later, so its error messages gain
an "in definition" line.

Both check_keys() and check_name_is_str() check the definition's name
is a string.  Since check_keys() now runs after check_name_is_str()
rather than before, its check is dead.  Bury it.  Checking values in
check_keys() is unclean anyway.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-21-armbru@redhat.com>
2019-09-28 17:17:19 +02:00
Markus Armbruster a6735a5743 qapi: Improve reporting of invalid flags
Split check_flags() off check_keys() and have check_exprs() call it
later, so its error messages gain an "in definition" line.  Tweak the
error messages.

Checking values in a function named check_keys() is unclean anyway.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-20-armbru@redhat.com>
2019-09-28 17:17:19 +02:00
Markus Armbruster 576f0b8a53 qapi: Improve reporting of invalid 'if' errors
Move check_if() from check_keys() to check_exprs() and call it later,
so its error messages gain an "in definition" line.

Checking values in a function named check_keys() is unclean anyway.
The original sin was commit 0545f6b887 "qapi: Better error messages
for bad expressions", which checks the value of key 'name'.  More
sinning in commit 2cbf09925a "qapi: More rigorous checking for type
safety bypass", commit c818408e44 "qapi: Implement boxed types for
commands/events", and commit 967c885108 "qapi: add 'if' to top-level
expressions".  This commit does penance for the latter.  The next
commits will do penance for the others.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-19-armbru@redhat.com>
2019-09-28 17:17:19 +02:00
Markus Armbruster 4ebda5abdb qapi: Move context-free checking to the proper place
QAPISchemaCommand.check() and QAPISchemaEvent().check() check 'data'
is present when 'boxed': true.  That's context-free.  Move to
check_command() and check_event().

Tweak the error message while there.

check_exprs() & friends now check exactly what qapi-code-gen.txt calls
the second layer of syntax.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-18-armbru@redhat.com>
2019-09-28 17:17:19 +02:00
Markus Armbruster fa110c6a9e qapi: Move context-sensitive checking to the proper place
When we introduced the QAPISchema intermediate representation (commit
ac88219a6c), we took a shortcut: we left check_exprs() & friends
alone instead of moving semantic checks into the
QAPISchemaFOO.check().  The .check() assert check_exprs() did its job.

Time to finish the conversion job.  Move exactly the context-sensitive
checks to the .check().  They replace assertions there.  Context-free
checks stay put.

Fixes the misleading optional tag error demonstrated by test
flat-union-optional-discriminator.

A few other error message improve.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-17-armbru@redhat.com>
2019-09-28 17:17:19 +02:00
Markus Armbruster 77daece3d9 qapi: Inline check_name() into check_union()
check_name() consists of check_name_is_str() and check_name_str().
check_union() relies on the latter to catch optional discriminators.
The next commit will replace that by a more straightforward check.
Inlining check_name() into check_union() now should make that easier
to review.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-16-armbru@redhat.com>
2019-09-28 17:17:19 +02:00
Markus Armbruster e6f9678da5 qapi: Plumb info to the QAPISchemaMember
Future commits will need info in the .check() methods of
QAPISchemaMember and its descendants.  Get it there.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-15-armbru@redhat.com>
2019-09-28 17:17:19 +02:00
Markus Armbruster 88112488cf qapi: Make check_type()'s array case a bit more obvious
check_type() checks the array's contents, then peels off the array and
falls through to the "not array" code without resetting allow_array
and allow_dict to False.  Works because the peeled value is a string,
and allow_array and allow_dict aren't used then.  Tidy up anyway:
recurse instead, defaulting allow_array and allow_dict to False.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-14-armbru@redhat.com>
2019-09-28 17:17:19 +02:00
Markus Armbruster 67fa64ce0e qapi: Move check for reserved names out of add_name()
The checks for reserved names are spread far and wide.  Move one from
add_name() to new check_defn_name_str().  This is a first step towards
collecting them all in dedicated name checking functions next to
check_name().

While there, drop the quotes around the meta-type in
check_name_str()'s error messages: "'command' uses ... name 'NAME'"
becomes "command uses ... name 'NAME'".

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-13-armbru@redhat.com>
2019-09-28 17:17:19 +02:00
Markus Armbruster 64e04f7149 qapi: Report invalid '*' prefix like any other invalid name
The special "does not allow optional name" error is well meant, but
confusing in practice.  Drop it.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-12-armbru@redhat.com>
2019-09-28 17:17:19 +02:00
Markus Armbruster 6ba1ba7f0e qapi: Use check_name_str() where it suffices
Replace check_name() by check_name_str() where the name is known to be
a string.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-11-armbru@redhat.com>
2019-09-28 17:17:19 +02:00
Markus Armbruster d7bc17c602 qapi: Improve reporting of invalid name errors
Split check_name() into check_name_is_str() and check_name_str(), keep
check_name() as a wrapper.

Move add_name()'s call into its caller check_exprs(), and inline.

This permits delaying check_name_str() there, so its error message
gains an "in definition" line.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-10-armbru@redhat.com>
2019-09-28 17:17:18 +02:00
Markus Armbruster c9efc984ca qapi: Reorder check_FOO() parameters for consistency
Most check_FOO() take the thing being checked as first argument.
check_name(), check_type(), check_known_keys() don't.  Clean that up.

While there, drop a "Todo" comment that should have been dropped in
commit 87adbbffd4 "qapi: add a dictionary form for TYPE".

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-9-armbru@redhat.com>
2019-09-28 17:17:18 +02:00
Markus Armbruster 481a6bd15c qapi: Improve reporting of member name clashes
We report name clashes like this:

    struct-base-clash.json: In struct 'Sub':
    struct-base-clash.json:5: 'name' (member of Sub) collides with 'name' (member of Base)

The "(member of Sub)" is redundant with "In struct 'Sub'".  Comes from
QAPISchemaMember.describe().  Pass info to it, so it can detect the
redundancy and avoid it.  Result:

    struct-base-clash.json: In struct 'Sub':
    struct-base-clash.json:5: member 'name' collides with member 'name' of type 'Base'

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-8-armbru@redhat.com>
2019-09-28 17:17:18 +02:00
Markus Armbruster 2ab218aad6 qapi: Change frontend error messages to start with lower case
Starting error messages with a capital letter complicates things when
text can get interpolated both at the beginning and in the middle of
an error message.  The next patch will do that.  Switch to lower case
to keep it simpler.

For what it's worth, the GNU Coding Standards advise the message
"should not begin with a capital letter when it follows a program name
and/or file name, because that isn’t the beginning of a sentence. (The
sentence conceptually starts at the beginning of the line.)"

While there, avoid breaking lines containing multiple arguments in the
middle of an argument.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-7-armbru@redhat.com>
2019-09-28 17:17:18 +02:00
Markus Armbruster 638c4af931 qapi: Clean up member name case checking
QAPISchemaMember.check_clash() checks for member names that map to the
same c_name().  Takes care of rejecting duplicate names.

It also checks a naming rule: no uppercase in member names.  That's a
rather odd place to do it.  Enforcing naming rules is
check_name_str()'s job.

qapi-code-gen.txt specifies the name case rule applies to the name as
it appears in the schema.  check_clash() checks c_name(name) instead.
No difference, as c_name() leaves alone case, but unclean.

Move the name case check into check_name_str(), less the c_name().
New argument @permit_upper suppresses it.  Pass permit_upper=True for
definitions (which are not members), and when the member's owner is
whitelisted with pragma name-case-whitelist.

Bonus: name-case-whitelist now applies to a union's inline base, too.
Update qapi/qapi-schema.json pragma to whitelist union CpuInfo instead
of CpuInfo's implicit base type's name q_obj_CpuInfo-base.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-6-armbru@redhat.com>
2019-09-28 17:17:18 +02:00
Markus Armbruster 7be6c51194 qapi: Prefix frontend errors with an "in definition" line
We take pains to include the offending expression in error messages,
e.g.

    tests/qapi-schema/alternate-any.json:2: alternate 'Alt' member 'one' cannot use type 'any'

But not always:

    tests/qapi-schema/enum-if-invalid.json:2: 'if' condition must be a string or a list of strings

Instead of improving them one by one, report the offending expression
whenever it is known, like this:

    tests/qapi-schema/enum-if-invalid.json: In enum 'TestIfEnum':
    tests/qapi-schema/enum-if-invalid.json:2: 'if' condition must be a string or a list of strings

Error messages that mention the offending expression become a bit
redundant, e.g.

    tests/qapi-schema/alternate-any.json: In alternate 'Alt':
    tests/qapi-schema/alternate-any.json:2: alternate 'Alt' member 'one' cannot use type 'any'

I'll take care of that later in this series.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-5-armbru@redhat.com>
2019-09-28 17:17:18 +02:00
Markus Armbruster 19e950d9d4 qapi: New QAPISourceInfo, replacing dict
We track source locations with a dict of the form

    {'file': FNAME, 'line': LINENO, 'parent': PARENT}

where PARENT is None for the main file, and the include directive's
source location for included files.

This is serviceable enough, but the next commit will add information,
and that's going to come out cleaner if we turn this into a class.  So
do that.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-4-armbru@redhat.com>
2019-09-28 17:17:18 +02:00
Markus Armbruster 57608a5299 qapi: Rename .owner to .defined_in
QAPISchemaMember.owner is the name of the defining entity.  That's a
confusing name when an object type inherits members from a base type.
Rename it to .defined_in.  Rename .set_owner() and ._pretty_owner() to
match.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-3-armbru@redhat.com>
2019-09-28 17:17:18 +02:00
Markus Armbruster 0ca7b11709 qapi: Tighten QAPISchemaFOO.check() assertions
When we introduced the QAPISchema intermediate representation (commit
ac88219a6c), we took a shortcut: we left check_exprs() & friends
alone instead of moving semantic checks into the
QAPISchemaFOO.check().  check_exprs() still checks and reports errors,
and the .check() assert check_exprs() did the job.  There are a few
gaps, though.

QAPISchemaArrayType.check() neglects to assert the element type is not
an array.  Add the assertion.

QAPISchemaObjectTypeVariants.check() neglects to assert the tag member
is not optional.  Add the assertion.

It neglects to assert the tag member is not conditional.  Add the
assertion.

It neglects to assert we actually have variants.  Add the assertion.

It asserts the variants are object types, but neglects to assert they
don't have variants.  Tighten the assertion.

QAPISchemaObjectTypeVariants.check_clash() has the same issue.
However, it can run only after .check().  Delete the assertion instead
of tightening it.

QAPISchemaAlternateType.check() neglects to assert the branch types
don't conflict.  Fixing that isn't trivial, so add just a TODO comment
for now.  It'll be resolved later in this series.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-2-armbru@redhat.com>
2019-09-28 17:17:18 +02:00
Markus Armbruster 56176412e7 qapi: Assert .visit() and .check_clash() run only after .check()
Easy since the previous commit provides .checked.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-20-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-09-24 14:07:23 +02:00
Markus Armbruster f9d1743b9b qapi: Fix excessive QAPISchemaEntity.check() recursion
Entity checking goes back to commit ac88219a6c "qapi: New QAPISchema
intermediate representation", v2.5.0.  It's designed to work as
follows: QAPISchema.check() calls .check() for all the schema's
entities.  An entity's .check() recurses into another entity's
.check() only if the C struct generated for the former contains the C
struct generated for the latter (pointers don't count).  This is used
to detect "object contains itself".

There are two instances of this:

* An object's C struct contains its base's C struct

  QAPISchemaObjectType.check() calls self.base.check()

* An object's C struct contains its variants' C structs

  QAPISchemaObjectTypeVariants().check calls v.type.check().  Since
  commit b807a1e1e3 "qapi: Check for QAPI collisions involving variant
  members", v2.6.0.

Thus, only object types can participate in recursion.

QAPISchemaObjectType.check() is made for that: it checks @self when
called the first time, recursing into base and variants, it reports an
"contains itself" error when this recursion reaches an object being
checked, and does nothing it reaches an object that has been checked
already.

The other .check() may safely assume they get called exactly once.

Sadly, this design has since eroded:

* QAPISchemaCommand.check() and QAPISchemaEvent.check() call
  .args_type.check().  Since commit c818408e44 "qapi: Implement boxed
  types for commands/events", v2.7.0.  Harmless, since args_type can
  only be an object type.

* QAPISchemaEntity.check() calls ._ifcond.check() when inheriting the
  condition from another type.  Since commit 4fca21c1b0 qapi: leave
  the ifcond attribute undefined until check(), v3.0.0.  This makes
  simple union wrapper types recurse into the wrapped type (nothing
  else uses this condition inheritance).  The .check() of types used
  as simple union branch type get called multiple times.

* QAPISchemaObjectType.check() calls its super type's .check()
  *before* the conditional handling multiple calls.  Also since commit
  4fca21c1b0.  QAPISchemaObjectType.check()'s guard against multiple
  checking doesn't protect QAPISchemaEntity.check().

* QAPISchemaArrayType.check() calls .element_type.check().  Also since
  commit 4fca21c1b0.  The .check() of types used as array element
  types get called multiple times.

  Commit 56a4689582 "qapi: Fix array first used in a different module"
  (v4.0.0) added more code relying on this .element_type.check().

The absence of explosions suggests the .check() involved happen to be
effectively idempotent.

Fix the unwanted recursion anyway:

* QAPISchemaCommand.check() and QAPISchemaEvent.check() calling
  .args_type.check() is unnecessary.  Delete the calls.

* Fix QAPISchemaObjectType.check() to call its super type's .check()
  after the conditional handling multiple calls.

* A QAPISchemaEntity's .ifcond becomes valid at .check().  This is due
  to arrays and simple unions.

  Most types get ifcond and info passed to their constructor.

  Array types don't: they get it from their element type, which
  becomes known only in .element_type.check().

  The implicit wrapper object types for simple union branches don't:
  they get it from the wrapped type, which might be an array.

  Ditch the idea to set .ifcond in .check().  Instead, turn it into a
  property and compute it on demand.  Safe because it's only used
  after the schema has been checked.

  Most types simply return the ifcond passed to their constructor.

  Array types forward to their .element_type instead, and the wrapper
  types forward to the wrapped type.

* A QAPISchemaEntity's .module becomes valid at .check().  This is
  because computing it needs info and schema.fname, and because array
  types get it from their element type instead.

  Make it a property just like .ifcond.

Additionally, have QAPISchemaEntity.check() assert it gets called at
most once, so the design invariant will stick this time.  Neglecting
that was entirely my fault.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-19-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[Commit message tidied up]
2019-09-24 14:07:23 +02:00
Markus Armbruster b1bc31f4b7 qapi: Fix to .check() empty structs just once
QAPISchemaObjectType.check() does nothing for types that have been
checked already.  Except the "has been checked" predicate is broken
for empty types: self.members is [] then, which isn't true.  The bug
is harmless, but fix it anyway: use self.member is not None instead.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-18-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-09-24 14:07:23 +02:00
Markus Armbruster e31fe1266c qapi: Delete useless check_exprs() code for simple union kind
Commit bceae7697f "qapi script: support enum type as discriminator in
union" made check_exprs() add the implicit enum types of simple unions
to global @enum_types.  I'm not sure it was needed even then.  It's
certainly not needed now.  Delete it.

discriminator_find_enum_define() and add_name() parameter @implicit
are now dead.  Bury them.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-17-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-09-24 14:07:23 +02:00
Markus Armbruster 6955397677 qapi: Clean up around check_known_keys()
All callers pass a dict argument to @keys, except check_keys() passes
a dict's .keys().  Drop .keys() there, and rename parameter @keys to
@value.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-16-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-09-24 14:07:23 +02:00
Markus Armbruster dc234189f8 qapi: Simplify check_keys()
check_keys() parameter expr_elem expects a dictionary with keys 'expr'
and 'info'.  Passing the two values separately is simpler, so do that.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-15-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-09-24 14:07:23 +02:00
Markus Armbruster fe9c4dcf90 qapi: Normalize 'if' in check_exprs(), like other sugar
We normalize shorthand to longhand forms in check_expr(): enumeration
values with normalize_enum(), feature values with
normalize_features(), struct members, union branches and alternate
branches with normalize_members().  If conditions are an exception: we
normalize them in QAPISchemaEntity.check() and
QAPISchemaMember.__init(), with listify_cond().  The idea goes back to
commit 2cbc94376e "qapi: pass 'if' condition into QAPISchemaEntity
objects", v3.0.0.

Normalize in check_expr() instead, with new helper normalize_if().

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-14-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-09-24 14:07:23 +02:00
Markus Armbruster dec0012ef8 qapi: Fix missing 'if' checks in struct, union, alternate 'data'
Commit 87adbbffd4..3e270dcacc "qapi: Add 'if' to (implicit
struct|union|alternate) members" (v4.0.0) neglected test coverage, and
promptly failed to check the conditions.  Review fail.

Recent commit "tests/qapi-schema: Demonstrate insufficient 'if'
checking" added test coverage, demonstrating the bug.  Fix it by add
the missing check_if().

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-13-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-09-24 14:07:23 +02:00
Markus Armbruster c2c7065e17 qapi: Reject blank 'if' conditions in addition to empty ones
"'if': 'COND'" generates "#if COND".  We reject empty COND because it
won't compile.  Blank COND won't compile any better, so reject that,
too.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-12-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-09-24 14:07:23 +02:00
Markus Armbruster 887a2069f7 qapi: Fix broken discriminator error messages
check_union() checks the discriminator exists in base and makes sense.
Two error messages mention the base.  These are broken for anonymous
bases, as demonstrated by tests flat-union-invalid-discriminator and
flat-union-invalid-if-discriminator.err.  The third one doesn't
bother.

First broken when commit ac4338f8eb "qapi: Allow anonymous base for
flat union" (v2.6.0) neglected to adjust the "not a member of base"
error message.  Commit ccadd6bcba "qapi: Add 'if' to implicit struct
members" (v4.0.0) then cloned the flawed error message.

Dumb them down not to mention the base.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-11-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-09-24 14:07:23 +02:00
Markus Armbruster 9d55380b5a qapi: Remove null from schema language
We represent the parse tree as OrderedDict.  We fetch optional dict
members with .get().  So far, so good.

We represent null literals as None.  .get() returns None both for
"absent" and for "present, value is the null literal".  Uh-oh.

Test features-if-invalid exposes this bug: "'if': null" is
misinterpreted as absent "if".

We added null to the schema language to "allow [...] an explicit
default value" (commit e53188ada5 "qapi: Allow true, false and null in
schema json", v2.4.0).  Hasn't happened; null is still unused except
as generic invalid value in tests/.

To fix, we'd have to replace .get() by something more careful, or
represent null differently.  Feasible, but we got more and bigger fish
to fry right now.  Remove the null literal from the schema language.
Replace null in tests by another invalid value.

Test features-if-invalid now behaves as it should.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-10-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-09-24 14:07:23 +02:00
Markus Armbruster 14c3279502 qapi: Improve reporting of lexical errors
Show text up to next structural character, whitespace, or quote
character instead of just the first character.

Forgotten quotes now get reported like "Stray 'command'" instead of
"Stray 'c'".

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-9-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-09-24 14:07:23 +02:00
Markus Armbruster 9f5e6b088a qapi: Use quotes more consistently in frontend error messages
Consistently enclose error messages in double quotes.  Use single
quotes within, except for one case of "'".

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-8-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-09-24 14:07:23 +02:00
Markus Armbruster 8d40738d2f qapi: Tweak code to match docs/devel/qapi-code-gen.txt
The previous commit made qapi-code-gen.txt define "(top-level)
expression" as either "directive" or "definition".  The code still
uses "expression" when it really means "definition".  Tidy up.

The previous commit made qapi-code-gen.txt use "object" rather than
"dictionary".  The code still uses "dictionary".  Tidy up.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190913201349.24332-17-armbru@redhat.com>
2019-09-24 14:07:22 +02:00
Markus Armbruster 398969fe1c qapi: Adjust frontend errors to say enum value, not member
For consistency with docs/devel/qapi-code-gen.txt.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190913201349.24332-12-armbru@redhat.com>
2019-09-24 14:07:22 +02:00
Markus Armbruster 0ced9531f1 qapi: Permit omitting all flat union branches
Absent flat union branches default to the empty struct (since commit
800877bb16 "qapi: allow empty branches in flat unions").  But an
attempt to omit all of them is rejected with "Union 'FOO' has no
branches".  Harmless oddity, but it's easy to avoid, so do that.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190913201349.24332-11-armbru@redhat.com>
[Commit message typo fixed]
2019-09-24 14:07:22 +02:00
Markus Armbruster f03255362a qapi: Permit alternates with just one branch
A union or alternate without branches makes no sense and doesn't work:
it can't be instantiated.  A union or alternate with just one branch
works, but is degenerate.  We accept the former, but reject the
latter.  Weird.  docs/devel/qapi-code-gen.txt doesn't mention the
difference.  It claims an alternate definition is "is similar to a
simple union type".

Permit degenerate alternates to make them consistent with unions.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190913201349.24332-10-armbru@redhat.com>
2019-09-24 14:07:22 +02:00
Markus Armbruster 675b214bc6 qapi: Permit 'boxed' with empty type
We reject empty types with 'boxed': true.  We don't really need that
to work, but making it work is actually simpler than rejecting it, so
do that.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190913201349.24332-9-armbru@redhat.com>
2019-09-24 14:07:22 +02:00
Markus Armbruster 9b4416bfc1 qapi: Drop support for escape sequences other than \\
Since the previous commit restricted strings to printable ASCII,
\uXXXX's only use is obfuscation.  Drop it.

This leaves \\, \/, \', and \".  Since QAPI schema strings are all
names, and names are restricted to ASCII letters, digits, hyphen, and
underscore, none of them is useful.

The latter three have no test coverage.  Drop them.

Keep \\ to avoid (more) gratuitous incompatibility with JSON.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190913201349.24332-8-armbru@redhat.com>
2019-09-24 14:07:22 +02:00
Markus Armbruster 56a8caff92 qapi: Restrict strings to printable ASCII
RFC 8259 on string contents:

   All Unicode characters may be placed within the quotation marks,
   except for the characters that MUST be escaped: quotation mark,
   reverse solidus, and the control characters (U+0000 through
   U+001F).

The QAPI schema parser accepts both less and more than JSON: it
accepts only ASCII with \u (less), and accepts control characters
other than LF (new line) unescaped.  How it treats unescaped non-ASCII
input differs between Python 2 and Python 3.

Make it accept strictly less: require printable ASCII.  Drop support
for \b, \f, \n, \r, \t.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190913201349.24332-7-armbru@redhat.com>
2019-09-24 14:07:22 +02:00
Markus Armbruster b22e86585b qapi: Drop support for boxed alternate arguments
Commands and events can define their argument type inline (default) or
by referring to another type ('boxed': true, since commit c818408e44
"qapi: Implement boxed types for commands/events", v2.7.0).  The
unboxed inline definition is an (anonymous) struct type.  The boxed
type may be a struct, union, or alternate type.

The latter is problematic: docs/interop/qemu-spec.txt requires the
value of the 'data' key to be a json-object, but any non-degenerate
alternate type has at least one branch that isn't.

Fortunately, we haven't made use of alternates in this context outside
tests/.  Drop support for them.

QAPISchemaAlternateType.is_empty() is now unused.  Drop it, too.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190913201349.24332-4-armbru@redhat.com>
2019-09-24 14:07:22 +02:00
Markus Armbruster dcca907bed qapi: Drop check_type()'s redundant parameter @allow_optional
check_type() uses @allow_optional only when @value is a dictionary and
@allow_dict is True.  All callers that pass allow_dict=True also pass
allow_optional=True.

Therefore, @allow_optional is always True when check_type() uses it.
Drop the redundant parameter.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190913201349.24332-3-armbru@redhat.com>
2019-09-24 14:07:22 +02:00
Markus Armbruster 157dd36395 qapi: Simplify how QAPIDoc implements its state machine
QAPIDoc uses a state machine to for processing of documentation lines.
Its state is encoded as an enum QAPIDoc._state (well, as enum-like
class actually, thanks to our infatuation with Python 2).

All we ever do with the state is calling the state's function to
process a line of documentation.  The enum values effectively serve as
handles for the functions.

Eliminate the rather wordy indirection: store the function to call in
QAPIDoc._append_line.  Update and improve comments.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190606153803.5278-8-armbru@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
[Commit message typo fixed]
2019-06-12 18:37:17 +02:00
Kevin Wolf f3ed93d545 qapi: Allow documentation for features
Features will be documented in a new part introduced by a "Features:"
line, after arguments and before named sections.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20190606153803.5278-6-armbru@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2019-06-12 18:35:57 +02:00
Kevin Wolf 03bf06bdc1 qapi: Disentangle QAPIDoc code
Documentation comments follow a certain structure: First, we have a text
with a general description (called QAPIDoc.body). After this,
descriptions of the arguments follow. Finally, we have a part that
contains various named sections.

The code doesn't show this structure, but just checks various attributes
that indicate indirectly which part is being processed, so it happens to
do the right set of things in the right phase. This is hard to follow,
and adding support for documentation of features would be even harder.

This patch restructures the code so that the three parts are clearly
separated. The code becomes a bit longer, but easier to follow. The
resulting output remains unchanged.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20190606153803.5278-5-armbru@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2019-06-12 18:35:31 +02:00
Kevin Wolf 6a8c0b5102 qapi: Add feature flags to struct types
Sometimes, the behaviour of QEMU changes without a change in the QMP
syntax (usually by allowing values or operations that previously
resulted in an error). QMP clients may still need to know whether
they can rely on the changed behavior.

Let's add feature flags to the QAPI schema language, so that we can make
such changes visible with schema introspection.

An example for a schema definition using feature flags looks like this:

    { 'struct': 'TestType',
      'data': { 'number': 'int' },
      'features': [ 'allow-negative-numbers' ] }

Introspection information then looks like this:

    { "name": "TestType", "meta-type": "object",
      "members": [
          { "name": "number", "type": "int" } ],
      "features": [ "allow-negative-numbers" ] }

This patch implements feature flags only for struct types. We'll
implement them more widely as needed.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20190606153803.5278-2-armbru@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2019-06-12 18:34:26 +02:00
Markus Armbruster a8d2532645 Include qemu-common.h exactly where needed
No header includes qemu-common.h after this commit, as prescribed by
qemu-common.h's file comment.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190523143508.25387-5-armbru@redhat.com>
[Rebased with conflicts resolved automatically, except for
include/hw/arm/xlnx-zynqmp.h hw/arm/nrf51_soc.c hw/arm/msf2-soc.c
block/qcow2-refcount.c block/qcow2-cluster.c block/qcow2-cache.c
target/arm/cpu.h target/lm32/cpu.h target/m68k/cpu.h target/mips/cpu.h
target/moxie/cpu.h target/nios2/cpu.h target/openrisc/cpu.h
target/riscv/cpu.h target/tilegx/cpu.h target/tricore/cpu.h
target/unicore32/cpu.h target/xtensa/cpu.h; bsd-user/main.c and
net/tap-bsd.c fixed up]
2019-06-12 13:20:20 +02:00
Markus Armbruster 0b8fa32f55 Include qemu/module.h where needed, drop it from qemu-common.h
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190523143508.25387-4-armbru@redhat.com>
[Rebased with conflicts resolved automatically, except for
hw/usb/dev-hub.c hw/misc/exynos4210_rng.c hw/misc/bcm2835_rng.c
hw/misc/aspeed_scu.c hw/display/virtio-vga.c hw/arm/stm32f205_soc.c;
ui/cocoa.m fixed up]
2019-06-12 13:18:33 +02:00
Markus Armbruster 56a4689582 qapi: Fix array first used in a different module
We generally put implicitly defined types in whatever module triggered
their definition.  This is wrong for array types, as the included test
case demonstrates.  Let's have a closer look at it.

Type 'Status' is defined sub-sub-module.json.  Array type ['Status']
occurs in main module qapi-schema-test.json and in
include/sub-module.json.  The main module's use is first, so the array
type gets put into the main module.

The generated C headers define StatusList in qapi-types.h.  But
include/qapi-types-sub-module.h uses it without including
qapi-types.h.  Oops.

To fix that, put the array type into its element type's module.

Now StatusList gets generated into qapi-types-sub-module.h, which all
its users include.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190301154051.23317-8-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-03-05 14:43:11 +01:00
Markus Armbruster 709395f8f6 qapi: Fix code generation for sub-modules in other directories
The #include directives to pull in sub-modules use file names relative
to the main module.  Works only when all modules are in the same
directory, or the main module's output directory is in the compiler's
include path.  Use relative file names instead.

The dummy variable we generate to avoid empty .o files has an invalid
name for sub-modules in other directories.  Fix that.

Both messed up in commit 252dc3105f "qapi: Generate separate .h, .c
for each module".  Escaped testing because tests/qapi-schema-test.json
doesn't cover sub-modules in other directories, only
tests/qapi-schema/include-relpath.json does, and we generate and
compile C code only for the former, not the latter.  Fold the latter
into the former.  This would have caught the mistakes fixed in this
commit.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190301154051.23317-5-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-03-05 14:43:11 +01:00
Markus Armbruster dddee4d7ba qapi: Pass file name to QAPIGen constructor instead of methods
Not much of an improvement now, but the next commit will profit.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190301154051.23317-4-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-03-05 14:43:11 +01:00
Markus Armbruster 093e367951 Revert "qapi-events: add 'if' condition to implicit event enum"
This reverts commit 7bd2634905.

The commit applied the events' conditions to the members of enum
QAPIEvent.  Awkward, because it renders QAPIEvent unusable in
target-independent code as soon as we make an event target-dependent.
Reverting this has the following effects:

* ui/vnc.c can remain target independent.

* monitor_qapi_event_conf[] doesn't have to muck around with #ifdef.

* query-events again doesn't reflect conditionals.  I'm going to
  deprecate it in favor of query-qmp-schema.

Another option would be to split target-dependent parts off enum
QAPIEvent into a target-dependent enum.  Doesn't seem worthwhile right
now.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20190214152251.2073-17-armbru@redhat.com>
2019-02-18 14:44:05 +01:00
Markus Armbruster 5d75648b56 qapi: Generate QAPIEvent stuff into separate files
Having to include qapi-events.h just for QAPIEvent is suboptimal, but
quite tolerable now.  It'll become problematic when we have events
conditional on the target, because then qapi-events.h won't be usable
from target-independent code anymore.  Avoid that by generating it
into separate files.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20190214152251.2073-6-armbru@redhat.com>
2019-02-18 14:44:04 +01:00
Markus Armbruster c2e196a9b4 qapi: Prepare for system modules other than 'builtin'
The next commit wants to generate qapi-emit-events.{c.h}.  To enable
that, extend QAPISchemaModularCVisitor to support additional "system
modules", i.e. modules that don't correspond to a (user-defined) QAPI
schema module.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20190214152251.2073-5-armbru@redhat.com>
2019-02-18 14:44:04 +01:00
Markus Armbruster dcac64711e qapi: Clean up modular built-in code generation a bit
We neglect to call .visit_module() for the special module we use for
built-ins.  Harmless, but clean it up anyway.  The
tests/qapi-schema/*.out now show the built-in module as 'module None'.

Subclasses of QAPISchemaModularCVisitor need to ._add_module() this
special module to enable code generation for built-ins.  When this
hasn't been done, QAPISchemaModularCVisitor.visit_module() does
nothing for the special module.  That looks like built-ins could
accidentally be generated into the wrong module when a subclass
neglects to call ._add_module().  Can't happen, because built-ins are
all visited before any other module.  But that's non-obvious.  Switch
off code generation explicitly.

Rename QAPISchemaModularCVisitor._begin_module() to
._begin_user_module().

New QAPISchemaModularCVisitor._is_builtin_module(), for clarity.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20190214152251.2073-4-armbru@redhat.com>
2019-02-18 14:44:04 +01:00
Markus Armbruster a95291007b qapi: Eliminate indirection through qmp_event_get_func_emit()
The qapi_event_send_FOO() functions emit events like this:

    QMPEventFuncEmit emit;

    emit = qmp_event_get_func_emit();
    if (!emit) {
        return;
    }

    qmp = qmp_event_build_dict("FOO");
    [put event arguments into @qmp...]

    emit(QAPI_EVENT_FOO, qmp);

The value of qmp_event_get_func_emit() depends only on the program:

* In qemu-system-FOO, it's always monitor_qapi_event_queue.

* In tests/test-qmp-event, it's always event_test_emit.

* In all other programs, it's always null.

This is exactly the kind of dependence the linker is supposed to
resolve; we don't actually need an indirection.

Note that things would fall apart if we linked more than one QAPI
schema into a single program: each set of qapi_event_send_FOO() uses
its own event enumeration, yet they share a single emit function.
Which takes the event enumeration as an argument.  Which one if
there's more than one?

More seriously: how does this work even now?  qemu-system-FOO wants
QAPIEvent, and passes a function taking that to
qmp_event_set_func_emit().  test-qmp-event wants test_QAPIEvent, and
passes a function taking that to qmp_event_set_func_emit().

It works by type trickery, of course:

    typedef void (*QMPEventFuncEmit)(unsigned event, QDict *dict);

    void qmp_event_set_func_emit(QMPEventFuncEmit emit);

    QMPEventFuncEmit qmp_event_get_func_emit(void);

We use unsigned instead of the enumeration type.  Relies on both
enumerations boiling down to unsigned, which happens to be true for
the compilers we use.

Clean this up as follows:

* Generate qapi_event_send_FOO() that call PREFIX_qapi_event_emit()
  instead of the value of qmp_event_set_func_emit().

* Generate a prototype for PREFIX_qapi_event_emit() into
  qapi-events.h.

* PREFIX_ is empty for qapi/qapi-schema.json, and test_ for
  tests/qapi-schema/qapi-schema-test.json.  It's qga_ for
  qga/qapi-schema.json, and doc-good- for
  tests/qapi-schema/doc-good.json, but those don't define any events.

* Rename monitor_qapi_event_queue() to qapi_event_emit() instead of
  passing it to qmp_event_set_func_emit().  This takes care of
  qemu-system-FOO.

* Rename event_test_emit() to test_qapi_event_emit() instead of
  passing it to qmp_event_set_func_emit().  This takes care of
  tests/test-qmp-event.

* Add a qapi_event_emit() that does nothing to stubs/monitor.c.  This
  takes care of all other programs that link code emitting QMP events.

* Drop qmp_event_set_func_emit(), qmp_event_get_func_emit().

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20181218182234.28876-3-armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
[Commit message typos fixed]
2019-01-24 10:01:05 +01:00
Marc-André Lureau ce1a1aec47 qapi: fix flat union on uncovered branches conditionals
Default branches variant should use the member conditional.

This fixes compilation with --disable-replication.

Fixes: 335d10cd8e

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20181217204046.14861-1-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Long line wrapped]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-12-18 07:57:36 +01:00
Marc-André Lureau 01ae9cc254 qapi: add condition to variants documentation
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20181213123724.4866-21-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-12-14 06:52:48 +01:00
Marc-André Lureau 8867bf0808 qapi: add 'If:' condition to struct members documentation
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20181213123724.4866-20-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-12-14 06:52:48 +01:00
Marc-André Lureau a35c9bf82a qapi: add 'If:' condition to enum values documentation
Use a common function to generate the "If:..." line.

While at it, get rid of the existing \n\n (no idea why it was
there). Use a line-break in member description, this seems to look
slightly better in the plaintext version.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20181213123724.4866-19-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-12-14 06:52:48 +01:00
Marc-André Lureau 8ee06f61e1 qapi: Add #if conditions to generated code members
Wrap generated enum and struct members and their supporting code with
#if/#endif, using the .ifcond members added in the previous patches.

We do enum and struct in a single patch because union tag enum and the
associated variants tie them together, and dealing with that to split
the patch doesn't seem worthwhile.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20181213123724.4866-18-marcandre.lureau@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-12-14 06:52:48 +01:00
Marc-André Lureau 3e270dcacc qapi: add 'if' to alternate members
Add 'if' key to alternate members:

{ 'alternate': 'TestIfAlternate', 'data':
  { 'alt': { 'type': 'TestStruct', 'if': 'COND' } } }

Generated code is not changed by this patch but with "qapi: add #if
conditions to generated code".

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20181213123724.4866-17-marcandre.lureau@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-12-13 19:20:11 +01:00
Marc-André Lureau a2724280fb qapi: add 'if' to union members
Add 'if' key to union members:

{ 'union': 'TestIfUnion', 'data':
    'mem': { 'type': 'str', 'if': 'COND'} }

The generated code remains unconditional for now. Later patches
generate the conditionals.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20181213123724.4866-16-marcandre.lureau@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-12-13 19:20:11 +01:00
Marc-André Lureau ccadd6bcba qapi: Add 'if' to implicit struct members
The generated code is for now *unconditional*.  Later patches generate
the conditionals.

Note that union discriminators may not have 'if' conditionals.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20181213123724.4866-14-marcandre.lureau@redhat.com>
Message-Id: <20181213123724.4866-15-marcandre.lureau@redhat.com>
[Patches squashed, commit message tweaked]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-12-13 19:20:11 +01:00
Marc-André Lureau 87adbbffd4 qapi: add a dictionary form for TYPE
Wherever a struct/union/alternate/command/event member with NAME: TYPE
form is accepted, desugar it to a NAME: { 'type': TYPE } form.

This will allow to add new member details, such as 'if' in the
following patch to introduce conditionals, or 'default' for default
values etc.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20181213123724.4866-13-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-12-13 19:20:11 +01:00
Marc-André Lureau 7bd2634905 qapi-events: add 'if' condition to implicit event enum
Add condition to QAPIEvent enum members based on the event 'if'.

The generated code remains unconditional for now. Later patches
generate the conditionals (also there is no additional coverage of
this change in qapi-schema-test.out since the event_names enum is an
implicit type created by qapi/events.py).

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20181213123724.4866-11-marcandre.lureau@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-12-13 19:20:11 +01:00
Marc-André Lureau 6cc32b0e14 qapi: add 'if' to enum members
QAPISchemaMember gains .ifcond for enum members: inherited classes,
such as QAPISchemaObjectTypeMember, will thus have an ifcond member
after this (those different types will also use the .ifcond to store
the condition and generate conditional code in the following patches).

The generated code remains unconditional for now. Later patches
generate the conditionals.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20181213123724.4866-10-marcandre.lureau@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-12-13 19:20:11 +01:00
Marc-André Lureau ea738b2168 qapi: add a dictionary form with 'name' key for enum members
Desugar the enum NAME form to { 'name': NAME }. This will allow to add
new enum members, such as 'if' in the following patch.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20181213123724.4866-7-marcandre.lureau@redhat.com>
Message-Id: <20181213123724.4866-8-marcandre.lureau@redhat.com>
Message-Id: <20181213123724.4866-9-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Harmless accidental move backed out, long line wrapped, patches
squashed]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-12-13 19:20:11 +01:00
Marc-André Lureau 7e80d48001 qapi: improve reporting of unknown or missing keys
Report the set of missing or unknown keys. And give a hint about the
accepted keys.

The error message for multiple meta type members (visible in
tests/qapi-schema/double-type.err) is not improved.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20181213123724.4866-6-marcandre.lureau@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-12-13 19:20:11 +01:00
Marc-André Lureau 563bd35d87 qapi: factor out checking for keys
Introduce a new helper function to check if the given keys are known,
and if mandatory keys are present. The function will be reused in
other places in the following code changes.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20181213123724.4866-5-marcandre.lureau@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-12-13 19:20:11 +01:00
Marc-André Lureau 1962bd39d5 qapi: change enum visitor and gen_enum* to take QAPISchemaMember
This will allow to add and access more properties associated with enum
values/members, like the associated 'if' condition. We may want to
have a specialized type QAPISchemaEnumMember, for now this will do.

Modify gen_enum() and gen_enum_lookup() for the same reason.

Suggested-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20181213123724.4866-3-marcandre.lureau@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-12-13 19:20:11 +01:00
Marc-André Lureau 9c2f56e9f9 qapi: Do not define enumeration value explicitly
The generated C enumeration types explicitly set the enumeration
constants to 0, 1, 2, ...  That's exactly what you get when you don't
supply values.

Drop the explicit values.  No change now, but it will avoid gaps in
the values when we later add support for 'if' conditions.  Avoiding
such gaps will save us the trouble of changing the ENUM_lookup[]
tables to work without a sentinel.

We'll have to take care to ensure the headers required by the 'if'
conditions get always included before the generated QAPI code.
Fortunately, our convention to include "qemu/osdep.h" first in any .c
ensures that's the case for our CONFIG_FOO macros.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20181213123724.4866-2-marcandre.lureau@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-12-13 19:20:11 +01:00
Marc-André Lureau 5751686364 qapi: rename QAPISchemaEnumType.values to .members
Rename QAPISchemaEnumType.values and related variables to members.
Makes sense ever since commit 93bda4dd4 changed .values from list of
string to list of QAPISchemaMember. Obvious no-op.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20181208111606.8505-4-marcandre.lureau@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-12-13 19:20:11 +01:00
Eric Blake 8c643361ee qapi: Add comments to aid debugging generated introspection
We consciously chose in commit 1a9a507b to hide QAPI type names
from the introspection output on the wire, but added a command
line option -u to unmask the type name when doing a debug build.
The unmask option still remains useful to some other forms of
automated analysis, so it will not be removed; however, when it
is not in use, the generated .c file can be hard to read.  At
the time when we first introduced masking, the generated file
consisted only of a monolithic C string, so there was no clean
way to inject any comments.

Later, in commit 7d0f982b, we switched the generation to output
a QLit object, in part to make it easier for future addition of
conditional compilation.  In fact, commit d626b6c1 took advantage
of this by passing a tuple instead of a bare object for encoding
the output of conditionals.  By extending that tuple, we can now
interject strategic comments.

For now, type name debug aid comments are only output once per
meta-type, rather than at all uses of the number used to encode
the type within the introspection data.  But this is still a lot
more convenient than having to regenerate the file with the
unmask operation temporarily turned on - merely search the
generated file for '"NNN" =' to learn the corresponding source
name and associated definition of type NNN.

The generated qapi-introspect.c changes only with the addition
of comments, such as:

| @@ -14755,6 +15240,7 @@
|          { "name", QLIT_QSTR("[485]"), },
|          {}
|      })),
| +    /* "485" = QCryptoBlockInfoLUKSSlot */
|      QLIT_QDICT(((QLitDictEntry[]) {
|          { "members", QLIT_QLIST(((QLitObject[]) {
|              QLIT_QDICT(((QLitDictEntry[]) {

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180827213943.33524-3-eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Rebased, update to qapi-code-gen.txt corrected]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-08-28 18:21:38 +02:00
Eric Blake 1aa806ccf0 qapi: Minor introspect.py cleanups
Commit 7d0f982b changed generated introspection output to no longer
produce long lines in the generated .c file, but failed to adjust
comments to match.  Add some clarity that the shorter length that
matters most is the overall QMP response on the wire.

Commit 25b1ef31 triggers a pep8 formatting nit.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180827213943.33524-2-eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-08-28 18:21:38 +02:00
Markus Armbruster 5f1450f544 qapi: Emit a blank line before dummy declaration
We emit a dummy variable in each .c file "to shut up OSX toolchain
warnings about empty .o files" (commit 252dc3105f).  Separate it from
the code preceding it (if any) with a blank line.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20180828120736.32323-2-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2018-08-28 18:21:38 +02:00
Peter Xu 3ab72385b2 qapi: Drop qapi_event_send_FOO()'s Error ** argument
The generated qapi_event_send_FOO() take an Error ** argument.  They
can't actually fail, because all they do with the argument is passing it
to functions that can't fail: the QObject output visitor, and the
@qmp_emit callback, which is either monitor_qapi_event_queue() or
event_test_emit().

Drop the argument, and pass &error_abort to the QObject output visitor
and @qmp_emit instead.

Suggested-by: Eric Blake <eblake@redhat.com>
Suggested-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180815133747.25032-4-peterx@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Commit message rewritten, update to qapi-code-gen.txt corrected]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-08-28 18:21:38 +02:00
Markus Armbruster bdd2d42b89 qapi: Fix build_params() for empty parameter list
build_params() returns '' instead of 'void' when there are no
parameters.  Can't happen now, but the next commit will change that.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
[peterx: compose the patch from email replies]
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180815133747.25032-3-peterx@redhat.com>
2018-08-28 18:21:38 +02:00
Markus Armbruster b736e25a18 qapi: Fix some pycodestyle-3 complaints
Fix the following issues:

    common.py:873:13: E129 visually indented line with same indent as next logical line
    common.py:1766:5: E741 ambiguous variable name 'l'
    common.py:1784:1: E305 expected 2 blank lines after class or function definition, found 1
    common.py:1833:1: E305 expected 2 blank lines after class or function definition, found 1
    common.py:1843:1: E305 expected 2 blank lines after class or function definition, found 1
    visit.py:181:18: E127 continuation line over-indented for visual indent

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20180621083551.775-1-armbru@redhat.com>
[Fixup squashed in:]
Message-ID: <871sd0nzw9.fsf@dusky.pond.sub.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2018-08-15 07:24:22 +02:00
Markus Armbruster 25b1ef31db qapi: Make 'allow-oob' optional in SchemaInfoCommand
Making 'allow-oob' optional in SchemaInfoCommand permits omitting it
in the common case.  Shrinks query-qmp-schema's output from 122.1KiB
to 118.6KiB for me.

Note that out-of-band execution is still experimental (you have to
configure the monitor with x-oob=on to use it).

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20180718090557.17248-1-armbru@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
2018-07-23 13:57:52 +02:00
Markus Armbruster 1f214ee1b8 qapi: Do not expose "allow-preconfig" in query-qmp-schema
According to commit 047f7038f5, option --preconfig

    [...] allows pausing QEMU in the new RUN_STATE_PRECONFIG state,
    allowing the configuration of QEMU from QMP before the machine
    jumps into board initialization code of machine_run_board_init()

    The intent is to allow management to query machine state and
    additionally configure it using previous query results within one
    QEMU instance (i.e. eliminate the need to start QEMU twice, 1st to
    query board specific parameters and 2nd for actual VM start using
    query results for additional parameters).

The implementation is a bit of a hack: it splices in an additional
main loop before machine creation, in special runstate preconfig.  New
command exit-preconfig exits that main loop.  QEMU continues
initializing, creates the machine, and runs the good old main loop.
The replacement of the main loop is transparent to monitors.

Sadly, some commands expect initialization to be complete.  Running
them in --preconfig's main loop violates their preconditions.  Since
we don't really know which commands are safe, we use a whitelist.
This drags the concept of run state into the QMP core.

The whitelist is done as a command flag in the QAPI schema (commit
d6fe3d02e9).  Drags the concept of run state further into the QAPI
language.

The command flag is exposed in query-qmp-schema (also commit
d6fe3d02e9).  This makes it ABI.

I consider the whole thing an offensively ugly hack, but sometimes an
ugly hack is the best we can do to solve a problem people have.

The need described by the commit message quote above is genuine.  The
proper solution would be a main loop that permits complete
configuration via QMP.  This is out of reach, thus the hack.

However, even though the need is genuine, it isn't urgent: libvirt is
not going to use this anytime soon.  Baking a hack into ABI before it
has any users is a bad idea.

This commit reverts the parts of commit d6fe3d02e9 that affect ABI
via query-qmp-schema.  The commit did the following:

(1) Add command flag 'allow-preconfig' to the QAPI schema language

(2) Pass it to code generators

(3) Have the commands.py code generator pass it to the command
    registry (so commit 047f7038f5 can use it as whitelist)

(4) Add 'allow-preconfig' to SchemaInfoCommand (neglecting to update
    qapi-code-gen.txt section "Client JSON Protocol introspection")

(5) Set 'allow-preconfig': true for commands qmp_capabilities,
    query-commands, query-command-line-options, query-status

Revert exactly (4), plus a bit of documentation added to
qemu-tech.info in commit 047f7038f5.

Shrinks query-qmp-schema's output from 126.5KiB to 121.8KiB for me.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20180705091402.26244-2-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Acked-by: Eduardo Habkost <ehabkost@redhat.com>
Acked-by: Igor Mammedov <imammedo@redhat.com>
[Straightforward conflict with commit d626b6c1ae resolved]
2018-07-16 15:35:57 +02:00
Marc-André Lureau 901a34a400 qapi: add 'If:' section to generated documentation
The documentation is generated only once, and doesn't know C
pre-conditions. Add 'If:' sections for top-level entities.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20180703155648.11933-13-marcandre.lureau@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-07-03 18:38:54 +02:00
Markus Armbruster 9f88c66211 qapi-types: add #if conditions to types & visitors
Types & visitors are coupled and must be handled together to avoid
temporary build regression.

Wrap generated types/visitor code with #if/#endif using the context
helpers. Derived from a patch by Marc-André.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20180703155648.11933-12-marcandre.lureau@redhat.com>
2018-07-03 18:38:53 +02:00
Marc-André Lureau c3cd6aa020 qapi/events: add #if conditions to events
Wrap generated code with #if/#endif using an 'ifcontext' on
QAPIGenCSnippet objects.

This makes a conditional event's qapi_event_send_FOO() compile-time
conditional, but its enum QAPIEvent member remains unconditional for
now. A follow up patch "qapi-event: add 'if' condition to implicit
event enum" will improve this.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20180703155648.11933-11-marcandre.lureau@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-07-03 18:38:54 +02:00
Marc-André Lureau 1f7b9f3181 qapi/commands: add #if conditions to commands
Wrap generated code with #if/#endif using an 'ifcontext' on
QAPIGenCSnippet objects.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20180703155648.11933-10-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Line breaks tweaked]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-07-03 18:38:54 +02:00
Marc-André Lureau d626b6c1ae qapi-introspect: add preprocessor conditions to generated QLit
This commit adds 'ifcond' conditions to top-level QLit objects.
Future work will add them to object and enum type members, i.e. within
QLit objects.

Extend the QLit generator to_qlit() to accept (@obj, @cond) tuples in
addition to just @obj.  The tuple causes the QLit generated for
objects for @obj with #if/#endif conditions for @cond.

See generated tests/test-qmp-introspect.c. Example diff after this
patch:

    --- before	2018-01-08 11:55:24.757083654 +0100
    +++ tests/test-qmp-introspect.c	2018-01-08 13:08:44.477641629 +0100
    @@ -51,6 +51,8 @@
             { "name", QLIT_QSTR("EVENT_F"), },
             {}
         })),
    +#if defined(TEST_IF_CMD)
    +#if defined(TEST_IF_STRUCT)
         QLIT_QDICT(((QLitDictEntry[]) {
             { "arg-type", QLIT_QSTR("5"), },
             { "meta-type", QLIT_QSTR("command"), },
    @@ -58,12 +60,16 @@
             { "ret-type", QLIT_QSTR("0"), },
             {}
         })),
    +#endif /* defined(TEST_IF_STRUCT) */
    +#endif /* defined(TEST_IF_CMD) */

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20180703155648.11933-9-marcandre.lureau@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-07-03 18:38:54 +02:00
Marc-André Lureau 40bb13766a qapi-introspect: modify to_qlit() to append ',' on level > 0
The following patch is going to break list entries with #if/#endif, so
they should have the trailing ',' as suffix.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20180703155648.11933-8-marcandre.lureau@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-07-03 18:38:53 +02:00
Marc-André Lureau ded9fc28b5 qapi: add #if/#endif helpers
Add helpers to wrap generated code with #if/#endif lines.

A later patch wants to use QAPIGen for generating C snippets rather
than full C files with copyright headers etc.  Splice in class
QAPIGenCCode between QAPIGen and QAPIGenC.

Add a 'with' statement context manager that will be used to wrap
generator visitor methods.  The manager will check if code was
generated before adding #if/#endif lines on QAPIGenCSnippet
objects. Used in the following patches.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20180703155648.11933-7-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-07-03 18:38:53 +02:00
Marc-André Lureau 485d948ce8 qapi: mcgen() shouldn't indent # lines
Skip preprocessor lines when adding indentation, since that would
likely result in invalid code.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20180703155648.11933-6-marcandre.lureau@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-07-03 18:38:53 +02:00
Marc-André Lureau fbf09a2fa4 qapi: add 'ifcond' to visitor methods
Modify the test visitor to check correct passing of values.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20180703155648.11933-5-marcandre.lureau@redhat.com>
[Accidental change to roms/seabios dropped]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-07-03 18:38:53 +02:00
Marc-André Lureau 4fca21c1b0 qapi: leave the ifcond attribute undefined until check()
We commonly initialize attributes to None in .init(), then set their
real value in .check().  Accessing the attribute before .check()
yields None.  If we're lucky, the code that accesses the attribute
prematurely chokes on None.

It won't for .ifcond, because None is a legitimate value.

Leave the ifcond attribute undefined until check().

Suggested-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20180703155648.11933-4-marcandre.lureau@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-07-03 18:38:53 +02:00
Marc-André Lureau 2cbc94376e qapi: pass 'if' condition into QAPISchemaEntity objects
Built-in objects remain unconditional.  Explicitly defined objects use
the condition specified in the schema.  Implicitly defined objects
inherit their condition from their users.  For most of them, there is
exactly one user, so the condition to use is obvious.  The exception
is wrapped types generated for simple union variants, which can be
shared by any number of simple unions.  The tight condition would be
the disjunction of the conditions of these simple unions.  For now,
use the wrapped type's condition instead.  Much simpler and good
enough for now.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20180703155648.11933-3-marcandre.lureau@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-07-03 18:38:53 +02:00
Marc-André Lureau 967c885108 qapi: add 'if' to top-level expressions
Accept 'if' key in top-level elements, accepted as string or list of
string type. The following patches will modify the test visitor to
check the value is correctly saved, and generate #if/#endif code (as a
single #if/endif line or a series for a list).

Example of 'if' key:
{ 'struct': 'TestIfStruct', 'data': { 'foo': 'int' },
  'if': 'defined(TEST_IF_STRUCT)' }

The generated code is for now *unconditional*. Later patches generate
the conditionals.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20180703155648.11933-2-marcandre.lureau@redhat.com>
[Commit message and Documentation improved]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-07-03 18:21:24 +02:00
Markus Armbruster da112e83c1 qapi/introspect: Eliminate pointless variable in .visit_end()
Commit 1a9a507b2e "qapi-introspect: Hide type names" added local
variable @jsons to improve sorting, but also removed the sorting.  It
was part of a big series that went to v8, and it made sense until v2
or so...

Commit 7d0f982bfb replaced @jsons by @qlits, preserving the
uselessness.

Get rid of it.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20180620124742.16979-1-armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
2018-06-22 16:33:47 +02:00
Markus Armbruster de685ae5e9 qapi: Open files with encoding='utf-8'
Python 2 happily reads UTF-8 files in text mode, but Python 3 requires
either UTF-8 locale or an explicit encoding passed to open().  Commit
d4e5ec877c fixed this by setting the en_US.UTF-8 locale.  Falls apart
when the locale isn't be available.

Matthias Maier and Arfrever Frehtes Taifersar Arahesis proposed to use
binary mode instead, with manual conversion from bytes to str.  Works,
but opening with an explicit encoding is simpler, so do that.

Since Python 2's open() doesn't support the encoding parameter, we
need to suppress it with a version check.

Reported-by: Arfrever Frehtes Taifersar Arahesis <arfrever.fta@gmail.com>
Reported-by: Matthias Maier <tamiko@43-1.org>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20180618175958.29073-2-armbru@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2018-06-22 16:33:46 +02:00
Anton Nefedov 800877bb16 qapi: allow empty branches in flat unions
It often happens that just a few discriminator values imply extra data in
a flat union. Existing checks did not make possible to leave other values
uncovered. Such cases had to be worked around by either stating a dummy
(empty) type or introducing another (subset) discriminator enumeration.

Both options create redundant entities in qapi files for little profit.

With this patch it is not necessary anymore to add designated union
fields for every possible value of a discriminator enumeration.

Signed-off-by: Anton Nefedov <anton.nefedov@virtuozzo.com>
Message-Id: <1529311206-76847-2-git-send-email-anton.nefedov@virtuozzo.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-06-22 16:33:46 +02:00
Marc-André Lureau f030ffd39d qapi/events: generate event enum in main module
The event generator produces an enum, and put it in the last visited
module. It fits better in the main module, since it's the set of all
visited events, from all modules.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20180321115211.17937-3-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-06-22 16:33:46 +02:00
Marc-André Lureau a48e7542be qapi/visit: remove useless prefix argument
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20180321115211.17937-2-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-06-22 16:33:46 +02:00
Eduardo Habkost c788341244 python: Remove scripts/ordereddict.py
Python 2.7 (the minimum Python version we require) provides
collections.OrderedDict on the standard library, so we don't need
to carry our own implementation.

Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Message-Id: <20180608175252.25110-3-ehabkost@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2018-06-08 16:40:54 -03:00
Igor Mammedov d6fe3d02e9 qapi: introduce new cmd option "allow-preconfig"
New option will be used to allow commands, which are prepared/need
to run, during preconfig state. Other commands that should be able
to run in preconfig state, should be amended to not expect machine
in initialized state or deal with it.

For compatibility reasons, commands that don't use new flag
'allow-preconfig' explicitly are not permitted to run in
preconfig state but allowed in all other states like they used
to be.

Within this patch allow following commands in preconfig state:
   qmp_capabilities
   query-qmp-schema
   query-commands
   query-command-line-options
   query-status
   exit-preconfig
to allow qmp connection, basic introspection and moving to the next
state.

PS:
set-numa-node and query-hotpluggable-cpus will be enabled later in
a separate patches.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Message-Id: <1526057503-39287-1-git-send-email-imammedo@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[ehabkost: Changed "since 2.13" to "since 3.0"]
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2018-05-30 13:19:09 -03:00
Laszlo Ersek 9a801c7d6c qapi: add SysEmuTarget to "common.json"
We'll soon need an enumeration type that lists all the softmmu targets
that QEMU (the project) supports. Introduce @SysEmuTarget to
"common.json".

The enum constant @x86_64 doesn't match the QAPI convention of preferring
hyphen ("-") over underscore ("_"). This is intentional; the @SysEmuTarget
constants are supposed to produce QEMU executable names when stringified
and appended to the "qemu-system-" prefix. Put differently, the
replacement text of the TARGET_NAME preprocessor macro must be possible to
look up in the list of (stringified) enum constants.

Like other enum types, @SysEmuTarget too can be used for discriminator
fields in unions. For the @i386 constant, a C-language union member called
"i386" would be generated. On mingw build hosts, "i386" is a macro
however. Add "i386" to "polluted_words" at once.

Cc: "Daniel P. Berrange" <berrange@redhat.com>
Cc: Eric Blake <eblake@redhat.com>
Cc: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20180427192852.15013-3-lersek@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-05-04 08:27:53 +02:00
Marc-André Lureau cb3e7f08ae qobject: Replace qobject_incref/QINCREF qobject_decref/QDECREF
Now that we can safely call QOBJECT() on QObject * as well as its
subtypes, we can have macros qobject_ref() / qobject_unref() that work
everywhere instead of having to use QINCREF() / QDECREF() for QObject
and qobject_incref() / qobject_decref() for its subtypes.

The replacement is mechanical, except I broke a long line, and added a
cast in monitor_qmp_cleanup_req_queue_locked().  Unlike
qobject_decref(), qobject_unref() doesn't accept void *.

Note that the new macros evaluate their argument exactly once, thus no
need to shout them.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180419150145.24795-4-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Rebased, semantic conflict resolved, commit message improved]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-05-04 08:27:53 +02:00
Peter Xu 9408860165 qapi: restrict allow-oob value to be "true"
It was missed in the first version of OOB series.  We should check this
to make sure we throw the right error when fault value is passed in.

Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180326063901.27425-5-peterx@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-27 10:17:45 -05:00
Peter Xu 876c67512e qapi: introduce new cmd option "allow-oob"
Here "oob" stands for "Out-Of-Band".  When "allow-oob" is set, it means
the command allows out-of-band execution.

The "oob" idea is proposed by Markus Armbruster in following thread:

  https://lists.gnu.org/archive/html/qemu-devel/2017-09/msg02057.html

This new "allow-oob" boolean will be exposed by "query-qmp-schema" as
well for command entries, so that QMP clients can know which commands
can be used in out-of-band calls. For example the command "migrate"
originally looks like:

  {"name": "migrate", "ret-type": "17", "meta-type": "command",
   "arg-type": "86"}

And it'll be changed into:

  {"name": "migrate", "ret-type": "17", "allow-oob": false,
   "meta-type": "command", "arg-type": "86"}

This patch only provides the QMP interface level changes.  It does not
contain the real out-of-band execution implementation yet.

Suggested-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180309090006.10018-18-peterx@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[eblake: rebase on introspection done by qlit]
Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-19 14:58:37 -05:00
Marc-André Lureau 7d0f982bfb qapi: generate a literal qobject for introspection
Replace the generated json string with a literal qobject. The later is
easier to deal with, at run time as well as compile time: adding #if
conditionals will be easier than in a json string.

The output of query-qmp-schema is not changed.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20180305172951.2150-5-marcandre.lureau@redhat.com>
[eblake: fix python 3 failure]
Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-19 10:00:14 -05:00
Marc-André Lureau 26ee12ad1f qapi2texi: minor python code simplification
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20180305172951.2150-2-marcandre.lureau@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-19 10:00:14 -05:00
Markus Armbruster eb815e248f qapi: Move qapi-schema.json to qapi/, rename generated files
Move qapi-schema.json to qapi/, so it's next to its modules, and all
files get generated to qapi/, not just the ones generated for modules.

Consistently name the generated files qapi-MODULE.EXT:
qmp-commands.[ch] become qapi-commands.[ch], qapi-event.[ch] become
qapi-events.[ch], and qmp-introspect.[ch] become qapi-introspect.[ch].
This gets rid of the temporary hacks in scripts/qapi/commands.py,
scripts/qapi/events.py, and scripts/qapi/common.py.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20180211093607.27351-28-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com>
[eblake: Fix trailing dot in tpm.c, undo temporary hack for OSX toolchain]
Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02 13:45:57 -06:00
Markus Armbruster 9af2398977 Include less of the generated modular QAPI headers
In my "build everything" tree, a change to the types in
qapi-schema.json triggers a recompile of about 4800 out of 5100
objects.

The previous commit split up qmp-commands.h, qmp-event.h, qmp-visit.h,
qapi-types.h.  Each of these headers still includes all its shards.
Reduce compile time by including just the shards we actually need.

To illustrate the benefits: adding a type to qapi/migration.json now
recompiles some 2300 instead of 4800 objects.  The next commit will
improve it further.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20180211093607.27351-24-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
[eblake: rebase to master]
Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02 13:45:50 -06:00
Markus Armbruster 252dc3105f qapi: Generate separate .h, .c for each module
Our qapi-schema.json is composed of modules connected by include
directives, but the generated code is monolithic all the same: one
qapi-types.h with all the types, one qapi-visit.h with all the
visitors, and so forth.  These monolithic headers get included all
over the place.  In my "build everything" tree, adding a QAPI type
recompiles about 4800 out of 5100 objects.

We wouldn't write such monolithic headers by hand.  It stands to
reason that we shouldn't generate them, either.

Split up generated qapi-types.h to mirror the schema's modular
structure: one header per module.  Name the main module's header
qapi-types.h, and sub-module D/B.json's header D/qapi-types-B.h.

Mirror the schema's includes in the headers, so that qapi-types.h gets
you everything exactly as before.  If you need less, you can include
one or more of the sub-module headers.  To be exploited shortly.

Split up qapi-types.c, qapi-visit.h, qapi-visit.c, qmp-commands.h,
qmp-commands.c, qapi-event.h, qapi-event.c the same way.
qmp-introspect.h, qmp-introspect.c and qapi.texi remain monolithic.

The split of qmp-commands.c duplicates static helper function
qmp_marshal_output_str() in qapi-commands-char.c and
qapi-commands-misc.c.  This happens when commands returning the same
type occur in multiple modules.  Not worth avoiding.

Since I'm going to rename qapi-event.[ch] to qapi-events.[ch], and
qmp-commands.[ch] to qapi-commands.[ch], name the shards that way
already, to reduce churn.  This requires temporary hacks in
commands.py and events.py.  Similarly, c_name() must temporarily
be taught to munge '/' in common.py.  They'll go away with the rename.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20180211093607.27351-23-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[eblake: declare a dummy variable in each .c file, to shut up OSX
toolchain warnings about empty .o files, including hacking c_name()]
Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02 13:44:24 -06:00
Markus Armbruster f9c146399d qapi/common: Fix guardname() for funny filenames
guardname() fails to return a valid C identifier for arguments
containing anything but [A-Za-z0-9_.-'].  Fix that.  Don't bother
protecting ticklish identifiers; header guards are all-caps, and no
ticklish identifiers are.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180211093607.27351-22-armbru@redhat.com>
Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02 13:14:10 -06:00
Markus Armbruster cdb6610ae4 qapi/types qapi/visit: Generate built-in stuff into separate files
Linking code from multiple separate QAPI schemata into the same
program is possible, but involves some weirdness around built-in
types:

* We generate code for built-in types into .c only with option
  --builtins.  The user is responsible for generating code for exactly
  one QAPI schema per program with --builtins.

* We generate code for built-in types into .h regardless of
  --builtins, but guarded by #ifndef QAPI_VISIT_BUILTIN.  Because all
  copies of this code are exactly the same, including any combination
  of these headers works.

Replace this contraption by something more conventional: generate code
for built-in types into their very own files: qapi-builtin-types.c,
qapi-builtin-visit.c, qapi-builtin-types.h, qapi-builtin-visit.h, but
only with --builtins.  Obey --output-dir, but ignore --prefix for
them.

Make qapi-types.h include qapi-builtin-types.h.  With multiple
schemata you now have multiple qapi-types.[ch], but only one
qapi-builtin-types.[ch].  Same for qapi-visit.[ch] and
qapi-builtin-visit.[ch].

Bonus: if all you need is built-in stuff, you can include a much
smaller header.  To be exploited shortly.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20180211093607.27351-21-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com>
[eblake: fix octal constant for python 3]
Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02 13:14:10 -06:00
Markus Armbruster 71b3f0459c qapi: Make code-generating visitors use QAPIGen more
The use of QAPIGen is rather shallow so far: most of the output
accumulation is not converted.  Take the next step: convert output
accumulation in the code-generating visitor classes.  Helper functions
outside these classes are not converted.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20180211093607.27351-20-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com>
[eblake: rebase to earlier guardstart cleanup]
Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02 13:14:10 -06:00
Markus Armbruster 834a3f3498 qapi: Rename generated qmp-marshal.c to qmp-commands.c
All generated .c are named like their .h, except for qmp-marshal.c and
qmp-commands.h.  To add to the confusion, tests-qmp-commands.c falsely
matches generated test-qmp-commands.h.

Get rid of this unnecessary complication.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20180211093607.27351-19-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02 13:14:10 -06:00
Markus Armbruster cf40a0a5c2 qapi: Record 'include' directives in intermediate representation
The include directive permits modular QAPI schemata, but the generated
code is monolithic all the same.  To permit generating modular code,
the front end needs to pass more information on inclusions to the back
ends.  The commit before last added the necessary information to the
parse tree.  This commit adds it to the intermediate representation
and its QAPISchemaVisitor.  A later commit will use this to to
generate modular code.

New entity QAPISchemaInclude represents inclusions.  Call new visitor
method visit_include() for it, so visitors can see the sub-modules a
module includes.

Note that unlike other entities, QAPISchemaInclude has no name, and is
therefore not added to entity_dict.

New QAPISchemaEntity attribute @module names the entity's source file.
Call new visitor method visit_module() when it changes during a visit,
so visitors can keep track of the module being visited.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20180211093607.27351-18-armbru@redhat.com>
[eblake: avoid accidental deletion of self._predefining]
Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02 13:14:10 -06:00
Markus Armbruster 8a84767cc4 qapi: Generate in source order
The generators' conversion to visitors (merge commit 9e72681d16)
changed the processing order of entities from source order to
alphabetical order.  The next commit needs source order, so change it
back.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20180211093607.27351-17-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02 13:14:10 -06:00
Markus Armbruster 97f0249474 qapi: Record 'include' directives in parse tree
The parse tree is a list of expressions.  Except include expressions
currently get replaced by the included file's parse tree.

Instead of throwing away the include expression, keep it with the file
name expanded so you don't have to track the including file's
directory to make sense of it.

A future commit will put this include expression to use.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20180211093607.27351-16-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[eblake: fix check of expr after assignment]
Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02 13:14:09 -06:00
Markus Armbruster 4257053083 qapi: Concentrate QAPISchemaParser.exprs updates in .__init__()
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20180211093607.27351-15-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02 13:14:09 -06:00
Markus Armbruster 181feaf355 qapi: Lift error reporting from QAPISchema.__init__() to callers
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20180211093607.27351-14-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02 13:14:09 -06:00
Markus Armbruster 71a7510baf qapi/common: Eliminate QAPISchema.exprs
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20180211093607.27351-13-armbru@redhat.com>
Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02 13:14:09 -06:00
Markus Armbruster af97502ce9 qapi: Improve include file name reporting in error messages
Error messages print absolute file names of included files even if the
user gave a relative one on the command line:

    $ PYTHONPATH=scripts python -B tests/qapi-schema/test-qapi.py tests/qapi-schema/include-cycle.json
    In file included from tests/qapi-schema/include-cycle.json:1:
    In file included from /work/armbru/qemu/tests/qapi-schema/include-cycle-b.json:1:
    /work/armbru/qemu/tests/qapi-schema/include-cycle-c.json:1: Inclusion loop for include-cycle.json

Improve this to

    In file included from tests/qapi-schema/include-cycle.json:1:
    In file included from tests/qapi-schema/include-cycle-b.json:1:
    tests/qapi-schema/include-cycle-c.json:1: Inclusion loop for include-cycle.json

The error message when an include file can't be opened prints the
include directive's file name, which is relative to the including
file.  Change this to print the file name relative to the working
directory.  Visible in tests/qapi-schema/include-no-file.err.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20180211093607.27351-12-armbru@redhat.com>
Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02 13:14:09 -06:00
Markus Armbruster 907b846653 qapi: Touch generated files only when they change
A massive number of objects depends on QAPI-generated headers.  In my
"build everything" tree, it's roughly 4800 out of 5100.  This is
particularly annoying when only some of the generated files change,
say for a doc fix.

Improve qapi-gen.py to touch its output files only if they actually
change.  Rebuild time for a QAPI doc fix drops from many minutes to a
few seconds.  Rebuilds get faster for certain code changes, too.  For
instance, adding a simple QMP event now recompiles less than 200
instead of 4800 objects.  But adding a QAPI type is as bad as ever;
we've clearly got more work to do.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180211093607.27351-11-armbru@redhat.com>
Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com>
[eblake: fix octal constant for python3]
Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02 13:14:09 -06:00
Markus Armbruster 3b446a1817 qapi-gen: Convert from getopt to argparse
argparse is nicer to use than getopt, and gives us --help almost for
free.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20180211093607.27351-10-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
[eblake: Fix --output-dir editing accident]
Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02 13:14:09 -06:00
Markus Armbruster fb0bc835e5 qapi-gen: New common driver for code and doc generators
Whenever qapi-schema.json changes, we run six programs eleven times to
update eleven files.  Similar for qga/qapi-schema.json.  This is
silly.  Replace the six programs by a single program that spits out
all eleven files.

The programs become modules in new Python package qapi, along with the
helper library.  This requires moving them to scripts/qapi/.  While
moving them, consistently drop executable mode bits.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20180211093607.27351-9-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com>
[eblake: move change to one-line 'blurb' earlier in series, mention mode
bit change as intentional, update qapi-code-gen.txt to match actual
generated events.c file]
Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02 13:14:09 -06:00