2011-07-19 21:50:39 +02:00
|
|
|
#
|
|
|
|
# QAPI helper library
|
|
|
|
#
|
|
|
|
# Copyright IBM, Corp. 2011
|
2016-01-29 14:48:41 +01:00
|
|
|
# Copyright (c) 2013-2016 Red Hat Inc.
|
2011-07-19 21:50:39 +02:00
|
|
|
#
|
|
|
|
# Authors:
|
|
|
|
# Anthony Liguori <aliguori@us.ibm.com>
|
qapi.py: Restructure lexer and parser
The parser has a rather unorthodox structure:
Until EOF:
Read a section:
Generator function get_expr() yields one section after the
other, as a string. An unindented, non-empty line that
isn't a comment starts a new section.
Lexing:
Split section into a list of tokens (strings), with help
of generator function tokenize().
Parsing:
Parse the first expression from the list of tokens, with
parse(), throw away any remaining tokens.
In parse_schema(): record value of an enum, union or
struct key (if any) in the appropriate global table,
append expression to the list of expressions.
Return list of expressions.
Known issues:
(1) Indentation is significant, unlike in real JSON.
(2) Neither lexer nor parser have any idea of source positions. Error
reporting is hard, let's go shopping.
(3) The one error we bother to detect, we "report" via raise.
(4) The lexer silently ignores invalid characters.
(5) If everything in a section gets ignored, the parser crashes.
(6) The lexer treats a string containing a structural character exactly
like the structural character.
(7) Tokens trailing the first expression in a section are silently
ignored.
(8) The parser accepts any token in place of a colon.
(9) The parser treats comma as optional.
(10) parse() crashes on unexpected EOF.
(11) parse_schema() crashes when a section's expression isn't a JSON
object.
Replace this piece of original art by a thoroughly unoriginal design.
Takes care of (1), (2), (5), (6) and (7), and lays the groundwork for
addressing the others. Generated source files remain unchanged.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-4-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-27 17:41:55 +02:00
|
|
|
# Markus Armbruster <armbru@redhat.com>
|
2011-07-19 21:50:39 +02:00
|
|
|
#
|
2014-03-01 08:40:34 +01:00
|
|
|
# This work is licensed under the terms of the GNU GPL, version 2.
|
|
|
|
# See the COPYING file in the top-level directory.
|
2011-07-19 21:50:39 +02:00
|
|
|
|
2015-04-02 14:46:39 +02:00
|
|
|
import errno
|
2015-04-02 13:12:21 +02:00
|
|
|
import getopt
|
2014-05-02 15:52:35 +02:00
|
|
|
import os
|
2017-03-15 13:57:35 +01:00
|
|
|
import re
|
2015-05-14 14:50:47 +02:00
|
|
|
import string
|
2017-03-15 13:57:35 +01:00
|
|
|
import sys
|
|
|
|
from ordereddict import OrderedDict
|
2011-07-19 21:50:39 +02:00
|
|
|
|
2015-05-04 17:05:00 +02:00
|
|
|
builtin_types = {
|
2017-06-26 19:25:14 +02:00
|
|
|
'null': 'QTYPE_QNULL',
|
2013-07-08 16:14:21 +02:00
|
|
|
'str': 'QTYPE_QSTRING',
|
2017-06-07 18:35:58 +02:00
|
|
|
'int': 'QTYPE_QNUM',
|
|
|
|
'number': 'QTYPE_QNUM',
|
2013-07-08 16:14:21 +02:00
|
|
|
'bool': 'QTYPE_QBOOL',
|
2017-06-07 18:35:58 +02:00
|
|
|
'int8': 'QTYPE_QNUM',
|
|
|
|
'int16': 'QTYPE_QNUM',
|
|
|
|
'int32': 'QTYPE_QNUM',
|
|
|
|
'int64': 'QTYPE_QNUM',
|
|
|
|
'uint8': 'QTYPE_QNUM',
|
|
|
|
'uint16': 'QTYPE_QNUM',
|
|
|
|
'uint32': 'QTYPE_QNUM',
|
|
|
|
'uint64': 'QTYPE_QNUM',
|
|
|
|
'size': 'QTYPE_QNUM',
|
2015-12-02 06:20:46 +01:00
|
|
|
'any': None, # any QType possible, actually
|
qapi: Convert QType into QAPI built-in enum type
What's more meta than using qapi to define qapi? :)
Convert QType into a full-fledged[*] builtin qapi enum type, so
that a subsequent patch can then use it as the discriminator
type of qapi alternate types. Fortunately, the judicious use of
'prefix' in the qapi definition avoids churn to the spelling of
the enum constants.
To avoid circular definitions, we have to flip the order of
inclusion between "qobject.h" vs. "qapi-types.h". Back in commit
28770e0, we had the latter include the former, so that we could
use 'QObject *' for our implementation of 'any'. But that usage
also works with only a forward declaration, whereas the
definition of QObject requires QType to be a complete type.
[*] The type has to be builtin, rather than declared in
qapi/common.json, because we want to use it for alternates even
when common.json is not included. But since it is the first
builtin enum type, we have to add special cases to qapi-types
and qapi-visit to only emit definitions once, even when two
qapi files are being compiled into the same binary (the way we
already handled builtin list types like 'intList'). We may
need to revisit how multiple qapi files share common types,
but that's a project for another day.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1449033659-25497-4-git-send-email-eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-12-02 06:20:47 +01:00
|
|
|
'QType': 'QTYPE_QSTRING',
|
2013-07-08 16:14:21 +02:00
|
|
|
}
|
|
|
|
|
2017-03-15 13:56:51 +01:00
|
|
|
# Are documentation comments required?
|
|
|
|
doc_required = False
|
|
|
|
|
2015-05-04 17:05:23 +02:00
|
|
|
# Whitelist of commands allowed to return a non-dictionary
|
2017-03-15 13:56:54 +01:00
|
|
|
returns_whitelist = []
|
2015-05-04 17:05:23 +02:00
|
|
|
|
qapi: Enforce (or whitelist) case conventions on qapi members
We document that members of enums and objects should be
'lower-case', although we were not enforcing it. We have to
whitelist a few pre-existing entities that violate the norms.
Add three new tests to expose the new error message, each of
which first uses the whitelisted name 'UuidInfo' to prove the
whitelist works, then triggers the failure (this is the same
pattern used in the existing returns-whitelist.json test).
Note that by adding this check, we have effectively forbidden
an entity with a case-insensitive clash of member names, for
any entity that is not on the whitelist (although there is
still the possibility to clash via '-' vs. '_').
Not done here: a future patch should also add naming convention
support and whitelist exceptions for command, event, and type
names.
The additions to QAPISchemaMember.check_clash() check whether
info['name'] is in the whitelist (the top-most entity name at
the point 'info' tracks), rather than self.owner (the type,
possibly implicit, that directly owns the member), because it
is easier to maintain the whitelist by the names actually in
the user's .json file, rather than worrying about the names
of implicit types.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1449033659-25497-14-git-send-email-eblake@redhat.com>
[Simplified a bit as per discussion with Eric]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-12-02 06:20:57 +01:00
|
|
|
# Whitelist of entities allowed to violate case conventions
|
2017-03-15 13:56:55 +01:00
|
|
|
name_case_whitelist = []
|
qapi: Enforce (or whitelist) case conventions on qapi members
We document that members of enums and objects should be
'lower-case', although we were not enforcing it. We have to
whitelist a few pre-existing entities that violate the norms.
Add three new tests to expose the new error message, each of
which first uses the whitelisted name 'UuidInfo' to prove the
whitelist works, then triggers the failure (this is the same
pattern used in the existing returns-whitelist.json test).
Note that by adding this check, we have effectively forbidden
an entity with a case-insensitive clash of member names, for
any entity that is not on the whitelist (although there is
still the possibility to clash via '-' vs. '_').
Not done here: a future patch should also add naming convention
support and whitelist exceptions for command, event, and type
names.
The additions to QAPISchemaMember.check_clash() check whether
info['name'] is in the whitelist (the top-most entity name at
the point 'info' tracks), rather than self.owner (the type,
possibly implicit, that directly owns the member), because it
is easier to maintain the whitelist by the names actually in
the user's .json file, rather than worrying about the names
of implicit types.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1449033659-25497-14-git-send-email-eblake@redhat.com>
[Simplified a bit as per discussion with Eric]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-12-02 06:20:57 +01:00
|
|
|
|
2017-03-15 13:57:31 +01:00
|
|
|
enum_types = {}
|
2017-03-15 13:57:32 +01:00
|
|
|
struct_types = {}
|
2017-03-15 13:57:33 +01:00
|
|
|
union_types = {}
|
2015-05-04 17:05:17 +02:00
|
|
|
all_names = {}
|
|
|
|
|
2015-06-10 10:04:36 +02:00
|
|
|
#
|
|
|
|
# Parsing the schema into expressions
|
|
|
|
#
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2014-05-07 20:46:15 +02:00
|
|
|
def error_path(parent):
|
2017-03-15 13:57:08 +01:00
|
|
|
res = ''
|
2014-05-07 20:46:15 +02:00
|
|
|
while parent:
|
2017-03-15 13:57:08 +01:00
|
|
|
res = ('In file included from %s:%d:\n' % (parent['file'],
|
2014-05-07 20:46:15 +02:00
|
|
|
parent['line'])) + res
|
|
|
|
parent = parent['parent']
|
|
|
|
return res
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2017-01-13 15:41:25 +01:00
|
|
|
class QAPIError(Exception):
|
|
|
|
def __init__(self, fname, line, col, incl_info, msg):
|
2015-09-30 00:21:01 +02:00
|
|
|
Exception.__init__(self)
|
2017-01-13 15:41:25 +01:00
|
|
|
self.fname = fname
|
|
|
|
self.line = line
|
|
|
|
self.col = col
|
|
|
|
self.info = incl_info
|
2013-07-27 17:41:56 +02:00
|
|
|
self.msg = msg
|
|
|
|
|
|
|
|
def __str__(self):
|
2017-03-15 13:57:08 +01:00
|
|
|
loc = '%s:%d' % (self.fname, self.line)
|
2017-01-13 15:41:25 +01:00
|
|
|
if self.col is not None:
|
2017-03-15 13:57:08 +01:00
|
|
|
loc += ':%s' % self.col
|
|
|
|
return error_path(self.info) + '%s: %s' % (loc, self.msg)
|
2013-07-27 17:41:56 +02:00
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2017-01-13 15:41:25 +01:00
|
|
|
class QAPIParseError(QAPIError):
|
|
|
|
def __init__(self, parser, msg):
|
|
|
|
col = 1
|
|
|
|
for ch in parser.src[parser.line_pos:parser.pos]:
|
|
|
|
if ch == '\t':
|
|
|
|
col = (col + 7) % 8 + 1
|
|
|
|
else:
|
|
|
|
col += 1
|
|
|
|
QAPIError.__init__(self, parser.fname, parser.line, col,
|
|
|
|
parser.incl_info, msg)
|
2014-03-05 03:44:34 +01:00
|
|
|
|
2017-01-13 15:41:25 +01:00
|
|
|
|
|
|
|
class QAPISemError(QAPIError):
|
|
|
|
def __init__(self, info, msg):
|
|
|
|
QAPIError.__init__(self, info['file'], info['line'], None,
|
|
|
|
info['parent'], msg)
|
2014-03-05 03:44:34 +01:00
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
qapi: add qapi2texi script
As the name suggests, the qapi2texi script converts JSON QAPI
description into a texi file suitable for different target
formats (info/man/txt/pdf/html...).
It parses the following kind of blocks:
Free-form:
##
# = Section
# == Subsection
#
# Some text foo with *emphasis*
# 1. with a list
# 2. like that
#
# And some code:
# | $ echo foo
# | -> do this
# | <- get that
#
##
Symbol description:
##
# @symbol:
#
# Symbol body ditto ergo sum. Foo bar
# baz ding.
#
# @param1: the frob to frobnicate
# @param2: #optional how hard to frobnicate
#
# Returns: the frobnicated frob.
# If frob isn't frobnicatable, GenericError.
#
# Since: version
# Notes: notes, comments can have
# - itemized list
# - like this
#
# Example:
#
# -> { "execute": "quit" }
# <- { "return": {} }
#
##
That's roughly following the following EBNF grammar:
api_comment = "##\n" comment "##\n"
comment = freeform_comment | symbol_comment
freeform_comment = { "# " text "\n" | "#\n" }
symbol_comment = "# @" name ":\n" { member | tag_section | freeform_comment }
member = "# @" name ':' [ text ] "\n" freeform_comment
tag_section = "# " ( "Returns:", "Since:", "Note:", "Notes:", "Example:", "Examples:" ) [ text ] "\n" freeform_comment
text = free text with markup
Note that the grammar is ambiguous: a line "# @foo:\n" can be parsed
both as freeform_comment and as symbol_comment. The actual parser
recognizes symbol_comment.
See docs/qapi-code-gen.txt for more details.
Deficiencies and limitations:
- the generated QMP documentation includes internal types
- union type support is lacking
- type information is lacking in generated documentation
- doc comment error message positions are imprecise, they point
to the beginning of the comment.
- a few minor issues, all marked TODO/FIXME in the code
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20170113144135.5150-16-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[test-qapi.py tweaked to avoid trailing empty lines in .out]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2017-01-13 15:41:29 +01:00
|
|
|
class QAPIDoc(object):
|
|
|
|
class Section(object):
|
|
|
|
def __init__(self, name=None):
|
|
|
|
# optional section name (argument/member or section name)
|
|
|
|
self.name = name
|
|
|
|
# the list of lines for this section
|
|
|
|
self.content = []
|
|
|
|
|
|
|
|
def append(self, line):
|
|
|
|
self.content.append(line)
|
|
|
|
|
|
|
|
def __repr__(self):
|
2017-03-15 13:57:08 +01:00
|
|
|
return '\n'.join(self.content).strip()
|
qapi: add qapi2texi script
As the name suggests, the qapi2texi script converts JSON QAPI
description into a texi file suitable for different target
formats (info/man/txt/pdf/html...).
It parses the following kind of blocks:
Free-form:
##
# = Section
# == Subsection
#
# Some text foo with *emphasis*
# 1. with a list
# 2. like that
#
# And some code:
# | $ echo foo
# | -> do this
# | <- get that
#
##
Symbol description:
##
# @symbol:
#
# Symbol body ditto ergo sum. Foo bar
# baz ding.
#
# @param1: the frob to frobnicate
# @param2: #optional how hard to frobnicate
#
# Returns: the frobnicated frob.
# If frob isn't frobnicatable, GenericError.
#
# Since: version
# Notes: notes, comments can have
# - itemized list
# - like this
#
# Example:
#
# -> { "execute": "quit" }
# <- { "return": {} }
#
##
That's roughly following the following EBNF grammar:
api_comment = "##\n" comment "##\n"
comment = freeform_comment | symbol_comment
freeform_comment = { "# " text "\n" | "#\n" }
symbol_comment = "# @" name ":\n" { member | tag_section | freeform_comment }
member = "# @" name ':' [ text ] "\n" freeform_comment
tag_section = "# " ( "Returns:", "Since:", "Note:", "Notes:", "Example:", "Examples:" ) [ text ] "\n" freeform_comment
text = free text with markup
Note that the grammar is ambiguous: a line "# @foo:\n" can be parsed
both as freeform_comment and as symbol_comment. The actual parser
recognizes symbol_comment.
See docs/qapi-code-gen.txt for more details.
Deficiencies and limitations:
- the generated QMP documentation includes internal types
- union type support is lacking
- type information is lacking in generated documentation
- doc comment error message positions are imprecise, they point
to the beginning of the comment.
- a few minor issues, all marked TODO/FIXME in the code
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20170113144135.5150-16-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[test-qapi.py tweaked to avoid trailing empty lines in .out]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2017-01-13 15:41:29 +01:00
|
|
|
|
|
|
|
class ArgSection(Section):
|
2017-03-15 13:57:03 +01:00
|
|
|
def __init__(self, name):
|
|
|
|
QAPIDoc.Section.__init__(self, name)
|
|
|
|
self.member = None
|
|
|
|
|
|
|
|
def connect(self, member):
|
|
|
|
self.member = member
|
qapi: add qapi2texi script
As the name suggests, the qapi2texi script converts JSON QAPI
description into a texi file suitable for different target
formats (info/man/txt/pdf/html...).
It parses the following kind of blocks:
Free-form:
##
# = Section
# == Subsection
#
# Some text foo with *emphasis*
# 1. with a list
# 2. like that
#
# And some code:
# | $ echo foo
# | -> do this
# | <- get that
#
##
Symbol description:
##
# @symbol:
#
# Symbol body ditto ergo sum. Foo bar
# baz ding.
#
# @param1: the frob to frobnicate
# @param2: #optional how hard to frobnicate
#
# Returns: the frobnicated frob.
# If frob isn't frobnicatable, GenericError.
#
# Since: version
# Notes: notes, comments can have
# - itemized list
# - like this
#
# Example:
#
# -> { "execute": "quit" }
# <- { "return": {} }
#
##
That's roughly following the following EBNF grammar:
api_comment = "##\n" comment "##\n"
comment = freeform_comment | symbol_comment
freeform_comment = { "# " text "\n" | "#\n" }
symbol_comment = "# @" name ":\n" { member | tag_section | freeform_comment }
member = "# @" name ':' [ text ] "\n" freeform_comment
tag_section = "# " ( "Returns:", "Since:", "Note:", "Notes:", "Example:", "Examples:" ) [ text ] "\n" freeform_comment
text = free text with markup
Note that the grammar is ambiguous: a line "# @foo:\n" can be parsed
both as freeform_comment and as symbol_comment. The actual parser
recognizes symbol_comment.
See docs/qapi-code-gen.txt for more details.
Deficiencies and limitations:
- the generated QMP documentation includes internal types
- union type support is lacking
- type information is lacking in generated documentation
- doc comment error message positions are imprecise, they point
to the beginning of the comment.
- a few minor issues, all marked TODO/FIXME in the code
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20170113144135.5150-16-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[test-qapi.py tweaked to avoid trailing empty lines in .out]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2017-01-13 15:41:29 +01:00
|
|
|
|
|
|
|
def __init__(self, parser, info):
|
|
|
|
# self.parser is used to report errors with QAPIParseError. The
|
|
|
|
# resulting error position depends on the state of the parser.
|
|
|
|
# It happens to be the beginning of the comment. More or less
|
|
|
|
# servicable, but action at a distance.
|
|
|
|
self.parser = parser
|
|
|
|
self.info = info
|
|
|
|
self.symbol = None
|
|
|
|
self.body = QAPIDoc.Section()
|
|
|
|
# dict mapping parameter name to ArgSection
|
|
|
|
self.args = OrderedDict()
|
|
|
|
# a list of Section
|
|
|
|
self.sections = []
|
|
|
|
# the current section
|
|
|
|
self.section = self.body
|
|
|
|
|
|
|
|
def has_section(self, name):
|
|
|
|
"""Return True if we have a section with this name."""
|
|
|
|
for i in self.sections:
|
|
|
|
if i.name == name:
|
|
|
|
return True
|
|
|
|
return False
|
|
|
|
|
|
|
|
def append(self, line):
|
|
|
|
"""Parse a comment line and add it to the documentation."""
|
|
|
|
line = line[1:]
|
|
|
|
if not line:
|
|
|
|
self._append_freeform(line)
|
|
|
|
return
|
|
|
|
|
|
|
|
if line[0] != ' ':
|
|
|
|
raise QAPIParseError(self.parser, "Missing space after #")
|
|
|
|
line = line[1:]
|
|
|
|
|
|
|
|
# FIXME not nice: things like '# @foo:' and '# @foo: ' aren't
|
|
|
|
# recognized, and get silently treated as ordinary text
|
|
|
|
if self.symbol:
|
|
|
|
self._append_symbol_line(line)
|
2017-03-15 13:57:08 +01:00
|
|
|
elif not self.body.content and line.startswith('@'):
|
|
|
|
if not line.endswith(':'):
|
qapi: add qapi2texi script
As the name suggests, the qapi2texi script converts JSON QAPI
description into a texi file suitable for different target
formats (info/man/txt/pdf/html...).
It parses the following kind of blocks:
Free-form:
##
# = Section
# == Subsection
#
# Some text foo with *emphasis*
# 1. with a list
# 2. like that
#
# And some code:
# | $ echo foo
# | -> do this
# | <- get that
#
##
Symbol description:
##
# @symbol:
#
# Symbol body ditto ergo sum. Foo bar
# baz ding.
#
# @param1: the frob to frobnicate
# @param2: #optional how hard to frobnicate
#
# Returns: the frobnicated frob.
# If frob isn't frobnicatable, GenericError.
#
# Since: version
# Notes: notes, comments can have
# - itemized list
# - like this
#
# Example:
#
# -> { "execute": "quit" }
# <- { "return": {} }
#
##
That's roughly following the following EBNF grammar:
api_comment = "##\n" comment "##\n"
comment = freeform_comment | symbol_comment
freeform_comment = { "# " text "\n" | "#\n" }
symbol_comment = "# @" name ":\n" { member | tag_section | freeform_comment }
member = "# @" name ':' [ text ] "\n" freeform_comment
tag_section = "# " ( "Returns:", "Since:", "Note:", "Notes:", "Example:", "Examples:" ) [ text ] "\n" freeform_comment
text = free text with markup
Note that the grammar is ambiguous: a line "# @foo:\n" can be parsed
both as freeform_comment and as symbol_comment. The actual parser
recognizes symbol_comment.
See docs/qapi-code-gen.txt for more details.
Deficiencies and limitations:
- the generated QMP documentation includes internal types
- union type support is lacking
- type information is lacking in generated documentation
- doc comment error message positions are imprecise, they point
to the beginning of the comment.
- a few minor issues, all marked TODO/FIXME in the code
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20170113144135.5150-16-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[test-qapi.py tweaked to avoid trailing empty lines in .out]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2017-01-13 15:41:29 +01:00
|
|
|
raise QAPIParseError(self.parser, "Line should end with :")
|
|
|
|
self.symbol = line[1:-1]
|
|
|
|
# FIXME invalid names other than the empty string aren't flagged
|
|
|
|
if not self.symbol:
|
|
|
|
raise QAPIParseError(self.parser, "Invalid name")
|
|
|
|
else:
|
|
|
|
self._append_freeform(line)
|
|
|
|
|
2017-03-15 13:57:23 +01:00
|
|
|
def end_comment(self):
|
|
|
|
self._end_section()
|
|
|
|
|
qapi: add qapi2texi script
As the name suggests, the qapi2texi script converts JSON QAPI
description into a texi file suitable for different target
formats (info/man/txt/pdf/html...).
It parses the following kind of blocks:
Free-form:
##
# = Section
# == Subsection
#
# Some text foo with *emphasis*
# 1. with a list
# 2. like that
#
# And some code:
# | $ echo foo
# | -> do this
# | <- get that
#
##
Symbol description:
##
# @symbol:
#
# Symbol body ditto ergo sum. Foo bar
# baz ding.
#
# @param1: the frob to frobnicate
# @param2: #optional how hard to frobnicate
#
# Returns: the frobnicated frob.
# If frob isn't frobnicatable, GenericError.
#
# Since: version
# Notes: notes, comments can have
# - itemized list
# - like this
#
# Example:
#
# -> { "execute": "quit" }
# <- { "return": {} }
#
##
That's roughly following the following EBNF grammar:
api_comment = "##\n" comment "##\n"
comment = freeform_comment | symbol_comment
freeform_comment = { "# " text "\n" | "#\n" }
symbol_comment = "# @" name ":\n" { member | tag_section | freeform_comment }
member = "# @" name ':' [ text ] "\n" freeform_comment
tag_section = "# " ( "Returns:", "Since:", "Note:", "Notes:", "Example:", "Examples:" ) [ text ] "\n" freeform_comment
text = free text with markup
Note that the grammar is ambiguous: a line "# @foo:\n" can be parsed
both as freeform_comment and as symbol_comment. The actual parser
recognizes symbol_comment.
See docs/qapi-code-gen.txt for more details.
Deficiencies and limitations:
- the generated QMP documentation includes internal types
- union type support is lacking
- type information is lacking in generated documentation
- doc comment error message positions are imprecise, they point
to the beginning of the comment.
- a few minor issues, all marked TODO/FIXME in the code
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20170113144135.5150-16-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[test-qapi.py tweaked to avoid trailing empty lines in .out]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2017-01-13 15:41:29 +01:00
|
|
|
def _append_symbol_line(self, line):
|
|
|
|
name = line.split(' ', 1)[0]
|
|
|
|
|
2017-03-15 13:57:08 +01:00
|
|
|
if name.startswith('@') and name.endswith(':'):
|
qapi: add qapi2texi script
As the name suggests, the qapi2texi script converts JSON QAPI
description into a texi file suitable for different target
formats (info/man/txt/pdf/html...).
It parses the following kind of blocks:
Free-form:
##
# = Section
# == Subsection
#
# Some text foo with *emphasis*
# 1. with a list
# 2. like that
#
# And some code:
# | $ echo foo
# | -> do this
# | <- get that
#
##
Symbol description:
##
# @symbol:
#
# Symbol body ditto ergo sum. Foo bar
# baz ding.
#
# @param1: the frob to frobnicate
# @param2: #optional how hard to frobnicate
#
# Returns: the frobnicated frob.
# If frob isn't frobnicatable, GenericError.
#
# Since: version
# Notes: notes, comments can have
# - itemized list
# - like this
#
# Example:
#
# -> { "execute": "quit" }
# <- { "return": {} }
#
##
That's roughly following the following EBNF grammar:
api_comment = "##\n" comment "##\n"
comment = freeform_comment | symbol_comment
freeform_comment = { "# " text "\n" | "#\n" }
symbol_comment = "# @" name ":\n" { member | tag_section | freeform_comment }
member = "# @" name ':' [ text ] "\n" freeform_comment
tag_section = "# " ( "Returns:", "Since:", "Note:", "Notes:", "Example:", "Examples:" ) [ text ] "\n" freeform_comment
text = free text with markup
Note that the grammar is ambiguous: a line "# @foo:\n" can be parsed
both as freeform_comment and as symbol_comment. The actual parser
recognizes symbol_comment.
See docs/qapi-code-gen.txt for more details.
Deficiencies and limitations:
- the generated QMP documentation includes internal types
- union type support is lacking
- type information is lacking in generated documentation
- doc comment error message positions are imprecise, they point
to the beginning of the comment.
- a few minor issues, all marked TODO/FIXME in the code
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20170113144135.5150-16-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[test-qapi.py tweaked to avoid trailing empty lines in .out]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2017-01-13 15:41:29 +01:00
|
|
|
line = line[len(name)+1:]
|
|
|
|
self._start_args_section(name[1:-1])
|
2017-03-15 13:57:08 +01:00
|
|
|
elif name in ('Returns:', 'Since:',
|
qapi: add qapi2texi script
As the name suggests, the qapi2texi script converts JSON QAPI
description into a texi file suitable for different target
formats (info/man/txt/pdf/html...).
It parses the following kind of blocks:
Free-form:
##
# = Section
# == Subsection
#
# Some text foo with *emphasis*
# 1. with a list
# 2. like that
#
# And some code:
# | $ echo foo
# | -> do this
# | <- get that
#
##
Symbol description:
##
# @symbol:
#
# Symbol body ditto ergo sum. Foo bar
# baz ding.
#
# @param1: the frob to frobnicate
# @param2: #optional how hard to frobnicate
#
# Returns: the frobnicated frob.
# If frob isn't frobnicatable, GenericError.
#
# Since: version
# Notes: notes, comments can have
# - itemized list
# - like this
#
# Example:
#
# -> { "execute": "quit" }
# <- { "return": {} }
#
##
That's roughly following the following EBNF grammar:
api_comment = "##\n" comment "##\n"
comment = freeform_comment | symbol_comment
freeform_comment = { "# " text "\n" | "#\n" }
symbol_comment = "# @" name ":\n" { member | tag_section | freeform_comment }
member = "# @" name ':' [ text ] "\n" freeform_comment
tag_section = "# " ( "Returns:", "Since:", "Note:", "Notes:", "Example:", "Examples:" ) [ text ] "\n" freeform_comment
text = free text with markup
Note that the grammar is ambiguous: a line "# @foo:\n" can be parsed
both as freeform_comment and as symbol_comment. The actual parser
recognizes symbol_comment.
See docs/qapi-code-gen.txt for more details.
Deficiencies and limitations:
- the generated QMP documentation includes internal types
- union type support is lacking
- type information is lacking in generated documentation
- doc comment error message positions are imprecise, they point
to the beginning of the comment.
- a few minor issues, all marked TODO/FIXME in the code
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20170113144135.5150-16-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[test-qapi.py tweaked to avoid trailing empty lines in .out]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2017-01-13 15:41:29 +01:00
|
|
|
# those are often singular or plural
|
2017-03-15 13:57:08 +01:00
|
|
|
'Note:', 'Notes:',
|
|
|
|
'Example:', 'Examples:',
|
|
|
|
'TODO:'):
|
qapi: add qapi2texi script
As the name suggests, the qapi2texi script converts JSON QAPI
description into a texi file suitable for different target
formats (info/man/txt/pdf/html...).
It parses the following kind of blocks:
Free-form:
##
# = Section
# == Subsection
#
# Some text foo with *emphasis*
# 1. with a list
# 2. like that
#
# And some code:
# | $ echo foo
# | -> do this
# | <- get that
#
##
Symbol description:
##
# @symbol:
#
# Symbol body ditto ergo sum. Foo bar
# baz ding.
#
# @param1: the frob to frobnicate
# @param2: #optional how hard to frobnicate
#
# Returns: the frobnicated frob.
# If frob isn't frobnicatable, GenericError.
#
# Since: version
# Notes: notes, comments can have
# - itemized list
# - like this
#
# Example:
#
# -> { "execute": "quit" }
# <- { "return": {} }
#
##
That's roughly following the following EBNF grammar:
api_comment = "##\n" comment "##\n"
comment = freeform_comment | symbol_comment
freeform_comment = { "# " text "\n" | "#\n" }
symbol_comment = "# @" name ":\n" { member | tag_section | freeform_comment }
member = "# @" name ':' [ text ] "\n" freeform_comment
tag_section = "# " ( "Returns:", "Since:", "Note:", "Notes:", "Example:", "Examples:" ) [ text ] "\n" freeform_comment
text = free text with markup
Note that the grammar is ambiguous: a line "# @foo:\n" can be parsed
both as freeform_comment and as symbol_comment. The actual parser
recognizes symbol_comment.
See docs/qapi-code-gen.txt for more details.
Deficiencies and limitations:
- the generated QMP documentation includes internal types
- union type support is lacking
- type information is lacking in generated documentation
- doc comment error message positions are imprecise, they point
to the beginning of the comment.
- a few minor issues, all marked TODO/FIXME in the code
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20170113144135.5150-16-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[test-qapi.py tweaked to avoid trailing empty lines in .out]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2017-01-13 15:41:29 +01:00
|
|
|
line = line[len(name)+1:]
|
|
|
|
self._start_section(name[:-1])
|
|
|
|
|
|
|
|
self._append_freeform(line)
|
|
|
|
|
|
|
|
def _start_args_section(self, name):
|
|
|
|
# FIXME invalid names other than the empty string aren't flagged
|
|
|
|
if not name:
|
|
|
|
raise QAPIParseError(self.parser, "Invalid parameter name")
|
|
|
|
if name in self.args:
|
|
|
|
raise QAPIParseError(self.parser,
|
|
|
|
"'%s' parameter name duplicated" % name)
|
|
|
|
if self.sections:
|
|
|
|
raise QAPIParseError(self.parser,
|
|
|
|
"'@%s:' can't follow '%s' section"
|
|
|
|
% (name, self.sections[0].name))
|
2017-03-15 13:57:23 +01:00
|
|
|
self._end_section()
|
qapi: add qapi2texi script
As the name suggests, the qapi2texi script converts JSON QAPI
description into a texi file suitable for different target
formats (info/man/txt/pdf/html...).
It parses the following kind of blocks:
Free-form:
##
# = Section
# == Subsection
#
# Some text foo with *emphasis*
# 1. with a list
# 2. like that
#
# And some code:
# | $ echo foo
# | -> do this
# | <- get that
#
##
Symbol description:
##
# @symbol:
#
# Symbol body ditto ergo sum. Foo bar
# baz ding.
#
# @param1: the frob to frobnicate
# @param2: #optional how hard to frobnicate
#
# Returns: the frobnicated frob.
# If frob isn't frobnicatable, GenericError.
#
# Since: version
# Notes: notes, comments can have
# - itemized list
# - like this
#
# Example:
#
# -> { "execute": "quit" }
# <- { "return": {} }
#
##
That's roughly following the following EBNF grammar:
api_comment = "##\n" comment "##\n"
comment = freeform_comment | symbol_comment
freeform_comment = { "# " text "\n" | "#\n" }
symbol_comment = "# @" name ":\n" { member | tag_section | freeform_comment }
member = "# @" name ':' [ text ] "\n" freeform_comment
tag_section = "# " ( "Returns:", "Since:", "Note:", "Notes:", "Example:", "Examples:" ) [ text ] "\n" freeform_comment
text = free text with markup
Note that the grammar is ambiguous: a line "# @foo:\n" can be parsed
both as freeform_comment and as symbol_comment. The actual parser
recognizes symbol_comment.
See docs/qapi-code-gen.txt for more details.
Deficiencies and limitations:
- the generated QMP documentation includes internal types
- union type support is lacking
- type information is lacking in generated documentation
- doc comment error message positions are imprecise, they point
to the beginning of the comment.
- a few minor issues, all marked TODO/FIXME in the code
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20170113144135.5150-16-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[test-qapi.py tweaked to avoid trailing empty lines in .out]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2017-01-13 15:41:29 +01:00
|
|
|
self.section = QAPIDoc.ArgSection(name)
|
|
|
|
self.args[name] = self.section
|
|
|
|
|
2017-03-15 13:57:08 +01:00
|
|
|
def _start_section(self, name=''):
|
|
|
|
if name in ('Returns', 'Since') and self.has_section(name):
|
qapi: add qapi2texi script
As the name suggests, the qapi2texi script converts JSON QAPI
description into a texi file suitable for different target
formats (info/man/txt/pdf/html...).
It parses the following kind of blocks:
Free-form:
##
# = Section
# == Subsection
#
# Some text foo with *emphasis*
# 1. with a list
# 2. like that
#
# And some code:
# | $ echo foo
# | -> do this
# | <- get that
#
##
Symbol description:
##
# @symbol:
#
# Symbol body ditto ergo sum. Foo bar
# baz ding.
#
# @param1: the frob to frobnicate
# @param2: #optional how hard to frobnicate
#
# Returns: the frobnicated frob.
# If frob isn't frobnicatable, GenericError.
#
# Since: version
# Notes: notes, comments can have
# - itemized list
# - like this
#
# Example:
#
# -> { "execute": "quit" }
# <- { "return": {} }
#
##
That's roughly following the following EBNF grammar:
api_comment = "##\n" comment "##\n"
comment = freeform_comment | symbol_comment
freeform_comment = { "# " text "\n" | "#\n" }
symbol_comment = "# @" name ":\n" { member | tag_section | freeform_comment }
member = "# @" name ':' [ text ] "\n" freeform_comment
tag_section = "# " ( "Returns:", "Since:", "Note:", "Notes:", "Example:", "Examples:" ) [ text ] "\n" freeform_comment
text = free text with markup
Note that the grammar is ambiguous: a line "# @foo:\n" can be parsed
both as freeform_comment and as symbol_comment. The actual parser
recognizes symbol_comment.
See docs/qapi-code-gen.txt for more details.
Deficiencies and limitations:
- the generated QMP documentation includes internal types
- union type support is lacking
- type information is lacking in generated documentation
- doc comment error message positions are imprecise, they point
to the beginning of the comment.
- a few minor issues, all marked TODO/FIXME in the code
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20170113144135.5150-16-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[test-qapi.py tweaked to avoid trailing empty lines in .out]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2017-01-13 15:41:29 +01:00
|
|
|
raise QAPIParseError(self.parser,
|
|
|
|
"Duplicated '%s' section" % name)
|
2017-03-15 13:57:23 +01:00
|
|
|
self._end_section()
|
qapi: add qapi2texi script
As the name suggests, the qapi2texi script converts JSON QAPI
description into a texi file suitable for different target
formats (info/man/txt/pdf/html...).
It parses the following kind of blocks:
Free-form:
##
# = Section
# == Subsection
#
# Some text foo with *emphasis*
# 1. with a list
# 2. like that
#
# And some code:
# | $ echo foo
# | -> do this
# | <- get that
#
##
Symbol description:
##
# @symbol:
#
# Symbol body ditto ergo sum. Foo bar
# baz ding.
#
# @param1: the frob to frobnicate
# @param2: #optional how hard to frobnicate
#
# Returns: the frobnicated frob.
# If frob isn't frobnicatable, GenericError.
#
# Since: version
# Notes: notes, comments can have
# - itemized list
# - like this
#
# Example:
#
# -> { "execute": "quit" }
# <- { "return": {} }
#
##
That's roughly following the following EBNF grammar:
api_comment = "##\n" comment "##\n"
comment = freeform_comment | symbol_comment
freeform_comment = { "# " text "\n" | "#\n" }
symbol_comment = "# @" name ":\n" { member | tag_section | freeform_comment }
member = "# @" name ':' [ text ] "\n" freeform_comment
tag_section = "# " ( "Returns:", "Since:", "Note:", "Notes:", "Example:", "Examples:" ) [ text ] "\n" freeform_comment
text = free text with markup
Note that the grammar is ambiguous: a line "# @foo:\n" can be parsed
both as freeform_comment and as symbol_comment. The actual parser
recognizes symbol_comment.
See docs/qapi-code-gen.txt for more details.
Deficiencies and limitations:
- the generated QMP documentation includes internal types
- union type support is lacking
- type information is lacking in generated documentation
- doc comment error message positions are imprecise, they point
to the beginning of the comment.
- a few minor issues, all marked TODO/FIXME in the code
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20170113144135.5150-16-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[test-qapi.py tweaked to avoid trailing empty lines in .out]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2017-01-13 15:41:29 +01:00
|
|
|
self.section = QAPIDoc.Section(name)
|
|
|
|
self.sections.append(self.section)
|
|
|
|
|
2017-03-15 13:57:23 +01:00
|
|
|
def _end_section(self):
|
|
|
|
if self.section:
|
|
|
|
contents = str(self.section)
|
|
|
|
if self.section.name and (not contents or contents.isspace()):
|
|
|
|
raise QAPIParseError(self.parser, "Empty doc section '%s'"
|
|
|
|
% self.section.name)
|
|
|
|
self.section = None
|
|
|
|
|
qapi: add qapi2texi script
As the name suggests, the qapi2texi script converts JSON QAPI
description into a texi file suitable for different target
formats (info/man/txt/pdf/html...).
It parses the following kind of blocks:
Free-form:
##
# = Section
# == Subsection
#
# Some text foo with *emphasis*
# 1. with a list
# 2. like that
#
# And some code:
# | $ echo foo
# | -> do this
# | <- get that
#
##
Symbol description:
##
# @symbol:
#
# Symbol body ditto ergo sum. Foo bar
# baz ding.
#
# @param1: the frob to frobnicate
# @param2: #optional how hard to frobnicate
#
# Returns: the frobnicated frob.
# If frob isn't frobnicatable, GenericError.
#
# Since: version
# Notes: notes, comments can have
# - itemized list
# - like this
#
# Example:
#
# -> { "execute": "quit" }
# <- { "return": {} }
#
##
That's roughly following the following EBNF grammar:
api_comment = "##\n" comment "##\n"
comment = freeform_comment | symbol_comment
freeform_comment = { "# " text "\n" | "#\n" }
symbol_comment = "# @" name ":\n" { member | tag_section | freeform_comment }
member = "# @" name ':' [ text ] "\n" freeform_comment
tag_section = "# " ( "Returns:", "Since:", "Note:", "Notes:", "Example:", "Examples:" ) [ text ] "\n" freeform_comment
text = free text with markup
Note that the grammar is ambiguous: a line "# @foo:\n" can be parsed
both as freeform_comment and as symbol_comment. The actual parser
recognizes symbol_comment.
See docs/qapi-code-gen.txt for more details.
Deficiencies and limitations:
- the generated QMP documentation includes internal types
- union type support is lacking
- type information is lacking in generated documentation
- doc comment error message positions are imprecise, they point
to the beginning of the comment.
- a few minor issues, all marked TODO/FIXME in the code
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20170113144135.5150-16-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[test-qapi.py tweaked to avoid trailing empty lines in .out]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2017-01-13 15:41:29 +01:00
|
|
|
def _append_freeform(self, line):
|
|
|
|
in_arg = isinstance(self.section, QAPIDoc.ArgSection)
|
|
|
|
if (in_arg and self.section.content
|
|
|
|
and not self.section.content[-1]
|
|
|
|
and line and not line[0].isspace()):
|
|
|
|
self._start_section()
|
|
|
|
if (in_arg or not self.section.name
|
2017-03-15 13:57:08 +01:00
|
|
|
or not self.section.name.startswith('Example')):
|
qapi: add qapi2texi script
As the name suggests, the qapi2texi script converts JSON QAPI
description into a texi file suitable for different target
formats (info/man/txt/pdf/html...).
It parses the following kind of blocks:
Free-form:
##
# = Section
# == Subsection
#
# Some text foo with *emphasis*
# 1. with a list
# 2. like that
#
# And some code:
# | $ echo foo
# | -> do this
# | <- get that
#
##
Symbol description:
##
# @symbol:
#
# Symbol body ditto ergo sum. Foo bar
# baz ding.
#
# @param1: the frob to frobnicate
# @param2: #optional how hard to frobnicate
#
# Returns: the frobnicated frob.
# If frob isn't frobnicatable, GenericError.
#
# Since: version
# Notes: notes, comments can have
# - itemized list
# - like this
#
# Example:
#
# -> { "execute": "quit" }
# <- { "return": {} }
#
##
That's roughly following the following EBNF grammar:
api_comment = "##\n" comment "##\n"
comment = freeform_comment | symbol_comment
freeform_comment = { "# " text "\n" | "#\n" }
symbol_comment = "# @" name ":\n" { member | tag_section | freeform_comment }
member = "# @" name ':' [ text ] "\n" freeform_comment
tag_section = "# " ( "Returns:", "Since:", "Note:", "Notes:", "Example:", "Examples:" ) [ text ] "\n" freeform_comment
text = free text with markup
Note that the grammar is ambiguous: a line "# @foo:\n" can be parsed
both as freeform_comment and as symbol_comment. The actual parser
recognizes symbol_comment.
See docs/qapi-code-gen.txt for more details.
Deficiencies and limitations:
- the generated QMP documentation includes internal types
- union type support is lacking
- type information is lacking in generated documentation
- doc comment error message positions are imprecise, they point
to the beginning of the comment.
- a few minor issues, all marked TODO/FIXME in the code
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20170113144135.5150-16-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[test-qapi.py tweaked to avoid trailing empty lines in .out]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2017-01-13 15:41:29 +01:00
|
|
|
line = line.strip()
|
2017-03-15 13:57:22 +01:00
|
|
|
match = re.match(r'(@\S+:)', line)
|
|
|
|
if match:
|
|
|
|
raise QAPIParseError(self.parser,
|
|
|
|
"'%s' not allowed in free-form documentation"
|
|
|
|
% match.group(1))
|
qapi: add qapi2texi script
As the name suggests, the qapi2texi script converts JSON QAPI
description into a texi file suitable for different target
formats (info/man/txt/pdf/html...).
It parses the following kind of blocks:
Free-form:
##
# = Section
# == Subsection
#
# Some text foo with *emphasis*
# 1. with a list
# 2. like that
#
# And some code:
# | $ echo foo
# | -> do this
# | <- get that
#
##
Symbol description:
##
# @symbol:
#
# Symbol body ditto ergo sum. Foo bar
# baz ding.
#
# @param1: the frob to frobnicate
# @param2: #optional how hard to frobnicate
#
# Returns: the frobnicated frob.
# If frob isn't frobnicatable, GenericError.
#
# Since: version
# Notes: notes, comments can have
# - itemized list
# - like this
#
# Example:
#
# -> { "execute": "quit" }
# <- { "return": {} }
#
##
That's roughly following the following EBNF grammar:
api_comment = "##\n" comment "##\n"
comment = freeform_comment | symbol_comment
freeform_comment = { "# " text "\n" | "#\n" }
symbol_comment = "# @" name ":\n" { member | tag_section | freeform_comment }
member = "# @" name ':' [ text ] "\n" freeform_comment
tag_section = "# " ( "Returns:", "Since:", "Note:", "Notes:", "Example:", "Examples:" ) [ text ] "\n" freeform_comment
text = free text with markup
Note that the grammar is ambiguous: a line "# @foo:\n" can be parsed
both as freeform_comment and as symbol_comment. The actual parser
recognizes symbol_comment.
See docs/qapi-code-gen.txt for more details.
Deficiencies and limitations:
- the generated QMP documentation includes internal types
- union type support is lacking
- type information is lacking in generated documentation
- doc comment error message positions are imprecise, they point
to the beginning of the comment.
- a few minor issues, all marked TODO/FIXME in the code
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20170113144135.5150-16-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[test-qapi.py tweaked to avoid trailing empty lines in .out]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2017-01-13 15:41:29 +01:00
|
|
|
self.section.append(line)
|
|
|
|
|
2017-03-15 13:57:03 +01:00
|
|
|
def connect_member(self, member):
|
|
|
|
if member.name not in self.args:
|
|
|
|
# Undocumented TODO outlaw
|
2017-03-15 13:57:04 +01:00
|
|
|
self.args[member.name] = QAPIDoc.ArgSection(member.name)
|
|
|
|
self.args[member.name].connect(member)
|
2017-03-15 13:57:03 +01:00
|
|
|
|
2017-03-15 13:57:27 +01:00
|
|
|
def check_expr(self, expr):
|
|
|
|
if self.has_section('Returns') and 'command' not in expr:
|
|
|
|
raise QAPISemError(self.info,
|
|
|
|
"'Returns:' is only valid for commands")
|
|
|
|
|
2017-03-15 13:57:26 +01:00
|
|
|
def check(self):
|
|
|
|
bogus = [name for name, section in self.args.iteritems()
|
|
|
|
if not section.member]
|
|
|
|
if bogus:
|
|
|
|
raise QAPISemError(
|
|
|
|
self.info,
|
|
|
|
"The following documented members are not in "
|
|
|
|
"the declaration: %s" % ", ".join(bogus))
|
|
|
|
|
qapi: add qapi2texi script
As the name suggests, the qapi2texi script converts JSON QAPI
description into a texi file suitable for different target
formats (info/man/txt/pdf/html...).
It parses the following kind of blocks:
Free-form:
##
# = Section
# == Subsection
#
# Some text foo with *emphasis*
# 1. with a list
# 2. like that
#
# And some code:
# | $ echo foo
# | -> do this
# | <- get that
#
##
Symbol description:
##
# @symbol:
#
# Symbol body ditto ergo sum. Foo bar
# baz ding.
#
# @param1: the frob to frobnicate
# @param2: #optional how hard to frobnicate
#
# Returns: the frobnicated frob.
# If frob isn't frobnicatable, GenericError.
#
# Since: version
# Notes: notes, comments can have
# - itemized list
# - like this
#
# Example:
#
# -> { "execute": "quit" }
# <- { "return": {} }
#
##
That's roughly following the following EBNF grammar:
api_comment = "##\n" comment "##\n"
comment = freeform_comment | symbol_comment
freeform_comment = { "# " text "\n" | "#\n" }
symbol_comment = "# @" name ":\n" { member | tag_section | freeform_comment }
member = "# @" name ':' [ text ] "\n" freeform_comment
tag_section = "# " ( "Returns:", "Since:", "Note:", "Notes:", "Example:", "Examples:" ) [ text ] "\n" freeform_comment
text = free text with markup
Note that the grammar is ambiguous: a line "# @foo:\n" can be parsed
both as freeform_comment and as symbol_comment. The actual parser
recognizes symbol_comment.
See docs/qapi-code-gen.txt for more details.
Deficiencies and limitations:
- the generated QMP documentation includes internal types
- union type support is lacking
- type information is lacking in generated documentation
- doc comment error message positions are imprecise, they point
to the beginning of the comment.
- a few minor issues, all marked TODO/FIXME in the code
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20170113144135.5150-16-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[test-qapi.py tweaked to avoid trailing empty lines in .out]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2017-01-13 15:41:29 +01:00
|
|
|
|
2015-09-16 13:06:04 +02:00
|
|
|
class QAPISchemaParser(object):
|
qapi.py: Restructure lexer and parser
The parser has a rather unorthodox structure:
Until EOF:
Read a section:
Generator function get_expr() yields one section after the
other, as a string. An unindented, non-empty line that
isn't a comment starts a new section.
Lexing:
Split section into a list of tokens (strings), with help
of generator function tokenize().
Parsing:
Parse the first expression from the list of tokens, with
parse(), throw away any remaining tokens.
In parse_schema(): record value of an enum, union or
struct key (if any) in the appropriate global table,
append expression to the list of expressions.
Return list of expressions.
Known issues:
(1) Indentation is significant, unlike in real JSON.
(2) Neither lexer nor parser have any idea of source positions. Error
reporting is hard, let's go shopping.
(3) The one error we bother to detect, we "report" via raise.
(4) The lexer silently ignores invalid characters.
(5) If everything in a section gets ignored, the parser crashes.
(6) The lexer treats a string containing a structural character exactly
like the structural character.
(7) Tokens trailing the first expression in a section are silently
ignored.
(8) The parser accepts any token in place of a colon.
(9) The parser treats comma as optional.
(10) parse() crashes on unexpected EOF.
(11) parse_schema() crashes when a section's expression isn't a JSON
object.
Replace this piece of original art by a thoroughly unoriginal design.
Takes care of (1), (2), (5), (6) and (7), and lays the groundwork for
addressing the others. Generated source files remain unchanged.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-4-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-27 17:41:55 +02:00
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
def __init__(self, fp, previously_included=[], incl_info=None):
|
2015-06-09 16:22:45 +02:00
|
|
|
abs_fname = os.path.abspath(fp.name)
|
2015-06-09 18:32:29 +02:00
|
|
|
fname = fp.name
|
2015-06-09 16:22:45 +02:00
|
|
|
self.fname = fname
|
|
|
|
previously_included.append(abs_fname)
|
|
|
|
self.incl_info = incl_info
|
qapi.py: Restructure lexer and parser
The parser has a rather unorthodox structure:
Until EOF:
Read a section:
Generator function get_expr() yields one section after the
other, as a string. An unindented, non-empty line that
isn't a comment starts a new section.
Lexing:
Split section into a list of tokens (strings), with help
of generator function tokenize().
Parsing:
Parse the first expression from the list of tokens, with
parse(), throw away any remaining tokens.
In parse_schema(): record value of an enum, union or
struct key (if any) in the appropriate global table,
append expression to the list of expressions.
Return list of expressions.
Known issues:
(1) Indentation is significant, unlike in real JSON.
(2) Neither lexer nor parser have any idea of source positions. Error
reporting is hard, let's go shopping.
(3) The one error we bother to detect, we "report" via raise.
(4) The lexer silently ignores invalid characters.
(5) If everything in a section gets ignored, the parser crashes.
(6) The lexer treats a string containing a structural character exactly
like the structural character.
(7) Tokens trailing the first expression in a section are silently
ignored.
(8) The parser accepts any token in place of a colon.
(9) The parser treats comma as optional.
(10) parse() crashes on unexpected EOF.
(11) parse_schema() crashes when a section's expression isn't a JSON
object.
Replace this piece of original art by a thoroughly unoriginal design.
Takes care of (1), (2), (5), (6) and (7), and lays the groundwork for
addressing the others. Generated source files remain unchanged.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-4-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-27 17:41:55 +02:00
|
|
|
self.src = fp.read()
|
|
|
|
if self.src == '' or self.src[-1] != '\n':
|
|
|
|
self.src += '\n'
|
|
|
|
self.cursor = 0
|
2014-03-05 03:44:33 +01:00
|
|
|
self.line = 1
|
|
|
|
self.line_pos = 0
|
qapi.py: Restructure lexer and parser
The parser has a rather unorthodox structure:
Until EOF:
Read a section:
Generator function get_expr() yields one section after the
other, as a string. An unindented, non-empty line that
isn't a comment starts a new section.
Lexing:
Split section into a list of tokens (strings), with help
of generator function tokenize().
Parsing:
Parse the first expression from the list of tokens, with
parse(), throw away any remaining tokens.
In parse_schema(): record value of an enum, union or
struct key (if any) in the appropriate global table,
append expression to the list of expressions.
Return list of expressions.
Known issues:
(1) Indentation is significant, unlike in real JSON.
(2) Neither lexer nor parser have any idea of source positions. Error
reporting is hard, let's go shopping.
(3) The one error we bother to detect, we "report" via raise.
(4) The lexer silently ignores invalid characters.
(5) If everything in a section gets ignored, the parser crashes.
(6) The lexer treats a string containing a structural character exactly
like the structural character.
(7) Tokens trailing the first expression in a section are silently
ignored.
(8) The parser accepts any token in place of a colon.
(9) The parser treats comma as optional.
(10) parse() crashes on unexpected EOF.
(11) parse_schema() crashes when a section's expression isn't a JSON
object.
Replace this piece of original art by a thoroughly unoriginal design.
Takes care of (1), (2), (5), (6) and (7), and lays the groundwork for
addressing the others. Generated source files remain unchanged.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-4-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-27 17:41:55 +02:00
|
|
|
self.exprs = []
|
qapi: add qapi2texi script
As the name suggests, the qapi2texi script converts JSON QAPI
description into a texi file suitable for different target
formats (info/man/txt/pdf/html...).
It parses the following kind of blocks:
Free-form:
##
# = Section
# == Subsection
#
# Some text foo with *emphasis*
# 1. with a list
# 2. like that
#
# And some code:
# | $ echo foo
# | -> do this
# | <- get that
#
##
Symbol description:
##
# @symbol:
#
# Symbol body ditto ergo sum. Foo bar
# baz ding.
#
# @param1: the frob to frobnicate
# @param2: #optional how hard to frobnicate
#
# Returns: the frobnicated frob.
# If frob isn't frobnicatable, GenericError.
#
# Since: version
# Notes: notes, comments can have
# - itemized list
# - like this
#
# Example:
#
# -> { "execute": "quit" }
# <- { "return": {} }
#
##
That's roughly following the following EBNF grammar:
api_comment = "##\n" comment "##\n"
comment = freeform_comment | symbol_comment
freeform_comment = { "# " text "\n" | "#\n" }
symbol_comment = "# @" name ":\n" { member | tag_section | freeform_comment }
member = "# @" name ':' [ text ] "\n" freeform_comment
tag_section = "# " ( "Returns:", "Since:", "Note:", "Notes:", "Example:", "Examples:" ) [ text ] "\n" freeform_comment
text = free text with markup
Note that the grammar is ambiguous: a line "# @foo:\n" can be parsed
both as freeform_comment and as symbol_comment. The actual parser
recognizes symbol_comment.
See docs/qapi-code-gen.txt for more details.
Deficiencies and limitations:
- the generated QMP documentation includes internal types
- union type support is lacking
- type information is lacking in generated documentation
- doc comment error message positions are imprecise, they point
to the beginning of the comment.
- a few minor issues, all marked TODO/FIXME in the code
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20170113144135.5150-16-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[test-qapi.py tweaked to avoid trailing empty lines in .out]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2017-01-13 15:41:29 +01:00
|
|
|
self.docs = []
|
2017-03-15 13:57:20 +01:00
|
|
|
self.cur_doc = None
|
qapi.py: Restructure lexer and parser
The parser has a rather unorthodox structure:
Until EOF:
Read a section:
Generator function get_expr() yields one section after the
other, as a string. An unindented, non-empty line that
isn't a comment starts a new section.
Lexing:
Split section into a list of tokens (strings), with help
of generator function tokenize().
Parsing:
Parse the first expression from the list of tokens, with
parse(), throw away any remaining tokens.
In parse_schema(): record value of an enum, union or
struct key (if any) in the appropriate global table,
append expression to the list of expressions.
Return list of expressions.
Known issues:
(1) Indentation is significant, unlike in real JSON.
(2) Neither lexer nor parser have any idea of source positions. Error
reporting is hard, let's go shopping.
(3) The one error we bother to detect, we "report" via raise.
(4) The lexer silently ignores invalid characters.
(5) If everything in a section gets ignored, the parser crashes.
(6) The lexer treats a string containing a structural character exactly
like the structural character.
(7) Tokens trailing the first expression in a section are silently
ignored.
(8) The parser accepts any token in place of a colon.
(9) The parser treats comma as optional.
(10) parse() crashes on unexpected EOF.
(11) parse_schema() crashes when a section's expression isn't a JSON
object.
Replace this piece of original art by a thoroughly unoriginal design.
Takes care of (1), (2), (5), (6) and (7), and lays the groundwork for
addressing the others. Generated source files remain unchanged.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-4-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-27 17:41:55 +02:00
|
|
|
self.accept()
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
while self.tok is not None:
|
2017-01-13 15:41:25 +01:00
|
|
|
info = {'file': fname, 'line': self.line,
|
|
|
|
'parent': self.incl_info}
|
qapi: add qapi2texi script
As the name suggests, the qapi2texi script converts JSON QAPI
description into a texi file suitable for different target
formats (info/man/txt/pdf/html...).
It parses the following kind of blocks:
Free-form:
##
# = Section
# == Subsection
#
# Some text foo with *emphasis*
# 1. with a list
# 2. like that
#
# And some code:
# | $ echo foo
# | -> do this
# | <- get that
#
##
Symbol description:
##
# @symbol:
#
# Symbol body ditto ergo sum. Foo bar
# baz ding.
#
# @param1: the frob to frobnicate
# @param2: #optional how hard to frobnicate
#
# Returns: the frobnicated frob.
# If frob isn't frobnicatable, GenericError.
#
# Since: version
# Notes: notes, comments can have
# - itemized list
# - like this
#
# Example:
#
# -> { "execute": "quit" }
# <- { "return": {} }
#
##
That's roughly following the following EBNF grammar:
api_comment = "##\n" comment "##\n"
comment = freeform_comment | symbol_comment
freeform_comment = { "# " text "\n" | "#\n" }
symbol_comment = "# @" name ":\n" { member | tag_section | freeform_comment }
member = "# @" name ':' [ text ] "\n" freeform_comment
tag_section = "# " ( "Returns:", "Since:", "Note:", "Notes:", "Example:", "Examples:" ) [ text ] "\n" freeform_comment
text = free text with markup
Note that the grammar is ambiguous: a line "# @foo:\n" can be parsed
both as freeform_comment and as symbol_comment. The actual parser
recognizes symbol_comment.
See docs/qapi-code-gen.txt for more details.
Deficiencies and limitations:
- the generated QMP documentation includes internal types
- union type support is lacking
- type information is lacking in generated documentation
- doc comment error message positions are imprecise, they point
to the beginning of the comment.
- a few minor issues, all marked TODO/FIXME in the code
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20170113144135.5150-16-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[test-qapi.py tweaked to avoid trailing empty lines in .out]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2017-01-13 15:41:29 +01:00
|
|
|
if self.tok == '#':
|
2017-03-15 13:57:20 +01:00
|
|
|
self.reject_expr_doc()
|
|
|
|
self.cur_doc = self.get_doc(info)
|
|
|
|
self.docs.append(self.cur_doc)
|
qapi: add qapi2texi script
As the name suggests, the qapi2texi script converts JSON QAPI
description into a texi file suitable for different target
formats (info/man/txt/pdf/html...).
It parses the following kind of blocks:
Free-form:
##
# = Section
# == Subsection
#
# Some text foo with *emphasis*
# 1. with a list
# 2. like that
#
# And some code:
# | $ echo foo
# | -> do this
# | <- get that
#
##
Symbol description:
##
# @symbol:
#
# Symbol body ditto ergo sum. Foo bar
# baz ding.
#
# @param1: the frob to frobnicate
# @param2: #optional how hard to frobnicate
#
# Returns: the frobnicated frob.
# If frob isn't frobnicatable, GenericError.
#
# Since: version
# Notes: notes, comments can have
# - itemized list
# - like this
#
# Example:
#
# -> { "execute": "quit" }
# <- { "return": {} }
#
##
That's roughly following the following EBNF grammar:
api_comment = "##\n" comment "##\n"
comment = freeform_comment | symbol_comment
freeform_comment = { "# " text "\n" | "#\n" }
symbol_comment = "# @" name ":\n" { member | tag_section | freeform_comment }
member = "# @" name ':' [ text ] "\n" freeform_comment
tag_section = "# " ( "Returns:", "Since:", "Note:", "Notes:", "Example:", "Examples:" ) [ text ] "\n" freeform_comment
text = free text with markup
Note that the grammar is ambiguous: a line "# @foo:\n" can be parsed
both as freeform_comment and as symbol_comment. The actual parser
recognizes symbol_comment.
See docs/qapi-code-gen.txt for more details.
Deficiencies and limitations:
- the generated QMP documentation includes internal types
- union type support is lacking
- type information is lacking in generated documentation
- doc comment error message positions are imprecise, they point
to the beginning of the comment.
- a few minor issues, all marked TODO/FIXME in the code
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20170113144135.5150-16-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[test-qapi.py tweaked to avoid trailing empty lines in .out]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2017-01-13 15:41:29 +01:00
|
|
|
continue
|
|
|
|
|
2014-05-07 20:46:15 +02:00
|
|
|
expr = self.get_expr(False)
|
2017-03-15 13:56:50 +01:00
|
|
|
if 'include' in expr:
|
2017-03-15 13:57:20 +01:00
|
|
|
self.reject_expr_doc()
|
2014-05-07 20:46:15 +02:00
|
|
|
if len(expr) != 1:
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPISemError(info, "Invalid 'include' directive")
|
2017-03-15 13:57:08 +01:00
|
|
|
include = expr['include']
|
2014-05-07 20:46:15 +02:00
|
|
|
if not isinstance(include, str):
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPISemError(info,
|
|
|
|
"Value of 'include' must be a string")
|
2017-03-15 13:56:50 +01:00
|
|
|
self._include(include, info, os.path.dirname(abs_fname),
|
|
|
|
previously_included)
|
2017-03-15 13:56:51 +01:00
|
|
|
elif "pragma" in expr:
|
2017-03-15 13:57:20 +01:00
|
|
|
self.reject_expr_doc()
|
2017-03-15 13:56:51 +01:00
|
|
|
if len(expr) != 1:
|
|
|
|
raise QAPISemError(info, "Invalid 'pragma' directive")
|
|
|
|
pragma = expr['pragma']
|
|
|
|
if not isinstance(pragma, dict):
|
|
|
|
raise QAPISemError(
|
|
|
|
info, "Value of 'pragma' must be a dictionary")
|
|
|
|
for name, value in pragma.iteritems():
|
|
|
|
self._pragma(name, value, info)
|
2014-05-07 20:46:15 +02:00
|
|
|
else:
|
|
|
|
expr_elem = {'expr': expr,
|
2017-01-13 15:41:25 +01:00
|
|
|
'info': info}
|
2017-03-15 13:57:20 +01:00
|
|
|
if self.cur_doc:
|
|
|
|
if not self.cur_doc.symbol:
|
|
|
|
raise QAPISemError(
|
|
|
|
self.cur_doc.info,
|
|
|
|
"Expression documentation required")
|
|
|
|
expr_elem['doc'] = self.cur_doc
|
2014-05-07 20:46:15 +02:00
|
|
|
self.exprs.append(expr_elem)
|
2017-03-15 13:57:20 +01:00
|
|
|
self.cur_doc = None
|
|
|
|
self.reject_expr_doc()
|
|
|
|
|
|
|
|
def reject_expr_doc(self):
|
|
|
|
if self.cur_doc and self.cur_doc.symbol:
|
|
|
|
raise QAPISemError(
|
|
|
|
self.cur_doc.info,
|
|
|
|
"Documentation for '%s' is not followed by the definition"
|
|
|
|
% self.cur_doc.symbol)
|
qapi.py: Restructure lexer and parser
The parser has a rather unorthodox structure:
Until EOF:
Read a section:
Generator function get_expr() yields one section after the
other, as a string. An unindented, non-empty line that
isn't a comment starts a new section.
Lexing:
Split section into a list of tokens (strings), with help
of generator function tokenize().
Parsing:
Parse the first expression from the list of tokens, with
parse(), throw away any remaining tokens.
In parse_schema(): record value of an enum, union or
struct key (if any) in the appropriate global table,
append expression to the list of expressions.
Return list of expressions.
Known issues:
(1) Indentation is significant, unlike in real JSON.
(2) Neither lexer nor parser have any idea of source positions. Error
reporting is hard, let's go shopping.
(3) The one error we bother to detect, we "report" via raise.
(4) The lexer silently ignores invalid characters.
(5) If everything in a section gets ignored, the parser crashes.
(6) The lexer treats a string containing a structural character exactly
like the structural character.
(7) Tokens trailing the first expression in a section are silently
ignored.
(8) The parser accepts any token in place of a colon.
(9) The parser treats comma as optional.
(10) parse() crashes on unexpected EOF.
(11) parse_schema() crashes when a section's expression isn't a JSON
object.
Replace this piece of original art by a thoroughly unoriginal design.
Takes care of (1), (2), (5), (6) and (7), and lays the groundwork for
addressing the others. Generated source files remain unchanged.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-4-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-27 17:41:55 +02:00
|
|
|
|
2017-03-15 13:56:50 +01:00
|
|
|
def _include(self, include, info, base_dir, previously_included):
|
|
|
|
incl_abs_fname = os.path.join(base_dir, include)
|
|
|
|
# catch inclusion cycle
|
|
|
|
inf = info
|
|
|
|
while inf:
|
|
|
|
if incl_abs_fname == os.path.abspath(inf['file']):
|
|
|
|
raise QAPISemError(info, "Inclusion loop for %s" % include)
|
|
|
|
inf = inf['parent']
|
|
|
|
|
|
|
|
# skip multiple include of the same file
|
|
|
|
if incl_abs_fname in previously_included:
|
|
|
|
return
|
|
|
|
try:
|
|
|
|
fobj = open(incl_abs_fname, 'r')
|
|
|
|
except IOError as e:
|
|
|
|
raise QAPISemError(info, '%s: %s' % (e.strerror, include))
|
|
|
|
exprs_include = QAPISchemaParser(fobj, previously_included, info)
|
|
|
|
self.exprs.extend(exprs_include.exprs)
|
|
|
|
self.docs.extend(exprs_include.docs)
|
|
|
|
|
2017-03-15 13:56:51 +01:00
|
|
|
def _pragma(self, name, value, info):
|
2017-03-15 13:56:55 +01:00
|
|
|
global doc_required, returns_whitelist, name_case_whitelist
|
2017-03-15 13:56:51 +01:00
|
|
|
if name == 'doc-required':
|
|
|
|
if not isinstance(value, bool):
|
|
|
|
raise QAPISemError(info,
|
|
|
|
"Pragma 'doc-required' must be boolean")
|
|
|
|
doc_required = value
|
2017-03-15 13:56:54 +01:00
|
|
|
elif name == 'returns-whitelist':
|
|
|
|
if (not isinstance(value, list)
|
|
|
|
or any([not isinstance(elt, str) for elt in value])):
|
|
|
|
raise QAPISemError(info,
|
|
|
|
"Pragma returns-whitelist must be"
|
|
|
|
" a list of strings")
|
|
|
|
returns_whitelist = value
|
2017-03-15 13:56:55 +01:00
|
|
|
elif name == 'name-case-whitelist':
|
|
|
|
if (not isinstance(value, list)
|
|
|
|
or any([not isinstance(elt, str) for elt in value])):
|
|
|
|
raise QAPISemError(info,
|
|
|
|
"Pragma name-case-whitelist must be"
|
|
|
|
" a list of strings")
|
|
|
|
name_case_whitelist = value
|
2017-03-15 13:56:51 +01:00
|
|
|
else:
|
|
|
|
raise QAPISemError(info, "Unknown pragma '%s'" % name)
|
|
|
|
|
qapi: add qapi2texi script
As the name suggests, the qapi2texi script converts JSON QAPI
description into a texi file suitable for different target
formats (info/man/txt/pdf/html...).
It parses the following kind of blocks:
Free-form:
##
# = Section
# == Subsection
#
# Some text foo with *emphasis*
# 1. with a list
# 2. like that
#
# And some code:
# | $ echo foo
# | -> do this
# | <- get that
#
##
Symbol description:
##
# @symbol:
#
# Symbol body ditto ergo sum. Foo bar
# baz ding.
#
# @param1: the frob to frobnicate
# @param2: #optional how hard to frobnicate
#
# Returns: the frobnicated frob.
# If frob isn't frobnicatable, GenericError.
#
# Since: version
# Notes: notes, comments can have
# - itemized list
# - like this
#
# Example:
#
# -> { "execute": "quit" }
# <- { "return": {} }
#
##
That's roughly following the following EBNF grammar:
api_comment = "##\n" comment "##\n"
comment = freeform_comment | symbol_comment
freeform_comment = { "# " text "\n" | "#\n" }
symbol_comment = "# @" name ":\n" { member | tag_section | freeform_comment }
member = "# @" name ':' [ text ] "\n" freeform_comment
tag_section = "# " ( "Returns:", "Since:", "Note:", "Notes:", "Example:", "Examples:" ) [ text ] "\n" freeform_comment
text = free text with markup
Note that the grammar is ambiguous: a line "# @foo:\n" can be parsed
both as freeform_comment and as symbol_comment. The actual parser
recognizes symbol_comment.
See docs/qapi-code-gen.txt for more details.
Deficiencies and limitations:
- the generated QMP documentation includes internal types
- union type support is lacking
- type information is lacking in generated documentation
- doc comment error message positions are imprecise, they point
to the beginning of the comment.
- a few minor issues, all marked TODO/FIXME in the code
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20170113144135.5150-16-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[test-qapi.py tweaked to avoid trailing empty lines in .out]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2017-01-13 15:41:29 +01:00
|
|
|
def accept(self, skip_comment=True):
|
qapi.py: Restructure lexer and parser
The parser has a rather unorthodox structure:
Until EOF:
Read a section:
Generator function get_expr() yields one section after the
other, as a string. An unindented, non-empty line that
isn't a comment starts a new section.
Lexing:
Split section into a list of tokens (strings), with help
of generator function tokenize().
Parsing:
Parse the first expression from the list of tokens, with
parse(), throw away any remaining tokens.
In parse_schema(): record value of an enum, union or
struct key (if any) in the appropriate global table,
append expression to the list of expressions.
Return list of expressions.
Known issues:
(1) Indentation is significant, unlike in real JSON.
(2) Neither lexer nor parser have any idea of source positions. Error
reporting is hard, let's go shopping.
(3) The one error we bother to detect, we "report" via raise.
(4) The lexer silently ignores invalid characters.
(5) If everything in a section gets ignored, the parser crashes.
(6) The lexer treats a string containing a structural character exactly
like the structural character.
(7) Tokens trailing the first expression in a section are silently
ignored.
(8) The parser accepts any token in place of a colon.
(9) The parser treats comma as optional.
(10) parse() crashes on unexpected EOF.
(11) parse_schema() crashes when a section's expression isn't a JSON
object.
Replace this piece of original art by a thoroughly unoriginal design.
Takes care of (1), (2), (5), (6) and (7), and lays the groundwork for
addressing the others. Generated source files remain unchanged.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-4-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-27 17:41:55 +02:00
|
|
|
while True:
|
|
|
|
self.tok = self.src[self.cursor]
|
2013-07-27 17:41:56 +02:00
|
|
|
self.pos = self.cursor
|
qapi.py: Restructure lexer and parser
The parser has a rather unorthodox structure:
Until EOF:
Read a section:
Generator function get_expr() yields one section after the
other, as a string. An unindented, non-empty line that
isn't a comment starts a new section.
Lexing:
Split section into a list of tokens (strings), with help
of generator function tokenize().
Parsing:
Parse the first expression from the list of tokens, with
parse(), throw away any remaining tokens.
In parse_schema(): record value of an enum, union or
struct key (if any) in the appropriate global table,
append expression to the list of expressions.
Return list of expressions.
Known issues:
(1) Indentation is significant, unlike in real JSON.
(2) Neither lexer nor parser have any idea of source positions. Error
reporting is hard, let's go shopping.
(3) The one error we bother to detect, we "report" via raise.
(4) The lexer silently ignores invalid characters.
(5) If everything in a section gets ignored, the parser crashes.
(6) The lexer treats a string containing a structural character exactly
like the structural character.
(7) Tokens trailing the first expression in a section are silently
ignored.
(8) The parser accepts any token in place of a colon.
(9) The parser treats comma as optional.
(10) parse() crashes on unexpected EOF.
(11) parse_schema() crashes when a section's expression isn't a JSON
object.
Replace this piece of original art by a thoroughly unoriginal design.
Takes care of (1), (2), (5), (6) and (7), and lays the groundwork for
addressing the others. Generated source files remain unchanged.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-4-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-27 17:41:55 +02:00
|
|
|
self.cursor += 1
|
|
|
|
self.val = None
|
|
|
|
|
2013-07-27 17:42:01 +02:00
|
|
|
if self.tok == '#':
|
qapi: add qapi2texi script
As the name suggests, the qapi2texi script converts JSON QAPI
description into a texi file suitable for different target
formats (info/man/txt/pdf/html...).
It parses the following kind of blocks:
Free-form:
##
# = Section
# == Subsection
#
# Some text foo with *emphasis*
# 1. with a list
# 2. like that
#
# And some code:
# | $ echo foo
# | -> do this
# | <- get that
#
##
Symbol description:
##
# @symbol:
#
# Symbol body ditto ergo sum. Foo bar
# baz ding.
#
# @param1: the frob to frobnicate
# @param2: #optional how hard to frobnicate
#
# Returns: the frobnicated frob.
# If frob isn't frobnicatable, GenericError.
#
# Since: version
# Notes: notes, comments can have
# - itemized list
# - like this
#
# Example:
#
# -> { "execute": "quit" }
# <- { "return": {} }
#
##
That's roughly following the following EBNF grammar:
api_comment = "##\n" comment "##\n"
comment = freeform_comment | symbol_comment
freeform_comment = { "# " text "\n" | "#\n" }
symbol_comment = "# @" name ":\n" { member | tag_section | freeform_comment }
member = "# @" name ':' [ text ] "\n" freeform_comment
tag_section = "# " ( "Returns:", "Since:", "Note:", "Notes:", "Example:", "Examples:" ) [ text ] "\n" freeform_comment
text = free text with markup
Note that the grammar is ambiguous: a line "# @foo:\n" can be parsed
both as freeform_comment and as symbol_comment. The actual parser
recognizes symbol_comment.
See docs/qapi-code-gen.txt for more details.
Deficiencies and limitations:
- the generated QMP documentation includes internal types
- union type support is lacking
- type information is lacking in generated documentation
- doc comment error message positions are imprecise, they point
to the beginning of the comment.
- a few minor issues, all marked TODO/FIXME in the code
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20170113144135.5150-16-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[test-qapi.py tweaked to avoid trailing empty lines in .out]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2017-01-13 15:41:29 +01:00
|
|
|
if self.src[self.cursor] == '#':
|
|
|
|
# Start of doc comment
|
|
|
|
skip_comment = False
|
qapi.py: Restructure lexer and parser
The parser has a rather unorthodox structure:
Until EOF:
Read a section:
Generator function get_expr() yields one section after the
other, as a string. An unindented, non-empty line that
isn't a comment starts a new section.
Lexing:
Split section into a list of tokens (strings), with help
of generator function tokenize().
Parsing:
Parse the first expression from the list of tokens, with
parse(), throw away any remaining tokens.
In parse_schema(): record value of an enum, union or
struct key (if any) in the appropriate global table,
append expression to the list of expressions.
Return list of expressions.
Known issues:
(1) Indentation is significant, unlike in real JSON.
(2) Neither lexer nor parser have any idea of source positions. Error
reporting is hard, let's go shopping.
(3) The one error we bother to detect, we "report" via raise.
(4) The lexer silently ignores invalid characters.
(5) If everything in a section gets ignored, the parser crashes.
(6) The lexer treats a string containing a structural character exactly
like the structural character.
(7) Tokens trailing the first expression in a section are silently
ignored.
(8) The parser accepts any token in place of a colon.
(9) The parser treats comma as optional.
(10) parse() crashes on unexpected EOF.
(11) parse_schema() crashes when a section's expression isn't a JSON
object.
Replace this piece of original art by a thoroughly unoriginal design.
Takes care of (1), (2), (5), (6) and (7), and lays the groundwork for
addressing the others. Generated source files remain unchanged.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-4-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-27 17:41:55 +02:00
|
|
|
self.cursor = self.src.find('\n', self.cursor)
|
qapi: add qapi2texi script
As the name suggests, the qapi2texi script converts JSON QAPI
description into a texi file suitable for different target
formats (info/man/txt/pdf/html...).
It parses the following kind of blocks:
Free-form:
##
# = Section
# == Subsection
#
# Some text foo with *emphasis*
# 1. with a list
# 2. like that
#
# And some code:
# | $ echo foo
# | -> do this
# | <- get that
#
##
Symbol description:
##
# @symbol:
#
# Symbol body ditto ergo sum. Foo bar
# baz ding.
#
# @param1: the frob to frobnicate
# @param2: #optional how hard to frobnicate
#
# Returns: the frobnicated frob.
# If frob isn't frobnicatable, GenericError.
#
# Since: version
# Notes: notes, comments can have
# - itemized list
# - like this
#
# Example:
#
# -> { "execute": "quit" }
# <- { "return": {} }
#
##
That's roughly following the following EBNF grammar:
api_comment = "##\n" comment "##\n"
comment = freeform_comment | symbol_comment
freeform_comment = { "# " text "\n" | "#\n" }
symbol_comment = "# @" name ":\n" { member | tag_section | freeform_comment }
member = "# @" name ':' [ text ] "\n" freeform_comment
tag_section = "# " ( "Returns:", "Since:", "Note:", "Notes:", "Example:", "Examples:" ) [ text ] "\n" freeform_comment
text = free text with markup
Note that the grammar is ambiguous: a line "# @foo:\n" can be parsed
both as freeform_comment and as symbol_comment. The actual parser
recognizes symbol_comment.
See docs/qapi-code-gen.txt for more details.
Deficiencies and limitations:
- the generated QMP documentation includes internal types
- union type support is lacking
- type information is lacking in generated documentation
- doc comment error message positions are imprecise, they point
to the beginning of the comment.
- a few minor issues, all marked TODO/FIXME in the code
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20170113144135.5150-16-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[test-qapi.py tweaked to avoid trailing empty lines in .out]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2017-01-13 15:41:29 +01:00
|
|
|
if not skip_comment:
|
|
|
|
self.val = self.src[self.pos:self.cursor]
|
|
|
|
return
|
2017-03-15 13:57:08 +01:00
|
|
|
elif self.tok in '{}:,[]':
|
qapi.py: Restructure lexer and parser
The parser has a rather unorthodox structure:
Until EOF:
Read a section:
Generator function get_expr() yields one section after the
other, as a string. An unindented, non-empty line that
isn't a comment starts a new section.
Lexing:
Split section into a list of tokens (strings), with help
of generator function tokenize().
Parsing:
Parse the first expression from the list of tokens, with
parse(), throw away any remaining tokens.
In parse_schema(): record value of an enum, union or
struct key (if any) in the appropriate global table,
append expression to the list of expressions.
Return list of expressions.
Known issues:
(1) Indentation is significant, unlike in real JSON.
(2) Neither lexer nor parser have any idea of source positions. Error
reporting is hard, let's go shopping.
(3) The one error we bother to detect, we "report" via raise.
(4) The lexer silently ignores invalid characters.
(5) If everything in a section gets ignored, the parser crashes.
(6) The lexer treats a string containing a structural character exactly
like the structural character.
(7) Tokens trailing the first expression in a section are silently
ignored.
(8) The parser accepts any token in place of a colon.
(9) The parser treats comma as optional.
(10) parse() crashes on unexpected EOF.
(11) parse_schema() crashes when a section's expression isn't a JSON
object.
Replace this piece of original art by a thoroughly unoriginal design.
Takes care of (1), (2), (5), (6) and (7), and lays the groundwork for
addressing the others. Generated source files remain unchanged.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-4-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-27 17:41:55 +02:00
|
|
|
return
|
|
|
|
elif self.tok == "'":
|
|
|
|
string = ''
|
|
|
|
esc = False
|
|
|
|
while True:
|
|
|
|
ch = self.src[self.cursor]
|
|
|
|
self.cursor += 1
|
|
|
|
if ch == '\n':
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPIParseError(self, 'Missing terminating "\'"')
|
qapi.py: Restructure lexer and parser
The parser has a rather unorthodox structure:
Until EOF:
Read a section:
Generator function get_expr() yields one section after the
other, as a string. An unindented, non-empty line that
isn't a comment starts a new section.
Lexing:
Split section into a list of tokens (strings), with help
of generator function tokenize().
Parsing:
Parse the first expression from the list of tokens, with
parse(), throw away any remaining tokens.
In parse_schema(): record value of an enum, union or
struct key (if any) in the appropriate global table,
append expression to the list of expressions.
Return list of expressions.
Known issues:
(1) Indentation is significant, unlike in real JSON.
(2) Neither lexer nor parser have any idea of source positions. Error
reporting is hard, let's go shopping.
(3) The one error we bother to detect, we "report" via raise.
(4) The lexer silently ignores invalid characters.
(5) If everything in a section gets ignored, the parser crashes.
(6) The lexer treats a string containing a structural character exactly
like the structural character.
(7) Tokens trailing the first expression in a section are silently
ignored.
(8) The parser accepts any token in place of a colon.
(9) The parser treats comma as optional.
(10) parse() crashes on unexpected EOF.
(11) parse_schema() crashes when a section's expression isn't a JSON
object.
Replace this piece of original art by a thoroughly unoriginal design.
Takes care of (1), (2), (5), (6) and (7), and lays the groundwork for
addressing the others. Generated source files remain unchanged.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-4-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-27 17:41:55 +02:00
|
|
|
if esc:
|
qapi: Support (subset of) \u escapes in strings
The handling of \ inside QAPI strings was less than ideal, and
really only worked JSON's \/, \\, \", and our extension of \'
(an obvious extension, when you realize we use '' instead of ""
for strings). For other things, like '\n', it resulted in a
literal 'n' instead of a newline.
Of course, at the moment, we really have no use for escaped
characters, as QAPI has to map to C identifiers, and we currently
support ASCII only for that. But down the road, we may add
support for default values for string parameters to a command
or struct; if that happens, it would be nice to correctly support
all JSON escape sequences, such as \n or \uXXXX. This gets us
closer, by supporting Unicode escapes in the ASCII range.
Since JSON does not require \OCTAL or \xXX escapes, and our QMP
implementation does not understand them either, I intentionally
reject it here, but it would be an easy addition if we desired it.
Likewise, intentionally refusing the NUL byte means we don't have
to worry about C strings being shorter than the qapi input.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-05-04 17:05:36 +02:00
|
|
|
if ch == 'b':
|
|
|
|
string += '\b'
|
|
|
|
elif ch == 'f':
|
|
|
|
string += '\f'
|
|
|
|
elif ch == 'n':
|
|
|
|
string += '\n'
|
|
|
|
elif ch == 'r':
|
|
|
|
string += '\r'
|
|
|
|
elif ch == 't':
|
|
|
|
string += '\t'
|
|
|
|
elif ch == 'u':
|
|
|
|
value = 0
|
2015-09-30 00:21:02 +02:00
|
|
|
for _ in range(0, 4):
|
qapi: Support (subset of) \u escapes in strings
The handling of \ inside QAPI strings was less than ideal, and
really only worked JSON's \/, \\, \", and our extension of \'
(an obvious extension, when you realize we use '' instead of ""
for strings). For other things, like '\n', it resulted in a
literal 'n' instead of a newline.
Of course, at the moment, we really have no use for escaped
characters, as QAPI has to map to C identifiers, and we currently
support ASCII only for that. But down the road, we may add
support for default values for string parameters to a command
or struct; if that happens, it would be nice to correctly support
all JSON escape sequences, such as \n or \uXXXX. This gets us
closer, by supporting Unicode escapes in the ASCII range.
Since JSON does not require \OCTAL or \xXX escapes, and our QMP
implementation does not understand them either, I intentionally
reject it here, but it would be an easy addition if we desired it.
Likewise, intentionally refusing the NUL byte means we don't have
to worry about C strings being shorter than the qapi input.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-05-04 17:05:36 +02:00
|
|
|
ch = self.src[self.cursor]
|
|
|
|
self.cursor += 1
|
2017-03-15 13:57:08 +01:00
|
|
|
if ch not in '0123456789abcdefABCDEF':
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPIParseError(self,
|
|
|
|
'\\u escape needs 4 '
|
|
|
|
'hex digits')
|
qapi: Support (subset of) \u escapes in strings
The handling of \ inside QAPI strings was less than ideal, and
really only worked JSON's \/, \\, \", and our extension of \'
(an obvious extension, when you realize we use '' instead of ""
for strings). For other things, like '\n', it resulted in a
literal 'n' instead of a newline.
Of course, at the moment, we really have no use for escaped
characters, as QAPI has to map to C identifiers, and we currently
support ASCII only for that. But down the road, we may add
support for default values for string parameters to a command
or struct; if that happens, it would be nice to correctly support
all JSON escape sequences, such as \n or \uXXXX. This gets us
closer, by supporting Unicode escapes in the ASCII range.
Since JSON does not require \OCTAL or \xXX escapes, and our QMP
implementation does not understand them either, I intentionally
reject it here, but it would be an easy addition if we desired it.
Likewise, intentionally refusing the NUL byte means we don't have
to worry about C strings being shorter than the qapi input.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-05-04 17:05:36 +02:00
|
|
|
value = (value << 4) + int(ch, 16)
|
|
|
|
# If Python 2 and 3 didn't disagree so much on
|
|
|
|
# how to handle Unicode, then we could allow
|
|
|
|
# Unicode string defaults. But most of QAPI is
|
|
|
|
# ASCII-only, so we aren't losing much for now.
|
|
|
|
if not value or value > 0x7f:
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPIParseError(self,
|
|
|
|
'For now, \\u escape '
|
|
|
|
'only supports non-zero '
|
|
|
|
'values up to \\u007f')
|
qapi: Support (subset of) \u escapes in strings
The handling of \ inside QAPI strings was less than ideal, and
really only worked JSON's \/, \\, \", and our extension of \'
(an obvious extension, when you realize we use '' instead of ""
for strings). For other things, like '\n', it resulted in a
literal 'n' instead of a newline.
Of course, at the moment, we really have no use for escaped
characters, as QAPI has to map to C identifiers, and we currently
support ASCII only for that. But down the road, we may add
support for default values for string parameters to a command
or struct; if that happens, it would be nice to correctly support
all JSON escape sequences, such as \n or \uXXXX. This gets us
closer, by supporting Unicode escapes in the ASCII range.
Since JSON does not require \OCTAL or \xXX escapes, and our QMP
implementation does not understand them either, I intentionally
reject it here, but it would be an easy addition if we desired it.
Likewise, intentionally refusing the NUL byte means we don't have
to worry about C strings being shorter than the qapi input.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-05-04 17:05:36 +02:00
|
|
|
string += chr(value)
|
2017-03-15 13:57:08 +01:00
|
|
|
elif ch in '\\/\'"':
|
qapi: Support (subset of) \u escapes in strings
The handling of \ inside QAPI strings was less than ideal, and
really only worked JSON's \/, \\, \", and our extension of \'
(an obvious extension, when you realize we use '' instead of ""
for strings). For other things, like '\n', it resulted in a
literal 'n' instead of a newline.
Of course, at the moment, we really have no use for escaped
characters, as QAPI has to map to C identifiers, and we currently
support ASCII only for that. But down the road, we may add
support for default values for string parameters to a command
or struct; if that happens, it would be nice to correctly support
all JSON escape sequences, such as \n or \uXXXX. This gets us
closer, by supporting Unicode escapes in the ASCII range.
Since JSON does not require \OCTAL or \xXX escapes, and our QMP
implementation does not understand them either, I intentionally
reject it here, but it would be an easy addition if we desired it.
Likewise, intentionally refusing the NUL byte means we don't have
to worry about C strings being shorter than the qapi input.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-05-04 17:05:36 +02:00
|
|
|
string += ch
|
|
|
|
else:
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPIParseError(self,
|
|
|
|
"Unknown escape \\%s" % ch)
|
qapi.py: Restructure lexer and parser
The parser has a rather unorthodox structure:
Until EOF:
Read a section:
Generator function get_expr() yields one section after the
other, as a string. An unindented, non-empty line that
isn't a comment starts a new section.
Lexing:
Split section into a list of tokens (strings), with help
of generator function tokenize().
Parsing:
Parse the first expression from the list of tokens, with
parse(), throw away any remaining tokens.
In parse_schema(): record value of an enum, union or
struct key (if any) in the appropriate global table,
append expression to the list of expressions.
Return list of expressions.
Known issues:
(1) Indentation is significant, unlike in real JSON.
(2) Neither lexer nor parser have any idea of source positions. Error
reporting is hard, let's go shopping.
(3) The one error we bother to detect, we "report" via raise.
(4) The lexer silently ignores invalid characters.
(5) If everything in a section gets ignored, the parser crashes.
(6) The lexer treats a string containing a structural character exactly
like the structural character.
(7) Tokens trailing the first expression in a section are silently
ignored.
(8) The parser accepts any token in place of a colon.
(9) The parser treats comma as optional.
(10) parse() crashes on unexpected EOF.
(11) parse_schema() crashes when a section's expression isn't a JSON
object.
Replace this piece of original art by a thoroughly unoriginal design.
Takes care of (1), (2), (5), (6) and (7), and lays the groundwork for
addressing the others. Generated source files remain unchanged.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-4-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-27 17:41:55 +02:00
|
|
|
esc = False
|
2017-03-15 13:57:08 +01:00
|
|
|
elif ch == '\\':
|
qapi.py: Restructure lexer and parser
The parser has a rather unorthodox structure:
Until EOF:
Read a section:
Generator function get_expr() yields one section after the
other, as a string. An unindented, non-empty line that
isn't a comment starts a new section.
Lexing:
Split section into a list of tokens (strings), with help
of generator function tokenize().
Parsing:
Parse the first expression from the list of tokens, with
parse(), throw away any remaining tokens.
In parse_schema(): record value of an enum, union or
struct key (if any) in the appropriate global table,
append expression to the list of expressions.
Return list of expressions.
Known issues:
(1) Indentation is significant, unlike in real JSON.
(2) Neither lexer nor parser have any idea of source positions. Error
reporting is hard, let's go shopping.
(3) The one error we bother to detect, we "report" via raise.
(4) The lexer silently ignores invalid characters.
(5) If everything in a section gets ignored, the parser crashes.
(6) The lexer treats a string containing a structural character exactly
like the structural character.
(7) Tokens trailing the first expression in a section are silently
ignored.
(8) The parser accepts any token in place of a colon.
(9) The parser treats comma as optional.
(10) parse() crashes on unexpected EOF.
(11) parse_schema() crashes when a section's expression isn't a JSON
object.
Replace this piece of original art by a thoroughly unoriginal design.
Takes care of (1), (2), (5), (6) and (7), and lays the groundwork for
addressing the others. Generated source files remain unchanged.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-4-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-27 17:41:55 +02:00
|
|
|
esc = True
|
|
|
|
elif ch == "'":
|
|
|
|
self.val = string
|
|
|
|
return
|
|
|
|
else:
|
|
|
|
string += ch
|
2017-03-15 13:57:08 +01:00
|
|
|
elif self.src.startswith('true', self.pos):
|
2015-06-10 08:24:58 +02:00
|
|
|
self.val = True
|
|
|
|
self.cursor += 3
|
|
|
|
return
|
2017-03-15 13:57:08 +01:00
|
|
|
elif self.src.startswith('false', self.pos):
|
2015-06-10 08:24:58 +02:00
|
|
|
self.val = False
|
|
|
|
self.cursor += 4
|
|
|
|
return
|
2017-03-15 13:57:08 +01:00
|
|
|
elif self.src.startswith('null', self.pos):
|
2015-06-10 08:24:58 +02:00
|
|
|
self.val = None
|
|
|
|
self.cursor += 3
|
|
|
|
return
|
qapi.py: Restructure lexer and parser
The parser has a rather unorthodox structure:
Until EOF:
Read a section:
Generator function get_expr() yields one section after the
other, as a string. An unindented, non-empty line that
isn't a comment starts a new section.
Lexing:
Split section into a list of tokens (strings), with help
of generator function tokenize().
Parsing:
Parse the first expression from the list of tokens, with
parse(), throw away any remaining tokens.
In parse_schema(): record value of an enum, union or
struct key (if any) in the appropriate global table,
append expression to the list of expressions.
Return list of expressions.
Known issues:
(1) Indentation is significant, unlike in real JSON.
(2) Neither lexer nor parser have any idea of source positions. Error
reporting is hard, let's go shopping.
(3) The one error we bother to detect, we "report" via raise.
(4) The lexer silently ignores invalid characters.
(5) If everything in a section gets ignored, the parser crashes.
(6) The lexer treats a string containing a structural character exactly
like the structural character.
(7) Tokens trailing the first expression in a section are silently
ignored.
(8) The parser accepts any token in place of a colon.
(9) The parser treats comma as optional.
(10) parse() crashes on unexpected EOF.
(11) parse_schema() crashes when a section's expression isn't a JSON
object.
Replace this piece of original art by a thoroughly unoriginal design.
Takes care of (1), (2), (5), (6) and (7), and lays the groundwork for
addressing the others. Generated source files remain unchanged.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-4-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-27 17:41:55 +02:00
|
|
|
elif self.tok == '\n':
|
|
|
|
if self.cursor == len(self.src):
|
|
|
|
self.tok = None
|
|
|
|
return
|
2014-03-05 03:44:33 +01:00
|
|
|
self.line += 1
|
|
|
|
self.line_pos = self.cursor
|
2013-07-27 17:41:57 +02:00
|
|
|
elif not self.tok.isspace():
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPIParseError(self, 'Stray "%s"' % self.tok)
|
qapi.py: Restructure lexer and parser
The parser has a rather unorthodox structure:
Until EOF:
Read a section:
Generator function get_expr() yields one section after the
other, as a string. An unindented, non-empty line that
isn't a comment starts a new section.
Lexing:
Split section into a list of tokens (strings), with help
of generator function tokenize().
Parsing:
Parse the first expression from the list of tokens, with
parse(), throw away any remaining tokens.
In parse_schema(): record value of an enum, union or
struct key (if any) in the appropriate global table,
append expression to the list of expressions.
Return list of expressions.
Known issues:
(1) Indentation is significant, unlike in real JSON.
(2) Neither lexer nor parser have any idea of source positions. Error
reporting is hard, let's go shopping.
(3) The one error we bother to detect, we "report" via raise.
(4) The lexer silently ignores invalid characters.
(5) If everything in a section gets ignored, the parser crashes.
(6) The lexer treats a string containing a structural character exactly
like the structural character.
(7) Tokens trailing the first expression in a section are silently
ignored.
(8) The parser accepts any token in place of a colon.
(9) The parser treats comma as optional.
(10) parse() crashes on unexpected EOF.
(11) parse_schema() crashes when a section's expression isn't a JSON
object.
Replace this piece of original art by a thoroughly unoriginal design.
Takes care of (1), (2), (5), (6) and (7), and lays the groundwork for
addressing the others. Generated source files remain unchanged.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-4-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-27 17:41:55 +02:00
|
|
|
|
|
|
|
def get_members(self):
|
|
|
|
expr = OrderedDict()
|
2013-07-27 17:41:58 +02:00
|
|
|
if self.tok == '}':
|
|
|
|
self.accept()
|
|
|
|
return expr
|
|
|
|
if self.tok != "'":
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPIParseError(self, 'Expected string or "}"')
|
2013-07-27 17:41:58 +02:00
|
|
|
while True:
|
qapi.py: Restructure lexer and parser
The parser has a rather unorthodox structure:
Until EOF:
Read a section:
Generator function get_expr() yields one section after the
other, as a string. An unindented, non-empty line that
isn't a comment starts a new section.
Lexing:
Split section into a list of tokens (strings), with help
of generator function tokenize().
Parsing:
Parse the first expression from the list of tokens, with
parse(), throw away any remaining tokens.
In parse_schema(): record value of an enum, union or
struct key (if any) in the appropriate global table,
append expression to the list of expressions.
Return list of expressions.
Known issues:
(1) Indentation is significant, unlike in real JSON.
(2) Neither lexer nor parser have any idea of source positions. Error
reporting is hard, let's go shopping.
(3) The one error we bother to detect, we "report" via raise.
(4) The lexer silently ignores invalid characters.
(5) If everything in a section gets ignored, the parser crashes.
(6) The lexer treats a string containing a structural character exactly
like the structural character.
(7) Tokens trailing the first expression in a section are silently
ignored.
(8) The parser accepts any token in place of a colon.
(9) The parser treats comma as optional.
(10) parse() crashes on unexpected EOF.
(11) parse_schema() crashes when a section's expression isn't a JSON
object.
Replace this piece of original art by a thoroughly unoriginal design.
Takes care of (1), (2), (5), (6) and (7), and lays the groundwork for
addressing the others. Generated source files remain unchanged.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-4-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-27 17:41:55 +02:00
|
|
|
key = self.val
|
|
|
|
self.accept()
|
2013-07-27 17:41:58 +02:00
|
|
|
if self.tok != ':':
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPIParseError(self, 'Expected ":"')
|
2013-07-27 17:41:58 +02:00
|
|
|
self.accept()
|
2014-03-05 03:44:32 +01:00
|
|
|
if key in expr:
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPIParseError(self, 'Duplicate key "%s"' % key)
|
2013-07-27 17:41:59 +02:00
|
|
|
expr[key] = self.get_expr(True)
|
2013-07-27 17:41:58 +02:00
|
|
|
if self.tok == '}':
|
qapi.py: Restructure lexer and parser
The parser has a rather unorthodox structure:
Until EOF:
Read a section:
Generator function get_expr() yields one section after the
other, as a string. An unindented, non-empty line that
isn't a comment starts a new section.
Lexing:
Split section into a list of tokens (strings), with help
of generator function tokenize().
Parsing:
Parse the first expression from the list of tokens, with
parse(), throw away any remaining tokens.
In parse_schema(): record value of an enum, union or
struct key (if any) in the appropriate global table,
append expression to the list of expressions.
Return list of expressions.
Known issues:
(1) Indentation is significant, unlike in real JSON.
(2) Neither lexer nor parser have any idea of source positions. Error
reporting is hard, let's go shopping.
(3) The one error we bother to detect, we "report" via raise.
(4) The lexer silently ignores invalid characters.
(5) If everything in a section gets ignored, the parser crashes.
(6) The lexer treats a string containing a structural character exactly
like the structural character.
(7) Tokens trailing the first expression in a section are silently
ignored.
(8) The parser accepts any token in place of a colon.
(9) The parser treats comma as optional.
(10) parse() crashes on unexpected EOF.
(11) parse_schema() crashes when a section's expression isn't a JSON
object.
Replace this piece of original art by a thoroughly unoriginal design.
Takes care of (1), (2), (5), (6) and (7), and lays the groundwork for
addressing the others. Generated source files remain unchanged.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-4-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-27 17:41:55 +02:00
|
|
|
self.accept()
|
2013-07-27 17:41:58 +02:00
|
|
|
return expr
|
|
|
|
if self.tok != ',':
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPIParseError(self, 'Expected "," or "}"')
|
2013-07-27 17:41:58 +02:00
|
|
|
self.accept()
|
|
|
|
if self.tok != "'":
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPIParseError(self, 'Expected string')
|
qapi.py: Restructure lexer and parser
The parser has a rather unorthodox structure:
Until EOF:
Read a section:
Generator function get_expr() yields one section after the
other, as a string. An unindented, non-empty line that
isn't a comment starts a new section.
Lexing:
Split section into a list of tokens (strings), with help
of generator function tokenize().
Parsing:
Parse the first expression from the list of tokens, with
parse(), throw away any remaining tokens.
In parse_schema(): record value of an enum, union or
struct key (if any) in the appropriate global table,
append expression to the list of expressions.
Return list of expressions.
Known issues:
(1) Indentation is significant, unlike in real JSON.
(2) Neither lexer nor parser have any idea of source positions. Error
reporting is hard, let's go shopping.
(3) The one error we bother to detect, we "report" via raise.
(4) The lexer silently ignores invalid characters.
(5) If everything in a section gets ignored, the parser crashes.
(6) The lexer treats a string containing a structural character exactly
like the structural character.
(7) Tokens trailing the first expression in a section are silently
ignored.
(8) The parser accepts any token in place of a colon.
(9) The parser treats comma as optional.
(10) parse() crashes on unexpected EOF.
(11) parse_schema() crashes when a section's expression isn't a JSON
object.
Replace this piece of original art by a thoroughly unoriginal design.
Takes care of (1), (2), (5), (6) and (7), and lays the groundwork for
addressing the others. Generated source files remain unchanged.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-4-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-27 17:41:55 +02:00
|
|
|
|
|
|
|
def get_values(self):
|
|
|
|
expr = []
|
2013-07-27 17:41:58 +02:00
|
|
|
if self.tok == ']':
|
|
|
|
self.accept()
|
|
|
|
return expr
|
2015-09-30 00:21:02 +02:00
|
|
|
if self.tok not in "{['tfn":
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPIParseError(self, 'Expected "{", "[", "]", string, '
|
|
|
|
'boolean or "null"')
|
2013-07-27 17:41:58 +02:00
|
|
|
while True:
|
2013-07-27 17:41:59 +02:00
|
|
|
expr.append(self.get_expr(True))
|
2013-07-27 17:41:58 +02:00
|
|
|
if self.tok == ']':
|
qapi.py: Restructure lexer and parser
The parser has a rather unorthodox structure:
Until EOF:
Read a section:
Generator function get_expr() yields one section after the
other, as a string. An unindented, non-empty line that
isn't a comment starts a new section.
Lexing:
Split section into a list of tokens (strings), with help
of generator function tokenize().
Parsing:
Parse the first expression from the list of tokens, with
parse(), throw away any remaining tokens.
In parse_schema(): record value of an enum, union or
struct key (if any) in the appropriate global table,
append expression to the list of expressions.
Return list of expressions.
Known issues:
(1) Indentation is significant, unlike in real JSON.
(2) Neither lexer nor parser have any idea of source positions. Error
reporting is hard, let's go shopping.
(3) The one error we bother to detect, we "report" via raise.
(4) The lexer silently ignores invalid characters.
(5) If everything in a section gets ignored, the parser crashes.
(6) The lexer treats a string containing a structural character exactly
like the structural character.
(7) Tokens trailing the first expression in a section are silently
ignored.
(8) The parser accepts any token in place of a colon.
(9) The parser treats comma as optional.
(10) parse() crashes on unexpected EOF.
(11) parse_schema() crashes when a section's expression isn't a JSON
object.
Replace this piece of original art by a thoroughly unoriginal design.
Takes care of (1), (2), (5), (6) and (7), and lays the groundwork for
addressing the others. Generated source files remain unchanged.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-4-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-27 17:41:55 +02:00
|
|
|
self.accept()
|
2013-07-27 17:41:58 +02:00
|
|
|
return expr
|
|
|
|
if self.tok != ',':
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPIParseError(self, 'Expected "," or "]"')
|
2013-07-27 17:41:58 +02:00
|
|
|
self.accept()
|
qapi.py: Restructure lexer and parser
The parser has a rather unorthodox structure:
Until EOF:
Read a section:
Generator function get_expr() yields one section after the
other, as a string. An unindented, non-empty line that
isn't a comment starts a new section.
Lexing:
Split section into a list of tokens (strings), with help
of generator function tokenize().
Parsing:
Parse the first expression from the list of tokens, with
parse(), throw away any remaining tokens.
In parse_schema(): record value of an enum, union or
struct key (if any) in the appropriate global table,
append expression to the list of expressions.
Return list of expressions.
Known issues:
(1) Indentation is significant, unlike in real JSON.
(2) Neither lexer nor parser have any idea of source positions. Error
reporting is hard, let's go shopping.
(3) The one error we bother to detect, we "report" via raise.
(4) The lexer silently ignores invalid characters.
(5) If everything in a section gets ignored, the parser crashes.
(6) The lexer treats a string containing a structural character exactly
like the structural character.
(7) Tokens trailing the first expression in a section are silently
ignored.
(8) The parser accepts any token in place of a colon.
(9) The parser treats comma as optional.
(10) parse() crashes on unexpected EOF.
(11) parse_schema() crashes when a section's expression isn't a JSON
object.
Replace this piece of original art by a thoroughly unoriginal design.
Takes care of (1), (2), (5), (6) and (7), and lays the groundwork for
addressing the others. Generated source files remain unchanged.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-4-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-27 17:41:55 +02:00
|
|
|
|
2013-07-27 17:41:59 +02:00
|
|
|
def get_expr(self, nested):
|
|
|
|
if self.tok != '{' and not nested:
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPIParseError(self, 'Expected "{"')
|
qapi.py: Restructure lexer and parser
The parser has a rather unorthodox structure:
Until EOF:
Read a section:
Generator function get_expr() yields one section after the
other, as a string. An unindented, non-empty line that
isn't a comment starts a new section.
Lexing:
Split section into a list of tokens (strings), with help
of generator function tokenize().
Parsing:
Parse the first expression from the list of tokens, with
parse(), throw away any remaining tokens.
In parse_schema(): record value of an enum, union or
struct key (if any) in the appropriate global table,
append expression to the list of expressions.
Return list of expressions.
Known issues:
(1) Indentation is significant, unlike in real JSON.
(2) Neither lexer nor parser have any idea of source positions. Error
reporting is hard, let's go shopping.
(3) The one error we bother to detect, we "report" via raise.
(4) The lexer silently ignores invalid characters.
(5) If everything in a section gets ignored, the parser crashes.
(6) The lexer treats a string containing a structural character exactly
like the structural character.
(7) Tokens trailing the first expression in a section are silently
ignored.
(8) The parser accepts any token in place of a colon.
(9) The parser treats comma as optional.
(10) parse() crashes on unexpected EOF.
(11) parse_schema() crashes when a section's expression isn't a JSON
object.
Replace this piece of original art by a thoroughly unoriginal design.
Takes care of (1), (2), (5), (6) and (7), and lays the groundwork for
addressing the others. Generated source files remain unchanged.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-4-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-27 17:41:55 +02:00
|
|
|
if self.tok == '{':
|
|
|
|
self.accept()
|
|
|
|
expr = self.get_members()
|
|
|
|
elif self.tok == '[':
|
|
|
|
self.accept()
|
|
|
|
expr = self.get_values()
|
qapi: Allow true, false and null in schema json
In the near term, we will use it for a sensible-looking
'gen':false inside command declarations, instead of the
current ugly 'gen':'no'.
In the long term, it will allow conversion from shorthand
with defaults mentioned only in side-band documentation:
'data':{'*flag':'bool', '*string':'str'}
into an explicit default value documentation, as in:
'data':{'flag':{'type':'bool', 'optional':true, 'default':true},
'string':{'type':'str', 'optional':true, 'default':null}}
We still don't parse integer values (also necessary before
we can allow explicit defaults), but that can come in a later
series.
Update the testsuite to match an improved error message.
Signed-off-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-05-04 17:05:18 +02:00
|
|
|
elif self.tok in "'tfn":
|
qapi.py: Restructure lexer and parser
The parser has a rather unorthodox structure:
Until EOF:
Read a section:
Generator function get_expr() yields one section after the
other, as a string. An unindented, non-empty line that
isn't a comment starts a new section.
Lexing:
Split section into a list of tokens (strings), with help
of generator function tokenize().
Parsing:
Parse the first expression from the list of tokens, with
parse(), throw away any remaining tokens.
In parse_schema(): record value of an enum, union or
struct key (if any) in the appropriate global table,
append expression to the list of expressions.
Return list of expressions.
Known issues:
(1) Indentation is significant, unlike in real JSON.
(2) Neither lexer nor parser have any idea of source positions. Error
reporting is hard, let's go shopping.
(3) The one error we bother to detect, we "report" via raise.
(4) The lexer silently ignores invalid characters.
(5) If everything in a section gets ignored, the parser crashes.
(6) The lexer treats a string containing a structural character exactly
like the structural character.
(7) Tokens trailing the first expression in a section are silently
ignored.
(8) The parser accepts any token in place of a colon.
(9) The parser treats comma as optional.
(10) parse() crashes on unexpected EOF.
(11) parse_schema() crashes when a section's expression isn't a JSON
object.
Replace this piece of original art by a thoroughly unoriginal design.
Takes care of (1), (2), (5), (6) and (7), and lays the groundwork for
addressing the others. Generated source files remain unchanged.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-4-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-27 17:41:55 +02:00
|
|
|
expr = self.val
|
|
|
|
self.accept()
|
2013-07-27 17:41:58 +02:00
|
|
|
else:
|
2017-03-15 13:57:36 +01:00
|
|
|
raise QAPIParseError(self, 'Expected "{", "[", string, '
|
|
|
|
'boolean or "null"')
|
qapi.py: Restructure lexer and parser
The parser has a rather unorthodox structure:
Until EOF:
Read a section:
Generator function get_expr() yields one section after the
other, as a string. An unindented, non-empty line that
isn't a comment starts a new section.
Lexing:
Split section into a list of tokens (strings), with help
of generator function tokenize().
Parsing:
Parse the first expression from the list of tokens, with
parse(), throw away any remaining tokens.
In parse_schema(): record value of an enum, union or
struct key (if any) in the appropriate global table,
append expression to the list of expressions.
Return list of expressions.
Known issues:
(1) Indentation is significant, unlike in real JSON.
(2) Neither lexer nor parser have any idea of source positions. Error
reporting is hard, let's go shopping.
(3) The one error we bother to detect, we "report" via raise.
(4) The lexer silently ignores invalid characters.
(5) If everything in a section gets ignored, the parser crashes.
(6) The lexer treats a string containing a structural character exactly
like the structural character.
(7) Tokens trailing the first expression in a section are silently
ignored.
(8) The parser accepts any token in place of a colon.
(9) The parser treats comma as optional.
(10) parse() crashes on unexpected EOF.
(11) parse_schema() crashes when a section's expression isn't a JSON
object.
Replace this piece of original art by a thoroughly unoriginal design.
Takes care of (1), (2), (5), (6) and (7), and lays the groundwork for
addressing the others. Generated source files remain unchanged.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-4-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-27 17:41:55 +02:00
|
|
|
return expr
|
2013-07-01 16:31:50 +02:00
|
|
|
|
qapi: add qapi2texi script
As the name suggests, the qapi2texi script converts JSON QAPI
description into a texi file suitable for different target
formats (info/man/txt/pdf/html...).
It parses the following kind of blocks:
Free-form:
##
# = Section
# == Subsection
#
# Some text foo with *emphasis*
# 1. with a list
# 2. like that
#
# And some code:
# | $ echo foo
# | -> do this
# | <- get that
#
##
Symbol description:
##
# @symbol:
#
# Symbol body ditto ergo sum. Foo bar
# baz ding.
#
# @param1: the frob to frobnicate
# @param2: #optional how hard to frobnicate
#
# Returns: the frobnicated frob.
# If frob isn't frobnicatable, GenericError.
#
# Since: version
# Notes: notes, comments can have
# - itemized list
# - like this
#
# Example:
#
# -> { "execute": "quit" }
# <- { "return": {} }
#
##
That's roughly following the following EBNF grammar:
api_comment = "##\n" comment "##\n"
comment = freeform_comment | symbol_comment
freeform_comment = { "# " text "\n" | "#\n" }
symbol_comment = "# @" name ":\n" { member | tag_section | freeform_comment }
member = "# @" name ':' [ text ] "\n" freeform_comment
tag_section = "# " ( "Returns:", "Since:", "Note:", "Notes:", "Example:", "Examples:" ) [ text ] "\n" freeform_comment
text = free text with markup
Note that the grammar is ambiguous: a line "# @foo:\n" can be parsed
both as freeform_comment and as symbol_comment. The actual parser
recognizes symbol_comment.
See docs/qapi-code-gen.txt for more details.
Deficiencies and limitations:
- the generated QMP documentation includes internal types
- union type support is lacking
- type information is lacking in generated documentation
- doc comment error message positions are imprecise, they point
to the beginning of the comment.
- a few minor issues, all marked TODO/FIXME in the code
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20170113144135.5150-16-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[test-qapi.py tweaked to avoid trailing empty lines in .out]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2017-01-13 15:41:29 +01:00
|
|
|
def get_doc(self, info):
|
|
|
|
if self.val != '##':
|
|
|
|
raise QAPIParseError(self, "Junk after '##' at start of "
|
|
|
|
"documentation comment")
|
|
|
|
|
|
|
|
doc = QAPIDoc(self, info)
|
|
|
|
self.accept(False)
|
|
|
|
while self.tok == '#':
|
|
|
|
if self.val.startswith('##'):
|
|
|
|
# End of doc comment
|
|
|
|
if self.val != '##':
|
|
|
|
raise QAPIParseError(self, "Junk after '##' at end of "
|
|
|
|
"documentation comment")
|
2017-03-15 13:57:23 +01:00
|
|
|
doc.end_comment()
|
qapi: add qapi2texi script
As the name suggests, the qapi2texi script converts JSON QAPI
description into a texi file suitable for different target
formats (info/man/txt/pdf/html...).
It parses the following kind of blocks:
Free-form:
##
# = Section
# == Subsection
#
# Some text foo with *emphasis*
# 1. with a list
# 2. like that
#
# And some code:
# | $ echo foo
# | -> do this
# | <- get that
#
##
Symbol description:
##
# @symbol:
#
# Symbol body ditto ergo sum. Foo bar
# baz ding.
#
# @param1: the frob to frobnicate
# @param2: #optional how hard to frobnicate
#
# Returns: the frobnicated frob.
# If frob isn't frobnicatable, GenericError.
#
# Since: version
# Notes: notes, comments can have
# - itemized list
# - like this
#
# Example:
#
# -> { "execute": "quit" }
# <- { "return": {} }
#
##
That's roughly following the following EBNF grammar:
api_comment = "##\n" comment "##\n"
comment = freeform_comment | symbol_comment
freeform_comment = { "# " text "\n" | "#\n" }
symbol_comment = "# @" name ":\n" { member | tag_section | freeform_comment }
member = "# @" name ':' [ text ] "\n" freeform_comment
tag_section = "# " ( "Returns:", "Since:", "Note:", "Notes:", "Example:", "Examples:" ) [ text ] "\n" freeform_comment
text = free text with markup
Note that the grammar is ambiguous: a line "# @foo:\n" can be parsed
both as freeform_comment and as symbol_comment. The actual parser
recognizes symbol_comment.
See docs/qapi-code-gen.txt for more details.
Deficiencies and limitations:
- the generated QMP documentation includes internal types
- union type support is lacking
- type information is lacking in generated documentation
- doc comment error message positions are imprecise, they point
to the beginning of the comment.
- a few minor issues, all marked TODO/FIXME in the code
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20170113144135.5150-16-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[test-qapi.py tweaked to avoid trailing empty lines in .out]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2017-01-13 15:41:29 +01:00
|
|
|
self.accept()
|
|
|
|
return doc
|
|
|
|
else:
|
|
|
|
doc.append(self.val)
|
|
|
|
self.accept(False)
|
|
|
|
|
|
|
|
raise QAPIParseError(self, "Documentation comment must end with '##'")
|
|
|
|
|
|
|
|
|
2015-06-10 10:04:36 +02:00
|
|
|
#
|
|
|
|
# Semantic analysis of schema expressions
|
2015-09-16 13:06:05 +02:00
|
|
|
# TODO fold into QAPISchema
|
|
|
|
# TODO catching name collisions in generated code would be nice
|
2015-06-10 10:04:36 +02:00
|
|
|
#
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2016-03-03 17:16:43 +01:00
|
|
|
def find_base_members(base):
|
2016-03-17 23:48:39 +01:00
|
|
|
if isinstance(base, dict):
|
|
|
|
return base
|
2017-03-15 13:57:32 +01:00
|
|
|
base_struct_define = struct_types.get(base)
|
2014-03-05 03:44:34 +01:00
|
|
|
if not base_struct_define:
|
|
|
|
return None
|
|
|
|
return base_struct_define['data']
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2015-05-04 17:05:10 +02:00
|
|
|
# Return the qtype of an alternate branch, or None on error.
|
|
|
|
def find_alternate_member_qtype(qapi_type):
|
2015-09-30 00:21:02 +02:00
|
|
|
if qapi_type in builtin_types:
|
2015-05-04 17:05:08 +02:00
|
|
|
return builtin_types[qapi_type]
|
2017-03-15 13:57:32 +01:00
|
|
|
elif qapi_type in struct_types:
|
2017-03-15 13:57:08 +01:00
|
|
|
return 'QTYPE_QDICT'
|
2017-03-15 13:57:31 +01:00
|
|
|
elif qapi_type in enum_types:
|
2017-03-15 13:57:08 +01:00
|
|
|
return 'QTYPE_QSTRING'
|
2017-03-15 13:57:33 +01:00
|
|
|
elif qapi_type in union_types:
|
2017-03-15 13:57:08 +01:00
|
|
|
return 'QTYPE_QDICT'
|
2015-05-04 17:05:08 +02:00
|
|
|
return None
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2014-03-07 02:08:56 +01:00
|
|
|
# Return the discriminator enum define if discriminator is specified as an
|
|
|
|
# enum type, otherwise return None.
|
|
|
|
def discriminator_find_enum_define(expr):
|
|
|
|
base = expr.get('base')
|
|
|
|
discriminator = expr.get('discriminator')
|
|
|
|
|
|
|
|
if not (discriminator and base):
|
|
|
|
return None
|
|
|
|
|
2016-03-03 17:16:43 +01:00
|
|
|
base_members = find_base_members(base)
|
|
|
|
if not base_members:
|
2014-03-07 02:08:56 +01:00
|
|
|
return None
|
|
|
|
|
2016-03-03 17:16:43 +01:00
|
|
|
discriminator_type = base_members.get(discriminator)
|
2014-03-07 02:08:56 +01:00
|
|
|
if not discriminator_type:
|
|
|
|
return None
|
|
|
|
|
2017-03-15 13:57:31 +01:00
|
|
|
return enum_types.get(discriminator_type)
|
2014-03-07 02:08:56 +01:00
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2015-11-18 09:52:56 +01:00
|
|
|
# Names must be letters, numbers, -, and _. They must start with letter,
|
|
|
|
# except for downstream extensions which must start with __RFQDN_.
|
|
|
|
# Dots are only valid in the downstream extension prefix.
|
2017-03-15 13:57:07 +01:00
|
|
|
valid_name = re.compile(r'^(__[a-zA-Z0-9.-]+_)?'
|
2015-11-18 09:52:56 +01:00
|
|
|
'[a-zA-Z][a-zA-Z0-9_-]*$')
|
2015-09-30 00:21:02 +02:00
|
|
|
|
|
|
|
|
2017-01-13 15:41:25 +01:00
|
|
|
def check_name(info, source, name, allow_optional=False,
|
2015-09-30 00:21:02 +02:00
|
|
|
enum_member=False):
|
2015-05-04 17:05:22 +02:00
|
|
|
global valid_name
|
|
|
|
membername = name
|
|
|
|
|
|
|
|
if not isinstance(name, str):
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPISemError(info, "%s requires a string name" % source)
|
2015-05-04 17:05:22 +02:00
|
|
|
if name.startswith('*'):
|
|
|
|
membername = name[1:]
|
|
|
|
if not allow_optional:
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPISemError(info, "%s does not allow optional name '%s'"
|
|
|
|
% (source, name))
|
2015-05-04 17:05:22 +02:00
|
|
|
# Enum members can start with a digit, because the generated C
|
|
|
|
# code always prefixes it with the enum name
|
2015-11-18 09:52:56 +01:00
|
|
|
if enum_member and membername[0].isdigit():
|
|
|
|
membername = 'D' + membername
|
2016-03-17 23:48:29 +01:00
|
|
|
# Reserve the entire 'q_' namespace for c_name(), and for 'q_empty'
|
|
|
|
# and 'q_obj_*' implicit type names.
|
qapi: Reserve 'q_*' and 'has_*' member names
c_name() produces names starting with 'q_' when protecting a
dictionary member name that would fail to directly compile, but
in doing so can cause clashes with any member name already
beginning with 'q-' or 'q_'. Likewise, we create a C name 'has_'
for any optional member that can clash with any member name
beginning with 'has-' or 'has_'.
Technically, rather than blindly reserving the namespace,
we could try to complain about user names only when an actual
collision occurs, or even teach c_name() how to munge names
to avoid collisions. But it is not trivial, especially when
collisions can occur across multiple types (such as via
inheritance or flat unions). Besides, no existing .json
files are trying to use these names. So it's easier to just
outright forbid the potential for collision. We can always
relax things in the future if a real need arises for QMP to
express member names that have been forbidden here.
'has_' only has to be reserved for struct/union member names,
while 'q_' is reserved everywhere (matching the fact that
only members can be optional, while we use c_name() for munging
both members and entities). Note that we could relax 'q_'
restrictions on entities independently from member names; for
example, c_name('qmp_' + 'unix') would result in a different
function name than our current 'qmp_' + c_name('unix').
Update and add tests to cover the new error messages.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1445898903-12082-6-git-send-email-eblake@redhat.com>
[Consistently pass protect=False to c_name(); commit message tweaked
slightly]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-10-26 23:34:44 +01:00
|
|
|
if not valid_name.match(membername) or \
|
|
|
|
c_name(membername, False).startswith('q_'):
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPISemError(info, "%s uses invalid name '%s'" % (source, name))
|
2015-05-04 17:05:22 +02:00
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
|
|
|
def add_name(name, info, meta, implicit=False):
|
2015-06-10 10:04:36 +02:00
|
|
|
global all_names
|
|
|
|
check_name(info, "'%s'" % meta, name)
|
2015-07-31 11:33:52 +02:00
|
|
|
# FIXME should reject names that differ only in '_' vs. '.'
|
|
|
|
# vs. '-', because they're liable to clash in generated C.
|
2015-06-10 10:04:36 +02:00
|
|
|
if name in all_names:
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPISemError(info, "%s '%s' is already defined"
|
|
|
|
% (all_names[name], name))
|
2015-10-26 23:34:43 +01:00
|
|
|
if not implicit and (name.endswith('Kind') or name.endswith('List')):
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPISemError(info, "%s '%s' should not end in '%s'"
|
|
|
|
% (meta, name, name[-4:]))
|
2015-06-10 10:04:36 +02:00
|
|
|
all_names[name] = meta
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2017-01-13 15:41:25 +01:00
|
|
|
def check_type(info, source, value, allow_array=False,
|
2015-09-30 00:21:02 +02:00
|
|
|
allow_dict=False, allow_optional=False,
|
|
|
|
allow_metas=[]):
|
2015-05-04 17:05:21 +02:00
|
|
|
global all_names
|
|
|
|
|
|
|
|
if value is None:
|
|
|
|
return
|
|
|
|
|
|
|
|
# Check if array type for value is okay
|
|
|
|
if isinstance(value, list):
|
|
|
|
if not allow_array:
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPISemError(info, "%s cannot be an array" % source)
|
2015-05-04 17:05:21 +02:00
|
|
|
if len(value) != 1 or not isinstance(value[0], str):
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPISemError(info,
|
|
|
|
"%s: array type must contain single type name" %
|
|
|
|
source)
|
2015-05-04 17:05:21 +02:00
|
|
|
value = value[0]
|
|
|
|
|
|
|
|
# Check if type name for value is okay
|
|
|
|
if isinstance(value, str):
|
2015-09-30 00:21:02 +02:00
|
|
|
if value not in all_names:
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPISemError(info, "%s uses unknown type '%s'"
|
|
|
|
% (source, value))
|
2015-05-04 17:05:21 +02:00
|
|
|
if not all_names[value] in allow_metas:
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPISemError(info, "%s cannot use %s type '%s'" %
|
|
|
|
(source, all_names[value], value))
|
2015-05-04 17:05:21 +02:00
|
|
|
return
|
|
|
|
|
|
|
|
if not allow_dict:
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPISemError(info, "%s should be a type name" % source)
|
2015-08-31 17:28:52 +02:00
|
|
|
|
|
|
|
if not isinstance(value, OrderedDict):
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPISemError(info,
|
|
|
|
"%s should be a dictionary or type name" % source)
|
2015-08-31 17:28:52 +02:00
|
|
|
|
|
|
|
# value is a dictionary, check that each member is okay
|
2015-05-04 17:05:21 +02:00
|
|
|
for (key, arg) in value.items():
|
2017-01-13 15:41:25 +01:00
|
|
|
check_name(info, "Member of %s" % source, key,
|
2015-05-04 17:05:22 +02:00
|
|
|
allow_optional=allow_optional)
|
2015-10-26 23:35:02 +01:00
|
|
|
if c_name(key, False) == 'u' or c_name(key, False).startswith('has_'):
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPISemError(info, "Member of %s uses reserved name '%s'"
|
|
|
|
% (source, key))
|
2015-05-04 17:05:33 +02:00
|
|
|
# Todo: allow dictionaries to represent default values of
|
|
|
|
# an optional argument.
|
2017-01-13 15:41:25 +01:00
|
|
|
check_type(info, "Member '%s' of %s" % (key, source), arg,
|
2015-09-16 13:06:27 +02:00
|
|
|
allow_array=True,
|
2015-05-04 17:05:21 +02:00
|
|
|
allow_metas=['built-in', 'union', 'alternate', 'struct',
|
2015-05-04 17:05:33 +02:00
|
|
|
'enum'])
|
2015-05-04 17:05:21 +02:00
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2017-01-13 15:41:25 +01:00
|
|
|
def check_command(expr, info):
|
2015-05-04 17:05:21 +02:00
|
|
|
name = expr['command']
|
2016-07-14 05:50:20 +02:00
|
|
|
boxed = expr.get('boxed', False)
|
2015-05-04 17:05:24 +02:00
|
|
|
|
2016-07-14 05:50:20 +02:00
|
|
|
args_meta = ['struct']
|
|
|
|
if boxed:
|
|
|
|
args_meta += ['union', 'alternate']
|
2017-01-13 15:41:25 +01:00
|
|
|
check_type(info, "'data' for command '%s'" % name,
|
2016-07-14 05:50:20 +02:00
|
|
|
expr.get('data'), allow_dict=not boxed, allow_optional=True,
|
|
|
|
allow_metas=args_meta)
|
2015-05-04 17:05:23 +02:00
|
|
|
returns_meta = ['union', 'struct']
|
|
|
|
if name in returns_whitelist:
|
|
|
|
returns_meta += ['built-in', 'alternate', 'enum']
|
2017-01-13 15:41:25 +01:00
|
|
|
check_type(info, "'returns' for command '%s'" % name,
|
qapi: Command returning anonymous type doesn't work, outlaw
Reproducer: with
{ 'command': 'user_def_cmd4', 'returns': { 'a': 'int' } }
added to qapi-schema-test.json, qapi-commands.py dies when it tries to
generate the command handler function
Traceback (most recent call last):
File "/work/armbru/qemu/scripts/qapi-commands.py", line 359, in <module>
ret = generate_command_decl(cmd['command'], arglist, ret_type) + "\n"
File "/work/armbru/qemu/scripts/qapi-commands.py", line 29, in generate_command_decl
ret_type=c_type(ret_type), name=c_name(name),
File "/work/armbru/qemu/scripts/qapi.py", line 927, in c_type
assert isinstance(value, str) and value != ""
AssertionError
because the return type doesn't exist.
Simply outlaw this usage, and drop or dumb down test cases accordingly.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2015-07-31 17:59:38 +02:00
|
|
|
expr.get('returns'), allow_array=True,
|
2015-09-16 13:06:27 +02:00
|
|
|
allow_optional=True, allow_metas=returns_meta)
|
2015-05-04 17:05:21 +02:00
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2017-01-13 15:41:25 +01:00
|
|
|
def check_event(expr, info):
|
2015-05-04 17:05:17 +02:00
|
|
|
name = expr['event']
|
2016-07-14 05:50:20 +02:00
|
|
|
boxed = expr.get('boxed', False)
|
2015-05-04 17:05:17 +02:00
|
|
|
|
2016-07-14 05:50:20 +02:00
|
|
|
meta = ['struct']
|
|
|
|
if boxed:
|
|
|
|
meta += ['union', 'alternate']
|
2017-01-13 15:41:25 +01:00
|
|
|
check_type(info, "'data' for event '%s'" % name,
|
2016-07-14 05:50:20 +02:00
|
|
|
expr.get('data'), allow_dict=not boxed, allow_optional=True,
|
|
|
|
allow_metas=meta)
|
2014-06-18 08:43:28 +02:00
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2017-01-13 15:41:25 +01:00
|
|
|
def check_union(expr, info):
|
2014-03-05 03:44:34 +01:00
|
|
|
name = expr['union']
|
|
|
|
base = expr.get('base')
|
|
|
|
discriminator = expr.get('discriminator')
|
|
|
|
members = expr['data']
|
|
|
|
|
2015-05-04 17:05:10 +02:00
|
|
|
# Two types of unions, determined by discriminator.
|
|
|
|
|
|
|
|
# With no discriminator it is a simple union.
|
|
|
|
if discriminator is None:
|
2014-03-05 03:44:34 +01:00
|
|
|
enum_define = None
|
2015-09-30 00:21:02 +02:00
|
|
|
allow_metas = ['built-in', 'union', 'alternate', 'struct', 'enum']
|
2015-05-04 17:05:08 +02:00
|
|
|
if base is not None:
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPISemError(info, "Simple union '%s' must not have a base" %
|
|
|
|
name)
|
2014-03-05 03:44:34 +01:00
|
|
|
|
|
|
|
# Else, it's a flat union.
|
|
|
|
else:
|
2016-03-17 23:48:39 +01:00
|
|
|
# The object must have a string or dictionary 'base'.
|
2017-01-13 15:41:25 +01:00
|
|
|
check_type(info, "'base' for union '%s'" % name,
|
2016-03-17 23:48:39 +01:00
|
|
|
base, allow_dict=True, allow_optional=True,
|
|
|
|
allow_metas=['struct'])
|
2015-09-30 00:21:07 +02:00
|
|
|
if not base:
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPISemError(info, "Flat union '%s' must have a base"
|
|
|
|
% name)
|
2016-03-03 17:16:43 +01:00
|
|
|
base_members = find_base_members(base)
|
2017-03-15 13:56:58 +01:00
|
|
|
assert base_members is not None
|
2015-05-04 17:05:08 +02:00
|
|
|
|
2015-05-04 17:05:22 +02:00
|
|
|
# The value of member 'discriminator' must name a non-optional
|
2015-05-04 17:05:25 +02:00
|
|
|
# member of the base struct.
|
2017-01-13 15:41:25 +01:00
|
|
|
check_name(info, "Discriminator of flat union '%s'" % name,
|
2015-05-04 17:05:22 +02:00
|
|
|
discriminator)
|
2016-03-03 17:16:43 +01:00
|
|
|
discriminator_type = base_members.get(discriminator)
|
2014-03-05 03:44:34 +01:00
|
|
|
if not discriminator_type:
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPISemError(info,
|
|
|
|
"Discriminator '%s' is not a member of base "
|
|
|
|
"struct '%s'"
|
|
|
|
% (discriminator, base))
|
2017-03-15 13:57:31 +01:00
|
|
|
enum_define = enum_types.get(discriminator_type)
|
2015-09-30 00:21:02 +02:00
|
|
|
allow_metas = ['struct']
|
2014-03-05 03:44:39 +01:00
|
|
|
# Do not allow string discriminator
|
|
|
|
if not enum_define:
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPISemError(info,
|
|
|
|
"Discriminator '%s' must be of enumeration "
|
|
|
|
"type" % discriminator)
|
2014-03-05 03:44:34 +01:00
|
|
|
|
2016-02-18 07:48:16 +01:00
|
|
|
# Check every branch; don't allow an empty union
|
|
|
|
if len(members) == 0:
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPISemError(info, "Union '%s' cannot have empty 'data'" % name)
|
2014-03-05 03:44:34 +01:00
|
|
|
for (key, value) in members.items():
|
2017-01-13 15:41:25 +01:00
|
|
|
check_name(info, "Member of union '%s'" % name, key)
|
2015-05-04 17:05:22 +02:00
|
|
|
|
2015-12-02 06:20:58 +01:00
|
|
|
# Each value must name a known type
|
2017-01-13 15:41:25 +01:00
|
|
|
check_type(info, "Member '%s' of union '%s'" % (key, name),
|
2015-06-10 13:07:43 +02:00
|
|
|
value, allow_array=not base, allow_metas=allow_metas)
|
2015-05-04 17:05:21 +02:00
|
|
|
|
2015-05-04 17:05:08 +02:00
|
|
|
# If the discriminator names an enum type, then all members
|
2015-11-18 09:52:49 +01:00
|
|
|
# of 'data' must also be members of the enum type.
|
2015-05-04 17:05:08 +02:00
|
|
|
if enum_define:
|
2017-03-15 13:57:29 +01:00
|
|
|
if key not in enum_define['data']:
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPISemError(info,
|
|
|
|
"Discriminator value '%s' is not found in "
|
|
|
|
"enum '%s'"
|
2017-03-15 13:57:29 +01:00
|
|
|
% (key, enum_define['enum']))
|
2015-05-04 17:05:08 +02:00
|
|
|
|
2016-07-14 05:50:13 +02:00
|
|
|
# If discriminator is user-defined, ensure all values are covered
|
|
|
|
if enum_define:
|
2017-03-15 13:57:29 +01:00
|
|
|
for value in enum_define['data']:
|
2016-07-14 05:50:13 +02:00
|
|
|
if value not in members.keys():
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPISemError(info, "Union '%s' data missing '%s' branch"
|
|
|
|
% (name, value))
|
2016-07-14 05:50:13 +02:00
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2017-01-13 15:41:25 +01:00
|
|
|
def check_alternate(expr, info):
|
2015-05-04 17:05:13 +02:00
|
|
|
name = expr['alternate']
|
2015-05-04 17:05:10 +02:00
|
|
|
members = expr['data']
|
|
|
|
types_seen = {}
|
|
|
|
|
2016-02-18 07:48:16 +01:00
|
|
|
# Check every branch; require at least two branches
|
|
|
|
if len(members) < 2:
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPISemError(info,
|
|
|
|
"Alternate '%s' should have at least two branches "
|
|
|
|
"in 'data'" % name)
|
2015-05-04 17:05:10 +02:00
|
|
|
for (key, value) in members.items():
|
2017-01-13 15:41:25 +01:00
|
|
|
check_name(info, "Member of alternate '%s'" % name, key)
|
2015-05-04 17:05:22 +02:00
|
|
|
|
2015-05-04 17:05:10 +02:00
|
|
|
# Ensure alternates have no type conflicts.
|
2017-01-13 15:41:25 +01:00
|
|
|
check_type(info, "Member '%s' of alternate '%s'" % (key, name),
|
2015-05-04 17:05:21 +02:00
|
|
|
value,
|
|
|
|
allow_metas=['built-in', 'union', 'struct', 'enum'])
|
2015-05-04 17:05:10 +02:00
|
|
|
qtype = find_alternate_member_qtype(value)
|
2016-02-18 07:48:17 +01:00
|
|
|
if not qtype:
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPISemError(info, "Alternate '%s' member '%s' cannot use "
|
|
|
|
"type '%s'" % (name, key, value))
|
qapi: Reject alternates that can't work with keyval_parse()
Alternates are sum types like unions, but use the JSON type on the
wire / QType in QObject instead of an explicit tag. That's why we
require alternate members to have distinct QTypes.
The recently introduced keyval_parse() (commit d454dbe) can only
produce string scalars. The qobject_input_visitor_new_keyval() input
visitor mostly hides the difference, so code using a QObject input
visitor doesn't have to care whether its input was parsed from JSON or
KEY=VALUE,... The difference leaks for alternates, as noted in commit
0ee9ae7: a non-string, non-enum scalar alternate value can't currently
be expressed.
In part, this is just our insufficiently sophisticated implementation.
Consider alternate type 'GuestFileWhence'. It has an integer member
and a 'QGASeek' member. The latter is an enumeration with values
'set', 'cur', 'end'. The meaning of b=set, b=cur, b=end, b=0, b=1 and
so forth is perfectly obvious. However, our current implementation
falls apart at run time for b=0, b=1, and so forth. Fixable, but not
today; add a test case and a TODO comment.
Now consider an alternate type with a string and an integer member.
What's the meaning of a=42? Is it the string "42" or the integer 42?
Whichever meaning you pick makes the other inexpressible. This isn't
just an implementation problem, it's fundamental. Our current
implementation will pick string.
So far, we haven't needed such alternates. To make sure we stop and
think before we add one that cannot sanely work with keyval_parse(),
let's require alternate members to have sufficiently distinct
representation in KEY=VALUE,... syntax:
* A string member clashes with any other scalar member
* An enumeration member clashes with bool members when it has value
'on' or 'off'.
* An enumeration member clashes with numeric members when it has a
value that starts with '-', '+', or a decimal digit. This is a
rather lazy approximation of the actual number syntax accepted by
the visitor.
Note that enumeration values starting with '-' and '+' are rejected
elsewhere already, but better safe than sorry.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <1495471335-23707-5-git-send-email-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
2017-05-22 18:42:15 +02:00
|
|
|
conflicting = set([qtype])
|
|
|
|
if qtype == 'QTYPE_QSTRING':
|
|
|
|
enum_expr = enum_types.get(value)
|
|
|
|
if enum_expr:
|
|
|
|
for v in enum_expr['data']:
|
|
|
|
if v in ['on', 'off']:
|
|
|
|
conflicting.add('QTYPE_QBOOL')
|
|
|
|
if re.match(r'[-+0-9.]', v): # lazy, could be tightened
|
2017-06-07 18:35:58 +02:00
|
|
|
conflicting.add('QTYPE_QNUM')
|
qapi: Reject alternates that can't work with keyval_parse()
Alternates are sum types like unions, but use the JSON type on the
wire / QType in QObject instead of an explicit tag. That's why we
require alternate members to have distinct QTypes.
The recently introduced keyval_parse() (commit d454dbe) can only
produce string scalars. The qobject_input_visitor_new_keyval() input
visitor mostly hides the difference, so code using a QObject input
visitor doesn't have to care whether its input was parsed from JSON or
KEY=VALUE,... The difference leaks for alternates, as noted in commit
0ee9ae7: a non-string, non-enum scalar alternate value can't currently
be expressed.
In part, this is just our insufficiently sophisticated implementation.
Consider alternate type 'GuestFileWhence'. It has an integer member
and a 'QGASeek' member. The latter is an enumeration with values
'set', 'cur', 'end'. The meaning of b=set, b=cur, b=end, b=0, b=1 and
so forth is perfectly obvious. However, our current implementation
falls apart at run time for b=0, b=1, and so forth. Fixable, but not
today; add a test case and a TODO comment.
Now consider an alternate type with a string and an integer member.
What's the meaning of a=42? Is it the string "42" or the integer 42?
Whichever meaning you pick makes the other inexpressible. This isn't
just an implementation problem, it's fundamental. Our current
implementation will pick string.
So far, we haven't needed such alternates. To make sure we stop and
think before we add one that cannot sanely work with keyval_parse(),
let's require alternate members to have sufficiently distinct
representation in KEY=VALUE,... syntax:
* A string member clashes with any other scalar member
* An enumeration member clashes with bool members when it has value
'on' or 'off'.
* An enumeration member clashes with numeric members when it has a
value that starts with '-', '+', or a decimal digit. This is a
rather lazy approximation of the actual number syntax accepted by
the visitor.
Note that enumeration values starting with '-' and '+' are rejected
elsewhere already, but better safe than sorry.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <1495471335-23707-5-git-send-email-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
2017-05-22 18:42:15 +02:00
|
|
|
else:
|
2017-06-07 18:35:58 +02:00
|
|
|
conflicting.add('QTYPE_QNUM')
|
qapi: Reject alternates that can't work with keyval_parse()
Alternates are sum types like unions, but use the JSON type on the
wire / QType in QObject instead of an explicit tag. That's why we
require alternate members to have distinct QTypes.
The recently introduced keyval_parse() (commit d454dbe) can only
produce string scalars. The qobject_input_visitor_new_keyval() input
visitor mostly hides the difference, so code using a QObject input
visitor doesn't have to care whether its input was parsed from JSON or
KEY=VALUE,... The difference leaks for alternates, as noted in commit
0ee9ae7: a non-string, non-enum scalar alternate value can't currently
be expressed.
In part, this is just our insufficiently sophisticated implementation.
Consider alternate type 'GuestFileWhence'. It has an integer member
and a 'QGASeek' member. The latter is an enumeration with values
'set', 'cur', 'end'. The meaning of b=set, b=cur, b=end, b=0, b=1 and
so forth is perfectly obvious. However, our current implementation
falls apart at run time for b=0, b=1, and so forth. Fixable, but not
today; add a test case and a TODO comment.
Now consider an alternate type with a string and an integer member.
What's the meaning of a=42? Is it the string "42" or the integer 42?
Whichever meaning you pick makes the other inexpressible. This isn't
just an implementation problem, it's fundamental. Our current
implementation will pick string.
So far, we haven't needed such alternates. To make sure we stop and
think before we add one that cannot sanely work with keyval_parse(),
let's require alternate members to have sufficiently distinct
representation in KEY=VALUE,... syntax:
* A string member clashes with any other scalar member
* An enumeration member clashes with bool members when it has value
'on' or 'off'.
* An enumeration member clashes with numeric members when it has a
value that starts with '-', '+', or a decimal digit. This is a
rather lazy approximation of the actual number syntax accepted by
the visitor.
Note that enumeration values starting with '-' and '+' are rejected
elsewhere already, but better safe than sorry.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <1495471335-23707-5-git-send-email-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
2017-05-22 18:42:15 +02:00
|
|
|
conflicting.add('QTYPE_QBOOL')
|
|
|
|
for qt in conflicting:
|
qapi: Fix error handling code on alternate conflict
The conflict check added by commit c0644771 ("qapi: Reject
alternates that can't work with keyval_parse()") doesn't work
with the following declaration:
{ 'alternate': 'Alt',
'data': { 'one': 'bool',
'two': 'str' } }
It crashes with:
Traceback (most recent call last):
File "./scripts/qapi-types.py", line 295, in <module>
schema = QAPISchema(input_file)
File "/home/ehabkost/rh/proj/virt/qemu/scripts/qapi.py", line 1468, in __init__
self.exprs = check_exprs(parser.exprs)
File "/home/ehabkost/rh/proj/virt/qemu/scripts/qapi.py", line 958, in check_exprs
check_alternate(expr, info)
File "/home/ehabkost/rh/proj/virt/qemu/scripts/qapi.py", line 830, in check_alternate
% (name, key, types_seen[qtype]))
KeyError: 'QTYPE_QSTRING'
This happens because the previously-seen conflicting member
('one') can't be found at types_seen[qtype], but at
types_seen['QTYPE_BOOL'].
Fix the bug by moving the error check to the same loop that adds
new items to types_seen, raising an exception if types_seen[qt]
is already set.
Add two additional test cases that can detect the bug.
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Message-Id: <20170717180926.14924-1-ehabkost@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2017-07-17 20:09:26 +02:00
|
|
|
if qt in types_seen:
|
|
|
|
raise QAPISemError(info, "Alternate '%s' member '%s' can't "
|
|
|
|
"be distinguished from member '%s'"
|
|
|
|
% (name, key, types_seen[qt]))
|
qapi: Reject alternates that can't work with keyval_parse()
Alternates are sum types like unions, but use the JSON type on the
wire / QType in QObject instead of an explicit tag. That's why we
require alternate members to have distinct QTypes.
The recently introduced keyval_parse() (commit d454dbe) can only
produce string scalars. The qobject_input_visitor_new_keyval() input
visitor mostly hides the difference, so code using a QObject input
visitor doesn't have to care whether its input was parsed from JSON or
KEY=VALUE,... The difference leaks for alternates, as noted in commit
0ee9ae7: a non-string, non-enum scalar alternate value can't currently
be expressed.
In part, this is just our insufficiently sophisticated implementation.
Consider alternate type 'GuestFileWhence'. It has an integer member
and a 'QGASeek' member. The latter is an enumeration with values
'set', 'cur', 'end'. The meaning of b=set, b=cur, b=end, b=0, b=1 and
so forth is perfectly obvious. However, our current implementation
falls apart at run time for b=0, b=1, and so forth. Fixable, but not
today; add a test case and a TODO comment.
Now consider an alternate type with a string and an integer member.
What's the meaning of a=42? Is it the string "42" or the integer 42?
Whichever meaning you pick makes the other inexpressible. This isn't
just an implementation problem, it's fundamental. Our current
implementation will pick string.
So far, we haven't needed such alternates. To make sure we stop and
think before we add one that cannot sanely work with keyval_parse(),
let's require alternate members to have sufficiently distinct
representation in KEY=VALUE,... syntax:
* A string member clashes with any other scalar member
* An enumeration member clashes with bool members when it has value
'on' or 'off'.
* An enumeration member clashes with numeric members when it has a
value that starts with '-', '+', or a decimal digit. This is a
rather lazy approximation of the actual number syntax accepted by
the visitor.
Note that enumeration values starting with '-' and '+' are rejected
elsewhere already, but better safe than sorry.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <1495471335-23707-5-git-send-email-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
2017-05-22 18:42:15 +02:00
|
|
|
types_seen[qt] = key
|
2014-03-05 03:44:34 +01:00
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2017-01-13 15:41:25 +01:00
|
|
|
def check_enum(expr, info):
|
2015-05-04 17:05:04 +02:00
|
|
|
name = expr['enum']
|
|
|
|
members = expr.get('data')
|
2015-08-26 15:21:20 +02:00
|
|
|
prefix = expr.get('prefix')
|
2015-05-04 17:05:04 +02:00
|
|
|
|
|
|
|
if not isinstance(members, list):
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPISemError(info,
|
|
|
|
"Enum '%s' requires an array for 'data'" % name)
|
2015-08-26 15:21:20 +02:00
|
|
|
if prefix is not None and not isinstance(prefix, str):
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPISemError(info,
|
|
|
|
"Enum '%s' requires a string for 'prefix'" % name)
|
2015-05-04 17:05:04 +02:00
|
|
|
for member in members:
|
2017-01-13 15:41:25 +01:00
|
|
|
check_name(info, "Member of enum '%s'" % name, member,
|
2015-05-04 17:05:22 +02:00
|
|
|
enum_member=True)
|
2015-05-04 17:05:04 +02:00
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2017-01-13 15:41:25 +01:00
|
|
|
def check_struct(expr, info):
|
2015-05-04 17:05:25 +02:00
|
|
|
name = expr['struct']
|
2015-05-04 17:05:21 +02:00
|
|
|
members = expr['data']
|
|
|
|
|
2017-01-13 15:41:25 +01:00
|
|
|
check_type(info, "'data' for struct '%s'" % name, members,
|
2015-05-04 17:05:22 +02:00
|
|
|
allow_dict=True, allow_optional=True)
|
2017-01-13 15:41:25 +01:00
|
|
|
check_type(info, "'base' for struct '%s'" % name, expr.get('base'),
|
2015-05-04 17:05:21 +02:00
|
|
|
allow_metas=['struct'])
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2015-05-04 17:05:15 +02:00
|
|
|
def check_keys(expr_elem, meta, required, optional=[]):
|
|
|
|
expr = expr_elem['expr']
|
|
|
|
info = expr_elem['info']
|
|
|
|
name = expr[meta]
|
|
|
|
if not isinstance(name, str):
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPISemError(info, "'%s' key must have a string value" % meta)
|
2015-09-30 00:21:02 +02:00
|
|
|
required = required + [meta]
|
2015-05-04 17:05:15 +02:00
|
|
|
for (key, value) in expr.items():
|
2015-09-30 00:21:02 +02:00
|
|
|
if key not in required and key not in optional:
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPISemError(info, "Unknown key '%s' in %s '%s'"
|
|
|
|
% (key, meta, name))
|
2015-09-30 00:21:02 +02:00
|
|
|
if (key == 'gen' or key == 'success-response') and value is not False:
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPISemError(info,
|
|
|
|
"'%s' of %s '%s' should only use false value"
|
|
|
|
% (key, meta, name))
|
2016-07-14 05:50:20 +02:00
|
|
|
if key == 'boxed' and value is not True:
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPISemError(info,
|
|
|
|
"'%s' of %s '%s' should only use true value"
|
|
|
|
% (key, meta, name))
|
2015-05-04 17:05:15 +02:00
|
|
|
for key in required:
|
2015-09-30 00:21:02 +02:00
|
|
|
if key not in expr:
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPISemError(info, "Key '%s' is missing from %s '%s'"
|
|
|
|
% (key, meta, name))
|
2015-05-04 17:05:15 +02:00
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2015-06-10 08:55:21 +02:00
|
|
|
def check_exprs(exprs):
|
2015-05-04 17:05:17 +02:00
|
|
|
global all_names
|
|
|
|
|
2017-03-15 13:57:21 +01:00
|
|
|
# Populate name table with names of built-in types
|
2015-06-10 08:55:21 +02:00
|
|
|
for builtin in builtin_types.keys():
|
|
|
|
all_names[builtin] = 'built-in'
|
2017-03-15 13:57:21 +01:00
|
|
|
|
|
|
|
# Learn the types and check for valid expression keys
|
2015-06-10 08:55:21 +02:00
|
|
|
for expr_elem in exprs:
|
|
|
|
expr = expr_elem['expr']
|
|
|
|
info = expr_elem['info']
|
2017-03-15 13:57:21 +01:00
|
|
|
doc = expr_elem.get('doc')
|
qapi: add qapi2texi script
As the name suggests, the qapi2texi script converts JSON QAPI
description into a texi file suitable for different target
formats (info/man/txt/pdf/html...).
It parses the following kind of blocks:
Free-form:
##
# = Section
# == Subsection
#
# Some text foo with *emphasis*
# 1. with a list
# 2. like that
#
# And some code:
# | $ echo foo
# | -> do this
# | <- get that
#
##
Symbol description:
##
# @symbol:
#
# Symbol body ditto ergo sum. Foo bar
# baz ding.
#
# @param1: the frob to frobnicate
# @param2: #optional how hard to frobnicate
#
# Returns: the frobnicated frob.
# If frob isn't frobnicatable, GenericError.
#
# Since: version
# Notes: notes, comments can have
# - itemized list
# - like this
#
# Example:
#
# -> { "execute": "quit" }
# <- { "return": {} }
#
##
That's roughly following the following EBNF grammar:
api_comment = "##\n" comment "##\n"
comment = freeform_comment | symbol_comment
freeform_comment = { "# " text "\n" | "#\n" }
symbol_comment = "# @" name ":\n" { member | tag_section | freeform_comment }
member = "# @" name ':' [ text ] "\n" freeform_comment
tag_section = "# " ( "Returns:", "Since:", "Note:", "Notes:", "Example:", "Examples:" ) [ text ] "\n" freeform_comment
text = free text with markup
Note that the grammar is ambiguous: a line "# @foo:\n" can be parsed
both as freeform_comment and as symbol_comment. The actual parser
recognizes symbol_comment.
See docs/qapi-code-gen.txt for more details.
Deficiencies and limitations:
- the generated QMP documentation includes internal types
- union type support is lacking
- type information is lacking in generated documentation
- doc comment error message positions are imprecise, they point
to the beginning of the comment.
- a few minor issues, all marked TODO/FIXME in the code
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20170113144135.5150-16-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[test-qapi.py tweaked to avoid trailing empty lines in .out]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2017-01-13 15:41:29 +01:00
|
|
|
|
2017-03-15 13:57:21 +01:00
|
|
|
if not doc and doc_required:
|
qapi: add qapi2texi script
As the name suggests, the qapi2texi script converts JSON QAPI
description into a texi file suitable for different target
formats (info/man/txt/pdf/html...).
It parses the following kind of blocks:
Free-form:
##
# = Section
# == Subsection
#
# Some text foo with *emphasis*
# 1. with a list
# 2. like that
#
# And some code:
# | $ echo foo
# | -> do this
# | <- get that
#
##
Symbol description:
##
# @symbol:
#
# Symbol body ditto ergo sum. Foo bar
# baz ding.
#
# @param1: the frob to frobnicate
# @param2: #optional how hard to frobnicate
#
# Returns: the frobnicated frob.
# If frob isn't frobnicatable, GenericError.
#
# Since: version
# Notes: notes, comments can have
# - itemized list
# - like this
#
# Example:
#
# -> { "execute": "quit" }
# <- { "return": {} }
#
##
That's roughly following the following EBNF grammar:
api_comment = "##\n" comment "##\n"
comment = freeform_comment | symbol_comment
freeform_comment = { "# " text "\n" | "#\n" }
symbol_comment = "# @" name ":\n" { member | tag_section | freeform_comment }
member = "# @" name ':' [ text ] "\n" freeform_comment
tag_section = "# " ( "Returns:", "Since:", "Note:", "Notes:", "Example:", "Examples:" ) [ text ] "\n" freeform_comment
text = free text with markup
Note that the grammar is ambiguous: a line "# @foo:\n" can be parsed
both as freeform_comment and as symbol_comment. The actual parser
recognizes symbol_comment.
See docs/qapi-code-gen.txt for more details.
Deficiencies and limitations:
- the generated QMP documentation includes internal types
- union type support is lacking
- type information is lacking in generated documentation
- doc comment error message positions are imprecise, they point
to the beginning of the comment.
- a few minor issues, all marked TODO/FIXME in the code
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20170113144135.5150-16-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[test-qapi.py tweaked to avoid trailing empty lines in .out]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2017-01-13 15:41:29 +01:00
|
|
|
raise QAPISemError(info,
|
|
|
|
"Expression missing documentation comment")
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
if 'enum' in expr:
|
2017-03-15 13:57:30 +01:00
|
|
|
meta = 'enum'
|
2015-08-26 15:21:20 +02:00
|
|
|
check_keys(expr_elem, 'enum', ['data'], ['prefix'])
|
2017-03-15 13:57:31 +01:00
|
|
|
enum_types[expr[meta]] = expr
|
2015-09-30 00:21:02 +02:00
|
|
|
elif 'union' in expr:
|
2017-03-15 13:57:30 +01:00
|
|
|
meta = 'union'
|
2015-06-10 08:55:21 +02:00
|
|
|
check_keys(expr_elem, 'union', ['data'],
|
|
|
|
['base', 'discriminator'])
|
2017-03-15 13:57:33 +01:00
|
|
|
union_types[expr[meta]] = expr
|
2015-09-30 00:21:02 +02:00
|
|
|
elif 'alternate' in expr:
|
2017-03-15 13:57:30 +01:00
|
|
|
meta = 'alternate'
|
2015-06-10 08:55:21 +02:00
|
|
|
check_keys(expr_elem, 'alternate', ['data'])
|
2015-09-30 00:21:02 +02:00
|
|
|
elif 'struct' in expr:
|
2017-03-15 13:57:30 +01:00
|
|
|
meta = 'struct'
|
2015-06-10 08:55:21 +02:00
|
|
|
check_keys(expr_elem, 'struct', ['data'], ['base'])
|
2017-03-15 13:57:32 +01:00
|
|
|
struct_types[expr[meta]] = expr
|
2015-09-30 00:21:02 +02:00
|
|
|
elif 'command' in expr:
|
2017-03-15 13:57:30 +01:00
|
|
|
meta = 'command'
|
2015-06-10 08:55:21 +02:00
|
|
|
check_keys(expr_elem, 'command', [],
|
2016-07-14 05:50:20 +02:00
|
|
|
['data', 'returns', 'gen', 'success-response', 'boxed'])
|
2015-09-30 00:21:02 +02:00
|
|
|
elif 'event' in expr:
|
2017-03-15 13:57:30 +01:00
|
|
|
meta = 'event'
|
2016-07-14 05:50:20 +02:00
|
|
|
check_keys(expr_elem, 'event', [], ['data', 'boxed'])
|
2015-06-10 08:55:21 +02:00
|
|
|
else:
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPISemError(expr_elem['info'],
|
|
|
|
"Expression is missing metatype")
|
2017-03-15 13:57:30 +01:00
|
|
|
name = expr[meta]
|
|
|
|
add_name(name, info, meta)
|
2017-03-15 13:57:21 +01:00
|
|
|
if doc and doc.symbol != name:
|
|
|
|
raise QAPISemError(info, "Definition of '%s' follows documentation"
|
|
|
|
" for '%s'" % (name, doc.symbol))
|
2013-07-27 17:41:56 +02:00
|
|
|
|
2015-06-10 08:55:21 +02:00
|
|
|
# Try again for hidden UnionKind enum
|
|
|
|
for expr_elem in exprs:
|
|
|
|
expr = expr_elem['expr']
|
2017-03-15 13:57:29 +01:00
|
|
|
if 'union' in expr and not discriminator_find_enum_define(expr):
|
|
|
|
name = '%sKind' % expr['union']
|
2015-09-30 00:21:02 +02:00
|
|
|
elif 'alternate' in expr:
|
2017-03-15 13:57:29 +01:00
|
|
|
name = '%sKind' % expr['alternate']
|
|
|
|
else:
|
|
|
|
continue
|
2017-03-15 13:57:31 +01:00
|
|
|
enum_types[name] = {'enum': name}
|
2017-03-15 13:57:30 +01:00
|
|
|
add_name(name, info, 'enum', implicit=True)
|
2015-06-10 08:55:21 +02:00
|
|
|
|
|
|
|
# Validate that exprs make sense
|
|
|
|
for expr_elem in exprs:
|
|
|
|
expr = expr_elem['expr']
|
|
|
|
info = expr_elem['info']
|
2017-03-15 13:57:27 +01:00
|
|
|
doc = expr_elem.get('doc')
|
2015-05-04 17:05:09 +02:00
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
if 'enum' in expr:
|
2015-06-10 08:55:21 +02:00
|
|
|
check_enum(expr, info)
|
2015-09-30 00:21:02 +02:00
|
|
|
elif 'union' in expr:
|
2015-06-10 08:55:21 +02:00
|
|
|
check_union(expr, info)
|
2015-09-30 00:21:02 +02:00
|
|
|
elif 'alternate' in expr:
|
2015-06-10 08:55:21 +02:00
|
|
|
check_alternate(expr, info)
|
2015-09-30 00:21:02 +02:00
|
|
|
elif 'struct' in expr:
|
2015-06-10 08:55:21 +02:00
|
|
|
check_struct(expr, info)
|
2015-09-30 00:21:02 +02:00
|
|
|
elif 'command' in expr:
|
2015-06-10 08:55:21 +02:00
|
|
|
check_command(expr, info)
|
2015-09-30 00:21:02 +02:00
|
|
|
elif 'event' in expr:
|
2015-06-10 08:55:21 +02:00
|
|
|
check_event(expr, info)
|
|
|
|
else:
|
|
|
|
assert False, 'unexpected meta type'
|
|
|
|
|
2017-03-15 13:57:27 +01:00
|
|
|
if doc:
|
|
|
|
doc.check_expr(expr)
|
2015-09-16 13:06:05 +02:00
|
|
|
|
2017-03-15 13:57:27 +01:00
|
|
|
return exprs
|
qapi: add qapi2texi script
As the name suggests, the qapi2texi script converts JSON QAPI
description into a texi file suitable for different target
formats (info/man/txt/pdf/html...).
It parses the following kind of blocks:
Free-form:
##
# = Section
# == Subsection
#
# Some text foo with *emphasis*
# 1. with a list
# 2. like that
#
# And some code:
# | $ echo foo
# | -> do this
# | <- get that
#
##
Symbol description:
##
# @symbol:
#
# Symbol body ditto ergo sum. Foo bar
# baz ding.
#
# @param1: the frob to frobnicate
# @param2: #optional how hard to frobnicate
#
# Returns: the frobnicated frob.
# If frob isn't frobnicatable, GenericError.
#
# Since: version
# Notes: notes, comments can have
# - itemized list
# - like this
#
# Example:
#
# -> { "execute": "quit" }
# <- { "return": {} }
#
##
That's roughly following the following EBNF grammar:
api_comment = "##\n" comment "##\n"
comment = freeform_comment | symbol_comment
freeform_comment = { "# " text "\n" | "#\n" }
symbol_comment = "# @" name ":\n" { member | tag_section | freeform_comment }
member = "# @" name ':' [ text ] "\n" freeform_comment
tag_section = "# " ( "Returns:", "Since:", "Note:", "Notes:", "Example:", "Examples:" ) [ text ] "\n" freeform_comment
text = free text with markup
Note that the grammar is ambiguous: a line "# @foo:\n" can be parsed
both as freeform_comment and as symbol_comment. The actual parser
recognizes symbol_comment.
See docs/qapi-code-gen.txt for more details.
Deficiencies and limitations:
- the generated QMP documentation includes internal types
- union type support is lacking
- type information is lacking in generated documentation
- doc comment error message positions are imprecise, they point
to the beginning of the comment.
- a few minor issues, all marked TODO/FIXME in the code
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20170113144135.5150-16-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[test-qapi.py tweaked to avoid trailing empty lines in .out]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2017-01-13 15:41:29 +01:00
|
|
|
|
|
|
|
|
2015-09-16 13:06:05 +02:00
|
|
|
#
|
|
|
|
# Schema compiler frontend
|
|
|
|
#
|
|
|
|
|
|
|
|
class QAPISchemaEntity(object):
|
2017-03-15 13:57:03 +01:00
|
|
|
def __init__(self, name, info, doc):
|
2015-09-16 13:06:05 +02:00
|
|
|
assert isinstance(name, str)
|
|
|
|
self.name = name
|
2015-10-13 06:22:32 +02:00
|
|
|
# For explicitly defined entities, info points to the (explicit)
|
|
|
|
# definition. For builtins (and their arrays), info is None.
|
|
|
|
# For implicitly defined entities, info points to a place that
|
|
|
|
# triggered the implicit definition (there may be more than one
|
|
|
|
# such place).
|
2015-09-16 13:06:05 +02:00
|
|
|
self.info = info
|
2017-03-15 13:57:03 +01:00
|
|
|
self.doc = doc
|
2015-09-16 13:06:05 +02:00
|
|
|
|
2015-09-16 13:06:06 +02:00
|
|
|
def c_name(self):
|
|
|
|
return c_name(self.name)
|
|
|
|
|
2015-09-16 13:06:05 +02:00
|
|
|
def check(self, schema):
|
|
|
|
pass
|
|
|
|
|
2015-10-13 06:22:27 +02:00
|
|
|
def is_implicit(self):
|
|
|
|
return not self.info
|
|
|
|
|
2015-09-16 13:06:07 +02:00
|
|
|
def visit(self, visitor):
|
|
|
|
pass
|
|
|
|
|
|
|
|
|
|
|
|
class QAPISchemaVisitor(object):
|
|
|
|
def visit_begin(self, schema):
|
|
|
|
pass
|
|
|
|
|
|
|
|
def visit_end(self):
|
|
|
|
pass
|
|
|
|
|
2015-10-13 06:22:21 +02:00
|
|
|
def visit_needed(self, entity):
|
|
|
|
# Default to visiting everything
|
|
|
|
return True
|
|
|
|
|
2015-09-16 13:06:07 +02:00
|
|
|
def visit_builtin_type(self, name, info, json_type):
|
|
|
|
pass
|
|
|
|
|
|
|
|
def visit_enum_type(self, name, info, values, prefix):
|
|
|
|
pass
|
|
|
|
|
|
|
|
def visit_array_type(self, name, info, element_type):
|
|
|
|
pass
|
|
|
|
|
|
|
|
def visit_object_type(self, name, info, base, members, variants):
|
|
|
|
pass
|
|
|
|
|
qapi: New QMP command query-qmp-schema for QMP introspection
qapi/introspect.json defines the introspection schema. It's designed
for QMP introspection, but should do for similar uses, such as QGA.
The introspection schema does not reflect all the rules and
restrictions that apply to QAPI schemata. A valid QAPI schema has an
introspection value conforming to the introspection schema, but the
converse is not true.
Introspection lowers away a number of schema details, and makes
implicit things explicit:
* The built-in types are declared with their JSON type.
All integer types are mapped to 'int', because how many bits we use
internally is an implementation detail. It could be pressed into
external interface service as very approximate range information,
but that's a bad idea. If we need range information, we better do
it properly.
* Implicit type definitions are made explicit, and given
auto-generated names:
- Array types, named by appending "List" to the name of their
element type, like in generated C.
- The enumeration types implicitly defined by simple union types,
named by appending "Kind" to the name of their simple union type,
like in generated C.
- Types that don't occur in generated C. Their names start with ':'
so they don't clash with the user's names.
* All type references are by name.
* The struct and union types are generalized into an object type.
* Base types are flattened.
* Commands take a single argument and return a single result.
Dictionary argument or list result is an implicit type definition.
The empty object type is used when a command takes no arguments or
produces no results.
The argument is always of object type, but the introspection schema
doesn't reflect that.
The 'gen': false directive is omitted as implementation detail.
The 'success-response' directive is omitted as well for now, even
though it's not an implementation detail, because it's not used by
QMP.
* Events carry a single data value.
Implicit type definition and empty object type use, just like for
commands.
The value is of object type, but the introspection schema doesn't
reflect that.
* Types not used by commands or events are omitted.
Indirect use counts as use.
* Optional members have a default, which can only be null right now
Instead of a mandatory "optional" flag, we have an optional default.
No default means mandatory, default null means optional without
default value. Non-null is available for optional with default
(possible future extension).
* Clients should *not* look up types by name, because type names are
not ABI. Look up the command or event you're interested in, then
follow the references.
TODO Should we hide the type names to eliminate the temptation?
New generator scripts/qapi-introspect.py computes an introspection
value for its input, and generates a C variable holding it.
It can generate awfully long lines. Marked TODO.
A new test-qmp-input-visitor test case feeds its result for both
tests/qapi-schema/qapi-schema-test.json and qapi-schema.json to a
QmpInputVisitor to verify it actually conforms to the schema.
New QMP command query-qmp-schema takes its return value from that
variable. Its reply is some 85KiBytes for me right now.
If this turns out to be too much, we have a couple of options:
* We can use shorter names in the JSON. Not the QMP style.
* Optionally return the sub-schema for commands and events given as
arguments.
Right now qmp_query_schema() sends the string literal computed by
qmp-introspect.py. To compute sub-schema at run time, we'd have to
duplicate parts of qapi-introspect.py in C. Unattractive.
* Let clients cache the output of query-qmp-schema.
It changes only on QEMU upgrades, i.e. rarely. Provide a command
query-qmp-schema-hash. Clients can have a cache indexed by hash,
and re-query the schema only when they don't have it cached. Even
simpler: put the hash in the QMP greeting.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2015-09-16 13:06:28 +02:00
|
|
|
def visit_object_type_flat(self, name, info, members, variants):
|
|
|
|
pass
|
|
|
|
|
2015-09-16 13:06:07 +02:00
|
|
|
def visit_alternate_type(self, name, info, variants):
|
|
|
|
pass
|
|
|
|
|
|
|
|
def visit_command(self, name, info, arg_type, ret_type,
|
2016-07-14 05:50:19 +02:00
|
|
|
gen, success_response, boxed):
|
2015-09-16 13:06:07 +02:00
|
|
|
pass
|
|
|
|
|
2016-07-14 05:50:19 +02:00
|
|
|
def visit_event(self, name, info, arg_type, boxed):
|
2015-09-16 13:06:07 +02:00
|
|
|
pass
|
|
|
|
|
2015-09-16 13:06:05 +02:00
|
|
|
|
|
|
|
class QAPISchemaType(QAPISchemaEntity):
|
2016-03-17 23:48:28 +01:00
|
|
|
# Return the C type for common use.
|
|
|
|
# For the types we commonly box, this is a pointer type.
|
|
|
|
def c_type(self):
|
|
|
|
pass
|
|
|
|
|
|
|
|
# Return the C type to be used in a parameter list.
|
|
|
|
def c_param_type(self):
|
|
|
|
return self.c_type()
|
|
|
|
|
|
|
|
# Return the C type to be used where we suppress boxing.
|
|
|
|
def c_unboxed_type(self):
|
|
|
|
return self.c_type()
|
2015-09-16 13:06:06 +02:00
|
|
|
|
|
|
|
def json_type(self):
|
|
|
|
pass
|
|
|
|
|
|
|
|
def alternate_qtype(self):
|
|
|
|
json2qtype = {
|
2017-06-26 19:25:14 +02:00
|
|
|
'null': 'QTYPE_QNULL',
|
2015-09-16 13:06:06 +02:00
|
|
|
'string': 'QTYPE_QSTRING',
|
2017-06-07 18:35:58 +02:00
|
|
|
'number': 'QTYPE_QNUM',
|
|
|
|
'int': 'QTYPE_QNUM',
|
2015-09-16 13:06:06 +02:00
|
|
|
'boolean': 'QTYPE_QBOOL',
|
|
|
|
'object': 'QTYPE_QDICT'
|
|
|
|
}
|
|
|
|
return json2qtype.get(self.json_type())
|
2015-09-16 13:06:05 +02:00
|
|
|
|
2017-03-15 13:57:14 +01:00
|
|
|
def doc_type(self):
|
|
|
|
if self.is_implicit():
|
|
|
|
return None
|
|
|
|
return self.name
|
|
|
|
|
2015-09-16 13:06:05 +02:00
|
|
|
|
|
|
|
class QAPISchemaBuiltinType(QAPISchemaType):
|
2016-03-17 23:48:36 +01:00
|
|
|
def __init__(self, name, json_type, c_type):
|
2017-03-15 13:57:03 +01:00
|
|
|
QAPISchemaType.__init__(self, name, None, None)
|
2015-09-16 13:06:06 +02:00
|
|
|
assert not c_type or isinstance(c_type, str)
|
|
|
|
assert json_type in ('string', 'number', 'int', 'boolean', 'null',
|
|
|
|
'value')
|
|
|
|
self._json_type_name = json_type
|
|
|
|
self._c_type_name = c_type
|
|
|
|
|
|
|
|
def c_name(self):
|
|
|
|
return self.name
|
|
|
|
|
2016-03-17 23:48:28 +01:00
|
|
|
def c_type(self):
|
|
|
|
return self._c_type_name
|
|
|
|
|
|
|
|
def c_param_type(self):
|
|
|
|
if self.name == 'str':
|
2015-09-16 13:06:06 +02:00
|
|
|
return 'const ' + self._c_type_name
|
|
|
|
return self._c_type_name
|
|
|
|
|
|
|
|
def json_type(self):
|
|
|
|
return self._json_type_name
|
2015-09-16 13:06:05 +02:00
|
|
|
|
2017-03-15 13:57:14 +01:00
|
|
|
def doc_type(self):
|
|
|
|
return self.json_type()
|
|
|
|
|
2015-09-16 13:06:07 +02:00
|
|
|
def visit(self, visitor):
|
|
|
|
visitor.visit_builtin_type(self.name, self.info, self.json_type())
|
|
|
|
|
2015-09-16 13:06:05 +02:00
|
|
|
|
|
|
|
class QAPISchemaEnumType(QAPISchemaType):
|
2017-03-15 13:57:03 +01:00
|
|
|
def __init__(self, name, info, doc, values, prefix):
|
|
|
|
QAPISchemaType.__init__(self, name, info, doc)
|
2015-09-16 13:06:05 +02:00
|
|
|
for v in values:
|
2015-12-02 06:20:55 +01:00
|
|
|
assert isinstance(v, QAPISchemaMember)
|
|
|
|
v.set_owner(name)
|
2015-09-16 13:06:05 +02:00
|
|
|
assert prefix is None or isinstance(prefix, str)
|
|
|
|
self.values = values
|
|
|
|
self.prefix = prefix
|
|
|
|
|
|
|
|
def check(self, schema):
|
2015-12-02 06:20:55 +01:00
|
|
|
seen = {}
|
|
|
|
for v in self.values:
|
|
|
|
v.check_clash(self.info, seen)
|
2017-03-15 13:57:03 +01:00
|
|
|
if self.doc:
|
|
|
|
self.doc.connect_member(v)
|
2015-09-16 13:06:05 +02:00
|
|
|
|
2015-10-13 06:22:32 +02:00
|
|
|
def is_implicit(self):
|
2017-03-15 13:57:02 +01:00
|
|
|
# See QAPISchema._make_implicit_enum_type() and ._def_predefineds()
|
|
|
|
return self.name.endswith('Kind') or self.name == 'QType'
|
2015-10-13 06:22:32 +02:00
|
|
|
|
2016-03-17 23:48:28 +01:00
|
|
|
def c_type(self):
|
2015-09-16 13:06:06 +02:00
|
|
|
return c_name(self.name)
|
|
|
|
|
2015-12-02 06:20:55 +01:00
|
|
|
def member_names(self):
|
|
|
|
return [v.name for v in self.values]
|
|
|
|
|
2015-09-16 13:06:06 +02:00
|
|
|
def json_type(self):
|
|
|
|
return 'string'
|
|
|
|
|
2015-09-16 13:06:07 +02:00
|
|
|
def visit(self, visitor):
|
|
|
|
visitor.visit_enum_type(self.name, self.info,
|
2015-12-02 06:20:55 +01:00
|
|
|
self.member_names(), self.prefix)
|
2015-09-16 13:06:07 +02:00
|
|
|
|
2015-09-16 13:06:05 +02:00
|
|
|
|
|
|
|
class QAPISchemaArrayType(QAPISchemaType):
|
|
|
|
def __init__(self, name, info, element_type):
|
2017-03-15 13:57:03 +01:00
|
|
|
QAPISchemaType.__init__(self, name, info, None)
|
2015-09-16 13:06:05 +02:00
|
|
|
assert isinstance(element_type, str)
|
|
|
|
self._element_type_name = element_type
|
|
|
|
self.element_type = None
|
|
|
|
|
|
|
|
def check(self, schema):
|
|
|
|
self.element_type = schema.lookup_type(self._element_type_name)
|
|
|
|
assert self.element_type
|
|
|
|
|
2015-10-13 06:22:32 +02:00
|
|
|
def is_implicit(self):
|
|
|
|
return True
|
|
|
|
|
2016-03-17 23:48:28 +01:00
|
|
|
def c_type(self):
|
|
|
|
return c_name(self.name) + pointer_suffix
|
|
|
|
|
2015-09-16 13:06:06 +02:00
|
|
|
def json_type(self):
|
|
|
|
return 'array'
|
|
|
|
|
2017-03-15 13:57:14 +01:00
|
|
|
def doc_type(self):
|
|
|
|
elt_doc_type = self.element_type.doc_type()
|
|
|
|
if not elt_doc_type:
|
|
|
|
return None
|
|
|
|
return 'array of ' + elt_doc_type
|
|
|
|
|
2015-09-16 13:06:07 +02:00
|
|
|
def visit(self, visitor):
|
|
|
|
visitor.visit_array_type(self.name, self.info, self.element_type)
|
|
|
|
|
2015-09-16 13:06:05 +02:00
|
|
|
|
|
|
|
class QAPISchemaObjectType(QAPISchemaType):
|
2017-03-15 13:57:03 +01:00
|
|
|
def __init__(self, name, info, doc, base, local_members, variants):
|
qapi: Track simple union tag in object.local_members
We were previously creating all unions with an empty list for
local_members. However, it will make it easier to unify struct
and union generation if we include the generated tag member in
local_members. That way, we can have a common code pattern:
visit the base (if any), visit the local members (if any), visit
the variants (if any). The local_members of a flat union
remains empty (because the discriminator is already visited as
part of the base). Then, by visiting tag_member.check() during
AlternateType.check(), we no longer need to call it during
Variants.check().
The various front end entities now exist as follows:
struct: optional base, optional local_members, no variants
simple union: no base, one-element local_members, variants with tag_member
from local_members
flat union: base, no local_members, variants with tag_member from base
alternate: no base, no local_members, variants
With the new local members, we require a bit of finesse to
avoid assertions in the clients.
No change to generated code.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-2-git-send-email-eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-18 09:52:36 +01:00
|
|
|
# struct has local_members, optional base, and no variants
|
|
|
|
# flat union has base, variants, and no local_members
|
|
|
|
# simple union has local_members, variants, and no base
|
2017-03-15 13:57:03 +01:00
|
|
|
QAPISchemaType.__init__(self, name, info, doc)
|
2015-09-16 13:06:05 +02:00
|
|
|
assert base is None or isinstance(base, str)
|
|
|
|
for m in local_members:
|
|
|
|
assert isinstance(m, QAPISchemaObjectTypeMember)
|
qapi: Track owner of each object member
Future commits will migrate semantic checking away from parsing
and over to the various QAPISchema*.check() methods. But to
report an error message about an incorrect semantic use of a
member of an object type, it helps to know which type, command,
or event owns the member. In particular, when a member is
inherited from a base type, it is desirable to associate the
member name with the base type (and not the type calling
member.check()).
Rather than packing additional information into the seen array
passed to each member.check() (as in seen[m.name] = {'member':m,
'owner':type}), it is easier to have each member track the name
of the owner type in the first place (keeping things simpler
with the existing seen[m.name] = m). The new member.owner field
is set via a new set_owner() method, called when registering
the members and variants arrays with an object or variant type.
Track only a name, and not the actual type object, to avoid
creating a circular python reference chain.
Note that Variants.set_owner() method does not set the owner
for the tag_member field; this field is set earlier either as
part of an object's non-variant members, or explicitly by
alternates.
The source information is intended for human consumption in
error messages, and a new describe() method is added to access
the resulting information. For example, given the qapi:
{ 'command': 'foo', 'data': { 'string': 'str' } }
an implementation of visit_command() that calls
arg_type.members[0].describe()
will see "'string' (parameter of foo)".
To make the human-readable name of implicit types work without
duplicating efforts, the describe() method has to reverse the
name of implicit types, via the helper _pretty_owner().
No change to generated code.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-16-git-send-email-eblake@redhat.com>
[Incorrect & unused -wrapper case in _pretty_owner() dropped]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-18 09:52:50 +01:00
|
|
|
m.set_owner(name)
|
|
|
|
if variants is not None:
|
|
|
|
assert isinstance(variants, QAPISchemaObjectTypeVariants)
|
|
|
|
variants.set_owner(name)
|
2015-09-16 13:06:05 +02:00
|
|
|
self._base_name = base
|
|
|
|
self.base = None
|
|
|
|
self.local_members = local_members
|
|
|
|
self.variants = variants
|
|
|
|
self.members = None
|
|
|
|
|
|
|
|
def check(self, schema):
|
qapi: Detect base class loops
It should be fairly obvious that qapi base classes need to
form an acyclic graph, since QMP cannot specify the same
key more than once, while base classes are included as flat
members alongside other members added by the child. But the
old check_member_clash() parser function was not prepared to
check for this, and entered an infinite recursion (at least
until Python gives up, complaining about nesting too deep).
Now that check_member_clash() has been recently removed,
attempts at self-inheritance trigger an assertion failure
introduced by commit ac88219a. The obvious fix is to turn
the assertion into a conditional.
This patch includes both the tests (base-cycle-direct and
base-cycle-indirect) and the fix, since the .err file output
for the unfixed case is not useful (particularly when it was
warning about unbounded recursion, as that limit may be
platform-specific).
We don't need to worry about cycles in flat unions (neither
the base type nor the type of a variant can be a union) nor
in alternates (alternate branches cannot themselves be an
alternate). But if we later allow a union type as a variant,
we will still be okay, as QAPISchemaObjectTypeVariants.check()
triggers the same QAPISchemaObjectType.check() that will
detect any loops.
Likewise, we need not worry about the case of diamond
inheritance where the same class is used for a flat union base
class and one of its variants; either both uses will introduce
a collision in trying to insert the same member name twice, or
the shared type is empty and changes nothing.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1449033659-25497-16-git-send-email-eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-12-02 06:20:59 +01:00
|
|
|
if self.members is False: # check for cycles
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPISemError(self.info,
|
|
|
|
"Object %s contains itself" % self.name)
|
2015-09-16 13:06:05 +02:00
|
|
|
if self.members:
|
|
|
|
return
|
|
|
|
self.members = False # mark as being checked
|
2015-11-18 09:52:43 +01:00
|
|
|
seen = OrderedDict()
|
2015-09-16 13:06:05 +02:00
|
|
|
if self._base_name:
|
|
|
|
self.base = schema.lookup_type(self._base_name)
|
|
|
|
assert isinstance(self.base, QAPISchemaObjectType)
|
|
|
|
self.base.check(schema)
|
2017-03-15 13:57:34 +01:00
|
|
|
self.base.check_clash(self.info, seen)
|
2015-09-16 13:06:05 +02:00
|
|
|
for m in self.local_members:
|
2015-11-18 09:52:40 +01:00
|
|
|
m.check(schema)
|
2015-11-18 09:52:51 +01:00
|
|
|
m.check_clash(self.info, seen)
|
2017-03-15 13:57:03 +01:00
|
|
|
if self.doc:
|
|
|
|
self.doc.connect_member(m)
|
2015-11-18 09:52:45 +01:00
|
|
|
self.members = seen.values()
|
2015-09-16 13:06:05 +02:00
|
|
|
if self.variants:
|
2015-11-18 09:52:41 +01:00
|
|
|
self.variants.check(schema, seen)
|
2015-11-18 09:52:45 +01:00
|
|
|
assert self.variants.tag_member in self.members
|
2017-03-15 13:57:34 +01:00
|
|
|
self.variants.check_clash(self.info, seen)
|
2017-03-15 13:57:26 +01:00
|
|
|
if self.doc:
|
|
|
|
self.doc.check()
|
2015-09-16 13:06:05 +02:00
|
|
|
|
2016-03-03 17:16:43 +01:00
|
|
|
# Check that the members of this type do not cause duplicate JSON members,
|
2015-11-18 09:52:51 +01:00
|
|
|
# and update seen to track the members seen so far. Report any errors
|
|
|
|
# on behalf of info, which is not necessarily self.info
|
2017-03-15 13:57:34 +01:00
|
|
|
def check_clash(self, info, seen):
|
2015-11-18 09:52:47 +01:00
|
|
|
assert not self.variants # not implemented
|
|
|
|
for m in self.members:
|
2015-11-18 09:52:51 +01:00
|
|
|
m.check_clash(info, seen)
|
2015-11-18 09:52:47 +01:00
|
|
|
|
2015-10-13 06:22:32 +02:00
|
|
|
def is_implicit(self):
|
2016-03-17 23:48:29 +01:00
|
|
|
# See QAPISchema._make_implicit_object_type(), as well as
|
|
|
|
# _def_predefineds()
|
|
|
|
return self.name.startswith('q_')
|
2015-10-13 06:22:32 +02:00
|
|
|
|
2016-07-14 05:50:16 +02:00
|
|
|
def is_empty(self):
|
|
|
|
assert self.members is not None
|
|
|
|
return not self.members and not self.variants
|
|
|
|
|
2015-09-16 13:06:06 +02:00
|
|
|
def c_name(self):
|
2016-07-14 05:50:14 +02:00
|
|
|
assert self.name != 'q_empty'
|
2015-09-16 13:06:06 +02:00
|
|
|
return QAPISchemaType.c_name(self)
|
|
|
|
|
2016-03-17 23:48:28 +01:00
|
|
|
def c_type(self):
|
2015-10-13 06:22:27 +02:00
|
|
|
assert not self.is_implicit()
|
qapi: Don't box struct branch of alternate
There's no reason to do two malloc's for an alternate type visiting
a QAPI struct; let's just inline the struct directly as the C union
branch of the struct.
Surprisingly, no clients were actually using the struct member prior
to this patch outside of the testsuite; an earlier patch in the series
added some testsuite coverage to make the effect of this patch more
obvious.
In qapi.py, c_type() gains a new is_unboxed flag to control when we
are emitting a C struct unboxed within the context of an outer
struct (different from our other two modes of usage with no flags
for normal local variable declarations, and with is_param for adding
'const' in a parameter list). I don't know if there is any more
pythonic way of collapsing the two flags into a single parameter,
as we never have a caller setting both flags at once.
Ultimately, we want to also unbox branches for QAPI unions, but as
that touches a lot more client code, it is better as separate
patches. But since unions and alternates share gen_variants(), I
had to hack in a way to test if we are visiting an alternate type
for setting the is_unboxed flag: look for a non-object branch.
This works because alternates have at least two branches, with at
most one object branch, while unions have only object branches.
The hack will go away in a later patch.
The generated code difference to qapi-types.h is relatively small:
| struct BlockdevRef {
| QType type;
| union { /* union tag is @type */
| void *data;
|- BlockdevOptions *definition;
|+ BlockdevOptions definition;
| char *reference;
| } u;
| };
The corresponding spot in qapi-visit.c calls visit_type_FOO(), which
first calls visit_start_struct() to allocate or deallocate the member
and handle a layer of {} from the JSON stream, then visits the
members. To peel off the indirection and the memory management that
comes with it, we inline this call, then suppress allocation /
deallocation by passing NULL to visit_start_struct(), and adjust the
member visit:
| switch ((*obj)->type) {
| case QTYPE_QDICT:
|- visit_type_BlockdevOptions(v, name, &(*obj)->u.definition, &err);
|+ visit_start_struct(v, name, NULL, 0, &err);
|+ if (err) {
|+ break;
|+ }
|+ visit_type_BlockdevOptions_fields(v, &(*obj)->u.definition, &err);
|+ error_propagate(errp, err);
|+ err = NULL;
|+ visit_end_struct(v, &err);
| break;
| case QTYPE_QSTRING:
| visit_type_str(v, name, &(*obj)->u.reference, &err);
The visit of non-object fields is unchanged.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1455778109-6278-13-git-send-email-eblake@redhat.com>
[Commit message tweaked]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2016-02-18 07:48:26 +01:00
|
|
|
return c_name(self.name) + pointer_suffix
|
2015-09-16 13:06:06 +02:00
|
|
|
|
2016-03-17 23:48:28 +01:00
|
|
|
def c_unboxed_type(self):
|
|
|
|
return c_name(self.name)
|
|
|
|
|
2015-09-16 13:06:06 +02:00
|
|
|
def json_type(self):
|
|
|
|
return 'object'
|
|
|
|
|
2015-09-16 13:06:07 +02:00
|
|
|
def visit(self, visitor):
|
|
|
|
visitor.visit_object_type(self.name, self.info,
|
|
|
|
self.base, self.local_members, self.variants)
|
qapi: New QMP command query-qmp-schema for QMP introspection
qapi/introspect.json defines the introspection schema. It's designed
for QMP introspection, but should do for similar uses, such as QGA.
The introspection schema does not reflect all the rules and
restrictions that apply to QAPI schemata. A valid QAPI schema has an
introspection value conforming to the introspection schema, but the
converse is not true.
Introspection lowers away a number of schema details, and makes
implicit things explicit:
* The built-in types are declared with their JSON type.
All integer types are mapped to 'int', because how many bits we use
internally is an implementation detail. It could be pressed into
external interface service as very approximate range information,
but that's a bad idea. If we need range information, we better do
it properly.
* Implicit type definitions are made explicit, and given
auto-generated names:
- Array types, named by appending "List" to the name of their
element type, like in generated C.
- The enumeration types implicitly defined by simple union types,
named by appending "Kind" to the name of their simple union type,
like in generated C.
- Types that don't occur in generated C. Their names start with ':'
so they don't clash with the user's names.
* All type references are by name.
* The struct and union types are generalized into an object type.
* Base types are flattened.
* Commands take a single argument and return a single result.
Dictionary argument or list result is an implicit type definition.
The empty object type is used when a command takes no arguments or
produces no results.
The argument is always of object type, but the introspection schema
doesn't reflect that.
The 'gen': false directive is omitted as implementation detail.
The 'success-response' directive is omitted as well for now, even
though it's not an implementation detail, because it's not used by
QMP.
* Events carry a single data value.
Implicit type definition and empty object type use, just like for
commands.
The value is of object type, but the introspection schema doesn't
reflect that.
* Types not used by commands or events are omitted.
Indirect use counts as use.
* Optional members have a default, which can only be null right now
Instead of a mandatory "optional" flag, we have an optional default.
No default means mandatory, default null means optional without
default value. Non-null is available for optional with default
(possible future extension).
* Clients should *not* look up types by name, because type names are
not ABI. Look up the command or event you're interested in, then
follow the references.
TODO Should we hide the type names to eliminate the temptation?
New generator scripts/qapi-introspect.py computes an introspection
value for its input, and generates a C variable holding it.
It can generate awfully long lines. Marked TODO.
A new test-qmp-input-visitor test case feeds its result for both
tests/qapi-schema/qapi-schema-test.json and qapi-schema.json to a
QmpInputVisitor to verify it actually conforms to the schema.
New QMP command query-qmp-schema takes its return value from that
variable. Its reply is some 85KiBytes for me right now.
If this turns out to be too much, we have a couple of options:
* We can use shorter names in the JSON. Not the QMP style.
* Optionally return the sub-schema for commands and events given as
arguments.
Right now qmp_query_schema() sends the string literal computed by
qmp-introspect.py. To compute sub-schema at run time, we'd have to
duplicate parts of qapi-introspect.py in C. Unattractive.
* Let clients cache the output of query-qmp-schema.
It changes only on QEMU upgrades, i.e. rarely. Provide a command
query-qmp-schema-hash. Clients can have a cache indexed by hash,
and re-query the schema only when they don't have it cached. Even
simpler: put the hash in the QMP greeting.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2015-09-16 13:06:28 +02:00
|
|
|
visitor.visit_object_type_flat(self.name, self.info,
|
|
|
|
self.members, self.variants)
|
2015-09-16 13:06:07 +02:00
|
|
|
|
2015-09-16 13:06:05 +02:00
|
|
|
|
2015-12-02 06:20:54 +01:00
|
|
|
class QAPISchemaMember(object):
|
qapi: Track owner of each object member
Future commits will migrate semantic checking away from parsing
and over to the various QAPISchema*.check() methods. But to
report an error message about an incorrect semantic use of a
member of an object type, it helps to know which type, command,
or event owns the member. In particular, when a member is
inherited from a base type, it is desirable to associate the
member name with the base type (and not the type calling
member.check()).
Rather than packing additional information into the seen array
passed to each member.check() (as in seen[m.name] = {'member':m,
'owner':type}), it is easier to have each member track the name
of the owner type in the first place (keeping things simpler
with the existing seen[m.name] = m). The new member.owner field
is set via a new set_owner() method, called when registering
the members and variants arrays with an object or variant type.
Track only a name, and not the actual type object, to avoid
creating a circular python reference chain.
Note that Variants.set_owner() method does not set the owner
for the tag_member field; this field is set earlier either as
part of an object's non-variant members, or explicitly by
alternates.
The source information is intended for human consumption in
error messages, and a new describe() method is added to access
the resulting information. For example, given the qapi:
{ 'command': 'foo', 'data': { 'string': 'str' } }
an implementation of visit_command() that calls
arg_type.members[0].describe()
will see "'string' (parameter of foo)".
To make the human-readable name of implicit types work without
duplicating efforts, the describe() method has to reverse the
name of implicit types, via the helper _pretty_owner().
No change to generated code.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-16-git-send-email-eblake@redhat.com>
[Incorrect & unused -wrapper case in _pretty_owner() dropped]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-18 09:52:50 +01:00
|
|
|
role = 'member'
|
|
|
|
|
2015-12-02 06:20:54 +01:00
|
|
|
def __init__(self, name):
|
2015-09-16 13:06:05 +02:00
|
|
|
assert isinstance(name, str)
|
|
|
|
self.name = name
|
qapi: Track owner of each object member
Future commits will migrate semantic checking away from parsing
and over to the various QAPISchema*.check() methods. But to
report an error message about an incorrect semantic use of a
member of an object type, it helps to know which type, command,
or event owns the member. In particular, when a member is
inherited from a base type, it is desirable to associate the
member name with the base type (and not the type calling
member.check()).
Rather than packing additional information into the seen array
passed to each member.check() (as in seen[m.name] = {'member':m,
'owner':type}), it is easier to have each member track the name
of the owner type in the first place (keeping things simpler
with the existing seen[m.name] = m). The new member.owner field
is set via a new set_owner() method, called when registering
the members and variants arrays with an object or variant type.
Track only a name, and not the actual type object, to avoid
creating a circular python reference chain.
Note that Variants.set_owner() method does not set the owner
for the tag_member field; this field is set earlier either as
part of an object's non-variant members, or explicitly by
alternates.
The source information is intended for human consumption in
error messages, and a new describe() method is added to access
the resulting information. For example, given the qapi:
{ 'command': 'foo', 'data': { 'string': 'str' } }
an implementation of visit_command() that calls
arg_type.members[0].describe()
will see "'string' (parameter of foo)".
To make the human-readable name of implicit types work without
duplicating efforts, the describe() method has to reverse the
name of implicit types, via the helper _pretty_owner().
No change to generated code.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-16-git-send-email-eblake@redhat.com>
[Incorrect & unused -wrapper case in _pretty_owner() dropped]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-18 09:52:50 +01:00
|
|
|
self.owner = None
|
|
|
|
|
|
|
|
def set_owner(self, name):
|
|
|
|
assert not self.owner
|
|
|
|
self.owner = name
|
2015-09-16 13:06:05 +02:00
|
|
|
|
2015-11-18 09:52:51 +01:00
|
|
|
def check_clash(self, info, seen):
|
|
|
|
cname = c_name(self.name)
|
2017-03-15 13:56:55 +01:00
|
|
|
if cname.lower() != cname and self.owner not in name_case_whitelist:
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPISemError(info,
|
|
|
|
"%s should not use uppercase" % self.describe())
|
2015-11-18 09:52:51 +01:00
|
|
|
if cname in seen:
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPISemError(info, "%s collides with %s" %
|
|
|
|
(self.describe(), seen[cname].describe()))
|
2015-11-18 09:52:51 +01:00
|
|
|
seen[cname] = self
|
2015-11-18 09:52:44 +01:00
|
|
|
|
qapi: Track owner of each object member
Future commits will migrate semantic checking away from parsing
and over to the various QAPISchema*.check() methods. But to
report an error message about an incorrect semantic use of a
member of an object type, it helps to know which type, command,
or event owns the member. In particular, when a member is
inherited from a base type, it is desirable to associate the
member name with the base type (and not the type calling
member.check()).
Rather than packing additional information into the seen array
passed to each member.check() (as in seen[m.name] = {'member':m,
'owner':type}), it is easier to have each member track the name
of the owner type in the first place (keeping things simpler
with the existing seen[m.name] = m). The new member.owner field
is set via a new set_owner() method, called when registering
the members and variants arrays with an object or variant type.
Track only a name, and not the actual type object, to avoid
creating a circular python reference chain.
Note that Variants.set_owner() method does not set the owner
for the tag_member field; this field is set earlier either as
part of an object's non-variant members, or explicitly by
alternates.
The source information is intended for human consumption in
error messages, and a new describe() method is added to access
the resulting information. For example, given the qapi:
{ 'command': 'foo', 'data': { 'string': 'str' } }
an implementation of visit_command() that calls
arg_type.members[0].describe()
will see "'string' (parameter of foo)".
To make the human-readable name of implicit types work without
duplicating efforts, the describe() method has to reverse the
name of implicit types, via the helper _pretty_owner().
No change to generated code.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-16-git-send-email-eblake@redhat.com>
[Incorrect & unused -wrapper case in _pretty_owner() dropped]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-18 09:52:50 +01:00
|
|
|
def _pretty_owner(self):
|
|
|
|
owner = self.owner
|
2016-03-17 23:48:29 +01:00
|
|
|
if owner.startswith('q_obj_'):
|
qapi: Track owner of each object member
Future commits will migrate semantic checking away from parsing
and over to the various QAPISchema*.check() methods. But to
report an error message about an incorrect semantic use of a
member of an object type, it helps to know which type, command,
or event owns the member. In particular, when a member is
inherited from a base type, it is desirable to associate the
member name with the base type (and not the type calling
member.check()).
Rather than packing additional information into the seen array
passed to each member.check() (as in seen[m.name] = {'member':m,
'owner':type}), it is easier to have each member track the name
of the owner type in the first place (keeping things simpler
with the existing seen[m.name] = m). The new member.owner field
is set via a new set_owner() method, called when registering
the members and variants arrays with an object or variant type.
Track only a name, and not the actual type object, to avoid
creating a circular python reference chain.
Note that Variants.set_owner() method does not set the owner
for the tag_member field; this field is set earlier either as
part of an object's non-variant members, or explicitly by
alternates.
The source information is intended for human consumption in
error messages, and a new describe() method is added to access
the resulting information. For example, given the qapi:
{ 'command': 'foo', 'data': { 'string': 'str' } }
an implementation of visit_command() that calls
arg_type.members[0].describe()
will see "'string' (parameter of foo)".
To make the human-readable name of implicit types work without
duplicating efforts, the describe() method has to reverse the
name of implicit types, via the helper _pretty_owner().
No change to generated code.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-16-git-send-email-eblake@redhat.com>
[Incorrect & unused -wrapper case in _pretty_owner() dropped]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-18 09:52:50 +01:00
|
|
|
# See QAPISchema._make_implicit_object_type() - reverse the
|
|
|
|
# mapping there to create a nice human-readable description
|
2016-03-17 23:48:29 +01:00
|
|
|
owner = owner[6:]
|
qapi: Track owner of each object member
Future commits will migrate semantic checking away from parsing
and over to the various QAPISchema*.check() methods. But to
report an error message about an incorrect semantic use of a
member of an object type, it helps to know which type, command,
or event owns the member. In particular, when a member is
inherited from a base type, it is desirable to associate the
member name with the base type (and not the type calling
member.check()).
Rather than packing additional information into the seen array
passed to each member.check() (as in seen[m.name] = {'member':m,
'owner':type}), it is easier to have each member track the name
of the owner type in the first place (keeping things simpler
with the existing seen[m.name] = m). The new member.owner field
is set via a new set_owner() method, called when registering
the members and variants arrays with an object or variant type.
Track only a name, and not the actual type object, to avoid
creating a circular python reference chain.
Note that Variants.set_owner() method does not set the owner
for the tag_member field; this field is set earlier either as
part of an object's non-variant members, or explicitly by
alternates.
The source information is intended for human consumption in
error messages, and a new describe() method is added to access
the resulting information. For example, given the qapi:
{ 'command': 'foo', 'data': { 'string': 'str' } }
an implementation of visit_command() that calls
arg_type.members[0].describe()
will see "'string' (parameter of foo)".
To make the human-readable name of implicit types work without
duplicating efforts, the describe() method has to reverse the
name of implicit types, via the helper _pretty_owner().
No change to generated code.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-16-git-send-email-eblake@redhat.com>
[Incorrect & unused -wrapper case in _pretty_owner() dropped]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-18 09:52:50 +01:00
|
|
|
if owner.endswith('-arg'):
|
|
|
|
return '(parameter of %s)' % owner[:-4]
|
2016-03-17 23:48:39 +01:00
|
|
|
elif owner.endswith('-base'):
|
|
|
|
return '(base of %s)' % owner[:-5]
|
qapi: Track owner of each object member
Future commits will migrate semantic checking away from parsing
and over to the various QAPISchema*.check() methods. But to
report an error message about an incorrect semantic use of a
member of an object type, it helps to know which type, command,
or event owns the member. In particular, when a member is
inherited from a base type, it is desirable to associate the
member name with the base type (and not the type calling
member.check()).
Rather than packing additional information into the seen array
passed to each member.check() (as in seen[m.name] = {'member':m,
'owner':type}), it is easier to have each member track the name
of the owner type in the first place (keeping things simpler
with the existing seen[m.name] = m). The new member.owner field
is set via a new set_owner() method, called when registering
the members and variants arrays with an object or variant type.
Track only a name, and not the actual type object, to avoid
creating a circular python reference chain.
Note that Variants.set_owner() method does not set the owner
for the tag_member field; this field is set earlier either as
part of an object's non-variant members, or explicitly by
alternates.
The source information is intended for human consumption in
error messages, and a new describe() method is added to access
the resulting information. For example, given the qapi:
{ 'command': 'foo', 'data': { 'string': 'str' } }
an implementation of visit_command() that calls
arg_type.members[0].describe()
will see "'string' (parameter of foo)".
To make the human-readable name of implicit types work without
duplicating efforts, the describe() method has to reverse the
name of implicit types, via the helper _pretty_owner().
No change to generated code.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-16-git-send-email-eblake@redhat.com>
[Incorrect & unused -wrapper case in _pretty_owner() dropped]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-18 09:52:50 +01:00
|
|
|
else:
|
|
|
|
assert owner.endswith('-wrapper')
|
|
|
|
# Unreachable and not implemented
|
|
|
|
assert False
|
2015-12-02 06:20:55 +01:00
|
|
|
if owner.endswith('Kind'):
|
|
|
|
# See QAPISchema._make_implicit_enum_type()
|
|
|
|
return '(branch of %s)' % owner[:-4]
|
qapi: Track owner of each object member
Future commits will migrate semantic checking away from parsing
and over to the various QAPISchema*.check() methods. But to
report an error message about an incorrect semantic use of a
member of an object type, it helps to know which type, command,
or event owns the member. In particular, when a member is
inherited from a base type, it is desirable to associate the
member name with the base type (and not the type calling
member.check()).
Rather than packing additional information into the seen array
passed to each member.check() (as in seen[m.name] = {'member':m,
'owner':type}), it is easier to have each member track the name
of the owner type in the first place (keeping things simpler
with the existing seen[m.name] = m). The new member.owner field
is set via a new set_owner() method, called when registering
the members and variants arrays with an object or variant type.
Track only a name, and not the actual type object, to avoid
creating a circular python reference chain.
Note that Variants.set_owner() method does not set the owner
for the tag_member field; this field is set earlier either as
part of an object's non-variant members, or explicitly by
alternates.
The source information is intended for human consumption in
error messages, and a new describe() method is added to access
the resulting information. For example, given the qapi:
{ 'command': 'foo', 'data': { 'string': 'str' } }
an implementation of visit_command() that calls
arg_type.members[0].describe()
will see "'string' (parameter of foo)".
To make the human-readable name of implicit types work without
duplicating efforts, the describe() method has to reverse the
name of implicit types, via the helper _pretty_owner().
No change to generated code.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-16-git-send-email-eblake@redhat.com>
[Incorrect & unused -wrapper case in _pretty_owner() dropped]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-18 09:52:50 +01:00
|
|
|
return '(%s of %s)' % (self.role, owner)
|
|
|
|
|
|
|
|
def describe(self):
|
|
|
|
return "'%s' %s" % (self.name, self._pretty_owner())
|
|
|
|
|
2015-09-16 13:06:05 +02:00
|
|
|
|
2015-12-02 06:20:54 +01:00
|
|
|
class QAPISchemaObjectTypeMember(QAPISchemaMember):
|
|
|
|
def __init__(self, name, typ, optional):
|
|
|
|
QAPISchemaMember.__init__(self, name)
|
|
|
|
assert isinstance(typ, str)
|
|
|
|
assert isinstance(optional, bool)
|
|
|
|
self._type_name = typ
|
|
|
|
self.type = None
|
|
|
|
self.optional = optional
|
|
|
|
|
|
|
|
def check(self, schema):
|
|
|
|
assert self.owner
|
|
|
|
self.type = schema.lookup_type(self._type_name)
|
|
|
|
assert self.type
|
|
|
|
|
|
|
|
|
2015-09-16 13:06:05 +02:00
|
|
|
class QAPISchemaObjectTypeVariants(object):
|
2015-10-13 06:22:29 +02:00
|
|
|
def __init__(self, tag_name, tag_member, variants):
|
|
|
|
# Flat unions pass tag_name but not tag_member.
|
|
|
|
# Simple unions and alternates pass tag_member but not tag_name.
|
|
|
|
# After check(), tag_member is always set, and tag_name remains
|
|
|
|
# a reliable witness of being used by a flat union.
|
|
|
|
assert bool(tag_member) != bool(tag_name)
|
|
|
|
assert (isinstance(tag_name, str) or
|
|
|
|
isinstance(tag_member, QAPISchemaObjectTypeMember))
|
2016-02-18 07:48:16 +01:00
|
|
|
assert len(variants) > 0
|
2015-09-16 13:06:05 +02:00
|
|
|
for v in variants:
|
|
|
|
assert isinstance(v, QAPISchemaObjectTypeVariant)
|
2016-07-14 05:50:15 +02:00
|
|
|
self._tag_name = tag_name
|
2015-10-13 06:22:29 +02:00
|
|
|
self.tag_member = tag_member
|
2015-09-16 13:06:05 +02:00
|
|
|
self.variants = variants
|
|
|
|
|
qapi: Track owner of each object member
Future commits will migrate semantic checking away from parsing
and over to the various QAPISchema*.check() methods. But to
report an error message about an incorrect semantic use of a
member of an object type, it helps to know which type, command,
or event owns the member. In particular, when a member is
inherited from a base type, it is desirable to associate the
member name with the base type (and not the type calling
member.check()).
Rather than packing additional information into the seen array
passed to each member.check() (as in seen[m.name] = {'member':m,
'owner':type}), it is easier to have each member track the name
of the owner type in the first place (keeping things simpler
with the existing seen[m.name] = m). The new member.owner field
is set via a new set_owner() method, called when registering
the members and variants arrays with an object or variant type.
Track only a name, and not the actual type object, to avoid
creating a circular python reference chain.
Note that Variants.set_owner() method does not set the owner
for the tag_member field; this field is set earlier either as
part of an object's non-variant members, or explicitly by
alternates.
The source information is intended for human consumption in
error messages, and a new describe() method is added to access
the resulting information. For example, given the qapi:
{ 'command': 'foo', 'data': { 'string': 'str' } }
an implementation of visit_command() that calls
arg_type.members[0].describe()
will see "'string' (parameter of foo)".
To make the human-readable name of implicit types work without
duplicating efforts, the describe() method has to reverse the
name of implicit types, via the helper _pretty_owner().
No change to generated code.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-16-git-send-email-eblake@redhat.com>
[Incorrect & unused -wrapper case in _pretty_owner() dropped]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-18 09:52:50 +01:00
|
|
|
def set_owner(self, name):
|
|
|
|
for v in self.variants:
|
|
|
|
v.set_owner(name)
|
|
|
|
|
2015-11-18 09:52:41 +01:00
|
|
|
def check(self, schema, seen):
|
2015-11-18 09:52:45 +01:00
|
|
|
if not self.tag_member: # flat union
|
2016-07-14 05:50:15 +02:00
|
|
|
self.tag_member = seen[c_name(self._tag_name)]
|
|
|
|
assert self._tag_name == self.tag_member.name
|
2015-09-16 13:06:05 +02:00
|
|
|
assert isinstance(self.tag_member.type, QAPISchemaEnumType)
|
|
|
|
for v in self.variants:
|
2015-11-18 09:52:48 +01:00
|
|
|
v.check(schema)
|
qapi: Simplify visiting of alternate types
Previously, working with alternates required two lookup arrays
and some indirection: for type Foo, we created Foo_qtypes[]
which maps each qtype to a value of the generated FooKind enum,
then look up that value in FooKind_lookup[] like we do for other
union types.
This has a couple of subtle bugs. First, the generator was
creating a call with a parameter '(int *) &(*obj)->type' where
type is an enum type; this is unsafe if the compiler chooses
to store the enum type in a different size than int, where
assigning through the wrong size pointer can corrupt data or
cause a SIGBUS.
Related bug, not not fixed in this patch: qapi-visit.py's
gen_visit_enum() generates a cast of its enum * argument to
int *. Marked FIXME.
Second, since the values of the FooKind enum start at zero, all
entries of the Foo_qtypes[] array that were not explicitly
initialized will map to the same branch of the union as the
first member of the alternate, rather than triggering a desired
failure in visit_get_next_type(). Fortunately, the bug seldom
bites; the very next thing the input visitor does is try to
parse the incoming JSON with the wrong parser, which normally
fails; the output visitor is not used with a C struct in that
state, and the dealloc visitor has nothing to clean up (so
there is no leak).
However, the second bug IS observable in one case: parsing an
integer causes unusual behavior in an alternate that contains
at least a 'number' member but no 'int' member, because the
'number' parser accepts QTYPE_QINT in addition to the expected
QTYPE_QFLOAT (that is, since 'int' is not a member, the type
QTYPE_QINT accidentally maps to FooKind 0; if this enum value
is the 'number' branch the integer parses successfully, but if
the 'number' branch is not first, some other branch tries to
parse the integer and rejects it). A later patch will worry
about fixing alternates to always parse all inputs that a
non-alternate 'number' would accept, for now this is still
marked FIXME in the updated test-qmp-input-visitor.c, to
merely point out that new undesired behavior of 'ans' matches
the existing undesired behavior of 'asn'.
This patch fixes the default-initialization bug by deleting the
indirection, and modifying get_next_type() to directly assign a
QTypeCode parameter. This in turn fixes the type-casting bug,
as we are no longer casting a pointer to enum to a questionable
size. There is no longer a need to generate an implicit FooKind
enum associated with the alternate type (since the QMP wire
format never uses the stringized counterparts of the C union
member names). Since the updated visit_get_next_type() does not
know which qtypes are expected, the generated visitor is
modified to generate an error statement if an unexpected type is
encountered.
Callers now have to know the QTYPE_* mapping when looking at the
discriminator; but so far, only the testsuite was even using the
C struct of an alternate types. I considered the possibility of
keeping the internal enum FooKind, but initialized differently
than most generated arrays, as in:
typedef enum FooKind {
FOO_KIND_A = QTYPE_QDICT,
FOO_KIND_B = QTYPE_QINT,
} FooKind;
to create nicer aliases for knowing when to use foo->a or foo->b
when inspecting foo->type; but it turned out to add too much
complexity, especially without a client.
There is a user-visible side effect to this change, but I
consider it to be an improvement. Previously,
the invalid QMP command:
{"execute":"blockdev-add", "arguments":{"options":
{"driver":"raw", "id":"a", "file":true}}}
failed with:
{"error": {"class": "GenericError",
"desc": "Invalid parameter type for 'file', expected: QDict"}}
(visit_get_next_type() succeeded, and the error comes from the
visit_type_BlockdevOptions() expecting {}; there is no mention of
the fact that a string would also work). Now it fails with:
{"error": {"class": "GenericError",
"desc": "Invalid parameter type for 'file', expected: BlockdevRef"}}
(the error when the next type doesn't match any expected types for
the overall alternate).
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1449033659-25497-5-git-send-email-eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-12-02 06:20:48 +01:00
|
|
|
# Union names must match enum values; alternate names are
|
|
|
|
# checked separately. Use 'seen' to tell the two apart.
|
|
|
|
if seen:
|
2015-12-02 06:20:55 +01:00
|
|
|
assert v.name in self.tag_member.type.member_names()
|
qapi: Simplify visiting of alternate types
Previously, working with alternates required two lookup arrays
and some indirection: for type Foo, we created Foo_qtypes[]
which maps each qtype to a value of the generated FooKind enum,
then look up that value in FooKind_lookup[] like we do for other
union types.
This has a couple of subtle bugs. First, the generator was
creating a call with a parameter '(int *) &(*obj)->type' where
type is an enum type; this is unsafe if the compiler chooses
to store the enum type in a different size than int, where
assigning through the wrong size pointer can corrupt data or
cause a SIGBUS.
Related bug, not not fixed in this patch: qapi-visit.py's
gen_visit_enum() generates a cast of its enum * argument to
int *. Marked FIXME.
Second, since the values of the FooKind enum start at zero, all
entries of the Foo_qtypes[] array that were not explicitly
initialized will map to the same branch of the union as the
first member of the alternate, rather than triggering a desired
failure in visit_get_next_type(). Fortunately, the bug seldom
bites; the very next thing the input visitor does is try to
parse the incoming JSON with the wrong parser, which normally
fails; the output visitor is not used with a C struct in that
state, and the dealloc visitor has nothing to clean up (so
there is no leak).
However, the second bug IS observable in one case: parsing an
integer causes unusual behavior in an alternate that contains
at least a 'number' member but no 'int' member, because the
'number' parser accepts QTYPE_QINT in addition to the expected
QTYPE_QFLOAT (that is, since 'int' is not a member, the type
QTYPE_QINT accidentally maps to FooKind 0; if this enum value
is the 'number' branch the integer parses successfully, but if
the 'number' branch is not first, some other branch tries to
parse the integer and rejects it). A later patch will worry
about fixing alternates to always parse all inputs that a
non-alternate 'number' would accept, for now this is still
marked FIXME in the updated test-qmp-input-visitor.c, to
merely point out that new undesired behavior of 'ans' matches
the existing undesired behavior of 'asn'.
This patch fixes the default-initialization bug by deleting the
indirection, and modifying get_next_type() to directly assign a
QTypeCode parameter. This in turn fixes the type-casting bug,
as we are no longer casting a pointer to enum to a questionable
size. There is no longer a need to generate an implicit FooKind
enum associated with the alternate type (since the QMP wire
format never uses the stringized counterparts of the C union
member names). Since the updated visit_get_next_type() does not
know which qtypes are expected, the generated visitor is
modified to generate an error statement if an unexpected type is
encountered.
Callers now have to know the QTYPE_* mapping when looking at the
discriminator; but so far, only the testsuite was even using the
C struct of an alternate types. I considered the possibility of
keeping the internal enum FooKind, but initialized differently
than most generated arrays, as in:
typedef enum FooKind {
FOO_KIND_A = QTYPE_QDICT,
FOO_KIND_B = QTYPE_QINT,
} FooKind;
to create nicer aliases for knowing when to use foo->a or foo->b
when inspecting foo->type; but it turned out to add too much
complexity, especially without a client.
There is a user-visible side effect to this change, but I
consider it to be an improvement. Previously,
the invalid QMP command:
{"execute":"blockdev-add", "arguments":{"options":
{"driver":"raw", "id":"a", "file":true}}}
failed with:
{"error": {"class": "GenericError",
"desc": "Invalid parameter type for 'file', expected: QDict"}}
(visit_get_next_type() succeeded, and the error comes from the
visit_type_BlockdevOptions() expecting {}; there is no mention of
the fact that a string would also work). Now it fails with:
{"error": {"class": "GenericError",
"desc": "Invalid parameter type for 'file', expected: BlockdevRef"}}
(the error when the next type doesn't match any expected types for
the overall alternate).
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1449033659-25497-5-git-send-email-eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-12-02 06:20:48 +01:00
|
|
|
assert isinstance(v.type, QAPISchemaObjectType)
|
qapi: Check for QAPI collisions involving variant members
Right now, our ad hoc parser ensures that we cannot have a
flat union that introduces any members that would clash with
non-variant members inherited from the union's base type (see
flat-union-clash-member.json). We want QAPISchemaObjectType.check()
to make the same check, so we can later reduce some of the ad
hoc checks.
We already have a map 'seen' of all non-variant members. We
still need to check for collisions between each variant type's
members and the non-variant ones.
To know the variant type's members, we need to call
variant.type.check(). This also detects when a type contains
itself in a variant, exactly like the existing base.check()
detects when a type contains itself as a base. (Except that
we currently forbid anything but a struct as the type of a
variant, so we can't actually trigger this type of loop yet.)
Slight complication: an alternate's variant can have arbitrary
type, but only an object type's check() may be called outside
QAPISchema.check(). We could either skip the call for variants
of alternates, or skip it for non-object types. For now, do
the latter, because it's easier.
Then we call each variant member's check_clash() with the
appropriate 'seen' map. Since members of different variants
can't clash, we have to clone a fresh seen for each variant.
Wrap this in a new helper method
QAPISchemaObjectTypeVariants.check_clash().
Note that cloning 'seen' inside .check_clash() resembles
the one we just removed from .check() in 'qapi: Drop
obsolete tag value collision assertions'; the difference here is
that we are now checking for clashes among the qapi members of
the variant type, rather than for a single clash with the variant
tag name itself.
Note that, by construction, collisions can't actually happen for
simple unions: each variant's type is a wrapper with a single
member 'data', which will never collide with the only non-variant
member 'type'.
For alternates, there's nothing for a variant object type's
members to clash with, and therefore no need to call the new
variants.check_clash().
No change to generated code.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-12-git-send-email-eblake@redhat.com>
[Commit message tweaked]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-18 09:52:46 +01:00
|
|
|
v.type.check(schema)
|
|
|
|
|
2017-03-15 13:57:34 +01:00
|
|
|
def check_clash(self, info, seen):
|
qapi: Check for QAPI collisions involving variant members
Right now, our ad hoc parser ensures that we cannot have a
flat union that introduces any members that would clash with
non-variant members inherited from the union's base type (see
flat-union-clash-member.json). We want QAPISchemaObjectType.check()
to make the same check, so we can later reduce some of the ad
hoc checks.
We already have a map 'seen' of all non-variant members. We
still need to check for collisions between each variant type's
members and the non-variant ones.
To know the variant type's members, we need to call
variant.type.check(). This also detects when a type contains
itself in a variant, exactly like the existing base.check()
detects when a type contains itself as a base. (Except that
we currently forbid anything but a struct as the type of a
variant, so we can't actually trigger this type of loop yet.)
Slight complication: an alternate's variant can have arbitrary
type, but only an object type's check() may be called outside
QAPISchema.check(). We could either skip the call for variants
of alternates, or skip it for non-object types. For now, do
the latter, because it's easier.
Then we call each variant member's check_clash() with the
appropriate 'seen' map. Since members of different variants
can't clash, we have to clone a fresh seen for each variant.
Wrap this in a new helper method
QAPISchemaObjectTypeVariants.check_clash().
Note that cloning 'seen' inside .check_clash() resembles
the one we just removed from .check() in 'qapi: Drop
obsolete tag value collision assertions'; the difference here is
that we are now checking for clashes among the qapi members of
the variant type, rather than for a single clash with the variant
tag name itself.
Note that, by construction, collisions can't actually happen for
simple unions: each variant's type is a wrapper with a single
member 'data', which will never collide with the only non-variant
member 'type'.
For alternates, there's nothing for a variant object type's
members to clash with, and therefore no need to call the new
variants.check_clash().
No change to generated code.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-12-git-send-email-eblake@redhat.com>
[Commit message tweaked]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-18 09:52:46 +01:00
|
|
|
for v in self.variants:
|
|
|
|
# Reset seen map for each variant, since qapi names from one
|
|
|
|
# branch do not affect another branch
|
|
|
|
assert isinstance(v.type, QAPISchemaObjectType)
|
2017-03-15 13:57:34 +01:00
|
|
|
v.type.check_clash(info, dict(seen))
|
2015-09-16 13:06:05 +02:00
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2015-09-16 13:06:05 +02:00
|
|
|
class QAPISchemaObjectTypeVariant(QAPISchemaObjectTypeMember):
|
qapi: Track owner of each object member
Future commits will migrate semantic checking away from parsing
and over to the various QAPISchema*.check() methods. But to
report an error message about an incorrect semantic use of a
member of an object type, it helps to know which type, command,
or event owns the member. In particular, when a member is
inherited from a base type, it is desirable to associate the
member name with the base type (and not the type calling
member.check()).
Rather than packing additional information into the seen array
passed to each member.check() (as in seen[m.name] = {'member':m,
'owner':type}), it is easier to have each member track the name
of the owner type in the first place (keeping things simpler
with the existing seen[m.name] = m). The new member.owner field
is set via a new set_owner() method, called when registering
the members and variants arrays with an object or variant type.
Track only a name, and not the actual type object, to avoid
creating a circular python reference chain.
Note that Variants.set_owner() method does not set the owner
for the tag_member field; this field is set earlier either as
part of an object's non-variant members, or explicitly by
alternates.
The source information is intended for human consumption in
error messages, and a new describe() method is added to access
the resulting information. For example, given the qapi:
{ 'command': 'foo', 'data': { 'string': 'str' } }
an implementation of visit_command() that calls
arg_type.members[0].describe()
will see "'string' (parameter of foo)".
To make the human-readable name of implicit types work without
duplicating efforts, the describe() method has to reverse the
name of implicit types, via the helper _pretty_owner().
No change to generated code.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-16-git-send-email-eblake@redhat.com>
[Incorrect & unused -wrapper case in _pretty_owner() dropped]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-18 09:52:50 +01:00
|
|
|
role = 'branch'
|
|
|
|
|
2015-09-16 13:06:05 +02:00
|
|
|
def __init__(self, name, typ):
|
|
|
|
QAPISchemaObjectTypeMember.__init__(self, name, typ, False)
|
|
|
|
|
|
|
|
|
|
|
|
class QAPISchemaAlternateType(QAPISchemaType):
|
2017-03-15 13:57:03 +01:00
|
|
|
def __init__(self, name, info, doc, variants):
|
|
|
|
QAPISchemaType.__init__(self, name, info, doc)
|
2015-09-16 13:06:05 +02:00
|
|
|
assert isinstance(variants, QAPISchemaObjectTypeVariants)
|
2016-07-14 05:50:15 +02:00
|
|
|
assert variants.tag_member
|
qapi: Track owner of each object member
Future commits will migrate semantic checking away from parsing
and over to the various QAPISchema*.check() methods. But to
report an error message about an incorrect semantic use of a
member of an object type, it helps to know which type, command,
or event owns the member. In particular, when a member is
inherited from a base type, it is desirable to associate the
member name with the base type (and not the type calling
member.check()).
Rather than packing additional information into the seen array
passed to each member.check() (as in seen[m.name] = {'member':m,
'owner':type}), it is easier to have each member track the name
of the owner type in the first place (keeping things simpler
with the existing seen[m.name] = m). The new member.owner field
is set via a new set_owner() method, called when registering
the members and variants arrays with an object or variant type.
Track only a name, and not the actual type object, to avoid
creating a circular python reference chain.
Note that Variants.set_owner() method does not set the owner
for the tag_member field; this field is set earlier either as
part of an object's non-variant members, or explicitly by
alternates.
The source information is intended for human consumption in
error messages, and a new describe() method is added to access
the resulting information. For example, given the qapi:
{ 'command': 'foo', 'data': { 'string': 'str' } }
an implementation of visit_command() that calls
arg_type.members[0].describe()
will see "'string' (parameter of foo)".
To make the human-readable name of implicit types work without
duplicating efforts, the describe() method has to reverse the
name of implicit types, via the helper _pretty_owner().
No change to generated code.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-16-git-send-email-eblake@redhat.com>
[Incorrect & unused -wrapper case in _pretty_owner() dropped]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-18 09:52:50 +01:00
|
|
|
variants.set_owner(name)
|
|
|
|
variants.tag_member.set_owner(self.name)
|
2015-09-16 13:06:05 +02:00
|
|
|
self.variants = variants
|
|
|
|
|
|
|
|
def check(self, schema):
|
2015-11-18 09:52:40 +01:00
|
|
|
self.variants.tag_member.check(schema)
|
qapi: Check for QAPI collisions involving variant members
Right now, our ad hoc parser ensures that we cannot have a
flat union that introduces any members that would clash with
non-variant members inherited from the union's base type (see
flat-union-clash-member.json). We want QAPISchemaObjectType.check()
to make the same check, so we can later reduce some of the ad
hoc checks.
We already have a map 'seen' of all non-variant members. We
still need to check for collisions between each variant type's
members and the non-variant ones.
To know the variant type's members, we need to call
variant.type.check(). This also detects when a type contains
itself in a variant, exactly like the existing base.check()
detects when a type contains itself as a base. (Except that
we currently forbid anything but a struct as the type of a
variant, so we can't actually trigger this type of loop yet.)
Slight complication: an alternate's variant can have arbitrary
type, but only an object type's check() may be called outside
QAPISchema.check(). We could either skip the call for variants
of alternates, or skip it for non-object types. For now, do
the latter, because it's easier.
Then we call each variant member's check_clash() with the
appropriate 'seen' map. Since members of different variants
can't clash, we have to clone a fresh seen for each variant.
Wrap this in a new helper method
QAPISchemaObjectTypeVariants.check_clash().
Note that cloning 'seen' inside .check_clash() resembles
the one we just removed from .check() in 'qapi: Drop
obsolete tag value collision assertions'; the difference here is
that we are now checking for clashes among the qapi members of
the variant type, rather than for a single clash with the variant
tag name itself.
Note that, by construction, collisions can't actually happen for
simple unions: each variant's type is a wrapper with a single
member 'data', which will never collide with the only non-variant
member 'type'.
For alternates, there's nothing for a variant object type's
members to clash with, and therefore no need to call the new
variants.check_clash().
No change to generated code.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-12-git-send-email-eblake@redhat.com>
[Commit message tweaked]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-18 09:52:46 +01:00
|
|
|
# Not calling self.variants.check_clash(), because there's nothing
|
|
|
|
# to clash with
|
2015-11-18 09:52:41 +01:00
|
|
|
self.variants.check(schema, {})
|
qapi: Simplify visiting of alternate types
Previously, working with alternates required two lookup arrays
and some indirection: for type Foo, we created Foo_qtypes[]
which maps each qtype to a value of the generated FooKind enum,
then look up that value in FooKind_lookup[] like we do for other
union types.
This has a couple of subtle bugs. First, the generator was
creating a call with a parameter '(int *) &(*obj)->type' where
type is an enum type; this is unsafe if the compiler chooses
to store the enum type in a different size than int, where
assigning through the wrong size pointer can corrupt data or
cause a SIGBUS.
Related bug, not not fixed in this patch: qapi-visit.py's
gen_visit_enum() generates a cast of its enum * argument to
int *. Marked FIXME.
Second, since the values of the FooKind enum start at zero, all
entries of the Foo_qtypes[] array that were not explicitly
initialized will map to the same branch of the union as the
first member of the alternate, rather than triggering a desired
failure in visit_get_next_type(). Fortunately, the bug seldom
bites; the very next thing the input visitor does is try to
parse the incoming JSON with the wrong parser, which normally
fails; the output visitor is not used with a C struct in that
state, and the dealloc visitor has nothing to clean up (so
there is no leak).
However, the second bug IS observable in one case: parsing an
integer causes unusual behavior in an alternate that contains
at least a 'number' member but no 'int' member, because the
'number' parser accepts QTYPE_QINT in addition to the expected
QTYPE_QFLOAT (that is, since 'int' is not a member, the type
QTYPE_QINT accidentally maps to FooKind 0; if this enum value
is the 'number' branch the integer parses successfully, but if
the 'number' branch is not first, some other branch tries to
parse the integer and rejects it). A later patch will worry
about fixing alternates to always parse all inputs that a
non-alternate 'number' would accept, for now this is still
marked FIXME in the updated test-qmp-input-visitor.c, to
merely point out that new undesired behavior of 'ans' matches
the existing undesired behavior of 'asn'.
This patch fixes the default-initialization bug by deleting the
indirection, and modifying get_next_type() to directly assign a
QTypeCode parameter. This in turn fixes the type-casting bug,
as we are no longer casting a pointer to enum to a questionable
size. There is no longer a need to generate an implicit FooKind
enum associated with the alternate type (since the QMP wire
format never uses the stringized counterparts of the C union
member names). Since the updated visit_get_next_type() does not
know which qtypes are expected, the generated visitor is
modified to generate an error statement if an unexpected type is
encountered.
Callers now have to know the QTYPE_* mapping when looking at the
discriminator; but so far, only the testsuite was even using the
C struct of an alternate types. I considered the possibility of
keeping the internal enum FooKind, but initialized differently
than most generated arrays, as in:
typedef enum FooKind {
FOO_KIND_A = QTYPE_QDICT,
FOO_KIND_B = QTYPE_QINT,
} FooKind;
to create nicer aliases for knowing when to use foo->a or foo->b
when inspecting foo->type; but it turned out to add too much
complexity, especially without a client.
There is a user-visible side effect to this change, but I
consider it to be an improvement. Previously,
the invalid QMP command:
{"execute":"blockdev-add", "arguments":{"options":
{"driver":"raw", "id":"a", "file":true}}}
failed with:
{"error": {"class": "GenericError",
"desc": "Invalid parameter type for 'file', expected: QDict"}}
(visit_get_next_type() succeeded, and the error comes from the
visit_type_BlockdevOptions() expecting {}; there is no mention of
the fact that a string would also work). Now it fails with:
{"error": {"class": "GenericError",
"desc": "Invalid parameter type for 'file', expected: BlockdevRef"}}
(the error when the next type doesn't match any expected types for
the overall alternate).
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1449033659-25497-5-git-send-email-eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-12-02 06:20:48 +01:00
|
|
|
# Alternate branch names have no relation to the tag enum values;
|
|
|
|
# so we have to check for potential name collisions ourselves.
|
|
|
|
seen = {}
|
|
|
|
for v in self.variants.variants:
|
|
|
|
v.check_clash(self.info, seen)
|
2017-03-15 13:57:03 +01:00
|
|
|
if self.doc:
|
|
|
|
self.doc.connect_member(v)
|
2017-03-15 13:57:26 +01:00
|
|
|
if self.doc:
|
|
|
|
self.doc.check()
|
2015-09-16 13:06:05 +02:00
|
|
|
|
2016-03-17 23:48:28 +01:00
|
|
|
def c_type(self):
|
|
|
|
return c_name(self.name) + pointer_suffix
|
|
|
|
|
2015-09-16 13:06:06 +02:00
|
|
|
def json_type(self):
|
|
|
|
return 'value'
|
|
|
|
|
2015-09-16 13:06:07 +02:00
|
|
|
def visit(self, visitor):
|
|
|
|
visitor.visit_alternate_type(self.name, self.info, self.variants)
|
|
|
|
|
2016-07-14 05:50:20 +02:00
|
|
|
def is_empty(self):
|
|
|
|
return False
|
|
|
|
|
2015-09-16 13:06:05 +02:00
|
|
|
|
|
|
|
class QAPISchemaCommand(QAPISchemaEntity):
|
2017-03-15 13:57:03 +01:00
|
|
|
def __init__(self, name, info, doc, arg_type, ret_type,
|
|
|
|
gen, success_response, boxed):
|
|
|
|
QAPISchemaEntity.__init__(self, name, info, doc)
|
2015-09-16 13:06:05 +02:00
|
|
|
assert not arg_type or isinstance(arg_type, str)
|
|
|
|
assert not ret_type or isinstance(ret_type, str)
|
|
|
|
self._arg_type_name = arg_type
|
|
|
|
self.arg_type = None
|
|
|
|
self._ret_type_name = ret_type
|
|
|
|
self.ret_type = None
|
|
|
|
self.gen = gen
|
|
|
|
self.success_response = success_response
|
2016-07-14 05:50:19 +02:00
|
|
|
self.boxed = boxed
|
2015-09-16 13:06:05 +02:00
|
|
|
|
|
|
|
def check(self, schema):
|
|
|
|
if self._arg_type_name:
|
|
|
|
self.arg_type = schema.lookup_type(self._arg_type_name)
|
2016-07-14 05:50:20 +02:00
|
|
|
assert (isinstance(self.arg_type, QAPISchemaObjectType) or
|
|
|
|
isinstance(self.arg_type, QAPISchemaAlternateType))
|
|
|
|
self.arg_type.check(schema)
|
|
|
|
if self.boxed:
|
|
|
|
if self.arg_type.is_empty():
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPISemError(self.info,
|
|
|
|
"Cannot use 'boxed' with empty type")
|
2016-07-14 05:50:20 +02:00
|
|
|
else:
|
|
|
|
assert not isinstance(self.arg_type, QAPISchemaAlternateType)
|
|
|
|
assert not self.arg_type.variants
|
|
|
|
elif self.boxed:
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPISemError(self.info, "Use of 'boxed' requires 'data'")
|
2015-09-16 13:06:05 +02:00
|
|
|
if self._ret_type_name:
|
|
|
|
self.ret_type = schema.lookup_type(self._ret_type_name)
|
|
|
|
assert isinstance(self.ret_type, QAPISchemaType)
|
|
|
|
|
2015-09-16 13:06:07 +02:00
|
|
|
def visit(self, visitor):
|
|
|
|
visitor.visit_command(self.name, self.info,
|
|
|
|
self.arg_type, self.ret_type,
|
2016-07-14 05:50:19 +02:00
|
|
|
self.gen, self.success_response, self.boxed)
|
2015-09-16 13:06:07 +02:00
|
|
|
|
2015-09-16 13:06:05 +02:00
|
|
|
|
|
|
|
class QAPISchemaEvent(QAPISchemaEntity):
|
2017-03-15 13:57:03 +01:00
|
|
|
def __init__(self, name, info, doc, arg_type, boxed):
|
|
|
|
QAPISchemaEntity.__init__(self, name, info, doc)
|
2015-09-16 13:06:05 +02:00
|
|
|
assert not arg_type or isinstance(arg_type, str)
|
|
|
|
self._arg_type_name = arg_type
|
|
|
|
self.arg_type = None
|
2016-07-14 05:50:19 +02:00
|
|
|
self.boxed = boxed
|
2015-09-16 13:06:05 +02:00
|
|
|
|
|
|
|
def check(self, schema):
|
|
|
|
if self._arg_type_name:
|
|
|
|
self.arg_type = schema.lookup_type(self._arg_type_name)
|
2016-07-14 05:50:20 +02:00
|
|
|
assert (isinstance(self.arg_type, QAPISchemaObjectType) or
|
|
|
|
isinstance(self.arg_type, QAPISchemaAlternateType))
|
|
|
|
self.arg_type.check(schema)
|
|
|
|
if self.boxed:
|
|
|
|
if self.arg_type.is_empty():
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPISemError(self.info,
|
|
|
|
"Cannot use 'boxed' with empty type")
|
2016-07-14 05:50:20 +02:00
|
|
|
else:
|
|
|
|
assert not isinstance(self.arg_type, QAPISchemaAlternateType)
|
|
|
|
assert not self.arg_type.variants
|
|
|
|
elif self.boxed:
|
2017-01-13 15:41:25 +01:00
|
|
|
raise QAPISemError(self.info, "Use of 'boxed' requires 'data'")
|
2015-09-16 13:06:05 +02:00
|
|
|
|
2015-09-16 13:06:07 +02:00
|
|
|
def visit(self, visitor):
|
2016-07-14 05:50:19 +02:00
|
|
|
visitor.visit_event(self.name, self.info, self.arg_type, self.boxed)
|
2015-09-16 13:06:07 +02:00
|
|
|
|
2015-09-16 13:06:05 +02:00
|
|
|
|
|
|
|
class QAPISchema(object):
|
|
|
|
def __init__(self, fname):
|
|
|
|
try:
|
2017-03-15 13:57:08 +01:00
|
|
|
parser = QAPISchemaParser(open(fname, 'r'))
|
qapi: add qapi2texi script
As the name suggests, the qapi2texi script converts JSON QAPI
description into a texi file suitable for different target
formats (info/man/txt/pdf/html...).
It parses the following kind of blocks:
Free-form:
##
# = Section
# == Subsection
#
# Some text foo with *emphasis*
# 1. with a list
# 2. like that
#
# And some code:
# | $ echo foo
# | -> do this
# | <- get that
#
##
Symbol description:
##
# @symbol:
#
# Symbol body ditto ergo sum. Foo bar
# baz ding.
#
# @param1: the frob to frobnicate
# @param2: #optional how hard to frobnicate
#
# Returns: the frobnicated frob.
# If frob isn't frobnicatable, GenericError.
#
# Since: version
# Notes: notes, comments can have
# - itemized list
# - like this
#
# Example:
#
# -> { "execute": "quit" }
# <- { "return": {} }
#
##
That's roughly following the following EBNF grammar:
api_comment = "##\n" comment "##\n"
comment = freeform_comment | symbol_comment
freeform_comment = { "# " text "\n" | "#\n" }
symbol_comment = "# @" name ":\n" { member | tag_section | freeform_comment }
member = "# @" name ':' [ text ] "\n" freeform_comment
tag_section = "# " ( "Returns:", "Since:", "Note:", "Notes:", "Example:", "Examples:" ) [ text ] "\n" freeform_comment
text = free text with markup
Note that the grammar is ambiguous: a line "# @foo:\n" can be parsed
both as freeform_comment and as symbol_comment. The actual parser
recognizes symbol_comment.
See docs/qapi-code-gen.txt for more details.
Deficiencies and limitations:
- the generated QMP documentation includes internal types
- union type support is lacking
- type information is lacking in generated documentation
- doc comment error message positions are imprecise, they point
to the beginning of the comment.
- a few minor issues, all marked TODO/FIXME in the code
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20170113144135.5150-16-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[test-qapi.py tweaked to avoid trailing empty lines in .out]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2017-01-13 15:41:29 +01:00
|
|
|
self.exprs = check_exprs(parser.exprs)
|
2017-03-15 13:57:27 +01:00
|
|
|
self.docs = parser.docs
|
2015-10-13 06:22:22 +02:00
|
|
|
self._entity_dict = {}
|
2015-10-13 06:22:32 +02:00
|
|
|
self._predefining = True
|
2015-10-13 06:22:22 +02:00
|
|
|
self._def_predefineds()
|
2015-10-13 06:22:32 +02:00
|
|
|
self._predefining = False
|
2015-10-13 06:22:22 +02:00
|
|
|
self._def_exprs()
|
|
|
|
self.check()
|
2017-01-13 15:41:25 +01:00
|
|
|
except QAPIError as err:
|
2015-09-16 13:06:05 +02:00
|
|
|
print >>sys.stderr, err
|
|
|
|
exit(1)
|
|
|
|
|
|
|
|
def _def_entity(self, ent):
|
2015-10-13 06:22:32 +02:00
|
|
|
# Only the predefined types are allowed to not have info
|
|
|
|
assert ent.info or self._predefining
|
2015-09-16 13:06:05 +02:00
|
|
|
assert ent.name not in self._entity_dict
|
|
|
|
self._entity_dict[ent.name] = ent
|
|
|
|
|
|
|
|
def lookup_entity(self, name, typ=None):
|
|
|
|
ent = self._entity_dict.get(name)
|
|
|
|
if typ and not isinstance(ent, typ):
|
|
|
|
return None
|
|
|
|
return ent
|
|
|
|
|
|
|
|
def lookup_type(self, name):
|
|
|
|
return self.lookup_entity(name, QAPISchemaType)
|
|
|
|
|
2016-03-17 23:48:36 +01:00
|
|
|
def _def_builtin_type(self, name, json_type, c_type):
|
|
|
|
self._def_entity(QAPISchemaBuiltinType(name, json_type, c_type))
|
qapi: Lazy creation of array types
Commit ac88219a had several TODO markers about whether we needed
to automatically create the corresponding array type alongside
any other type. It turns out that most of the time, we don't!
There are a few exceptions: 1) We have a few situations where we
use an array type in internal code but do not expose that type
through QMP; fix it by declaring a dummy type that forces the
generator to see that we want to use the array type.
2) The builtin arrays (such as intList for QAPI ['int']) must
always be generated, because of the way our QAPI_TYPES_BUILTIN
compile guard works: we have situations (at the very least
tests/test-qmp-output-visitor.c) that include both top-level
"qapi-types.h" (via "error.h") and a secondary
"test-qapi-types.h". If we were to only emit the builtin types
when used locally, then the first .h file would not include all
types, but the second .h does not declare anything at all because
the first .h set QAPI_TYPES_BUILTIN, and we would end up with
compilation error due to things like unknown type 'int8List'.
Actually, we may need to revisit how we do type guards, and
change from a single QAPI_TYPES_BUILTIN over to a different
usage pattern that does one #ifdef per qapi type - right now,
the only types that are declared multiple times between two qapi
.json files for inclusion by a single .c file happen to be the
builtin arrays. But now that we have QAPI 'include' statements,
it is logical to assume that we will soon reach a point where
we want to reuse non-builtin types (yes, I'm thinking about what
it will take to add introspection to QGA, where we will want to
reuse the SchemaInfo type and friends). One #ifdef per type
will help ensure that generating the same qapi type into more
than one qapi-types.h won't cause collisions when both are
included in the same .c file; but we also have to solve how to
avoid creating duplicate qapi-types.c entry points. So that
is a problem left for another day.
Generated code for qapi-types and qapi-visit is drastically
reduced; less than a third of the arrays that were blindly
created were actually needed (a quick grep shows we dropped
from 219 to 69 *List types), and the .o files lost more than
30% of their bulk. [For best results, diff the generated
files with 'git diff --patience --no-index pre post'.]
Interestingly, the introspection output is unchanged - this is
because we already cull all types that are not indirectly
reachable from a command or event, so introspection was already
using only a subset of array types. The subset of types
introspected is now a much larger percentage of the overall set
of array types emitted in qapi-types.h (since the larger set
shrunk), but still not 100% (evidence that the array types
emitted for our new Dummy structs, and the new struct itself,
don't affect QMP).
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1444710158-8723-9-git-send-email-eblake@redhat.com>
[Moved array info tracking to a later patch]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-10-13 06:22:28 +02:00
|
|
|
# TODO As long as we have QAPI_TYPES_BUILTIN to share multiple
|
|
|
|
# qapi-types.h from a single .c, all arrays of builtins must be
|
|
|
|
# declared in the first file whether or not they are used. Nicer
|
|
|
|
# would be to use lazy instantiation, while figuring out how to
|
|
|
|
# avoid compilation issues with multiple qapi-types.h.
|
2015-10-13 06:22:32 +02:00
|
|
|
self._make_array_type(name, None)
|
2015-09-16 13:06:05 +02:00
|
|
|
|
|
|
|
def _def_predefineds(self):
|
2016-03-17 23:48:36 +01:00
|
|
|
for t in [('str', 'string', 'char' + pointer_suffix),
|
|
|
|
('number', 'number', 'double'),
|
|
|
|
('int', 'int', 'int64_t'),
|
|
|
|
('int8', 'int', 'int8_t'),
|
|
|
|
('int16', 'int', 'int16_t'),
|
|
|
|
('int32', 'int', 'int32_t'),
|
|
|
|
('int64', 'int', 'int64_t'),
|
|
|
|
('uint8', 'int', 'uint8_t'),
|
|
|
|
('uint16', 'int', 'uint16_t'),
|
|
|
|
('uint32', 'int', 'uint32_t'),
|
|
|
|
('uint64', 'int', 'uint64_t'),
|
|
|
|
('size', 'int', 'uint64_t'),
|
|
|
|
('bool', 'boolean', 'bool'),
|
2017-06-26 19:25:14 +02:00
|
|
|
('any', 'value', 'QObject' + pointer_suffix),
|
|
|
|
('null', 'null', 'QNull' + pointer_suffix)]:
|
2015-09-16 13:06:06 +02:00
|
|
|
self._def_builtin_type(*t)
|
2017-03-15 13:57:03 +01:00
|
|
|
self.the_empty_object_type = QAPISchemaObjectType(
|
|
|
|
'q_empty', None, None, None, [], None)
|
qapi: New QMP command query-qmp-schema for QMP introspection
qapi/introspect.json defines the introspection schema. It's designed
for QMP introspection, but should do for similar uses, such as QGA.
The introspection schema does not reflect all the rules and
restrictions that apply to QAPI schemata. A valid QAPI schema has an
introspection value conforming to the introspection schema, but the
converse is not true.
Introspection lowers away a number of schema details, and makes
implicit things explicit:
* The built-in types are declared with their JSON type.
All integer types are mapped to 'int', because how many bits we use
internally is an implementation detail. It could be pressed into
external interface service as very approximate range information,
but that's a bad idea. If we need range information, we better do
it properly.
* Implicit type definitions are made explicit, and given
auto-generated names:
- Array types, named by appending "List" to the name of their
element type, like in generated C.
- The enumeration types implicitly defined by simple union types,
named by appending "Kind" to the name of their simple union type,
like in generated C.
- Types that don't occur in generated C. Their names start with ':'
so they don't clash with the user's names.
* All type references are by name.
* The struct and union types are generalized into an object type.
* Base types are flattened.
* Commands take a single argument and return a single result.
Dictionary argument or list result is an implicit type definition.
The empty object type is used when a command takes no arguments or
produces no results.
The argument is always of object type, but the introspection schema
doesn't reflect that.
The 'gen': false directive is omitted as implementation detail.
The 'success-response' directive is omitted as well for now, even
though it's not an implementation detail, because it's not used by
QMP.
* Events carry a single data value.
Implicit type definition and empty object type use, just like for
commands.
The value is of object type, but the introspection schema doesn't
reflect that.
* Types not used by commands or events are omitted.
Indirect use counts as use.
* Optional members have a default, which can only be null right now
Instead of a mandatory "optional" flag, we have an optional default.
No default means mandatory, default null means optional without
default value. Non-null is available for optional with default
(possible future extension).
* Clients should *not* look up types by name, because type names are
not ABI. Look up the command or event you're interested in, then
follow the references.
TODO Should we hide the type names to eliminate the temptation?
New generator scripts/qapi-introspect.py computes an introspection
value for its input, and generates a C variable holding it.
It can generate awfully long lines. Marked TODO.
A new test-qmp-input-visitor test case feeds its result for both
tests/qapi-schema/qapi-schema-test.json and qapi-schema.json to a
QmpInputVisitor to verify it actually conforms to the schema.
New QMP command query-qmp-schema takes its return value from that
variable. Its reply is some 85KiBytes for me right now.
If this turns out to be too much, we have a couple of options:
* We can use shorter names in the JSON. Not the QMP style.
* Optionally return the sub-schema for commands and events given as
arguments.
Right now qmp_query_schema() sends the string literal computed by
qmp-introspect.py. To compute sub-schema at run time, we'd have to
duplicate parts of qapi-introspect.py in C. Unattractive.
* Let clients cache the output of query-qmp-schema.
It changes only on QEMU upgrades, i.e. rarely. Provide a command
query-qmp-schema-hash. Clients can have a cache indexed by hash,
and re-query the schema only when they don't have it cached. Even
simpler: put the hash in the QMP greeting.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2015-09-16 13:06:28 +02:00
|
|
|
self._def_entity(self.the_empty_object_type)
|
2017-06-07 18:35:58 +02:00
|
|
|
qtype_values = self._make_enum_members(['none', 'qnull', 'qnum',
|
2015-12-02 06:20:55 +01:00
|
|
|
'qstring', 'qdict', 'qlist',
|
2017-06-07 18:35:58 +02:00
|
|
|
'qbool'])
|
2017-03-15 13:57:03 +01:00
|
|
|
self._def_entity(QAPISchemaEnumType('QType', None, None,
|
|
|
|
qtype_values, 'QTYPE'))
|
2015-09-16 13:06:05 +02:00
|
|
|
|
2015-12-02 06:20:55 +01:00
|
|
|
def _make_enum_members(self, values):
|
|
|
|
return [QAPISchemaMember(v) for v in values]
|
|
|
|
|
2015-10-13 06:22:32 +02:00
|
|
|
def _make_implicit_enum_type(self, name, info, values):
|
2015-12-02 06:20:55 +01:00
|
|
|
# See also QAPISchemaObjectTypeMember._pretty_owner()
|
2015-10-13 06:22:27 +02:00
|
|
|
name = name + 'Kind' # Use namespace reserved by add_name()
|
2015-12-02 06:20:55 +01:00
|
|
|
self._def_entity(QAPISchemaEnumType(
|
2017-03-15 13:57:03 +01:00
|
|
|
name, info, None, self._make_enum_members(values), None))
|
2015-09-16 13:06:05 +02:00
|
|
|
return name
|
|
|
|
|
2015-10-13 06:22:32 +02:00
|
|
|
def _make_array_type(self, element_type, info):
|
2015-10-26 23:34:43 +01:00
|
|
|
name = element_type + 'List' # Use namespace reserved by add_name()
|
2015-09-16 13:06:05 +02:00
|
|
|
if not self.lookup_type(name):
|
2015-10-13 06:22:32 +02:00
|
|
|
self._def_entity(QAPISchemaArrayType(name, info, element_type))
|
2015-09-16 13:06:05 +02:00
|
|
|
return name
|
|
|
|
|
2017-03-15 13:57:03 +01:00
|
|
|
def _make_implicit_object_type(self, name, info, doc, role, members):
|
2015-09-16 13:06:05 +02:00
|
|
|
if not members:
|
|
|
|
return None
|
qapi: Track owner of each object member
Future commits will migrate semantic checking away from parsing
and over to the various QAPISchema*.check() methods. But to
report an error message about an incorrect semantic use of a
member of an object type, it helps to know which type, command,
or event owns the member. In particular, when a member is
inherited from a base type, it is desirable to associate the
member name with the base type (and not the type calling
member.check()).
Rather than packing additional information into the seen array
passed to each member.check() (as in seen[m.name] = {'member':m,
'owner':type}), it is easier to have each member track the name
of the owner type in the first place (keeping things simpler
with the existing seen[m.name] = m). The new member.owner field
is set via a new set_owner() method, called when registering
the members and variants arrays with an object or variant type.
Track only a name, and not the actual type object, to avoid
creating a circular python reference chain.
Note that Variants.set_owner() method does not set the owner
for the tag_member field; this field is set earlier either as
part of an object's non-variant members, or explicitly by
alternates.
The source information is intended for human consumption in
error messages, and a new describe() method is added to access
the resulting information. For example, given the qapi:
{ 'command': 'foo', 'data': { 'string': 'str' } }
an implementation of visit_command() that calls
arg_type.members[0].describe()
will see "'string' (parameter of foo)".
To make the human-readable name of implicit types work without
duplicating efforts, the describe() method has to reverse the
name of implicit types, via the helper _pretty_owner().
No change to generated code.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-16-git-send-email-eblake@redhat.com>
[Incorrect & unused -wrapper case in _pretty_owner() dropped]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-18 09:52:50 +01:00
|
|
|
# See also QAPISchemaObjectTypeMember._pretty_owner()
|
2016-03-17 23:48:29 +01:00
|
|
|
name = 'q_obj_%s-%s' % (name, role)
|
2015-09-16 13:06:05 +02:00
|
|
|
if not self.lookup_entity(name, QAPISchemaObjectType):
|
2017-03-15 13:57:03 +01:00
|
|
|
self._def_entity(QAPISchemaObjectType(name, info, doc, None,
|
2015-09-16 13:06:05 +02:00
|
|
|
members, None))
|
|
|
|
return name
|
|
|
|
|
2017-03-15 13:57:03 +01:00
|
|
|
def _def_enum_type(self, expr, info, doc):
|
2015-09-16 13:06:05 +02:00
|
|
|
name = expr['enum']
|
|
|
|
data = expr['data']
|
|
|
|
prefix = expr.get('prefix')
|
2015-12-02 06:20:55 +01:00
|
|
|
self._def_entity(QAPISchemaEnumType(
|
2017-03-15 13:57:03 +01:00
|
|
|
name, info, doc, self._make_enum_members(data), prefix))
|
2015-09-16 13:06:05 +02:00
|
|
|
|
2015-10-13 06:22:32 +02:00
|
|
|
def _make_member(self, name, typ, info):
|
2015-09-16 13:06:05 +02:00
|
|
|
optional = False
|
|
|
|
if name.startswith('*'):
|
|
|
|
name = name[1:]
|
|
|
|
optional = True
|
|
|
|
if isinstance(typ, list):
|
|
|
|
assert len(typ) == 1
|
2015-10-13 06:22:32 +02:00
|
|
|
typ = self._make_array_type(typ[0], info)
|
2015-09-16 13:06:05 +02:00
|
|
|
return QAPISchemaObjectTypeMember(name, typ, optional)
|
|
|
|
|
2015-10-13 06:22:32 +02:00
|
|
|
def _make_members(self, data, info):
|
|
|
|
return [self._make_member(key, value, info)
|
2015-09-16 13:06:05 +02:00
|
|
|
for (key, value) in data.iteritems()]
|
|
|
|
|
2017-03-15 13:57:03 +01:00
|
|
|
def _def_struct_type(self, expr, info, doc):
|
2015-09-16 13:06:05 +02:00
|
|
|
name = expr['struct']
|
|
|
|
base = expr.get('base')
|
|
|
|
data = expr['data']
|
2017-03-15 13:57:03 +01:00
|
|
|
self._def_entity(QAPISchemaObjectType(name, info, doc, base,
|
2015-10-13 06:22:32 +02:00
|
|
|
self._make_members(data, info),
|
2015-09-16 13:06:05 +02:00
|
|
|
None))
|
|
|
|
|
|
|
|
def _make_variant(self, case, typ):
|
|
|
|
return QAPISchemaObjectTypeVariant(case, typ)
|
|
|
|
|
2015-10-13 06:22:32 +02:00
|
|
|
def _make_simple_variant(self, case, typ, info):
|
2015-09-16 13:06:05 +02:00
|
|
|
if isinstance(typ, list):
|
|
|
|
assert len(typ) == 1
|
2015-10-13 06:22:32 +02:00
|
|
|
typ = self._make_array_type(typ[0], info)
|
|
|
|
typ = self._make_implicit_object_type(
|
2017-03-15 13:57:03 +01:00
|
|
|
typ, info, None, 'wrapper', [self._make_member('data', typ, info)])
|
2015-09-16 13:06:05 +02:00
|
|
|
return QAPISchemaObjectTypeVariant(case, typ)
|
|
|
|
|
2017-03-15 13:57:03 +01:00
|
|
|
def _def_union_type(self, expr, info, doc):
|
2015-09-16 13:06:05 +02:00
|
|
|
name = expr['union']
|
|
|
|
data = expr['data']
|
|
|
|
base = expr.get('base')
|
|
|
|
tag_name = expr.get('discriminator')
|
2015-10-13 06:22:29 +02:00
|
|
|
tag_member = None
|
2016-03-17 23:48:39 +01:00
|
|
|
if isinstance(base, dict):
|
|
|
|
base = (self._make_implicit_object_type(
|
2017-03-15 13:57:35 +01:00
|
|
|
name, info, doc, 'base', self._make_members(base, info)))
|
2015-09-16 13:06:05 +02:00
|
|
|
if tag_name:
|
|
|
|
variants = [self._make_variant(key, value)
|
|
|
|
for (key, value) in data.iteritems()]
|
qapi: Track simple union tag in object.local_members
We were previously creating all unions with an empty list for
local_members. However, it will make it easier to unify struct
and union generation if we include the generated tag member in
local_members. That way, we can have a common code pattern:
visit the base (if any), visit the local members (if any), visit
the variants (if any). The local_members of a flat union
remains empty (because the discriminator is already visited as
part of the base). Then, by visiting tag_member.check() during
AlternateType.check(), we no longer need to call it during
Variants.check().
The various front end entities now exist as follows:
struct: optional base, optional local_members, no variants
simple union: no base, one-element local_members, variants with tag_member
from local_members
flat union: base, no local_members, variants with tag_member from base
alternate: no base, no local_members, variants
With the new local members, we require a bit of finesse to
avoid assertions in the clients.
No change to generated code.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-2-git-send-email-eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-18 09:52:36 +01:00
|
|
|
members = []
|
2015-09-16 13:06:05 +02:00
|
|
|
else:
|
2015-10-13 06:22:32 +02:00
|
|
|
variants = [self._make_simple_variant(key, value, info)
|
2015-09-16 13:06:05 +02:00
|
|
|
for (key, value) in data.iteritems()]
|
2015-12-02 06:20:50 +01:00
|
|
|
typ = self._make_implicit_enum_type(name, info,
|
|
|
|
[v.name for v in variants])
|
|
|
|
tag_member = QAPISchemaObjectTypeMember('type', typ, False)
|
qapi: Track simple union tag in object.local_members
We were previously creating all unions with an empty list for
local_members. However, it will make it easier to unify struct
and union generation if we include the generated tag member in
local_members. That way, we can have a common code pattern:
visit the base (if any), visit the local members (if any), visit
the variants (if any). The local_members of a flat union
remains empty (because the discriminator is already visited as
part of the base). Then, by visiting tag_member.check() during
AlternateType.check(), we no longer need to call it during
Variants.check().
The various front end entities now exist as follows:
struct: optional base, optional local_members, no variants
simple union: no base, one-element local_members, variants with tag_member
from local_members
flat union: base, no local_members, variants with tag_member from base
alternate: no base, no local_members, variants
With the new local members, we require a bit of finesse to
avoid assertions in the clients.
No change to generated code.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-2-git-send-email-eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-18 09:52:36 +01:00
|
|
|
members = [tag_member]
|
2015-09-16 13:06:05 +02:00
|
|
|
self._def_entity(
|
2017-03-15 13:57:03 +01:00
|
|
|
QAPISchemaObjectType(name, info, doc, base, members,
|
2015-09-16 13:06:05 +02:00
|
|
|
QAPISchemaObjectTypeVariants(tag_name,
|
2015-10-13 06:22:29 +02:00
|
|
|
tag_member,
|
2015-09-16 13:06:05 +02:00
|
|
|
variants)))
|
|
|
|
|
2017-03-15 13:57:03 +01:00
|
|
|
def _def_alternate_type(self, expr, info, doc):
|
2015-09-16 13:06:05 +02:00
|
|
|
name = expr['alternate']
|
|
|
|
data = expr['data']
|
|
|
|
variants = [self._make_variant(key, value)
|
|
|
|
for (key, value) in data.iteritems()]
|
qapi: Simplify visiting of alternate types
Previously, working with alternates required two lookup arrays
and some indirection: for type Foo, we created Foo_qtypes[]
which maps each qtype to a value of the generated FooKind enum,
then look up that value in FooKind_lookup[] like we do for other
union types.
This has a couple of subtle bugs. First, the generator was
creating a call with a parameter '(int *) &(*obj)->type' where
type is an enum type; this is unsafe if the compiler chooses
to store the enum type in a different size than int, where
assigning through the wrong size pointer can corrupt data or
cause a SIGBUS.
Related bug, not not fixed in this patch: qapi-visit.py's
gen_visit_enum() generates a cast of its enum * argument to
int *. Marked FIXME.
Second, since the values of the FooKind enum start at zero, all
entries of the Foo_qtypes[] array that were not explicitly
initialized will map to the same branch of the union as the
first member of the alternate, rather than triggering a desired
failure in visit_get_next_type(). Fortunately, the bug seldom
bites; the very next thing the input visitor does is try to
parse the incoming JSON with the wrong parser, which normally
fails; the output visitor is not used with a C struct in that
state, and the dealloc visitor has nothing to clean up (so
there is no leak).
However, the second bug IS observable in one case: parsing an
integer causes unusual behavior in an alternate that contains
at least a 'number' member but no 'int' member, because the
'number' parser accepts QTYPE_QINT in addition to the expected
QTYPE_QFLOAT (that is, since 'int' is not a member, the type
QTYPE_QINT accidentally maps to FooKind 0; if this enum value
is the 'number' branch the integer parses successfully, but if
the 'number' branch is not first, some other branch tries to
parse the integer and rejects it). A later patch will worry
about fixing alternates to always parse all inputs that a
non-alternate 'number' would accept, for now this is still
marked FIXME in the updated test-qmp-input-visitor.c, to
merely point out that new undesired behavior of 'ans' matches
the existing undesired behavior of 'asn'.
This patch fixes the default-initialization bug by deleting the
indirection, and modifying get_next_type() to directly assign a
QTypeCode parameter. This in turn fixes the type-casting bug,
as we are no longer casting a pointer to enum to a questionable
size. There is no longer a need to generate an implicit FooKind
enum associated with the alternate type (since the QMP wire
format never uses the stringized counterparts of the C union
member names). Since the updated visit_get_next_type() does not
know which qtypes are expected, the generated visitor is
modified to generate an error statement if an unexpected type is
encountered.
Callers now have to know the QTYPE_* mapping when looking at the
discriminator; but so far, only the testsuite was even using the
C struct of an alternate types. I considered the possibility of
keeping the internal enum FooKind, but initialized differently
than most generated arrays, as in:
typedef enum FooKind {
FOO_KIND_A = QTYPE_QDICT,
FOO_KIND_B = QTYPE_QINT,
} FooKind;
to create nicer aliases for knowing when to use foo->a or foo->b
when inspecting foo->type; but it turned out to add too much
complexity, especially without a client.
There is a user-visible side effect to this change, but I
consider it to be an improvement. Previously,
the invalid QMP command:
{"execute":"blockdev-add", "arguments":{"options":
{"driver":"raw", "id":"a", "file":true}}}
failed with:
{"error": {"class": "GenericError",
"desc": "Invalid parameter type for 'file', expected: QDict"}}
(visit_get_next_type() succeeded, and the error comes from the
visit_type_BlockdevOptions() expecting {}; there is no mention of
the fact that a string would also work). Now it fails with:
{"error": {"class": "GenericError",
"desc": "Invalid parameter type for 'file', expected: BlockdevRef"}}
(the error when the next type doesn't match any expected types for
the overall alternate).
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1449033659-25497-5-git-send-email-eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-12-02 06:20:48 +01:00
|
|
|
tag_member = QAPISchemaObjectTypeMember('type', 'QType', False)
|
2015-09-16 13:06:05 +02:00
|
|
|
self._def_entity(
|
2017-03-15 13:57:03 +01:00
|
|
|
QAPISchemaAlternateType(name, info, doc,
|
2015-09-16 13:06:05 +02:00
|
|
|
QAPISchemaObjectTypeVariants(None,
|
2015-10-13 06:22:29 +02:00
|
|
|
tag_member,
|
2015-09-16 13:06:05 +02:00
|
|
|
variants)))
|
|
|
|
|
2017-03-15 13:57:03 +01:00
|
|
|
def _def_command(self, expr, info, doc):
|
2015-09-16 13:06:05 +02:00
|
|
|
name = expr['command']
|
|
|
|
data = expr.get('data')
|
|
|
|
rets = expr.get('returns')
|
|
|
|
gen = expr.get('gen', True)
|
|
|
|
success_response = expr.get('success-response', True)
|
2016-07-14 05:50:19 +02:00
|
|
|
boxed = expr.get('boxed', False)
|
2015-09-16 13:06:05 +02:00
|
|
|
if isinstance(data, OrderedDict):
|
2015-10-13 06:22:32 +02:00
|
|
|
data = self._make_implicit_object_type(
|
2017-03-15 13:57:03 +01:00
|
|
|
name, info, doc, 'arg', self._make_members(data, info))
|
2015-09-16 13:06:05 +02:00
|
|
|
if isinstance(rets, list):
|
|
|
|
assert len(rets) == 1
|
2015-10-13 06:22:32 +02:00
|
|
|
rets = self._make_array_type(rets[0], info)
|
2017-03-15 13:57:03 +01:00
|
|
|
self._def_entity(QAPISchemaCommand(name, info, doc, data, rets,
|
|
|
|
gen, success_response, boxed))
|
2015-09-16 13:06:05 +02:00
|
|
|
|
2017-03-15 13:57:03 +01:00
|
|
|
def _def_event(self, expr, info, doc):
|
2015-09-16 13:06:05 +02:00
|
|
|
name = expr['event']
|
|
|
|
data = expr.get('data')
|
2016-07-14 05:50:19 +02:00
|
|
|
boxed = expr.get('boxed', False)
|
2015-09-16 13:06:05 +02:00
|
|
|
if isinstance(data, OrderedDict):
|
2015-10-13 06:22:32 +02:00
|
|
|
data = self._make_implicit_object_type(
|
2017-03-15 13:57:03 +01:00
|
|
|
name, info, doc, 'arg', self._make_members(data, info))
|
|
|
|
self._def_entity(QAPISchemaEvent(name, info, doc, data, boxed))
|
2015-09-16 13:06:05 +02:00
|
|
|
|
|
|
|
def _def_exprs(self):
|
|
|
|
for expr_elem in self.exprs:
|
|
|
|
expr = expr_elem['expr']
|
|
|
|
info = expr_elem['info']
|
2017-03-15 13:57:03 +01:00
|
|
|
doc = expr_elem.get('doc')
|
2015-09-16 13:06:05 +02:00
|
|
|
if 'enum' in expr:
|
2017-03-15 13:57:03 +01:00
|
|
|
self._def_enum_type(expr, info, doc)
|
2015-09-16 13:06:05 +02:00
|
|
|
elif 'struct' in expr:
|
2017-03-15 13:57:03 +01:00
|
|
|
self._def_struct_type(expr, info, doc)
|
2015-09-16 13:06:05 +02:00
|
|
|
elif 'union' in expr:
|
2017-03-15 13:57:03 +01:00
|
|
|
self._def_union_type(expr, info, doc)
|
2015-09-16 13:06:05 +02:00
|
|
|
elif 'alternate' in expr:
|
2017-03-15 13:57:03 +01:00
|
|
|
self._def_alternate_type(expr, info, doc)
|
2015-09-16 13:06:05 +02:00
|
|
|
elif 'command' in expr:
|
2017-03-15 13:57:03 +01:00
|
|
|
self._def_command(expr, info, doc)
|
2015-09-16 13:06:05 +02:00
|
|
|
elif 'event' in expr:
|
2017-03-15 13:57:03 +01:00
|
|
|
self._def_event(expr, info, doc)
|
2015-09-16 13:06:05 +02:00
|
|
|
else:
|
|
|
|
assert False
|
|
|
|
|
|
|
|
def check(self):
|
|
|
|
for ent in self._entity_dict.values():
|
|
|
|
ent.check(self)
|
2015-06-10 08:55:21 +02:00
|
|
|
|
2015-09-16 13:06:07 +02:00
|
|
|
def visit(self, visitor):
|
2015-10-13 06:22:21 +02:00
|
|
|
visitor.visit_begin(self)
|
|
|
|
for (name, entity) in sorted(self._entity_dict.items()):
|
|
|
|
if visitor.visit_needed(entity):
|
|
|
|
entity.visit(visitor)
|
2015-09-16 13:06:07 +02:00
|
|
|
visitor.visit_end()
|
|
|
|
|
2014-03-05 03:44:34 +01:00
|
|
|
|
2015-06-10 10:04:36 +02:00
|
|
|
#
|
|
|
|
# Code generation helpers
|
|
|
|
#
|
|
|
|
|
2011-07-19 21:50:39 +02:00
|
|
|
def camel_case(name):
|
|
|
|
new_name = ''
|
|
|
|
first = True
|
|
|
|
for ch in name:
|
|
|
|
if ch in ['_', '-']:
|
|
|
|
first = True
|
|
|
|
elif first:
|
|
|
|
new_name += ch.upper()
|
|
|
|
first = False
|
|
|
|
else:
|
|
|
|
new_name += ch.lower()
|
|
|
|
return new_name
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2015-05-14 14:50:53 +02:00
|
|
|
# ENUMName -> ENUM_NAME, EnumName1 -> ENUM_NAME1
|
|
|
|
# ENUM_NAME -> ENUM_NAME, ENUM_NAME1 -> ENUM_NAME1, ENUM_Name2 -> ENUM_NAME2
|
|
|
|
# ENUM24_Name -> ENUM24_NAME
|
|
|
|
def camel_to_upper(value):
|
|
|
|
c_fun_str = c_name(value, False)
|
|
|
|
if value.isupper():
|
|
|
|
return c_fun_str
|
|
|
|
|
|
|
|
new_name = ''
|
|
|
|
l = len(c_fun_str)
|
|
|
|
for i in range(l):
|
|
|
|
c = c_fun_str[i]
|
2017-03-15 13:57:08 +01:00
|
|
|
# When c is upper and no '_' appears before, do more checks
|
|
|
|
if c.isupper() and (i > 0) and c_fun_str[i - 1] != '_':
|
2015-09-30 00:21:02 +02:00
|
|
|
if i < l - 1 and c_fun_str[i + 1].islower():
|
|
|
|
new_name += '_'
|
|
|
|
elif c_fun_str[i - 1].isdigit():
|
2015-05-14 14:50:53 +02:00
|
|
|
new_name += '_'
|
|
|
|
new_name += c
|
|
|
|
return new_name.lstrip('_').upper()
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2015-08-26 15:21:20 +02:00
|
|
|
def c_enum_const(type_name, const_name, prefix=None):
|
|
|
|
if prefix is not None:
|
|
|
|
type_name = prefix
|
qapi: Change munging of CamelCase enum values
When munging enum values, the fact that we were passing the entire
prefix + value through camel_to_upper() meant that enum values
spelled with CamelCase could be turned into CAMEL_CASE. However,
this provides a potential collision (both OneTwo and One-Two would
munge into ONE_TWO) for enum types, when the same two names are
valid side-by-side as QAPI member names. By changing the generation
of enum constants to always be prefix + '_' + c_name(value,
False).upper(), and ensuring that there are no case collisions (in
the next patches), we no longer have to worry about names that
would be distinct as QAPI members but collide as variant tag names,
without having to think about what munging the heuristics in
camel_to_upper() will actually perform on an enum value.
Making the change will affect enums that did not follow coding
conventions, using 'CamelCase' rather than desired 'lower-case'.
Thankfully, there are only two culprits: InputButton and ErrorClass.
We already tweaked ErrorClass to make it an alias of QapiErrorClass,
where only the alias needs changing rather than the whole tree. So
the bulk of this change is modifying INPUT_BUTTON_WHEEL_UP to the
new INPUT_BUTTON_WHEELUP (and likewise for WHEELDOWN). That part
of this commit may later need reverting if we rename the enum
constants from 'WheelUp' to 'wheel-up' as part of moving
x-input-send-event to a stable interface; but at least we have
documentation bread crumbs in place to remind us (commit 513e7cd),
and it matches the fact that SDL constants are also spelled
SDL_BUTTON_WHEELUP.
Suggested by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-27-git-send-email-eblake@redhat.com>
[Commit message tweaked]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-18 09:53:01 +01:00
|
|
|
return camel_to_upper(type_name) + '_' + c_name(const_name, False).upper()
|
2015-05-14 14:50:53 +02:00
|
|
|
|
2015-05-14 14:50:48 +02:00
|
|
|
c_name_trans = string.maketrans('.-', '__')
|
2015-05-14 14:50:47 +02:00
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
qapi: Make c_type() consistently convert qapi names
Continuing the string of cleanups for supporting downstream names
containing '.', this patch focuses on ensuring c_type() can
handle a downstream name. This patch alone does not fix the
places where generator output should be calling this function
but was open-coding things instead, but it gets us a step closer.
In particular, the changes to c_list_type() and type_name() mean
that type_name(FOO) now handles the case when FOO contains '.',
'-', or is a ticklish identifier other than a builtin (builtins
are exempted because ['int'] must remain mapped to 'intList' and
not 'q_intList'). Meanwhile, ['unix'] now maps to 'q_unixList'
rather than 'unixList', to match the fact that 'unix' is ticklish;
however, our naming conventions state that complex types should
start with a capital, so no type name following conventions will
ever have the 'q_' prepended.
Likewise, changes to c_type() mean that c_type(FOO) properly
handles an enum or complex type FOO with '.' or '-' in the
name, or is a ticklish identifier (again, a ticklish identifier
as a type name violates conventions).
Signed-off-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-05-14 14:50:55 +02:00
|
|
|
# Map @name to a valid C identifier.
|
|
|
|
# If @protect, avoid returning certain ticklish identifiers (like
|
2017-03-15 13:57:08 +01:00
|
|
|
# C keywords) by prepending 'q_'.
|
qapi: Make c_type() consistently convert qapi names
Continuing the string of cleanups for supporting downstream names
containing '.', this patch focuses on ensuring c_type() can
handle a downstream name. This patch alone does not fix the
places where generator output should be calling this function
but was open-coding things instead, but it gets us a step closer.
In particular, the changes to c_list_type() and type_name() mean
that type_name(FOO) now handles the case when FOO contains '.',
'-', or is a ticklish identifier other than a builtin (builtins
are exempted because ['int'] must remain mapped to 'intList' and
not 'q_intList'). Meanwhile, ['unix'] now maps to 'q_unixList'
rather than 'unixList', to match the fact that 'unix' is ticklish;
however, our naming conventions state that complex types should
start with a capital, so no type name following conventions will
ever have the 'q_' prepended.
Likewise, changes to c_type() mean that c_type(FOO) properly
handles an enum or complex type FOO with '.' or '-' in the
name, or is a ticklish identifier (again, a ticklish identifier
as a type name violates conventions).
Signed-off-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-05-14 14:50:55 +02:00
|
|
|
#
|
|
|
|
# Used for converting 'name' from a 'name':'type' qapi definition
|
|
|
|
# into a generated struct member, as well as converting type names
|
|
|
|
# into substrings of a generated C function name.
|
|
|
|
# '__a.b_c' -> '__a_b_c', 'x-foo' -> 'x_foo'
|
|
|
|
# protect=True: 'int' -> 'q_int'; protect=False: 'int' -> 'int'
|
2015-05-14 14:50:48 +02:00
|
|
|
def c_name(name, protect=True):
|
2012-07-30 17:46:55 +02:00
|
|
|
# ANSI X3J11/88-090, 3.1.1
|
|
|
|
c89_words = set(['auto', 'break', 'case', 'char', 'const', 'continue',
|
2015-09-30 00:21:02 +02:00
|
|
|
'default', 'do', 'double', 'else', 'enum', 'extern',
|
|
|
|
'float', 'for', 'goto', 'if', 'int', 'long', 'register',
|
|
|
|
'return', 'short', 'signed', 'sizeof', 'static',
|
|
|
|
'struct', 'switch', 'typedef', 'union', 'unsigned',
|
|
|
|
'void', 'volatile', 'while'])
|
2012-07-30 17:46:55 +02:00
|
|
|
# ISO/IEC 9899:1999, 6.4.1
|
|
|
|
c99_words = set(['inline', 'restrict', '_Bool', '_Complex', '_Imaginary'])
|
|
|
|
# ISO/IEC 9899:2011, 6.4.1
|
2015-09-30 00:21:02 +02:00
|
|
|
c11_words = set(['_Alignas', '_Alignof', '_Atomic', '_Generic',
|
|
|
|
'_Noreturn', '_Static_assert', '_Thread_local'])
|
2012-07-30 17:46:55 +02:00
|
|
|
# GCC http://gcc.gnu.org/onlinedocs/gcc-4.7.1/gcc/C-Extensions.html
|
|
|
|
# excluding _.*
|
|
|
|
gcc_words = set(['asm', 'typeof'])
|
2013-08-07 17:39:43 +02:00
|
|
|
# C++ ISO/IEC 14882:2003 2.11
|
|
|
|
cpp_words = set(['bool', 'catch', 'class', 'const_cast', 'delete',
|
|
|
|
'dynamic_cast', 'explicit', 'false', 'friend', 'mutable',
|
|
|
|
'namespace', 'new', 'operator', 'private', 'protected',
|
|
|
|
'public', 'reinterpret_cast', 'static_cast', 'template',
|
|
|
|
'this', 'throw', 'true', 'try', 'typeid', 'typename',
|
|
|
|
'using', 'virtual', 'wchar_t',
|
|
|
|
# alternative representations
|
|
|
|
'and', 'and_eq', 'bitand', 'bitor', 'compl', 'not',
|
|
|
|
'not_eq', 'or', 'or_eq', 'xor', 'xor_eq'])
|
2012-09-19 16:31:07 +02:00
|
|
|
# namespace pollution:
|
2016-02-02 15:51:41 +01:00
|
|
|
polluted_words = set(['unix', 'errno', 'mips', 'sparc'])
|
2015-11-18 09:52:52 +01:00
|
|
|
name = name.translate(c_name_trans)
|
2015-09-30 00:21:02 +02:00
|
|
|
if protect and (name in c89_words | c99_words | c11_words | gcc_words
|
|
|
|
| cpp_words | polluted_words):
|
2017-03-15 13:57:08 +01:00
|
|
|
return 'q_' + name
|
2015-11-18 09:52:52 +01:00
|
|
|
return name
|
2011-07-19 21:50:39 +02:00
|
|
|
|
2014-06-10 13:25:53 +02:00
|
|
|
eatspace = '\033EATSPACE.'
|
2015-05-14 14:50:54 +02:00
|
|
|
pointer_suffix = ' *' + eatspace
|
2014-06-10 13:25:53 +02:00
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2011-07-19 21:50:39 +02:00
|
|
|
def genindent(count):
|
2017-03-15 13:57:08 +01:00
|
|
|
ret = ''
|
2015-09-30 00:21:02 +02:00
|
|
|
for _ in range(count):
|
2017-03-15 13:57:08 +01:00
|
|
|
ret += ' '
|
2011-07-19 21:50:39 +02:00
|
|
|
return ret
|
|
|
|
|
|
|
|
indent_level = 0
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2011-07-19 21:50:39 +02:00
|
|
|
def push_indent(indent_amount=4):
|
|
|
|
global indent_level
|
|
|
|
indent_level += indent_amount
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2011-07-19 21:50:39 +02:00
|
|
|
def pop_indent(indent_amount=4):
|
|
|
|
global indent_level
|
|
|
|
indent_level -= indent_amount
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2015-06-24 19:27:32 +02:00
|
|
|
# Generate @code with @kwds interpolated.
|
|
|
|
# Obey indent_level, and strip eatspace.
|
2011-07-19 21:50:39 +02:00
|
|
|
def cgen(code, **kwds):
|
2015-06-24 19:27:32 +02:00
|
|
|
raw = code % kwds
|
|
|
|
if indent_level:
|
|
|
|
indent = genindent(indent_level)
|
2015-09-07 17:45:55 +02:00
|
|
|
# re.subn() lacks flags support before Python 2.7, use re.compile()
|
2017-03-15 13:57:07 +01:00
|
|
|
raw = re.subn(re.compile(r'^.', re.MULTILINE),
|
2015-09-07 17:45:55 +02:00
|
|
|
indent + r'\g<0>', raw)
|
2015-06-24 19:27:32 +02:00
|
|
|
raw = raw[0]
|
2017-03-15 13:57:07 +01:00
|
|
|
return re.sub(re.escape(eatspace) + r' *', '', raw)
|
2011-07-19 21:50:39 +02:00
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2011-07-19 21:50:39 +02:00
|
|
|
def mcgen(code, **kwds):
|
2015-06-24 19:27:32 +02:00
|
|
|
if code[0] == '\n':
|
|
|
|
code = code[1:]
|
|
|
|
return cgen(code, **kwds)
|
2011-07-19 21:50:39 +02:00
|
|
|
|
|
|
|
|
|
|
|
def guardname(filename):
|
2015-06-27 07:27:21 +02:00
|
|
|
return c_name(filename, protect=False).upper()
|
2013-05-11 00:46:00 +02:00
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2013-05-11 00:46:00 +02:00
|
|
|
def guardstart(name):
|
|
|
|
return mcgen('''
|
|
|
|
|
|
|
|
#ifndef %(name)s
|
|
|
|
#define %(name)s
|
|
|
|
|
|
|
|
''',
|
|
|
|
name=guardname(name))
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2013-05-11 00:46:00 +02:00
|
|
|
def guardend(name):
|
|
|
|
return mcgen('''
|
|
|
|
|
|
|
|
#endif /* %(name)s */
|
|
|
|
|
|
|
|
''',
|
|
|
|
name=guardname(name))
|
2015-04-02 13:12:21 +02:00
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2015-09-16 13:06:16 +02:00
|
|
|
def gen_enum_lookup(name, values, prefix=None):
|
2015-09-16 13:06:12 +02:00
|
|
|
ret = mcgen('''
|
|
|
|
|
2017-08-24 10:46:10 +02:00
|
|
|
const QEnumLookup %(c_name)s_lookup = {
|
|
|
|
.array = (const char *const[]) {
|
2015-09-16 13:06:12 +02:00
|
|
|
''',
|
2015-09-16 13:06:16 +02:00
|
|
|
c_name=c_name(name))
|
2015-09-16 13:06:12 +02:00
|
|
|
for value in values:
|
|
|
|
index = c_enum_const(name, value, prefix)
|
|
|
|
ret += mcgen('''
|
2017-08-24 10:46:10 +02:00
|
|
|
[%(index)s] = "%(value)s",
|
2015-09-16 13:06:12 +02:00
|
|
|
''',
|
2015-09-16 13:06:16 +02:00
|
|
|
index=index, value=value)
|
2015-09-16 13:06:12 +02:00
|
|
|
|
|
|
|
ret += mcgen('''
|
2017-08-24 10:46:10 +02:00
|
|
|
},
|
|
|
|
.size = %(max_index)s
|
2015-09-16 13:06:12 +02:00
|
|
|
};
|
|
|
|
''',
|
2017-08-24 10:46:11 +02:00
|
|
|
max_index=c_enum_const(name, '_MAX', prefix))
|
2015-09-16 13:06:12 +02:00
|
|
|
return ret
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2015-09-16 13:06:16 +02:00
|
|
|
def gen_enum(name, values, prefix=None):
|
|
|
|
# append automatically generated _MAX value
|
qapi: Don't let implicit enum MAX member collide
Now that we guarantee the user doesn't have any enum values
beginning with a single underscore, we can use that for our
own purposes. Renaming ENUM_MAX to ENUM__MAX makes it obvious
that the sentinel is generated.
This patch was mostly generated by applying a temporary patch:
|diff --git a/scripts/qapi.py b/scripts/qapi.py
|index e6d014b..b862ec9 100644
|--- a/scripts/qapi.py
|+++ b/scripts/qapi.py
|@@ -1570,6 +1570,7 @@ const char *const %(c_name)s_lookup[] = {
| max_index = c_enum_const(name, 'MAX', prefix)
| ret += mcgen('''
| [%(max_index)s] = NULL,
|+// %(max_index)s
| };
| ''',
| max_index=max_index)
then running:
$ cat qapi-{types,event}.c tests/test-qapi-types.c |
sed -n 's,^// \(.*\)MAX,s|\1MAX|\1_MAX|g,p' > list
$ git grep -l _MAX | xargs sed -i -f list
The only things not generated are the changes in scripts/qapi.py.
Rejecting enum members named 'MAX' is now useless, and will be dropped
in the next patch.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-23-git-send-email-eblake@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
[Rebased to current master, commit message tweaked]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-18 09:52:57 +01:00
|
|
|
enum_values = values + ['_MAX']
|
2015-09-16 13:06:12 +02:00
|
|
|
|
2015-09-16 13:06:16 +02:00
|
|
|
ret = mcgen('''
|
2015-09-16 13:06:12 +02:00
|
|
|
|
2015-09-16 13:06:16 +02:00
|
|
|
typedef enum %(c_name)s {
|
2015-09-16 13:06:12 +02:00
|
|
|
''',
|
2015-09-16 13:06:16 +02:00
|
|
|
c_name=c_name(name))
|
2015-09-16 13:06:12 +02:00
|
|
|
|
|
|
|
i = 0
|
|
|
|
for value in enum_values:
|
2015-09-16 13:06:16 +02:00
|
|
|
ret += mcgen('''
|
|
|
|
%(c_enum)s = %(i)d,
|
2015-09-16 13:06:12 +02:00
|
|
|
''',
|
2015-09-16 13:06:16 +02:00
|
|
|
c_enum=c_enum_const(name, value, prefix),
|
2015-09-16 13:06:12 +02:00
|
|
|
i=i)
|
|
|
|
i += 1
|
|
|
|
|
2015-09-16 13:06:16 +02:00
|
|
|
ret += mcgen('''
|
|
|
|
} %(c_name)s;
|
2015-09-16 13:06:12 +02:00
|
|
|
''',
|
2015-09-16 13:06:16 +02:00
|
|
|
c_name=c_name(name))
|
|
|
|
|
|
|
|
ret += mcgen('''
|
2015-09-16 13:06:12 +02:00
|
|
|
|
2017-08-24 10:46:07 +02:00
|
|
|
#define %(c_name)s_str(val) \\
|
2017-08-24 10:46:10 +02:00
|
|
|
qapi_enum_lookup(&%(c_name)s_lookup, (val))
|
2017-08-24 10:46:07 +02:00
|
|
|
|
2017-08-24 10:46:10 +02:00
|
|
|
extern const QEnumLookup %(c_name)s_lookup;
|
2015-09-16 13:06:16 +02:00
|
|
|
''',
|
|
|
|
c_name=c_name(name))
|
|
|
|
return ret
|
2015-09-16 13:06:12 +02:00
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2017-06-01 14:41:41 +02:00
|
|
|
def build_params(arg_type, boxed, extra):
|
2015-09-16 13:06:20 +02:00
|
|
|
if not arg_type:
|
2016-07-14 05:50:20 +02:00
|
|
|
assert not boxed
|
2015-09-16 13:06:20 +02:00
|
|
|
return extra
|
|
|
|
ret = ''
|
|
|
|
sep = ''
|
2016-07-14 05:50:19 +02:00
|
|
|
if boxed:
|
2016-07-14 05:50:20 +02:00
|
|
|
ret += '%s arg' % arg_type.c_param_type()
|
|
|
|
sep = ', '
|
2016-07-14 05:50:19 +02:00
|
|
|
else:
|
|
|
|
assert not arg_type.variants
|
|
|
|
for memb in arg_type.members:
|
|
|
|
ret += sep
|
|
|
|
sep = ', '
|
|
|
|
if memb.optional:
|
|
|
|
ret += 'bool has_%s, ' % c_name(memb.name)
|
|
|
|
ret += '%s %s' % (memb.type.c_param_type(),
|
|
|
|
c_name(memb.name))
|
2015-09-16 13:06:20 +02:00
|
|
|
if extra:
|
|
|
|
ret += sep + extra
|
|
|
|
return ret
|
|
|
|
|
2015-09-30 00:21:13 +02:00
|
|
|
|
2015-06-10 10:04:36 +02:00
|
|
|
#
|
|
|
|
# Common command line parsing
|
|
|
|
#
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2017-03-15 13:57:08 +01:00
|
|
|
def parse_command_line(extra_options='', extra_long_options=[]):
|
2015-04-02 13:12:21 +02:00
|
|
|
|
|
|
|
try:
|
|
|
|
opts, args = getopt.gnu_getopt(sys.argv[1:],
|
2017-03-15 13:57:08 +01:00
|
|
|
'chp:o:' + extra_options,
|
|
|
|
['source', 'header', 'prefix=',
|
|
|
|
'output-dir='] + extra_long_options)
|
2015-12-18 08:52:41 +01:00
|
|
|
except getopt.GetoptError as err:
|
2015-04-02 13:17:34 +02:00
|
|
|
print >>sys.stderr, "%s: %s" % (sys.argv[0], str(err))
|
2015-04-02 13:12:21 +02:00
|
|
|
sys.exit(1)
|
|
|
|
|
2017-03-15 13:57:08 +01:00
|
|
|
output_dir = ''
|
|
|
|
prefix = ''
|
2015-04-02 13:12:21 +02:00
|
|
|
do_c = False
|
|
|
|
do_h = False
|
|
|
|
extra_opts = []
|
|
|
|
|
|
|
|
for oa in opts:
|
|
|
|
o, a = oa
|
2017-03-15 13:57:08 +01:00
|
|
|
if o in ('-p', '--prefix'):
|
2017-03-15 13:57:07 +01:00
|
|
|
match = re.match(r'([A-Za-z_.-][A-Za-z0-9_.-]*)?', a)
|
2015-07-01 13:13:54 +02:00
|
|
|
if match.end() != len(a):
|
|
|
|
print >>sys.stderr, \
|
|
|
|
"%s: 'funny character '%s' in argument of --prefix" \
|
|
|
|
% (sys.argv[0], a[match.end()])
|
|
|
|
sys.exit(1)
|
2015-04-02 13:12:21 +02:00
|
|
|
prefix = a
|
2017-03-15 13:57:08 +01:00
|
|
|
elif o in ('-o', '--output-dir'):
|
|
|
|
output_dir = a + '/'
|
|
|
|
elif o in ('-c', '--source'):
|
2015-04-02 13:12:21 +02:00
|
|
|
do_c = True
|
2017-03-15 13:57:08 +01:00
|
|
|
elif o in ('-h', '--header'):
|
2015-04-02 13:12:21 +02:00
|
|
|
do_h = True
|
|
|
|
else:
|
|
|
|
extra_opts.append(oa)
|
|
|
|
|
|
|
|
if not do_c and not do_h:
|
|
|
|
do_c = True
|
|
|
|
do_h = True
|
|
|
|
|
2015-04-02 13:32:16 +02:00
|
|
|
if len(args) != 1:
|
|
|
|
print >>sys.stderr, "%s: need exactly one argument" % sys.argv[0]
|
2015-04-02 13:17:34 +02:00
|
|
|
sys.exit(1)
|
2015-06-09 16:22:45 +02:00
|
|
|
fname = args[0]
|
2015-04-02 13:17:34 +02:00
|
|
|
|
2015-06-09 16:22:45 +02:00
|
|
|
return (fname, output_dir, do_c, do_h, prefix, extra_opts)
|
2015-04-02 14:46:39 +02:00
|
|
|
|
2015-06-10 10:04:36 +02:00
|
|
|
#
|
|
|
|
# Generate output files with boilerplate
|
|
|
|
#
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2015-04-02 14:46:39 +02:00
|
|
|
def open_output(output_dir, do_c, do_h, prefix, c_file, h_file,
|
|
|
|
c_comment, h_comment):
|
2015-06-27 07:27:21 +02:00
|
|
|
guard = guardname(prefix + h_file)
|
2015-04-02 14:46:39 +02:00
|
|
|
c_file = output_dir + prefix + c_file
|
|
|
|
h_file = output_dir + prefix + h_file
|
|
|
|
|
2015-09-03 10:24:25 +02:00
|
|
|
if output_dir:
|
|
|
|
try:
|
|
|
|
os.makedirs(output_dir)
|
2015-12-18 08:52:41 +01:00
|
|
|
except os.error as e:
|
2015-09-03 10:24:25 +02:00
|
|
|
if e.errno != errno.EEXIST:
|
|
|
|
raise
|
2015-04-02 14:46:39 +02:00
|
|
|
|
|
|
|
def maybe_open(really, name, opt):
|
|
|
|
if really:
|
|
|
|
return open(name, opt)
|
|
|
|
else:
|
|
|
|
import StringIO
|
|
|
|
return StringIO.StringIO()
|
|
|
|
|
|
|
|
fdef = maybe_open(do_c, c_file, 'w')
|
|
|
|
fdecl = maybe_open(do_h, h_file, 'w')
|
|
|
|
|
|
|
|
fdef.write(mcgen('''
|
|
|
|
/* AUTOMATICALLY GENERATED, DO NOT MODIFY */
|
|
|
|
%(comment)s
|
|
|
|
''',
|
2015-09-30 00:21:02 +02:00
|
|
|
comment=c_comment))
|
2015-04-02 14:46:39 +02:00
|
|
|
|
|
|
|
fdecl.write(mcgen('''
|
|
|
|
/* AUTOMATICALLY GENERATED, DO NOT MODIFY */
|
|
|
|
%(comment)s
|
|
|
|
#ifndef %(guard)s
|
|
|
|
#define %(guard)s
|
|
|
|
|
|
|
|
''',
|
2015-09-30 00:21:02 +02:00
|
|
|
comment=h_comment, guard=guard))
|
2015-04-02 14:46:39 +02:00
|
|
|
|
|
|
|
return (fdef, fdecl)
|
|
|
|
|
2015-09-30 00:21:02 +02:00
|
|
|
|
2015-04-02 14:46:39 +02:00
|
|
|
def close_output(fdef, fdecl):
|
|
|
|
fdecl.write('''
|
|
|
|
#endif
|
|
|
|
''')
|
|
|
|
fdecl.close()
|
|
|
|
fdef.close()
|