Commit graph

63562 commits

Author SHA1 Message Date
Markus Armbruster 037f244088 json: Have lexer call streamer directly
json_lexer_init() takes the function to process a token as an
argument.  It's always json_message_process_token().  Makes the code
harder to understand for no actual gain.  Drop the indirection.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180823164025.12553-34-armbru@redhat.com>
2018-08-24 20:26:37 +02:00
Marc-André Lureau e8b19d7d73 json-parser: simplify and avoid JSONParserContext allocation
parser_context_new/free() are only used from json_parser_parse(). We
can fold the code there and avoid an allocation altogether.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20180719184111.5129-9-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20180823164025.12553-33-armbru@redhat.com>
2018-08-24 20:26:37 +02:00
Marc-André Lureau 7c1e1d5481 json: remove useless return value from lexer/parser
The lexer always returns 0 when char feeding. Furthermore, none of the
caller care about the return value.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20180326150916.9602-10-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20180823164025.12553-32-armbru@redhat.com>
2018-08-24 20:26:37 +02:00
Markus Armbruster c473c379e1 check-qjson: Fix and enable utf8_string()'s disabled part
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180823164025.12553-31-armbru@redhat.com>
2018-08-24 20:26:37 +02:00
Markus Armbruster dc45a07c36 json: Fix \uXXXX for surrogate pairs
The JSON parser treats each half of a surrogate pair as unpaired
surrogate.  Fix it to recognize surrogate pairs.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180823164025.12553-30-armbru@redhat.com>
2018-08-24 20:26:37 +02:00
Markus Armbruster 46a628b139 json: Reject invalid \uXXXX, fix \u0000
The JSON parser translates invalid \uXXXX to garbage instead of
rejecting it, and swallows \u0000.

Fix by using mod_utf8_encode() instead of flawed wchar_to_utf8().

Valid surrogate pairs are now differently broken: they're rejected
instead of translated to garbage.  The next commit will fix them.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180823164025.12553-29-armbru@redhat.com>
2018-08-24 20:26:37 +02:00
Markus Armbruster de6decfe8e json: Simplify parse_string()
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180823164025.12553-28-armbru@redhat.com>
2018-08-24 20:26:37 +02:00
Markus Armbruster b2da4a4d75 json: Leave rejecting invalid escape sequences to parser
Both lexer and parser reject invalid escape sequences in strings.  The
parser's check is useless.

The lexer ends the token right after the first non-well-formed byte.
This tends to lead to suboptimal error reporting.  For instance, input

    {"abc\@ijk": 1}

produces the tokens

    JSON_LCURLY   {
    JSON_ERROR    "abc\@
    JSON_KEYWORD  ijk
    JSON_ERROR   ": 1}\n

The parser then reports three errors

    Invalid JSON syntax
    JSON parse error, invalid keyword 'ijk'
    Invalid JSON syntax

before it recovers at the newline.

Drop the lexer's escape sequence checking, and make it accept the same
characters after backslash it accepts elsewhere in strings.  It now
produces

    JSON_LCURLY   {
    JSON_STRING   "abc\@ijk"
    JSON_COLON    :
    JSON_INTEGER  1
    JSON_RCURLY

and the parser reports just

    JSON parse error, invalid escape sequence in string

While there, fix parse_string()'s inaccurate function comment.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180823164025.12553-27-armbru@redhat.com>
2018-08-24 20:26:37 +02:00
Markus Armbruster 4b1c0cd7c7 json: Accept overlong \xC0\x80 as U+0000 ("modified UTF-8")
Since the JSON grammer doesn't accept U+0000 anywhere, this merely
exchanges one kind of parse error for another.  It's purely for
consistency with qobject_to_json(), which accepts \xC0\x80 (see commit
e2ec3f9768).

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180823164025.12553-26-armbru@redhat.com>
2018-08-24 20:26:37 +02:00
Markus Armbruster de930f45cb json: Leave rejecting invalid UTF-8 to parser
Both the lexer and the parser (attempt to) validate UTF-8 in JSON
strings.

The lexer rejects bytes that can't occur in valid UTF-8: \xC0..\xC1,
\xF5..\xFF.  This rejects some, but not all invalid UTF-8.  It also
rejects ASCII control characters \x00..\x1F, in accordance with RFC
8259 (see recent commit "json: Reject unescaped control characters").

When the lexer rejects, it ends the token right after the first bad
byte.  Good when the bad byte is a newline.  Not so good when it's
something like an overlong sequence in the middle of a string.  For
instance, input

    {"abc\xC0\xAFijk": 1}\n

produces the tokens

    JSON_LCURLY   {
    JSON_ERROR    "abc\xC0
    JSON_ERROR    \xAF
    JSON_KEYWORD  ijk
    JSON_ERROR   ": 1}\n

The parser then reports four errors

    Invalid JSON syntax
    Invalid JSON syntax
    JSON parse error, invalid keyword 'ijk'
    Invalid JSON syntax

before it recovers at the newline.

The commit before previous made the parser reject invalid UTF-8
sequences.  Since then, anything the lexer rejects, the parser would
reject as well.  Thus, the lexer's rejecting is unnecessary for
correctness, and harmful for error reporting.

However, we want to keep rejecting ASCII control characters in the
lexer, because that produces the behavior we want for unclosed
strings.

We also need to keep rejecting \xFF in the lexer, because we
documented that as a way to reset the JSON parser
(docs/interop/qmp-spec.txt section 2.6 QGA Synchronization), which
means we can't change how we recover from this error now.  I wish we
hadn't done that.

I think we should treat \xFE the same as \xFF.

Change the lexer to accept \xC0..\xC1 and \xF5..\xFD.  It now rejects
only \x00..\x1F and \xFE..\xFF.  Error reporting for invalid UTF-8 in
strings is much improved, except for \xFE and \xFF.  For the example
above, the lexer now produces

    JSON_LCURLY   {
    JSON_STRING   "abc\xC0\xAFijk"
    JSON_COLON    :
    JSON_INTEGER  1
    JSON_RCURLY

and the parser reports just

    JSON parse error, invalid UTF-8 sequence in string

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180823164025.12553-25-armbru@redhat.com>
2018-08-24 20:26:37 +02:00
Markus Armbruster 574bf16ff1 json: Report first rather than last parse error
Quiz time!  When a parser reports multiple errors, but the user gets
to see just one, which one is (on average) the least useful one?

Yes, you're right, it's the last one!  You're clearly familiar with
compilers.

Which one does QEMU report?

Right again, the last one!  You're clearly familiar with QEMU.

Reproducer: feeding

    {"abc\xC2ijk": 1}\n

to QMP produces

    {"error": {"class": "GenericError", "desc": "JSON parse error, key is not a string in object"}}

Report the first error instead.  The reproducer now produces

    {"error": {"class": "GenericError", "desc": "JSON parse error, invalid UTF-8 sequence in string"}}

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180823164025.12553-24-armbru@redhat.com>
2018-08-24 20:26:37 +02:00
Markus Armbruster e59f39d403 json: Reject invalid UTF-8 sequences
We reject bytes that can't occur in valid UTF-8 (\xC0..\xC1,
\xF5..\xFF in the lexer.  That's insufficient; there's plenty of
invalid UTF-8 not containing these bytes, as demonstrated by
check-qjson:

* Malformed sequences

  - Unexpected continuation bytes

  - Missing continuation bytes after start bytes other than
    \xC0..\xC1, \xF5..\xFD.

* Overlong sequences with start bytes other than \xC0..\xC1,
  \xF5..\xFD.

* Invalid code points

Fixing this in the lexer would be bothersome.  Fixing it in the parser
is straightforward, so do that.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180823164025.12553-23-armbru@redhat.com>
2018-08-24 20:26:37 +02:00
Markus Armbruster a89d3104a2 check-qjson: Document we expect invalid UTF-8 to be rejected
The JSON parser rejects some invalid sequences, but accepts others
without correcting the problem.

We should either reject all invalid sequences, or minimize overlong
sequences and replace all other invalid sequences by a suitable
replacement character.  A common choice for replacement is U+FFFD.

I'm going to implement the former.  Update the comments in
utf8_string() to expect this.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180823164025.12553-22-armbru@redhat.com>
2018-08-24 20:26:37 +02:00
Markus Armbruster 00ea57fadc json: Tighten and simplify qstring_from_escaped_str()'s loop
Simplify loop control, and assert that the string ends with the
appropriate quote (the lexer ensures it does).

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180823164025.12553-21-armbru@redhat.com>
2018-08-24 20:26:37 +02:00
Markus Armbruster eddc0a7f0a json: Revamp lexer documentation
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180823164025.12553-20-armbru@redhat.com>
2018-08-24 20:26:37 +02:00
Markus Armbruster 340db1ed82 json: Reject unescaped control characters
Fix the lexer to reject unescaped control characters in JSON strings,
in accordance with RFC 8259 "The JavaScript Object Notation (JSON)
Data Interchange Format".

Bonus: we now recover more nicely from unclosed strings.  E.g.

    {"one: 1}\n{"two": 2}

now recovers cleanly after the newline, where before the lexer
remained confused until the next unpaired double quote or lexical
error.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180823164025.12553-19-armbru@redhat.com>
2018-08-24 20:26:37 +02:00
Markus Armbruster a2ec6be72b json: Fix lexer to include the bad character in JSON_ERROR token
json_lexer[] maps (lexer state, input character) to the new lexer
state.  The input character is consumed unless the new state is
terminal and the input character doesn't belong to this token,
i.e. the state transition uses look-ahead.  When this is the case,
input character '\0' would result in the same state transition.
TERMINAL_NEEDED_LOOKAHEAD() exploits this.

Except this is wrong for transitions to IN_ERROR.  There, the
offending input character is in fact consumed: case IN_ERROR returns.
It isn't added to the JSON_ERROR token, though.

Fix that by making TERMINAL_NEEDED_LOOKAHEAD() return false for
transitions to IN_ERROR.

There's a slight complication.  json_lexer_flush() passes input
character '\0' to flush an incomplete token.  If this results in
JSON_ERROR, we'd now add the '\0' to the token.  Suppress that.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180823164025.12553-18-armbru@redhat.com>
2018-08-24 20:26:37 +02:00
Markus Armbruster 2e933f5701 check-qjson: Cover interpolation more thoroughly
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180823164025.12553-17-armbru@redhat.com>
2018-08-24 20:26:37 +02:00
Markus Armbruster 6bc93a3401 check-qjson qmp-test: Cover control characters more thoroughly
RFC 8259 "The JavaScript Object Notation (JSON) Data Interchange
Format" requires control characters in strings to be escaped.
Demonstrate the JSON parser accepts U+0001 .. U+001F unescaped.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180823164025.12553-16-armbru@redhat.com>
2018-08-24 20:26:37 +02:00
Markus Armbruster 5f454e662e check-qjson: Fix utf8_string() to test all invalid sequences
Some of utf8_string()'s test_cases[] contain multiple invalid
sequences.  Testing that qobject_from_json() fails only tests we
reject at least one invalid sequence.  That's incomplete.

Additionally test each non-space sequence in isolation.

This demonstrates that the JSON parser accepts invalid sequences
starting with \xC2..\xF4.  Add a FIXME comment.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180823164025.12553-15-armbru@redhat.com>
2018-08-24 20:26:37 +02:00
Markus Armbruster 32846e9304 check-qjson: Simplify utf8_string()
The previous commit made utf8_string()'s test_cases[].utf8_in
superfluous: we can use .json_in instead.  Except for the case testing
U+0000.  \x00 doesn't work in C strings, so it tests \\u0000 instead.
But testing \\uXXXX is escaped_string()'s job.  It's covered there.
Test U+0001 here, and drop .utf8_in.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180823164025.12553-14-armbru@redhat.com>
2018-08-24 20:26:37 +02:00
Markus Armbruster 6ad8444f6a check-qjson: Cover UTF-8 in single quoted strings
utf8_string() tests only double quoted strings.  Cover single quoted
strings, too: store the strings to test without quotes, then wrap them
in either kind of quote.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180823164025.12553-13-armbru@redhat.com>
2018-08-24 20:26:37 +02:00
Markus Armbruster 069946f402 check-qjson: Consolidate partly redundant string tests
simple_string() and single_quote_string() have become redundant with
escaped_string(), except for embedded single and double quotes.
Replace them by a test that covers just that.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180823164025.12553-12-armbru@redhat.com>
2018-08-24 20:26:37 +02:00
Markus Armbruster e0fe2a978e check-qjson: Cover escaped characters more thoroughly, part 2
Cover escaped single quote, surrogates, invalid escapes, and
noncharacters.  This demonstrates that valid surrogate pairs are
misinterpreted, and invalid surrogates and noncharacters aren't
rejected.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180823164025.12553-11-armbru@redhat.com>
2018-08-24 20:26:37 +02:00
Markus Armbruster f3cfdd3a30 check-qjson: Streamline escaped_string()'s test strings
Merge a few closely related test strings, and drop a few redundant
ones.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180823164025.12553-10-armbru@redhat.com>
2018-08-24 20:26:37 +02:00
Markus Armbruster 4e1df9b734 check-qjson: Cover escaped characters more thoroughly, part 1
escaped_string() first tests double quoted strings, then repeats a few
tests with single quotes.  Repeat all of them: store the strings to
test without quotes, and wrap them in either kind of quote for
testing.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180823164025.12553-9-armbru@redhat.com>
2018-08-24 20:26:37 +02:00
Markus Armbruster e2f64a688b test-qga: Clean up how we test QGA synchronization
To permit recovering from arbitrary JSON parse errors, the JSON parser
resets itself on lexical errors.  We recommend sending a 0xff byte for
that purpose, and test-qga covers this usage since commit 5229564b83.
That commit had to add an ugly hack to qmp_fd_vsend() to make capable
of sending this byte (it's designed to send only valid JSON).

The previous commit added a way to send arbitrary text.  Put that to
use for this purpose, and drop the hack from qmp_fd_vsend().

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180823164025.12553-8-armbru@redhat.com>
2018-08-24 20:26:37 +02:00
Markus Armbruster aed877c53b qmp-test: Cover syntax and lexical errors
qmp-test neglects to cover QMP input that isn't valid JSON.  libqtest
doesn't let us send such input.  Add qtest_qmp_send_raw() for this
purpose, and put it to use in qmp-test.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180823164025.12553-7-armbru@redhat.com>
[Commit message typo fixed]
2018-08-24 20:26:37 +02:00
Markus Armbruster d93bb9d5c3 qmp-cmd-test: Split off qmp-test
qmp-test is for QMP protocol tests.  Commit e4a426e75e added generic,
basic tests of query commands to it.  Move them to their own test
program qmp-cmd-test, to keep qmp-test focused on the protocol.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180823164025.12553-6-armbru@redhat.com>
2018-08-24 20:25:48 +02:00
Markus Armbruster 5365490879 check-qjson: Cover whitespace more thoroughly
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180823164025.12553-5-armbru@redhat.com>
2018-08-24 20:25:48 +02:00
Markus Armbruster a3694181e3 check-qjson: Cover blank and lexically erroneous input
qobject_from_json() can return null without setting an error on
lexical errors.  I call that a bug.  Add test coverage to demonstrate
it.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180823164025.12553-4-armbru@redhat.com>
2018-08-24 20:25:48 +02:00
Markus Armbruster 956a104a6c check-qjson: Cover multiple JSON objects in same string
qobject_from_json() & friends misbehave when the JSON text has more
than one JSON value.  Add test coverage to demonstrate the bugs.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180823164025.12553-3-armbru@redhat.com>
2018-08-24 20:25:48 +02:00
Markus Armbruster 72e9e569d0 docs/interop/qmp-spec: How to force known good parser state
Section "QGA Synchronization" specifies that sending "a raw 0xFF
sentinel byte" makes the server "reset its state and discard all
pending data prior to the sentinel."  What actually happens there is a
lexical error, which will produce one or more error responses.
Moreover, it's not specific to QGA.

Create new section "Forcing the JSON parser into known-good state" to
document the technique properly.  Rewrite section "QGA
Synchronization" to document just the other direction, i.e. command
guest-sync-delimited.

Section "Protocol Specification" mentions "synchronization bytes
(documented below)".  Delete that.

While there, fix it not to claim '"Server" is QEMU itself', but
'"Server" is either QEMU or the QEMU Guest Agent'.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180823164025.12553-2-armbru@redhat.com>
2018-08-24 20:25:48 +02:00
Peter Maydell 1dfb85a875 check/next for 20180822
-----BEGIN PGP SIGNATURE-----
 
 iQIcBAABCAAGBQJbfRjcAAoJEPSH7xhYctcj6vkP/0CxdFLEJ5zfmQCT9plrcenc
 4CtR3syKimF1CSk9eKPE8V3oiZSBxuM1FJYPhH8d9UbYcWrItGLr/dgh1EAgurAI
 P4oeWqI21CGeCWQGduhmQ51vSw1b8JTdNYWmb3QAsMBUugZYla4lvC3R5h63vmBI
 4U1RzQrZmmN9svnNMx22dCInbPNoayR3Ekr7z/bF6sRG/B+ZwecenVoD9X8T/Ozu
 epx9OOoBfMGDB5wbEEx/RUKrMsGH5D712QeMHUtGYmLRs1Wl4AV7Si+bSd3oi+GI
 aL6ZjuaOofGaESOuH7fTkTGhGgmcPd7+pLPqpEYIJ3wmQOOQP/dp9B+6VXCxTQcA
 y5F9FBEP5nQL+OIusvi+l65PqzstrKtxrSzWPGHgmosdLead15znZ4Z6YdOtHWHr
 ZZOW55M2ZvlZvEWB3hHmT9rjZFP4Uu9XFIW05gzQiqVhcKemtgQ3hBiX+OvxRpPM
 RpGLqGK/oDwadvEsNitYqbRJDe74VSAxOtmvEsDfJLzRoyHM1zHw3au00NdyGhMp
 89Xc5AnkuHJLCFZ9duXErt5GQz/7EzHkLpQ16pqyuetgLc50ytED1tgqiJl6+Td1
 IS4me8wwBpk+IvRD0zsh4p/FHocL684CSP5+AZwcy8RljtbBacbvmAS6484RSx4c
 rWL39dB2uhvHEErmS9/+
 =2bke
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/juanquintela/tags/check/20180822' into staging

check/next for 20180822

# gpg: Signature made Wed 22 Aug 2018 09:03:40 BST
# gpg:                using RSA key F487EF185872D723
# gpg: Good signature from "Juan Quintela <quintela@redhat.com>"
# gpg:                 aka "Juan Quintela <quintela@trasno.org>"
# Primary key fingerprint: 1899 FF8E DEBF 58CC EE03  4B82 F487 EF18 5872 D723

* remotes/juanquintela/tags/check/20180822:
  check: Only test tpm devices when they are compiled in
  check: Only test usb-ehci when it is compiled in
  check: Only test usb-uhci devices when they are compiled in
  check: Only test usb-ohci when it is compiled in
  check: Only test nvme when it is compiled in
  check: Only test pvpanic when it is compiled in
  check: Only test wdt_ib700 when it is compiled in
  check: Only test sdhci when it is compiled in
  check: Only test i82801b11 when it is compiled in
  check: Only test ioh3420 when it is compiled in
  check: Only test ipack when it is compiled in
  check: Only test hda when it is compiled in
  check: Only test ac97 when it is compiled in
  check: Only test es1370 when it is compiled in
  check: Only test rtl8139 when it is compiled in
  check: Only test pcnet when it is compiled in
  check: Only test eepro100 when it is compiled in
  check: Only test ne2000 when it is compiled in
  check: Only test vmxnet3 when it is compiled in

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-08-24 14:46:31 +01:00
Peter Maydell f4e8428b9a target-arm queue:
* Fix rounding errors in scaling float-to-int and int-to-float operations
  * Connect virtualization-related IRQs and memory regions of GICv2
    in boards that use Cortex-A7 or Cortex-A15
  * Support taking exceptions to AArch32 Hyp mode
  * Clear CPSR.IL and CPSR.J on 32-bit exception entry
    (a minor bug fix that won't affect non-buggy guest code)
  * mps2-an505: Implement various missing devices:
    dual timer, watchdogs, counters in the FPGAIO registers,
    some missing ID/control registers, TrustZone Master Security
    Controllers, PL081 DMA controllers, PL022 SPI controllers
  * correct ID register values for mps2-an385, -an511, -an505
  * fix some hardcoded tabs in untouched backwaters of the
    target/arm codebase
  * raspi: Refactor framebuffer property handling code and implement
    support for the virtual framebuffer/viewport
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1
 
 iQIcBAABCAAGBQJbf/e4AAoJEDwlJe0UNgze+PwP/jOwmu1UXHiFlegv3DgTCd3O
 ZHXxVZH1yqXY1DDXU4LwAv6dtnrxmd8w2Q352FlETdXeXYhG9MOlLIAq7VKseLMe
 oeTxDB4fGyd1sLjovxYVjpcELW2P7Oz0oXuCX/IgxycAu7nDZqUVdy5jyqdZdeEI
 YoIcpEjG8wigO2GN5ccbuwy2bo9OAvKGyD3qGvcENxc9JgmkE5rhfTWsuZhxlhKe
 ONU9kOPK0AnRGOfbr6S0fcXZYZcZ4vzc0h/seAkkrjwkkdfUAQ7lhZEBRfX+ENYL
 KgShmPHfu+IxTeAfdIlWz70FEM6gxxUxut/Bta0tl/OGqUAcHaKvGxUEoTrIqcLe
 XLTd5ae3imFerVlLrwIfJj2Lk3nVWDdArG2isDcIVAQ9uWhMY37x5IrRK55USgcw
 GSFuNUU3dQmKVqD3e0i6fCE/+ZAVaZU/2RNWiH8s5Y1e38DajxMgw4ed4KZ5lG/b
 lyR2q5xcCMpryaNqqMP3eIZhS00KP9n7CZblIj01//txcpMijXMWLXatjt71RUlk
 AnSv8yjcBh2s91OCt4sqD1zjYEECDw1xLEi2SGyVo5WOod4vGAoFqHbD4JF9anq0
 TTsS5TBXsG2tFztKpyftqzffLCpHepN8VcptIRWcS22PYaMjT2ibvbV9ojEgQH8Y
 Wj9rqLgU0aqgIwVObrzu
 =4kLg
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-20180824-1' into staging

target-arm queue:
 * Fix rounding errors in scaling float-to-int and int-to-float operations
 * Connect virtualization-related IRQs and memory regions of GICv2
   in boards that use Cortex-A7 or Cortex-A15
 * Support taking exceptions to AArch32 Hyp mode
 * Clear CPSR.IL and CPSR.J on 32-bit exception entry
   (a minor bug fix that won't affect non-buggy guest code)
 * mps2-an505: Implement various missing devices:
   dual timer, watchdogs, counters in the FPGAIO registers,
   some missing ID/control registers, TrustZone Master Security
   Controllers, PL081 DMA controllers, PL022 SPI controllers
 * correct ID register values for mps2-an385, -an511, -an505
 * fix some hardcoded tabs in untouched backwaters of the
   target/arm codebase
 * raspi: Refactor framebuffer property handling code and implement
   support for the virtual framebuffer/viewport

# gpg: Signature made Fri 24 Aug 2018 13:19:04 BST
# gpg:                using RSA key 3C2525ED14360CDE
# gpg: Good signature from "Peter Maydell <peter.maydell@linaro.org>"
# gpg:                 aka "Peter Maydell <pmaydell@gmail.com>"
# gpg:                 aka "Peter Maydell <pmaydell@chiark.greenend.org.uk>"
# Primary key fingerprint: E1A5 C593 CD41 9DE2 8E83  15CF 3C25 25ED 1436 0CDE

* remotes/pmaydell/tags/pull-target-arm-20180824-1: (52 commits)
  hw/arm/mps2: Fix ID register errors on AN511 and AN385
  hw/display/bcm2835_fb: Validate bcm2835_fb_mbox_push() config
  hw/display/bcm2835_fb: Validate config settings
  hw/display/bcm2835_fb: Fix handling of virtual framebuffer
  hw/display/bcm2835_fb: Abstract out calculation of pitch, size
  hw/display/bcm2835_fb: Reset resolution, etc correctly
  hw/display/bcm2835_fb: Drop unused size and pitch fields
  hw/misc/bcm2835_property: Track fb settings using BCM2835FBConfig
  hw/misc/bcm2835_fb: Move config fields to their own struct
  target/arm: Remove a handful of stray tabs
  target/arm: Untabify iwmmxt_helper.c
  target/arm: Untabify translate.c
  hw/arm/mps2-tz: Fix MPS2 SCC config register values
  hw/arm/mps2-tz: Instantiate SPI controllers
  hw/ssi/pl022: Correct wrong DMACR and ICR handling
  hw/ssi/pl022: Correct wrong value for PL022_INT_RT
  hw/ssi/pl022: Use DeviceState::realize rather than SysBusDevice::init
  hw/ssi/pl022: Don't directly call vmstate_register()
  hw/ssi/pl022: Set up reset function in class init
  hw/ssi/pl022: Allow use as embedded-struct device
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-08-24 13:29:07 +01:00
Peter Maydell 239cb6feb2 hw/arm/mps2: Fix ID register errors on AN511 and AN385
Fix MPS2 SCC config register values for the mps2-an511
and mps2-an385 boards:
 * the SCC_AID bits [23:20] specify the FPGA build target board revision,
   and the SCC_CFG4 register specifies the actual board revision, so
   these should have matching values. Claim to be board revision C,
   consistently -- we had the revision in the wrong part of SCC_AID.
 * SCC_ID bits [15:4] should be the board number in hex, not decimal

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20180823175225.22612-1-peter.maydell@linaro.org
2018-08-24 13:17:50 +01:00
Peter Maydell cfb7ba9838 hw/display/bcm2835_fb: Validate bcm2835_fb_mbox_push() config
Refactor bcm2835_fb_mbox_push() to work by calling
bcm2835_fb_validate_config() and bcm2835_fb_reconfigure(),
so that config set this way is also validated.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20180814144436.679-9-peter.maydell@linaro.org
2018-08-24 13:17:50 +01:00
Peter Maydell f8add62c0c hw/display/bcm2835_fb: Validate config settings
Validate the config settings that the guest tries to set.

The wiki page documentation is not really accurate here:
generally rather than failing requests to set bad parameters,
the hardware will just clip them to something sensible.

Validate the most important parameters: sizes and
the viewport offsets. This prevents the framebuffer
code from trying to read out-of-range memory.

In the property handling code, we validate the new parameters every
time we encounter a tag that sets them. This means we validate the
config multiple times if the request includes multiple config-setting
tags, but the code would require significant restructuring to do a
validation only once but still return the clipped settings for
get-parameter tags and the buffer allocation tag.

Validation of settings made via the older bcm2835_fb_mbox_push()
function will be done in the next commit.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20180814144436.679-8-peter.maydell@linaro.org
2018-08-24 13:17:50 +01:00
Peter Maydell 01f18af98b hw/display/bcm2835_fb: Fix handling of virtual framebuffer
The raspi framebuffir in bcm2835_fb supports the definition
of a virtual "viewport", which is smaller than the full
physical framebuffer size and at an adjustable offset within
it. Only the viewport area is sent to the screen. This allows
the guest to do things like double buffering, or scrolling
by adjusting the viewport origin. Currently QEMU doesn't
implement this at all.

Add support for this feature:
 * the property mailbox code needs to distinguish the
   virtual width/height from the physical width/height
 * the framebuffer code needs to do something with the
   virtual width/height/origin information

Note that the wiki documentation on the semantics of the
virtual and physical height and width has it the wrong way
around -- the virtual size is the size of the allocated
buffer, and the physical size is the size of the display,
so the virtual size is always the same as or larger than
the physical.

If the viewport size is set smaller than the physical
screen size, we ignore the viewport settings completely
and just display the physical screen area.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20180814144436.679-7-peter.maydell@linaro.org
2018-08-24 13:17:49 +01:00
Peter Maydell 9a1f03f4ee hw/display/bcm2835_fb: Abstract out calculation of pitch, size
Abstract out the calculation of the pitch and size of the
framebuffer into functions that operate on the BCM2835FBConfig
struct -- these are about to get a little more complicated
when we add support for virtual and physical sizes differing.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20180814144436.679-6-peter.maydell@linaro.org
2018-08-24 13:17:49 +01:00
Peter Maydell 9e2938a0fd hw/display/bcm2835_fb: Reset resolution, etc correctly
The bcm2835_fb's initial resolution and other parameters are set
via QOM properties. We should reset to those initial values on
device reset, which means we need to save the QOM property
values somewhere that they are not overwritten by guest
changes to the framebuffer configuration.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20180814144436.679-5-peter.maydell@linaro.org
2018-08-24 13:17:49 +01:00
Peter Maydell ea662f7cc8 hw/display/bcm2835_fb: Drop unused size and pitch fields
The BCM2835FBState struct has a 'pitch' field which is a
cached copy of xres * (bpp >> 3), and a 'size' field which is
a cached copy of pitch * yres. However we don't actually do
anything with these fields; delete them. We retain the
now-unused slots in the VMState struct for migration
compatibility.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20180814144436.679-4-peter.maydell@linaro.org
2018-08-24 13:17:49 +01:00
Peter Maydell 193100b571 hw/misc/bcm2835_property: Track fb settings using BCM2835FBConfig
Refactor the fb property setting code so that rather than
using a set of pointers to local variables to track
whether a config value has been updated in the current
mbox and if so what its new value is, we just copy
all the current settings of the fb at the start, and
then update that copy as we go along, before asking
the fb to switch to it at the end.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20180814144436.679-3-peter.maydell@linaro.org
2018-08-24 13:17:48 +01:00
Peter Maydell a02755ece0 hw/misc/bcm2835_fb: Move config fields to their own struct
The handling of framebuffer properties in the bcm2835_property code
is a bit clumsy, because for each of the many fb related properties
we try to track the value we're about to set and whether we're going
to be setting a value, and then we hand all the new values off
to the framebuffer via a function which takes them all as separate
arguments. It would be simpler if the property code could easily
copy all the framebuffer's current settings, update them with
the new specified values and then ask the framebuffer to switch
to the new set.

As the first part of this refactoring, pull all the fb config
settings fields in BCM2835FBState out into their own struct.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20180814144436.679-2-peter.maydell@linaro.org
2018-08-24 13:17:48 +01:00
Peter Maydell 6e0fafe2ef target/arm: Remove a handful of stray tabs
Following the bulk conversion of the iwMMXt code, there are
just a handful of hard coded tabs in target/arm; fix them.
This is a whitespace-only patch.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Message-id: 20180821165215.29069-4-peter.maydell@linaro.org
2018-08-24 13:17:48 +01:00
Peter Maydell 67aed15551 target/arm: Untabify iwmmxt_helper.c
Untabify the arm iwmmxt_helper.c.  This affects only the iwMMXt code.
We've never touched that code in years, so it's not going to get
fixed up by our "change when touched" process, and a bulk change is
not going to be too disruptive.

This commit was produced using Emacs "untabify" (plus one
by-hand removal of a space to fix a checkpatch nit); it is
a whitespace-only change.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Message-id: 20180821165215.29069-3-peter.maydell@linaro.org
2018-08-24 13:17:48 +01:00
Peter Maydell d00584b7cf target/arm: Untabify translate.c
Untabify the arm translate.c. This affects only some lines,
mostly comments, in the iwMMXt code. We've never touched
that code in years, so it's not going to get fixed up
by our "change when touched" process, and a bulk change
is not going to be too disruptive.

This commit was produced using Emacs "untabify"; it is
a whitespace-only change.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Message-id: 20180821165215.29069-2-peter.maydell@linaro.org
2018-08-24 13:17:47 +01:00
Peter Maydell cb159db944 hw/arm/mps2-tz: Fix MPS2 SCC config register values
Some of the config register values we were setting for the MPS2 SCC
weren't correct:
 * the SCC_AID bits [23:20] specify the FPGA build target board revision,
   and the SCC_CFG4 register specifies the actual board revision, so
   these should have matching values. Claim to be board revision C,
   consistently -- we had the revision in the wrong part of SCC_AID.
 * SCC_ID bits [15:4] should be 0x505, not decimal 505

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Message-id: 20180820141116.9118-23-peter.maydell@linaro.org
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2018-08-24 13:17:47 +01:00
Peter Maydell 0d49759b97 hw/arm/mps2-tz: Instantiate SPI controllers
The SPI controllers in the MPS2 AN505 board are PL022s.
We have a model of the PL022, so create these devices.

We don't currently model the LCD controller that sits behind
one of the PL022s; the others are intended to control devices
that sit on the FPGA's general purpose SPI connector or
"shield" expansion connectors.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Message-id: 20180820141116.9118-22-peter.maydell@linaro.org
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2018-08-24 13:17:47 +01:00
Peter Maydell 7d3912f54e hw/ssi/pl022: Correct wrong DMACR and ICR handling
In the PL022, register offset 0x20 is the ICR, a write-only
interrupt-clear register.  Register offset 0x24 is DMACR, the DMA
control register.  We were incorrectly implementing (a stub version
of) DMACR at 0x20, and not implementing anything at 0x24.  Fix this
bug.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Message-id: 20180820141116.9118-21-peter.maydell@linaro.org
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2018-08-24 13:17:46 +01:00