Replace "raw_" file prefixes with a directory path. This mirrors the
directory structure of the module and is cleaner to work
with.
Change-Id: I067b7772a83f9707633828119319376324c04e10
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/52862
Commit-Queue: Wyatt Hepler <hepler@google.com>
Pigweed-Auto-Submit: Wyatt Hepler <hepler@google.com>
Reviewed-by: Alexei Frolov <frolv@google.com>
- Support codegen for raw and Nanopb RPC services.
- Leave a TODO to implement client and bidirectional streaming for the
Nanopb client.
Change-Id: Ifb69bec13a8ac912d25ddda28a4e26f392d4fed5
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/52522
Reviewed-by: Alexei Frolov <frolv@google.com>
Pigweed-Auto-Submit: Wyatt Hepler <hepler@google.com>
Commit-Queue: Auto-Submit <auto-submit@pigweed.google.com.iam.gserviceaccount.com>
Update the server-to-client RPC packet types so that the packet type
unambiguously indicates whether it is the first or last packet. This is
already the case for client-to-server RPC packet types.
- Have RESPONSE always be the last packet in the stream. For RPCs
without a server stream, it includes a payload. Remove
SERVER_STREAM_END.
- Introduce the SERVER_STREAM packet, to parallel the CLIENT_STREAM
packet.
- Update the server and client code and tests. Test that old-style
streaming RPCs still work correctly.
- Refactor the duplicate MessageOutput class into a FakeChannelOutput
used by both Nanopb and raw RPCs.
- In C++, don't encode default-valued payload and status fields.
Change-Id: I218772dad6c2981dda5f032f298ea43ee5e08b4d
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/49822
Commit-Queue: Wyatt Hepler <hepler@google.com>
Pigweed-Auto-Submit: Wyatt Hepler <hepler@google.com>
Reviewed-by: Alexei Frolov <frolv@google.com>
- Add a CLIENT_STREAM packet for messages in client streams.
- Rename CANCEL_SERVER_STREAM to CANCEL, since it applies to all RPC
types.
- Remove server responses to CANCEL packets from protocol description.
- Document cancellation of unary RPCs.
Change-Id: I3b15beb2de94f318fac58fc3d682aefe41056676
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/48800
Commit-Queue: Wyatt Hepler <hepler@google.com>
Reviewed-by: Alexei Frolov <frolv@google.com>
Adds initial support for c++ RPC libraries. This change only
supports the raw RPC protocol.
Change-Id: Icdecc14f1b8be03120330094793315f807deb3ec
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/43921
Reviewed-by: Wyatt Hepler <hepler@google.com>
Commit-Queue: Wyatt Hepler <hepler@google.com>
This updates the nanopb RPC client to use pw::Function for callbacks
instead of custom virtual interfaces.
Change-Id: I7c548cde00223ec32e45cc831fe7f9f79577842b
Requires: pigweed-internal:13201
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/48484
Reviewed-by: Wyatt Hepler <hepler@google.com>
Nanopb client calls are templated and can have long type names. For
example, the client call type for the nanopb EchoService.Echo method is
NanopbClientCall<UnaryResponseHandler<pw_rpc_EchoMessage>>
This updates the generated RPC client code to provide aliases for the
client call types of each method within the service client. The above
type can now be replaced with
EchoServiceClient::EchoCall
which is much more legible.
Change-Id: I184e34b2b52b120b83227be82c4a411dcd4941ec
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/42929
Commit-Queue: Alexei Frolov <frolv@google.com>
Reviewed-by: Keir Mierle <keir@google.com>
Reviewed-by: Wyatt Hepler <hepler@google.com>
- The alias_deprecated_command function creates an alias that redirects
from an old RPC command name to a new name.
- Move console_tools.Watchdog tests to their own file.
Change-Id: Ie7071462e5c17e5782fb549cc2f15aa19e64bbf0
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/44120
Reviewed-by: Alexei Frolov <frolv@google.com>
Commit-Queue: Wyatt Hepler <hepler@google.com>
- Rework the cumbersome multi_client_terminal_variables function into a
pw_rpc.console_tools.Context class. The class can be extended if
needed.
- Provide functions.help_as_repr that wraps a function so that its
__repr__ displays help information. This is helpful for defining
commands in an interactive console.
- Display all variables, not just RPCs, in the CommandHelper.
Requires: pigweed-internal:11382
Change-Id: I9a4dfc25473f6cf878f7855602a4f2334313de29
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/41120
Reviewed-by: Keir Mierle <keir@google.com>
Commit-Queue: Wyatt Hepler <hepler@google.com>
The multi_client_terminal_variables function sets up local variables for
use in an RPC terminal with one or more RPC clients.
Change-Id: I6e53472af157aea3074671491528ec0e9b84bca4
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/40141
Reviewed-by: Alexei Frolov <frolv@google.com>
Commit-Queue: Wyatt Hepler <hepler@google.com>
- Support opening an RPC without calling it in the core pw_rpc client.
- Provide a keep_open option so the client keep RPCs going even if they
are completed.
Change-Id: Idec8247f9ee0b432c0bfb5dbcf1d08ae1e27daea
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/39823
Pigweed-Auto-Submit: Wyatt Hepler <hepler@google.com>
Commit-Queue: Auto-Submit <auto-submit@pigweed.google.com.iam.gserviceaccount.com>
Reviewed-by: Alexei Frolov <frolv@google.com>
- Define an interface for generating stubs.
- Generate the method implementation stubs outside the class.
- Share more code between raw and Nanopb RPC stub generation.
- Skip empty names in the C++ namespace to avoid outputting "::".
Change-Id: Ic00b81ae5af07e89408962454432fa227033b904
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/39201
Commit-Queue: Wyatt Hepler <hepler@google.com>
Pigweed-Auto-Submit: Wyatt Hepler <hepler@google.com>
Reviewed-by: Alexei Frolov <frolv@google.com>
Cancel server streaming RPCs if they time out or if iteration is
aborted by an exception (such as KeyboardInterrupt).
Change-Id: Ib6a29c1456e600937129830fdfa2cabeb76d2aca
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/39323
Reviewed-by: Alexei Frolov <frolv@google.com>
Commit-Queue: Wyatt Hepler <hepler@google.com>
Ensure that Nanopb method implementations use the correct request and
response types. Using the wrong struct type will be a compilation error.
Change-Id: I9d300a108d76d014cb7699a72b7526b5b3e2d82e
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/33701
Commit-Queue: Auto-Submit <auto-submit@pigweed.google.com.iam.gserviceaccount.com>
Pigweed-Auto-Submit: Wyatt Hepler <hepler@google.com>
Reviewed-by: Alexei Frolov <frolv@google.com>
Create a simple watchdog class that could be used to track connection
state in a pw_rpc console.
Change-Id: I943b514d7994e78ef52898e639c691e5b6c2c046
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/33100
Reviewed-by: Keir Mierle <keir@google.com>
Reviewed-by: Jennifer Silva <jennifersilva@google.com>
Reviewed-by: Alexei Frolov <frolv@google.com>
Commit-Queue: Wyatt Hepler <hepler@google.com>
- Support nested messages, enums, oneof, mappings, optional fields, and
repeated fields in the proto_repr function.
- Display only hexadecimal for bytes when fewer than half the characters
are ASCII.
- Add the flag for optional fields in proto3 to dynamic proto
compilation so that optional fields can be used in tests.
Change-Id: I87dd646de1e339e80ceec729476af89548af21e5
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/32880
Commit-Queue: Wyatt Hepler <hepler@google.com>
Reviewed-by: Alexei Frolov <frolv@google.com>
- Add separate methods for response, completion, and error to the
ClientImpl interface. This allows explicit handling of errors by the
client.
- In the callback_client impl, raise exceptions for server errors.
- Simplify the callback_client's invoke method.
- Add pw_rpc_timeout_s to RPC method tab completion.
Change-Id: I1a2214057147c0d1b2547973da1b775f76e36d60
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/31582
Commit-Queue: Wyatt Hepler <hepler@google.com>
Reviewed-by: Alexei Frolov <frolv@google.com>
- Check that the first message is a "test_case_start" message. This
prevents undefined variable errors if the first message is dropped.
- Support providing a timeout to use for the pw.unit_test.Run RPC.
Change-Id: I3f1df140cb07097969c6d7cf79b9c30a21172458
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/32140
Reviewed-by: Ewout van Bekkum <ewout@google.com>
Reviewed-by: Alexei Frolov <frolv@google.com>
Commit-Queue: Wyatt Hepler <hepler@google.com>
- Pass all files to Mypy rather than the directory, in case tests
happen to reside at different paths.
- Remove unnecessary aliases of packet_pb2 members.
- Move Python requirements that apply throughout the Python build from
pw_presubmit to a pw_python_requirements target in pw_build.
- Update to Mypy 0.800 and mypy-protobuf 2.2.
Change-Id: I07ea07fb864ca4cd79fa5c0389e03146eca2b755
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/31600
Reviewed-by: Rob Mohr <mohrr@google.com>
For compatibility with Python packaging, each pw_protobuf_library target
must have a unique root directory for its protos. If this is violated,
the setup.py for the Python package is generated multiple times,
resulting in flaky builds.
This change ensures that no two pw_proto_library targets generate the
same Python package and fixes existing conflicts in the Pigweed repo.
Change-Id: Icbc13ceeb2fd329bc4fbf1e14ec039e98dc7e483
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/31740
Reviewed-by: Alexei Frolov <frolv@google.com>
This removes the generated packet proto code that was checked in and
replaces it with the proto package generated by the build.
Change-Id: I640f7e55cbd759af1672702597f15fe19fd968d5
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/26702
Reviewed-by: Wyatt Hepler <hepler@google.com>
Commit-Queue: Alexei Frolov <frolv@google.com>
proto3 now allows specifying optional fields. These fields behave
equivalently to proto2 optional fields and allow API users to detect
whether the field is present in the proto.
- Have the pw_rpc protoc plugins indicate proto3 optional field support.
- Require mypy-protobuf 1.24 or later for proto3 optional field support.
- Provide the flag for proto3 optional fields to protoc.
Change-Id: I66ba3a280cfc9b5c4d1ff42109a6e1d3d691b271
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/31320
Commit-Queue: Wyatt Hepler <hepler@google.com>
Reviewed-by: Alexei Frolov <frolv@google.com>
The callback_client implementation allows specifying a default timeout
for unary and server streaming RPCs. The timeout can be overridden for
an individual RPC call. If no response is received by the timeout, an
RpcTimeout exception is raised.
Change-Id: I932bb6ebbf3bf157adb40bc0536a529c8411c221
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/31140
Reviewed-by: Ewout van Bekkum <ewout@google.com>
Commit-Queue: Wyatt Hepler <hepler@google.com>
Split python_test_deps from python_deps. This makes the true
dependencies clearer and prevents circular dependencies in some cases
(pw_protobuf_compiler's tests depend on compiled protobufs).
Change-Id: I97dcba908995ef281850105d0cecafd0d87522e8
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/30561
Reviewed-by: Keir Mierle <keir@google.com>
Commit-Queue: Wyatt Hepler <hepler@google.com>
- Since protobuf Python packages are generated and installed from the
build directory now, add dependencies on package installs in the
build. This ensures packages are properly installed before tests or
linting.
- Instantiate all Python actions in the default toolchain. This prevents
duplicate instantiations of Python actions for each toolchain, which
causes flakiness as packages are reinstalled for each toolchain.
- Add python_deps to pw_python_action that Python actions can depend on
Python package installations.
- Add various missing dependencies between Python packages.
Change-Id: I4ec34f06c04014b3552cc1b60aa506ac9a89b329
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/30000
Commit-Queue: Wyatt Hepler <hepler@google.com>
Reviewed-by: Alexei Frolov <frolv@google.com>
Reviewed-by: Rob Mohr <mohrr@google.com>
- Provide function that prints protobufs in a repr-like style.
- Return a UnaryResponse named tuple instead of a plain tuple from unary
RPCs in pw_rpc.callback_client.
Change-Id: Iaccdb1b599ab4052b63f2dec810e768dc588fcb6
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/30020
Reviewed-by: Alexei Frolov <frolv@google.com>
Commit-Queue: Wyatt Hepler <hepler@google.com>
Since Python packages are installed by GN now, packages are installed in
the correct order. This allows local deps to be fully expressed.
Change-Id: I057cc0902b16da3480f73520281eb52391236a63
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/27901
Pigweed-Auto-Submit: Wyatt Hepler <hepler@google.com>
Reviewed-by: Rob Mohr <mohrr@google.com>
Commit-Queue: Auto-Submit <auto-submit@pigweed.google.com.iam.gserviceaccount.com>
- google.protobuf.message_factory may generate new, unique message types
rather than reusing previously generated classes. The new types are
essentially identical to others and share the same descriptor
instance. When working with pw_rpc methods, accept any message type
that shares the same descriptors.
- Move pw_rpc.descriptors.Method tests to a new file and expand them to
cover this case.
Change-Id: I46a7823e086c0bbf26ecd5db3017bed0f7ae27e0
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/27160
Commit-Queue: Wyatt Hepler <hepler@google.com>
Pigweed-Auto-Submit: Wyatt Hepler <hepler@google.com>
Reviewed-by: Alexei Frolov <frolv@google.com>
- Fix the RPC help strings, which were showing the request protobuf as
the function name.
- Use __dir__ and __getattr__ instead of actually setting members on
nested packages. This makes it simpler to update the packages after
they're created.
Change-Id: I4dc594c74c452a14a7da4520a90b461606751413
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/27060
Reviewed-by: Ewout van Bekkum <ewout@google.com>
Reviewed-by: Alexei Frolov <frolv@google.com>
Commit-Queue: Auto-Submit <auto-submit@pigweed.google.com.iam.gserviceaccount.com>
Pigweed-Auto-Submit: Wyatt Hepler <hepler@google.com>
- Load request and response protos from their own files instead of
assuming they're defined in the same module.
- Don't invoke protoc if no proto file paths are provided.
Change-Id: I44ceedf0ddcb0751368d41d75de3316cf21c34f2
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/26740
Reviewed-by: Wyatt Hepler <hepler@google.com>
Commit-Queue: Alexei Frolov <frolv@google.com>
- Add pylintrc and mypy_ini options to the pw_python_package template.
These can be used to specify configuration files to use for Pylint and
Mypy.
- Run pylint and mypy from the setup directory. This allows tools to
find per-package configuration files (if they aren't specified by the
pylintrc or mypy_ini arguments).
- Fix some incorrect import ordering that Pylint detects now that it
runs from package directories. PEP8 states imports should be grouped
by standard library, third party, and local package imports.
Change-Id: I8017341178ac5920d623ebbed4535432d69527c3
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/26700
Reviewed-by: Keir Mierle <keir@google.com>
Reviewed-by: Alexei Frolov <frolv@google.com>
Update RPC function signatures to have request proto fields as keyword
arguments. Include type annotations and default values. This allows for
tab completion of arguments in the IPython shell, as well as more
detailed and useful help messages.
Change-Id: I479a344be249a013f6325cf20b3abf0175977b9c
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/26560
Reviewed-by: Ewout van Bekkum <ewout@google.com>
Reviewed-by: Alexei Frolov <frolv@google.com>
Commit-Queue: Wyatt Hepler <hepler@google.com>
- Generate a service implementation stub in the RPC proto header. This
can be used as a reference or copied when implementing a service.
- Expand the documentation for code generation.
- Reduce code duplication in codegen.
- Remove codegen_test.py, which was not running because of a missing
unittest.main() line. The test was intended as an end-to-end test for
RPC codegen, but it was out of date and is no longer necessary because
the C++ codegen tests cover it.
Change-Id: Ie63f75da523b8746a849909d00f6a66c37767f40
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/26161
Commit-Queue: Wyatt Hepler <hepler@google.com>
Reviewed-by: Keir Mierle <keir@google.com>
- Replace the public internal method lookup generated functions with a
private array of method IDs. Create a MethodLookup class that is
friended by generated services to access this array.
- Use the generated method ID array to support method lookup in services
with multiple different protobuf implementations (Nanopb and raw).
- Ensure that the correct method implementation is selected when looking
up a method at compile time. This prevents accidentally accessing a
Nanopb method as a raw method or vice versa in a test method context.
Change-Id: I0ea334c02ae7c29e50099aa5bcfa975df625d15e
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/25740
Commit-Queue: Wyatt Hepler <hepler@google.com>
Reviewed-by: Keir Mierle <keir@google.com>
- Fix issue where an RPC method could be implemented with the signature
of a different RPC type. Pass the expected method type when methods
are created and check that it matches the type deduced from the method
signature.
- Reduce code duplication to limit the number of places where
compilation errors occur.
- Combine variable RPC method traits classes into MethodTraits.
- Move all Method creation functionality into the method classes.
- Put the final method invocation logic into one function that
supports member functions or static functions.
- Consolidate error messages for incorrect method signatures into one
place. Expand the messages and describe the expected function
signature.
Change-Id: Iae3c6d32df715e877e8c977ccd2232814ef590a7
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/25641
Commit-Queue: Wyatt Hepler <hepler@google.com>
Reviewed-by: Keir Mierle <keir@google.com>
- Rather than relying on Python entry point scripts to get paths to
protoc plugins, pass the paths to the generate_protos.py script. This
works in environments where the Python entry points are not available.
- Use unique paths for generated proto code in the GN build. This allows
multiple GN targets to generate code from the same .proto files. The
CMake build already worked like this.
- Cleanup and reduce code duplication in the CMake and GN proto
generation code.
- Support Windows builds: generate .bat wrapper script for plugins, fix
compilation issue that affects some older compilers.
Change-Id: Iab27a3d193d6008567b324dc1c4e5f540894b8ff
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/24380
Commit-Queue: Wyatt Hepler <hepler@google.com>
Reviewed-by: Keir Mierle <keir@google.com>
Reviewed-by: Zoltan Szatmary-Ban <szatmz@google.com>
This change adds a TestMethodContext class for raw RPC methods. Its API
is identical to the nanopb one, but using byte spans for requests and
responses instead of structs.
Change-Id: Icf92f026882bcb6623478762d01bfde4e2deadc2
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/23170
Commit-Queue: Alexei Frolov <frolv@google.com>
Reviewed-by: Wyatt Hepler <hepler@google.com>
This updates the pw_rpc's nanopb codegen to output a service client
class for every service in a proto file, containing member functions to
invoke each of the service's methods.
Change-Id: I6bfbb76982e7a8be03eb2f13d50085b781e880a2
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/19260
Commit-Queue: Alexei Frolov <frolv@google.com>
Reviewed-by: Wyatt Hepler <hepler@google.com>
