Rim/s1-mod
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 1 Wiki Activity

Add dependencies locally

Browse Source
This commit is contained in:
Ahrimdon
2024-02-27 03:09:30 -05:00
parent 1679ef60cc
commit 70e8a8502b
5698 changed files with 2770161 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files

184
deps/protobuf/docs/csharp/proto2.md vendored Normal file
View File

@ -0,0 +1,184 @@
As part of the 3.10 release of Google.Protobuf, experimental proto2 support has been released. This document outlines the new changes brought about to include proto2 support. This does not break existing proto3 support and users may continue to use proto3 features without changing their current code. Again the generated code and public API associated with proto2 is experimental and subject to change in the future. APIs for proto2 may be added, removed, or adjusted as feedback is received.
Generated code for proto2 may also be modified by adding, removing, or adjusting APIs as feedback is received.
### Enabling proto2 features
For information about specific proto2 features, please read the [proto2 language guide](https://developers.google.com/protocol-buffers/docs/proto).
Much like other languages, proto2 features are used with proto2 files with the syntax declaration `syntax = "proto2";`. However, please note, proto3 is still the recommended version of protobuf and proto2 support is meant for legacy system interop and advanced uses.
# Generated code
### Messages
Messages in proto2 files are very similar to their proto3 counterparts. They expose the usual property for getting and setting, but they also include properties and methods to handle field presence.
For `optional`/`required` field XYZ, a `HasXYZ` property is included for checking presence and a `ClearXYZ` method is included for clearing the value.
```proto
message Foo {
optional Bar bar = 1;
required Baz baz = 2;
}
```
```cs
var foo = new Foo();
Assert.IsNull(foo.Bar);
Assert.False(foo.HasBar);
foo.Bar = new Bar();
Assert.True(foo.HasBar);
foo.ClearBar();
```
### Messages with extension ranges
Messages which define extension ranges implement the `IExtendableMessage` interface as shown below.
See inline comments for more info.
```cs
public interface IExtendableMessage<T> : IMessage<T> where T : IExtendableMessage<T>
{
// Gets the value of a single value extension. If the extension isn't present, this returns the default value.
TValue GetExtension<TValue>(Extension<T, TValue> extension);
// Gets the value of a repeated extension. If the extension hasn't been set, this returns null to prevent unnecessary allocations.
RepeatedField<TValue> GetExtension<TValue>(RepeatedExtension<T, TValue> extension);
// Gets the value of a repeated extension. This will initialize the value of the repeated field and will never return null.
RepeatedField<TValue> GetOrInitializeExtension<TValue>(RepeatedExtension<T, TValue> extension);
// Sets the value of the extension
void SetExtension<TValue>(Extension<T, TValue> extension, TValue value);
// Returns whether the extension is present in the message
bool HasExtension<TValue>(Extension<T, TValue> extension);
// Clears the value of the extension, removing it from the message
void ClearExtension<TValue>(Extension<T, TValue> extension);
// Clears the value of the repeated extension, removing it from the message. Calling GetExtension after this will always return null.
void ClearExtension<TValue>(RepeatedExtension<T, TValue> extension);
}
```
### Extensions
Extensions are generated in static containers like reflection classes and type classes.
For example for a file called `foo.proto` containing extensions in the file scope, a
`FooExtensions` class is created containing the extensions defined in the file scope.
For easy access, this class can be used with `using static` to bring all extensions into scope.
```proto
option csharp_namespace = "FooBar";
extend Foo {
optional Baz foo_ext = 124;
}
message Baz {
extend Foo {
repeated Baz repeated_foo_ext = 125;
}
}
```
```cs
public static partial class FooExtensions {
public static readonly Extension<Foo, Baz> FooExt = /* initialization */;
}
public partial class Baz {
public partial static class Extensions {
public static readonly RepeatedExtension<Foo, Baz> RepeatedFooExt = /* initialization */;
}
}
```
```cs
using static FooBar.FooExtensions;
using static FooBar.Baz.Extensions;
var foo = new Foo();
foo.SetExtension(FooExt, new Baz());
foo.GetOrInitializeExtension(RepeatedFooExt).Add(new Baz());
```
# APIs
### Message initialization
Initialization refers to checking the status of required fields in a proto2 message. If a message is uninitialized, not all required fields are set in either the message itself or any of its submessages. In other languages, missing required fields throw errors depending on the merge method used. This could cause unforeseen errors at runtime if the incorrect method is used.
However, in this implementation, parsers and input streams don't check messages for initialization on their own and throw errors. Instead it's up to you to handle messages with missing required fields in whatever way you see fit.
Checking message initialization can be done manually via the `IsInitialized` extension method in `MessageExtensions`.
### Extension registries
Just like in Java, extension registries can be constructed to parse extensions when reading new messages
from input streams. The API is fairly similar to the Java API with some added bonuses with C# syntax sugars.
```proto
message Baz {
extend Foo {
optional Baz foo_ext = 124;
}
}
```
```cs
var registry = new ExtensionRegistry()
{
Baz.Extensions.FooExt
};
var foo = Foo.Factory.WithExtensionRegistry(registry).ParseFrom(input);
Assert.True(foo.HasExtension(Bas.Extensions.FooExt));
var fooNoRegistry = Foo.Factory.ParseFrom(input);
Assert.False(foo.HasExtension(Bas.Extensions.FooExt));
```
### Custom options
Due to their limited use and lack of type safety, the original `CustomOptions` APIs are now deprecated. Using the new generated extension identifiers, you can access extensions safely through the GetOption APIs. Note that cloneable values such as
repeated fields and messages will be deep cloned.
Example based on custom options usage example [here](https://github.com/protocolbuffers/protobuf/issues/5007#issuecomment-411604515).
```cs
foreach (var service in input.Services)
{
Console.WriteLine($" {service.Name}");
foreach (var method in service.Methods)
{
var rule = method.GetOption(AnnotationsExtensions.Http);
if (rule != null)
{
Console.WriteLine($" {method.Name}: {rule}");
}
else
{
Console.WriteLine($" {method.Name}: no HTTP binding");
}
}
}
```
### Reflection
Reflection APIs have been extended to enable accessing the new proto2 portions of the library and generated code.
* FieldDescriptor.Extension
* Gets the extension identifier behind an extension field, allowing it to be added to an ExtensionRegistry
* FieldDescriptor.IsExtension
* Returns whether a field is an extension of another type.
* FieldDescriptor.ExtendeeType
* Returns the extended type of an extension field
* IFieldAccessor.HasValue
* Returns whether a field's value is set. For proto3 fields, throws an InvalidOperationException.
* FileDescriptor.Syntax
* Gets the syntax of a file
* FileDescriptor.Extensions
* An immutable list of extensions defined in the file
* MessageDescriptor.Extensions
* An immutable list of extensions defined in the message
```cs
var extensions = Baz.Descriptor.Extensions.GetExtensionsInDeclarationOrder(Foo.Descriptor);
var registry = new ExtensionRegistry();
registry.AddRange(extensions.Select(f => f.Extension));
var baz = Foo.Descriptor.Parser.WithExtensionRegistry(registry).ParseFrom(input);
foreach (var field in extensions)
{
if (field.Accessor.HasValue(baz))
{
Console.WriteLine($"{field.Name}: {field.Accessor.GetValue(baz)}");
}
}
```

476
deps/protobuf/docs/field_presence.md vendored Normal file
View File

@ -0,0 +1,476 @@
# Application note: Field presence
This application note explains the various presence tracking disciplines for protobuf fields. It also explains the behaviour of explicit presence tracking for singular proto3 fields with basic types.
## Background
_Field presence_ is the notion of whether a protobuf field has a value. There are two different manifestations of presence for protobufs: _no presence_, where the generated message API stores field values (only), and _explicit presence_, where the API also stores whether or not a field has been set.
Historically, proto2 has mostly followed _explicit presence_, while proto3 exposes only _no presence_ semantics. Singular proto3 fields of basic types (numeric, string, bytes, and enums) which are defined with the `optional` label have _explicit presence_, like proto2 (this feature is enabled by default as release 3.15).
### Presence disciplines
_Presence disciplines_ define the semantics for translating between the _API representation_ and the _serialized representation_. The _no presence_ discipline relies upon the field value itself to make decisions at (de)serialization time, while the _explicit presence_ discipline relies upon the explicit tracking state instead.
### Presence in _tag-value stream_ (wire format) serialization
The wire format is a stream of tagged, _self-delimiting_ values. By definition, the wire format represents a sequence of _present_ values. In other words, every value found within a serialization represents a _present_ field; furthermore, the serialization contains no information about not-present values.
The generated API for a proto message includes (de)serialization definitions which translate between API types and a stream of definitionally _present_ (tag, value) pairs. This translation is designed to be forward- and backward-compatibile across changes to the message definition; however, this compatibility introduces some (perhaps surprising) considerations when deserializing wire-formatted messages:
- When serializing, fields with _no presence_ are not serialized if they contain their default value.
- For numeric types, the default is 0.
- For enums, the default is the zero-valued enumerator.
- For strings, bytes, and repeated fields, the default is the zero-length value.
- For messages, the default is the language-specific null value.
- "Empty" length-delimited values (such as empty strings) can be validly represented in serialized values: the field is "present," in the sense that it appears in the wire format. However, if the generated API does not track presence, then these values may not be re-serialized; i.e., the empty field may be "not present" after a serialization round-trip.
- When deserializing, duplicate field values may be handled in different ways depending on the field definition.
- Duplicate `repeated` fields are typically appended to the field's API representation. (Note that serializing a _packed_ repeated field produces only one, length-delimited value in the tag stream.)
- Duplicate `optional` field values follow the rule that "the last one wins."
- `oneof` fields expose the API-level invariant that only one field is set at a time. However, the wire format may include multiple (tag, value) pairs which notionally belong to the `oneof`. Similar to `optional` fields, the generated API follows the "last one wins" rule.
- Out-of-range values are not returned for enum fields in generated proto2 APIs. However, out-of-range values may be stored as _unknown fields_ in the API, even though the wire-format tag was recognized.
### Presence in _named-field mapping_ formats
Protobufs can be represented in human-readable, textual forms. Two notable formats are TextFormat (the output format produced by generated message `DebugString` methods) and JSON.
These formats have correctness requirements of their own, and are generally stricter than _tagged-value stream_ formats. However, TextFormat more closely mimics the semantics of the wire format, and does, in certain cases, provide similar semantics (for example, appending repeated name-value mappings to a repeated field). In particular, similar to the wire format, TextFormat only includes fields which are present.
JSON is a much stricter format, however, and cannot validly represent some semantics of the wire format or TextFormat.
- Notably, JSON _elements_ are semantically unordered, and each member must have a unique name. This is different from TextFormat rules for repeated fields.
- JSON may include fields that are "not present," unlike the _no presence_ discipline for other formats:
- JSON defines a `null` value, which may be used to represent a _defined but not-present field_.
- Repeated field values may be included in the formatted output, even if they are equal to the default (an empty list).
- Because JSON elements are unordered, there is no way to unambiguously interpret the "last one wins" rule.
- In most cases, this is fine: JSON elements must have unique names: repeated field values are not valid JSON, so they do not need to be resolved as they are for TextFormat.
- However, this means that it may not be possible to interpret `oneof` fields unambiguously: if multiple cases are present, they are unordered.
In theory, JSON _can_ represent presence in a semantic-preserving fashion. In practice, however, presence correctness can vary depending upon implementation choices, especially if JSON was chosen as a means to interoperate with clients not using protobufs.
### Presence in proto2 APIs
This table outlines whether presence is tracked for fields in proto2 APIs (both for generated APIs and using dynamic reflection):
Field type | Explicit Presence
-------------------------------------------- | -----------------
Singular numeric (integer or floating point) | ✔️
Singular enum | ✔️
Singular string or bytes | ✔️
Singular message | ✔️
Repeated |
Oneofs | ✔️
Maps |
Singular fields (of all types) track presence explicitly in the generated API. The generated message interface includes methods to query presence of fields. For example, the field `foo` has a corresponding `has_foo` method. (The specific name follows the same language-specific naming convention as the field accessors.) These methods are sometimes referred to as "hazzers" within the protobuf implementation.
Similar to singular fields, `oneof` fields explicitly track which one of the members, if any, contains a value. For example, consider this example `oneof`:
```protobuf
oneof foo {
int32 a = 1;
float b = 2;
}
```
Depending on the target language, the generated API would generally include several methods:
- A hazzer for the oneof: `has_foo`
- A _oneof case_ method: `foo`
- Hazzers for the members: `has_a`, `has_b`
- Getters for the members: `a`, `b`
Repeated fields and maps do not track presence: there is no distinction between an _empty_ and a _not-present_ repeated field.
### Presence in proto3 APIs
This table outlines whether presence is tracked for fields in proto3 APIs (both for generated APIs and using dynamic reflection):
Field type | `optional` | Explicit Presence
-------------------------------------------- | ---------- | -----------------
Singular numeric (integer or floating point) | No |
Singular enum | No |
Singular string or bytes | No |
Singular numeric (integer or floating point) | Yes | ✔️
Singular enum | Yes | ✔️
Singular string or bytes | Yes | ✔️
Singular message | Yes | ✔️
Singular message | No | ✔️
Repeated | N/A |
Oneofs | N/A | ✔️
Maps | N/A |
Similar to proto2 APIs, proto3 does not track presence explicitly for repeated fields. Without the `optional` label, proto3 APIs do not track presence for basic types (numeric, string, bytes, and enums), either. Oneof fields affirmatively expose presence, although the same set of hazzer methods may not generated as in proto2 APIs.
Under the _no presence_ discipline, the default value is synonymous with "not present" for purposes of serialization. To notionally "clear" a field (so it won't be serialized), an API user would set it to the default value.
The default value for enum-typed fields under _no presence_ is the corresponding 0-valued enumerator. Under proto3 syntax rules, all enum types are required to have an enumerator value which maps to 0. By convention, this is an `UNKNOWN` or similarly-named enumerator. If the zero value is notionally outside the domain of valid values for the application, this behavior can be thought of as tantamount to _explicit presence_.
## Semantic differences
The _no presence_ serialization discipline results in visible differences from the _explicit presence_ tracking discipline, when the default value is set. For a singular field with numeric, enum, or string type:
- _No presence_ discipline:
- Default values are not serialized.
- Default values are _not_ merged-from.
- To "clear" a field, it is set to its default value.
- The default value may mean:
- the field was explicitly set to its default value, which is valid in the application-specific domain of values;
- the field was notionally "cleared" by setting its default; or
- the field was never set.
- _Explicit presence_ discipline:
- Explicitly set values are always serialized, including default values.
- Un-set fields are never merged-from.
- Explicitly set fields -- including default values -- _are_ merged-from.
- A generated `has_foo` method indicates whether or not the field `foo` has been set (and not cleared).
- A generated `clear_foo` method must be used to clear (i.e., un-set) the value.
### Considerations for merging
Under the _no presence_ rules, it is effectively impossible for a target field to merge-from its default value (using the protobuf's API merging functions). This is because default values are skipped, similar to the _no presence_ serialization discipline. Merging only updates the target (merged-to) message using the non-skipped values from the update (merged-from) message.
The difference in merging behavior has further implications for protocols which rely on partial "patch" updates. If field presence is not tracked, then an update patch alone cannot represent an update to the default value, because only non-default values are merged-from.
Updating to set a default value in this case requires some external mechanism, such as `FieldMask`. However, if presence _is_ tracked, then all explicitly-set values -- even default values -- will be merged into the target.
### Considerations for change-compatibility
Changing a field between _explicit presence_ and _no presence_ is a binary-compatible change for serialized values in wire format. However, the serialized representation of the message may differ, depending on which version of the message definition was used for serialization. Specifically, when a "sender" explicitly sets a field to its default value:
- The serialized value following _no presence_ discipline does not contain the default value, even though it was explicitly set.
- The serialized value following _explicit presence_ discipline contains every "present" field, even if it contains the default value.
This change may or may not be safe, depending on the application's semantics. For example, consider two clients with different versions of a message definition.
Client A uses this definition of the message, which follows the _explicit presence_ serialization discipline for field `foo`:
```protobuf
syntax = "proto3";
message Msg {
optional int32 foo = 1;
}
```
Client B uses a definition of the same message, except that it follows the _no presence_ discipline:
```protobuf
syntax = "proto3";
message Msg {
int32 foo = 1;
}
```
Now, consider a scenario where client A observes `foo`'s presence as the clients repeatedly exchange the "same" message by deserializing and reserializing:
```protobuf
// Client A:
Msg m_a;
m_a.set_foo(1); // non-default value
assert(m_a.has_foo()); // OK
Send(m_a.SerializeAsString()); // to client B
// Client B:
Msg m_b;
m_b.ParseFromString(Receive()); // from client A
assert(m_b.foo() == 1); // OK
Send(m_b.SerializeAsString()); // to client A
// Client A:
m_a.ParseFromString(Receive()); // from client B
assert(m_a.foo() == 1); // OK
assert(m_a.has_foo()); // OK
m_a.set_foo(0); // default value
Send(m_a.SerializeAsString()); // to client B
// Client B:
Msg m_b;
m_b.ParseFromString(Receive()); // from client A
assert(m_b.foo() == 0); // OK
Send(m_b.SerializeAsString()); // to client A
// Client A:
m_a.ParseFromString(Receive()); // from client B
assert(m_a.foo() == 0); // OK
assert(m_a.has_foo()); // FAIL
```
If client A depends on _explicit presence_ for `foo`, then a "round trip" through client B will be lossy from the perspective of client A. In the example, this is not a safe change: client A requires (by `assert`) that the field is present; even without any modifications through the API, that requirement fails in a value- and peer-dependent case.
## How to enable _explicit presence_ in proto3
These are the general steps to use field tracking support for proto3:
1. Add an `optional` field to a `.proto` file.
1. Run `protoc` (at least v3.15, or v3.12 using `--experimental_allow_proto3_optional` flag).
1. Use the generated "hazzer" methods and "clear" methods in application code, instead of comparing or setting default values.
### `.proto` file changes
This is an example of a proto3 message with fields which follow both _no presence_ and _explicit presence_ semantics:
```protobuf
syntax = "proto3";
package example;
message MyMessage {
// No presence:
int32 not_tracked = 1;
// Explicit presence:
optional int32 tracked = 2;
}
```
### `protoc` invocation
Presence tracking for proto3 messages is enabled by default [since v3.15.0](https://github.com/protocolbuffers/protobuf/releases/tag/v3.15.0) release, formerly up until [v3.12.0](https://github.com/protocolbuffers/protobuf/releases/tag/v3.12.0) the `--experimental_allow_proto3_optional` flag was required when using presence tracking with protoc.
### Using the generated code
The generated code for proto3 fields with _explicit presence_ (the `optional` label) will be the same as it would be in a proto2 file.
This is the definition used in the "no presence" examples below:
```protobuf
syntax = "proto3";
package example;
message Msg {
int32 foo = 1;
}
```
This is the definition used in the "explicit presence" examples below:
```protobuf
syntax = "proto3";
package example;
message Msg {
optional int32 foo = 1;
}
```
In the examples, a function `GetProto` constructs and returns a message of type `Msg` with unspecified contents.
#### C++ example
No presence:
```C++
Msg m = GetProto();
if (m.foo() != 0) {
// "Clear" the field:
m.set_foo(0);
} else {
// Default value: field may not have been present.
m.set_foo(1);
}
```
Explicit presence:
```C++
Msg m = GetProto();
if (m.has_foo()) {
// Clear the field:
m.clear_foo();
} else {
// Field is not present, so set it.
m.set_foo(1);
}
```
#### C# example
No presence:
```C#
var m = GetProto();
if (m.Foo != 0) {
// "Clear" the field:
m.Foo = 0;
} else {
// Default value: field may not have been present.
m.Foo = 1;
}
```
Explicit presence:
```C#
var m = GetProto();
if (m.HasFoo) {
// Clear the field:
m.ClearFoo();
} else {
// Field is not present, so set it.
m.Foo = 1;
}
```
#### Go example
No presence:
```go
m := GetProto()
if m.Foo != 0 {
// "Clear" the field:
m.Foo = 0
} else {
// Default value: field may not have been present.
m.Foo = 1
}
```
Explicit presence:
```go
m := GetProto()
if m.Foo != nil {
// Clear the field:
m.Foo = nil
} else {
// Field is not present, so set it.
m.Foo = proto.Int32(1)
}
```
#### Java example
These examples use a `Builder` to demonstrate clearing. Simply checking presence and getting values from a `Builder` follows the same API as the message type.
No presence:
```java
Msg.Builder m = GetProto().toBuilder();
if (m.getFoo() != 0) {
// "Clear" the field:
m.setFoo(0);
} else {
// Default value: field may not have been present.
m.setFoo(1);
}
```
Explicit presence:
```java
Msg.Builder m = GetProto().toBuilder();
if (m.hasFoo()) {
// Clear the field:
m.clearFoo()
} else {
// Field is not present, so set it.
m.setFoo(1);
}
```
#### Python example
No presence:
```python
m = example.Msg()
if m.foo != 0:
# "Clear" the field:
m.foo = 0
else:
# Default value: field may not have been present.
m.foo = 1
```
Explicit presence:
```python
m = example.Msg()
if m.HasField('foo'):
# Clear the field:
m.ClearField('foo')
else:
# Field is not present, so set it.
m.foo = 1
```
#### Ruby example
No presence:
```ruby
m = Msg.new
if m.foo != 0
# "Clear" the field:
m.foo = 0
else
# Default value: field may not have been present.
m.foo = 1
end
```
Explicit presence:
```ruby
m = Msg.new
if m.has_foo?
# Clear the field:
m.clear_foo
else
# Field is not present, so set it.
m.foo = 1
end
```
#### Javascript example
No presence:
```js
var m = new Msg();
if (m.getFoo() != 0) {
// "Clear" the field:
m.setFoo(0);
} else {
// Default value: field may not have been present.
m.setFoo(1);
}
```
Explicit presence:
```js
var m = new Msg();
if (m.hasFoo()) {
// Clear the field:
m.clearFoo()
} else {
// Field is not present, so set it.
m.setFoo(1);
}
```
#### Objective C example
No presence:
```Objective-C
Msg *m = [[Msg alloc] init];
if (m.foo != 0) {
// "Clear" the field:
m.foo = 0;
} else {
// Default value: field may not have been present.
m.foo = 1;
}
```
Explicit presence:
```Objective-C
Msg *m = [[Msg alloc] init];
if (m.hasFoo()) {
// Clear the field:
[m clearFoo];
} else {
// Field is not present, so set it.
[m setFoo:1];
}
```

400
deps/protobuf/docs/implementing_proto3_presence.md vendored Normal file
View File

@ -0,0 +1,400 @@
# How To Implement Field Presence for Proto3
Protobuf release 3.12 adds experimental support for `optional` fields in
proto3. Proto3 optional fields track presence like in proto2. For background
information about what presence tracking means, please see
[docs/field_presence](field_presence.md).
## Document Summary
This document is targeted at developers who own or maintain protobuf code
generators. All code generators will need to be updated to support proto3
optional fields. First-party code generators developed by Google are being
updated already. However third-party code generators will need to be updated
independently by their authors. This includes:
- implementations of Protocol Buffers for other languages.
- alternate implementations of Protocol Buffers that target specialized use
cases.
- RPC code generators that create generated APIs for service calls.
- code generators that implement some utility code on top of protobuf generated
classes.
While this document speaks in terms of "code generators", these same principles
apply to implementations that dynamically generate a protocol buffer API "on the
fly", directly from a descriptor, in languages that support this kind of usage.
## Background
Presence tracking was added to proto3 in response to user feedback, both from
inside Google and [from open-source
users](https://github.com/protocolbuffers/protobuf/issues/1606). The [proto3
wrapper
types](https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/wrappers.proto)
were previously the only supported presence mechanism for proto3. Users have
pointed to both efficiency and usability issues with the wrapper types.
Presence in proto3 uses exactly the same syntax and semantics as in proto2.
Proto3 Fields marked `optional` will track presence like proto2, while fields
without any label (known as "singular fields"), will continue to omit presence
information. The `optional` keyword was chosen to minimize differences with
proto2.
Unfortunately, for the current descriptor protos and `Descriptor` API (as of
3.11.4) it is not possible to use the same representation as proto2. Proto3
descriptors already use `LABEL_OPTIONAL` for proto3 singular fields, which do
not track presence. There is a lot of existing code that reflects over proto3
protos and assumes that `LABEL_OPTIONAL` in proto3 means "no presence." Changing
the semantics now would be risky, since old software would likely drop proto3
presence information, which would be a data loss bug.
To minimize this risk we chose a descriptor representation that is semantically
compatible with existing proto3 reflection. Every proto3 optional field is
placed into a one-field `oneof`. We call this a "synthetic" oneof, as it was not
present in the source `.proto` file.
Since oneof fields in proto3 already track presence, existing proto3
reflection-based algorithms should correctly preserve presence for proto3
optional fields with no code changes. For example, the JSON and TextFormat
parsers/serializers in C++ and Java did not require any changes to support
proto3 presence. This is the major benefit of synthetic oneofs.
This design does leave some cruft in descriptors. Synthetic oneofs are a
compatibility measure that we can hopefully clean up in the future. For now
though, it is important to preserve them across different descriptor formats and
APIs. It is never safe to drop synthetic oneofs from a proto schema. Code
generators can (and should) skip synthetic oneofs when generating a user-facing
API or user-facing documentation. But for any schema representation that is
consumed programmatically, it is important to keep the synthetic oneofs around.
In APIs it can be helpful to offer separate accessors that refer to "real"
oneofs (see [API Changes](#api-changes) below). This is a convenient way to omit
synthetic oneofs in code generators.
## Updating a Code Generator
When a user adds an `optional` field to proto3, this is internally rewritten as
a one-field oneof, for backward-compatibility with reflection-based algorithms:
```protobuf
syntax = "proto3";
message Foo {
// Experimental feature, not generally supported yet!
optional int32 foo = 1;
// Internally rewritten to:
// oneof _foo {
// int32 foo = 1 [proto3_optional=true];
// }
//
// We call _foo a "synthetic" oneof, since it was not created by the user.
}
```
As a result, the main two goals when updating a code generator are:
1. Give `optional` fields like `foo` normal field presence, as described in
[docs/field_presence](field_presence.md) If your implementation already
supports proto2, a proto3 `optional` field should use exactly the same API
and internal implementation as proto2 `optional`.
2. Avoid generating any oneof-based accessors for the synthetic oneof. Its only
purpose is to make reflection-based algorithms work properly if they are
not aware of proto3 presence. The synthetic oneof should not appear anywhere
in the generated API.
### Satisfying the Experimental Check
If you try to run `protoc` on a file with proto3 `optional` fields, you will get
an error because the feature is still experimental:
```
$ cat test.proto
syntax = "proto3";
message Foo {
// Experimental feature, not generally supported yet!
optional int32 a = 1;
}
$ protoc --cpp_out=. test.proto
test.proto: This file contains proto3 optional fields, but --experimental_allow_proto3_optional was not set.
```
There are two options for getting around this error:
1. Pass `--experimental_allow_proto3_optional` to protoc.
2. Make your filename (or a directory name) contain the string
`test_proto3_optional`. This indicates that the proto file is specifically
for testing proto3 optional support, so the check is suppressed.
These options are demonstrated below:
```
# One option:
$ ./src/protoc test.proto --cpp_out=. --experimental_allow_proto3_optional
# Another option:
$ cp test.proto test_proto3_optional.proto
$ ./src/protoc test_proto3_optional.proto --cpp_out=.
$
```
The experimental check will be removed in a future release, once we are ready
to make this feature generally available. Ideally this will happen for the 3.13
release of protobuf, sometime in mid-2020, but there is not a specific date set
for this yet. Some of the timing will depend on feedback we get from the
community, so if you have questions or concerns please get in touch via a
GitHub issue.
### Signaling That Your Code Generator Supports Proto3 Optional
If you now try to invoke your own code generator with the test proto, you will
run into a different error:
```
$ ./src/protoc test_proto3_optional.proto --my_codegen_out=.
test_proto3_optional.proto: is a proto3 file that contains optional fields, but
code generator --my_codegen_out hasn't been updated to support optional fields in
proto3. Please ask the owner of this code generator to support proto3 optional.
```
This check exists to make sure that code generators get a chance to update
before they are used with proto3 `optional` fields. Without this check an old
code generator might emit obsolete generated APIs (like accessors for a
synthetic oneof) and users could start depending on these. That would create
a legacy migration burden once a code generator actually implements the feature.
To signal that your code generator supports `optional` fields in proto3, you
need to tell `protoc` what features you support. The method for doing this
depends on whether you are using the C++
`google::protobuf::compiler::CodeGenerator`
framework or not.
If you are using the CodeGenerator framework:
```c++
class MyCodeGenerator : public google::protobuf::compiler::CodeGenerator {
// Add this method.
uint64_t GetSupportedFeatures() const override {
// Indicate that this code generator supports proto3 optional fields.
// (Note: don't release your code generator with this flag set until you
// have actually added and tested your proto3 support!)
return FEATURE_PROTO3_OPTIONAL;
}
}
```
If you are generating code using raw `CodeGeneratorRequest` and
`CodeGeneratorResponse` messages from `plugin.proto`, the change will be very
similar:
```c++
void GenerateResponse() {
CodeGeneratorResponse response;
response.set_supported_features(CodeGeneratorResponse::FEATURE_PROTO3_OPTIONAL);
// Generate code...
}
```
Once you have added this, you should now be able to successfully use your code
generator to generate a file containing proto3 optional fields:
```
$ ./src/protoc test_proto3_optional.proto --my_codegen_out=.
```
### Updating Your Code Generator
Now to actually add support for proto3 optional to your code generator. The goal
is to recognize proto3 optional fields as optional, and suppress any output from
synthetic oneofs.
If your code generator does not currently support proto2, you will need to
design an API and implementation for supporting presence in scalar fields.
Generally this means:
- allocating a bit inside the generated class to represent whether a given field
is present or not.
- exposing a `has_foo()` method for each field to return the value of this bit.
- make the parser set this bit when a value is parsed from the wire.
- make the serializer test this bit to decide whether to serialize.
If your code generator already supports proto2, then most of your work is
already done. All you need to do is make sure that proto3 optional fields have
exactly the same API and behave in exactly the same way as proto2 optional
fields.
From experience updating several of Google's code generators, most of the
updates that are required fall into one of several patterns. Here we will show
the patterns in terms of the C++ CodeGenerator framework. If you are using
`CodeGeneratorRequest` and `CodeGeneratorReply` directly, you can translate the
C++ examples to your own language, referencing the C++ implementation of these
methods where required.
#### To test whether a field should have presence
Old:
```c++
bool MessageHasPresence(const google::protobuf::Descriptor* message) {
return message->file()->syntax() ==
google::protobuf::FileDescriptor::SYNTAX_PROTO2;
}
```
New:
```c++
// Presence is no longer a property of a message, it's a property of individual
// fields.
bool FieldHasPresence(const google::protobuf::FieldDescriptor* field) {
return field->has_presence();
// Note, the above will return true for fields in a oneof.
// If you want to filter out oneof fields, write this instead:
// return field->has_presence && !field->real_containing_oneof()
}
```
#### To test whether a field is a member of a oneof
Old:
```c++
bool FieldIsInOneof(const google::protobuf::FieldDescriptor* field) {
return field->containing_oneof() != nullptr;
}
```
New:
```c++
bool FieldIsInOneof(const google::protobuf::FieldDescriptor* field) {
// real_containing_oneof() returns nullptr for synthetic oneofs.
return field->real_containing_oneof() != nullptr;
}
```
#### To iterate over all oneofs
Old:
```c++
bool IterateOverOneofs(const google::protobuf::Descriptor* message) {
for (int i = 0; i < message->oneof_decl_count(); i++) {
const google::protobuf::OneofDescriptor* oneof = message->oneof(i);
// ...
}
}
```
New:
```c++
bool IterateOverOneofs(const google::protobuf::Descriptor* message) {
// Real oneofs are always first, and real_oneof_decl_count() will return the
// total number of oneofs, excluding synthetic oneofs.
for (int i = 0; i < message->real_oneof_decl_count(); i++) {
const google::protobuf::OneofDescriptor* oneof = message->oneof(i);
// ...
}
}
```
## Updating Reflection
If your implementation offers reflection, there are a few other changes to make:
### API Changes
The API for reflecting over fields and oneofs should make the following changes.
These match the changes implemented in C++ reflection.
1. Add a `FieldDescriptor::has_presence()` method returning `bool`
(adjusted to your language's naming convention). This should return true
for all fields that have explicit presence, as documented in
[docs/field_presence](field_presence.md). In particular, this includes
fields in a oneof, proto2 scalar fields, and proto3 `optional` fields.
This accessor will allow users to query what fields have presence without
thinking about the difference between proto2 and proto3.
2. As a corollary of (1), please do *not* expose an accessor for the
`FieldDescriptorProto.proto3_optional` field. We want to avoid having
users implement any proto2/proto3-specific logic. Users should use the
`has_presence()` function instead.
3. You may also wish to add a `FieldDescriptor::has_optional_keyword()` method
returning `bool`, which indicates whether the `optional` keyword is present.
Message fields will always return `true` for `has_presence()`, so this method
can allow a user to know whether the user wrote `optional` or not. It can
occasionally be useful to have this information, even though it does not
change the presence semantics of the field.
4. If your reflection API may be used for a code generator, you may wish to
implement methods to help users tell the difference between real and
synthetic oneofs. In particular:
- `OneofDescriptor::is_synthetic()`: returns true if this is a synthetic
oneof.
- `FieldDescriptor::real_containing_oneof()`: like `containing_oneof()`,
but returns `nullptr` if the oneof is synthetic.
- `Descriptor::real_oneof_decl_count()`: like `oneof_decl_count()`, but
returns the number of real oneofs only.
### Implementation Changes
Proto3 `optional` fields and synthetic oneofs must work correctly when
reflected on. Specifically:
1. Reflection for synthetic oneofs should work properly. Even though synthetic
oneofs do not really exist in the message, you can still make reflection work
as if they did. In particular, you can make a method like
`Reflection::HasOneof()` or `Reflection::GetOneofFieldDescriptor()` look at
the hasbit to determine if the oneof is present or not.
2. Reflection for proto3 optional fields should work properly. For example, a
method like `Reflection::HasField()` should know to look for the hasbit for a
proto3 `optional` field. It should not be fooled by the synthetic oneof into
thinking that there is a `case` member for the oneof.
Once you have updated reflection to work properly with proto3 `optional` and
synthetic oneofs, any code that *uses* your reflection interface should work
properly with no changes. This is the benefit of using synthetic oneofs.
In particular, if you have a reflection-based implementation of protobuf text
format or JSON, it should properly support proto3 optional fields without any
changes to the code. The fields will look like they all belong to a one-field
oneof, and existing proto3 reflection code should know how to test presence for
fields in a oneof.
So the best way to test your reflection changes is to try round-tripping a
message through text format, JSON, or some other reflection-based parser and
serializer, if you have one.
### Validating Descriptors
If your reflection implementation supports loading descriptors at runtime,
you must verify that all synthetic oneofs are ordered after all "real" oneofs.
Here is the code that implements this validation step in C++, for inspiration:
```c++
// Validation that runs for each message.
// Synthetic oneofs must be last.
int first_synthetic = -1;
for (int i = 0; i < message->oneof_decl_count(); i++) {
const OneofDescriptor* oneof = message->oneof_decl(i);
if (oneof->is_synthetic()) {
if (first_synthetic == -1) {
first_synthetic = i;
}
} else {
if (first_synthetic != -1) {
AddError(message->full_name(), proto.oneof_decl(i),
DescriptorPool::ErrorCollector::OTHER,
"Synthetic oneofs must be after all other oneofs");
}
}
}
if (first_synthetic == -1) {
message->real_oneof_decl_count_ = message->oneof_decl_count_;
} else {
message->real_oneof_decl_count_ = first_synthetic;
}
```

18
deps/protobuf/docs/jvm_aot.md vendored Normal file
View File

@ -0,0 +1,18 @@
# Ahead Of Time (AOT) compilation for the Java Virtual Machine (JVM)"
Ahead Of Time (AOT) compilation build tools such as those provided by [GraalVM's `native-image`](https://www.graalvm.org/reference-manual/native-image/) can require some configuration when using protobuf.
Protobuf for the JVM uses reflection and some of its target classes are not possible to determine in advance.
Historically, there were good reasons to use reflection based on APIs that were published effectively requiring them, and this situation is unlikely to change.
[The Lite version of protobuf for the JVM](https://github.com/protocolbuffers/protobuf/blob/master/java/lite.md)
avoids reflection and may be better suited for use with AOT compilation tooling. This Lite version was originally targeted for use on Android which has similar AOT compilation
goals as GraalVM's native-image tool.
## GraalVM native-image
This section addresses GraalVM's `native-image` configuration specifically as this AOT compilation tool due to its popularity. The `native-image` tool can be configured
with respect to: the [Java Native Interface](https://en.wikipedia.org/wiki/Java_Native_Interface) (JNI), http proxying, reflection, and other resources. While these
considerations can be manually declared as JSON files, we recommend that a JVM application is exercised along with
[the assisted configuration agent](https://www.graalvm.org/reference-manual/native-image/BuildConfiguration/#assisted-configuration-of-native-image-builds). The agent
will generate files that you can then subsequently point at when invoking `native-image`. We recommend that the generated files are retained with a project's source
code.

306
deps/protobuf/docs/options.md vendored Normal file
View File

@ -0,0 +1,306 @@
# Protobuf Global Extension Registry
This file contains a global registry of known extensions for descriptor.proto,
so that any developer who wishes to use multiple 3rd party projects, each with
their own extensions, can be confident that there won't be collisions in
extension numbers.
If you need an extension number for your custom option (see [custom options](
https://developers.google.com/protocol-buffers/docs/proto#customoptions)),
please [send us a pull request](https://github.com/protocolbuffers/protobuf/pulls) to
add an entry to this doc, or [create an issue](https://github.com/protocolbuffers/protobuf/issues)
with info about your project (name and website) so we can add an entry for you.
## Existing Registered Extensions
1. C# port of protocol buffers
* Website: https://github.com/jskeet/protobuf-csharp-port
* Extensions: 1000
1. Perl/XS port of protocol buffers
* Website: http://code.google.com/p/protobuf-perlxs
* Extensions: 1001
1. Objective-C port of protocol buffers
* Website: http://code.google.com/p/protobuf-objc
* Extensions: 1002
1. Google Wave Federation Protocol open-source release (FedOne)
* Website: http://code.google.com/p/wave-protocol
* Extensions: 1003
1. PHP code generator plugin
* Website: ???
* Extensions: 1004
1. GWT code generator plugin (third-party!)
* Website: http://code.google.com/p/protobuf-gwt/
* Extensions: 1005
1. Unix Domain RPC code generator plugin
* Website: http://go/udrpc
* Extensions: 1006
1. Object-C generator plugin (Plausible Labs)
* Website: http://www.plausible.coop
* Extensions: 1007
1. TBD (code42.com)
* Website: ???
* Extensions: 1008
1. Goby Underwater Autonomy Project
* Website: https://github.com/GobySoft/goby
* Extensions: 1009
1. Nanopb
* Website: http://kapsi.fi/~jpa/nanopb
* Extensions: 1010
1. Bluefin AUV Communication Extensions
* Website: http://www.bluefinrobotics.com
* Extensions: 1011
1. Dynamic Compact Control Language
* Website: http://github.com/GobySoft/dccl
* Extensions: 1012
1. ScaleOut StateServer® Native C++ API
* Website: http://www.scaleoutsoftware.com
* Extensions: 1013
1. FoundationDB SQL Layer
* Website: https://github.com/FoundationDB/sql-layer
* Extensions: 1014
1. Fender
* Website: https://github.com/hassox/fender
* Extensions: 1015
1. Vortex
* Website: http://www.prismtech.com/vortex
* Extensions: 1016
1. tresorit
* Website: https://tresorit.com/
* Extensions: 1017
1. CRIU (Checkpoint Restore In Userspace)
* Website: http://criu.org/Main_Page
* Extensions: 1018
1. protobuf-c
* Website: https://github.com/protobuf-c/protobuf-c
* Extensions: 1019
1. ScalaPB
* Website: https://scalapb.github.io/
* Extensions: 1020
1. protoc-gen-bq-schema
* Website: https://github.com/GoogleCloudPlatform/protoc-gen-bq-schema
* Extensions: 1021
1. grpc-gateway
* Website: https://github.com/gengo/grpc-gateway
* Extensions: 1022
1. Certificate Transparency
* Website: https://github.com/google/certificate-transparency
* Extensions: 1023
1. JUNOS Router Telemetry
* Website: http://www.juniper.net
* Extensions: 1024
1. Spine Event Engine
* Website: https://github.com/SpineEventEngine/core-java
* Extensions: 1025
1. Aruba cloud platform
* Website: ???
* Extensions: 1026 -> 1030
1. Voltha
* Website: ???
* Extensions: 1031 -> 1033
1. gator
* Website: ???
* Extensions: 1034
1. protoc-gen-flowtypes
* Website: https://github.com/tmc/grpcutil/tree/master/protoc-gen-flowtypes
* Extensions: 1035
1. ProfaneDB
* Website: https://gitlab.com/ProfaneDB/ProfaneDB
* Extensions: 1036
1. protobuf-net
* Website: https://github.com/mgravell/protobuf-net
* Extensions: 1037
1. FICO / StreamEngine
* Website: http://www.fico.com/
* Extensions: 1038
1. GopherJS
* Website: https://github.com/johanbrandhorst/protobuf
* Extensions: 1039
1. ygot
* Website: https://github.com/openconfig/ygot
* Extensions: 1040
1. go-grpcmw
* Website: https://github.com/MarquisIO/go-grpcmw
* Extensions: 1041
1. grpc-gateway protoc-gen-swagger
* Website: https://github.com/grpc-ecosystem/grpc-gateway
* Extensions: 1042
1. AN Message
* Website: TBD
* Extensions: 1043
1. protofire
* Website: https://github.com/ribrdb/protofire
* Extensions: 1044
1. Gravity
* Website: https://github.com/aphysci/gravity
* Extensions: 1045
1. SEMI Standards – I&C Technical Committee
* Website: http://downloads.semi.org/web/wstdsbal.nsf/9c2b317e76523cca88257641005a47f5/88a5863a580e323088256e7b00707489!OpenDocument
* Extensions: 1046
1. Elixir plugin
* Website: https://github.com/tony612/grpc-elixir
* Extensions: 1047
1. API client generators
* Website: ???
* Extensions: 1048-1056
1. Netifi Proteus
* Website: https://github.com/netifi-proteus
* Extensions: 1057
1. CGSN Mooring Project
* Website: https://bitbucket.org/ooicgsn/cgsn-mooring
* Extensions: 1058
1. Container Storage Interface
* Website: https://github.com/container-storage-interface/spec
* Extensions: 1059-1069
1. TwirpQL Plugin
* Website: https://twirpql.dev
* Extensions: 1070
1. Protoc-gen-validate
* Website: https://github.com/envoyproxy/protoc-gen-validate
* Extensions: 1071
1. Protokt
* Website: https://github.com/open-toast/protokt
* Extensions: 1072
1. Dart port of protocol buffers
* Website https://github.com/dart-lang/protobuf
* Extensions: 1073
1. Ocaml-protoc-plugin
* Website: https://github.com/issuu/ocaml-protoc-plugin
* Extensions: 1074
1. Analyze Re Graphene
* Website: https://analyzere.com
* Extensions: 1075
1. Wire since and until
* Website: https://square.github.io/wire/
* Extensions: 1076, 1077
1. Bazel, Failure Details
* Website: https://github.com/bazelbuild/bazel
* Extensions: 1078
1. grpc-graphql-gateway
* Website: https://github.com/ysugimoto/grpc-graphql-gateway
* Extensions: 1079
1. Cloudstate
* Website: https://cloudstate.io
* Extensions: 1080-1084
1. SummaFT protoc-plugins
* Website: https://summaft.com/
* Extensions: 1085
1. ADLINK EdgeSDK
* Website: https://www.adlinktech.com/en/Edge-SDK-IoT
* Extensions: 1086
1. Wire wire_package
* Website: https://square.github.io/wire/
* Extensions: 1087
1. Confluent Schema Registry
* Website: https://github.com/confluentinc/schema-registry
* Extensions: 1088
1. ScalaPB Validate
* Website: https://scalapb.github.io/docs/validation
* Extension: 1089
1. Astounding (Currently Private)
* Website: https://github.com/PbPipes/Astounding
* Extension: 1090
1. Protoc-gen-psql
* Website: https://github.com/Intrinsec/protoc-gen-psql
* Extension: 1091-1101
1. Protoc-gen-sanitize
* Website: https://github.com/Intrinsec/protoc-gen-sanitize
* Extension: 1102-1106
1. Coach Client Connect (planned release in March 2021)
* Website: https://www.coachclientconnect.com
* Extension: 1107
1. Kratos API Errors
* Website: https://go-kratos.dev
* Extension: 1108
1. Glitchdot (Currently Private)
* Website: https://go.glitchdot.com
* Extension: 1109
1. eigr/protocol
* Website: https://eigr.io
* Extension: 1110-1114
1. Container Object Storage Interface (COSI)
* Website: https://github.com/kubernetes-sigs/container-object-storage-interface-spec
* Extension: 1115-1124
1. Protoc-gen-jsonschema
* Website: https://github.com/chrusty/protoc-gen-jsonschema
* Extension: 1125-1129
1. Protoc-gen-checker
* Website: https://github.com/Intrinsec/protoc-gen-checker
* Extension: 1130-1139
1. Protoc-gen-go-svc
* Website: https://github.com/dane/protoc-gen-go-svc
* Extension: 1140
1. Embedded Proto
* Website: https://EmbeddedProto.com
* Extension: 1141

304
deps/protobuf/docs/performance.md vendored Normal file
View File

@ -0,0 +1,304 @@
# Protobuf Performance
The following benchmark test results were produced on a workstation utilizing an Intel® Xeon® Processor E5-2630 with 32GB of RAM.
This table contains the results of three separate languages:
* **C++** - For C++, there are three parsing methods:
* **new** - This is for using a new operator for creating a message instance.
* **new arena** - This is for using arena for creating a new message instance.
* **reuse** - This is for reusing the same message instance for parsing.
* **Java** - For Java, there are three parsing/serialization methods:
* **byte[]** - This is for parsing from a Byte Array.
* **ByteString** - This is for parsing from a
com.google.protobuf.ByteString.
* **InputStream** - This is for parsing from an InputStream.
* **Python** - For Python, there are three types of Python protobuf for testing:
* **C++-generated-code** - This is for using C++ generated code of the
proto file as a dynamic linked library.
* **C++-reflection** - This is for using C++ reflection, for which there's no
generated code, but still using C++ protobuf library as a dynamic linked
library.
* **pure-Python** - This is for the pure version of Python, which does not link with
any C++ protobuf library.
## Parsing performance
<table>
<tbody><tr>
<th rowspan="2"> </th>
<th colspan="3" rowspan="1">C++</th>
<th colspan="3" rowspan="1">C++ with tcmalloc</th>
<th colspan="3" rowspan="1">java</th>
<th colspan="3" rowspan="1">python</th>
</tr>
<tr>
<th colspan="1">new</th>
<th colspan="1">new arena</th>
<th colspan="1">reuse</th>
<th colspan="1">new</th>
<th colspan="1">new arena</th>
<th colspan="1">reuse</th>
<th colspan="1">byte[]</th>
<th colspan="1">ByteString</th>
<th colspan="1">InputStream</th>
<th colspan="1">C++-generated-code</th>
<th colspan="1">C++-reflection</th>
<th colspan="1">pure-Python</th>
</tr>
<tr>
<td>google_message1_proto2</td>
<td>368.717MB/s</td>
<td>261.847MB/s</td>
<td>799.403MB/s</td>
<td>645.183MB/s</td>
<td>441.023MB/s</td>
<td>1.122GB/s</td>
<td>425.437MB/s</td>
<td>425.937MB/s</td>
<td>251.018MB/s</td>
<td>82.8314MB/s</td>
<td>47.6763MB/s</td>
<td>3.76299MB/s</td>
</tr>
<tr>
<td>google_message1_proto3</td>
<td>294.517MB/s</td>
<td>229.116MB/s</td>
<td>469.982MB/s</td>
<td>434.510MB/s</td>
<td>394.701MB/s</td>
<td>591.931MB/s</td>
<td>357.597MB/s</td>
<td>378.568MB/s</td>
<td>221.676MB/s</td>
<td>82.0498MB/s</td>
<td>39.9467MB/s</td>
<td>3.77751MB/s</td>
</tr>
<tr>
<td>google_message2</td>
<td>277.242MB/s</td>
<td>347.611MB/s</td>
<td>793.67MB/s</td>
<td>503.721MB/s</td>
<td>596.333MB/s</td>
<td>922.533MB/s</td>
<td>416.778MB/s</td>
<td>419.543MB/s</td>
<td>367.145MB/s</td>
<td>241.46MB/s</td>
<td>71.5723MB/s</td>
<td>2.73538MB/s</td>
</tr>
<tr>
<td>google_message3_1</td>
<td>213.478MB/s</td>
<td>291.58MB/s</td>
<td>543.398MB/s</td>
<td>539.704MB/s</td>
<td>717.300MB/s</td>
<td>927.333MB/s</td>
<td>684.241MB/s</td>
<td>704.47MB/s</td>
<td>648.624MB/s</td>
<td>209.036MB/s</td>
<td>142.356MB/s</td>
<td>15.3324MB/s</td>
</tr>
<tr>
<td>google_message3_2</td>
<td>672.685MB/s</td>
<td>802.767MB/s</td>
<td>1.21505GB/s</td>
<td>985.790MB/s</td>
<td>1.136GB/s</td>
<td>1.367GB/s</td>
<td>1.54439GB/s</td>
<td>1.60603GB/s</td>
<td>1.33443GB/s</td>
<td>573.835MB/s</td>
<td>314.33MB/s</td>
<td>15.0169MB/s</td>
</tr>
<tr>
<td>google_message3_3</td>
<td>207.681MB/s</td>
<td>140.591MB/s</td>
<td>535.181MB/s</td>
<td>369.743MB/s</td>
<td>262.301MB/s</td>
<td>556.644MB/s</td>
<td>279.385MB/s</td>
<td>304.853MB/s</td>
<td>107.575MB/s</td>
<td>32.248MB/s</td>
<td>26.1431MB/s</td>
<td>2.63541MB/s</td>
</tr>
<tr>
<td>google_message3_4</td>
<td>7.96091GB/s</td>
<td>7.10024GB/s</td>
<td>9.3013GB/s</td>
<td>8.518GB/s</td>
<td>8.171GB/s</td>
<td>9.917GB/s</td>
<td>5.78006GB/s</td>
<td>5.85198GB/s</td>
<td>4.62609GB/s</td>
<td>2.49631GB/s</td>
<td>2.35442GB/s</td>
<td>802.061MB/s</td>
</tr>
<tr>
<td>google_message3_5</td>
<td>76.0072MB/s</td>
<td>51.6769MB/s</td>
<td>237.856MB/s</td>
<td>178.495MB/s</td>
<td>111.751MB/s</td>
<td>329.569MB/s</td>
<td>121.038MB/s</td>
<td>132.866MB/s</td>
<td>36.9197MB/s</td>
<td>10.3962MB/s</td>
<td>8.84659MB/s</td>
<td>1.25203MB/s</td>
</tr>
<tr>
<td>google_message4</td>
<td>331.46MB/s</td>
<td>404.862MB/s</td>
<td>427.99MB/s</td>
<td>589.887MB/s</td>
<td>720.367MB/s</td>
<td>705.373MB/s</td>
<td>606.228MB/s</td>
<td>589.13MB/s</td>
<td>530.692MB/s</td>
<td>305.543MB/s</td>
<td>174.834MB/s</td>
<td>7.86485MB/s</td>
</tr>
</tbody></table>
## Serialization performance
<table>
<tbody><tr>
<th rowspan="2"> </th>
<th colspan="1" rowspan="2">C++</th>
<th colspan="1" rowspan="2">C++ with tcmalloc</th>
<th colspan="3" rowspan="1">java</th>
<th colspan="3" rowspan="1">python</th>
</tr>
<tr>
<th colspan="1">byte[]</th>
<th colspan="1">ByteString</th>
<th colspan="1">InputStream</th>
<th colspan="1">C++-generated-code</th>
<th colspan="1">C++-reflection</th>
<th colspan="1">pure-Python</th>
</tr>
<tr>
<td>google_message1_proto2</td>
<td>1.39698GB/s</td>
<td>1.701GB/s</td>
<td>1.12915GB/s</td>
<td>1.13589GB/s</td>
<td>758.609MB/s</td>
<td>260.911MB/s</td>
<td>58.4815MB/s</td>
<td>5.77824MB/s</td>
</tr>
<tr>
<td>google_message1_proto3</td>
<td>959.305MB/s</td>
<td>939.404MB/s</td>
<td>1.15372GB/s</td>
<td>1.07824GB/s</td>
<td>802.337MB/s</td>
<td>239.4MB/s</td>
<td>33.6336MB/s</td>
<td>5.80524MB/s</td>
</tr>
<tr>
<td>google_message2</td>
<td>1.27429GB/s</td>
<td>1.402GB/s</td>
<td>1.01039GB/s</td>
<td>1022.99MB/s</td>
<td>798.736MB/s</td>
<td>996.755MB/s</td>
<td>57.9601MB/s</td>
<td>4.09246MB/s</td>
</tr>
<tr>
<td>google_message3_1</td>
<td>1.31916GB/s</td>
<td>2.049GB/s</td>
<td>991.496MB/s</td>
<td>860.332MB/s</td>
<td>662.88MB/s</td>
<td>1.48625GB/s</td>
<td>421.287MB/s</td>
<td>18.002MB/s</td>
</tr>
<tr>
<td>google_message3_2</td>
<td>2.15676GB/s</td>
<td>2.632GB/s</td>
<td>2.14736GB/s</td>
<td>2.08136GB/s</td>
<td>1.55997GB/s</td>
<td>2.39597GB/s</td>
<td>326.777MB/s</td>
<td>16.0527MB/s</td>
</tr>
<tr>
<td>google_message3_3</td>
<td>650.456MB/s</td>
<td>1.040GB/s</td>
<td>593.52MB/s</td>
<td>580.667MB/s</td>
<td>346.839MB/s</td>
<td>123.978MB/s</td>
<td>35.893MB/s</td>
<td>2.32834MB/s</td>
</tr>
<tr>
<td>google_message3_4</td>
<td>8.70154GB/s</td>
<td>9.825GB/s</td>
<td>5.88645GB/s</td>
<td>5.93946GB/s</td>
<td>2.44388GB/s</td>
<td>5.9241GB/s</td>
<td>4.05837GB/s</td>
<td>876.87MB/s</td>
</tr>
<tr>
<td>google_message3_5</td>
<td>246.33MB/s</td>
<td>443.993MB/s</td>
<td>283.278MB/s</td>
<td>259.167MB/s</td>
<td>206.37MB/s</td>
<td>37.0285MB/s</td>
<td>12.2228MB/s</td>
<td>1.1979MB/s</td>
</tr>
<tr>
<td>google_message4</td>
<td>1.56674GB/s</td>
<td>2.19601GB/s</td>
<td>776.907MB/s</td>
<td>770.707MB/s</td>
<td>702.931MB/s</td>
<td>1.49623GB/s</td>
<td>205.116MB/s</td>
<td>8.93428MB/s</td>
</tr>
</tbody></table>
\* The cpp performance can be improved by using [tcmalloc](https://gperftools.github.io/gperftools/tcmalloc.html), please follow the (instruction)[https://github.com/protocolbuffers/protobuf/blob/master/benchmarks/README.md] to link with tcmalloc to get the faster result.

204
deps/protobuf/docs/third_party.md vendored Normal file
View File

@ -0,0 +1,204 @@
# Third-Party Add-ons for Protocol Buffers
This page lists code related to Protocol Buffers which is developed and maintained by third parties. You may find this code useful, but note that **these projects are not affiliated with or endorsed by Google (unless explicitly marked)**; try them at your own risk. Also note that many projects here are in the early stages of development and not production-ready.
If you have a project that should be listed here, please
[send us a pull request](https://github.com/protocolbuffers/protobuf/pulls) to update this page.
## Programming Languages
These are projects we know about implementing Protocol Buffers for other programming languages:
* Action Script: https://code.google.com/p/protobuf-actionscript3/
* Action Script: https://code.google.com/p/protoc-gen-as3/
* Action Script: https://github.com/matrix3d/JProtoc
* Action Script: https://github.com/zhongfq/protobuf-as3/
* Ada: https://github.com/reznikmm/protobuf
* C: https://github.com/protobuf-c/protobuf-c
* C: https://koti.kapsi.fi/jpa/nanopb/
* C: https://github.com/cloudwu/pbc/
* C: https://github.com/haberman/upb/wiki
* C: https://github.com/squidfunk/protobluff
* C: https://github.com/eerimoq/pbtools
* C++: https://github.com/protocolbuffers/protobuf (Google-official implementation)
* C++: https://EmbeddedProto.com
* C/C++: http://spbc.sf.net/
* C#: https://code.google.com/p/protobuf-csharp-port
* C#: https://silentorbit.com/protobuf/
* C#/.NET/WCF/VB: https://code.google.com/p/protobuf-net/
* Clojure: http://github.com/ninjudd/clojure-protobuf
* Clojure: https://github.com/clojusc/protobuf
* Clojure: https://protojure.readthedocs.io
* Common Lisp: http://github.com/brown/protobuf
* Common Lisp: http://github.com/qitab/cl-protobuf
* D: https://github.com/dcarp/protobuf-d
* D: https://github.com/msoucy/dproto
* D: https://github.com/opticron/ProtocolBuffer
* Dart: https://github.com/dart-lang/dart-protobuf (runtime) https://github.com/dart-lang/dart-protoc-plugin (code generator)
* Delphi: http://sourceforge.net/projects/protobuf-delphi/
* Delphi: http://fundementals.sourceforge.net/dl.html
* Elixir: https://github.com/jeremyong/exprotoc
* Elixir: https://github.com/tony612/protobuf-elixir
* Elixir: https://github.com/ahamez/protox
* Elm: https://github.com/tiziano88/elm-protobuf
* Erlang: https://github.com/tomas-abrahamsson/gpb
* Erlang: http://piqi.org/
* Erlang: https://github.com/basho/erlang_protobuffs (no longer maintained, use gpb instead)
* Hacklang/HHVM: https://github.com/y3llowcake/proto-hack
* GDScript: https://github.com/oniksan/godobuf (Godot v3 engine plugin)
* Go: https://github.com/golang/protobuf (Google-official implementation)
* Go: https://github.com/akunspy/gopbuf
* Go: https://github.com/gogo/protobuf
* GopherJS: https://github.com/johanbrandhorst/protobuf
* Haskell: https://hackage.haskell.org/package/hprotoc
* Haskell: https://github.com/google/proto-lens (Google-unofficial implementation)
* Haskell: https://github.com/awakesecurity/proto3-suite (code generator) https://github.com/awakesecurity/proto3-wire (binary serializer/deserializer)
* Haxe: https://github.com/Atry/protoc-gen-haxe
* Java: https://github.com/protocolbuffers/protobuf (Google-official implementation)
* Java/Android: https://github.com/square/wire
* Java: https://github.com/HebiRobotics/QuickBuffers/
* Java ME: https://code.google.com/p/protobuf-javame/
* Java ME: http://swingme.sourceforge.net/encode.shtml
* Javascript: https://code.google.com/p/protobuf-js/
* Javascript: http://github.com/sirikata/protojs
* Javascript: https://github.com/dcodeIO/ProtoBuf.js
* Javascript: https://code.google.com/p/protobuf-for-node/
* Javascript: https://code.google.com/p/protostuff/
* Javascript: https://github.com/seishun/node-protoc-plugin (Node.js port of plugin.h)
* Javascript: https://github.com/seishun/node-protoc-gen-javascript (Node.js port of the Google-official implementation)
* Javascript: https://github.com/ButterCam/sisyphus-js
* Julia: https://github.com/tanmaykm/ProtoBuf.jl
* Kotlin: https://github.com/marcoferrer/kroto-plus
* Kotlin: https://github.com/Kotlin/kotlinx.serialization
* Kotlin: https://github.com/ButterCam/sisyphus
* Kotlin: https://github.com/open-toast/protokt
* Lua: https://code.google.com/p/protoc-gen-lua/
* Lua: http://github.com/indygreg/lua-protobuf
* Lua: https://github.com/Neopallium/lua-pb
* Matlab: https://code.google.com/p/protobuf-matlab/
* Mercury: https://code.google.com/p/protobuf-mercury/
* Objective C: https://code.google.com/p/protobuf-objc/
* Objective C: https://github.com/alexeyxo/protobuf-objc
* OCaml: http://piqi.org/
* Perl: http://groups.google.com/group/protobuf-perl
* Perl: https://metacpan.org/pod/Google::ProtocolBuffers
* Perl: https://metacpan.org/pod/Google::ProtocolBuffers::Dynamic
* Perl/XS: https://code.google.com/p/protobuf-perlxs/
* PHP: https://code.google.com/p/pb4php/
* PHP: https://github.com/allegro/php-protobuf/
* PHP: https://github.com/chobie/php-protocolbuffers
* Prolog: http://www.swi-prolog.org/pldoc/package/protobufs.html
* Purescript: https://github.com/xc-jp/purescript-protobuf
* Python: https://github.com/protocolbuffers/protobuf (Google-official implementation)
* Python: https://github.com/eigenein/protobuf
* Python: https://github.com/danielgtaylor/python-betterproto
* R: http://cran.r-project.org/package=RProtoBuf
* Ruby: https://code.google.com/p/ruby-protobuf/
* Ruby: http://github.com/mozy/ruby-protocol-buffers
* Ruby: https://github.com/bmizerany/beefcake/tree/master/lib/beefcake
* Ruby: https://github.com/localshred/protobuf
* Rust: https://github.com/tokio-rs/prost
* Rust: https://github.com/stepancheg/rust-protobuf/
* Rust: https://github.com/tafia/quick-protobuf
* Scala: http://github.com/jeffplaisance/scala-protobuf
* Scala: https://code.google.com/p/protobuf-scala
* Scala: https://github.com/SandroGrzicic/ScalaBuff
* Scala: https://scalapb.github.io
* Solidity: https://github.com/celer-network/pb3-gen-sol
* Swift: https://github.com/alexeyxo/protobuf-swift
* Swift: https://github.com/apple/swift-protobuf/
* Typescript: https://github.com/thesayyn/protoc-gen-ts
* Typescript: https://github.com/pbkit/pbkit
* Vala: https://launchpad.net/protobuf-vala
* Visual Basic: https://code.google.com/p/protobuf-net/
## RPC Implementations
GRPC (http://www.grpc.io/) is Google's RPC implementation for Protocol Buffers. There are other third-party RPC implementations as well. Some of these actually work with Protocol Buffers service definitions (defined using the `service` keyword in `.proto` files) while others just use Protocol Buffers message objects.
* https://github.com/grpc/grpc (C++, Node.js, Python, Ruby, Objective-C, PHP, C#, Google-official implementation)
* https://zeroc.com/products/ice (Multiple languages)
* https://github.com/protobuf-net/protobuf-net (C#/.NET/WCF/VB)
* http://www.deltavsoft.com/doc/_external_serialization.html#Protobufs (C++)
* https://protojure.github.io (Clojure)
* https://code.google.com/p/protobuf-rpc-pro/ (Java)
* https://github.com/baidu/sofa-pbrpc (C++)
* https://github.com/madhon/protobuf-csharp-rpc (C#)
* https://github.com/madwyn/libpbrpc (C++)
* https://github.com/SeriousMa/grpc-protobuf-validation (Java)
* https://github.com/elixir-grpc/grpc (Elixir)
* https://github.com/johanbrandhorst/protobuf (GopherJS)
* https://github.com/awakesecurity/gRPC-haskell (Haskell)
* https://github.com/Yeolar/raster (C++)
* https://github.com/jnordberg/wsrpc (JavaScript Node.js/Browser)
* https://github.com/pbkit/npm-packages/blob/main/frpc-test/src/index.spec.ts (TypeScript Node.js/Browser)
* https://github.com/ppissias/xsrpcj (Java)
* https://github.com/twitchtv/twirp (Multiple languages)
Inactive:
* https://launchpad.net/txprotobuf/ (Python)
* https://github.com/modeswitch/protobuf-rpc (Python)
* https://github.com/sdeo/protobuf-socket-rpc (Java, Python)
* https://code.google.com/p/proto-streamer/ (Java)
* https://code.google.com/p/server1/ (C++)
* https://code.google.com/p/protobuf-mina-rpc/ (Python client, Java server)
* https://code.google.com/p/casocklib/ (C++)
* https://code.google.com/p/cxf-protobuf/ (Java)
* https://code.google.com/p/protobuf-remote/ (C++/C#)
* https://code.google.com/p/eneter-protobuf-serializer/ (Java/.NET)
* https://github.com/robbinfan/claire/tree/master/protorpc (C++)
* https://github.com/ebencheung/arab (C++)
* https://code.google.com/p/protobuf-csharp-rpc/ (C#)
* https://github.com/thesamet/rpcz (C++/Python, based on ZeroMQ)
* https://github.com/w359405949/libmaid (C++, Python)
## Other Utilities
There are miscellaneous other things you may find useful as a Protocol Buffers developer.
* [Bazel Build](https://bazel.build)
* [rules_closure](https://github.com/bazelbuild/rules_closure) `js-closure`
* [rules_go](https://github.com/bazelbuild/rules_go) `go`
* [rules_protobuf](https://github.com/pubref/rules_protobuf) `java` `c++` `c#` `go` `js-closure` `js-node` `python` `ruby`
* [NetBeans IDE plugin](https://code.google.com/p/protobuf-netbeans-plugin/)
* [Wireshark/Ethereal packet sniffer plugin](https://code.google.com/p/protobuf-wireshark/)
* [Alternate encodings (JSON, XML, HTML) for Java protobufs](https://code.google.com/p/protobuf-java-format/)
* [Another JSON encoder/decoder for Java](https://github.com/sijuv/protobuf-codec)
* [Editor for serialized protobufs](https://code.google.com/p/protobufeditor/)
* [IntelliJ IDEA plugin](http://github.com/jvolkman/intellij-protobuf-editor)
* [IntelliJ Protobuf Plugin](https://github.com/devkanro/intellij-protobuf-plugin)
* [TextMate syntax highlighting](http://github.com/michaeledgar/protobuf-tmbundle)
* [Oracle PL SQL plugin](https://code.google.com/p/protocol-buffer-plsql/)
* [Eclipse editor for protobuf (from Google)](https://code.google.com/p/protobuf-dt/)
* [C++ Builder compatible protobuf](https://github.com/saadware/protobuf-cppbuilder)
* Maven Protobuf Compiler Plugin
* By xolstice.org ([Documentation](https://www.xolstice.org/protobuf-maven-plugin/)) ([Source](https://github.com/xolstice/protobuf-maven-plugin/)) [![Maven Central](https://img.shields.io/maven-central/v/org.xolstice.maven.plugins/protobuf-maven-plugin.svg)](https://repo1.maven.org/maven2/org/xolstice/maven/plugins/protobuf-maven-plugin/)
* https://code.google.com/p/maven-protoc-plugin/
* https://github.com/os72/protoc-jar-maven-plugin
* [Documentation generator plugin (Markdown/HTML/DocBook/...)](https://github.com/pseudomuto/protoc-gen-doc)
* [DocBook generator for .proto files](https://code.google.com/p/protoc-gen-docbook/)
* [Protobuf for nginx module](https://github.com/dbcode/protobuf-nginx/)
* [RSpec matchers and Cucumber step defs for testing Protocol Buffers](https://github.com/connamara/protobuf_spec)
* [Sbt plugin for Protocol Buffers](https://github.com/Atry/sbt-cppp)
* [Gradle Protobuf Plugin](https://github.com/aantono/gradle-plugin-protobuf)
* [Multi-platform executable JAR and Java API for protoc](https://github.com/os72/protoc-jar)
* [Python scripts to convert between Protocol Buffers and JSON](https://github.com/NextTuesday/py-pb-converters)
* [Visual Studio Language Service support for Protocol Buffers](http://visualstudiogallery.msdn.microsoft.com/4bc0f38c-b058-4e05-ae38-155e053c19c5)
* [Visual Studio Code Support for Protocol Buffers](https://marketplace.visualstudio.com/items?itemName=zxh404.vscode-proto3)
* [C++ library for serialization/de-serialization between Protocol Buffers and JSON.](https://github.com/yinqiwen/pbjson)
* [ProtoBuf with Java EE7 Expression Language 3.0; pure Java ProtoBuf Parser and Builder.](https://github.com/protobufel/protobuf-el)
* [Notepad++ Syntax Highlighting for .proto files](https://github.com/chai2010/notepadplus-protobuf)
* [Linter for .proto files](https://github.com/ckaznocha/protoc-gen-lint)
* [Protocol Buffers Dynamic Schema - create protobuf schemas programmatically (Java)](https://github.com/os72/protobuf-dynamic)
* [Make protoc plugins in NodeJS](https://github.com/konsumer/node-protoc-plugin)
* [ProfaneDB - A Protocol Buffers database](https://profanedb.gitlab.io)
* [Protocol Buffer property-based testing utility and example message generator (Python / Hypothesis)](https://github.com/CurataEng/hypothesis-protobuf)
* [Protolock - CLI utility to prevent backward-incompatible changes to .proto files](https://github.com/nilslice/protolock)
* [Optional GRPC - GRPC for testable microservices (Python)](https://github.com/mattpaletta/optional-grpc.git)
* [Protobuf Parser - Yet another Go package which parses a Protocol Buffer file (proto2+proto3)](https://github.com/yoheimuta/go-protoparser)
* [Protolint - A tool to enforce Protocol Buffer style and conventions.](https://github.com/yoheimuta/protolint)
* [vscode-protolint: A protobuf linter for visual studio code](https://github.com/plexsystems/vscode-protolint)
* [intellij-protolint: A protobuf linter for JetBrains IDEs](https://github.com/yoheimuta/intellij-protolint)
* [vim-protolint: A protobuf linter for Vim](https://github.com/yoheimuta/vim-protolint)
* [super-linter: Protocol Buffer lint as GitHub Action](https://github.com/github/super-linter)
* [protoc-gen-fieldmask - A plugin to generate static type fieldmask paths](https://github.com/idodod/protoc-gen-fieldmask)