gRPC-Gateway 简介
gRPC-Gateway 是 protoc 的一个插件,工作机制是读取一个 gRPC 服务定义并生成一个反向代理服务器,将 RESTful JSON API 翻译成 gRPC。
这个服务器是根据编写的 gRPC 定义中的自定义选项来生成的。
安装使用
依赖工具
工具 | 简介 | 安装 |
---|---|---|
protobuf | protocol buffer 编译所需的命令行 | http://google.github.io/proto-lens/installing-protoc.html |
protoc-gen-go | 从 proto 文件,生成 .go 文件 | https://grpc.io/docs/languages/go/quickstart/ |
protoc-gen-go-grpc | 从 proto 文件,生成 gRPC 相关的 .go 文件 | https://grpc.io/docs/languages/go/quickstart/ |
protoc-gen-grpc-gateway | 从 proto 文件,生成 gRPC-gateway 相关的 .go 文件 | https://github.com/grpc-ecosystem/grpc-gateway#installation |
protoc-gen-openapiv2 | 从 proto 文件,生成 swagger 文档所需的参数文件 | https://github.com/grpc-ecosystem/grpc-gateway#installation |
buf | protobuf 管理工具,可选,简化命令行操作和protobuf 文件管理 | https://docs.buf.build/installation |
步骤
编写buf配置
buf.gen.yaml
version: v1beta1
plugins:
- name: go
out: internal/proto
opt:
- paths=source_relative
- name: go-grpc
out: internal/proto
opt:
- paths=source_relative
- require_unimplemented_servers=false
- name: grpc-gateway
out: internal/proto
opt:
- paths=source_relative
- name: openapiv2
out: openapi
opt:
- json_names_for_fields=false
buf.yaml
version: v1beta1
name: buf.build/myworkspace/grpc
deps:
- buf.build/beta/googleapis
- buf.build/grpc-ecosystem/grpc-gateway
build:
roots:
- proto
lint:
use:
- DEFAULT
rpc_allow_google_protobuf_empty_requests: true
rpc_allow_google_protobuf_empty_responses: true
breaking:
use:
- FILE
编写proto
和 gRPC 接口 proto 文件不一样的是,其中包含了对 HTTP 的注释。
- 通过 gRPC options 添加 HTTP API 信息,比如 HTTP method、path、body、额外绑定等,需引用 google/api/annotations.proto
- 通过 protobuf message field,添加额外信息,来实现 HTTP 请求参数和响应 body 的约束,并且可以生成 swagger 文档,需引用 protoc-gen-openapiv2/options/annotations.proto
demo.proto
syntax = "proto3";
package console.v1;
option go_package = "git.yourcompany.com/yourgroup/grpc-gateway-demo/proto/console/v1";
import "google/protobuf/empty.proto";
import "google/api/annotations.proto";
import "protoc-gen-openapiv2/options/annotations.proto";
// DemoService is the demo service definition
service DemoService {
rpc Hello(HelloRequest) returns (HelloResponse) {
option (google.api.http) = {
get: "/web/v1/hello-messages/{name}"
additional_bindings {
get: "/client/v1/hello-messages/{name}"
}
};
}
}
message HelloRequest {
string name = 1 [(grpc.gateway.protoc_gen_openapiv2.options.openapiv2_field) = {
description: "名称",
required: ['name'],
type: STRING,
}];
}
message HelloResponse {
string message = 1 [(grpc.gateway.protoc_gen_openapiv2.options.openapiv2_field) = {
description: "响应消息",
required: ['message'],
type: STRING,
}];
}
生成代码
命令
grpc.sh
#!/usr/bin/env bash
buf lint
buf breaking --against .git``#branch=master
buf generate
目录结构
demo.pb.gw.go 为 gRPC-gateway 生成的文件,处理 HTTP RESTful 到 gRPC 的请求。
grpc-gateway-demo/internal/proto
qtmf-demo``/internal/proto
➜ tree
.
└── console
``└── v1
``├── demo.pb.go
``├── demo.pb.gw.go
``└── demo_grpc.pb.go
生成的 swagger 文档
[DemoService]
GET /web/v1/hello-messages/{name}
GET /client/v1/hello-messages/{name}
编写 go rpc service
grpc/main.go
type DemoHandler struct{}
// Hello 编写实现方法
func (s *DemoHandler) Hello(ctx context.Context, request *consoleV1.HelloRequest) (*consoleV1.HelloResponse, error) {
``return` `&consoleV1.HelloResponse{
``Message: ``"hello: "` `+ request.GetName(),
``}, nil
}
func main() {
lis, err := net.Listen("tcp", port)
if err != nil {
log.Fatalf("failed to listen: %v", err)
}
s := grpc.NewServer()
pb.RegisterDemoServer(s, &server{})
// Register reflection service on gRPC server.
reflection.Register(s)
if err := s.Serve(lis); err != nil {
log.Fatalf("failed to serve: %v", err)
}
}
启动服务
main.go
package main
import (
"context"
"flag"
"net/http"
"github.com/golang/glog"
"github.com/grpc-ecosystem/grpc-gateway/v2/runtime"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials/insecure"
gw "github.com/yourorg/yourrepo/proto/gen/go/your/service/v1/your_service" // Update
)
var (
// command-line options:
// gRPC server endpoint
grpcServerEndpoint = flag.String("grpc-server-endpoint", "localhost:9090", "gRPC server endpoint")
)
func run() error {
ctx := context.Background()
ctx, cancel := context.WithCancel(ctx)
defer cancel()
// Register gRPC server endpoint
// Note: Make sure the gRPC server is running properly and accessible
mux := runtime.NewServeMux()
opts := []grpc.DialOption{grpc.WithTransportCredentials(insecure.NewCredentials())}
err := gw.RegisterYourServiceHandlerFromEndpoint(ctx, mux, *grpcServerEndpoint, opts)
if err != nil {
return err
}
// Start HTTP server (and proxy calls to gRPC server endpoint)
return http.ListenAndServe(":8081", mux)
}
func main() {
flag.Parse()
defer glog.Flush()
if err := run(); err != nil {
glog.Fatal(err)
}
}
原理
回到官方示意图,清晰的画出了从 proto 文件,生成 gRPC Service 和 反向代理的过程。
源码分析
gRPC-gateway 主要分代码生成、生成的反向代理以及 HTTP 和 gRPC 协议处理的 runtime 等几大部分。
代码生成
https://github.com/grpc-ecosystem/grpc-gateway/blob/master/protoc-gen-grpc-gateway/main.go#L65
func main() {
flag.Parse()
defer glog.Flush()
if *versionFlag {
fmt.Printf("Version %v, commit %v, built at %v\n", version, commit, date)
os.Exit(0)
}
protogen.Options{
ParamFunc: flag.CommandLine.Set,
}.Run(func(gen *protogen.Plugin) error {
reg := descriptor.NewRegistry()
err := applyFlags(reg)
if err != nil {
return err
}
codegenerator.SetSupportedFeaturesOnPluginGen(gen)
generator := gengateway.New(reg, *useRequestContext, *registerFuncSuffix, *allowPatchFeature, *standalone)
glog.V(1).Infof("Parsing code generator request")
if err := reg.LoadFromPlugin(gen); err != nil {
return err
}
unboundHTTPRules := reg.UnboundExternalHTTPRules()
if len(unboundHTTPRules) != 0 {
return fmt.Errorf("HTTP rules without a matching selector: %s", strings.Join(unboundHTTPRules, ", "))
}
var targets []*descriptor.File
for _, target := range gen.Request.FileToGenerate {
f, err := reg.LookupFile(target)
if err != nil {
return err
}
targets = append(targets, f)
}
files, err := generator.Generate(targets)
for _, f := range files {
glog.V(1).Infof("NewGeneratedFile %q in %s", f.GetName(), f.GoPkg)
genFile := gen.NewGeneratedFile(f.GetName(), protogen.GoImportPath(f.GoPkg.Path))
if _, err := genFile.Write([]byte(f.GetContent())); err != nil {
return err
}
}
glog.V(1).Info("Processed code generator request")
return err
})
}
生成的反向代理
// 处理 HTTP 请求
func RegisterGreeterHandlerServer(ctx context.Context, mux *runtime.ServeMux, server GreeterServer) error {
mux.Handle("GET", pattern_Greeter_SayHello_0, func(w http.ResponseWriter, req *http.Request, pathParams map[string]string) {
ctx, cancel := context.WithCancel(req.Context())
defer cancel()
var stream runtime.ServerTransportStream
ctx = grpc.NewContextWithServerTransportStream(ctx, &stream)
inboundMarshaler, outboundMarshaler := runtime.MarshalerForRequest(mux, req)
rctx, err := runtime.AnnotateIncomingContext(ctx, mux, req, "/grpc.gateway.examples.internal.helloworld.Greeter/SayHello", runtime.WithHTTPPathPattern("/say/{name}"))
if err != nil {
runtime.HTTPError(ctx, mux, outboundMarshaler, w, req, err)
return
}
resp, md, err := local_request_Greeter_SayHello_0(rctx, inboundMarshaler, server, req, pathParams)
md.HeaderMD, md.TrailerMD = metadata.Join(md.HeaderMD, stream.Header()), metadata.Join(md.TrailerMD, stream.Trailer())
ctx = runtime.NewServerMetadataContext(ctx, md)
if err != nil {
runtime.HTTPError(ctx, mux, outboundMarshaler, w, req, err)
return
}
forward_Greeter_SayHello_0(ctx, mux, outboundMarshaler, w, req, resp, mux.GetForwardResponseOptions()...)
})
// 请求 gRPC Client
func request_Greeter_SayHello_0(ctx context.Context, marshaler runtime.Marshaler, client GreeterClient, req *http.Request, pathParams map[string]string) (proto.Message, runtime.ServerMetadata, error) {
var protoReq HelloRequest
var metadata runtime.ServerMetadata
var (
val string
ok bool
err error
_ = err
)
val, ok = pathParams["name"]
if !ok {
return nil, metadata, status.Errorf(codes.InvalidArgument, "missing parameter %s", "name")
}
protoReq.Name, err = runtime.String(val)
if err != nil {
return nil, metadata, status.Errorf(codes.InvalidArgument, "type mismatch, parameter: %s, error: %v", "name", err)
}
if err := req.ParseForm(); err != nil {
return nil, metadata, status.Errorf(codes.InvalidArgument, "%v", err)
}
if err := runtime.PopulateQueryParameters(&protoReq, req.Form, filter_Greeter_SayHello_0); err != nil {
return nil, metadata, status.Errorf(codes.InvalidArgument, "%v", err)
}
msg, err := client.SayHello(ctx, &protoReq, grpc.Header(&metadata.HeaderMD), grpc.Trailer(&metadata.TrailerMD))
return msg, metadata, err
错误映射
从 gRPC status code 转换到 HTTP status code
https://github.com/grpc-ecosystem/grpc-gateway/blob/master/runtime/errors.go
// HTTPStatusFromCode converts a gRPC error code into the corresponding HTTP response status.
// See: https://github.com/googleapis/googleapis/blob/master/google/rpc/code.proto
func HTTPStatusFromCode(code codes.Code) int {
switch code {
case codes.OK:
return http.StatusOK
case codes.Canceled:
return http.StatusRequestTimeout
case codes.Unknown:
return http.StatusInternalServerError
case codes.InvalidArgument:
return http.StatusBadRequest
case codes.DeadlineExceeded:
return http.StatusGatewayTimeout
case codes.NotFound:
return http.StatusNotFound
case codes.AlreadyExists:
return http.StatusConflict
case codes.PermissionDenied:
return http.StatusForbidden
case codes.Unauthenticated:
return http.StatusUnauthorized
case codes.ResourceExhausted:
return http.StatusTooManyRequests
case codes.FailedPrecondition:
// Note, this deliberately doesn't translate to the similarly named '412 Precondition Failed' HTTP response status.
return http.StatusBadRequest
case codes.Aborted:
return http.StatusConflict
case codes.OutOfRange:
return http.StatusBadRequest
case codes.Unimplemented:
return http.StatusNotImplemented
case codes.Internal:
return http.StatusInternalServerError
case codes.Unavailable:
return http.StatusServiceUnavailable
case codes.DataLoss:
return http.StatusInternalServerError
}
grpclog.Infof("Unknown gRPC error code: %v", code)
return http.StatusInternalServerError
}
参考
https://github.com/grpc-ecosystem/grpc-gateway#readme
https://grpc-ecosystem.github.io/grpc-gateway/
https://github.com/grpc/grpc-go
https://github.com/googleapis/googleapis/tree/master/google/api
https://github.com/grpc-ecosystem/grpc-gateway/tree/master/protoc-gen-openapiv2/options