gRPC-Gateway 简介

gRPC-Gateway 是 protoc 的一个插件,工作机制是读取一个 gRPC 服务定义并生成一个反向代理服务器,将 RESTful JSON API 翻译成 gRPC。

这个服务器是根据编写的 gRPC 定义中的自定义选项来生成的。

安装使用

依赖工具

工具简介安装
protobufprotocol buffer 编译所需的命令行http://google.github.io/proto-lens/installing-protoc.html
protoc-gen-go从 proto 文件,生成 .go 文件https://grpc.io/docs/languages/go/quickstart/
protoc-gen-go-grpc从 proto 文件,生成 gRPC 相关的 .go 文件https://grpc.io/docs/languages/go/quickstart/
protoc-gen-grpc-gateway从 proto 文件,生成 gRPC-gateway 相关的 .go 文件https://github.com/grpc-ecosystem/grpc-gateway#installation
protoc-gen-openapiv2从 proto 文件,生成 swagger 文档所需的参数文件https://github.com/grpc-ecosystem/grpc-gateway#installation
bufprotobuf 管理工具,可选,简化命令行操作和protobuf 文件管理https://docs.buf.build/installation

步骤

编写buf配置

buf.gen.yaml

version: v1beta1
plugins:
- name: go
  out: internal/proto
  opt:
  - paths=source_relative
- name: go-grpc
  out: internal/proto
  opt:
  - paths=source_relative
  - require_unimplemented_servers=false
- name: grpc-gateway
  out: internal/proto
  opt:
  - paths=source_relative
- name: openapiv2
  out: openapi
  opt:
  - json_names_for_fields=false

buf.yaml

version: v1beta1
name: buf.build/myworkspace/grpc
deps:
- buf.build/beta/googleapis
- buf.build/grpc-ecosystem/grpc-gateway
build:
  roots:
  - proto
lint:
  use:
  - DEFAULT
  rpc_allow_google_protobuf_empty_requests: true
  rpc_allow_google_protobuf_empty_responses: true
breaking:
  use:
  - FILE

编写proto

和 gRPC 接口 proto 文件不一样的是,其中包含了对 HTTP 的注释。

demo.proto

syntax = "proto3";
 
package console.v1;
option go_package = "git.yourcompany.com/yourgroup/grpc-gateway-demo/proto/console/v1";
 
import "google/protobuf/empty.proto";
import "google/api/annotations.proto";
import "protoc-gen-openapiv2/options/annotations.proto";
 
// DemoService is the demo service definition
service DemoService {
    rpc Hello(HelloRequest) returns (HelloResponse) {
        option (google.api.http) = {
            get: "/web/v1/hello-messages/{name}"
            additional_bindings {
                get: "/client/v1/hello-messages/{name}"
            }
        };
    }
}
 
message HelloRequest {
    string name = 1 [(grpc.gateway.protoc_gen_openapiv2.options.openapiv2_field) = {
        description: "名称",
        required: ['name'],
        type: STRING,
    }];
}
 
message HelloResponse {
    string message = 1 [(grpc.gateway.protoc_gen_openapiv2.options.openapiv2_field) = {
        description: "响应消息",
        required: ['message'],
        type: STRING,
    }];
}

生成代码

命令

grpc.sh

#!/usr/bin/env bash

buf lint
buf breaking --against .git``#branch=master
buf generate

目录结构

demo.pb.gw.go 为 gRPC-gateway 生成的文件,处理 HTTP RESTful 到 gRPC 的请求。

grpc-gateway-demo/internal/proto

qtmf-demo``/internal/proto
➜ tree
.
└── console
  ``└── v1
    ``├── demo.pb.go
    ``├── demo.pb.gw.go
    ``└── demo_grpc.pb.go

生成的 swagger 文档

[DemoService]

GET /web/v1/hello-messages/{name}

GET /client/v1/hello-messages/{name}

编写 go rpc service

grpc/main.go


type DemoHandler struct{}

// Hello 编写实现方法
func (s *DemoHandler) Hello(ctx context.Context, request *consoleV1.HelloRequest) (*consoleV1.HelloResponse, error) {
  ``return` `&consoleV1.HelloResponse{
    ``Message: ``"hello: "` `+ request.GetName(),
  ``}, nil
}

func main() {
	lis, err := net.Listen("tcp", port)
	if err != nil {
		log.Fatalf("failed to listen: %v", err)
	}
	s := grpc.NewServer()
	pb.RegisterDemoServer(s, &server{})
	// Register reflection service on gRPC server.
	reflection.Register(s)
	if err := s.Serve(lis); err != nil {
		log.Fatalf("failed to serve: %v", err)
	}
}

启动服务

main.go

package main
 
import (
  "context"
  "flag"
  "net/http"
 
  "github.com/golang/glog"
  "github.com/grpc-ecosystem/grpc-gateway/v2/runtime"
  "google.golang.org/grpc"
  "google.golang.org/grpc/credentials/insecure"
 
  gw "github.com/yourorg/yourrepo/proto/gen/go/your/service/v1/your_service"  // Update
)
 
var (
  // command-line options:
  // gRPC server endpoint
  grpcServerEndpoint = flag.String("grpc-server-endpoint",  "localhost:9090", "gRPC server endpoint")
)
 
func run() error {
  ctx := context.Background()
  ctx, cancel := context.WithCancel(ctx)
  defer cancel()
 
  // Register gRPC server endpoint
  // Note: Make sure the gRPC server is running properly and accessible
  mux := runtime.NewServeMux()
  opts := []grpc.DialOption{grpc.WithTransportCredentials(insecure.NewCredentials())}
  err := gw.RegisterYourServiceHandlerFromEndpoint(ctx, mux,  *grpcServerEndpoint, opts)
  if err != nil {
    return err
  }
 
  // Start HTTP server (and proxy calls to gRPC server endpoint)
  return http.ListenAndServe(":8081", mux)
}
 
func main() {
  flag.Parse()
  defer glog.Flush()
 
  if err := run(); err != nil {
    glog.Fatal(err)
  }
}

原理

回到官方示意图,清晰的画出了从 proto 文件,生成 gRPC Service 和 反向代理的过程。

源码分析

gRPC-gateway 主要分代码生成、生成的反向代理以及 HTTP 和 gRPC 协议处理的 runtime 等几大部分。

代码生成

https://github.com/grpc-ecosystem/grpc-gateway/blob/master/protoc-gen-grpc-gateway/main.go#L65

func main() {
    flag.Parse()
    defer glog.Flush()
 
    if *versionFlag {
        fmt.Printf("Version %v, commit %v, built at %v\n", version, commit, date)
        os.Exit(0)
    }
 
    protogen.Options{
        ParamFunc: flag.CommandLine.Set,
    }.Run(func(gen *protogen.Plugin) error {
        reg := descriptor.NewRegistry()
 
        err := applyFlags(reg)
        if err != nil {
            return err
        }
 
        codegenerator.SetSupportedFeaturesOnPluginGen(gen)
 
        generator := gengateway.New(reg, *useRequestContext, *registerFuncSuffix, *allowPatchFeature, *standalone)
 
        glog.V(1).Infof("Parsing code generator request")
 
        if err := reg.LoadFromPlugin(gen); err != nil {
            return err
        }
 
        unboundHTTPRules := reg.UnboundExternalHTTPRules()
        if len(unboundHTTPRules) != 0 {
            return fmt.Errorf("HTTP rules without a matching selector: %s", strings.Join(unboundHTTPRules, ", "))
        }
 
        var targets []*descriptor.File
        for _, target := range gen.Request.FileToGenerate {
            f, err := reg.LookupFile(target)
            if err != nil {
                return err
            }
            targets = append(targets, f)
        }
 
        files, err := generator.Generate(targets)
        for _, f := range files {
            glog.V(1).Infof("NewGeneratedFile %q in %s", f.GetName(), f.GoPkg)
            genFile := gen.NewGeneratedFile(f.GetName(), protogen.GoImportPath(f.GoPkg.Path))
            if _, err := genFile.Write([]byte(f.GetContent())); err != nil {
                return err
            }
        }
 
        glog.V(1).Info("Processed code generator request")
 
        return err
    })
}

生成的反向代理

https://github.com/grpc-ecosystem/grpc-gateway/blob/master/examples/internal/helloworld/helloworld.pb.gw.go#L738

// 处理 HTTP 请求
func RegisterGreeterHandlerServer(ctx context.Context, mux *runtime.ServeMux, server GreeterServer) error {
 
    mux.Handle("GET", pattern_Greeter_SayHello_0, func(w http.ResponseWriter, req *http.Request, pathParams map[string]string) {
        ctx, cancel := context.WithCancel(req.Context())
        defer cancel()
        var stream runtime.ServerTransportStream
        ctx = grpc.NewContextWithServerTransportStream(ctx, &stream)
        inboundMarshaler, outboundMarshaler := runtime.MarshalerForRequest(mux, req)
        rctx, err := runtime.AnnotateIncomingContext(ctx, mux, req, "/grpc.gateway.examples.internal.helloworld.Greeter/SayHello", runtime.WithHTTPPathPattern("/say/{name}"))
        if err != nil {
            runtime.HTTPError(ctx, mux, outboundMarshaler, w, req, err)
            return
        }
        resp, md, err := local_request_Greeter_SayHello_0(rctx, inboundMarshaler, server, req, pathParams)
        md.HeaderMD, md.TrailerMD = metadata.Join(md.HeaderMD, stream.Header()), metadata.Join(md.TrailerMD, stream.Trailer())
        ctx = runtime.NewServerMetadataContext(ctx, md)
        if err != nil {
            runtime.HTTPError(ctx, mux, outboundMarshaler, w, req, err)
            return
        }
 
        forward_Greeter_SayHello_0(ctx, mux, outboundMarshaler, w, req, resp, mux.GetForwardResponseOptions()...)
 
    })
 
 
 
// 请求 gRPC Client
func request_Greeter_SayHello_0(ctx context.Context, marshaler runtime.Marshaler, client GreeterClient, req *http.Request, pathParams map[string]string) (proto.Message, runtime.ServerMetadata, error) {
    var protoReq HelloRequest
    var metadata runtime.ServerMetadata
 
    var (
        val string
        ok  bool
        err error
        _   = err
    )
 
    val, ok = pathParams["name"]
    if !ok {
        return nil, metadata, status.Errorf(codes.InvalidArgument, "missing parameter %s", "name")
    }
 
    protoReq.Name, err = runtime.String(val)
    if err != nil {
        return nil, metadata, status.Errorf(codes.InvalidArgument, "type mismatch, parameter: %s, error: %v", "name", err)
    }
 
    if err := req.ParseForm(); err != nil {
        return nil, metadata, status.Errorf(codes.InvalidArgument, "%v", err)
    }
    if err := runtime.PopulateQueryParameters(&protoReq, req.Form, filter_Greeter_SayHello_0); err != nil {
        return nil, metadata, status.Errorf(codes.InvalidArgument, "%v", err)
    }
 
    msg, err := client.SayHello(ctx, &protoReq, grpc.Header(&metadata.HeaderMD), grpc.Trailer(&metadata.TrailerMD))
    return msg, metadata, err

错误映射

从 gRPC status code 转换到 HTTP status code

https://github.com/grpc-ecosystem/grpc-gateway/blob/master/runtime/errors.go

// HTTPStatusFromCode converts a gRPC error code into the corresponding HTTP response status.
// See: https://github.com/googleapis/googleapis/blob/master/google/rpc/code.proto
func HTTPStatusFromCode(code codes.Code) int {
    switch code {
    case codes.OK:
        return http.StatusOK
    case codes.Canceled:
        return http.StatusRequestTimeout
    case codes.Unknown:
        return http.StatusInternalServerError
    case codes.InvalidArgument:
        return http.StatusBadRequest
    case codes.DeadlineExceeded:
        return http.StatusGatewayTimeout
    case codes.NotFound:
        return http.StatusNotFound
    case codes.AlreadyExists:
        return http.StatusConflict
    case codes.PermissionDenied:
        return http.StatusForbidden
    case codes.Unauthenticated:
        return http.StatusUnauthorized
    case codes.ResourceExhausted:
        return http.StatusTooManyRequests
    case codes.FailedPrecondition:
        // Note, this deliberately doesn't translate to the similarly named '412 Precondition Failed' HTTP response status.
        return http.StatusBadRequest
    case codes.Aborted:
        return http.StatusConflict
    case codes.OutOfRange:
        return http.StatusBadRequest
    case codes.Unimplemented:
        return http.StatusNotImplemented
    case codes.Internal:
        return http.StatusInternalServerError
    case codes.Unavailable:
        return http.StatusServiceUnavailable
    case codes.DataLoss:
        return http.StatusInternalServerError
    }
 
    grpclog.Infof("Unknown gRPC error code: %v", code)
    return http.StatusInternalServerError
}

参考

https://github.com/grpc-ecosystem/grpc-gateway#readme

https://grpc-ecosystem.github.io/grpc-gateway/

https://github.com/grpc/grpc-go

https://grpc.io/docs/

https://github.com/googleapis/googleapis/tree/master/google/api

https://github.com/grpc-ecosystem/grpc-gateway/tree/master/protoc-gen-openapiv2/options