Building MCP Servers from Protobuf (Part1) - Protobuf to REST API

Introduction

In this blog series, we’ll show you how to build an MCP (Model Context Protocol) server packed with useful tools. Rather than starting from scratch, we’ll take advantage of our existing Protocol Buffers and Google’s gRPC transcoding. By creating a custom protoc (Protocol Buffer compiler) plugin, we can automatically generate the MCP server. This unified approach lets us produce gRPC services, OpenAPI specifications, REST APIs, and the MCP server all from the same source.

What You'll Build

By the end of this blog, you'll have:

• A single Protocol Buffer definition that generates both gRPC and REST API
• Automatic OpenAPI specification generation
• A foundation ready for MCP server generation (covered in Part 2)

Prerequisites

Before we start, make sure you have:

• Go 1.19+ installed
• Protocol Buffer compiler (protoc) installed
• Basic familiarity with gRPC and REST APIs

The Problem We're Solving

Our architecture started out simple: we used gRPC for internal services, REST APIs for external clients, and kept OpenAPI specs in a separate, manually maintained repository. As the system grew, this split became a headache—every change meant updating service code, the REST gateway, and documentation in multiple places. This led to inconsistencies, forgotten updates, and a lot of repeated work.

What is gRPC Transcoding?

gRPC Transcoding lets you expose your gRPC services as REST APIs, so clients can use familiar HTTP calls while your backend logic stays in gRPC. This is made possible by adding special HTTP annotations—known as Google API annotations—to your Protocol Buffer definitions. These annotations define how HTTP requests map to gRPC methods, including routes, path variables, and request bodies.

How Do Google API Annotations Work?

Google API annotations are a set of Protocol Buffer extensions that define how gRPC services map to REST APIs. They were originally developed by Google for their own APIs and have become the standard for gRPC-to-REST transcoding.

Key Components:

• google/api/annotations.proto: This file provides the extension for HTTP options on RPC methods.
• google.api.http: An option defined in google/api/http.proto that specifies the HTTP mapping for a gRPC method.
• Path Variables: Placeholders in the HTTP path (like {book_id}) that are automatically mapped to fields in the request message.
• Body Mapping: The body field in the HTTP option determines which part of the request message is used as the HTTP request body.

In short, Google API annotations make it easy to maintain and extend your APIs, keeping everything in sync and reducing manual work.

Set Up Your Project 

Let's start by creating a new project:

mkdir proto-to-mcp-tutorial
cd proto-to-mcp-tutorial

# Create directory structure
mkdir -p {proto,googleapis,generated/{go,openapi},cmd}

Your project structure should look like:

proto-to-mcp-tutorial/
├── proto/
├── googleapis/
├── generated/
│   ├── go/
│   └── openapi/
└── cmd/

Install Required Dependencies

First, let's install go protoc plugins:

go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.34.1
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.3.0
go install github.com/google/gnostic/cmd/protoc-gen-openapi@latest
go install github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-grpc-gateway@v2.24.0

Then we need the Google API definitions for HTTP annotations:

# Create directory for Google APIs
mkdir -p googleapis/google/api

# Download the required proto files
curl -o googleapis/google/api/annotations.proto \
https://raw.githubusercontent.com/googleapis/googleapis/master/google/api/annotations.proto
curl -o googleapis/google/api/http.proto \
https://raw.githubusercontent.com/googleapis/googleapis/master/google/api/http.proto

Create Your Protocol Buffer Definition

Create a file proto/bookstore.proto with our bookstore service definition:

syntax = "proto3";

package bookstore.v1;

import "google/api/annotations.proto";

option go_package = "generated/go/bookstore/v1";

service BookstoreService {
    // Get a book by ID
    rpc GetBook(GetBookRequest) returns (Book) {
        option (google.api.http) = {
            get: "/v1/books/{book_id}"
        };
    }
 
    // Create a new book
    rpc CreateBook(CreateBookRequest) returns (Book) {
        option (google.api.http) = {
            post: "/v1/books"
            body: "book"
        };
    }
}

message Book {
    string book_id = 1;
    string title = 2;
    string author = 3;
    int32 pages = 4;
}

message GetBookRequest {
    string book_id = 1;
}

message CreateBookRequest {
    Book book = 1;
}

The magic happens in those google.api.http annotations. The get: "/v1/books/{book_id}" tells the generator to create a GET endpoint where the book_id URL parameter automatically maps to the corresponding field in our GetBookRequest message. Similarly, body: "book" specifies that the entire book field becomes the HTTP request body for the CreateBook endpoint.

What's particularly elegant about this approach is that the same annotations work across multiple generators. The gRPC gateway uses them to create REST endpoints, the OpenAPI generator uses them to create proper path parameters and request body schemas, and as we'll see in Part 2, our MCP plugin can use them to understand how to structure tool inputs.

Generate Code

Once we had our proto file with annotations, the build process became surprisingly straightforward:

export GOOGLEAPIS_DIR=./googleapis

# Generate gRPC service code
protoc -I${GOOGLEAPIS_DIR} --proto_path=proto --go_out=./generated/go  --go_opt paths=source_relative --go-grpc_out=./generated/go --go-grpc_opt paths=source_relative bookstore.proto

# Generate gRPC Gateway for REST endpoints  
protoc -I${GOOGLEAPIS_DIR} --proto_path=proto --grpc-gateway_out=./generated/go --grpc-gateway_opt paths=source_relative bookstore.proto

# Generate OpenAPI specifications
protoc -I${GOOGLEAPIS_DIR} -I./proto --openapi_out=./generated/openapi \
      --openapi_opt=fq_schema_naming=true \
      --openapi_opt=version="1.0.0" \
      --openapi_opt=title="Bookstore API" \
      bookstore.proto

Each protoc command reads the same bookstore.proto file but generates different outputs based on the plugin used. The HTTP annotations provide the routing information that each generator needs to create proper interfaces.

Implement the gRPC Server

Create cmd/server/main.go:

package main

import (
    "context"
    "fmt"
    "log"
    "net"
    "net/http"
    "github.com/grpc-ecosystem/grpc-gateway/v2/runtime"
    "google.golang.org/grpc"
    "google.golang.org/grpc/credentials/insecure"
    pb "proto-to-mcp-tutorial/generated/go"
)

type server struct {
    pb.UnimplementedBookstoreServiceServer
    books map[string]*pb.Book
}

func (s *server) GetBook(ctx context.Context, req *pb.GetBookRequest) (*pb.Book, error) {
    book, exists := s.books[req.BookId]
    if !exists {
        return nil, fmt.Errorf("book not found")
    }
    return book, nil
}

func (s *server) CreateBook(ctx context.Context, req *pb.CreateBookRequest) (*pb.Book, error) {
    book := req.Book
    if book.BookId == "" {
        book.BookId = fmt.Sprintf("book-%d", len(s.books)+1)
    }
    s.books[book.BookId] = book
    return book, nil
}

func main() {
    // Initialize server with some sample data
    srv := &server{
        books: map[string]*pb.Book{
            "book-1": {
                BookId: "book-1",
                Title:  "The Go Programming Language",
                Author: "Alan Donovan",
                Pages:  380,
            },
        },
    }
    
    // Start gRPC server
    go func() {
        lis, err := net.Listen("tcp", ":9090")
        if err != nil {
            log.Fatalf("failed to listen: %v", err)
        }
        s := grpc.NewServer()
        pb.RegisterBookstoreServiceServer(s, srv)
        log.Println("gRPC server starting on :9090")
        if err := s.Serve(lis); err != nil {
            log.Fatalf("failed to serve: %v", err)
        }
    }()
    
    // Start REST gateway
    ctx := context.Background()
    mux := runtime.NewServeMux()
    opts := []grpc.DialOption{grpc.WithTransportCredentials(insecure.NewCredentials())}
    err := pb.RegisterBookstoreServiceHandlerFromEndpoint(ctx, mux, "localhost:9090", opts)
    if err != nil {
        log.Fatalf("failed to register gateway: %v", err)
    }
    
    log.Println("REST server starting on :8080")
    if err := http.ListenAndServe(":8080", mux); err != nil {
        log.Fatalf("failed to serve REST: %v", err)
    }
}

Test Your Implementation

Run the server:

# Init go module
go mod init proto-to-mcp-tutorial && go mod tidy
go run cmd/server/main.go

You should see output like:

gRPC server starting on :9090
REST server starting on :8080

Open a new terminal for testing (keep the server running).

And our clients could interact with the exact same service logic through either gRPC or HTTP, with identical data models and behavior. A curl request to get a book, for example:

curl -X GET http://localhost:8080/v1/books/book-1

And returns exactly as expected:

{
 "bookId": "book-1",
 "title": "The Go Programming Language",
 "author": "Alan Donovan",
 "pages": 380
}

Verify OpenAPI Generation

Check the generated OpenAPI specification:

cat generated/openapi/openapi.yaml

You should see a complete OpenAPI spec that matches your REST endpoints! Look for:

• Path definitions for /v1/books/{bookId} and /v1/books
• Schema definitions for bookstore.v1.Book
• HTTP method specifications (get, post)

Conclusion

Congratulations! You've just built a unified API system where:

1. Single Source of Truth: One proto file defines your entire API
2. Dual Interface: Your service is accessible via both gRPC and REST
3. Automatic Documentation: OpenAPI spec is generated automatically
4. No Sync Issues: Everything stays in sync because it's generated from the same source

Here's what the architecture now looks like:

Next Steps

This foundation sets you up perfectly for Part 2, where we'll extend this same pattern to generate MCP servers for AI integration. The HTTP annotations you've added will help our MCP plugin understand how to structure tool inputs and outputs.

References

1. Google API HTTP Annotations Documentation
2. Protocol Buffers Documentation
3. gRPC Gateway GitHub
4. protoc Plugin Development