gRPC

Understanding The Problem

Before gRPC, let's understand why it exists.

Imagine Your Current System:

Auth Service
Logger Service
Notification Service

Suppose:

User Login
↓
Auth Service
↓
Logger Service
↓
Notification Service

Now Auth Service needs to ask Notification Service:

"Send Login Alert"

How?

Option 1: Direct Function Call

Inside same application:

sendNotification(user);

Easy.

Because:

Same Process
Same Memory
Same Runtime

Option 2: Different Servers

Now Notification Service lives elsewhere:

Auth Service -> 10.0.0.1

Notification Service -> 10.0.0.2

Function call no longer works.

You need:

Network Communication

Traditional Approach (REST)

Auth service sends:

POST /send-notification

{
	"userId": 100,
	"message": "Login detected"
}

Notification Service responds:

{
	"success": true
}

What Happens Internally?

Step 1: Create JSON

{
	"userId": 100
}

Step 2: Convert JSON to string

"{\"userId\":100}"

Step 3: Send over network

Step 4: Receiver parses JSON

{
	"userId": 100
}

Step 5: Create response JSON

Again serialize.

Again deserialize.

Problem 1: Large Payloads

Look:

{
  "userId": 100,
  "name": "TheJat",
  "email": "TheJat@thejat.in"
}

Every field name travels over network.

userId
name
email

Again and again

For millions of requests:

Huge bandwidth waste

Problem 2: Parsing Cost

JSON is text.

Server must:

String
↓
Parse
↓
Object

For every request.

CPU cost increases.

Problem 3: Weak Contracts

REST API docs may say:

{
	"userId": 100
}

But client sends:

{
	"userid": 100
}

Notice:

userId
userid

Bug.

Detected only at runtime.

Problem 4: SDK Generation

For every service:

Frontend SDK
Java SDK
Go SDK
Node SDK
Python SDK

Developers often write these manually.

Maintenance nightmare.

Google's Challenge

Google had:

Thousands of services
Millions of requests/sec

REST became expensive internally.

They needed:

Fast
Small Payloads
Strong Contracts
Auto-generated SDKs
Streaming

Result:

gRPC

Core Idea of gRPC

Instead of sending:

{
  "userId":100,
  "name":"Manish"
}

Send:

000101010101001010

Smaller.

Faster.

Machine-friendly.

But How Does Client Know Structure?

Answer:

.proto file

Think of it as:

API Blueprint
+
Contract
+
Documentation
+
Code Generator

Example:

message User {
	int32 id = 1;
	string name = 2;
}

This says:

Field 1 = id
Field 2 = name

Instead of sending:

id
name

every request, protobuf sends:

1
2

Much smaller.

First Mental Model

Think of gRPC as:

REST = Human readable

gRPC = Machine optimized

How gRPC Actually Works Internally

Most developers use gRPC like this:

const user = await userClient.getUser({
  id: 101
});

Looks like a normal function call.

But behind the scenes, a lot happens.

The Magic Trick

When you write:

const user = await userClient.getUser({
  id: 101
});

It feels like:

Your Code
    ↓
getUser()
    ↓
Returns User

Reality:

Client Service
     ↓
Serialize Request
     ↓
HTTP/2
     ↓
Network
     ↓
Server
     ↓
Execute Method
     ↓
Serialize Response
     ↓
Network
     ↓
Client

gRPC hides all of this.

RPC Architecture:

A gRPC system has four main components:

Application Code
        ↓
Client Stub
        ↓
gRPC Runtime
        ↓
Network
        ↓
gRPC Runtime
        ↓
Server Stub
        ↓
Application Code

Component 1: Application Code

Suppose you are building your Payment Service.

You write:

const user = await userClient.getUser({
  id: 101
});

You don't care about:

• TCP
• HTTP/2
• Serialization
• Retries
• Packet framing

You just call a method.

Component 2: Client Stub

The Client Stub is generated automatically from the .proto file.

Example:

service UserService {
  rpc GetUser(GetUserRequest)
      returns (UserResponse);
}

gRPC generates something similar to:

class UserServiceClient {
  async getUser(req) {}
}

This generated code is called a:

Stub

Think of a stub as:

Local Proxy

It looks like the real service.

But it isn't.

Visual

Your Code
     ↓
Client Stub
     ↓
Actual Network Call

Component 3: Channel

Every gRPC client maintains a Channel.

Client
   ↓
Channel
   ↓
Server

A channel is:

Persistent Connection

Usually one HTTP/2 connection.

Why Important?

REST often creates many connections.

Request 1
Connection A

Request 2
Connection B

Request 3
Connection C

gRPC:

Single HTTP/2 Connection

Connection
 ├─ Stream 1
 ├─ Stream 2
 ├─ Stream 3
 └─ Stream 4

This dramatically improves performance.

Component 4: Server Stub

Server side also has generated code.

Example:

service UserService {
  rpc GetUser(GetUserRequest)
      returns (UserResponse);
}

Generated interface:

class UserService {
  getUser() {}
}

You implement it:

class UserService {
  async getUser(request) {
      return findUser(request.id);
  }
}

This is the actual business logic.

Complete Request Journey

Let's trace one request.

Step 1: Application Code

await client.getUser({
	id: 101
}0;

Step 2: Client Stub Receives Request

GetUserRequest

{
  "id":101
}

Step 3: Serialization

Convert into protobuf binary.

Not:

{
  "id":101
}

Instead:

000010101011010

Much smaller.

Step 4: gRPC Frame Creation

gRPC adds metadata.

Message Length
Compression Info
Flags
Payload

Result:

gRPC Frame

Step 5: HTTP/2 Stream

Place into a stream.

Connection
   ├─ Stream 15
   ├─ Stream 16
   └─ Stream 17

You request may be Stream 16.

Step 6: Network Transfer

Request Travels:

Payment Service
       ↓
Switch
       ↓
Router
       ↓
Kubernetes Network
       ↓
User Service

Step 7: Server Receives Request

Server gRPC runtime reads:

Stream 16

Extracts:

GetUserRequest

Step 8: Deserialilzation

Binary:

000010101011010

becomes:

{
  "id":101
}

(or language-specific object)

Step 9: Method Dispatch

Server checks:

Requested Method?

Header say:

/UserService/GetUser

So gRPC calls:

getUser()

Step 10: Business Logic Executes

async getUser(req) {
   return db.user.findById(req.id);
}

Database queried.

User found.

Step 11: Response Created

{
  "id":101,
  "name":"TheJat"
}

Step 12: Serialize Again

Response converted to protobuf binary.

10101010110101001010

Step 13: Send Throudh Same HTTP/2 Stream

Steam 16

returns response.

Step 14: Client Receives

gRPC runtime decodes.

Stub converts it back.

Step 15: Your Code Gets Result

const user = await client.getUser({
  id:101
});

console.log(user.name);

Output:

TheJat

Looks like a local function.

Actually crossed the network.

How Does Server Know Which Method to Call?

Every gRPC request contains:

/service/method

Example:

/UserService/GetUser

Similar to REST:

GET /users/101

What Happens If Server Is Down?

Client sends:

GetUser()

Server unreachable.

gRPC returns:

UNAVAILABLE

Error code.

Common gRPC Status Codes

These are standardized across languages.

Protocol Buffers

If gRPC is the car,

gRPC = Car

then Protocol Buffers (Protobuf) is the engine.

Protocol Buffers
=
Data Definition Language
+
Serialization Format
+
Code Generator

Without Protobuf, gRPC would not exist.

What Problem Does Protobuf Solve?

Suppose your Payment Service sends:

{
  "paymentId": 1001,
  "amount": 500,
  "currency": "INR"
}

This is human-readable.

But machine don't care about readability.

They care about:

Size
Speed
Compatibility

Google wanted:

Smaller payloads
Faster serialization
Cross-language support
Versioning

Result:

Protocol Buffers

What Is a .proto File?

Think of .proto as:

Database Schema
+
API Contract
+
Documentation
+
SDK Generator

Example:

syntax = "proto3";

message User {
  int32 id = 1;
  string name = 2;
}

This defines:

A User Object

First Rule

Every proto file starts with:

syntax = "proto3";

Example:

syntax = "proto3";

message User {
  int32 id = 1;
}

Why Proto3?

Google had:

proto1
proto2
proto3

Today:

proto3 = standard

Almost all modern systems use it.

Understanding Messages

A message is similar to:

interface User {}

or

class User {}

Example:

message User {
  int32 id = 1;
  string name = 2;
}

Equivalent TypeScript:

interface User {
   id: number;
   name: string;
}

The Most Important Thing: Field Numbers

Look carefully:

message User {
  int32 id = 1;
  string name = 2;
}

People often think:

1 and 2 are indexes

But they are not.

They are:

Field tags

These tags are what travel on the wire.

Why Not Send Field Names?

JSON sends:

{
  "id": 10,
  "name": "TheJat"
}

Network carries:

id
name

every time.

Protobuf sends:

1-> 10
2-> TheJat

Much smaller.

Golden Rule

Never change field numbers.

Bad:

message User {
  int32 id = 1;
  string name = 2;
}

Later:

message User {
  int32 id = 2;
  string name = 1;
}

Disaster.

Because old clients still think:

1 = id
2 = name

Data corruption.

Real Production Rule

Once released:

id = 1

must remain:

id = 1

forever.

Google follows this rule religiously.

Primitive Types

Common types:

double
float

int32
int64

uint32
uint64

bool

string

bytes

int32 vs int64

Use:

int32

for:

1
100
100000

Use:

int64

for:

Database IDs
Timestamps
Large counters

Example:

int64 user_id = 1;

Most production systems use int64 for IDs.

Strings

message User {
	string email = 1;
}

Example:

thejat@thejat.in

Boolean

bool is_active = 1;

Possibly values:

true
false

Bytes

Raw binary.

bytes image = 1;

Useful for:

Files
Images
Certificates
Encryption Keys

Repeated Fields (Arrays)

JSON:

{
  "roles": [
      "admin",
      "manager"
  ]
}

Proto:

message User {
	repeated string roles = 1;
}

Equivalent:

roles: string[]

Nested Messages

Example:

message Address {
	string city = 1;
	string state = 2;
}

message User {
	string name = 1;
	Address address = 2;
}

Result:

{
  "name":"TheJat",
  "address":{
      "city":"KKR",
      "state":"Haryana"
  }
}

Enums

Suppose payment status.

Bad:

string status = 1;

People may send:

success
SUCCESS
done
completed

Inconsistent.

Use enum.

enum PaymentStatus {
  PENDING = 0;
  SUCCESS = 1;
  FAILED = 2;
}

Then:

message Payment {
	Payment Status = 1;
}

Why Enum Starts At 0?

Proto3 requires:

enum PaymentStatus {
	PENDING = 0;
}

First value must be zero.

It is the best practice.

Maps

JSON:

{
  "metadata": {
      "ip":"1.2.3.4",
      "browser":"chrome"
  }
}

Proto:

map<string,string> metadata = 1;

Equivalent:

Record<string,string>

Optional Fields

Proto3 originally removed optional fields.

Now they are back.

Example:

message User {
	optional string middle_name = 1;
}

Useful when:

Value absent

must be different from

Empty string

Oneof

Suppose login can happen using:

Email
OR
Phone

Never both.

Use:

message LoginRequest {
	oneof identifier {
		string email = 1;
		string phone = 2;
	}
}

Valid:

{
	"email": "thejat@thejat.in
}

Valid:

{
	"phone": "123456790"
}

Invalid:

{
	"email": "thejat@thejat.in"
	"phone": "123456790"
}

Only one field can exist.

Services

Now let's connect messages with gRPC.

service UserService {
	rpc GetUser(GetUserRequest)
		returns (UserResponse);
}

This means:

Method: GetUser()

Input: GetUserRequest

Output: UserResponse

Example Complete Proto

syntax = "proto3";

package user;

message GetUserRequest {
  int64 user_id = 1;
}

message UserResponse {
  int64 id = 1;
  string name = 2;
  string email = 3;
}

service UserService {
  rpc GetUser(GetUserRequest)
      returns (UserResponse);
}

This single file generates:

NodeJS SDK
Java SDK
Go SDK
Pythong SDK
C# SDK
Rust SDK

Automatically.