First glimpse into gRPC through Python (Part 2)
Tags: python, grpc, microservices, restapi
Published: 2023-01-19 18:52:00

## 1. Previous Post
[First glimpse into gRPC through Python (Part 1)](https://ivanyu2021.hashnode.dev/first-glimpse-into-grpc-through-python-part-1)

## 2. Implementation and Examples
After introduced gRPC concept and development flow in previous post, we will show how to implement gRPC in python.

[Example Code link](https://github.com/ivanyu199012/7.-gRPC-app) (To run the below examples, please go through `README.md` in the link)

### 2.1. 4 kinds of service method
There are 4 kinds of service methods in gRPC and the difference between them is to apply **steaming** or not in request or response.

#### 2.1.1. Simple RPC
This type is just like the http request and response. Client send a single request to server and server reply a single reponse.

Proto file
```protobuf
rpc doSimple ( Request ) returns ( Response ) {}
```

Server
```python
def doSimple(self, request, context):
	output = f"Hello { request.input }!"
	return Response( output=f"Hello { request.input }!" )
```

Client
```python
def call_doSimple( self ):
	with grpc.insecure_channel('localhost:50051') as channel:
		stub = SampleServiceStub( channel )
		request = Request( input=self.faker.name() )
		logger.info( f"doSimple client sent: { request.input }" )
		response = stub.doSimple( request )
		logger.info( f"doSimple client received: { response.output }" )
```

Command line output
```cmd
[ 2022-07-15 09:43:27,530 ][ INFO ][ call_doSimple ] doSimple client sent: Lindsay Ross
[ 2022-07-15 09:43:27,531 ][ INFO ][ call_doSimple ] doSimple client received: Hello Lindsay Ross!
```

#### 2.1.2. Response-streaming RPC
For this type, streaming happens in the response. Client sends a single request to server and server returns a streaming of multiple responses.

Proto file
```protobuf
rpc doResponseStreaming( Request ) returns ( stream Response ) {}
```

Server
```python
def doResponseStreaming(self, request, context):
	faker = Faker()
	name_list = [ *[ faker.name() for i in range( 3 ) ], request.input ]

	for name in name_list:
		time.sleep( 0.5 )
		yield Response( output=name )
```

Client
```python
def call_doResponseStreaming( self ):
	with grpc.insecure_channel('localhost:50051') as channel:
		stub = SampleServiceStub( channel )
		request = Request( input=self.faker.name() )
		logger.info( f"doResponseStreaming client sent: { request.input }" )
		response_generator = stub.doResponseStreaming( request )
		for response in response_generator:
			logger.info( f"doResponseStreaming client received: { response.output }" )
```

Command line output
```cmd
[ 2022-07-15 10:14:27,347 ][ INFO ][ call_doResponseStreaming ] doResponseStreaming client sent: Veronica Good
[ 2022-07-15 10:14:27,971 ][ INFO ][ call_doResponseStreaming ] doResponseStreaming client received: Richard Torres
[ 2022-07-15 10:14:28,472 ][ INFO ][ call_doResponseStreaming ] doResponseStreaming client received: Monica Russo
[ 2022-07-15 10:14:28,985 ][ INFO ][ call_doResponseStreaming ] doResponseStreaming client received: Sean Lane
[ 2022-07-15 10:14:29,498 ][ INFO ][ call_doResponseStreaming ] doResponseStreaming client received: Veronica Good
```

#### 2.1.3. Request-streaming RPC
Similar to the above type, but the streaming happens in the request. Client sends a streaming of multiple requests to server and server returns a single response.

Proto file
```protobuf
rpc doRequestStreaming ( stream Request ) returns ( Response ) {}
```

Server
```python
def doRequestStreaming(self, request_iterator, context):
	result_list = []
	for request in request_iterator:
		result_list.append( request.input.upper() )

	return Response( output=", ".join( result_list ) )
```

Client
```python
def call_doRequestStreaming( self ):

	def get_fake_name_generator():
		faker = Faker()
		for _ in range( 10 ):
			time.sleep( 0.5 )
			name = faker.name()
			logger.info( f"doRequestStreaming client sent: { name }." )
			yield Request( input=name )

	with grpc.insecure_channel('localhost:50051') as channel:
		stub = SampleServiceStub( channel )
		request = get_fake_name_generator()
		response = stub.doRequestStreaming( request )
		logger.info( f"doRequestStreaming client received: { response.output }" )
```

Command line output
```cmd
[ 2022-07-15 10:21:08,058 ][ INFO ][ get_fake_name_generator ] doRequestStreaming client sent: Courtney Hammond.
[ 2022-07-15 10:21:08,562 ][ INFO ][ get_fake_name_generator ] doRequestStreaming client sent: James Petersen.
[ 2022-07-15 10:21:09,070 ][ INFO ][ get_fake_name_generator ] doRequestStreaming client sent: Tom Anderson.
[ 2022-07-15 10:21:09,073 ][ INFO ][ call_doRequestStreaming ] doRequestStreaming client received: COURTNEY HAMMOND, JAMES PETERSEN, TOM ANDERSON
```

#### 2.1.4. Bidirectionally-streaming RPC
As you may already guess the meaning from the name, the streaming happens in both request and response.

Proto file
```protobuf
rpc doBidirectional ( stream Request ) returns ( stream Response ) {}
```

Server
```python
def doBidirectional(self, request_iterator, context):
	for request in request_iterator:
		yield Response( output=request.input + " is excellent!" )
```

Client
```python
def call_doBidirectional( self ):

	def get_fake_name_generator():
		faker = Faker()
		for _ in range( 3 ):
			time.sleep( 0.5 )
			name = faker.name()
			logger.info( f"doRequestStreaming client sent: { name }." )
			yield Request( input=name )

	with grpc.insecure_channel('localhost:50051') as channel:
		stub = SampleServiceStub( channel )
		request = get_fake_name_generator()
		response_generator = stub.doBidirectional( request )
		for response in response_generator:
			logger.info( f"doBidirectional client received: { response.output }" )
```

Command line output
```cmd
[ 2022-07-15 10:41:11,994 ][ INFO ][ get_fake_name_generator ] doRequestStreaming client sent: Sherry Hanson.
[ 2022-07-15 10:41:11,996 ][ INFO ][ call_doBidirectional ] doBidirectional client received: Sherry Hanson is excellent!
[ 2022-07-15 10:41:12,497 ][ INFO ][ get_fake_name_generator ] doRequestStreaming client sent: Danielle Jones.
[ 2022-07-15 10:41:12,499 ][ INFO ][ call_doBidirectional ] doBidirectional client received: Danielle Jones is excellent!
[ 2022-07-15 10:41:12,999 ][ INFO ][ get_fake_name_generator ] doRequestStreaming client sent: Alexis Goodwin.
[ 2022-07-15 10:41:13,001 ][ INFO ][ call_doBidirectional ] doBidirectional client received: Alexis Goodwin is excellent!
```
### 2.2. Special Data Types and default value
gRPC can handles some speical data types, like date time, list or map, but need to pay some attention to avoid bug.

#### 2.2.1. Date Time
gRPC use timestamp to handle date time and it uses timezone.

In proto file
```protobuf
google.protobuf.Timestamp date = 1;
```

You need to include timezone in the input date time in order to transfer the data correctly to the server side.

Below is the output send the date **WITHOUT** timezone from client to server side:
```cmd
[ 2022-07-15 11:04:52,633 ][ INFO ][ call_doSpecialDataType ] doSpecialDataType client sent: request.date=2022-07-15 11:04:52
[ 2022-07-15 11:04:52,654 ][ INFO ][ doSpecialDataType ] doSpecialDataType Server received: request.date=2022-07-15 20:04:52
```
We can see there are 9 hours difference between client and server since gRPC regards the date time without timezone as UTC time. When the date input received by the server, it automatically add 9 hours as we are in UTC+09:00 region.

#### 2.2.2. List
gRPC can handle list datatype by simply adding `repeated` in front of the field in proto file.
```protobuf
repeated string names = 1;
```

#### 2.2.3. Map
gRPC can handle map datatype by defining map as the type and specify key and value type in proto file.
```protobuf
map<string, string> name2phoneNumMap = 3;
```

#### 2.2.4. Default value
According to this [link](https://developers.google.com/protocol-buffers/docs/proto3#default), it mentioned that
> if a scalar message field is set to its default, the value will not be serialized on the wire.

We will explain the meaning in the following example:

proto file
```protobuf
repeated CardInfo cardInfos = 4;

message CardInfo {
	string name = 1;
	int32 numberOfCreditCard = 2;
}
```

We have a list of cardInfo and each card info contains an integer field called `numberOfCreditCard`. In the response, we set `numberOfCreditCard` of last CardInfo of the cardInfo list to be 0.

Server
```python
cardInfos = request.cardInfos
cardInfos.append( CardInfo( name="Boris Lee", numberOfCreditCard=0 ) )
```

Command line output
```cmd
[ 2022-07-15 11:21:39,200 ][ INFO ][ call_doSpecialDataType ] doSpecialDataType client received:  response.cardInfos= [name: "Katherine Soto"
numberOfCreditCard: 1
, name: "Kerry Powell"
numberOfCreditCard: 1
, name: "Mrs. Christina Hicks DDS"
numberOfCreditCard: 1
, name: "Boris Lee" <- No "numberOfCreditCard"
]
```

We can see that Boris Lee does **NOT** have numberOfCreditCard. Since 0 is regarded as default value, it will not be serialized on the wire and transfer back to client.

To solve this problem, we need to add `optional` in front of the field in proto file.
```protobuf
optional int32 numberOfCreditCard = 2;
```
Generate the code and run the program again, you can see the zero appeared.
```cmd
, name: "Boris Lee"
numberOfCreditCard: 0
]
```

## 3. Unit test
We can use python native [unit test framework](https://docs.python.org/3.9/library/unittest.html) to write unit test for gRPC. This is crucial as you do not need to switch on/off the gRPC server again and again whenever there is change in your code in order to test your code manually.

### 3.1. Create test server
First, you need to create a test server for receiving gRPC call in the `setUp` method so that whenever the test method is called, it will first set up the test server.

```python
class SampleServiceTest(unittest.TestCase):

	def setUp(self):
		logger.info( f"=== Method: { self._testMethodName } =======" )
		servicers = {
			sample_service_pb2.DESCRIPTOR.services_by_name['SampleService']: SampleService()
		}

		self.test_server = grpc_testing.server_from_dictionary(
			servicers, grpc_testing.strict_real_time())
```

### 3.2. Create test method
Next, you need to create at least one test method for each gRPC method you want to test.

```python
def test_doSimple(self):
	faker = Faker()
	request = sample_service_pb2.Request( input=faker.name() )

	doSimple_method = self.test_server.invoke_unary_unary(
		method_descriptor=(sample_service_pb2.DESCRIPTOR
			.services_by_name['SampleService']
			.methods_by_name['doSimple']),
		invocation_metadata={},
		request=request, timeout=1)

	response, _, code, _ = doSimple_method.termination()
	self.assertEqual( code, grpc.StatusCode.OK )
	self.assertEqual( response.output, f"Hello { request.input }!" )
```

### 3.3. Run test
Finally, run your test in the main thread

```python
if __name__ == '__main__':
	unittest.main()
```

For detail, take a look in [sample_service_test.py](https://github.com/ivanyu199012/7.-gRPC-app/blob/master/sampleGrpcProj/sample_service_test.py)

## 4. Reason I wrote this series of blogs

I wrote this blog as I found that when I first learnt about gRPC, it is a little bit difficult to grasp the development method in a fast manner. There are a lot of built-in examples in the gRPC site, but it is a little bit too complicate for a beginner.

At first, what I wanted to know the most are:

	1. How can I make a new method?
	2. How can I call the method in a microservice through gRPC?
	3. How can I handle the input and output?
	4. How can I do the unit test?

There are resources in Internet, but they are either scattered around or they used a lot of words to explain without a diagram. As a result, I decided to write this blog to present gRPC (using Python) in a simple and practical way so newcomer can pick up the concept easily and then start development immediately.