Section 12: Microservice Development Guide

Quantra microservices are standalone gRPC-based processes that perform heavy or specialised processing tasks outside the main Django platform process. Each microservice runs as an independent server, communicates via Protocol Buffers over gRPC with mutual TLS (mTLS) authentication, and follows a standardised frame format for streaming data. This guide covers every aspect of building, registering, and operating a Quantra microservice.

12.1 Microservice Architecture

Quantra microservices are gRPC-based, standalone processes. They run independently of the Django web application and communicate exclusively through gRPC streaming calls secured with mTLS. This architecture provides several benefits:

• Process isolation. A crash or memory leak in a microservice does not affect the web platform or other microservices.
• Language flexibility. Although most microservices are written in Python, any language with gRPC support could be used.
• Independent scaling. Each microservice can be scaled horizontally by running multiple instances behind a load balancer.
• Security. mTLS ensures that only authenticated services can communicate with each other.

Each microservice listens on a dedicated port in the range 50051–50090 and is registered in the platform ServiceEndpoint database table. The main platform discovers available microservices by querying this table.

12.2 Creating a New Microservice

Follow these steps to create a new microservice from scratch:

1. Choose a name. Use a descriptive, lowercase, underscore-separated name (e.g., pdf_extractor).
2. Create the directory. Create a new folder under /ms/ with the microservice name.
3. Copy the proto definition. Use the standard microservice.proto from /ms/grpc/.
4. Implement the service. Write a Python gRPC server implementing the MicroService interface.
5. Add CLI argument support. Implement all required command-line arguments.
6. Configure mTLS. Load certificates from /ms/certs/.
7. Register the service endpoint. Add an entry to the ServiceEndpoint table.
8. Add to the qm script. Integrate startup into the qm management script.

mkdir -p /ms/pdf_extractor
cp /ms/grpc/microservice.proto /ms/pdf_extractor/
# Generate Python gRPC stubs
python -m grpc_tools.protoc -I/ms/grpc --python_out=/ms/pdf_extractor --grpc_python_out=/ms/pdf_extractor /ms/grpc/microservice.proto

12.3 Proto Definition

All Quantra microservices share a common proto definition located at /ms/grpc/microservice.proto. This ensures a uniform interface across all services.

syntax = "proto3";

package quantra;

service MicroService {
    // Bidirectional streaming RPC.
    // The client sends a stream of Frame messages and receives
    // a stream of Frame messages in return.
    rpc Call (stream Frame) returns (stream Frame);
}

message Frame {
    string payload = 1;  // JSON-encoded frame data
}

The Call RPC uses bidirectional streaming: both the client and server send a stream of Frame messages. Each Frame contains a JSON-encoded payload string conforming to the Standard Frame Format described in Section 12.6. This design allows microservices to handle arbitrarily large documents by streaming chunks, report progress incrementally, and support long-running operations without HTTP timeouts.

12.4 Implementing the Service

The microservice implementation is a Python gRPC server that inherits from the generated MicroServiceServicer class and implements the Call method.

# /ms/pdf_extractor/server.py
import grpc
import json
import logging
import sys
import os
from concurrent import futures

# Add libs to path
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..', 'libs'))
from version_mixin import VersionMixin

import microservice_pb2
import microservice_pb2_grpc

logger = logging.getLogger(__name__)

SERVICE_VERSION = "1.0.0"

class PdfExtractorServicer(microservice_pb2_grpc.MicroServiceServicer, VersionMixin):

    def Call(self, request_iterator, context):
        """
        Bidirectional streaming handler.
        Receives Frame messages from the client, processes them,
        and yields Frame messages back.
        """
        for request_frame in request_iterator:
            try:
                data = json.loads(request_frame.payload)
                oauth2 = data.get("oauth2", {})
                records = data.get("data", [])
                tod = data.get("tod", "object")
                eos = data.get("eos", False)

                # Process each record
                for record in records:
                    meta = record.get("meta", {})
                    content = record.get("content", {})

                    # Perform extraction logic here...
                    result_content = {"extracted": True, "fields": {}}

                    # Build response frame
                    response_data = {
                        "oauth2": oauth2,
                        "data": [{"meta": meta, "content": result_content}],
                        "tod": "object",
                        "eos": False,
                    }
                    yield microservice_pb2.Frame(
                        payload=json.dumps(response_data)
                    )

                if eos:
                    # Send end-of-stream marker
                    yield microservice_pb2.Frame(
                        payload=json.dumps({"oauth2": oauth2, "data": [], "tod": "object", "eos": True})
                    )

            except Exception as e:
                logger.error("Error processing frame: %s", e)
                error_frame = {"error": str(e), "eos": True}
                yield microservice_pb2.Frame(payload=json.dumps(error_frame))

12.5 CLI Arguments

All Quantra microservices must support a standard set of command-line arguments. This ensures uniform startup behaviour and allows the qm management script to launch all services consistently.

Argument	Required	Description
--port	Yes	The port number the gRPC server will listen on (e.g., 50051).
--ca-cert	Yes	Path to the Certificate Authority certificate file for mTLS verification.
--server-cert	Yes	Path to the server TLS certificate file.
--server-key	Yes	Path to the server TLS private key file.
--max-workers	No	Maximum number of thread pool workers (default: 10).
--log-level	No	Logging level: DEBUG, INFO, WARNING, ERROR (default: INFO).

import argparse

def parse_args():
    parser = argparse.ArgumentParser(description="PDF Extractor Microservice")
    parser.add_argument("--port", type=int, required=True, help="gRPC listen port")
    parser.add_argument("--ca-cert", required=True, help="CA certificate path")
    parser.add_argument("--server-cert", required=True, help="Server certificate path")
    parser.add_argument("--server-key", required=True, help="Server private key path")
    parser.add_argument("--max-workers", type=int, default=10, help="Thread pool size")
    parser.add_argument("--log-level", default="INFO", help="Logging level")
    return parser.parse_args()

12.6 mTLS Support

All microservice communication is secured with mutual TLS (mTLS). Both the client and server present certificates, and each verifies the other against a shared Certificate Authority (CA). Certificates are stored in /ms/certs/.

def create_server_credentials(args):
    """Load certificates and create gRPC server credentials."""
    with open(args.ca_cert, 'rb') as f:
        ca_cert = f.read()
    with open(args.server_cert, 'rb') as f:
        server_cert = f.read()
    with open(args.server_key, 'rb') as f:
        server_key = f.read()

    credentials = grpc.ssl_server_credentials(
        [(server_key, server_cert)],
        root_certificates=ca_cert,
        require_client_auth=True,
    )
    return credentials

def serve(args):
    credentials = create_server_credentials(args)
    server = grpc.server(futures.ThreadPoolExecutor(max_workers=args.max_workers))
    microservice_pb2_grpc.add_MicroServiceServicer_to_server(
        PdfExtractorServicer(), server
    )
    server.add_secure_port(f"[::]:{args.port}", credentials)
    server.start()
    logger.info("PDF Extractor listening on port %d with mTLS", args.port)
    server.wait_for_termination()

The certificate directory structure under /ms/certs/ typically contains:

• ca.crt — The Certificate Authority certificate (shared by all services).
• server.crt — The server certificate for this microservice.
• server.key — The server private key.
• client.crt — The client certificate used by the platform to connect.
• client.key — The client private key.

12.7 Standard Frame Format

All data exchanged between the platform and microservices uses a standardised JSON frame format. Every frame is a JSON object with the following top-level fields:

Field	Type	Description
oauth2	object	Authentication context. Contains at minimum a url field identifying the originating platform URL. May contain additional OAuth2 token information for downstream API calls.
data	array	An array of data objects. Each object contains two dictionaries: meta (metadata about the record such as filename, content type, source identifier) and content (the actual data payload, which varies by service).
tod	string	Type of data. Either "object" (the data array contains document/object records) or "table" (the data array contains tabular row records).
eos	boolean	End of stream marker. When true, indicates this is the final frame in the stream. The data array may be empty in the final frame.

Example Frame

{
    "oauth2": {
        "url": "https://quantra.example.com"
    },
    "data": [
        {
            "meta": {
                "filename": "invoice_001.pdf",
                "content_type": "application/pdf",
                "source_id": "batch_2024_q1",
                "page_number": 1
            },
            "content": {
                "text": "Invoice #1001
Date: 2024-01-15
Vendor: Acme Corp...",
                "confidence": 0.97,
                "fields": {
                    "invoice_number": "1001",
                    "date": "2024-01-15",
                    "vendor": "Acme Corp",
                    "total": "1,234.56"
                }
            }
        }
    ],
    "tod": "object",
    "eos": false
}

End-of-Stream Frame

{
    "oauth2": {"url": "https://quantra.example.com"},
    "data": [],
    "tod": "object",
    "eos": true
}

12.8 Streaming Pattern

The bidirectional streaming pattern allows microservices to handle large files and long-running operations efficiently. Key aspects of the pattern include:

• Chunked file handling. Large documents are split across multiple frames. Each frame carries a chunk of the document in its content dictionary. The microservice reassembles chunks before processing, or processes them incrementally.
• Progress reporting. The microservice can yield intermediate frames with progress metadata (e.g., percentage complete, pages processed) before the final result frames.
• Backpressure. gRPC flow control automatically applies backpressure when the receiver cannot consume frames fast enough, preventing memory exhaustion.
• Error handling. If processing fails for a particular record, the microservice should yield a frame with an error field rather than terminating the entire stream.
• End-of-stream. The final frame must have eos: true to signal that no more data will follow.

12.9 Service Registration

Every microservice must be registered in the Django ServiceEndpoint table so the platform can discover and connect to it. Before assigning a port, always check the database for existing allocations.

# In Django shell or via admin:
from core.models import ServiceEndpoint

# List all current endpoints to find available ports
for ep in ServiceEndpoint.objects.all().order_by("port"):
    print(f"{ep.port}  {ep.name}  {'ACTIVE' if ep.is_active else 'INACTIVE'}")

# Register the new microservice
ServiceEndpoint.objects.create(
    name="pdf_extractor",
    host="localhost",
    port=50058,          # Use the next available port in 50051-50090
    protocol="grpc",
    is_active=True,
)

Alternatively, use the Django admin interface to add the entry. Navigate to Core > Service Endpoints > Add and fill in the fields.

12.10 Adding to the qm Script

The qm script is the central management tool for starting, stopping, and restarting all Quantra services. Every new microservice must be added to this script so it starts automatically with the rest of the platform.

Add an entry for your microservice in the services array of the qm script, specifying the Python executable path, port, and certificate paths:

# Example entry in qm script
start_service "pdf_extractor"     python3 /ms/pdf_extractor/server.py     --port 50058     --ca-cert /ms/certs/ca.crt     --server-cert /ms/certs/server.crt     --server-key /ms/certs/server.key

After adding the entry, use qm restart to restart all services including the new one.

12.11 Shared Libraries

Common functionality shared across multiple microservices is located in /ms/libs/. These libraries are added to the Python path at runtime by each microservice.

Library	Purpose
version_mixin.py	A mixin class that adds version reporting capabilities to microservice servicers. When mixed into a servicer class, it enables the service to respond to version queries with its current version string.
version_utils.py	Utility functions for parsing, comparing, and incrementing version strings. Used by the bump_versions.py deployment script and by microservices that need to report or validate version information.

# Using shared libraries in your microservice
import sys, os
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..', 'libs'))

from version_mixin import VersionMixin
from version_utils import parse_version, bump_build

class MyServicer(microservice_pb2_grpc.MicroServiceServicer, VersionMixin):
    VERSION = "1.0.0"
    # VersionMixin adds version reporting to gRPC metadata

12.12 Testing Microservices

Microservices should be tested at multiple levels:

• Unit tests. Test individual processing functions in isolation, without starting the gRPC server. Mock the frame input and verify frame output.
• Integration tests. Start the microservice on a test port, connect a gRPC client, send test frames, and verify responses. Use test certificates for mTLS.
• End-to-end tests. Test the full pipeline from the platform through the microservice and back, verifying that data flows correctly through the entire system.

# Example unit test
import json
from server import PdfExtractorServicer

def test_extraction():
    servicer = PdfExtractorServicer()
    test_frame = microservice_pb2.Frame(
        payload=json.dumps({
            "oauth2": {"url": "https://test.example.com"},
            "data": [{"meta": {"filename": "test.pdf"}, "content": {"raw": "..."}}],
            "tod": "object",
            "eos": True,
        })
    )
    responses = list(servicer.Call(iter([test_frame]), None))
    assert len(responses) > 0
    last = json.loads(responses[-1].payload)
    assert last["eos"] is True

12.13 Version Management

Each microservice maintains a SERVICE_VERSION constant in its server.py. This version follows the same MAJOR.MINOR.BUILD convention as plugins. The build count must be incremented for every code change. The VersionMixin from shared libraries exposes this version via gRPC metadata, allowing the platform to verify that it is communicating with the expected version of each service.