Microservice messaging: The event that changed everything

By -

Unknown Event

In event driven microservice architectures, it is usual that the message only carries meaning for the event producer and end consumer(s). Any generic middleware, broker, gateway, or cache are left unable to act on the content because they don’t understand it.

What a waste of opportunity.

But, what if the event message could be understood?

What if a generic microservice middleware could interpret it to update the database, or the cache could use it to update itself, or the client could automatically update its own state?

This article will show a full example, written in Go but applicable to any other language, demonstrating the power you get from using messages with a defined meaning and purpose; how a single event can change everything.

The example will use the following parts:

  • RES protocol - a simple JSON based protocol which defines event messages for common mutations of data
  • NATS server - a message broker focused on simplicity and performance
  • Resgate - a real-time API gateway built upon the RES protocol
  • go-res - a Go package for writing services using the RES protocol
  • BadgerDB - an embeddable, fast, key-value database for Go

The example is based on the Edit Text Example from the go-res package. The example page on GitHub contains instructions on how to run it.

Scope of the example

The RES protocol defines what a resource is, and how to represent it in JSON. In the example, we will serve a most basic resource:

GET: /api/example/greeting

{ "message": "Hello, World!" }

In addition, we will add a method to update the resource:

POST: /api/example/greeting/set

Body:

{ "message": "Hello, Resgate.io!" }

Finally, all the changes should be persisted in the database - BadgerDB in this example.

The real-time synchronization of the clients, and the self-updating cache, are additional bonuses we get from Resgate.

The code

This is what the entire service looks like:

package main

import (
	"log"

	"github.com/dgraph-io/badger"
	res "github.com/jirenius/go-res"
	"github.com/jirenius/go-res/middleware"
)

type Greeting struct {
	Message string `json:"message"`
}

func main() {
	opts := badger.DefaultOptions
	opts.Dir = "./db"
	opts.ValueDir = "./db"
	opts.Truncate = true
	db, err := badger.Open(opts)
	if err != nil {
		log.Fatal(err)
	}
	defer db.Close()

	s := res.NewService("example")
	s.Handle("greeting",
		res.Model,
		res.Access(res.AccessGranted),
		middleware.BadgerDB{
			DB:      db,
			Type:    Greeting{},
			Default: Greeting{"Hello, World!"},
		},
		res.Set(func(r res.CallRequest) {
			var p Greeting
			r.ParseParams(&p)
			r.ChangeEvent(map[string]interface{}{
				"message": p.Message,
			})
			r.OK(nil)
		}),
	)

	s.ListenAndServe("nats://localhost:4222")
}

Let’s break it down.

Explaining the service

We define our data model; a simple json object with a single property:

type Greeting struct {
    Message string `json:"message"`
}

Then some boiler plate to create the BadgerDB database:

opts := badger.DefaultOptions
opts.Dir = "./db"
opts.ValueDir = "./db"
opts.Truncate = true // For Windows users
db, err := badger.Open(opts)
if err != nil {
	log.Fatal(err)
}
defer db.Close()

After that, it is time to create the service instance. We give it the namespace "example":

s := res.NewService("example")

We then add a resource to the service, called "greeting" (the URL path will be /api/example/greeting). To help the middleware, we specify that it is a Model (a json object), and that anyone should be able to access it:

s.Handle("greeting",
	res.Model,
	res.Access(res.AccessGranted),
	/* ... */
)

Then we add our BadgerDB middleware. The middleware will do two things:

  • Serve the resource from the database (or use Default if not found)
  • Update the stored data on any event
middleware.BadgerDB{
	DB:      db,
	Type:    Greeting{},
	Default: Greeting{"Hello, World!"},
},

We also add a handler for calling the set method. When handling the request, we will parse the provided JSON body to get the new greeting message, and before replying with OK to the request, we will send… THE EVENT

res.Set(func(r res.CallRequest) {
	var p Greeting
	r.ParseParams(&p)
	r.ChangeEvent(map[string]interface{}{
		"message": p.Message,
	})
	r.OK(nil)
}),

Last we tell it to connect to NATS server, and start serving the resource through Resgate:

s.ListenAndServe("nats://localhost:4222")

The Change Event

An event message everyone understands

That little ChangeEvent is enough. Any technology familiar with the RES protocol will know what it means, and how to use it to update the state of the resource. It will result in the following:

  • BadgerDB middleware will update the stored data
  • The API gateway, Resgate, will update its cache using only the event
  • All clients connected to Resgate through WebSocket, subscribing to the resource, will automatically update their internal state (and hopefully their reactive views)

All of this with a simple event - no extra code needed.

Conclusion

By using a protocol that defines how to represent data, together with pre-defined events used to mutate it, we can highly reduce the amount of code required to serve and update that data.

No need for custom code to read and store data to the database.
No need for sending events on a separate stream to update the client.
No need for client side code to interpret the event and update the state.

Additionally we can gain benefits such improved cache efficiency, seamless updating of client state, and easier scaling with generic synchronizations between microservice replicas.

Visit Resgate.io for more information about the RES protocol, Resgate, and related resources, including live demos of the resulting real-time API.

Note: The Go gopher was designed by Renee French.