Google Cloud Pub/Sub is a messaging service which lets independent applicaitons exchange messages with Publish/Subscribe model with others. There are pros and cons when using message servcie as a messaging middleware:
- reduce coupling between applications
- use message queue as buffer to hanlde large message request
- publish/subscribe to different topic depending on the situation
- the instantaneity, ordering and duplication of message are not guaranteed
- understand how the message is being delivered to avoid unexpected behavior of applicaitons.
Here are couple things that we may consider before choosing which message service to use:
- performance of message delivery
- scalability, reliability and availability
Here is the Pub/Sub’s product announcement
Google Cloud Pub/Sub brings the scalability, flexibility, and reliability of enterprise message-oriented middleware to the cloud.
Of course we can host our own message service with NSQ, RabbitMQ or other open source projects. However, if you dont have enough time/experience, it’s better to use 3-rd party service to help the team move forward fast.
Above is the message flow between Pub/Sub components: (from https://cloud.google.com/pubsub/docs/overview)
- A publisher application creates a topic in the Google Cloud Pub/Sub service and sends messages to the topic. A message contains a payload and optional attributes that describe the payload content.
- Messages are persisted in a message store until they are delivered and acknowledged by subscribers.
- The Pub/Sub service forwards messages from a topic to all of its subscriptions, individually. Each subscription receives messages either by Pub/Sub pushing them to the subscriber’s chosen endpoint, or by the subscriber pulling them from the service.
- The subscriber receives pending messages from its subscription and acknowledges each one to the Pub/Sub service.
- When a message is acknowledged by the subscriber, it is removed from the subscription’s queue of messages.
Here are the things you need to notice before using the Pub/Sub:
- Messages that are not acknowledge stay in message storage, up to 7 days after publication time(Retry Policy). Add timestamp to message is a good way to filter timeout message.
- Ordering of message is not guaranteed due to performance consideration
- Message duplication is possible. If the duplication of message is NOT allowed, you have to do duplication checking.
- DO NOT remove topic before all messages send to that topic is being received, or you may lost the messages.
- When subscriber is coming up, it’s before to register subscription with its own uuid to prevent name confliction.
And now lets see how to implement the Publisher and Subscriber with Golang!
- Implementation of Cloud Pub/Sub Publisher
- Implementation of Cloud Pub/Sub Pull Subscription
- Implementation of Cloud Pub/Sub Push Subscription