Deploying the Agosto IoT Broker in Google Cloud Launcher

 

Getting Started

The Agosto IoT Broker includes the fabulous RabbitMQ polyglot messaging broker and sends/receives messages to Google Cloud Pub/Sub. It is most commonly used to route messages from MQTT Devices ← → Google Cloud Pub/Sub.

With the broker, users can rapidly deploy a solution that can handle many thousands of MQTT clients. The broker architecture can be expanded to support millions of devices through load balancing connections to multiple VMs running RabbitMQ in a clustered configuration (the cloud-launcher version currently does not automatically handle a clustered config - but can be manually set up in more advanced configurations).


Create IoT Broker VM

1. Locate the Agosto IoT Broker in Google Cloud Launcher and click “Launch on Compute Engine”

2. Verify details of your instance. Name your instance and select your Zone.

Note that the default firewall configuration will open the following firewall ports:

  • 15672 - RabbitMQ Web Management

  • 1883 - MQTT


Broker Configuration

You will need to configure the broker to allow connections from MQTT clients.

An admin account used to log into RabbitMQ management console will be created as part of the Cloud Launcher setup. You can log into RabbitMQ management console via http://vm_external_ip_address:15672 with those credentials.

The following steps can be performed from the RabbitMQ admin console or via SSH login to the VM. The steps below show setup from an SSH login into the VM.

Create any “normal” RabbitMQ accounts - these are for the devices (MQTT Clients) that will connect to the broker:

sudo rabbitmqctl add_user mqtt-user <password>

sudo rabbitmqctl set_permissions mqtt-user ".*" ".*" ".*"

(Note that mqtt-user can be any username you want, and you can create several as needed)

 

Optional: The RabbitMQ Admin account’s password can also be changed from the RabbitMQ management console or by logging into the VM.

You can also change the admin password with the following shell command:

sudo rabbitmqctl change_password admin <newpassword>

Advanced Connection Configuration

The main config file is /etc/gcp-iot-adapter.conf. The Cloud Launcher installer has configured the adapter with default values that should work for most people. It will set the current GCP project and a gcpiotadapter user with a random password. It will also create a gcpiotadapter RabbitMQ user to match. If you change the gcpiotadapter user info, the username/password set in RabbitMQ must match the username/password set in the config file.

Any changes made to /etc/gcp-iot-adapter.conf will require a restart of the adapter service.


Troubleshooting

Most problems can be diagnosed by looking at the relevant logs located in these directories: /var/log/gcp-iot-adapter and /var/log/rabbitmq.


Testing the Server

The follow optional activities walk you through sending and receiving messages from an MQTT client to Google Cloud Pub/Sub through the broker.

 

In the config file, there is an example route configured to forward all messages published to the MQTT topic “gateway/#” to a pubsub topic named “projects/<gcp project id>/topics/gateways”.

 

There is also a topic named “projects/<gcp project id>/topics/to-gateway” that is responsible for sending messages from Pub/Sub to MQTT subscribers (ie your devices). When sending a message from Pub/Sub, you will publish the message to this Pub/Sub topic and set the MQTT topic by setting a Pub/Sub attribute named “topic”. More on that later.


Test the Flow of Messages

Make sure that your broker and gcp-iot-adapter are running.

Pub/Sub → Broker → Device

This scenario shows sending a command message to your “device”, which is a “gateway” in our example.

Install your favorite MQTT client (such as the MQTTLens Chrome App) and connect to the broker’s external IP address - this will be the simulated “gateway” - let’s say the gateway’s name is “foo”.

Login with mqtt-user & password set for the MQTT Client access as described in Broker Configuration.

Subscribe to the topic “gateway-command/foo” in MQTTLens. Now your foo gateway is subscribed, waiting for messages:
 

Log into the GCP Cloud Console, go to Pub/Sub, find the “to-gateway” topic:

Click on the dots, hit Publish message, type your message text, add an attribute with key = “topic” and value = “gateway-command/foo” (no quotes), hit Publish button at the bottom of the form:

You’ve just sent a message from Pub/Sub to your gateway! If you did everything correctly, the message will appear in MQTTLens:

 

Device → Broker → Pub/Sub

This scenario shows sending a status message from your “gateway” to Pub/Sub. The default example in the gcp-iot-adapter configuration file (/etc/gcp-iot-adapter.conf) forwards messages from the “gateway/#” MQTT topic to the ”projects/<gcp-project>/topics/gateways” Pub/Sub topic. You’ll need a Pub/Sub client to receive messages, as there is no interface in the Cloud Console to view messages.

Luckily, the latest version of gcloud alpha supports subscribing and pulling messages directly from Pub/Sub.

Note: This assumes that gcloud is installed on your workstation. You can also use the Google Cloud Shell to perform these tasks.


Create a subscription

You only need to do this once:

# Command below goes all on one line
gcloud alpha pubsub subscriptions create my-test-sub --topic=projects/<project                                                  
id>/topics/gateways --project <project id>

 

In MQTTLens, Publish a message to the gateway/foo/status topic:


Pull messages

From your workstation or the Google Cloud Shell, pull the message:

gcloud alpha pubsub subscriptions pull my-test-sub --project <project id> --auto-ack

Notice that the MQTT Topic has been added to a Pub/Sub Attribute named “mqtt_topic”. Your application logic can use that to determine you received a message from the foo gateway. The mqtt_topic_params attribute will contain parameters after the topic name as configured in the adapter config file.

 

Problems?  Verify that you followed the steps in order and that you have no typos in your commands.  Check the logs in /var/log/gcp-iot-adapter and /var/log/rabbitmq.

Congratulations, you’ve just created an end-to-end MQTT → Google Cloud Pub/Sub solution!