Get Start with RabbitMQ

RabbitMQ is an open source & enterprise level messsage broker using AMQP protocal. The AMQP protocol is designed for asychronization and message dispatch. RabbitMQ can be used in time-consuming backend jobs such as sending emails to client, generating file for user to download, and so on.

Install on Mac

brew update
brew install rabbitmq

Then install the client Pika for RabbitMQ

pip install pika

Start RabbitMQ

/usr/local/sbin/rabbitmq-server

Consumer & Producer Problem:

In parallel programming, Consumer & Producer Problem refers to one type of the most frequently used cases. Simply, Producers create data in a fixed size cache, while Consumers would take data from that cache. To make the whole process working properly, it is a must that producers would not produce new data while the cache is full, and consumers will not take more when the cache is empty.

VHOST (Virtual host) in RabbitMQ

Vhost is an important concept in RabbitMQ, which you can create queue, binding and exchange seperately. By default, RabbitMQ comes with a inner-built vhost named ‘/’. However we need more vhosts, queues to organize our tasks when there are too many. Just like in programing we have different naming space and modules.

? Step 1:

Create a new user with name ben and password 1234

sudo /usr/local/sbin/rabbitmqctl add_user ben 1234

? Step 2:

Create a new vhost

sudo /usr/local/sbin/rabbitmqctl add_vhost new_host

? Step 3:

Set permissions on the new vhost

sudo /usr/local/sbin/rabbitmqctl set_permissions -p new_host ben ".*" ".*" ".*"

1st “.*” means enable user ben to add and delete Queue and Exchange
2nd “.*” means enable user ben to publish new messages
3rd “.*” means enable user ben to read new messages

? Step 4:

Now we can gete all the vhosts in current system

/usr/local/sbin/rabbitmqctl list_vhosts

Results:

➜  ~ /usr/local/sbin/rabbitmqctl list_vhosts
Listing vhosts
/
new_host
➜  ~

? Step 5:

Sure thing we can list all the queues in a vhost:

/usr/local/sbin/rabbitmqctl list_queues -p /

Results

➜  ~ /usr/local/sbin/rabbitmqctl list_queues -p /
Listing queues
standard    0

? Step 6:

There is also an additional cmd to get all the users:

/usr/local/sbin/rabbitmqctl  list_users

Results:

➜  ~ /usr/local/sbin/rabbitmqctl  list_users
Listing users
ben []
guest   [administrator]
➜  ~

Show me the code!

The following is an full working example:

rabbitmq_producer.py
https://gist.github.com/arkilis/e48606239023759b5bbc5d70f5396e3b

import sys
import pika

parameters = pika.URLParameters('amqp://guest:guest@localhost:5672/%2F')
connection = pika.BlockingConnection(parameters)
channel = connection.channel()

channel.exchange_declare(exchange='web_develop', exchange_type='direct',
                         passive=False, durable=True, auto_delete=False)
if len(sys.argv) != 1:
    msg = sys.argv[1]
else:
    msg = 'hah'

props = pika.BasicProperties(content_type='text/plain', delivery_mode=2)
channel.basic_publish('web_develop', 'xxx_routing_key', msg,
                      properties=props)
connection.close()

rabbitmq_consumer.py
https://gist.github.com/arkilis/5d73b47ff55bf95545db520a5cd43fe2

import pika

def on_message(channel, method_frame, header_frame, body):
    channel.basic_ack(delivery_tag=method_frame.delivery_tag)
    print body

parameters = pika.URLParameters('amqp://guest:guest@localhost:5672/%2F')
connection = pika.BlockingConnection(parameters)
channel = connection.channel()

channel.exchange_declare(exchange='web_develop', exchange_type='direct',
                         passive=False, durable=True, auto_delete=False)

channel.queue_declare(queue='standard', auto_delete=True)
channel.queue_bind(queue='standard', exchange='web_develop',
                   routing_key='xxx_routing_key')

channel.basic_consume(on_message, 'standard')

try:
    channel.start_consuming()
except KeyboardInterrupt:
    channel.stop_consuming()

connection.close()

Run with consumer first, python rabbitmq_consumer.py

The run python rabbitmq_producer.py msg in another terminal. You would find out a new message showing up from consumer terminal.

Update

By default, RabbitMQ provides a default account with Guest/Guest with admin privilege. When publishing to production, password must be changed!