Let’s create a Chatbot which can execute tasks asynchronously using Celery and communicate via WebSockets.
We will use Django as a base for our application:
- HTTP requests handler
- App configuration
- Template engine
We need to use Redis in a couple of places:
- Cache Storage
- Celery Message Broker and Result Backend
- Channel layer for the WebSocket communication
We will use Heroku for the deployment, so we don’t need to install and configure Redis manually. Redis DSN will be available as an environment variable REDIS_URL so we can use it everywhere we need it.
Redis as a Cache Storage
Django does not support Redis internally, so we need to use the extra package. We are going to usedjango-redis. It’s full-featured Redis cache backend for Django.
pip install django-redis
This will install a couple more dependencies, including redis-py — Python interface to the Redis.
We only need to update our Django project configuration with the CACHES settings.
Celery with Redis as a Message Broker
We will have some tasks which may take a while. For example, getting a response from the remote server. So we need to do them in the background and send the result back to the client when it will be available.
Configuration for Celery is pretty simple, we are going to reuse our REDIS_URL for the CeleryBROKER_URL and RESULT_BACKEND.
Let’s define our Celery instance inside project/celery.py:
And we need to import Celery instance in our project, to ensure the app is loaded when Django starts.
So now our Celery is ready and we can add background tasks to our project. For example this one:
WebSockets with Django Channels
We want to send a result from to the client immediately as it is available. Without querying it every couple seconds via scheduled AJAX calls.
WebSocket is a computer communications protocol, providing full-duplex communication channels over a single TCP connection. So user can send data to the backend and backend can send data to the user.
Django Channels are the way to go if you need to have WebSockets. There two versions available now 1.x and 2.x. They have major changes and upgrade from 1.x to 2.x can be a big project if you have a fairly big application. Version 1.x is no longer supported, so for new projects, you should use the 2.x branch.
Django-Channels has really good documentation available on the official website, so I encourage you to check.
Key moments are — we need to change couple things in our application to support WebSockets.
There is nothing to configure for our application, we can just run it same way as we run WSGI application, but instead of WSGI we need to point it to the ASGI application:
ASGI configuration looks the same as WSGI but points to the Django Channels application instead:
Django-Channels and Redis
Django channels are some kind of communication system, which allow multiple consumer instances to talk with each other, and with other parts of Django.
A channel layer provides the following abstractions:
- A channel is a mailbox where messages can be sent to. Each channel has a name. Anyone who has the name of a channel can send a message to the channel.
- A group is a group of related channels. A group has a name. Anyone who has the name of a group can add/remove a channel to the group by name and send a message to all channels in the group.
Every consumer instance has an automatically generated unique channel name, and so can be communicated with via a channel layer.
So let’s define CHANNEL_LAYERS for that project. We are going to use channels_redis for that. So we need to define backend and provide Redis DSN:
WebSocket Consumers and Routing
Now we need to define how we will interact with our clients — frontend JS calls. So we will receive every message (as a JSON string) from the client and do something with it.
Here how it may look:
And as our application can have multiple consumers (same way as we may have multiple Django views) we need to have URL routing similar to the Django urls.py file:
So when the client sent a message to the WebSocket /ws/chat/ path — it will be processed by our ChatConsumer.
And finally, we need to include our application URLs to the main router — project/routing.py:
And point to it from out project/settings.py file:
ASGI_APPLICATION = "project.routing.application"
Deployment to Heroku
I will use Heroku for deployment as it’s the quickest way to deploy everything in one step. And also — it’s free (they required an account verification though).
That file contains processes types definition:
Here we define what we need to do during the application release phase and configure two application instances:
- web — Daphne server which will listen on $PORT and handle our https:// and wss:// requests.
- worker — Celery worker instance to process our asynchronous task queue.
Heroku app.json manifest
This is the file which will be used to create a new Heroku website instance from scratch.
- Environment variables we need to be set for out app (like Django SECRET_KEY value)
- Which services we need to have (PostgreSQL, Redis)
- Which type of the virtual machine we want to have heroku/python.
That file can also be used to create a nice Deploy to Heroku button. So it can be deployed in one click next time.
This was a brief explanation without going too much in the details (which will be covered in the next articles).
Project available on Github: https://github.com/inoks/django-chatbot
Demo Chatbot: https://django-chat-bot.herokuapp.com/