Hostly

Example: Background worker with a queue ​

This example moves slow work, such as sending e-mails or generating PDFs, out of a web app into a background worker. The app puts jobs into a Redis queue, the worker takes them out and processes them.

text
Internet ──HTTPS──> app (app.example.hostly.ninja, port 3000)
                     │
                     └──> queue:6379 (internal)
                            ^
                            │
                     worker (no port)

The examples use the workspace domain fragment example. Replace it and the images of the app and the worker with your own.

Paid tier

This example runs three services in one workspace. The free tier allows one service per workspace, so this requires the paid tier.

1. Deploy the queue ​

Create a new service:

StepSettingValue
SubdomainOnly reachable internallyon
Service namequeue
ImageImageredis:8
Container app port6379

After the deployment, the service details page shows Internally reachable at queue:6379.

Redis keeps the queue in memory. Jobs that are still waiting are lost when the queue restarts. If that is not acceptable, add a volume at /data (paid tier) and enable Redis persistence in your image, or store jobs in a database instead.

2. Deploy the web app ​

Create a public service for the app:

StepSettingValue
SubdomainOnly reachable internallyoff
Subdomainapp
SSLon
ImageImageregistry.hostly.ninja/example/app:1.0.0
Container app port3000
ConfigurationEnvironment variablesREDIS_URL=redis://queue:6379

3. Deploy the worker ​

The worker only connects outward to the queue. Nobody connects to it, so it needs neither a domain nor a port:

StepSettingValue
SubdomainOnly reachable internallyon
Service nameworker
ImageImageregistry.hostly.ninja/example/worker:1.0.0
This service has no portchecked
ConfigurationEnvironment variablesREDIS_URL=redis://queue:6379

Many frameworks run the app and the worker from the same image with a different start command. Hostly uses the start command of the image, so build a separate image or tag for the worker, or let the image choose its mode from an environment variable, e.g. MODE=worker.

Good to know ​

  • Health of the worker: Hostly cannot send requests to a service without port. The worker counts as healthy as long as its container runs. Check its logs to see whether jobs are processed.
  • More throughput: Enable high availability for the worker to run three instances that process jobs in parallel. This only works if the worker keeps no local state.
  • Scheduled jobs: A worker can also run tasks on a schedule, e.g. a nightly cleanup, with a scheduler library inside the container.
  • Other brokers work the same way, e.g. RabbitMQ on 5672 or NATS on 4222.