Some time ago I built an API for local fintech companies alongside Emmanuel. While trying to sell this product, we were asked several times for webhooks. We didn't find a great tool that was easy to use, language-agnostic and cloud-native; something that could scale independently of our core service. We decided to build it ourselves.
Webhooks are the glue of the modern internet. They are used for asynchronous communication by API providers to notify apps/integrators about events that have occurred in their account. I like to say it as: if you make a payment on Shopify with Stripe’s checkout and Stripe fails to send a webhook event to Shopify’s servers about your payment, you won’t get a delivery of your package even though your card has been debited. This explains the importance of webhook events to many consumer apps.
Today, implementing a webhooks delivery system is a highly fragmented problem. Stripe's system is designed for security (with features like replay attacks, rolling secrets etc), Twilio for performance (with features like connection override), Pagerduty for flexibility (with features like payload filtering) and many more. We'd love to make these features available to everyone with Convoy. We're already helping a few companies solve problems they didn't know they had because of the visibility Convoy gives.
API providers need to push webhook events reliably. It’s a pain to build this from scratch. At first glance, it seems like good old HTTP POST, but with time it becomes more tricky as the requirements pile up: dealing with bad endpoints, pushing events to multiple endpoints as configured by your user, rate-limiting of customer endpoints, security of delivery, UX for managing retries. We take care of all that and give you one less infrastructure to worry about.
Convoy is open-source software that exposes a REST API. Developers push events to it using the REST API (we aim to support other means of ingesting data in the future for high volume environments – this is a feature Shopify provides today) then we sign the payload, publish the events, apply retries, etc. We also provide a management UI to manually retry events to user endpoints.
Convoy consists of 3 core components—in addition to the REST API, there is a job queue and a storage layer. The REST API server is used to create applications, endpoints and events. The job queue currently supports 2 backends: in-memory and Redis. The storage layer currently supports 2 backends: on-disk and MongoDB. All events pushed to Convoy are saved to storage and enqueued on the job queue for workers to deliver.
Most of our users use our product to push webhook events to their customers’ endpoints. Customers also use Convoy as a broker for inter-service communication.
We make money by providing a managed service. We've been running the managed service for a while now and charge $1/5k events, but those details are not on our website yet.
The repository is available at https://github.com/frain-dev/convoy and a getting started doc exists here: https://getconvoy.io/docs/guide. Anyone can download the binary or Docker image from here: https://getconvoy.io/download and run it wherever they choose - AWS, Azure, GCP, etc.
We’d love you to try it out and hear your thoughts! We’d also love to hear about your experiences building custom webhook software and the challenges you’ve faced?
by Andrey
Azimov