Over-engineering a telegram bot with microservices
A bit of history
Almost 3 years ago I started my fun side project to code. As a heavy telegram user, I thought it would be awesome if I can also integrate my music library into telegram as well. The only dynamic way for this I thought is using bot in telegram. I checked a lot of bots but seemingly most of them were either broken or they were sending tons of ads which makes your music library really messy. So, I decided to build my own using youtube as main source of the music (as I use youtube to listen the music). It took only around 10 minutes to build very first version of youtube to mp3 bot. All it was capable of doing was accepting the youtube video url and sending back a mp3 version. Later on, I saw that people are somehow finding my bot in telegram and started to use it. I have built user management for this to keep track of number of users. Also added bunch of functionalities like banning some users, sending some notification to all users, checking on in-active users to ask if they have any suggestion or complaint. Number of users are currently 7000. I integrated sentry to be able to monitor and detect the errors and connected it to discord channel so I can receive everything in time. Later on, I decided to add payment as I spend money on hosting a VPS and separate database. I integrated buymeacoffee link to the bot and I limited number of free usage to only 1 download a day. But to be honest, I would say buymeacoffee is a complete rip off, they charge a lot and hard to payout. So in the last month, as I saw that paying customers increased, I thought about integrating stripe into the bot, but it comes with its difficulties. Stripe has excellent documentation and in fact, I should accept that it is the best documented platform I have ever seen in my entire development career. In stipe, once you integrate payment to your application using stripe API, event for each action on stripe will be generated accordingly, and the good part is that you can setup a webhook endpoint to receive these events on your application and take some actions based on those events. Hence, I needed to implement some more stuff if I want to achieve custom payment integration.
Current architecture
Above is the current architecture of the bot that I host. Previously it was just bot and MongoDB. Now I kind of implemented microservice design and we have event bus with redis.
Bot
Bot itself is responsible for processing user inputs and providing output for the clients. It also handles conversion of the youtube videos to mp3 and there is currently no separate microservice for this as well. It stores the user info in the database and some needed metadata for conversion. Bot also consumes events that are produced by payment webhook API microservice. For the events, redis stream is used. Separate worker in the bot microservice is responsible for consuming the events continuously.
Payment Webhook API
Its main purpose is to have an API endpoint that can receive events from the stripe, validate it, and write it as stream to redis. For this purpose I have chosen FastAPI to build that single API endpoint. With this, we ensure that we process every single payment event, hand it over to event bus and then consume it using the bot microservice to process and in the end give proper access to the user.
Deployment
As I do not see need for the replication for the moment, I do not use docker swarm or kubernetes, but only docker compose. All the microservices are containerized and can be easily run. Maybe a CI/CD pipeline would come handy soon.
Thanks for reading it!
