Implementing a Mutex for ActiveJob

At Chatwoot, we rely heavily on background jobs to process incoming messages and events from various platforms like Facebook. A while back we noticed a strange issue, two messages intended for the same conversation thread would lead into two different threads al-together. The problem was that sometimes webhooks from Facebook would arrive too close together. These webhook events were picked by the backend and queued for processing with Sidekiq. Our background workers would pick up these jobs simultaneously, each of these threads would check for an existing conversation. Since neither found one (because neither had created it yet), both workers ended up creating separate conversations for the same user message. This race condition resulted in duplicate conversation threads for a single interaction.