Next Generation Out of Band Garbage Collection | Rails at Scale

In 2023, I wrote about how we’ve tuned Ruby’s garbage collector for Shopify’s monolith, including how we implemented out-of-band garbage collection to reduce the impact of major collection on latency. While the latency improvements were massive, we weren’t entirely satisfied with the heuristics used to trigger out-of-band garbage collection. It was purely based on averages, so we had to trade latency for capacity. More importantly, it didn’t fully eliminate major collection from request cycles, it only made it very rare. But in December 2023, while discussing with Koichi Sasada, we came up with a new idea.