External Garbage Collection — Census Engineering

A common pattern in SaaS apps is the need to create, and then later clean up, external resources. What do I mean by “external”? Basically, anything that isn’t a row in your transactional database. In the normal course of running Census, we create and then later destroy lots of things:Google Cloud Service Accounts for fine-grained access to customers’ BigQuery dataAWS S3 objects storing temporary data during a Reverse ETL runSnapshot tables in customers’ data warehouses used to detect data changesAuth0 connections so users can use their IdPs to sign in to Censusetc….