Data pipelines in Ruby on Rails - Corey Martin

If you need to bring a lot of data into your app, a data pipeline can help. This article describes how to build a simple data pipeline with Ruby on Rails, Sidekiq, Redis, and Postgres.What’s a data pipeline?A data pipeline takes information from one place and puts it somewhere else. It likely transforms that data along the way. In enterprise circles, this is called ETL, or extract-transform-load.Generally, data pipelines:Ingest a lot of data at onceIngest new or updated data on a schedule.