sidekiq_batch_chaining

🚀 Master Complex Batch Chaining with Callback Pipelines

Utilize Sidekiq::Batch to orchestrate multi-stage processing, with on-success and on-complete callbacks for each phase. You can nest batches or dynamically spawn sub-batches to handle dependent workflows.

class MainJob
  include Sidekiq::Worker
  def perform(data)
    batch = Sidekiq::Batch.new
    batch.description = "Process data pipeline for #{data['id']}"
    batch.on(:success, PipelineCallback, {'parent_id' => data['id']})

    batch.jobs do
      StageOneWorker.perform_async(data)
    end
  end
end

class PipelineCallback
  include Sidekiq::Worker
  def perform(status, options)
    # status contains :total, :failures, :bid
    StageTwoWorker.perform_async(options['parent_id'])
  end
end