Skip to main content

httparty_typhoeus_adapter

🚀 Integrating Typhoeus for Parallel Requests​

HTTParty doesn’t natively support parallel requests, but you can plug in the Typhoeus HTTP engine to run concurrent calls. Override perform_request to queue calls in a shared Hydra:

require 'httparty'
require 'typhoeus'

class ParallelClient
include HTTParty
base_uri 'https://api.example.com'

# Custom adapter to use Typhoeus::Hydra
module TyphoeusAdapter
def perform_request(http_method, path, options, &block)
hydra = Typhoeus::Hydra.hydra
req = Typhoeus::Request.new(
self.class.base_uri + path,
method: http_method,
headers: options[:headers],
body: options[:body]
)

response = nil
req.on_complete { |resp| response = resp }
hydra.queue(req)
hydra.run

build_response(response)
end
end

# Inject the adapter
class << self
prepend TyphoeusAdapter
end

def fetch(id)
self.class.get("/resources/#{id}")
end
end

# Fire off 50 parallel requests
requests = 50.times.map do |i|
Thread.new { ParallelClient.new.fetch(i).body }
end
requests.each(&:join)

This approach uses a shared Hydra instance to batch and run all requests concurrently, drastically reducing total execution time when hitting many endpoints at once.