OSCON 2016 - Ruby can too scale: Highly performant microservices in Ruby

Requesting queuing time is the time after a request enters your production infrastructure and before it reaches your application. It may include an actual queue that requests enter, or it may represent other functions (such as load balancing or internal network latency). If you are monitoring request queuing along with your application metrics, you may find that under heavy loads, your request queuing time shoots up, however your service/application performance stays uniform.

The images below show just that. The 1st image shows overall latency including request queuing time while the 2nd image shows latency with the queuing time taken out.

For more information on how to measure and optimize request queuing time checkout the following links:

Unit tests can be a useful tool for understanding the performance of your system. During unit testing you can run a profiling tool that shows how long tests, functions, and execution paths take to execute. Armed with this information, you can look at a list of the slowest tests / execution paths and try to understand why they are slow. In ruby a useful tool is minitest-perf.

Use smaller, more focused frameworks. For example, if you don't need everything that Rails provides use Sinatra instead, same goes with using Sequel instead of ActiveRecord. In an experiment Tim conducted, he was able to optimize his Rails app to produce a 9.4% increase in client requests per minute by getting rid of jbuilder templates and using the oj gem. By replacing ActiveRecord with Sequel he was able to stretch that performance gain to 13.5%. But by replacing Rails with Sinatra his performance jumped by 372%, from 18,011 requests per minute to 85,017 requests per minute!

You can also take a look at running your ruby code on interpreters that are faster like JRuby and Rubinious

• Set reasonable timeouts on your client calls. The Ruby HTTP default timeout of 60 seconds is NOT a reasonable timeout. In tandem, use the circuit breaker pattern to avoid flooding struggling servers. Gems that can help here are the Timeout module and cb2 gem.
• Parallelize operations over collections whenever possible. This is very easy to do with HTTP requests using Typhoeus Hydra. There's also the Parallel library that allows you to operate over collections using multiple processes or threads.
• Execute non-core tasks as background jobs using Sidekiq or Resque
• Rate limit by client. If you are using Rack there's a middleware called rack/throttle that can enforce a minimum time interval between subsequent HTTP requests from a particular client and can also define a maximum number of allowed HTTP requests per a given time period (per minute, hourly, or daily).