I made a simple Capistrano task to run rake gems:install on the server, for Rails Gem dependencies:

namespace :gems do
  desc "Install gems"
  task :install, :roles => :app do
    run "cd #{current_path} && #{sudo} rake RAILS_ENV=production gems:install"
  end
end

The one non-obvious thing here is how sudo and cd interacts. Explained further in this thread.

Also, I ran into a catch 22 with an app that has will_paginate as a Gem dependency.

The rake gems:install task will load the app environment, but as I was using the WillPaginate constant in my app (in a helper), the task failed with

uninitialized constant WillPaginate

So the Rake task could not run to install the gem, because the WillPaginate constant was not available, because the gem was not installed…

The fix was simply to check for the constant in a conditional:

class CustomRenderer < WillPaginate::LinkRenderer
  # ⋮
end if defined?(WillPaginate)  # avoid catch 22 with "rake gems:install"

Another solution would, of course, be to install the gem outside of the Rake task, but that kind of does away with the point of having the task in the first place.