soveran/ohm — pull request #93 by Erol (2013-07-27 06:59:15+00:00)
Title: Allow passing of attributes to #select & #reject for better performance with large collections.

`Ohm::Collection#select` & `Ohm::Collection#reject` creates whole objects when enumerating, which can be slow on large collections or models with a lot of attributes. I was thinking of allowing those two methods to accept optional attributes to enumerate by. When attributes are given, the method will use `SORT` with a `GET` parameter to fetch only those attributes, minimizing network traffic, object creation and GC runs.

Here's the benchmark script I used. I used a collection size of 20,000 records.

require "ohm"
require "benchmark"

class User < Ohm::Model
attribute :fname
attribute :lname
attribute :bday
attribute :gender
attribute :city
attribute :state
attribute :country
attribute :zip
end

create = lambda do |i|
User.new(:fname => "John#{i}",
:lname => "Doe#{i}",
:bday => Time.now.to_s,
:gender => "Male",
:city => "Los Angeles",
:state => "CA",
:country => "US",
:zip => "90210").save
end

20000.times(&create)

Benchmark.bm(20) do |bm|
bm.report('#select by given attributes') { User.all.select(:fname) { |fname| fname =~ /John\d$/ } }
bm.report('#select by object') { User.all.select { |user| user.fname =~ /John\d$/ } }
end

And the results:

user system total real
select by given attributes 0.370000 0.030000 0.400000 ( 0.489133)
select by object 4.140000 0.400000 4.540000 ( 4.723038)

My only hesitation with this approach is that it might be a case of a golden hammer. The use-case above (regexp searching) could have been done better by querying a synced SQL database or a search tool like Solr or ES.

What do you guys think?