Skip to main content

Posts

Featured

Ruby Lazy chunked hash like behavior

When we want to iterate a long list, we can simply write a query and get a cursor, ActiveRecord will do all the heavy lifting for us.

What happens when we need to do some complicated computations on a set of data, which sometimes can be too big to be stored in memory for the entire computing process?

This is when we need to start being more creational.

I'd like to introduce what I came up with.

The problem:

- Complex calculation on time based data series for a period of 3 months.
- Each calculation may depends on previous one and on future and past data.
- Must be in order.
- When fetching all data server crash on memory.

The solution:

I wanted to do the most minor code change possible, and currently the data was accessed via a hash. 
I decided to encapsulate the hash with something I called lazy chunked hash (tried google it see it as standard behavior in clojure).

It looks like this:

class ValuesProviderdef initialize() @loaded_date = nil @hash= Hash.new(0) end def [](time_slot) …

Latest Posts

Rubymine on OSX memory issues

Use FactoryGirl And Faker for easy data generation in unit testing (Part1)

Get the user locale from http headers

Dedicating a DJ(Delayed job) worker to a specific queue in heroku.

Overriding default id in mongoid

Nice post about cyber war

ClusteTV is hiring

Karma Rails Angular Jasmine testing - set up in 5 minutes

Export Hebrew (or any non english language) with php to excel

Home Base SDK