opensoul.org - Home

awesome_nested_set: making nested sets cool

2008-11-17T06:32:00Z

Yes, I’m making the assertion that preordered tree traversal is now cool. And I don’t mean just “pocket protector” cool, because it’s always been that, but now it’s “show your friends” cool.

For those that have no idea what I’m talking about, and don’t really care, but still want to be cool, skip to the next section. For all three of you that want to understand all the gory details, check out this MySQL DevZone article on managing hierarchical data.

What are you talking about?

I’m talking about putting hierarchical data into a relational database, and a plugin to make that easier. There are lots of reasons for trying to do this: organizational structures, genealogies, taxonomies, nested pages of a website, etc. It’s kinda like putting a square peg into a round hole, except that the square peg is made out of Play-Doh, so we can force it through the hole anyway, and we just have a little extra mess to deal with.

There were several Active Record plugins out there that tried to clean up the mess, but they were either buggy or incomplete.

We created awesome_nested_set to try to remedy that.

What makes this so awesome?

There’s a lot of things that makes this awesome, but my personal favorite is that awesome_nested_set makes use of Rails 2.1’s named_scope features¹, so most of the nested set methods return a scope that works as a finder. You can call find methods on it or access other named scopes.

class Department < ActiveRecord::Base
  acts_as_nested_set
  named_scope :in_need_of_review, :lambda => {{
    :conditions => {:reviewed_at > 1.year.ago
  }}
end

chancellor = Department.create(:name => 'Chancellor')
aa = Department.create(:name => 'Academic Affairs').move_to_child_of(chancellor)
Department.create(:name => 'Admissions').move_to_child_of aa
Department.create(:name => 'Student Services',
  :reviewed_at => 3.months.ago).move_to_child_of aa

chancellor.descendants.in_need_of_review
rogue = chancellor.descendants.all(:conditions => 'manager_id IS NULL')

There’s lots more info in the README on GitHub, so check it out. Let us know if you have any suggestions or feedback.

It also backports named_scope for those still on Rails 2.0

Capistrano 2.5 and older versions of git

2008-11-13T16:43:00Z

Git support in Capistrano 2.5 got a little lovin’, but as a result, it may have broke your deploys. By default, Capistrano now uses the -q flag to tell git to not be so chatty. But older versions of git (1.4.x and maybe some early 1.5.x versions) don’t support the -q flag. If upgrading git isn’t an option, the solution is simple. Just tell Capistrano to take the muzzle off of git:

set :scm_verbose, true

Rails Training: January 20–23 in San Antonio, TX

2008-11-12T17:32:00Z

We’ve been providing highly-praised Ruby on Rails training all over the world, and now we’re bringing it to San Antonio, Texas. We will be offering hands-on Ruby on Rails training in downtown San Antonio on January 20-23, 2009.

I’ve never been to San Antonio, so I’m really exited for this class. Any recommendations for things to check out?

Making RSpec concise

2008-11-10T03:07:00Z

A common criticism of RSpec is that it is very verbose. I don’t necessarily agree (or care), but I thought it would be fun to see how concise I could make my specs.

Here are some simple specs from a client project:

describe Company do
  before do
    @company = Company.new
  end

  it "should have many classifications" do
    @company.should have_many(:classifications)
  end

  it "should have many industries through companies" do
    @company.should have_many(:industries, :through => :classifications)
  end

  it "should have many locations" do
    @company.should have_many(:locations)
  end

  it "should have many leads" do
    @company.should have_many(:leads)
  end

  it "should have many jobs" do
    @company.should have_many(:jobs)
  end

  it "should have many notes" do
    @company.should have_many(:notes)
  end

  it "should have many phones ordered by phone type position" do
    @company.should have_many(:phones, :as => :phonable,
      :include => :phone_type, :order => 'phone_types.position')
  end

  it "should have many events" do
    @company.should have_many(:events)
  end

  it "should have many interests" do
    @company.should have_many(:interests)
  end

  it "should belong_to an exchange" do
    @company.should belong_to(:exchange)
  end

  it 'should belong_to a primary_contact' do
    @company.should belong_to(:primary_contact, :class_name => 'Job')
  end

  it 'should have many articles' do
    @company.should have_many(:articles,
      :order => 'articles.date DESC, articles.created_at DESC')
  end
end

These specs check the declared associations on our company model using some custom matchers. They are not very complicated, but are somewhat repetitive. Each example has a description that is basically a duplication of the implementation.

Step 1: remove the description

For a while now, RSpec has had the ability for matchers to be self describing. If you don’t pass a block to #it, it uses the description provided by the matcher.

it do
  @company.should have_many(:jobs)
end

When that spec is run, it gives the output “should have a has_many association called :jobs”. Depending on what you’re speccing, the built in description isn’t always clear, but in this case it’s great.

See #simple_matcher if you want to create custom matchers with useful error messages.

Step 2: remove the subject

So the duplication within each example is gone, but if you look at the full spec above, each example calls @company.should. Accessing an instance variable isn’t what I would consider “duplication”, but thanks to a nifty new feature added to RSpec today, it’s now unnecessary noise. We can simply call #should within our example, and it will use a new instance of the described type as the “subject”.

describe Company do
  it do
    should have_many(:jobs)
  end
end

You can customize the subject if you don’t simply want a new instance.

describe Company, 'validations' do
  subject { Company.new(valid_company_attributes) }

  it do
    should be_valid
  end
end

Note: As David Chelimsky points out in the comments, this is not released yet and is subject to change.

Step 3: One-liner

Lastly, we can use the one line block:

describe Company do
  it { should have_many(:classifications) }
  it { should have_many(:events) }
  it { should have_many(:interests) }
  it { should have_many(:jobs) }
  it { should have_many(:leads) }
  it { should have_many(:locations) }
  it { should have_many(:notes) }
  it { should have_many(:articles, :order => 'articles.date DESC, articles.created_at DESC') }
  it { should have_many(:industries, :through => :classifications) }
  it { should have_many(:phones, :as => :phonable, :include => :phone_type, :order => 'phone_types.position') }
  it { should belong_to(:exchange) }
  it { should belong_to(:primary_contact, :class_name => 'Job') }
end

That’s pretty sexy. I wasn’t able to do this with all of the specs in my app, but it worked with quite a few of them.

It's a search party!

2008-11-08T20:03:00Z

While chatting with Dr. John Nunemaker at RubyConf, I realized that I have several problems. Ignoring the many character flaws that are beyond the scope of this post, my problems are:

I tag things I find useful on Delicious. But I rarely look back to delicious because it’s just easier to search Google, resorting to Delicious if I can’t find something that I remember tagging.
It’s easier to search Google because not everything that I find useful is in Delicious. I don’t want to have to think about where useful things are, I just want to search for them.

I want a search engine that prioritizes things that I’ve found useful in the past. Ideally, google would let me tag things and take that into account when calculating page rank. But, in the mean time…

Let’s have a search party!

I threw together a simple search interface that pulls in results from Google, Delicious, and just for fun, GitHub. It shows primarily Google results, but then in the sidebar, it shows results from Delicious, with my bookmarks highlighted at the top, and also results from GitHub. Check it out.

And if you use Firefox, you can add it as your search provider:

It is an extremely simple app that has 2 pages and uses JavaScript to read in JSON from all of the services. There is one Ruby class that does screen-scraping since Delicious doesn’t provide an API to their full search.

I first implemented it in Merb, but upon realizing how simplistic it is, I switched it over to Sinatra. And I used jQuery to do the JSON and other JavaScripty goodness. Thanks to Mark Van Holstyn for helping implement it.

The code is available on GitHub, so check it out, fork it, and make it more awesome.

I will be making a few other posts about specific things I learned when building this, including deploying sinatra apps and using jQuery to do JSON with Merb and Sinatra.

Ajax and Request Forgery Protection

2008-10-24T04:52:00Z

Rails 2.1 added protection for cross-site request forgery by embedding a session-based token in generated forms. Rails will not process a POSTed request without the token. For the most part, this protection is transparent. But occasionally, an Ajax request request gets left out in the cold without a token.

If your Ajax request is tied to a form on the page, then all is good because the form already has the authenticity token in it. It only happens when your Ajax request is not tied to a form but makes a POST request, which is a rare but occasionally useful.

So how do we let those Ajax requests in on the fun? We came up with the ingenious idea of just embedding the authenticity token in a meta tag on every page, which can then be used in the Javascript.

<meta name="authenticity-token" id="authenticity-token" content="<%= form_authenticity_token %>" />

The authenticity token is unique for each visitor, and already included in other parts of the page, so this doesn’t defeat the purpose of the request forgery protection.

We usually add a couple convenience methods for accessing the token in Javascript.

var Application = {
  authenticityToken: function() {
    return $('authenticity-token').content;
  },

  authenticityTokenParameter: function(){
   return 'authenticity_token=' + encodeURIComponent(Application.authenticityToken());
  }
}

Now, we have easy access to it whenever we need it.

new Ajax.Request(url, {
  parameters: Application.authenticityTokenParameter()
});

Push Upstream

2008-10-22T04:18:00Z

Scenario 1: You’re half way through a really productive day on a wicked new feature for an app and everything is going smoothly. Then, from out of nowhere…SMACK! A nasty bug in a library you depend on splatters right in your face. “Seriously, nobody has come across this before?” you mumble.

Scenario 2: You’re using some fancy library and you think to yourself, “Geez, wouldn’t it be sweet if it did <insert fancy feature here> for you?”

The temptation, especially with Ruby, is to solve both of these problems in your app’s code base by modifying your copy of the library. This may seem like the path of least resistance and the quickest way to move forward in the short term, but in the long term all you are doing is delaying the cost of properly solving that problem. You are incurring debt, which over time compounds and will end up costing you more to fix.

First, when you’re making the change just for yourself, you’re less likely to solve the problem properly or do the appropriate testing. You’ll do some hack job for the problem you have, without considering how it might affect other scenarios.

Second, if you found a bug or have a desired feature, chances are that someone else has or will run into the same situation. Fixing it locally doesn’t help everyone else.

Lastly, fixing the bug or adding a new feature directly in your app makes it nearly impossible to upgrade the library. I spent over 4 hours this week trying to upgrade an open source app to the latest version of Ruby on Rails, and still have over 100 failing tests. The biggest problem isn’t Rails itself, but all the plugins that also need upgraded, many of which had been patched in some form.

At Collective Idea, we use git submodules for as many external libraries as possible. While submodules have their own issues, and probably aren’t a recommended approach, the positive side-effect is that it strongly discourages us from patching libraries in place. If we do need to make a change, each library is already its own git repository, so we can push our changes and send a pull request to the original maintainer.

If you need to modify an open source library, don’t just do it in place. Push your changes upstream. Irresponsible patching hurts us all.

Money with precision

2008-10-18T20:26:00Z

I’ve been working on a project that needs to store mileage reimbursement rates to the nearest tenth of a cent. We were using the money gem, which stores money amounts in cents, so it looked like it was going to be a pain.

But without too much suffering, I modified the money gem to take a precision (in powers of 10), which defaults to 2. It can now store amounts in any precision.

>> amount = 20.to_money + 0.505.to_money
=> #<Money @precision=3, @currency="USD", @cents=20505>
>> amount.to_s
=> "20.505"
>> amount.format
=> "$20.51"

You can also store amounts in negative precisions, like millions:

>> amount = Money.new(50, 'USD', -6)
>> amount.to_s
=> "50"
>> amount.format
=> "$50000000.00"

Check out our fork of the money gem on github. There are lots of other goodies in there.

Behavior Driven Development with Cucumber

2008-10-14T04:14:00Z

The Great Lakes Ruby Bash was a good time. There were several quality presentations, including Jim Weirich’s talk “Playing it Safe – How to write library friendly code in Ruby”, Larry Karnowski’s talk “Usability on Rails”, and Brandon Dimcheff’s “Metawhat? A look into the mysterious metaclass”.

I presented a talk titled “Behavior Driven Development with Cucumber”. Despite the fact that half of the audience didn’t know what the hell I was talking about, I think it went well. I uploaded the slides from my talk to slideshare if you’re interested. I’m not sure that they’ll really be very helpful, but I may try to record audio to go with them at some point.

As far as I know, no audio or video was captured at the conference.

acts_as_audited in development

2008-10-13T02:10:00Z

Thanks to metatribe, a huge annoyance in acts_as_audited is now fixed: it works in development mode. Due to it’s dependance on Rails’ cache sweepers, which are only enabled when caching is, acts_as_audited didn’t work in development mode.

metatribe has a crafty solution, which took me a few minutes to believe it would even work. So pull the latest version and happy auditing.

The importance of var in JavaScript

2008-09-04T00:37:00Z

I would consider myself proficient at JavaScript. Not a rockstar, but I can hold my own. But I didn’t learn it out of a book; I picked it up slowly over several years. So occasionally I come across something that I probably would have learned in the first couple chapters of a decent JavaScript book.

Today, I experienced the importance of declaring variables with var.

For the unenlightened like myself, JavaScript basically has two scopes for variables: global and local. Variables assigned outside of a function are global, and variables assigned inside of a function, using the var keyword, are local (not rocket surgery). However, if you leave the var keyword off, it assigns a global variable, regardless of where it’s declared.

Here’s a bit of code that demonstrates this:

function fail() {
  date = new Date();
  console.log('Before: ' + Number(date));
  setTimeout(function() {
    console.log('After: ' + Number(date));
  }, 1000);
}

fail();
fail();

The code defines a function that sets a variable, logs the variable, then defines a callback that logs the variable again on second later. If you call the function twice in a row, you can see the effect:

Before: 1220487263486
Before: 1220487263499
After: 1220487263499 <- This should be 1220487263486
After: 1220487263499

Declaring the date variable with var gives us this output:

Before: 1220487287985
Before: 1220487287994
After: 1220487287985 <- see, this is the same as the first line
After: 1220487287994

So remember kids, declare local variables with var!

Autotest mapping for Rails test conventions

2008-08-22T11:00:00Z

A while ago I posted a configuration for getting autotest to work with Test::Unit outside of Rails. Ryan Davis, author of autotest, commented on that post saying that it should “Just Work™” without any custom configuration. I was perplexed because I’ve never been able to get it to work on my gems and Rails plugins.

I finally took time to look into the issue, and realized it’s because I always use the Rails naming conventions for my test files. I name them foo_test.rb, instead of test_foo.rb, which is what Autotest looks for.

That’s easily solvable. Here’s an Autotest configuration, tested with ZenTest 3.10.0, that should make it work for either naming convention. You can throw this in your ~/.autotest file, or in a .autotest file inside your project.

Autotest.add_hook :initialize do |at|
  at.clear_mappings

  at.add_mapping %r%/^lib/(.*)\.rb$% do |_, m|
    possible = File.basename(m[1])
    files_matching %r%^test/.*(#{possible}_test|test_#{possible})\.rb$%
  end

  at.add_mapping(%r%^test/.*\.rb$%) {|filename, _| filename }
end

Happy autotesting.

Merging Active Record models

2008-08-21T09:00:00Z

We’ve been working on a project that involves importing a massive amount of data from multiple sources. The data is somewhat complicated, so we occasionally end up with duplicate records that need merged together. The data is highly normalized, so there are a bunch of associations that also need merged. If you’ve ever done this by hand, you know how painful it can be.

To alleviate that pain, I introduce you to merger, a Rails plugin for merging Active Record models.

@person.merge!(Person.find_all_by_email(@person.email))

The plugin is pretty simple right now. All it does is:

Given a set of records, picks the oldest record (the one with the lowest id) as the one to keep
Moves any associated has_many and habtm records from the duplicates to the record that is being kept
Deletes the duplicate records

We intend to add a lot to it, including:

Strategies for choosing which record to keep
Strategies for merging the individual attributes of the records
Recursively merge associations based on certain attributes
Options for what to do with the duplicate records

Check it out on Github and let us know what you think.

Photo adapted from http://flickr.com/photos/xrrr/2478140383/

Life without fixtures

2008-08-21T02:12:00Z

All the cool kids tell me that fixtures just aren’t the “in” thing anymore. Speaking of which, when did fanny-packs stop being cool? I wish someone would have said something…

Anyway, while I still haven’t fallen out of love with (foxy) fixtures, I have taken some of the home-grown alternative methods for a spin, but so far, I’ve been slightly frustrated with my experience.

The predominant replacement seems to be the “factory” pattern, talked about by Dan Manges and then followed up by the FixtureReplacement plugin. The basic idea is that all your tests/specs should setup the data that they require, and there is a factory that makes it easy to create the necessary valid test data. For example, a homegrown solution might look like this:

class Generate
  def self.person(options={})
    p = Person.create!({
      :name     => 'Brandon',
      :login    => 'brandon',
      :password => 'testing'
    }.merge(options))
    p.account ||= Generate.account
  end
end

describe Person, '#authenticate' do
  it 'should return the person record if successful' do
    person = Generate.person
    Person.authenticate(person.login, 'testing').should == person
  end
end

This method scales pretty well. There are some issues, such as the use of create!. Sometimes you intentionally want an invalid record, or you want a new record with just some valid attributes initialized. It’s also slower than fixtures, but I don’t care too much about that right now.

But there’s a bigger issue. Like a good little BDDer, I have also been stubbing and mocking all my “unit tests” so there is no interaction with the database or code outside of what is being tested. To make the factory method work, I now need to define 3 different methods depending on what I’m testing: one using #create!, one using #new, and another one using stubbing.

What do we do about it?

I think the FixtureReplacement plugin is on the right track. It handles the new vs. create problem very nicely.

module FixtureReplacement
  attributes_for :person do |u|
    u.name     = String.random
    u.login    = String.random
    u.password = String.random
    u.account  = default_account
  end
end

@person = new_person(:login => 'brandon')
@person = create_person

Before I heard about the FixtureReplacement plugin, I actually concocted my own little solution that handles all three scenarios. I’m not crazy about the syntax, but it works for my needs.

Generate.add Person, :name => 'Brandon', :login => 'brandon' do |generator, u|
  u.account ||= generator.account
end

@person = Generate.person(:login => 'brandon')
@person = Generate.new_person
@person = Generate.stub_person

Anywho, for the fixture-less approach to work, I think stubs need to be supported.

Is all this really worth it?

That’s the question I find myself asking. What’s wrong with fixtures anyway?

One thing I really like about fixtures, when done right, is that they help tell the story of your application. You get to know the fixture data as you work on the app.

The factory method also seems to go against convention-over-configuration. Instead of having a default “configuration” when your tests run, you have to configure each test. Call me lazy, but that just seems like too much work.

I find that testing takes a lot more effort now than it did with fixtures. And as a result, I’m more hesitant to test everything. So while the factory method and stubbing is theoretically supposed to help you test better, I feel like they’ve had the opposite effect.

What do you think? Have you had success using the factory pattern for tests?

Win a free week of Rails training

2008-08-18T13:00:00Z

Want to learn Rails but can’t scrounge up the cash for training? You can win a free spot at either our Rails session starting next Monday, August 25, or to an advanced Rails session whenever it’s announced.

All you have to do is Complete this brief survey about our upcoming Advanced Rails session. We’ll be drawing a winner at 5:00 TOMORROW, August 19.

Also, we still have some spots available for our Rails session next week, so head on over to the Sessions site to check it out.