Thursday, August 14, 2008

ActiveRecord: Can haz namespaces?

There are plenty of blog postings from has_many :through, err the blog and Pratik's blog describing how to completely avoid namespaced models. I unfortunately discovered these sites only after struggling with namespaces myself.

If namespaces were completely unsupported by ActiveRecord, I would leave it at that. Contrarily, namespaces are moderately supported, but lead first timers to the spooky edges of the Rails framework.

After giving it some thought, I determined that the best way to describe the state of namespacing in ActiveRecord is to act as a first time novice, pretending to use them for the first time!

The Requirements

I am an intern working for a company that has built their entire website with Ruby on Rails. My boss wants me to add statistics tracking similar to Google Analytics, and he wants it all to go into the Statistics:: namespace. The initial requirements include tracking the milliseconds that each HTTP request took to complete, how long users remained logged into their session, and which session each request belonged to.

This sounds like a really easy project, and I want to impress my boss by using all the best practices and conventions of Rails.

Table Name Surprise

I will start with a model that stores the length of each request. It makes sense to use a Rails script to generate Statistics::Request:
ruby script/generate model Statistics::Request milliseconds:integer
  create  app/models/statistics
  create  test/unit/statistics
  create  test/fixtures/statistics
  create  app/models/statistics/request.rb
  create  test/unit/statistics/request_test.rb
  create  test/fixtures/statistics_requests.yml
  create  db/migrate
  create  db/migrate/20080814034925_create_statistics_requests.rb
The generator conveniently created a statistics folder inside both app/models and test/unit. I am a little bit confused as to why the folder test/fixtures/statistics was created, when the actual fixture was placed in test/fixtures/statistics_requests.yml, but I will figure that out later.

The table name in the migration is statistics_requests, so I immediately assume that table names include the namespace. After migrating the database, I fire up the console to try out the model:
>> Statistics::Request.create(:milliseconds => 10)
ActiveRecord::StatementInvalid: Could not find table 'requests'
How strange, I thought the table's name is statistics_requests. Just to make sure:
>> Statistics::Request.table_name
=> "requests"
Is this a bug in Rails? Apparently not. I guess it's time to do some dirty monkey patching:

Good, now the model can find its table. Time to write some fixtures and tests.

Tests are looking good

Feeling confident after getting my namespaced model to work, it would be nice to make sure the the new test runs:
rake test:units
... "test/unit/statistics/request_test.rb"
1 tests, 1 assertions, 0 failures, 0 errors
Sweet - It looks like tests are discovered recursively in the test/unit folder.


I need to add the Statistics::Session model, and also update Statistics::Request to belong to a session.

The migration:

The model:

I use the console to test out the association:
>> session = Statistics::Session.create(:start => 4.hours.ago, :end =>
=> #<Statistics::Session id: 1, start: "2008-08-14 00:42:00", end: "2008-08-14 04:42:00">
>> request = Statistics::Request.create(:milliseconds => 10, :session => session)
ArgumentError: Statistics is not missing constant Session!
This is a bizarre error. It turns out that Rails can't find Statistics::Request from Statistics::Session. I finally figured out that I need to do this:

It would make more sense if I only had to specify :class_name if it existed in a different namespace.

Namespaced Fixtures = WTF?

Development and testing will be easier if I generate some fixture data. Remember when the model generator script created an unused folder?
create  test/fixtures/statistics
create  test/fixtures/statistics_requests.yml
Based on the other conventions, it seems like this file belongs in test/fixtures/statistics/requests.yml. I'll move it there, and load up some fake data:
rake db:fixtures:load
It will be fun to play around with all that fixture data:
>> Statistics::Request.count
=> 0
How bizarre, the data did not even get loaded. I soon find out that unlike unit tests, fixture files are not recursively loaded. The file can't be moved after all.

I need to get the request -> session association working in the fixtures too. Let me edit statistics_sessions.yml and statistics_requests.yml

Unfortunately, this does not work:
rake db:fixtures:load
rake aborted!
SQLite3::SQLException: table statistics_requests has no column named session...
For some reason, the label referencing is not working with my namespaced models. I guess I need to hand hold Rails on this one:

Observers - OMG they work!

Feeling a little down after my battle with fixtures, I decide to make a request observer that logs whenever we get a request:
ruby script/generate observer Statistics::Request
  exists  app/models/statistics
  exists  test/unit/statistics
  create  app/models/statistics/request_observer.rb
  create  test/unit/statistics/request_observer_test.rb
Looking good so far. Let me implement the class:

I also won't forget to add the following line to environment.rb:

And viola, it all works:
>> Statistics::Request.create(:milliseconds => 10)
We got a hit!!!

In Summary

Ok, I'm done playing stupid. My final word is that namespaced models can be made to work, but there are some rough edges. Some final tips for you:
  • Don't have conflicting namespaces and class names. For example, I should not introduce a class named Statistics in the above example. Otherwise, I will end up seeing the following message: "warning: toplevel constant Statistics referenced by Statistics::Statistics".
  • Even when referencing other classes in the same namespace, it is best to always include the namespace prefix. Otherwise, you will run into issues with conflicting class names across different namespaces and Rails' automagical class loading.

Some things that Rails should do:
  • All APIs in Rails that support defining classes by Symbols should also support Strings. I was fortunate that I could pass in 'statistics/request_observer' to active_record.observers. However, other methods such as ActiveRecord::Observer.observe do not support strings.
  • I believe that there is enough reason to support prefixing database table names with the namespace. This is as simple as introducing a new option to ActiveRecord called 'include_namespace_in_table_name'.
  • Fix the fixtures.


Hongli Lai said...

Good stuff.

I see these problems as bugs that should be fixed. There are valid reasons why one wants to namespace models.

nachokb said...


I once needed namespaces and had to sort all that out by myself. Nice of you to post about it.

About your issue with Symbols and Strings, even though inconsistencies should be fixed, you could always use :"ns/klazz" or "ns/klazz".to_sym...


Nicolás Sanguinetti said...

Exactly what nachokb says. Only one more thing.... fixtures?! WTF? Stop hurting yourself and use a cleaner alternative.

Object Daddy, for example, looks great.

Matthew Higgins said...

Thanks for the feedback.

I personally understand the workarounds for namespacing, but I do not believe that Rails should force us to use workarounds for this common requirement. My original goal of this blog post was to capture the difficulties that any newcomer will have with namespaces.

The patches to fix these 'bugs' are not difficult. The more difficult problem is convincing Rails core team that namespacing is a valid and common way to organize models.

lifo said...

There is no need to convince rails core team really. Namespaced model should work as expected and they behave a lot better on latest rails than they did before. Patches are most welcome.

Matthew Higgins said...

Pratik - I was generally referring to the DHH quote "I am generally not a huge fan of namespaces for models. As I don’t think that’s a good fit for splitting up your domain." I don't think anyone is entirely against the idea, but unfavorable opinions do exist.

Here's a patch -

Harri said...

A while ago I was also quite frustrated with the namespaced models. I'm somewhat new to the Rails, but I decided a make a little patch that improves the support. It changes the way the tables are named and fixes the fixtures. It also make generators write fully working code (e.g. when generating scaffolds with namespaced models).

Dave Redpath said...

I found that the table naming problem (at least) goes away when a model class is defined for the namespace eg models/admin.rb for stuff namespaced for an admin area.

JezC said...

+1 for Dave Redpath. Create an app/models/(namespace}.rb and insert the "class (namespace) < ActiveRecord::Base ... end' and the database tables for the namespace are then found automagically.

Christophe said...

To load fixtures add :
set_fixture_class :statistics_requests => Statistics::Request, :statistics_sessions => Statistics::Session

before your fixtures :all or fixtures :statistics_requests

Then you can call statistics_requests(:one) in your tests

Christophe said...

It prevents to use :
Thats wonderful no ?

Christophe said...

This fix allows STI and singular namespace ;-)

ActiveRecord::Base.class_eval do
def self.table_name
class_of_active_record_descendant(self).name.split("::").map { |package| package.underscore}.join("_").pluralize