Musings of a Trained Monkey

There is an interesting thread going on the rails-core mailing list about testing coverage and documentation. I learned about rcov which provides testing coverage metrics for Ruby applications. Here are links to the results so farTook this content down due to it being stale:

• ActionMailer
  rcov -x activerecord,activesupport,actionpack test/*.rb
• ActionPack(top level only)
  rcov -x activerecord,activesupport,actionmailer test/*.rb
• ActionWebService
  rcov -x activerecord,activesupport,actionpack,actionmailer test/*.rb

I modified rcov a bit trying to get it to work covering the Rails codebase. The results were somewhat mixed, but only railties blew up strictly in rcov with a SystemStackError. Showing the complete test coverage of ActiveRecord in particular will present a challenge due to the number of database connectors it has.

As an example of differences between running rake test versus rcov test*.rb, in Activerecord there is a test case named aaa_create_tables_test that runs fine with rake, but with rcov there is a load error because aaa_create_tables_test requires abstract_unit.rb which simple test/*.rb has loaded yet because aaa_create_tables_test comes before abstract_unit. I have gotten farther with loading abstract_unit.rb explicitly rcov -x actionpack,activesupport test/abstract_unit.rb test/*.rb but it would be better if I could figure out how to wrap rake inside of rcov...

People that have worked with me in the past know how much I think unit tests are worth. In fact I won't say any more about them. White box testing has a place, it just isn't on my blog.

Selenium is an open source testing framework that will be making products like Mercury Interactive's WinRunner quake in their boots. Being able to do acceptance/functional/regression/Black Box testing of web applications across browser and OS platform boundaries from a single script file is already possible with Selenium. The major testing/QA vendors can't even boast that level of functionality yet. Even better AJAX applications are supported by Selenium. There is a Selenium IDE that works as an extension to Firefox. You can simply record your actions "macro-like" as you navigate through the web application you want to test.

What makes this all the more exciting is the recent selenium_on_rails plugin. It allows you to embed and run your Selenium tests as rake tasks. Automated quality assurance has never been this compelling to include as part of the development process.

So far the only short coming I've observed is the lack of i18n support, however the new .rsel format probably could be used to overcome this. Rsel lets you define your Selenium tests in terms of Ruby code. To see all of this in action take a look at the excellent screencast over at and then nothing.net.

UPDATE: Someone made a good point that I maybe didn't make clear here, Selenium can be used to test ANY web application, not just Rails powered, but even legacy Java, .NET, PHP, or any other caveman languages from the last century. :-)