RubyGems: Single/Multi-tons and memorysize

Steve Longdo — Wed, 09 Aug 2006 03:41:00 +0000

There is a line in the RubyGems source code, @gempath_searcher ||= Gem::GemPathSearcher.new, from custom_require.rb that seeks to make Gem::GemPathSearcher a Singleton class. For some reason this isn't sufficient as I have seen as many as five instances of it existing in the same thread during profiling of Typo. I am not sure why more than one gets generated. It may relate to Rails aliasing the Kernel.require method after RubyGems has in custom_require.rb.

Review of the source makes me understand why a Gem::GemPathSearcher instance can be so large, it holds the gem specifications for every gem and version on the load path. Running locally this is not a big deal, as you probably don't have a lot of gems and their versions installed. On a shared host though there are tons of gems and versions of gems installed (350+ on my TextDrive server!).

A permanent solution to fix Gem::GemPathSearcher to truly be a Singleton class would probably be something as simple as checking Object.const_defined?(:GemPathSearcher) to see if the class is already in memory.

A quick alternative would be restricting the gem library path to a gem repository other than the shared host's repository. This can be done by exporting the GEM_HOME variable in your shell. There is also a GEM_PATH variable that can be used to support multiple repositories. Examples of how to work with these variables can be found on the Rails wiki. It should be possible to make a repository containing just the gems your app needs and nothing else. This will reduce the memory footprint of Gem::GemPathSearcher quite a bit, no matter how many instances of them there might be!

Obviously I am still learning the implications of RubyGems working together with Rails apps. I apologize for you the reader being dragged along on my quest to learn. Anyone wishing to bring this quest to a quick end just tell me how it ends.

Update: I will continue my research until we arrive at the truth. I still have some other RubyGems optimizing techniques to apply. My hope is using something like the gemconfigure file described in the documentation will be the difference between the Whole Milk Memory and Skim Milk Memory. Also I will reformat the code in the post to be pretty tomorrow. need to add some stuff to scribbish to accomplish this and I want to go to bed now...

"RubyGems: Single/Multi-tons and memorysize" by Matthew Margolis

Thu, 10 Aug 2006 14:33:32 +0000

Great to hear that someone is looking into this. I look forward to your findings.

"RubyGems: Single/Multi-tons and memorysize" by Piers Cawley

Wed, 09 Aug 2006 05:13:58 +0000

Yikes! That explains a whole lot.

Musings of a Trained Monkey: RubyGems: Single/Multi-tons and memorysize

RubyGems: Single/Multi-tons and memorysize

"RubyGems: Single/Multi-tons and memorysize" by Matthew Margolis

"RubyGems: Single/Multi-tons and memorysize" by Piers Cawley