Datamapper DataMapper

Comments and changes to this ticket

• 

  Alex Coles November 16th, 2009 @ 09:31 AM

  • Assigned user set to “Dan Kubb (dkubb)”

  Assigning this to Dan.

  @dkubb, how do we go about configuring which adapters we test against in a CI run?

• 

  Alex Coles November 16th, 2009 @ 10:24 AM

  Ideally we'd be able to run a few more combinations (next, master) and next and current release versions of JRuby. I don't know what's possible with current resources?

  http://gist.github.com/223478 : this gist is a brainstorm of all of the combinations that we'd like to run specs against (in an ideal scenario, without resource constraints).

• You flagged this item as spam.
  

  ronin-6253 (at lighthouseapp) November 16th, 2009 @ 11:21 AM

  With the entire test suite enabled JRuby was consuming an unsustainable amount of memory.

• 

  Alex Coles November 16th, 2009 @ 11:44 AM

  OK, that's interesting to know. I've had to tweak some JVM memory settings locally, so we'll look into that.

• 

  Dan Kubb (dkubb) November 16th, 2009 @ 11:48 AM

  • State changed from “unconfirmed” to “confirmed”

  I still haven't figured out if it's a memory leak in DM/DO, or if it's simply RSpec using a ton of memory to store pending spec state. We have alot of pending specs for things like the in-memory adapter and m:m specs.

• 

  Alex Coles November 30th, 2009 @ 11:50 AM

  rsim looked into this and the problem may lie with RSpec. In NetBean's (JVM Memory) Profiler, I was seeing similar behaviour running with the in_memory adapter as for DO adapters. Here's the transcript from today's IRC:

  rsim: myabc: dbussink: generated 1 million RSpec "it" blocks which takes more than 1GB of memory  as I see it grabs memory during definition of tests and then is costant during execution. I suspect that because dm-core have so many shared examples then each time shared examples are reused they are copied in memory and therefore memory consumption grows all the time
  [5:07pm] dbussink: rsim: so they are never cleaned up?
  [5:08pm] rsim: dbussink: it seems so that RSpec test definitions are not released from memory after execution
  [5:08pm] myabc: rsim: damn..
  [5:09pm] pietia_: nice 
  [5:16pm] rsim: dbussink: myabc: pietia_: each RSpec describe block creates new dynamic class definition and each "it" test creates new Example object which is stored in array in class variable - as class definitions are not released from memory then all test definition stays in memory
  [5:17pm] dbussink: rsim: do those objects get called by rspec later on?
  [5:17pm] dbussink: rsim: i can imagine failures and pending specs being remembered for display later, but passing ones shouldn't need to be remembered
  [5:17pm] pietia_: rsim, and those objects are in that _big_ hashmap  ?
  [5:17pm] pietia_: i bet
  [5:18pm] rsim: dbussink: at first _spec.rb file is executed and decribe / it methods create test structure in memory. After that RSpec calls run method on created tests
  [5:23pm] rsim: dbussink: probably one option could be to split RSpec test run into several processes - e.g. public / semipublic or even more detailed. Then each process will consume smaller amount of memory
  [5:26pm] rsim: each shared example group is created as dynamic module and then included in other describe classes - so it seems that it is not copying it there
  [5:28pm] somebee left the chat room.
  [5:31pm] somebee joined the chat room.
  [5:33pm] dbussink: rsim: separate processing would solve it yeah, but we have to make sure return status etc. is properly checked
  

• 

  Dan Kubb (dkubb) February 1st, 2010 @ 04:25 PM

  • State changed from “confirmed” to “resolved”

  Removing this ticket because CI server is now disabled, and memory leak causing this has since been resolved.