#1090 ✓resolved
Alex Coles

CI only running dm-core specs with InMemory adapter on JRuby

Reported by Alex Coles | October 11th, 2009 @ 10:09 AM

Appears that in CI, dm-core specs using JRuby implementation are only running with the in-memory adapter.

This ticket is to track investigation of what's incorrectly configured/failing.

Comments and changes to this ticket

  • Alex Coles

    Alex Coles November 16th, 2009 @ 09:31 AM

    • Assigned user set to “Dan Kubb (dkubb)”

    Assigning this to Dan.

    @dkubb, how do we go about configuring which adapters we test against in a CI run?

  • Alex Coles

    Alex Coles November 16th, 2009 @ 10:24 AM

    Ideally we'd be able to run a few more combinations (next, master) and next and current release versions of JRuby. I don't know what's possible with current resources?

    http://gist.github.com/223478 : this gist is a brainstorm of all of the combinations that we'd like to run specs against (in an ideal scenario, without resource constraints).

  • ronin-6253 (at lighthouseapp)

    ronin-6253 (at lighthouseapp) November 16th, 2009 @ 11:21 AM

    With the entire test suite enabled JRuby was consuming an unsustainable amount of memory.

  • Alex Coles

    Alex Coles November 16th, 2009 @ 11:44 AM

    OK, that's interesting to know. I've had to tweak some JVM memory settings locally, so we'll look into that.

  • Dan Kubb (dkubb)

    Dan Kubb (dkubb) November 16th, 2009 @ 11:48 AM

    • State changed from “unconfirmed” to “confirmed”

    I still haven't figured out if it's a memory leak in DM/DO, or if it's simply RSpec using a ton of memory to store pending spec state. We have alot of pending specs for things like the in-memory adapter and m:m specs.

  • Alex Coles

    Alex Coles November 30th, 2009 @ 11:50 AM

    rsim looked into this and the problem may lie with RSpec. In NetBean's (JVM Memory) Profiler, I was seeing similar behaviour running with the in_memory adapter as for DO adapters. Here's the transcript from today's IRC:

    rsim: myabc: dbussink: generated 1 million RSpec "it" blocks which takes more than 1GB of memory  as I see it grabs memory during definition of tests and then is costant during execution. I suspect that because dm-core have so many shared examples then each time shared examples are reused they are copied in memory and therefore memory consumption grows all the time
    [5:07pm] dbussink: rsim: so they are never cleaned up?
    [5:08pm] rsim: dbussink: it seems so that RSpec test definitions are not released from memory after execution
    [5:08pm] myabc: rsim: damn..
    [5:09pm] pietia_: nice 
    [5:16pm] rsim: dbussink: myabc: pietia_: each RSpec describe block creates new dynamic class definition and each "it" test creates new Example object which is stored in array in class variable - as class definitions are not released from memory then all test definition stays in memory
    [5:17pm] dbussink: rsim: do those objects get called by rspec later on?
    [5:17pm] dbussink: rsim: i can imagine failures and pending specs being remembered for display later, but passing ones shouldn't need to be remembered
    [5:17pm] pietia_: rsim, and those objects are in that _big_ hashmap  ?
    [5:17pm] pietia_: i bet
    [5:18pm] rsim: dbussink: at first _spec.rb file is executed and decribe / it methods create test structure in memory. After that RSpec calls run method on created tests
    [5:23pm] rsim: dbussink: probably one option could be to split RSpec test run into several processes - e.g. public / semipublic or even more detailed. Then each process will consume smaller amount of memory
    [5:26pm] rsim: each shared example group is created as dynamic module and then included in other describe classes - so it seems that it is not copying it there
    [5:28pm] somebee left the chat room.
    [5:31pm] somebee joined the chat room.
    [5:33pm] dbussink: rsim: separate processing would solve it yeah, but we have to make sure return status etc. is properly checked
  • Dan Kubb (dkubb)

    Dan Kubb (dkubb) February 1st, 2010 @ 04:25 PM

    • State changed from “confirmed” to “resolved”

    Removing this ticket because CI server is now disabled, and memory leak causing this has since been resolved.

Please Sign in or create a free account to add a new ticket.

With your very own profile, you can contribute to projects, track your activity, watch tickets, receive and update tickets through your email and much more.

New-ticket Create new ticket

Create your profile

Help contribute to this project by taking a few moments to create your personal profile. Create your profile »

People watching this ticket