#1413 unconfirmed
Andre Ben Hamou

DataObjects Scavenger Interval Threading Problem

Reported by Andre Ben Hamou | September 16th, 2010 @ 06:11 PM

We use multi-repository access throughout our DM code. One practical upshot of this is that batch scripts that stripe through all of the repositories end up holding open (due to DO connection pooling) huge numbers of surplus connections to the DB. In order to combat this, we've wrapped batch tasks in the following genre of code...

    scavenger_interval = DataObjects::Pooling.scavenger_interval
    DataObjects::Pooling.scavenger_interval = 2
    
    ...some code here...
    
    DataObjects::Pooling.scavenger_interval = scavenger_interval

We're seeing this blow up for certain kinds of operations. The error raised is actually a SystemExit...

    from /home/vjuser/.bundle/ruby/1.8/gems/data_objects-0.10.1/lib/data_objects/pooling.rb:60:in `scavenger'
    from /usr/lib/ruby/1.8/set.rb:194:in `each'
    from /usr/lib/ruby/1.8/set.rb:194:in `each_key'
    from /usr/lib/ruby/1.8/set.rb:194:in `each'
    from /home/vjuser/.bundle/ruby/1.8/gems/data_objects-0.10.1/lib/data_objects/pooling.rb:57:in `scavenger'
    from /home/vjuser/.bundle/ruby/1.8/gems/data_objects-0.10.1/lib/data_objects/pooling.rb:56:in `synchronize'
    from /home/vjuser/.bundle/ruby/1.8/gems/data_objects-0.10.1/lib/data_objects/pooling.rb:56:in `scavenger'
    from /home/vjuser/.bundle/ruby/1.8/gems/data_objects-0.10.1/lib/data_objects/pooling.rb:49:in `initialize'
    from /home/vjuser/.bundle/ruby/1.8/gems/data_objects-0.10.1/lib/data_objects/pooling.rb:49:in `new'
    from /home/vjuser/.bundle/ruby/1.8/gems/data_objects-0.10.1/lib/data_objects/pooling.rb:49:in `scavenger'
    from /home/vjuser/.bundle/ruby/1.8/gems/data_objects-0.10.1/lib/data_objects/pooling.rb:90:in `append_pool'
    from /home/vjuser/.bundle/ruby/1.8/gems/data_objects-0.10.1/lib/data_objects/pooling.rb:154:in `initialize'
    from /home/vjuser/.bundle/ruby/1.8/gems/data_objects-0.10.1/lib/data_objects/pooling.rb:119:in `new'
    from /home/vjuser/.bundle/ruby/1.8/gems/data_objects-0.10.1/lib/data_objects/pooling.rb:119:in `new'
    from /home/vjuser/.bundle/ruby/1.8/gems/data_objects-0.10.1/lib/data_objects/pooling.rb:119:in `synchronize'
    from /home/vjuser/.bundle/ruby/1.8/gems/data_objects-0.10.1/lib/data_objects/pooling.rb:119:in `new'
    from /home/vjuser/.bundle/ruby/1.8/gems/data_objects-0.10.1/lib/data_objects/connection.rb:63:in `new'
    from /home/vjuser/.bundle/ruby/1.8/gems/dm-core-0.10.2/lib/dm-core/adapters/data_objects_adapter.rb:233:in `open_connection'
    from /home/vjuser/.bundle/ruby/1.8/gems/dm-core-0.10.2/lib/dm-core/transaction.rb:412:in `open_connection'
    from /home/vjuser/.bundle/ruby/1.8/gems/dm-core-0.10.2/lib/dm-core/adapters/data_objects_adapter.rb:269:in `with_connection'
    from /home/vjuser/.bundle/ruby/1.8/gems/dm-core-0.10.2/lib/dm-core/adapters/data_objects_adapter.rb:136:in `read'
    from /home/vjuser/.bundle/ruby/1.8/gems/dm-core-0.10.2/lib/dm-core/repository.rb:145:in `read'
    from /home/vjuser/.bundle/ruby/1.8/gems/dm-core-0.10.2/lib/dm-core/collection.rb:1111:in `lazy_load'
    from /home/vjuser/.bundle/ruby/1.8/gems/extlib-0.9.15/lib/extlib/lazy_array.rb:411:in `each'
    from /home/vjuser/.bundle/ruby/1.8/gems/dm-core-0.10.2/lib/dm-core/collection.rb:506:in `each'

I note this is occurring on/in a Mutex.synchronize so I assume we're dealing with a hairy threading problem.

Comments and changes to this ticket

  • Dan Kubb (dkubb)

    Dan Kubb (dkubb) September 17th, 2010 @ 12:13 AM

    • State changed from “new” to “unconfirmed”
    • Assigned user set to “Dirkjan Bussink”
  • Andre Ben Hamou

    Andre Ben Hamou September 20th, 2010 @ 03:08 AM

    Just realised the stack trace was missing the all-important first line...

      /home/vjuser/.bundle/ruby/1.8/gems/data_objects-0.10.1/lib/data_objects/pooling.rb:60:in `synchronize': Thread#join: deadlock 0xb57c9fb0 - mutual join(0xb755a1b0) (ThreadError)
    

Please Sign in or create a free account to add a new ticket.

With your very own profile, you can contribute to projects, track your activity, watch tickets, receive and update tickets through your email and much more.

New-ticket Create new ticket

Create your profile

Help contribute to this project by taking a few moments to create your personal profile. Create your profile »

Pages