#1413 unconfirmed
Andre Ben Hamou

DataObjects Scavenger Interval Threading Problem

Reported by Andre Ben Hamou | September 16th, 2010 @ 06:11 PM

We use multi-repository access throughout our DM code. One practical upshot of this is that batch scripts that stripe through all of the repositories end up holding open (due to DO connection pooling) huge numbers of surplus connections to the DB. In order to combat this, we've wrapped batch tasks in the following genre of code...

    scavenger_interval = DataObjects::Pooling.scavenger_interval
    DataObjects::Pooling.scavenger_interval = 2
    
    ...some code here...
    
    DataObjects::Pooling.scavenger_interval = scavenger_interval

We're seeing this blow up for certain kinds of operations. The error raised is actually a SystemExit...

    from /home/vjuser/.bundle/ruby/1.8/gems/data_objects-0.10.1/lib/data_objects/pooling.rb:60:in `scavenger'
    from /usr/lib/ruby/1.8/set.rb:194:in `each'
    from /usr/lib/ruby/1.8/set.rb:194:in `each_key'
    from /usr/lib/ruby/1.8/set.rb:194:in `each'
    from /home/vjuser/.bundle/ruby/1.8/gems/data_objects-0.10.1/lib/data_objects/pooling.rb:57:in `scavenger'
    from /home/vjuser/.bundle/ruby/1.8/gems/data_objects-0.10.1/lib/data_objects/pooling.rb:56:in `synchronize'
    from /home/vjuser/.bundle/ruby/1.8/gems/data_objects-0.10.1/lib/data_objects/pooling.rb:56:in `scavenger'
    from /home/vjuser/.bundle/ruby/1.8/gems/data_objects-0.10.1/lib/data_objects/pooling.rb:49:in `initialize'
    from /home/vjuser/.bundle/ruby/1.8/gems/data_objects-0.10.1/lib/data_objects/pooling.rb:49:in `new'
    from /home/vjuser/.bundle/ruby/1.8/gems/data_objects-0.10.1/lib/data_objects/pooling.rb:49:in `scavenger'
    from /home/vjuser/.bundle/ruby/1.8/gems/data_objects-0.10.1/lib/data_objects/pooling.rb:90:in `append_pool'
    from /home/vjuser/.bundle/ruby/1.8/gems/data_objects-0.10.1/lib/data_objects/pooling.rb:154:in `initialize'
    from /home/vjuser/.bundle/ruby/1.8/gems/data_objects-0.10.1/lib/data_objects/pooling.rb:119:in `new'
    from /home/vjuser/.bundle/ruby/1.8/gems/data_objects-0.10.1/lib/data_objects/pooling.rb:119:in `new'
    from /home/vjuser/.bundle/ruby/1.8/gems/data_objects-0.10.1/lib/data_objects/pooling.rb:119:in `synchronize'
    from /home/vjuser/.bundle/ruby/1.8/gems/data_objects-0.10.1/lib/data_objects/pooling.rb:119:in `new'
    from /home/vjuser/.bundle/ruby/1.8/gems/data_objects-0.10.1/lib/data_objects/connection.rb:63:in `new'
    from /home/vjuser/.bundle/ruby/1.8/gems/dm-core-0.10.2/lib/dm-core/adapters/data_objects_adapter.rb:233:in `open_connection'
    from /home/vjuser/.bundle/ruby/1.8/gems/dm-core-0.10.2/lib/dm-core/transaction.rb:412:in `open_connection'
    from /home/vjuser/.bundle/ruby/1.8/gems/dm-core-0.10.2/lib/dm-core/adapters/data_objects_adapter.rb:269:in `with_connection'
    from /home/vjuser/.bundle/ruby/1.8/gems/dm-core-0.10.2/lib/dm-core/adapters/data_objects_adapter.rb:136:in `read'
    from /home/vjuser/.bundle/ruby/1.8/gems/dm-core-0.10.2/lib/dm-core/repository.rb:145:in `read'
    from /home/vjuser/.bundle/ruby/1.8/gems/dm-core-0.10.2/lib/dm-core/collection.rb:1111:in `lazy_load'
    from /home/vjuser/.bundle/ruby/1.8/gems/extlib-0.9.15/lib/extlib/lazy_array.rb:411:in `each'
    from /home/vjuser/.bundle/ruby/1.8/gems/dm-core-0.10.2/lib/dm-core/collection.rb:506:in `each'

I note this is occurring on/in a Mutex.synchronize so I assume we're dealing with a hairy threading problem.

Comments and changes to this ticket

Please Sign in or create a free account to add a new ticket.

With your very own profile, you can contribute to projects, track your activity, watch tickets, receive and update tickets through your email and much more.

New-ticket Create new ticket

Create your profile

Help contribute to this project by taking a few moments to create your personal profile. Create your profile »

Pages