
DataObjects Scavenger Interval Threading Problem
Reported by Andre Ben Hamou | September 16th, 2010 @ 06:11 PM
We use multi-repository access throughout our DM code. One practical upshot of this is that batch scripts that stripe through all of the repositories end up holding open (due to DO connection pooling) huge numbers of surplus connections to the DB. In order to combat this, we've wrapped batch tasks in the following genre of code...
scavenger_interval = DataObjects::Pooling.scavenger_interval
DataObjects::Pooling.scavenger_interval = 2
...some code here...
DataObjects::Pooling.scavenger_interval = scavenger_interval
We're seeing this blow up for certain kinds of operations. The error raised is actually a SystemExit...
from /home/vjuser/.bundle/ruby/1.8/gems/data_objects-0.10.1/lib/data_objects/pooling.rb:60:in `scavenger'
from /usr/lib/ruby/1.8/set.rb:194:in `each'
from /usr/lib/ruby/1.8/set.rb:194:in `each_key'
from /usr/lib/ruby/1.8/set.rb:194:in `each'
from /home/vjuser/.bundle/ruby/1.8/gems/data_objects-0.10.1/lib/data_objects/pooling.rb:57:in `scavenger'
from /home/vjuser/.bundle/ruby/1.8/gems/data_objects-0.10.1/lib/data_objects/pooling.rb:56:in `synchronize'
from /home/vjuser/.bundle/ruby/1.8/gems/data_objects-0.10.1/lib/data_objects/pooling.rb:56:in `scavenger'
from /home/vjuser/.bundle/ruby/1.8/gems/data_objects-0.10.1/lib/data_objects/pooling.rb:49:in `initialize'
from /home/vjuser/.bundle/ruby/1.8/gems/data_objects-0.10.1/lib/data_objects/pooling.rb:49:in `new'
from /home/vjuser/.bundle/ruby/1.8/gems/data_objects-0.10.1/lib/data_objects/pooling.rb:49:in `scavenger'
from /home/vjuser/.bundle/ruby/1.8/gems/data_objects-0.10.1/lib/data_objects/pooling.rb:90:in `append_pool'
from /home/vjuser/.bundle/ruby/1.8/gems/data_objects-0.10.1/lib/data_objects/pooling.rb:154:in `initialize'
from /home/vjuser/.bundle/ruby/1.8/gems/data_objects-0.10.1/lib/data_objects/pooling.rb:119:in `new'
from /home/vjuser/.bundle/ruby/1.8/gems/data_objects-0.10.1/lib/data_objects/pooling.rb:119:in `new'
from /home/vjuser/.bundle/ruby/1.8/gems/data_objects-0.10.1/lib/data_objects/pooling.rb:119:in `synchronize'
from /home/vjuser/.bundle/ruby/1.8/gems/data_objects-0.10.1/lib/data_objects/pooling.rb:119:in `new'
from /home/vjuser/.bundle/ruby/1.8/gems/data_objects-0.10.1/lib/data_objects/connection.rb:63:in `new'
from /home/vjuser/.bundle/ruby/1.8/gems/dm-core-0.10.2/lib/dm-core/adapters/data_objects_adapter.rb:233:in `open_connection'
from /home/vjuser/.bundle/ruby/1.8/gems/dm-core-0.10.2/lib/dm-core/transaction.rb:412:in `open_connection'
from /home/vjuser/.bundle/ruby/1.8/gems/dm-core-0.10.2/lib/dm-core/adapters/data_objects_adapter.rb:269:in `with_connection'
from /home/vjuser/.bundle/ruby/1.8/gems/dm-core-0.10.2/lib/dm-core/adapters/data_objects_adapter.rb:136:in `read'
from /home/vjuser/.bundle/ruby/1.8/gems/dm-core-0.10.2/lib/dm-core/repository.rb:145:in `read'
from /home/vjuser/.bundle/ruby/1.8/gems/dm-core-0.10.2/lib/dm-core/collection.rb:1111:in `lazy_load'
from /home/vjuser/.bundle/ruby/1.8/gems/extlib-0.9.15/lib/extlib/lazy_array.rb:411:in `each'
from /home/vjuser/.bundle/ruby/1.8/gems/dm-core-0.10.2/lib/dm-core/collection.rb:506:in `each'
I note this is occurring on/in a Mutex.synchronize so I assume we're dealing with a hairy threading problem.
Comments and changes to this ticket
-
Dan Kubb (dkubb) September 17th, 2010 @ 12:13 AM
- State changed from new to unconfirmed
- Assigned user set to Dirkjan Bussink
Please Sign in or create a free account to add a new ticket.
With your very own profile, you can contribute to projects, track your activity, watch tickets, receive and update tickets through your email and much more.
Create your profile
Help contribute to this project by taking a few moments to create your personal profile. Create your profile »