
Generated relationship- / SEL-queries are more complex than neccecary
Reported by Sindre Aarsaether | July 30th, 2009 @ 09:36 AM | in 0.10.0
Examples of some queries generated by DM. SEL-loading parents of a collection (with belongs_to relationship to the parents):
SELECT *fields* FROM *model* WHERE "id" IN (NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, 1358128, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL) ORDER BY "id"
Solution for queries without composite keys could be as simple
as .compact.uniq. But at the same time, maybe make it a little
smart about composite-key loading to? So that loading resources
[1,234], [1,235], [1,236], [1,1213] is compacted to:
WHERE pk1 = 1 AND pk2 IN (234,235,236,1213)
Or, if more complex, like [1,234], [1,235], [1,236], [2,1213],
[2,1214] to:
WHERE ((pk1 = 1 AND pk2 IN (..)) OR (pk1 = 2 AND pk2 IN (...))
But beware, this might not be any faster (might take longer to clean up in ruby, and mysql might not care at all). Still, looks cleaner in the logs. That's gotta account for something right :)
Comments and changes to this ticket
-
Sindre Aarsaether August 1st, 2009 @ 02:51 PM
By the way, strange that mysql cares, but the queries with many NULLs greatly decrease performance.
SELECT * FROM people WHERE id IN (NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, 629705, NULL, NULL, NULL, NULL, 629705, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, 41315, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, 629705, NULL, NULL, NULL) ORDER BY id;
2 rows in set (2.42 sec)SELECT * FROM people WHERE id IN (41315,629705) ORDER BY id;
2 rows in set (0.00 sec)Query-cache etc is turned of. I've seen in the logs that some of the NULL-filled queries took a mysteriously long time to execute.
-
Dan Kubb (dkubb) September 10th, 2009 @ 04:04 PM
- Milestone set to 0.10.0
- State changed from unconfirmed to resolved
- Assigned user set to Dan Kubb (dkubb)
This should now be resolved in edge dm-core.
ATM I don't believe we're going to compact [1,234], [1,235], [1,236], [1,1213] to something like "WHERE pk1 = 1 AND pk2 IN (234,235,236,1213)". While I do like the idea of having the SQL query become minimal, I don't see it in the immediate future, unless you want to take a stab at it? (and it doesn't hurt the benchmarks)
I had the idea once that we'd have a Comparison#compact method that would walk through the Query conditions, and remove redundant entries, as well as factor out common conditions, and generally optimize the conditions. When I tried to implement it, I had to do a ton of work to walk through the conditions, more than I think we'd save when executing the query.
Please Sign in or create a free account to add a new ticket.
With your very own profile, you can contribute to projects, track your activity, watch tickets, receive and update tickets through your email and much more.
Create your profile
Help contribute to this project by taking a few moments to create your personal profile. Create your profile »