#328 ✓not-applicable

sharding support

Reported by jeff | May 30th, 2008 @ 04:39 PM

would be nice for DM to have support for sharding like http://www.hivedb.org does for java.

Comments and changes to this ticket

  • Dan Kubb (dkubb)

    Dan Kubb (dkubb) June 5th, 2008 @ 01:35 AM

    jeff, I'm curious how you think this would work? I've not used hivedb before, but how do you see it fitting in with DataMapper?

  • Sam Smoot

    Sam Smoot June 5th, 2008 @ 11:03 AM

    Yehuda and I have been talking about this a bit. I think we can do something in Repository to handle this. So instead of Repository -> Adapter being 1:1, it'd become 1:N(readers)+N(writers).

    Obviously the idea needs more fleshing out, but whatever the solution, I think we can extend the capabilities of the existing system rather than adding a bunch of new classes, and we have to be ultimately concerned that the Adapters should not be aware of the sharding at all.

    This way, eventually, you might have in-memory storage for readers for example, and database storage for a writer, and the readers are updated through a background synchronization process.

    Just to throw out something wild. ;-) But it's vitally important the Adapters remain ignorant as much as possible. dm-core should do all the heavy lifting as a general philosophy.

  • Adam French
  • DBA

    DBA August 21st, 2008 @ 05:31 AM

    • Tag set to feature, suggestion

    For reference:

    ActiveRecord sharding with DataFabric, by FiveRuns: http://blog.fiveruns.com/2008/7/...

    Read slaves with ActsAsReadOnlyable plugin (not sharding but a lot of sharding setups use read slave clusters): http://revolutiononrails.blogspo...

  • Carl

    Carl November 10th, 2008 @ 08:43 AM

    I think that's a great idea, but it seems like support for master/slave replication setups "out of the box" would be even more useful for the vast majority.

    So, count me in for sharding, but I'd prefer replication support if you had to implement just 1 of these features.

  • Carl

    Carl November 25th, 2008 @ 06:55 PM

    • no changes were found...
  • Jonathan Stott (namelessjon)

    Jonathan Stott (namelessjon) November 30th, 2008 @ 07:21 AM

    • Tag changed from feature, suggestion to dataobjects, feature, suggestion
    • State changed from “new” to “open”
    • Assigned user cleared.
  • Dan Kubb (dkubb)

    Dan Kubb (dkubb) January 7th, 2009 @ 04:19 PM

    • State changed from “open” to “not-applicable”

    I am going to close this ticket as per: http://is.gd/eOHY

    However, this is something I would like to discuss on the mailing list if anyone seeing this is interested in starting it there.

    Sharding is an interesting problem, one that I've not really seen solved to any degree of satisfaction before by other ORMs. I believe we're closer than others I've seen, but there should be some real world discussion about sharding strategies and how that should fit in with DM.

Please Sign in or create a free account to add a new ticket.

With your very own profile, you can contribute to projects, track your activity, watch tickets, receive and update tickets through your email and much more.

New-ticket Create new ticket

Create your profile

Help contribute to this project by taking a few moments to create your personal profile. Create your profile »

People watching this ticket