Marshal problem with DO and Ruby 1.9.1
Reported by Dan Kubb (dkubb) | May 11th, 2010 @ 12:46 PM
Today, when testing the Property/Type refactoring I ran into an issue in Ruby 1.9.1, DO, and the Object type.
Previously values of an Object type were serialized using Marshal and base64 encoded. The base64 encoding is no longer being done, and I think it exposed a bug in DO.
To reproduce, go into dm-core edge and run the following commands:
NOTE: Make sure you have the bundler gem installed, and ruby 1.9.1p378 (or ruby-head)
- rake local_gemfile
- ADAPTER="sqlite" BUNDLE_GEMFILE="Gemfile.local" bundle
install
- ADAPTER="sqlite" BUNDLE_GEMFILE="Gemfile.local" bundle exec spec spec/public/property/object_spec.rb
You should see a stacktrace similar to this: http://pastie.org/955692.txt
The error is "marshal data too short", which after some initial research shows that perhaps the data marshal is working on is being truncated in some way. My guess is it's an encoding problem, but I don't know for sure.
This problem is repeatable with sqlite, or postgres; but mysql does not have any problems with this code.
Comments and changes to this ticket
-
Martin Gamsjaeger (snusnu) May 12th, 2010 @ 07:42 AM
- State changed from “unconfirmed” to “confirmed”
-
Collin May 12th, 2010 @ 09:42 AM
Some other permutations of the Object de/serialization issue: http://gist.github.com/397839
-
Collin May 12th, 2010 @ 09:52 AM
Also, we run into these issues with mysql.
I have become quite confused while looking into this issue. Sometimes the failure happens inside the c extension of the dm-*sql-adapter class. Last line of the stack trace is the call to reader.next! which is implemented in c and eventually gets here: http://github.com/datamapper/do/blob/master/do_sqlite3/ext/do_sqlit...
The dm-mysql-adapter has analogous code.
However, other times it get's here: http://github.com/datamapper/dm-core/blob/master/lib/dm-core/proper... and fails with "format version 4.8 required; 72.69 given". This is an error that indicates the data attempting to be loaded by Marshal does not start with the proper format. I have not confirmed this for the stack-trace in my gist but in the non-isolated case this the result of Marshal trying to load Base64 encoded data. It tries to interpret the first two bytes as the version, and gets something like: "HE"
-
Piotr Solnica (solnic) May 14th, 2010 @ 08:47 AM
I had a short discussion with Dirkjan today and we think this should be fixed on the DO adapters side rather then doing various workarounds in dm-core.
-
Dan Kubb (dkubb) May 22nd, 2010 @ 01:04 AM
- State changed from “confirmed” to “resolved”
Dirkjan: Object seems to be working again with the latest DO, but are there other things planned? Is DO going to handle Object natively again, or are you going to just defer to handling it in DM?
For now I'm going to mark this ticket as resolved, since the problem I reported is fixed.
Please Sign in or create a free account to add a new ticket.
With your very own profile, you can contribute to projects, track your activity, watch tickets, receive and update tickets through your email and much more.
Create new ticket
Create your profile
Help contribute to this project by taking a few moments to create your personal profile. Create your profile »
Dan Kubb (dkubb)
Dirkjan Bussink
Martin Gamsjaeger (snusnu)
Piotr Solnica (solnic)