#999 ✓resolved
siplux

dm-types YAML causing "Column length too big for colum" error during migrations with MySQL

Reported by siplux | August 2nd, 2009 @ 08:15 PM

Getting the same error as this ticket with latest revision of the 0.10.0 branch, http://datamapper.lighthouseapp.com/projects/20609-datamapper/ticke...


diff --git a/dm-types/lib/dm-types/yaml.rb b/dm-types/lib/dm-types/yaml.rb
index 311d6a6..f44c4c7 100644
--- a/dm-types/lib/dm-types/yaml.rb
+++ b/dm-types/lib/dm-types/yaml.rb
@@ -3,8 +3,9 @@ require 'yaml'
 module DataMapper
   module Types
     class Yaml < DataMapper::Type
-      primitive Text
-      lazy true
+      primitive String
+      length    65535
+      lazy      true
 
       def self.load(value, property)
         if value.nil?
@@ -12,7 +13,7 @@ module DataMapper
         elsif value.is_a?(String)
           ::YAML.load(value)
         else
-          raise ArgumentError.new("+value+ must be nil or a String")
+          raise ArgumentError.new("+value+ of a property of YAML type must be nil or a String")
         end
       end

Comments and changes to this ticket

  • phatmann

    phatmann October 2nd, 2009 @ 12:39 PM

    Here is a lame monkeypatch for people that need a fix now. I am not yet sure what the best long-term fix is.

    module DataMapper
      module Migrations
        module DataObjectsAdapter
          module ClassMethods
            alias _type_map type_map
            def type_map
              t = _type_map.dup
              t[Types::Yaml]  = { :primitive => 'TEXT' }
              t
            end
          end
        end
      end
    end
    
  • Jordan Ritter

    Jordan Ritter October 2nd, 2009 @ 03:04 PM

    I solved this with a different approach.

    FWIW, I'll speculate that the Text "class primitive" went away because DM already had a "text" class primitive (called String). Given DM's stated intent of remaining generalizable over RDBMS-specific implementations: having two class primitives for "text" objects -- which were/are necessary because of specific RDBMS implementations (VARCHAR vs. CLOB/*TEXT) -- maybe was seen both as a high-level philosophical DRY contradiction and a general "this is against our stated religion". Not that DM devs are inflexible -- because they've been quite happy to fix and extend things.

    However, for the vast majority of us, DM is our window into an RDBMS. And those RDBMS's sometimes use different fields for (objectively) the same data (i.e. VARCHAR vs. TEXT), just with different limitations. So we need this functionality.

    My approach was to change how the type_map looks up what the underlying schema primitive is, instead looking at the property.type's ancestry and using theirs. There are places in the DM code that recognize the class type ::DM::Types::Text and convert its underlying schema primitive to 'TEXT'; therefore deriving a YAML type from ::Text type would result in the same schema primitive being used. This has the added bonus that the recently-added field auto-sizing feature (via :length) works as expected.

    For example:

    module DataMapper
    module Types

    class Yaml < DataMapper::Types::Text
      primitive String
      # ...
    end
    

    end end

    Then:

    class Foo
    property :bar, Yaml, :length => 2**24-1 end

    Would result in a MEDIUMTEXT being used in MySQL for the field.

    This requires a change in dm-core (http://github.com/jpr5/dm-core/commit/bd8ad8979a66cc47f885d9e2a5bdc..., and updates to dm-more's Csv, Yaml and Json types (http://github.com/jpr5/dm-more/commit/9c970d2120bb4b740989597ec02f5....

  • Dan Kubb (dkubb)

    Dan Kubb (dkubb) October 3rd, 2009 @ 02:20 AM

    • State changed from “new” to “accepted”
    • Assigned user set to “Dan Kubb (dkubb)”

    I actually think perhaps we should make Text a primitive in the short-term, and then make sure the Csv, Yaml and Object types use Text as a primitive.

    The long term approach will be to have DM::Property subclasses, and put custom types and properties on equal footing. There won't be anything special about custom types vs. Property
    objects, and serialization methods will be able to be defined on a per Property basis.

  • Dan Kubb (dkubb)

    Dan Kubb (dkubb) October 6th, 2009 @ 12:38 PM

    • State changed from “accepted” to “resolved”

    This is now resolved in dm-core/master.

Please Sign in or create a free account to add a new ticket.

With your very own profile, you can contribute to projects, track your activity, watch tickets, receive and update tickets through your email and much more.

New-ticket Create new ticket

Create your profile

Help contribute to this project by taking a few moments to create your personal profile. Create your profile »

Pages