data_migrate v1.1.0

By vargonaut | Published:

Only a few days, and I already have a new version of my gem Data Migrate. If you are not sure what this is all about, I encourage you to check out the project on Github, or check out my introductory post.

What changed?

Installation has changed to merely including the gem in your project. The last version required you generate and migrate the data schema table. This version does that automatically for you. When you use one of the supplied rake tasks, it will check to see if the table exists, and create it if not.

I’ve also updated the README to tell folks this, and add some information about why you ought to use Data Migrate.

If you installed v1.0.0, you’ll want to remove the migration ‘create_data_migrations_table’. If you don’t you’ll get an error that the table exists if you re-migrate everything using one of the data related tasks.

Why’d you do that?

KB was giving it a run for me, and discovered a bit of a frustration. While it worked as promised, deploying was a bit wonky. If you only had schema migrations, it was smooth, but he updated his cap:deploy to use ‘db:migrate:with_data’. Because it needs to get pending migrations, it needs the data schema table in place before it can run this task. As a result, he had to migrate through creating the table, and then could proceed with the new task.

It works fine in development, but I doubt folks would be pleased having to go through the hassle during a deploy. I thought it would be more friendly if it acted more like your stand migrations, where the table is created at need. After reading through the source to see how its done with ActiveRecord, I added the functionality to Data Migrate.

I hope this didn’t cause anyone grief. Of course, let me know if you notice anything else wonky.

data_migrate

By vargonaut | Published:

[edited for clarity and to add what itch data migrate scratches]

Data Migrate is a gem I created to migrate data alongside schema changes. As you will see below, it provides a number of rake tasks (with some other support) that allow you to create and migrate data as you would database changes.

Why you want to use this

Its seems when a project hits a certain size, I get to manipulate data outside the application itself. Changing defaults, new validations, one-to-one to one-to-many… I found it a pain and dodgy to have to step up migrations one by one, run a ruby script of some sort, then resume migrations. It tanks a lot of the automation of deploy.

If you don’t use the one off scripts, you could do it as a regular migration. It’d be much better to keep concerns separate. The benefit of having them separate has to do with your data model.

For instance, lets take an absurd example, to illustrate: You have your infamous Rails blog that has posts with many comments. After some use, you decide you are going to be a trend setter, and want only one comment per post, and just the text. “Frist!” rules the day. Given that you:
– write a migration to add a comment column to Post
– write a migration to move the contents of the first comments to the Post
– drop the column_id column from Post
– drop the Comment model
– fix all your test/controller/view mojo.

You’ve just got bit. When you rake setup:development, the mess gets mad at you after it creates your database, and starts cranking through migrations. It gets to the part where you iterate over the comments and it blows up. You don’t have a comment model anymore for it to even try and get ‘all’ from. You think you are smarter, and wrap the AR call in a conditional based on the environment. That’s fine until you get that QA gal, and she wants her own thing. Then the UI people get tired of waiting for the full stack to load on page refreshes, so you have to edit past migrations…

With Data Migrate, you have the control. You can generate your migrations as schema or data as you would as your work flow. For setting tasks that don’t require any intermediate AR activity, like dev and test, you stick with db:migrate. For your prod, and qa, you change their scripts to db:migrate:with_data. Of course you want to test your migration, so you have the choice of db:migrate:with_data or data:migrate to just capture that data change.

So, what’s it going to do?

Once installed you will have a new table that tracks ‘data_migrations’ in the same way Rails handles ‘schema_migrations’ for you. That’s a good thing to point out about my language. When I say ‘data’, I mean just that. Mucking with the models as it were. By ‘schema’, I mean structural changes to the database – new tables, changed columns, etc.

As you use the gem, data migrations are stored in db/data. They act like schema migrations, except they should be reserved for data migrations. For instance, if you realize you need to titleize all yours titles, this is the place to do it.

Read More »

Create or Select a value via Nest Resources

By vargonaut | Published:

I am currently working on a small project and intend to do an auto-complete for a field. To make it friendly, it’d be great if a user could choose an existing value via the auto-complete, or have a new value saved. For instance, if you have say, our friend the blog, and that has Post which belongs_to Category. My goal is a text field for category (as opposed to the usual select) that will allow both an existing value, or a new one.

I also wanted it to be safe. My first attempts at this ended rather poorly, and just didn’t feel right. I tried both some madness on the models, and some transactions in the controller when saving. The transactions do work, but you wind up with a lot of book keeping for such a small thing.

The approach I settled on (as you will see) is using nested attributes, but in reverse. All the examples I found assumed you wanted the has_many end of the relation in the form. Some time in irb later, I nailed it.

Read More »