Matt Connolly's Blog

my brain dumps here…

Tag Archives: rails

Passenger apache module for OpenIndiana

I did a bit of hunting and made some patches to the ‘passenger’ gem so that it’s apache module would compile for OpenIndiana. Changes are in my github fork:

https://github.com/mattconnolly/passenger

And I just noticed that one of the fixes was in a patch in Joyent’s SmartOS instructions for using passenger.

I tested this also on a VM guest installation of Solaris 11 Express, and it worked too. I’d be interested to hear if it works for others on OpenIndiana, Solaris or SmartOS.

So with updates to rvm, latest version of ruby and with this patched version of passenger, I’m finally good to go to deploy rails apps on OpenIndiana. Woot!

Advertisements

A Ruby gem to backup and migrate data from a Rails app

Ever wanted to quickly duplicate your database in a rails app so you can do some development work on real data, but not risk breaking it?

Background:

Rails has some very good strategies for managing your schema with rake tasks such as db:migrate, db:schema:load, etc. Switching between production and development environments lets you use different databases.

Great, but how do you get the data from one environment to another?

With MySQL you can duplicate a database (easy with phpMyAdmin), and with Sqlite3 you can simply copy the database.db file.

But what if I want to go from MySQL to Sqlite3 just for working??

There are some database differences, and there’s a rake task that’s already managing the database structure. I need something to move the data as well.

In my particular case, I’m using redmine as an issue tracker, hacking on it, and also evaluating the Chiliproject fork. So I’d like to be able to move all my data: schema, data and files easily from one to another.

The Rails-Backup-Migrate Gem

So, I put together a gem to do this. It’s inspired by Tobias Lütke’s backup.rake file he posted a good 5 years ago: http://blog.leetsoft.com/2006/5/29/easy-migration-between-databases

It appears to be for an early version of rails. It also had some issues with tables belonging to “many-to-many” relationships, and was also slow because it instantiated every single ActiveRecord object as it went.

I’ve sped the process up by skipping the ActiveRecord object steps, and updated the table definitions for rails 2.3 (redmine runs on this) and it appears to be working for rails 3.0 as well.

This gem provides the following rake tasks to a rails app:

rake site:backup[backup_file]                          # Backup everything: schema, database to yml, and all files in 'files' directory.
rake site:backup:db[backup_file]                       # Dump schema and entire db in YML files to a backup file.
rake site:backup:files[backup_file]                    # Archive all files in the `files` directory into a backup file.
rake site:backup:schema[backup_file]                   # Dump schema to a backup file.
rake site:restore[backup_file]                         # Erase and reload db schema and data from backup files, and restore all files in the 'files' directory.
rake site:restore:db[backup_file]                      # Erase and reload entire db schema and data from backup file.
rake site:restore:files[backup_file]                   # Restore all files in the 'files' directory.
rake site:restore:schema[backup_file]                  # Erase and reload db schema from backup file.

The default backup_file is "site-backup.tgz" and is relative to rails app root directory.

Installing:

Installing manually:

$ gem install rails-backup-migrate

Installing with Bundler:

Add the following to your Gemfile:

gem rails-backup-migrate

And then

$ bundle install

Using rails-backup-migrate:

To use the gem in your rails app, simply add the following line to the end of your Rakefile:

require 'rails-backup-migrate'

Which will load the above tasks into your rails app’s task list. If you’re using bundler

$ # backup everything into the default file 'site-backup.tgz'
$ rake site:backup
$ # now, restore into another instance (eg: clone/checkout another copy of your project)
$ cd /path/to/another/instance/of/my/rails/app
$ rake site:restore[/path/to/original/instance/of/my/rails/app/site-backup.tgz]

Done.

The source code is available on github: https://github.com/mattconnolly/rails-backup-migrate

This is the first gem I’ve made and published, and I hope it makes life a little bit easier for someone!

Rails data backups, independent of database

With my continuing my interest in redmine / ChiliProject, I’m really wanting a way of backing up my data that is database independent. I did a bunch of searching around, and the best solution I found was here. However, it appears a little out dated, and didn’t work in Rails 2.3 for a few reasons:

  • There isn’t always a 1:1 mapping between tables and models. (Example: many-many relationships create extra tables).
  • The excluded tables list was outdated (probably from an earlier Rails)[1]

There were also some comments about the export being quite slow. This could have been because every record of every table was loaded up in an ActiveRecord model instance before being dumped to the YML files.

I addressed these issues and made a new version. Can’t attach a file to this blog, so it’s on pastie for now: http://pastie.org/1877530

I’ve tested this getting data out of mysql and into sqlite3 and that works just fine. This is really handy for getting the data into a test environment quickly without having to set up more mysql databases, users, etc.

[1]: I’m testing this for Rails 2.3.11 as Redmine and ChiliProject are Rails 2.3.x apps.

If this saves you some time, or could be improved, please let me know.

Using git to move from redmine to chiliproject

I originally got onto Redmine because of its great out of the box support for git as well as remote subversion repositories. I’m keeping an eye on the “chiliproject” fork of “redmine” at the moment.

I have a redmine installation which includes a few custom modifications. So I have my “local” branch, which forked from redmine’s 1.1-stable some time ago, but when there’s a new version to do, I merge the new redmine version into my local, giving me, for example, redmine 1.1.3 + my changes.

So in order to evaluate chiliproject, and to see if my custom changes would go across, I figured I’d just add the chiliproject git remote and merge it across. Easy right? Well, kind of.

Simply doing this, didn’t work:

$ git remote add chiliproject git://github.com/chiliproject/chiliproject.git
$ git fetch chiliproject
$ git merge chiliproject/stable

This resulted in a *whole bunch* of merge conflicts; many of which actually looked to me like they should have been resolved automatically.

It appears those two repositories have a lot in common, but the main branches I’m interested in (redmine/1.1-stable and chiliproject/stable) have quite a distant common ancestor, which is what git will use to do the merge.

So, check this out:

$ # how many changes have I made from `redmine/1.1-stable`... out of curiosity?
$ git diff redmine/1.1-stable | wc
    3340   11917  126173
$ git branch chili-stable chiliproject/stable
$ git checkout chili-stable
$ git merge -s ours redmine/1.1-stable
$ git branch local_chili local_redmine
$ git checkout local_chili
$ git merge chili-stable
<--snip-->
$ # merge succeeded, how many changes do I now have from `chiliproject/stable`?
$ git diff chiliproject/stable | wc
    3340   11917  126173

And Voila! All of my changes since redmine/1.1-stable can be applied to chiliproject/stable without any conflict. So, when two git repositories are related by difficult to merge, let `git merge -s ours` come to the rescue!

Redmine: Ruby on Rails without Virtual Hosts

I’ve been playing around with Redmine, a Rails app, on Mac OS X 10.6. It’s been quite a pain to get set up. It seems that the only easy way to get passenger to work is with virtual hosts. It’s not really obvious or intuitive how virtual hosts work. After a lot of RTFMing the apache docs, and passenger google group, I got it working. But Virtual hosts are still a pain, because you have to set them up in your /etc/hosts file more for every machine on the network, and there’s still a high likelihood of breaking other things (like php apps)

So, is it possible to install redmine (or any ruby on rails app) in a directory without using a separate virtual host just for the app, AND without taking down other php apps (drupal, phpmyadmin, etc) which are running on the same apache server?

Yes. After lots of trial and error, the solution is fairly simple, but has to be done in a specific way.

Step 1. Configure httpd.conf files

Add this to your httpd.conf file, (or an included file) before your users/other conf files are loaded:

LoadModule passenger_module /Library/Ruby/Gems/1.8/gems/passenger-2.2.15/ext/apache2/mod_passenger.so
PassengerRoot /Library/Ruby/Gems/1.8/gems/passenger-2.2.15
PassengerRuby /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/ruby
PassengerEnabled Off

Follow the passenger install instructions to make sure that you’ve got the right paths and versions in there. “PassengerEnabled Off” in the global scope disables it by default. We’ll enable it just for where we want it.

Step 2. Setup a .conf file for the rails app

In my case, I’ve added a “redmine.conf” file which is imported by httpd.conf. In this file, I have:

<Location "/redmine">
    Options Indexes ExecCGI FollowSymLinks -MultiViews
    Order allow,deny
    Allow from all
    AllowOverride all

    PassengerEnabled On
    RailsBaseURI /redmine
    RailsEnv production
</Location>

Here we enable Passenger, set the rails environment and the rails base URI.

Step 3. Setup a symlink

I had tried this with real directories and apache “Alias” commands to no luck. The passenger module docs does have instructions, but only for use with virtual hosts, if you ignore the virtual hosts bit, it works. On the mac, I have it set up like this:

$ # /Library/WebServer/Documents -- document root
$ # /Library/WebServer/Sites -- for rails and other apps.
$ cd /Library/WebServer/Documents
$ ln -s ../Sites/redmine/public redmine

Finally, a solution that other network machines can access without Virtual Host nightmare.