Sunday, June 28, 2009

Updating multiple form items with Ajax.InPlaceEditor

I have a few places where I want to do in place editing in my Rails app, so I tried using the official Rails in_place_editing plugin, but my form setup was just too complex for it to work. (From doing some reading up online, the plugin also has some other issues and hasn't been maintained. Because of this there are several alternative plugins out there.) But I eventually wound up finding a solution I could adapt and use.

So after implementing that, simple in-place editing worked beautifully! Unfortunately, simple in-place editing isn't quite enough for my app. When a page item is updated, sometimes I need it to update another item on the page. The script.aculo.us Ajax.InPlaceEditor is only designed to update one page item--the one that was edited. I tried working around it by changing something like this (in my controller action where I handled the AJAX call):

render :text => new_value

To something like this:

render :update do |page|
page.replace_html(original_page_id_to_update, :text => new_quantity)
page.replace_html(secondary_page_id_to_update, :text => new_value)
end

That didn't quite work. It updated the secondary_page_id_to_update element correctly, but instead of putting new_quantity into original_page_id_to_update, it put in something like this:

try { Element.update("original_page_id_to_update", new_quantity); Element.update("secondary_page_id_to_update", new_value); } catch (e) { alert('RJS error:\n\n' + e.toString()); alert('Element.update(\"original_page_id_to_update\", new_quantity);\nElement.update(\"secondary_page_id_to_update\", new_value);'); throw e }

I was stumped on this for quite a while. Spent a lot of time scouring The Google for answers, digging into the script.aculo.us documentation for Ajax.InPlaceEditor, and examining the actual Ajax.InPlaceEditor code. What my web searches turned up was that I'm not the only person with this problem--I found several forum posts on different sites from people trying to do the same, or similar, things. Unfortunately, none of them had a solution. :(

So I went for a run to take a break, then came home to take a shower and like Archimedes, I had a "Eureka!" moment. (Though his was in the bath.) I realized what was happening was that both of my replace_html calls were actually working, but as part of what Ajax.InPlaceEditor does it returns the final value of the render call, and sticks it into the field that was edited. This is just the default behavior, as it assumes you only want to update that one field.

Looking into the Ajax.InPlaceEditor code, I found this (relevant lines in bold):

handleFormSubmission: function(e) {
var form = this._form;
var value = $F(this._controls.editor);
this.prepareSubmission();
var params = this.options.callback(form, value) || '';
if (Object.isString(params))
params = params.toQueryParams();
params.editorId = this.element.id;
if (this.options.htmlResponse) {
var options = Object.extend({ evalScripts: true }, this.options.ajaxOptions);
Object.extend(options, {
parameters: params,
onComplete: this._boundWrapperHandler,
onFailure: this._boundFailureHandler
});
new Ajax.Updater({ success: this.element }, this.url, options);
} else {
var options = Object.extend({ method: 'get' }, this.options.ajaxOptions);
Object.extend(options, {
parameters: params,
onComplete: this._boundWrapperHandler,
onFailure: this._boundFailureHandler
});
new Ajax.Request(this.url, options);
}
if (e) Event.stop(e);
}

What was happening was that it was creating a new Ajax.Updater, which is meant to update a single element with the return value from the url it calls, which turned out to be that chunk of Javascript above when I used the render :update block. The Javascript was being eval'd correctly, and updating both my fields, but then Ajax.Updater was re-updating the 1st field with the result of the render block, which was that chunk of Javascript. (BTW, I think this Rails bug is the same thing, but they didn't see that anything was being eval'd because they only had the one field being updated, so their guess was that the eval was failing.)

I tweaked the Ajax.InPlaceEditor code to replace the Ajax.Updater line with the Ajax.Request line, and suddenly everything worked! But that didn't seem like the right solution, so I looked at the logic in that block. It checks options.htmlResponse, so I tried setting :htmlResponse to false where I set up the Ajax.InPlaceEditor, and voila! It worked.

So hopefully this will be useful to people, and turns out to be very easy. Here's the summary:

  1. Use a render :update block with as many replace_html calls as you need (including one to update the in-place edit field itself).

  2. Set :htmlResponse to false when you set up your Ajax.InPlaceEditor.


And that's it!

Saturday, June 13, 2009

Beware trying to do an Array find() with an ActiveRecord collection, and other various lessons.

Wow, this is something that's been haunting me for a week or two, and I just couldn't figure it out. I've been on a major performance improvement spree on Project Unblowuppable, which has always been a very database intensive application. After building up a decent sized database in production through the private beta, I pulled it down to play around with locally and find the real bottlenecks. One particular page (the one that does by far the most work) for one user was doing over 1200 db queries! Ouch. I've been trying to optimize that for a few weeks by using :include (in several cases nested includes) in my original queries to eager load as much as possible up front, and by rearranging the code to do things like pull db queries out of loops where possible. On most of the pages this was a big win, but not on that One Evil Page. I used the same techniques there, and spent literally a few weeks (in my spare time) going over the code again and again, and looking at what was going on in a debugger (thank you, RubyMine!!). Nothing worked, but watching exactly what was happening in the debugger and doing lots of research on Google was very interesting.

If you watch what gets retrieved in an ActiveRecord find() call in a debugger, you'll see (of course) that by default it doesn't retrieve any of the associated records (via belongs_to, has_many, etc.) automatically. If you use the :include option, you can tell Rails that you want to pull those associations at the same time, so you're not doing more queries later--quite possibly one at a time in a loop, which is terrible if db queries are a bottleneck for your app. An interesting thing if you peek at the query result is that the :included record(s) may not actually be there in the model if you drill down through it. In fact, what you may see is an new attribute for an association you loaded that's nil! I kept seeing this and thinking "WTF?" But looking at the SQL in the server log, I could see that indeed the associated records were being loaded--but where did Rails put the data?

In most cases, using records with eager loaded associated data worked well, but not always. Eventually I was satisfied that the data was somewhere, even though I couldn't drill down to it and view it directly in the debugger. It turns out ActiveRecord does some method_missing magic to pull that data in. Ok, no problem--there's lots of PFM in AR. :) But on the One Evil Page, even though I was pre-loading the data, it was still doing a bunch of queries to pull it from the DB again. What I finally decided to do was switch from using an ActiveRecord find() (find_by() really) to a standard Ruby Array find(). Looking at the types in the debugger, my collection was an Array, so this should work, right? Not necessarily. Rails does some magic with the find() method on these collections (which are actually AssociationProxies) and instead of just doing an in-memory search through the array, it was still doing a db lookup. So I did some looking through the pickaxe book and saw that in Enumerable collections, find() is actually just an alias for detect(). So I replaced 'find' with 'detect' and PRESTO: with all the eager loading set up, I dropped the number of db queries on the Evil Page from over 1200 to under 150! (And I'm still not done--I think I can still cut it a lot more.)

So the moral of the story? I'm not sure. I don't know if this is a bug or a feature in AR, or if it's just a side effect of my design, but wanted to put this out there in case anybody else is having a similar issue. Do some playing around and see if using eager loading and Ruby's Array detect() can cut down your db queries. (Note: the query_reviewer plug-in is an essential tool for checking your db usage. Not only will it give you the total query count, but it will show you duplicate queries and give you the stack trace of exactly where the problem is! It also shows when indices are missing or not being used, which may just be because mysql decided there wasn't enough data yet to bother.) A related note about Array find_all(): according to this thread, it is also hijacked by AR and the equivalent thing to do is use select() instead. But in my case whether I use find_all() or select() the effect is the same. Not sure why this seems to work differently than find()/detect(). Anyway, just be aware of this sort of thing and play around. Hopefully this isn't some bug that future versions of Rails will fix and break in my app. ;)

A similar thing that may help cut down db queries is using Array length() (or size()) instead of count() to get the number of records in a collection. This is great if you've already loaded the records, especially if the count is being done in a loop, which was my case. I was able to shave off a lot of SQL count queries this way.

An aside about using :include: I guess Rails (pre 2.0?) used to do a big Cartesian join when you did includes, which I can imagine was pretty horrible. The things that caused me to do some head scratching following my AR model objects in the debugger are probably a side effect of the fix for that, which is to do separate (and simple) db queries to load the data separately instead of joined. Looking at the number of SQL queries this looks very efficient, and like a very clever solution. I'm betting there was a lot of work to make this happen so transparently--my hat's off to the Rails team for this and all their other hard work!

Sunday, June 7, 2009

hpricot and Ruby 1.8.7

So the problems from my Ruby 1.8.7 upgrade aren't over yet...! None of the Rails scripts (including script/server) will run from the terminal on my MacBook, because of an hpricot incompatibility. I've ignored it because everything still seems to work from RubyMine--I run the app there, and can do other things like script/generate. But I decided to try to fix it anyway.

Found a page that said yes indeed, hpricot and Ruby 1.8.7 don't get along, so I pulled down the git fork, tried to build and install, but had build problems with it (it was trying to build for an old darwin target, I think). So I tried pulling down and building Why's git repo per the instructions. Sure enough, that built and installed a new hpricot version--apparently the gem is not up to date.

Some notes: You'll need ragel, which can be installed via macport, and fast_xs, which is available as a gem to build and install hpricot.

Wednesday, June 3, 2009

Finally fixed 'bundled mysql.rb driver has been removed from Rails 2.2' issue

Well, I finally sorted out what the problem was that was causing the 'bundled mysql.rb driver has been removed from Rails 2.2' error. If you read back a few posts you'll see my saga of updating ruby from 1.8.6 to 1.8.7 and the issues it caused with my gems. Well, I fixed those for my user account, but not for when I used sudo--they give different lists of installed gems when I do 'gem environment'/'sudo gem environment'. Ran the old gem tool and got the same list with and without sudo. Hmmm. So I tried uninstalling/reinstalling the mysql gem with the old gem tool, and bam! problem solved.

Well, almost. After all the stuff I'd done I nuked all my mysql databases (no big deal, only dev/test data, and easy to restore with Rails migrations) but more importantly all my mysql users. So used the MySQL Administrator tool to add back my user, get the permissions set up correctly, and everything running again.

Well...almost. My app runs fine inside RubyMine, but doing script/server from Terminal still fails for some reason (and as far as I can tell, script/server doesn't have a 'verbose' option to give details). Whatever. If I can run it in RubyMine, I can work again. Woohoo!

Oh--and one other handy thing I found: Start and Stop MySql in Mac Os X 10.5 Leopard

Tuesday, June 2, 2009

Upgrading mysql from 5.0 to 5.1

Ugh. So I saw there was a newer version of mysql out there (5.1) so I decided to upgrade the MacBook Pro to use it instead of 5.0. Before I did that, my Rails app was running fine. After doing the upgrade I started getting this error:

The bundled mysql.rb driver has been removed from Rails 2.2. Please install the mysql gem and try again: gem install mysql.

I've spent hours searching Google (and Bing, even!) for an answer, but haven't found anything that works. I've uninstalled and reinstalled the mysql gem many times, with different command line param combinations, no effect. I even totally cleaned mysql and the mysql gem off the computer (http://akrabat.com/2008/09/11/uninstalling-mysql-on-mac-os-x-leopard/), reinstalled mysql 5.0, and it's still fucked. I'm totally stumped. Put up a question on stackoverflow, but no luck yet.

I don't know how I'm going to fix this, and I'm dead in the water for development because of it. FUCK!!