I recently spent some time working on a Rails application that needed to have various kinds of attachments, such as PDFs and images, for various types of resources, such as Programs and Questions. The app in question lets you create programs and questionnaires for the purpose of granting continuing medical education (CME) credits to doctors — so they’re apprised of the bleeding edge techniques for sawing off your arm or tying your tooth to a doorknob. Not that this matters for the piece, but context — like sugar or salt — makes things go down easier.

If you can picture an office of harried administrators dealing with ever-changing AMA requirements — “This brochure needs to go with this program!”, “This xray picture needs to go with Question 7″, “This program needs to sing ‘Missing You’ while showing you diagrams of a root canal”.. (OK, maybe not the second item), you’ll see why I wanted to build a system that could expand easily. I did not want to find myself in the position of trying to look tough while shamefacedly muttering, “I’m sorry, PDFs can only go with *Programs*, as you initially specified” and watching these not-so-gentle administrative souls stare at me with fully warranted incredulity. Blaming the client is like [fill-in-the-blank]: weirdly fun for about 10 minutes before the hangover sets in.

This project turned into a tour of attachment_fu, single table inheritance, polymorphism and functional testing of uploads. Thanks to input from friends, blogs and the usual Internet suspects, I got it up and running in its first form and decided to write it up. Hopefully what I’ve learned can help someone else. “Link-outs”, the reclusive programmer’s form of the “Shout Out” (aka a list of great resources), can be found at the bottom of the article.

Goal

To implement a system that would allow for many types of files to be uploaded and attached to many types of resources.

Requirements

1. Many types of assets (image, pdf, etc.)
2. Assets belong to many types of resources (program, question, etc)
3. Assets upload to the file system
4. Assets validate in the context of their parent resource [you upload the pdf in the program form; the image in the question form, etc]
5. All functional tests pass for uploads

Design

Rather than have database tables for each attachment type (PDFs and images, then eventually MP3s) I made one table called Assets. Listening to my requirements, a friend suggested setting up my Assets model using polymorphism and single table inheritance and that I look into attachment_fu for the uploads. For this exercise I will only be focusing on the PDF upload. Readers can extrapolate from there for image or other media types. This article does not address image resizing, or anything related to uploading images. There are some great links at the bottom that deal with these topics in depth.

Modeling the domain:

Assets (the table) has a field called type that is be [PDF, Image (etc)]; a field called resource_type which is the name of the class it belongs to, [Program, Question, etc.] and resource_id which is the id of the resource it belongs to.

The Asset class will inherit from ActiveRecord. PDF and Image (and any future file types we add to the system) inherit from Asset. Line items from Assets (the table) will look like this (leaving out the other fields for now)

type   | resource_type | resource_id

Pdf      | Program         | 82
Image  | Question         | 79

For my assets, I created the following models:
asset.rb, image.rb and pdf.rb. The latter two inherit from Asset.

# [Asset.rb]
class Asset < ActiveRecord::Base
  belongs_to :resource, :polymorphic=>true
end

# [Pdf.rb]
class Pdf < Asset
  belongs_to :program
  has_attachment  :content_type=>‘application/pdf’,
                         :storage=>:file_system,
                         :path_prefix=>’public/uploads/pdfs/’,
                         :size => 1.megabyte..3.megabytes
  validates_presence_of   :content_type, :filename

   def validate
     if filename &&  /pdf$/.match(filename).nil?
         errors.add(:filename, “must be a PDF “)
     end
  end

end

Because I am using attachment_fu, I need to have a set of basic fields in my table that will handle image and file uploads. The migration I created looks like this:

t.column :parent_id,  :integer     # for the thumbnails of resized images, required for plugin
t.column :type, :string               # for STI on this table (curent types are image, PDF)
t.column :resource_type, :string  # for polymorphism on this table (current types are Program, Question)
t.column :resource_id, :integer    # for polymorphism, (program_id, question_id, etc)
t.column :content_type, :string    # Fields here [content_type] and below all attachment_fu requirements
t.column :filename, :string
t.column :thumbnail, :string
t.column :size, :integer
t.column :width, :integer
t.column :height, :integer

To create the relationship for my parent models (question and program) I set up the polymorphic relationship like so:

# [program.rb]
class Program < ActiveRecord::Base
   has_many    :pdfs, :as=>:resource,  :dependent => :destroy
end

# [question.rb]
class Question < ActiveRecord::Base
  has_many      :images, :as=>:resource, :dependent => :destroy
end

Note the difference between delete and destroy. Destroy will remove the entry from the database and the file from the filesystem. Delete will only delete the database entry.

Once this is set up, you can test it in the console and see the magic of Rails at work. Through the column names and relationships we’ve built, entries in Assets are automatically created. It’s one of the things that’s great about Rails. It doesn’t simply describe different modeling patterns: it implements them if you set up your code correctly. It’s a beautiful thing.

Uploading

Without moving files to the server, the system is not complete. So, onto attachment_fu. The attachment_fu tutorials I found online all made the assumption that the attachment you are uploading is an entity of its own, in its own form: a mugshot, for instance. My case was different: I needed to embed the attachment in the forms for their parent resources (pdf in the program form; image in the question form, etc) rather than independently. So I needed to validate a form for more than one model. In my _form.rhtml partial for program I have the pdf field like so:

# [views/program/_form.rhtml]

< label for="program_notes" >Notes

<%= f.text_area :notes  %>

# … more fields …
< label for="program_status" >Add Brochure? [PDF Files only]< /label >
<%= file_field("pdf", "uploaded_data") %>

This means that both a pdf object and a program object are being passed back to the program controller. Since we know that a program has_many pdfs, we can use program.pdfs.build in order to create — then validate and ultimately save — pdf objects.

Validating in context of parent resource:

I needed to check for errors with my program fields and my pdf field. This snippet is what I ended up with. The second line, validate_and_build_pdf is just a result of refactoring: it allows the validation methods to be run if you select a file through the upload file field (so if you try to upload a gif, the validation in Pdf.rb will squak) — but not if you just leave that field empty.

#[programs_controller.rb]
def create
  @program = Program.new(params[:program])
   validate_and_build_pdf(params[:pdf],params[:pdf][:uploaded_data])
   if @program.save
      flash[:notice] = ‘Program was successfully created.’
      redirect_to :action => ‘list’ and return
   else
      render :action => ‘new’
   end
end

# this method stops program from saving the asset if the field is blank,
# and returns type-errors if relevant (files not PDF)
def validate_and_build_pdf(fileParams,file)
  if Asset.is_valid_file?(file)
     @pdf = @program.pdfs.build(fileParams)
  end
end

The reason for the Asset.is_valid_file? method was the same: if no file was uploaded an entry was still being made in the assets table due to the polymorphic relationship. So I had to make sure I was dealing with an actual fileobject, not an empty field. Since this would be used across the application for all types of file uploads, I made this a class method of Asset.

[Asset.rb]
# Checks that the object is one of the following types before running validation on it
def self.is_valid_file?(fileObj)
  if fileObj.is_a?(Tempfile)|| fileObj.is_a?(StringIO)
     || (defined?(ActionController::TestUploadedFile) && fileObj.instance_of?(ActionController::TestUploadedFile))
     true
  else
     false
  end
end

You may be wondering what’s up with the ActionController::TestUploadedFile condition — it’s because when you run functional tests of uploaded assets, that is what they are — they are not instances of fileObj.

Once it’s determined that you’re dealing with a valid file and you hit custom validation methods (such as informing the user that the file needs to be a pdf), you can include them with your program error messages so you only get one messages box at the top of your form, not two:

<%= error_messages_for :program, :pdf %>

Pass Functional Tests

This part is where you test all your uploads from your functional tests. To test uploaded files, you need to put files in your fixtures/files directory then build your tests.

# [programs_controller_test.rb]
def test_create_with_pdf
  fdata = fixture_file_upload(’/files/semi.pdf’, ‘application/pdf’)
  num_programs = Program.count
  post :create,   :multipart => true,
  :pdf=>{”uploaded_data”=>fdata},
  :program => {:name=>”Newest Created Program”,
                    :is_ongoing=>1,
                    :status=>”Edit”}
  assert_response :redirect
  assert_redirected_to :action => ‘list’
  assert_equal num_programs + 1, Program.count
end

I had a gotcha here — I realized that I has to have the pdf array separate from the program array (since in the form it’s field_for pdf). I fixed it by submitting two objects, program and pdf, to the controller, seen above.

Finally, running the functional testing leaves files on the file system and they should be deleted at the end of all tests. So I added a destroy method in the teardown, which removes them.

def teardown
  Pdf.find(:all).each { |p| p.destroy }
end

Failing Upload Tests
Just to check that only Pdfs can be uploaded for programs, I added a handful of other file types to my directory and created an array of invalid file types in my setup:

# [programs_controller_test.rb]
def setup
 # other stuff….
 @invalid_files = [['lipsum.doc',"application/doc"],['lipsum.rtf',"application/rtf"],
             ['lipsum.txt',"text/plain"],['squiggle.gif','image/gif' ],
             ['squiggle.jpg','image/jpg' ], ['squiggle.png','image/png' ],
             ['squiggle.psd',"image/x-photoshop"],['squiggle.zip', 'application/zip']
             ]
end

and created a test to check these uploads will fail:

  def test_create_with_invalid_file_types
    @invalid_files.each do |name, type|
      fdata = fixture_file_upload('/files/' + name, type)
      post :create,   :multipart => true,
                      :pdf=>{"uploaded_data"=>fdata},
                      :program => {:name=>"Newerest Created Program",
                             :is_ongoing=>1,
                             :status=>"Edit",
                             :start_date=>Time.now.tomorrow,
                             :end_date=>''}
      assert_response 200
    end
  end

This gave me peace of mind that my files are being uploaded, only the right types are accepted, and all is good in the land of PDFs and Programs. And voila, we are done and on to the next task.

Link Outs

Here’s a listing of various online resources on the topics covered in this post. This should be enough for a good start. Happy Uploading!

• Attachment Fu itself
• Significant Bits fu tutorial:
• Almost Effortless’ Fu-tutorial
• Mike Clark’s tutorial
• Changing upload location tutorial
• Testing file uploads: A broad overview (not using fu)
• Jonathan Leighton (who created the fixture_file_upload patch)
• More functional testing of attachment-fu
• An overview of Single Table Inheritance

The first time I was played with Selenium (the IDE) was a jaw-droppping moment. It was as if little sprites had taken over my computer and were gleefully doing my job. Sitting back and watching a form being filled in, a button selected, links linked — it was an incredible, mafia-man type experience: things were just getting done, and fast. The little guy rushed through my tasks in seconds, cheerfully, leaving green check-marks in his midst, while I sat back and kind of stared, the way you’re supposed to stare at a national monument. These days there are too many hot-dog stands and camera-d tourists at national monuments to merit my awe, but it’s safe to say that my little browser sprite was stepping up to the plate nicely. One could get used to this, I thought. Me, the sprite, and Firefox: what could be wrong in the world?

This didn’t last (amazing how quickly one gets spoiled). The problem with Selenium IDE is that the tests only play back in Firefox. I love Firefox, but the whole gist of my job is that the stuff we do needs to work in all browsers, on all machines (ok, most on most, but you get the idea). At any rate, I went looking for how I could extend the power of Selenium so I could test my web apps cross browser.

There are two approaches: Selenium on Rails (not covered in this article but you can find info here), and Selenium RC (Remote Control). After poking around a bit it seemed that RC was the solution for me. I was testing old apps, not Rails ones, so the Rails solution was overkill — and I had heard a general buzz that RC is a more robust, ‘best-practice’ solution. Whether that’s commonly held wisdom I can’t say (and if anyone wants to do a comparison, I’d love to hear), but it fit my requirements so I dove in.

If you’re not familiar with what Selenium RC lets you do, here: it lets you write tests in most languages (java, perl, python, ruby, php (NOTE: at the time of my testing I could not locate 0.9.1, with php support. It may be available now)) and then run them from the command line. Assuming the selenium server is running (see steps below), a proxy browser is fired up and your script is executed. In short, a browser opens and is interacted with. Each test is run and you’re fed a success or error message. It’s the same process you see when you run tests back using the IDE: your application being lovingly and quickly manhandled by invisible testing men (though based on the current online discussions I should probably make it clear that the gender of the magic sprites is unknown and irrelevant — and that they do not want to have sex with you, even if they are running in heels :))!

To the docs

The openqa docs are great resources. My one caveat is I would suggest a larger font and a clearer navigation scheme. It’s all there though, and it works, so read carefully, try the examples, and be patient.

From here on out I’m going to transcribe the steps I took to get up and running. This is the order I did things, so hopefully this is helpful as a list of steps and guidelines:

STEPS To set up Selenium RC on Windows

1. Read the openQA tutorial:
2. Download selenium RC here: [NOTE: 0.9.1 says it has php but I couldn’t find it anywhere. I got 0.9.0]
3. Extract it using winzip
4. Put in your c dir (or wherever you keep programs), like so: c:\selenium-remote-control-0.9.0.. I renamed it to selenium, so mine reads c:\selenium. NOTE: make sure you have right java version by typing java –version in your cmd shell. You must have at least 1.5 (1.4???)]. If you don’t, upgrade your java.
5. Put selenium.rb in your ruby lib dir so it will be accessible to ruby using require ’selenium’. If you don’t do this you’ll have to run your scripts from w/in the same directory as selenium.rb: you don’t wanna do that. For me I put it in C:\ruby\lib\ruby\1.8\selenium.rb
6. Start Server. You need to run from cmd line from WITHIN your selenium\server dir [ in my case c:\selenium\server] or feed it the path. [NOTE jar is Java Archive: To run an application packaged as a JAR java -jar app.jar]

java -jar selenium-server.jar –interactive.

At this point you can execute commands on the command line or you can execute a standalone script. There are commands in the tutorial that you can follow along with. To do the latter (execute a program), simply navigate to a test (there’s one in ruby/samples called selenium.rb ) and run it. The one that shipped with my ruby opens google and searches for the string “Hello World”.

C:ruby\\samples>ruby selenium.rb

Once this was done, it was time to start working on a real app. I had an 4 y/o php app in a frameset that I decided to use for testing. Motivation: if I got this stuff working, I would never have to manually slog through all the test cases on that app again. Motivation enough!

Making your tests

You can do this one of two ways: Read the API and manually code your tests in your language of choice (I was using Ruby), or, create your test cases by recording your actions in Selenium IDE (in Firefox) and then save as a .rb script. [File-->export tests as...]. The latter is the much more efficient solution: it’s quicker (do instead of write), and it’s a quicker way to learn the API. Not sure what the method for selecting a frameset is? Just select the frameset in the IDE, export, and the script will come out with that method written right in. It’s @selenium.select_frame “framename” btw.

The way I proceeded was to create (record) an individual test for each action. Because it’s a web app and not a unit test, each test has lots of smaller steps in it that create the overall story of what you are doing. Some of my examples are: create_program, create_test_one_question_no_domains, add_evaluation_to_test, add_test_to_program, etc. The same sorts of things we’d give in a list to QA. You can tell what they are by the titles, which is a common-sense way to keep things clear: descriptive naming is the name of the game. When done, I could run each test from my command line, like so:

ruby create_program.rb

and the browser would open and go through all the steps to, in the case above, create a new program in my application. However, this was repetitive: each individual test used the same setup method and only ran itself. So, I created a master suite called createProgramAndTestSuite.rb. I pasted my individual tests as methods into this class, so that in running the suite I’d be executing all the individual actions (tests) necessary to test this part of the app. Now, one line could run my 16 tests at once:

ruby createProgramAndTestSuite.rb

And, bingo: all my previously individual tests: create_program, create_test_one_question_no_domains, add_evaluation_to_test, etc. were all run in one fell swoop.

The other beauty of consolidation it allows you to take advantage of the setup method. The setup method is run before each test so you can start each individual test where it needs to start — rather than backing up to login or some equivalent “setup” (hence the name) actions. You get most of this for free: tests exported from Selenium IDE come with auto-generated setup and teardown methods that you can leave or modify. Also, note that you can customize your variable names and script conventions in the IDE by going to Options->format and choosing the language you’re working with. Putting all my tests in one suite allowed me to DRY up my repetitive setup actions in one place.

My final setup for the suite looks something like this — I added a bunch of stuff to log me in, select the main frame and gets me ready for each individual action.

def setup
@verification_errors = []
if $selenium
@selenium = $selenium
else
@selenium = Selenium::SeleneseInterpreter.new(”localhost”, 4444, “*firefox”, “http://urlofapp.com/”, 10000);
@selenium.start
@selenium.open “http://urlofapp.com/admin/admin.php”
@selenium.type “staff_email”, “secret@email.com”
@selenium.type “level_passwd”, “magic_password”
@selenium.click(”login”)
@selenium.select_window “main”
@selenium.wait_for_page_to_load “30000″
# more stuff
end
end

the teardown (again run after each test) is the default from the IDE (I did not change this).

def teardown
@selenium.stop unless $selenium
assert_equal [], @verification_errors
end

And my test methods are pretty straightforward — here’s a few lines just to give you an idea of what it looks like:

def test_create_new_program
@selenium.click "link=Programs"
@selenium.wait_for_page_to_load "30000"
@selenium.click "link=create program"
# more stuff
end

def test_create_test_one_question_no_domains
@selenium.click "link=Questionnaires"
@selenium.wait_for_page_to_load "30000"
@selenium.click "link=create questionnaire"
@selenium.wait_for_page_to_load "30000"
@selenium.click "submit"
@selenium.wait_for_page_to_load "30000"
@selenium.type "test_name", "Selenium: Test
# more stuff
end

Testing is verbose and simple. You write a lot, but, like with most tests, I found I relied on a few basic calls again and again. The typical browser work-horses are going to be select, type, click, wait, etc. I found I had to use wait_for_page_to_load a lot. But there’s an API in the contextual menu if you have the IDE installed (right click when you’re recording tests and you’ll see the showAllAvailableCommands submenu that you can insert into your recording tests). Or, you can check out the Selenium API online.

As a result of executing (and recording) all the actions once in the IDE and doing some copy-pasting and rearranging, in a few days you can have a library of test suites for your application, and voila! your app is not only running itself, but, more importantly, testing itself. You no longer have to run these by hand, you can spot errors more quickly, you can test simultaneously on multiple platforms, and you can easily pinpoint, and fix, failures. It’s a beautiful thing.

Of course, there are lots of complicated browser interactions, especially nowadays, that are not easy (or possible) to test in Selenium. Apps using drag-and-drop, for instance. And if you get into RJS or other types of javascript browser effects that we associate with “web 2.0″, you’ll need to look elsewhere. Actually, if folks know resources or have experiences in this domain, please post to the comments so we can build a resource. But, outside of these cases, Selenium RC is great for running acceptance tests. It’s easy to set up, well integrated, and, most importantly, highly addictive.

Happy testing!