The first time I was played with Selenium (the IDE) was a jaw-droppping moment. It was as if little sprites had taken over my computer and were gleefully doing my job. Sitting back and watching a form being filled in, a button selected, links linked — it was an incredible, mafia-man type experience: things were just getting done, and fast. The little guy rushed through my tasks in seconds, cheerfully, leaving green check-marks in his midst, while I sat back and kind of stared, the way you’re supposed to stare at a national monument. These days there are too many hot-dog stands and camera-d tourists at national monuments to merit my awe, but it’s safe to say that my little browser sprite was stepping up to the plate nicely. One could get used to this, I thought. Me, the sprite, and Firefox: what could be wrong in the world?

This didn’t last (amazing how quickly one gets spoiled). The problem with Selenium IDE is that the tests only play back in Firefox. I love Firefox, but the whole gist of my job is that the stuff we do needs to work in all browsers, on all machines (ok, most on most, but you get the idea). At any rate, I went looking for how I could extend the power of Selenium so I could test my web apps cross browser.

There are two approaches: Selenium on Rails (not covered in this article but you can find info here), and Selenium RC (Remote Control). After poking around a bit it seemed that RC was the solution for me. I was testing old apps, not Rails ones, so the Rails solution was overkill — and I had heard a general buzz that RC is a more robust, ‘best-practice’ solution. Whether that’s commonly held wisdom I can’t say (and if anyone wants to do a comparison, I’d love to hear), but it fit my requirements so I dove in.

If you’re not familiar with what Selenium RC lets you do, here: it lets you write tests in most languages (java, perl, python, ruby, php (NOTE: at the time of my testing I could not locate 0.9.1, with php support. It may be available now)) and then run them from the command line. Assuming the selenium server is running (see steps below), a proxy browser is fired up and your script is executed. In short, a browser opens and is interacted with. Each test is run and you’re fed a success or error message. It’s the same process you see when you run tests back using the IDE: your application being lovingly and quickly manhandled by invisible testing men (though based on the current online discussions I should probably make it clear that the gender of the magic sprites is unknown and irrelevant — and that they do not want to have sex with you, even if they are running in heels :))!

To the docs

The openqa docs are great resources. My one caveat is I would suggest a larger font and a clearer navigation scheme. It’s all there though, and it works, so read carefully, try the examples, and be patient.

From here on out I’m going to transcribe the steps I took to get up and running. This is the order I did things, so hopefully this is helpful as a list of steps and guidelines:

STEPS To set up Selenium RC on Windows

1. Read the openQA tutorial:
2. Download selenium RC here: [NOTE: 0.9.1 says it has php but I couldn’t find it anywhere. I got 0.9.0]
3. Extract it using winzip
4. Put in your c dir (or wherever you keep programs), like so: c:\selenium-remote-control-0.9.0.. I renamed it to selenium, so mine reads c:\selenium. NOTE: make sure you have right java version by typing java –version in your cmd shell. You must have at least 1.5 (1.4???)]. If you don’t, upgrade your java.
5. Put selenium.rb in your ruby lib dir so it will be accessible to ruby using require ’selenium’. If you don’t do this you’ll have to run your scripts from w/in the same directory as selenium.rb: you don’t wanna do that. For me I put it in C:\ruby\lib\ruby\1.8\selenium.rb
6. Start Server. You need to run from cmd line from WITHIN your selenium\server dir [ in my case c:\selenium\server] or feed it the path. [NOTE jar is Java Archive: To run an application packaged as a JAR java -jar app.jar]

java -jar selenium-server.jar –interactive.

At this point you can execute commands on the command line or you can execute a standalone script. There are commands in the tutorial that you can follow along with. To do the latter (execute a program), simply navigate to a test (there’s one in ruby/samples called selenium.rb ) and run it. The one that shipped with my ruby opens google and searches for the string “Hello World”.

C:ruby\\samples>ruby selenium.rb

Once this was done, it was time to start working on a real app. I had an 4 y/o php app in a frameset that I decided to use for testing. Motivation: if I got this stuff working, I would never have to manually slog through all the test cases on that app again. Motivation enough!

Making your tests

You can do this one of two ways: Read the API and manually code your tests in your language of choice (I was using Ruby), or, create your test cases by recording your actions in Selenium IDE (in Firefox) and then save as a .rb script. [File-->export tests as...]. The latter is the much more efficient solution: it’s quicker (do instead of write), and it’s a quicker way to learn the API. Not sure what the method for selecting a frameset is? Just select the frameset in the IDE, export, and the script will come out with that method written right in. It’s @selenium.select_frame “framename” btw.

The way I proceeded was to create (record) an individual test for each action. Because it’s a web app and not a unit test, each test has lots of smaller steps in it that create the overall story of what you are doing. Some of my examples are: create_program, create_test_one_question_no_domains, add_evaluation_to_test, add_test_to_program, etc. The same sorts of things we’d give in a list to QA. You can tell what they are by the titles, which is a common-sense way to keep things clear: descriptive naming is the name of the game. When done, I could run each test from my command line, like so:

ruby create_program.rb

and the browser would open and go through all the steps to, in the case above, create a new program in my application. However, this was repetitive: each individual test used the same setup method and only ran itself. So, I created a master suite called createProgramAndTestSuite.rb. I pasted my individual tests as methods into this class, so that in running the suite I’d be executing all the individual actions (tests) necessary to test this part of the app. Now, one line could run my 16 tests at once:

ruby createProgramAndTestSuite.rb

And, bingo: all my previously individual tests: create_program, create_test_one_question_no_domains, add_evaluation_to_test, etc. were all run in one fell swoop.

The other beauty of consolidation it allows you to take advantage of the setup method. The setup method is run before each test so you can start each individual test where it needs to start — rather than backing up to login or some equivalent “setup” (hence the name) actions. You get most of this for free: tests exported from Selenium IDE come with auto-generated setup and teardown methods that you can leave or modify. Also, note that you can customize your variable names and script conventions in the IDE by going to Options->format and choosing the language you’re working with. Putting all my tests in one suite allowed me to DRY up my repetitive setup actions in one place.

My final setup for the suite looks something like this — I added a bunch of stuff to log me in, select the main frame and gets me ready for each individual action.

def setup
@verification_errors = []
if $selenium
@selenium = $selenium
else
@selenium = Selenium::SeleneseInterpreter.new(”localhost”, 4444, “*firefox”, “http://urlofapp.com/”, 10000);
@selenium.start
@selenium.open “http://urlofapp.com/admin/admin.php”
@selenium.type “staff_email”, “secret@email.com”
@selenium.type “level_passwd”, “magic_password”
@selenium.click(”login”)
@selenium.select_window “main”
@selenium.wait_for_page_to_load “30000″
# more stuff
end
end

the teardown (again run after each test) is the default from the IDE (I did not change this).

def teardown
@selenium.stop unless $selenium
assert_equal [], @verification_errors
end

And my test methods are pretty straightforward — here’s a few lines just to give you an idea of what it looks like:

def test_create_new_program
@selenium.click "link=Programs"
@selenium.wait_for_page_to_load "30000"
@selenium.click "link=create program"
# more stuff
end

def test_create_test_one_question_no_domains
@selenium.click "link=Questionnaires"
@selenium.wait_for_page_to_load "30000"
@selenium.click "link=create questionnaire"
@selenium.wait_for_page_to_load "30000"
@selenium.click "submit"
@selenium.wait_for_page_to_load "30000"
@selenium.type "test_name", "Selenium: Test
# more stuff
end

Testing is verbose and simple. You write a lot, but, like with most tests, I found I relied on a few basic calls again and again. The typical browser work-horses are going to be select, type, click, wait, etc. I found I had to use wait_for_page_to_load a lot. But there’s an API in the contextual menu if you have the IDE installed (right click when you’re recording tests and you’ll see the showAllAvailableCommands submenu that you can insert into your recording tests). Or, you can check out the Selenium API online.

As a result of executing (and recording) all the actions once in the IDE and doing some copy-pasting and rearranging, in a few days you can have a library of test suites for your application, and voila! your app is not only running itself, but, more importantly, testing itself. You no longer have to run these by hand, you can spot errors more quickly, you can test simultaneously on multiple platforms, and you can easily pinpoint, and fix, failures. It’s a beautiful thing.

Of course, there are lots of complicated browser interactions, especially nowadays, that are not easy (or possible) to test in Selenium. Apps using drag-and-drop, for instance. And if you get into RJS or other types of javascript browser effects that we associate with “web 2.0″, you’ll need to look elsewhere. Actually, if folks know resources or have experiences in this domain, please post to the comments so we can build a resource. But, outside of these cases, Selenium RC is great for running acceptance tests. It’s easy to set up, well integrated, and, most importantly, highly addictive.

Happy testing!

On my current project we are building a web based application that allows the user to sign up for a service. The signup process is a series of pages for gathering a variety of information (about 8 or so pages). Now… I am a bit on the lazy side so a while back I started using the Selenium IDE to record scripts for moving me to the location in the app that I was interested in. The other day one of the non-techinical folk on the team asked me how to use the Selenium IDE and so I decided to write up a blog post about it. There are a couple of posts out there already in addition to the information on OpenQA but I figure one more post won’t hurt. It really is a very good tool for all types of people from sys admins, PM’s, Business Analysts, developers, to end users doing beta testing. I decided to break up the information based on questions I was asked when showing a couple of folks how to use the app.

What is Selenium?
OpenQA says: Selenium is a UI test tool for web applications. Selenium tests run directly in a browser, just as real users do. Those tests can run in Internet Explorer, Mozilla and Firefox on Windows, Linux, and Macintosh. No other test tool covers such a wide array of platforms.

What is Selenium IDE?
OpenQA says: Selenium IDE is an integrated development environment (thats what IDE stands for for those who don’t know) for Selenium tests. It is implemented as a Firefox extension, and allows you to record, edit, and debug tests. Selenium IDE includes the entire Selenium Core, allowing you to easily and quickly record and play back tests in the actual environment that they will run. Selenium IDE is not only a recording tool: it is a complete IDE. You can choose to use its recording capability, or you may edit your scripts by hand. With autocomplete support and the ability to move commands around quickly, Selenium IDE is the ideal environment for creating Selenium tests no matter what style of tests you prefer.

Installation:
A firefox extension is an installable enhancement to the browser’s functionality and add features to the application or allows existing features to be modified. Since Selenium IDE is a Firefox extension you get it by downloading and installing the firefox extension. Oh and installing extensions does require you to restart firefox so just keep that in mind before you go any further.
1. Go to OpenQA
2. Click the firefox extensions link under the download bullet of the latest version. This will pop up a box and a button should appear briefly that says “Install Now”. Click on that.
3. This will pop up a box and install the extension. Once its finished click the button on the bottom right corner that says “Restart Firefox”.

How do I run/use the Selenium IDE?
The folks over at OpenQA already did a nice movie so you can go watch that to get the basics.

How do I run a script that I recorded in Firefox against Internet Explorer?
You would need to install Selenium Core or Selenium RC in order to be able to do this. Selenium Core has to be run on the same webserver as the application you are trying to test. So this probably won’t work for non-developer types who don’t have access to what is put on the webserver. If that is the case then you might want to look into using Selenium RC.For information on using and setting up Selenium Core take a look Here and for using Selenium RC look Here.

What is the difference between run, walk, and step?
The only difference between run and walk is that run is faster. They behave exactly the same way. Step on the other requires you to actually push the blue downward right angle arrow key to continue to the next step. When you use Run and Walk with breakpoints the pause button will turn into a pause/resume button. In order to keep going to the next breakpoint just hit the pause/resume button. If you hit play again from a breakpoint or from any command it will start the script over from the beginning.

How do I use breakpoints?
Sometimes when you are running your scripts something breaks and you might want to see the state of the application just before that break or maybe the script exits before you get a chance to see the error on the screen in these cases breakpoints are awesome. Simply chose the line in your script where you want it to stop and put a break point. You do this by right clicking and selecting “Toggle Breakpoint”. Now when you hit the play button it will stop at that command. Additionally you can set a break point and then step through the application from that point on using the step feature.

Can you give an example of when you would want to save the scripts in a different language.
Sometimes QA people or Buisness Analysts or even end users may want to submit the test they used to create the error back to the developers, in this case the developers may like to have the test in the language which they are writing the application in. It may also be the case that the developers themselves change the language based on what they were given or who they may be giving the tests to. In order to change the language the test is output in go to the Selenium IDE tool bar select Options -> Format and then select the language you want the test in.

What is the difference between deleteCookie and deleteCookieAndWait?
In a nut shell don’t use the “andWait” commands. The “andWait” commands should be gone as of the latest version (they are still there but they all do the same things as their couterpart commands i.e deleteCookieAndWait does exactly what deleteCookie does) but for those still on the old version here is the explanation. Typically anytime you see the “andWait” or “withWait” piece tacked on to the end of a command it simply means that you want your script to stop executing any additional commands until it gets back a response saying the command has finished.

How would you handle situations where you would need to change a piece of data entered each time you run the script?
To be honest I haven’t quite figured out how to do this. I am going to try to play around with it a bit and see if I can figure something out. As soon as I do I’ll post an answer to this.

What does the log at the bottom tell me and how do I interpret what it is saying?
From the point of view of a non-technical person the log basically just tells you information about what has happened. Maybe you clicked the back button and the ide didn’t recognize that click and you then clicked some image. In this case you would get an error stating it could not find the element you told it to find. As it turns out the script isn’t on the right page so of course it wouldn’t find it. There are lots of reasons why you might get an error and usually if you step through your script you will find that you are not on the page you expect to be when the command was trying to be executed or some other trivial thing has happened. The log also has general information in the info tag that tells you which commands have been executed etc. All this maybe helpful when trying to figure out what has gone wrong.

Gotchas?
The most common gotchas are around timing and caching/session issues. There are a few commands that you can insert into your scripts in order to help you get around this. For instance if you have a page which has some ajax calls then you might have an issue where your script trys to do something before the call has actually finished. In this case you can insert the command “pause” into the script.
In another situation you may want to make sure that you are starting with a fresh session i.e. no cookies. The command “deleteCookie” can be used in order to make sure this happens.
A full list of Selenium commands can be found Here

So now you guys and girls go play around with it and post back your questions here. I’ll do my best to answer them or at least point you in the right direction.

Next post: Selenium On Rails!

Also check out these posts for information on Selenium IDE:

http://dynamitemap.com/selenium/ - very good introduction

http://ajaxian.com/archives/selenium-ide-07-released - some good questions and answers in the comments section