Book Review: "Refactoring in Ruby"


BookDesign processEventsIntroductionsRubyTestingTips and Tricks

“Refactoring in Ruby” written by William C. Wake and Kevin Rutherford.
Published by Addison-Wesley

This is more like a “workbook” then a “how to write awesome code” book. You can download the code from github http://github.com/kevinrutherford/rrwb-code and you will find tests/specs for the exercises.

The book is arranged in three parts, The Art of Refactoring, Code Smells, and Programs to Refactor.

There are explanations of “code smells” which are one characteristic of code that could be improved. Some of them are long parameter lists, unnecessarily complex, global variable, feature envy sections, etc. One thing I find interesting is the “How did it get this way?” section. It gives some insight into the thought process and reasoning behind the smell. I think this is good, as programmers our ego may be rather miffed to hear “This code stinks” but with some reasoning, it makes the pain less and I think firms up in our minds when this happens again, to do it this other way. I always want to know why when someone says I could do such and such thing better.

In addition to the code smell examples there are three programs to refactor in the end of the book. In a conversational tone, it walks through and gives some hints on what needs refactoring. Its almost as if you had a pair programming buddy working with you and identifying in small chunks what can be improved. This is definitely something I want to work through more carefully.

What I find odd, is that not all the code smells have code examples. The inspiration for the book I think is the Martin Fowler book “Refactoring Improving the design of Existing Code” which has examples for every code smell. Maybe Ruby smells less than Java? Or those fixes are really trivial? I don’t know. Overall, this is a great book and is certainly worth the price and investment and you will be a better programmer because of it!

I Love Python: ReSTful DB CRUD dispatching using CherryPy


BookDatabasePythonServersThoughts

CherryPy has been one of my favorite Python tools for several years. It should be mentioned here that a ReSTful dispatcher could easily be written in web.py, or pylons as well, and even comes for free in the latest TurboGears implementation.

But if you’re looking for a small, easily manageable and extremely dynamic ReST dispatching solution without the heft of an entire web framework, I’m about to show you how CherryPy can help you in three different ways, depending on your model.

Assuming this mapping:

HTTP GET or HEAD = DB Read
HTTP POST = DB update
HTTP PUT = DB insert
HTTP DELETE = DB delete

Let’s also standardize on one common method across all examples, for determining the HTTP request type, and matching it to the function of the same name. Here is the full code snippet for accomplishing this task:

methods = ('OPTIONS','GET','HEAD','POST',
'PUT','DELETE','TRACE','CONNECT')

if cherrypy.request.method not in self.methods:
    raise cherrypy.HTTPError(400,'Bad Request')

# If request method is HEAD, return the page handler
# for GET, and let CherryPy take care of dropping
# the response body
method = cherrypy.request.method

if cherrypy.request.method == "HEAD":
    method = "GET"

http_method = getattr(self,method)

#print "HTTP Method: %s" % method

result=(http_method)(args,kwargs)

In our examples, we’re going to shorten this to:

http_method = getattr(self.m,cherrypy.request.method)
return (http_method)(args,kwargs)

All of this essentially determines how HTTP was called (GET/PUT/POST/DELETE), and calls the method in a class which exactly matches this name (self.GET(), self.PUT(), etc)
When you see this code, know that it’s just the HTTP method resolving code.

Now for the fun. Let’s look at the dispatcher options we have.

Way 1: A hard-coded URL pointing to fixed resources:

CherryPy can be used in a manner similar to this to establish a fixed URL, and corresponding resources, driven from predefined classes instantiated in the ‘root’ hierarchy:

import cherrypy

class ReSTPaths1:
	@cherrypy.expose
	def index(self):
		http_method = getattr(self,cherrypy.request.method)
		return (http_method)()

	def GET(self):
		return "In GET 1.."

class ReSTPaths2:
	@cherrypy.expose
	def index(self):
		http_method = getattr(self,cherrypy.request.method)
		return (http_method)()

	def GET(self):
		return "In GET 2.."

class ReSTPaths3:
	@cherrypy.expose
	def index(self,client_id=None):
		http_method = getattr(self,cherrypy.request.method)
		return (http_method)(client_id)

	def GET(self,client_id=None):
		return "IN Get 3, your client_id is %s\n" % (client_id)


cherrypy.server.socket_port=8081

root=ReSTPaths1()
root.client = ReSTPaths2()
root.client.address = ReSTPaths3()
cherrypy.quickstart(root)

Once this is running, the URL to invoke it looks like this:

http://localhost:8081/

http://localhost:8081/client/

http://localhost:8081/client/address/

http://localhost:8081/client/address/?client_id=34567

http://localhost:8081/client/address/34567

Output looks something like this:

In GET 1..
In GET 2..
IN Get 3, your client_id is None

If you’re new to CherryPy or Python in general, I’ll reiterate for you how we are calling the GET method in our class.

When we issue this request, we’re issuing what HTTP calls a GET request:

http://localhost:8081/

The CherryPy service above, listening on port 8081, calls the index() method on the root class. The root class was set to:

root=ReSTPaths1()

at the bottom of that file. The index() method from the ReSTPaths1 Class looks like this, at the top of that file:

	def index(self):
		http_method = getattr(self,cherrypy.request.method)
		return (http_method)()

If we were to insert a print cherrypy.request.method statement before the return, we would see it set to “GET”.

getattr simply says: “get me the function name in self, matching the string “GET”.
it returns a reference to self.GET(), which is set directly below the index:

	def GET(self):
		return "In GET 1.."

Notice that the index() method has a @cherrrypy.expose decorator above it. This makes the index method callable by the public. The GET method does not have it, which means we could never invoke the GET method by typing:

http://localhost:8081/GET

If you try this, you’ll get a 404 Not Found error, because it’s not visible through the CherryPy interface.

GET() has to be invoked through index(), which means GET can only be called if an HTTP GET request is issued. If we posted form data to this same URL from, say, a form entry asking people for data input, we would need to add a POST method to this ReSTPaths1() class, to receive the POST data entered in the form fields.

Now back to our example:

In this example, no part of the URL or associated resources are dynamic, in either initialization or run time. This is fine, and suits the needs of most ReSTful CRUD interfaces.

Way 2: URL paths and associated components dynamically set once, upon dispatcher init/startup:

Now let’s say we want to determine the contents of the root, and therefore the URLs and associated resources for our ReSTful interface, dynamically during initialization/startup.

We can assign the root setting by using a Python metaclass to generate classes in our CherryPy startup code, and set the root components to each generated class. This goes beyond the average needs for CRUD access, but it’s such a nice implementation that I must show it off:

import cherrypy

class MetaCRUD(type):
	@cherrypy.expose
	def index(cls):
		http_method = getattr(cls,cherrypy.request.method)
		return (http_method)()

	def GET(cls): return "In class ", cls.__name__, ', received a GET request.'

	def PUT(cls): return "In class ", cls.__name__, ', received a PUT request.'

	def POST(cls): return "In class ", cls.__name__, ', received a POST request.'

	def DELETE(cls): return "In class ", cls.__name__, ', received a DELETE request.'


baseCRUD = MetaCRUD('baseCRUD',(),{})
root = baseCRUD

dynamic_class = {}

for d in ['legacy_dbi','new_dbi','some_other_dbi']:
	dynamic_class[d] = MetaCRUD(d,(),{})
	setattr(root,d,dynamic_class[d])

cherrypy.server.socket_port=8081
cherrypy.quickstart(root)

Here we’re using a metaclass, with CherryPy exposed methods, to generate a dictionary of dynamic classes. We set the root.classname = the_new_class by using the setattr() method.

After initialization, URL components and resources are fixed in this model. But wow, the awesome power we have during initialization, in 28 lines really rocks. I wrote this in 30 minutes, and realized again why I am so head-over-heels in love with this language.

When we hit these URLs:


http://localhost:8081/


http://localhost:8081/legacy_dbi/


http://localhost:8081/new_dbi/


http://localhost:8081/some_other_dbi/

We see this output:

In class baseCRUD, received a GET request.
In class legacy_dbi, received a GET request.
In class new_dbi, received a GET request.
In class some_other_dbi, received a GET request.

Let’s issue a POST request via curl, on the command line. The response is returned:

[gloriajw@g-monster ~]$ curl http://localhost:8081/some_other_dbi/ -d ""
In class some_other_dbi, received a POST request.

This model could be used for, say, reading the contents of the Postgres template1 databases list or the mysql ‘show databases’ command, and auto-generating a ReSTful CRUD interface for each. Access of each resources can be controlled via HTTP Auth methods. This is a great solution to providing, and restricting, legacy database access for new processes through a standard interface.

Way 3: Live, ever-dynamic determination of URL and associated component:

Some ReSTful URL models may need to be ‘run-time dynamic’, especially in the case where databases are dynamically created, and the associated resources per new database could vary. There is a simple example of a dynamic URL and resource model:

import cherrypy
import pprint

class ReSTPaths:
	@cherrypy.expose
	def __init__(self):
		pass

	@cherrypy.expose
	def client(self,*args,**kwargs):
		return "Your HTTP method was %s. Your args are: %s and your kwargs are: %s\n" \
		% (cherrypy.request.method, pprint.pformat(args), pprint.pformat(kwargs))

	@cherrypy.expose
	def address(self,*args,**kwargs):
		return "Your HTTP method was %s. Your args are: %s and your kwargs are: %s\n" \
		% (cherrypy.request.method, pprint.pformat(args), pprint.pformat(kwargs))

cherrypy.quickstart(ReSTPaths())

This allows for dynamic URLs such as:

http://localhost:8080/client/address/34567

http://localhost:8080/client/address?client_id=34567

http://localhost:8080/address/client?client_id=34567

http://localhost:8080/address/client/34567

http://localhost:8080/address/anything/anything_else

The output from this code looks like this:

Your HTTP method was GET. Your args are: ('address', '34567') and your kwargs are: {}
Your HTTP method was GET. Your args are: ('address',) and your kwargs are: {'client_id': '34567'}
Your HTTP method was GET. Your args are: ('client',) and your kwargs are: {'client_id': '34567'}
Your HTTP method was GET. Your args are: ('client', '34567') and your kwargs are: {}
Your HTTP method was GET. Your args are: ('anything', 'anything_else') and your kwargs are: {}

Notice that we only have keyword args (kwargs) when we pass a named parameter, such as client_id=34567

Let’s try a POST request from curl, on the command line:

[gloriajw@g-monster ~]$ curl -d "something_else=whatever_i_want" http://localhost:8080/address/anything/anything_else
Your HTTP method was POST. Your args are: ('anything', 'anything_else') and your kwargs are: {'something_else': 'whatever_i_want'}

In this code, the sky is the limit. You can place whatever code you like in these methods, dynamically creating classes and resources as needed, letting them only persist until the result is returned. This may add some inefficiency, but in exchange offer more secure network resources.

Code is attached, Enjoy!

Gloria

http://www.devchix.com/wp-content/uploads/2009/04/restfixedargs.py

http://www.devchix.com/wp-content/uploads/2009/04/restmeta.py

http://www.devchix.com/wp-content/uploads/2009/04/restvarargs.py

The Creative Commons license, and change.gov: Our new president gets it!


BookIntroductionsPythonThoughts

I was really excited and moved to see this:

http://change.gov/newsroom/entry/towards_a_21st_century_government/

This copyright license affects most content on the website change.gov, guaranteeing transparency and the feedom to redistribute and discuss it’s content.

All I can say is, wow. This president not only believes in transparency, but understands how to implement it using existing tools and conventions. He understands the spirit of Open Source software development, and everything that has come about since this movement began. Imagine an “Open Government”, genuinely implemented and influenced by the people, and it’s hard not to want to get involved.

The change.gov web site has a very nice, simple form, requesting your ideas and suggestions. They already have proposals for nationwide fiber optic broadband, automation and transparency in the patent process, and automation of health insurance data to bring healthcare costs down. There are many more places where technology can automate away inefficient and error prone processes.

We here on DevChix all know how technology has changed our lives. Let them know how it can change a nation. Submit your ideas and suggestions, and let’s help educate and influence this new administration.

Gloria

From Python 2.6 to PHP 5.2: A circuitous journey


BookIntroductionsPeoplePHPPythonThoughts

When I started heavily using PHP 5.2 (not by choice, I’ll admit), I was impressed, but I suffered from some incorrect assumptions about what PHP5 is and is not capable of doing. The good news is that it is more object oriented than it’s predecessor, but has some caveats to consider. Here are some things to be aware of when switching from a pure OO language to PHP5:

1: A nonexistent PHP array key generates no error or warning. When trying to iterate over a nonexistent array key, a warning occurs. In other languages, both of these conditions throw an exception.

Try this code for example:

<?
$dictionary=array('one'=>'got one','two'=>'have two','four'=>'missing three?');
foreach (array_keys($dictionary) as $key)
{
	print "Key is:".$key.", value is:".$dictionary[$key]."\n";
}
print "Try undefined key three, no warning occurs:".$dictionary['three']."\n";
foreach ($dictionary['three'] as $value)
{
	print "Now we're iterating over a nonexistent key:";
	print "Key is: three, value is:".$dictionary['three']."\n";
}
?>

Running it results in this output:

php test.php
Key is:one, value is:got one
Key is:two, value is:have two
Key is:four, value is:missing three?
Try undefined key three, no warning occurs:

Warning: Invalid argument supplied for foreach() in /root/test.php on line 8

If it is vital to me to make sure I am aware of missing keys, I only have two choices. If I need a proactive solution, I have to use the array_key_exists() function to do existence checking before use. If I want a reactive solution, I write a log scanner, to pick up on these warnings. In every other OO language I have used, an exception was thrown for this condition, and my exception handling determined if the error was vital enough to have to exit immediately or not. This seems like a more efficient way to handle this condition. I would imaging PHP5 does not do this because of it’s need to be backward compatible with PHP4, but this is a guess.

It would be wonderful to have a -OO flag for PHP, which gives you the option to run PHP and expect more standard, stricter OO behavior in these instances.

2: Warnings cannot be “caught” like exceptions. Exceptions and warnings are distinctly separate beasts, and never the twain shall meet. Fine, I thought, maybe I could detect warnings similar to how we detect errors. But it seems like warnings cannot be detected when they happen. There is no PHP code I know of which can check if a warning had occurred in runtime. I tried to detect it using array error_get_last() but to no avail. if you know how, post your trick here.

3: In PHP, ‘true’ evaluates to an integer ’1′. To get the boolean ‘true’ value from a ‘true’ statement, one needs to var_export() a true statement. Similarly, or maybe not, ‘false’ evaluates to no output. Here is an example:

<?
print "\nThe raw value of a true statement in PHP:".true;
print "\nThe raw value of a false statement in PHP:".false;
print "\nThe exported value of a true statement in PHP:".var_export(true,true);
print "\nThe exported value of a false statement in PHP:".var_export(false,true);
print "\n";
?>

And the output:

The raw value of a true statement in PHP:1
The raw value of a false statement in PHP:
The exported value of a true statement in PHP:true
The exported value of a false statement in PHP:false

This may not be noticeable to you in a standard expression. But if you’re doing funky stuff, like using the evaluated expression values as key references into the dictionary of a decision tree, for example, 1 does not equal ‘true’, and the difference matters quite a bit.

4: Long running processes with recursive circular references (such as Doctrine code) run out of memory. This is documented in many places, and the free() function works sometimes. A fix is coming in PHP 5.3. The foolproof solution for my code in production today (youch!) is to periodically restart the daemon. If you’re cringing right now, know that you’re not cringing alone.

There may be a part II to this article. Feel free to add your own PHP5 observations.

Gloria

Book Review, "Pro Active Record"


BookReviewsRuby

Published by Apress
By Kevin Marshall, Chad Pytel, Jon Yurek
Book Info
Sample Chapter: Ch. 01 – Introducing Active Record
Table of Contents

Years ago when I was in PHP Land (now I travel quite a bit more! haha), I strugged for months with how to write a good ORM . It was tough, because I wanted to abstract the “boring logic” of retrieving records from a database without writing SQL but still remain flexible enough. I never really came up with a good model. I used the DAO from “extreme php” library which I think was a knock off from java. It was ok, but I still didn’t feel like I had “arrived”.

When I discovered Ruby on Rails, I found ActiveRecord. Ahh HA! Finally, this is what I was looking for. At first I thought it was part of Rails, but its not. Its a standalone library and you can use it with straight up ruby scripts.

I got a review copy of “Pro Active Record” some time ago and read it some when I got it, then some later, and now I am going to officially write up a review!

If you do anything with Active Record, get this book. The things that are briefly mentioned in most Rails books are described in detail in this book.

Chapter 1 – Introducing Active Record

Most of the time, the first chapters of a book are boring to me. I don’t need another “History of the Internet” or how “HTML was developed” … blah blah. But this one, the story is only 1 page. And it actually has some introductory scripts on using Active Record, so you can see right away how it works. It also explains the benefits of MVC and why ORMs are good. Some people still don’t get it!

Chapter 2 – Active Record and SQL

This chapter helps you translate the “sql in your head” to how to write it with Active Record. I’ve used Active Record so much that now I have forgotten most of my SQL, which is kind of embarrassing. :) I now find writing sql tedious and boring! I would have actually called this chapter “Demystifying Active Record” since it explains why all the dynamic finders work. You’ll also find transactions and locking explained here.

Chapter 3 – Setting up Your Database

Migrations! The Awesome Thing that can turn into a nightmare for large rails projects with multiple developers…. definitely have to decide on some best practices with your team on this one. The chapter has only one thing to say about this — assume any checked in migration has already been run by your team and the migration should not be edited and checked back in! You’ll have to make another migration file with your changes.

[tip]
Nola's Note: When you make a migration, test it both UP and DOWN!! Here's what I do --
write a migratiion
rake db:migrate (go up to the version with new code)
rake db:migrate VERSION=n-1, (go to version before the latest)
rake db:migrate (back to lastest)
rake db:migrate VERSION=0 (back to blank db)
rake db:migrate (back to latest)
[/tip]

Just to be sure its all good — even on a new database!

Chapter 4 – Core Features of Active Record

Now is the fun stuff – Callbacks. This is magic. This makes Active Record so flexible, and is one thing I could never figure out how to do with my PHP ORMs. I use call backs to set defaults for fields. If its just a straight default, then I set it in the database but if I need to make a decision, (if this field then this field..) then I can use it in a callback.

Associations – at first this is very confusing! I don’t know how many times I got “has_many” and “belongs_to” mixed around in the beginning.

Validations – Awesome. I had to do some ruby code without a database and I really really really missed the validations. It took me like 5x longer than it should! Understanding all of these validation methods will make your life so much more enjoyable. I really really hate doing boring, repetitive stuff…it seems so wasteful to me.

Chapter 5 – Bonus Features

Everybody likes a bonus and this isn’t even the last chapter of the book.

Java people will like the Active Record Observers — seems a little AOP to me (aspect orienteted programming) and something I probably have neglected to use to their fullest extent.

Acting up — Learn how to “save time” with the “acts_as” magic: List, Tree, Nested Sets. If your data needs these structures, you got it made. I can imaging how much longer it would take to write this stuff in perl or php.

Composed of – I haven’t used this, but this looks like a good way to make sensible objects out of database tables. There is quite a bit of explanation and examples of this, it will come in handy.

There are a few other in depth explanations of things, such as method_missing which is how alot of the magic happens. Rock on.

Chapter 6 – Active Record Testing and Debugging

Ahh yes, Testing. My favorite subject. My friends who know how much I love testing say I am sick. I must have an inner need to PROVE I am right or something, haha.

The chapter goes into depth about using test_unit with Active Record, sadly no RSpec. But, it does go into all the error messages that Active Record throws so you can write good try/catch blocks and make very exact error messages (probably best logged for the admin rather then displayed to the user!)

Chapter 7 – Working with Legacy Schema

Here’s how you work with that old database that just won’t die… or that management won’t let you totally redo. Active Record follows some of the principles of Rails “convention over configuration” … relying on table and column naming conventions to figure out how to build your object….but still giving you a way out if you want your tables singular and your primary id field called “myawesomeid” instead of “id”

I’ve used some of these things on an older database and it was possible! Not too bad if thats what you have to work with.

[soapbox]
Some people find this annoying "oh gosh! my library can't make decisions for me! OMG! That sucks" .. to that I say, "Umm ok. But if you follow these conventions then I can come into your project and know exactly what is going on" ... like with web standards, we all harp on how IE and FF do things differently, yet people want to bellyache about Active Record preferring to have plural names and id field called "id". Right.

Follow the dang convention and find something worth complaining about to complain about. :)
[/soapbox]


Chapter 8 – Active Record and The Real World

This chapter goes into depth about the library and encourages you to go read the Active Record code. Always a good idea to know what it is you are using :) I’ve actually learned ruby better by reading source code. The chapter walks you through basic structure of the files. Very cool.

[soapbox]
I used to work at a place that didn't like any "outside code" because they were afraid "OMG ... it will send our passwords to Russia!" ... ok, well I am not an idiot. I read over any code that I use that I didn't write. I look at the tests to see if I am using it right. I even RUN the tests so I can be sure its working as advertised.
[/soapbox]

Alternatives to Active Record – with EXAMPLES! If something about Active Record doesn’t set too well with you, take a look at the alternatives. Sometimes I look at the alternatives and decide that the first wasn’t so bad after all. You’ll find examples of DBI, Og, ActiveRelation.

Finally a section on Q and A finishes up this book. The Appendix has a complete reference of ActiveRecord methods to make this book a well rounded reference, tips, documentation and very handy to have at your desk!