A colleague came to me about a month ago and asked: “Have you seen this?” “No, but it looks great, if it really works the way it claims.” I replied. We were ogling this link, with expressions of “Ooo”, “Ahhh”, and “Wow”:
He installed it on his XP desktop, I installed it on my Fedora Core 6 Desktop, and on our Linux test server with eight CPUs. The installation in an existing Python distribution took five minutes. Reading the daemon instructions and starting Parallel Python daemons on all of these machines took another three minutes.
We modified a test example, a distributed sum of primes, to run for many minutes, giving it a huge upper limit. On the Linux machines, running the top command, then pressing the number 1 reveals the activity on all CPUs. We started the modified test example, closely watching top and the task table on XP. We saw an even distribution of jobs across machines, and a consistently even balance amongst all CPUs. Parallel Python is written so well that it balances jobs according to a machine’s capacity and speed.
I decided to integrate it into my web scraper project. I need to spawn jobs to scrape different pages in parallel. The problem I had was mapping my OO model to the existing procedural examples. I have a scraper class, and I want Parallel Python to run it’s run() method, and see it’s self variables.
Looking for help, I find a help forum on the Parallel Python web site. Not a bad start. I post my question, and get a response within 24 to 48 hours from the most helpful geek I have ever dealt with.
Vitalii Vanovschi, the author of the examples, and probably also the author of most of the Parallel Python code, offers damned near flawless, quick, and ego-free assistance with this software. To answer my question, he rewrote an example ‘on the fly’, to show how classes can be used in Parallel Python. His responses to everyone are consistently respectful, concise, and helpful, with a strong undertone of of the genuine love for the Python language and it’s capabilities. This is the best tech support I have ever gotten for any product, EVER.
To top it off, there is a link on his main page, asking people to talk about what drew people to Python. This adds a ‘Ruby-like’ social element to his web site which is still unfortunately rare in the Python realm. I was thrilled to find some social aspect to this web site and it’s subject matter. It is refreshing to be able to talk about the technical and social aspects of the language and the tools, simultaneously, with the same crowd of people. Rubyists seem to understand this, and Vitalii’s hope to bring this aspect into the Python world does not go unnoticed.
Vitalli has brought so many good things together in one application, on one web site. He has created a simple, easy-to-navigate web interface with related forums, great documentation and examples. The software itself is robust, well-written and so easy to use that one can integrate it into an existing project very quickly.
This project is an excellent example of good software, good presentation, good tech support, and good social support. I think Python programmers can learn a lot from this example. My hopes are that more Python project developers make better attempts at the social aspect of their projects, asking people to share their experiences with the project and the related languages in an ego-free environment. Maybe user communities will form from these environments. And maybe, just maybe, Python programmers can one day experience the sense of community that Ruby developers already enjoy.
Vitalli, kudos, and thank you for such a great, well-rounded project. Thank you for setting the the technical, social and presentation precedents. May all new projects follow this model.