Sep 13, 2009

Fabric factory

This is a project I have been working on recently after I spent a day to look at the existing solution to run periodically a test suite. Most of the project I look at were either difficult to setup or require to learn yet another specific domain specific language or had dependency on a larger software stack.

As a reaction to this situation I have decided to see if I could write something simple that achieves gracefully this task. I also try to make it as easy to setup as possible.

I have decided to use cpython as platform, django as web framework for the server and Fabric as library to automate the task execution.

The result of this mix can be found on bitbucket in a project called Fabric Factory. This will eventually become a complete Job Server that could be used to distribute any kind of task scripted in Fabric.


This assumes that python is installed on your computer and that you have an internet conection.

You can download the code using mercurial:
* hg clone
A fabfile will help you to quickly setup your environment.
* fab quickstart

Note : In order to run the command above you will need the latest version of Fabric the following command will take care of this:

pip install -e git://


"quickstart" has created a virtualenv which must be actived before you continue.

. ve/bin/activate

Once the virtualenv is activated you can go inside "src/project". This is a django project so from there you can do several things :

* create an sqlite db : python syncdb
* run the server : python runserver
* run the test suite : python test

The main app of this django project called fabric factory is called "factory".

Once the server is started and that you have created some "Build" in django's admin interface you can open a new terminal and run the client side of the project:

cd src/worker
python --daemon=start
python --daemon=stop

Use case

Now that you have understood the layout of the project. Let us see how we can achieve something useful with it.

We are going to create a Build that will :
* download the Fabric Factory
* setup the environement
* run the test suite
* Report the result

1> Direct your browser to that url and key in the username/password you have chosen for your administrator.
2> Add the fabfile recipe store in docs and call it "fabric factory use case"
3> Replace by in sites :
4> Create a Build that will download setup and run the test here :

The fabfile recipe that we have downloaded earlier contains a task called : 'download_setup_and_test' This task as been writen to do what we want.

We are now going to configure the client to run this task. However before doing this let us see how the server publish the tasks that need to be executed. Point your browser to this url :

5> Open a new terminal and move into the the worker directory then you can start the worker in daemon mode :

. ve/bin/activate
cd src/worker
python --daemon=start

If you want to look at what is happening in the background you can watch at the log file in realtime

tail -f worker.log

6> You can see the status of the build here : Keep in mind this task is pretty long to run because we are downloading all the dependancies (django, fabric). It took me almost 5 minutes to execute this task and to see the result in the admin.


This project which is still very immature seems to prove that this stack is well suited to build this kind of tool. I would be glad to hear your experience about this kind of tool. Please do not hesitate to copy, fork, contribute to this project to make sure that soon we have a simple easy to setup yet flexible tool to distribute tasks.
blog comments powered by Disqus