Paginated feed for cmsplugin_feed

In my previous post [1] I wrote about my experience of writing a new plugins for django-cms [2]. This plugin gives you  the capability to add a feed to a django-cms' Page.

I ended my post by asking about the best practice used to paginate the content of a plugin. Since I didn't get flooded by the answers I assumed that this shouldn't be different than doing it on a django app. Here it is what the fine django documentation says about this topic.

So I have decided to implement this approach, the key point here is to understand that django-cms' plugin has access to the context :

class FeedPlugin(CMSPluginBase):
    [...]
    def render(self, context, instance, placeholder):
        feed = get_cached_feed(instance)
        if instance.paginate_by:
            is_paginated =True
            request = context['request']
        [...]

This capability lets your plugin react to the GET and POST parameters. Once I have understood this the only thing that I had to do is to use the django's Paginator on the list of feed entries.

As you can see in the code below there is nothing specific to django-cms there.

            is_paginated =True
            request = context['request']
            feed_page_param = "feed_%s_page" %str(instance.id)

            feed_paginator = Paginator(feed["entries"], instance.paginate_by) 
            # Make sure page request is an int. If not, deliver first page.
            try:
                page = int(request.GET.get(feed_page_param, '1'))
            except ValueError:
                page = 1
            # If page request (9999) is out of range, deliver last page of results.
            try:
                entries = feed_paginator.page(page)
            except (EmptyPage, InvalidPage):
                entries = feed_paginator.page(paginator.num_pages)

The complete implementation of this feature is available in the bitbucket repository of cmsplugin_feed.

I would be glad to hear from you if this implementation could be enhanced.

[1] http://yml-blog.blogspot.com/2009/12/feed-extension-for-django-cms.html
[2] http://www.django-cms.org/
[3] http://docs.djangoproject.com/en/dev/topics/pagination/
[4] http://bitbucket.org/yml/cmsplugin-feed/changeset/7d0644c7668f/

feed extension for django-cms

I have been lately looking at the open source CMS alternatives based on django. After some times investigating I have decided to give django-cms [1] a try.

django-cms provides an API to write plugin it is well documented [2] and you can also look the implementation of plugins listed there [3]. I have decided to experiment with it by writing a plugin that displays the content of a feed into a page. The source code is available on my bitbucket account in a project called cmsplugin-feed. The development of this plugin has been relatively straight forward and once you get to know the conventions defined to implement an extension. You feel like you are writing python code for a django application. The CMS part does not stand on your way at least it didn't in my experimentation.

Here it is 2 screenshots showing this plugin in action :
In order to add a feed you need give a name and an URL to your feed

Then the plugin displays the field on a page.

I would be interested to read from someone the "best practice" to paginate the content of a plugin. Ideally I would like to add a parameter for every feed indicating how many items I want per page. Then I would like to paginate the feed in the placeholder of every page and this will allow the user to navigate between the pages.

[1] http://www.django-cms.org
[2] http://www.django-cms.org/en/documentation/2.0/custom_plugins/
[3] http://www.django-cms.org/en/extensions/
[4] http://bitbucket.org/yml/cmsplugin-feed/

uWSGI reaches the 0.9.3 Milestone

uWSGI project [1] reaches the 0.9.3 milestone. You can review the complete announcement [2] on the mailing list  and download the code here [3]

This new release brings some new exiting features :

- Nginx 0.7.x module
- configuration via python module
- support (non-standard) for Python 3.x
- Twisted client resource adapter
- graceful restart of worker processes and hot-plug substitution/upgrade
of the uWSGI server
- shared memory area to share data between workers/processes
- Tomcat handler
- support for virtualenv

In this post [4] I have covered the usage of both the "configuration via a python module" and the "virtualenv support".

uWSGI allows you to run your favorite wsgi application on top of prefered web server (apache, cherokee or nginx).

[1] http://projects.unbit.it/uwsgi/
[2] http://lists.unbit.it/pipermail/uwsgi/2009-November/000020.html
[3] http://projects.unbit.it/downloads/uwsgi-0.9.3.tar.gz
[4] http://yml-blog.blogspot.com/2009/11/setting-up-django-project-with-cherokee.html

Continous sphinx build

I have always found the cycle : Edit the documentation source => build the sphinx based documentation [1] => open a browser pointing the updated documentation and re iterate until your are satisfied a bit painful.

Today I have finally found an automated version of this workflow that
gave me one of this nice WAHOUUUU feeling that happens after you remove this little stone from your shoe.

The idea is to continuously build the documentation using watch [2]

watch -n 2 make htlm

This will update the generated documentation every 2 seconds. This is nice in itself and it leads me to discover that Epiphany automatically reload the page when it changes.

[1] http://sphinx.pocoo.org/
[2] http://linux.die.net/man/1/watch
[3] http://projects.gnome.org/epiphany/

Setting up a django project with Cherokee, uWSGI and vitualenv (continued)

In my previous post [1] I have started to talk about this stack but I have avoided to mention that the django project was indeed encapsulated into a virtualenv. This brought a couple small quirks and questions to the surface. So I spent some times investigating and exchanging with the community behind uWSGI and cherokee in order to solve them. The enhancements I discuss here require that you use the tip of uWSGI [2] because some of them are new capability added in the past days or small improvements to my recipe of using these products.

Before I start to dive into the enhancements let me sum up what we had to do in order to serve our WSGI project, in our case a django project.

1> I had to add 2 configurations files : uwsgi.conf, django_wsgi.py
The first one is an XML file where you define the PYTHONPATH and the mount point for your application, the second one is a python module where you define your WSGI configuration.

2> I had to use the cherokee's wizard to create the appropriate configuration to serve my django app using uWSGI. This step add an "Information Sources" with the following settings in the interpreter section :

/usr/local/bin/uwsgi -s /tmp/cherokee-source1.sock -C -x /opt/webapps/example-ve/src/project/uwsgi.conf

The first thing that can be improved in this recipe is that you can reduce the configuration to only one python module. This allows you to avoid the uwsgi.conf file. uWSGI comes with an option that allows you to directly specify a python module.

-w     name of wsgi config module (no ROCK_SOLID)

This would give us the following command :

uwsgi -s /tmp/cherokee-source1.sock -C -w project.django_wsgi

The django_wsgi file is very simple and looks like this :

import os
import django.core.handlers.wsgi

# Set the django settings and define the wsgi app
os.environ['DJANGO_SETTINGS_MODULE'] = 'project.settings'
application = django.core.handlers.wsgi.WSGIHandler()

# Mount the application to the url
applications = {'/':'application', }

This is all good and simple but it only works if "project" is in the PYHTONPATH. I handle all my projects in a dedicated virtualenv. uWSGI handle this gracefully now, there is a new option that allows you to define your Python Home.

-H     set python home/virtualenv

Here it the improved settings for the interpreter section of the information sources :

uwsgi -s /tmp/cherokee-source1.sock -C -w project.django_wsgi -H /opt/webapps/example-ve

I would be glad to hear from you and very happy if you give this setup a spin on your django project and let me know how it performs compare to your actual setup.

[1] http://yml-blog.blogspot.com/2009/11/how-to-set-up-cherokee-and-uwsgi.html
[2] http://projects.unbit.it/uwsgi/changeset/87%3Ae78ef65c833e

How to set up cherokee and uWSGI

A couple of post ago I have "blogged" about cherokee which is a light weight web server, fast and extremely simple to configure thanks to its great admin interface.

The most common configuration to use cherokee with django is described here and to put is in a nutshell it was based on flup using SCGI or FASTCGI protocol.

However a couple of weeks ago a new contender pop up on my radar. It is called uWSGI and it might become in the near future very popular to power django application. You can find more information about it on on the pages below :
* example [2]
* Documentation [3]

From an architecture stand point it might provide us python web developer an interesting piece of software that could operate with virtually any webserver. There is already an apache module and cherokee handler. The documentation mentions that the core developers have started to develop an nginx adaptor.


Software prerequisites

cherokee 0.99.26
uWSGI (changeset: 85:61dcfd718023)

I have installed cherokee using the PPA on launchpad [1] and uWSGI directly from the mercurial repository.

hg clone http://projects.unbit.it/hg/uwsgi

Installation

Installation of both product is relatively straight forward and well documented. I will not detail it more here but rather point you to cherokee [4] and uWSGI website [5].
Here it is how you would compile uWSGI on Ubuntu 9.04

make -f Makefile.Linux.Py26

Note: The compilation will create a binairy called uwsgi26. It is important to add it into your PATH under the name of uwsgi. If you don't do this cherokee's wizard  will complain about not being able to find uWSGI server. I have created a symbolic link but you could also copy it into /usr/local/bin.

Configuration

The amazing admin interface of cherokee make it dead simple once you know what need to be done. In the latest version of cherokee a wizzard has been added to make the configuration even simpler.

First thing first you need to create 2 configurations file into your django project directory called respectively : django_wsgi.py and uwsgi.conf

------ django_wsgi.py ------------------

  import os import django.core.handlers.wsgi  
  os.environ['DJANGO_SETTINGS_MODULE'] = 'project.settings' 
  application = django.core.handlers.wsgi.WSGIHandler() 
 

------ uwsgi.conf --------------------

<uwsgi>

    <pythonpath>/home/yml/workdir/webdev/fabric_factory/20091023/fabric_factory/ve/lib/python2.6/site-packages/</pythonpath>

    <pythonpath>/home/yml/workdir/webdev/fabric_factory/20091023/fabric_factory/src/</pythonpath>

    <app mountpoint="/">

        <script>project.django_wsgi</script>

    </app>

</uwsgi>

Once this is done you can launch cherokee admin : 'cherokee-admin -u' and point your browser to this URL :
* http://127.0.0.1:9090/

Once in cherokee you can use the uWSGI wizard to create automatically your information sources and add a new rule. In order to do so you need to go into the virtual server section on the behavior tab [6]. Below the table there is an hyperlink called wizards. It will take you to a list of wizards, click on uWSGI.

Cherokee asks for the path to the uwsgi.conf file that we have created earlier. The last bit you might want to do is to add a rule to serve the admin media.

Troubleshooting

Here it is some basic technics to understand what is going on and solve your issues. Look at cherokee's log files :

sudo tail -f /var/log/cherokee/cherokee.error
and
sudo tail -f /var/log/cherokee/cherokee.access 

Look into the section called "information sources" in cherokee-admin. Explore the information sources created by the uWSGI wizard. Copy the text in the interpreter field, in my case it is :

/usr/local/bin/uwsgi -s /tmp/cherokee-source1.sock -C -x /[Path to my django project]/uwsgi.conf

Then you can paste it in a terminal this will give you a way to see what it is going on.

This new combination is still very young but it looks promising to me. I am sure that if we get enough people to hammer on it can quickly becomes popular.

[1] https://launchpad.net/~cherokee-webserver/+archive/ppa
[2] http://projects.unbit.it/uwsgi/wiki/Example
[3] http://projects.unbit.it/uwsgi/wiki/Doc
[4] http://www.cherokee-project.com/doc/basics_installation.html
[5] http://projects.unbit.it/uwsgi/wiki/Install
[6] http://127.0.0.1:9090/vserver/10

django-piston authentication against django auth -- part 4

The previous post ends with one known "Open Issue". The authentication for the ajax call was sitll hard coded [1]. In this post I am going to show you how you can extend django-piston to authenticate your users against django.contrib.auth. This might be important if your web app already take advantage of this module to manage and authenticate users.


A bit of reading of the source code and the documentation of django-piston [2] leads me to understand that django-piston has been designed from the ground to enable you to easily write your own authentication handler. Out of the box it comes with "HttpBasicAuthentication" and OAuthAuthentication. "HttpBasicAuthentication" is a very good example how to implement a DjangoAuthentication.

An authentication handler is a class, which must have 2 methods: is_authenticated, challenge

class DjangoAuthentication(object):
    """
    Django authentication. 
    """
    def __init__(self, login_url=None, redirect_field_name=REDIRECT_FIELD_NAME):
        if not login_url:
            login_url = settings.LOGIN_URL
        self.login_url = login_url
        self.redirect_field_name = redirect_field_name
        self.request = None
    
    def is_authenticated(self, request):
        """
        This method call the `is_authenticated` method of django
        User in django.contrib.auth.models.
        
        `is_authenticated`: Will be called when checking for
        authentication. It returns True if the user is authenticated
        False otherwise.
        """
        self.request = request
        return request.user.is_authenticated()
        
    def challenge(self):
        """
        `challenge`: In cases where `is_authenticated` returns
        False, the result of this method will be returned.
        This will usually be a `HttpResponse` object with
        some kind of challenge headers and 401 code on it.
        """
        path = urlquote(self.request.get_full_path())
        tup = self.login_url, self.redirect_field_name, path 
        return HttpResponseRedirect('%s?%s=%s' %tup)

The implementation is relatively simple, is_authenticated uses the method with the same name available from a django User and the challenge redirect the user to the login page.

This post will end the series on django-piston it was for me a pretext to get familliar with this great app that make creating a RESTFUL api for django easy. You can find all the modifications I did into my fork on bitbucket [3]


[1] http://bitbucket.org/yml/django-piston/src/a8bcb7f9756e/examples/blogserver/templates/edit_ajaxy_post.html#cl-6
[2] http://bitbucket.org/jespern/django-piston/wiki/Documentation#authentication
[3] http://bitbucket.org/yml/django-piston/