Feb 21, 2009

django full text search with solango

2 months ago I wrote a post titled "django fulltext search part-1" this post was explaining how to take advantage of djangosearch to interface between django and solr.

The big advantage of djangosearch is the fact that it comes with a plugable backend architecture. This can be a strength since in theory it enables you to abstract the details of the the full text engine you are using however in practice I ended up writing a patch to bypath the abstraction layer because it was preventing me of doing the query I wanted. So to make a long story short djangosearch was not working out of the box for my needs.

However 2 week ago Sean Creeley released solango and this significantly changed landscape of full text search in the django eco-system. I will not go in the details about solango in this post. It comes with some management commands that are so convenient that I still wonder why I haven't thought at implementing them on top of djangosearch. I will copy below a short extract form the documentation which is excellent :
./manage.py solr --help
#solango schema options
--fields Prints out the fields the schema.xml will create
--flush Will remove the data directory from Solr.
--reindex Will reindex Solr from the registry.
--schema Will create the schema.xml in SOLR_SCHEMA_PATH or in the --path.
--start Start solr running java -jar start.jar
--path=SCHEMA_PATH Tells Solango where to create config file

In the rest of the post I am going to assume that you have installed and configured solango. When I have done this I have not seen any major obstacle. Once again Sean has done an excellent job at documenting this project. one thing that annoys me while implementing solango in one of the project I am working on is the fact that you cannot restrict your search in what is often call an "advanced search". The good news is that solango has been recently improved to be easily extended. The only thing you will have to do is to defined a django form to represent your advanced search and to add an url that use it. This can literally be done in less than 30 lines of code including the comments and the imports.

Let us start by the writing the forms.py :

from django import forms
from solango.solr import get_model_from_key
import solango

def model_choices():
"""
Return a list of tuple with all the models that have been indexed.
This tuple is used in the AdvancedSearchForm to selects the models
you want to search your term in.
"""
models = [(model_key, get_model_from_key(model_key)._meta.verbose_name_plural)
for model_key in solango.registry.keys()]
return models

class AdvancedSearchForm(forms.Form):
"""
Form that represents an advanced search
"""
q = forms.CharField(required=False)
model = forms.MultipleChoiceField(choices=model_choices(), required=False,
widget=forms.CheckboxSelectMultiple)
def clean_q(self):
q = self.cleaned_data.get("q")
if q == '':
raise forms.ValidationError("You cannot query for an empty string")
return q

Nothing really complex there we have defined a form with a charfield named "q" that will be used to enter your search terms and a set of check boxes displaying all the models indexed. The user will be able to select one or several models to restrict its query.
Now that the form is defined we are going to use it. In order to do so you need to add the following url somewhere in your project.

from django.conf.urls.defaults import *

from solango.views import select
from populous.search.forms import AdvancedSearchForm

urlpatterns = patterns('',
url(r'^advanced/$', select,
{
"form_class":AdvancedSearchForm,
"template_name":None,
},name="search-advanced"),
)

You will recognize there the same pattern used in the generic views. You can customize the solango's view, called "select", by passing a "form_class" and a "template_name". In this example I have not over loaded the template_name thus django will used the solango default template.

I would be glad to read from you the customized form you have built on top of solango. Please do not hesitate to post them as comment.

Updated the 22th Feb 2009 : Correct the name of Sean Creeley, sorry for that.