Social Product Development Network

I am a bit overwhelmed by the number of excelent PLM related blog posts. Today I am going to try to answer the question asked by Oleg : How to implement Social Networking for PLM ?

PLM has always been the concept of managing the information produce by a variety of Computer Aided Authoring tools. In a PLM system everything is organized around the BOM, so all this information is linked to Part and the Parts are organized in a tree.
PLM vendor love to claim that they are also capturing the communication thanks to their workflow system.
But the reality is that these workflows capture at best 2 or 3 very well formalized process : RFQ, ECN, ...

But everybody in the companies knows that the most important discussion about the product developement are done else where : by emails, over phone, meetings... This is even more true with the concept of global engineering team located in several countries. Some companies have a very aggressive objective in term offshore engineering and design, I know a couple of companies that are targeting 50 % of these activities in low cost countries.

I think that time has come to invent yet another acronym Social Product Development Network -- SPDN. This new marketing beast will offer :

• an internal blog to every employee with the possibility to publish article on internet.
• a wiki to capture the user generated documentation around the Product
  
• An issue tracker to follow the issue related to the product development
• A twitter clone to exchange short messages
• gallery to share pictures of the product prototypes, tests , ...
  
• BOM management
• PDM
• Workflow to control the 2 or 3 key processes

You are going to tell me that this is impossible because PLM vendor does not have the resources/skills to implement all this software stack. You are right but the good news is that you don't have to do it. All this and much more is already available out there in many flavors. For example I have been recently contributing and using such platform recently. It is called Pinax a video presenting the solution is available there. The only pieces that you will have to worry about are the core of the PLM (BOM, PDM, Workflow) and for each of them you will have only to focuse on implementing the vision on an leveraging a modern infrastructure.
No you don't have to develop yet another ORM it is there. No you don't have to invent yet another template language, or an url dispatcher, a cache stragetgy. You use all your ressources to focus on the implementation of the PLM vision rather than the technical details. This technology power web site that receive million hits a day. So I assume it will scale the PLM needs.

But all this requires a complete shift from selling software licences to sell a vision.

Conclusion :
If you really have a vision for PLM sell it with the service to implement it rather than the licences.

Django fulltext search -- part 1

I have been recently working on adding full text search to a django based web application since this is not yet particularly well documented. Here it is my little stone to the amazing pyramid already in place waiting for you to use it. In order to add this "must have" feature to your great web application you will need 3 components :

• djangosearch
• pysolr
• solr
  

Starting by the latest the first challenge is to get it installed, I haven't had luck with the usual "apt-get install solr". It must be me but after an installation I was only getting a blank page instead of solr admin interface. On #solr channel someone advise me to just grab the latest release tar ball and to run with it. Then I have performed the tutorial this help me a lot to get started with solr. Believe it or not when you are there you have done the hardest part of it.

Now you need to go inside the django project you want to work on and follow the following recipe :

1. download and put into your PYHTONPATH djangosearch and pysolr
   
2. Add djangosearch inside the INSTALLED_APPS tupple
3. add the following settings :

• SEARCH_ENGINE = "solr" # solr, lucene, xapian, estraier
• SEARCH_RESULTS_PER_PAGE = 10
• SOLR_URL= "http://localhost:8983/solr"
  

At this point you are almost done the last operations you will need to do is to define inside models.py the fields you want to index with solr. In order to do this djangosearch come with a class called : ModelIndex
For example if you want to index : tease, and story fields of your Story model you will do something like this :

from djangosearch import ModelIndex
class Story(models.Model):
  ...
 tease = models.TextField(_('tease'), blank=True)
 story = models.TextField(_('story'), blank=True)
  ...
  index = ModelIndex(fields=["tease","story"])

Now you need to tell solr about this 2 new fields this is in fact easier that it seems to be, the theory is explained there. In practice what you need to do open this file (apache-solr-1.3.0/example/solr/conf/schema.xml) with your editor of choice and add this 2 lines inside the fields section:




You will now need to restart solr in order to take this modification into account.

The meal is now ready you can go and taste it.

• Open a browser go to the admin interface and add a new story or update an existing one.
• Open a django shell and enjoy your new full text search

from dajngosearch import search
search_results = search("foo")

I hope this will be useful to someone and will avoid you the pain I had to get all these pieces dancing together.
I would be glad to read from you what can be improve in this raw recipe and also what you are doing with your search and search_results.

Blindness of closed source software company

Since yesterday I am using this blog in a slightly different way than usual. For some reasons it seems that something is happening in PLM realm.

This post is again a reaction to the post from Oleg Shilovitsky's. The post was titled :

Who will play role of Google Maps in PLM?

I think this is an excelent question. I am taking this opportunity to add some information on how this could happen. I have been working closely with both concept, recently I have been working hard to developed a Geographic Information System (GIS) platform to manage Geo localized content. I will make a post related to this platform as soon as it is ready for prime time.
Let us get back to the question, just a bit of history before google map GIS was complex, expensive, not web based software, closed source, understood and used by a very small subset of the population. Do you see any resemblance with PLM ? :-)
Google maps brings it to the mass.


I agree with you that none of the application or technology below deliver something close to what google maps bring.

• Enterprise Mashups (application area emerged from ability to mix data coming from different sources and present it to user.
• Microsoft Excel Services (high acceptance of Excel user interface make it usable everywhere)
• Dassault Systemes 3DLive (very promising lifelike user experience comes last year in new DS products).

My explanation to this is that google maps was just the tip of the iceberg the cherry on the cake. The nice UI that you present to your user. But the truth is that in fact there is now a complete open source stacked available to build geo enabled mashup:

• postgis
• geodjango
• ...

This is just to named few but in reality there is plenty of component available with a vibrant community around them.
So if PLM want to get close to that ubiquity PLM vendor will need to open the doors and the windows in order to get some fresh air, idea in. This put them in front of 2 challenges :
- Ability to deliver code able to compete with other open source software. In my eyes that means improve their quality and documentation
- Build a business model to keep up this effort and make benefits from it.

Afresco manage to do it in a different realm I am sure someone will do it the only questions left are who and when.

What if SMB as a vision for PLM and PLM vendors don't undertand it.

This post has been started as a comment of a post from the virtualducthman but since it start to be too long I have decided to write my own post. This is not a rant but rather the impression I had after I moved from a famous PLM Vendor to tier one automotive supplier.

As always extremely well written post without any flaw in the reasoning if you buy in the initial hypothesis. However I disagree with the statement that you use as foundation : Software vendor are proposing excellent product with a good ROI and SMB customer don't understand it because they do not have a vision.

In fact as PLM manager for a tier one supplier in automotive I have seen quite the opposite. I have seen all the big player coming and trying to approach me with solutions that have all the exact same problem :

• No interoperability between authoring tools in the Engineering department and Plant
• No backward or forward compatibility between version with the tools that they are supposed to integrate with.
• All these tools share an interesting feature, no open standard to exchange data (xml, json, rss, atom, ...)
• Impossible to mash up the information you put in their repository with other source of information. Ok, I agree impossible is a bit to strong you can always develop this feature and add it to your application. At least by designed these applications are making such thing hard.

What if rather than saying that SMB do not have a vision we try to find a different explanation to your observation. I tend to believe that they have an extremely well defined vision. SMB know that if they want to be in the business in five years from know they need to be agile in order to transform the way they do things to adapt themselves to the new constraints of the OEM. It is interesting to note that nobody is blaming Ford, GM, ... of not being able to see that they have good chance to go bankrupt in some month from now. It is interesting that many people blame now these companies of not being able re-invent/adapt their products to their market. when all of them where using PLM systems and had huge PLM projects on going. May be this can be an interesting post to develop this idea. Why PLM system only help you to develop faster/cheaper product that you are used to produce ? or Why PLM system can hurt your creativity in introducing innovative to your customer ?

In order to develop this agility SMB need to put very high in the list of capabilities for their information system the following features : Agility, re-configuration, continuous evolution / transformation, openness, ease of integration with unknown system, overall strategy of the PLM vendor.

That put us in front of a slightly different picture, isn't it? I am not going to develop in that post why and how each of this term could help to contribute to convince the SMB to change their mind on the value proposition your are doing but I am convince that the impact will be significant. Also I hope that you understand that now known on the solution on the market give them such freedom.

As an example, Alfresco has proven that this approach can work, I understand that this is not in the PLM realm but in content management and web content management but this area shared a couple of similarity with PLM. The market was dominated by closed source vendors providing the same kind of solution for the last 15 years, with no major evolution nor integration of the new tools introduced by the development of internet : wiki, Blog, chat, tag, fulltext search, low cost, commodity software, ... Of course each PLM product have changed several times their names but unfortunately the code, concepts, people behind them are the same it was just a marketing transformation.

Where is the fresh air, idea, innovation ... May be this industry need to show some code rather than powerpoint. It would be nice also if they change their approach in trying to force their solution rather than listening to their customer SMB is not OEM. They do not have will/money to spend in top/down project they need to be able to scale their solution with a bottom up approach. They need to be able to scale from 1 project to a department then a division and at the end to the whole company.

Any way Mr virtualduchman thanks for writing your excellent posts. This post shouldn't be read as PLM do not bring value to SMB but rather like PLM bring value but this can be highly increased by listening to their needs and feedbacks.

How to explore dynamically your django code

I have been using the methods presented by Simon Willison on this post.
For quite sometimes now and I always felt that it would be nice to have the tab completion when using the debugger inside my views or elsewhere in my django code.

Here it is the magical line:
import ipdb; ipdb.set_trace()

Note : That "pdb" as been replace by "ipdb" which is the IPython debugger.

You then get all the nice stuff from IPython (tab completion, nice tracebacks) right in pdb. In order to install it you can for example do : easy_python ipdb

How to create a view to add an object...

How to create a view to add an object with a PointField

All the files described below are part of a django app called dj_cartographe. The objective of this short wiki page is to help you to create a web page where you will be able to add object RunningWaterOutage.

Let us take the following models that represents a simplified version of what could be a RunningWaterOutage :

# dj_cartographe/models.py
from django.contrib.gis.db import models

class RunningWaterOutage(models.Model):
       """A spatial model for interesting locations """
       name = models.CharField(max_length=50, )
       description = models.TextField()
       creation_date = models.DateTimeField(auto_now_add=True)
       start_date = models.DateTimeField()
       end_date = models.DateTimeField()

       geometry = models.PointField(srid=4326) #EPSG:4236 is the spatial reference for our data
       objects = models.GeoManager() # so we can use spatial queryset methods

       def __unicode__(self): return self.name

In order to use this model in django admin you should configure it as follow :

# dj_cartographe/admins.py
from django.contrib.gis import admin
from django.contrib.gis.maps.google import GoogleMap
from dj_cartographe.models import *
class RunningWaterOutageAdminOptions(admin.OSMGeoAdmin):
       list_display = ('name', 'description', 'start_date', 'end_date')
       list_filter = ('name', 'description', 'start_date', 'end_date')
       fieldsets = (
          ('Location Attributes', {'fields': (('name', 'description', 'start_date', 'end_date',))}),
          ('Editable Map View', {'fields': ('geometry',)}),
        )
       # Default GeoDjango OpenLayers map options
       scrollable = False
    map_width = 700
       map_height = 325
    GMAP = GoogleMap(key='<YOUR GOOGLE KEY THERE>') # Can also set GOOGLE_MAPS_API_KEY in settings
class RunningWaterOutageGoogleAdminOptions(admin.OSMGeoAdmin):
       extra_js = [GMAP.api_url + GMAP.key]
       map_template = 'gis/admin/google.html'
       list_display = ('name', 'description', 'start_date', 'end_date')
       list_filter = ('name', 'description', 'start_date', 'end_date')
       fieldsets = (
         ('Location Attributes', {'fields': (('name', 'description', 'start_date', 'end_date',))}),
         ('Editable Map View', {'fields': ('geometry',)}),
       )
    # Default GeoDjango OpenLayers map options
       scrollable = False
       map_width = 700
       map_height = 325
   # Register our model and admin options with the admin site
    admin.site.register(RunningWaterOutage, RunningWaterOutageAdminOptions)
   # Register the google enabled admin site
   google_admin = admin.AdminSite()
   google_admin.register(RunningWaterOutage, RunningWaterOutageGoogleAdminOptions)

It is interesting to note that in the example above we have created in fact 2 instances of admin site.

Now the interesting part of this recipe, you will find below a method that will enable you to provide a map as user interface to enter the location of the RunningWaterOutage.

#dj_cartographe/forms.py
from django.forms import ModelForm
from django.forms.fields import CharField
from django.contrib.gis.admin.options import GeoModelAdmin
from dj_cartographe.admin import google_admin

from dj_cartographe.models import RunningWaterOutage

geomodeladmin =  GeoModelAdmin(RunningWaterOutage, google_admin)
db_field = RunningWaterOutage._meta.get_field('geometry')

class RunningWaterOutageForm(ModelForm):
       # Overiding the default Field type
    geometry = CharField(widget=geomodeladmin.get_map_widget(db_field))
       class Meta:
    model = RunningWaterOutage

       class Media:
                js = (
                    "http://openlayers.org/api/2.6/OpenLayers.js",
                 )

This RunningWaterOutageForm will contain when rendered on a template a nice OpenLayers map that your users will be able to use instead of having to key in the Point in a textarea.

Now the template :

# dj_cartographe/templates/running_water_outage_edit.html
{% extends "base.html" %}

{% block title %}my first map{% endblock %}
{% block media %}
{{ form.media }}
{% endblock %}

{% block content %}
<h1>my first map</h1>
<form action="." method="POST">
{{form}}
<input type="submit" value="Submit" />
</form>
{% endblock %}

There is nothing special there, however it is interesting to note that we use {{ form.media }} to pull in the javascript.



The last piece is to create the view :

#dj_cartographe/views.py
from django.shortcuts import render_to_response
from dj_cartographe.models import RunningWaterOutage
from dj_cartographe.forms import RunningWaterOutageForm


def running_water_outage_add(request):

        if request.method == "POST":
                running_water_outage_form = RunningWaterOutageForm(request.POST)
            if running_water_outage_form.is_valid():
                    running_water_outage_form.save()
       else:
                running_water_outage_form = RunningWaterOutageForm()

        return render_to_response("running_water_outage_edit.html",
                                      {"form" : running_water_outage_form,
                                     })

Here it is the result of this small recipe :

How to evaluate the coverage of a django test suite

I am using this recipe to estimate the coverage of the test suites arounds my django's projects.

Software Prerequisites

In order to follow this recipe you will need to have the following software installed :

• Django
• python (obviously)
• coverage.py

I will not explain how to get the first 2 items of this list installed since if you are reading this I am assuming that you are familiar with them. "Coverage" is a bit different, python package index make installing this component a piece of cake.

# easy_install coverage

Usage of coverage with django test suite

This presentation give a recipe to evaluate the coverage of your test suite. I found this page useful to understand how to use it. If you are looking for a project to test this recipe on you can checkout "django-survey". The 3 lines below is all you need to get a report on the coverage of your test suite.

# coverage.py -e                            (1)
# coverage.py -x manage.py test survey      (2)
# coverage.py -r -m >report.txt             (3)

The first line erases collected coverage data, the second executes the module and collect the coverage data, the third line reports on the statement coverage for the given files and show line numbers of the statements that weren't executed.

Then you need to analyze the file called "report.txt" and extract the information usefull to your project. In our case all the files located in django-survey :

Name 
------------------------------------------------------------------------------------------------------------------------------------------------
__init__                                                                                                               0      0   100%
[...]
c:\yml\_myscript_\dj_things\web_development\django-survey\survey\__init__                                              0      0   100%
c:\yml\_myscript_\dj_things\web_development\django-survey\survey\forms                                               172    117    68%   32, 47-49, 56, 74-86, 96, 107, 117-120, 126-136, 139-145, 147-159, 212, 226-227, 230
c:\yml\_myscript_\dj_things\web_development\django-survey\survey\models                                              157    142    90%   28-29, 70, 82-83, 88-89, 102-104, 109, 126, 159, 211, 236
c:\yml\_myscript_\dj_things\web_development\django-survey\survey\templatetags\__init__                                 0      0   100%
c:\yml\_myscript_\dj_things\web_development\django-survey\survey\templatetags\survey                                  10      7    70%   8, 16-17
c:\yml\_myscript_\dj_things\web_development\django-survey\survey\tests\__init__                                        2      2   100%
c:\yml\_myscript_\dj_things\web_development\django-survey\survey\tests\test_images                                     1      1   100%
c:\yml\_myscript_\dj_things\web_development\django-survey\survey\tests\test_models                                     1      1   100%
c:\yml\_myscript_\dj_things\web_development\django-survey\survey\tests\test_urls                                       1      1   100%
c:\yml\_myscript_\dj_things\web_development\django-survey\survey\urls                                                  6      6   100%
c:\yml\_myscript_\dj_things\web_development\django-survey\survey\views                                               183    126    68%   47, 54-65, 79-83, 90, 94, 96, 124-125, 150-151, 157-158, 183-186, 230-231, 246, 261-263, 280-281, 311-313, 328, 341-342, 359-360, 379, 419-425, 444-456, 462-474
[...]
manage                                                                                                                13      9    69%   10-13
management\__init__                                                                                                    0      0   100%
settings                                                                                                              31     31   100%
urls                                                                                                                   4      4   100%