Django on Glassfish via Jython

Introduction

For the last major web application that I created, I used Django, and absolutely loved it. Since I'm now working on a project that uses Glassfish, I decided to make an attempt at running Django in Glassfish. Since Glassfish is primarly Java-based, it makes the most sense to get Django running on Jython, and run it in Glassfish that way.

Luckily for me, some people have already done considerable work in getting Django running in a Jython environment, and on Glassfish. The information from Frank Wierzbicki's blog gave me a good sense of the state of this task. His instructions for running Django in Jython, and then running the combination in Glassfish can be found here:

Jython and Django Progress Part I
Jython and Django Progress Part II

Upon further investigation, it seems that in the months following Frank's blog posts, some other users have created a much neater way of getting everything running. In fact, getting Django running on Jython should now be as simple as following the steps listed here:

Django on Jython

And then deploying the Jython+Django on Glassfish has also been simplified:

Django .war Deployment

So, I began by following these instructions. However, along the way, I ran into a few speedbumps. In case others have a similar configuration to me, and run into the same problems, it seems prudent to document my experience here.

I'm going to post a step-by-step account of the process that I followed. I want to make it very clear that a large portion of this process involved following the steps in the above links, so much of the credit should go to those authors. Consider this an incremental contribution.

Environment

For purposes of repeatability, here is the environment in which I'm working:

• Ubuntu Server 8.04 LTS


• Java 1.5.0_15-b04


• Jython SVN r5530


• Django SVN r9294


• Django-Jython SVN r33


• Glassfish v2ur2-b04


• Modjy_0_22_2

It sure is hard not to type Djython!

Process

1. Install jython-dev
   

   
   svn co https://jython.svn.sourceforge.net/svnroot/jython/trunk/jython/  jython-dev
   cd jython-dev
   ant
   

   
   

You also want to add related environment variables and aliases to your environment. You can either run these each time at the command line, or add them to the appropriate shell login script (I used .bash_profile). I used the absolute path the the SVN version of jython that I'd just built.


$JAVA_HOME = /path/to/java
$JYTHON = /path/to/jython
alias jython25=/path/to/jython-dev/dist/bin/jython





2. Install Django
   

   
   svn co http://code.djangoproject.com/svn/django/trunk/ django-dev
   cd django-dev
   jython25 setup.py install
   

   
   You might also want to add a shortcut to django-admin through jython:
   

   
   alias django-admin-jy="jython25  /path/to/jython-dev/dist/bin/django-admin.py"
   

   
   



3. Get the Django-Jython helper scripts
   

   
   svn co http://django-jython.googlecode.com/svn/trunk/ django-jython
   cd django-jython
   jython25 setup.py install
   

   
   



4. Create a new project
   

   
   django-admin-jy startproject myproject
   

   
   



5. Try to run the project:
   

   
   jython25 myproject/manage.py runserver
   

   
   
   It was at this point I encountered my first error:
   

   
   Traceback (most recent call last):
    File "myproject/manage.py", line 4, in 
      import settings # Assumed to be in the same directory.
   java.lang.ArrayIndexOutOfBoundsException: 6696
   

   
   
   I remedied this error by removing the first line of myproject/settings.py -- I'm not sure why this caused a problem, and I haven't spent much more time investigating it. It doesn't seem consistently repeatable.
   

Now, I successfully had my Django app running in Jython. All in all, those instructions were pretty good! Next, I followed the procedure to deploy the Django/Jython app as a war.

1. Add 'doj' to the app list in settings.py. In case you're wondering, this was added to your jython site-packages when you installed django-jython.


2. Build the war using the manage.py module built from django-jython:
   

   
   jython25 myproject/manage.py war
   

   
   Wow, now you have war! It's that simple! Well, not quite, for me.
   



3. Deploy the war:
   

   asadmin deploy myproject.war

   
   Depending on your Glassfish configuration, you may need to use the absolute path for the asadmin program.
   
   This is where my real problems began
   



4. Problem:
   WARNING: com.sun.enterprise.deployment.backend.IASDeploymentException: PWC1651: Class com.xhaus.modjy.ModjyJServlet has unsupported major or minor version numbers, which are greater than those found in the Java Runtime Environment version 1.5.0_15
   
   That one is easy. The modjy included in django-jython was compiled with Java 1.6. I'm running Java 1.5. So, I downloaded Modjy 0.22_2 and re-built it with my version of Java. The resulting jar is copied to:django-jython/doj/management/commands/war_skel/WEB-INF/lib. Then, rebuild django-jython, and then rebuild your war.
   



5. Problem:
   Command deploy executed successfully with following warning messages: Error occurred during application loading phase. The application will not run properly. Please fix your application and redeploy.
   WARNING: com.sun.enterprise.deployment.backend.IASDeploymentException: ContainerBase.addChild: start: LifecycleException: javax.servlet.ServletException: Unable to import 'modjy_servlet' from /var/glassfish/domains/domain1/applications/j2ee-modules/myproject/WEB-INF/lib/modjy.jar: do you maybe need to set the 'modjy_jar.location' parameter?
   
   This was tricky to debug, because the actual Python exception was being caught and re-raised, hiding the original cause of the error. The exception was being thrown when the xhaus Java servlet tried to import the Python code (grepping for strings helped me to determine this). Since I really wanted to see the Python error, I made the following modification to the Java part of the modjy component:
   

   
   StringWriter sw = new StringWriter();
        PrintWriter pw = new PrintWriter(sw);
        ix.printStackTrace(pw);
        throw new ServletException(sw.toString()+"--Unable to import '"+MODJY_PYTHON_CLASSNAME+"' from "+modjyJarLocation+
        ": do you maybe need to set the 'modjy_jar.location' parameter?");}
   

   
   
   After rebuilding modjy as described above, and redeploying, I was able to see the real cause of the problem.
   
   
6. WARNING: com.sun.enterprise.deployment.backend.IASDeploymentException: ContainerBase.addChild: start: LifecycleException: javax.servlet.ServletException: Traceback (most recent call last):
   File "", line 1, in
   File "/var/glassfish/domains/domain1/applications/j2ee-modules/myproject/WEB-INF/lib/modjy.jar/modjy.py", line 24, in
   ImportError: No module named types
   
   Well.... crap. Jython was having trouble importing standard modules. According to modjy, it added to the python path every subdirectory of WEB-INFO/lib-python in the .war. But it turns out, after peeking in the code, that this is not actually the case. Modjy actually just searches for .pth files (python path files), and uses those to modify the system path. So, this was an easy remedy. For each subdirectory of lib-python that contained a library I wanted, I had to create <name>.pth, and include one line in the file, indicating the path that should be included in sys.path. I did this for django, Lib, doj, and myproject. For example: django.pth contained one line:

   /django

   
   
   This solved the problem, and the app was successfully deployed. Hooray!
   



7. Browsing to localhost:8080/myproject allows me to see the deployed Django/Jython app, and results in an instant "un-hooray":
   Traceback (most recent call last):
   File "/var/glassfish/domains/domain1/applications/j2ee-modules/myproject/WEB-INF/lib/modjy.jar/modjy.py", line 80, in service
   File "/var/glassfish/domains/domain1/applications/j2ee-modules/myproject/WEB-INF/lib/modjy.jar/modjy_exceptions$py.class", line 91, in handle
   modjy_exceptions.NoCallable: Error loading jython callable 'handler': No module named core
   
   This problem took a lot of poking around to debug. The problem occurs in the servlet's application.py, a small module that generates the WSGI servlet handler for Glassfish. The problem was that the system path provided seemed to include too much, causing the django import to fail. It seemed that django was resolving to a javapackage, instead of a Python module. I'm not sure exactly why this fixed it, but moving the sys.path element '__classpath__' from the middle to the end of sys.path fixed it. So, I added these lines at the top of application.py:
   
   

   
   import sys
   sys.path.remove('__classpath__')
   sys.path.append('__classpath__')
   

   
   
   



8. Problem:
   Traceback (most recent call last):
   File "/var/glassfish/domains/domain1/applications/j2ee-modules/myproject/WEB-INF/lib/modjy.jar/modjy.py", line 77, in service
   File "/var/glassfish/domains/domain1/applications/j2ee-modules/myproject/WEB-INF/lib/modjy.jar/modjy.py", line 94, in dispatch_to_application
   File "/var/glassfish/domains/domain1/applications/j2ee-modules/myproject/WEB-INF/lib/modjy.jar/modjy_wsgi$py.class", line 141, in set_wsgi_environment
   File "/var/glassfish/domains/domain1/applications/j2ee-modules/myproject/WEB-INF/lib/modjy.jar/modjy_wsgi$py.class", line 115, in set_wsgi_streams
   TypeError: coercing to Unicode: need string, 'javainstance' type found
   
   This occurred when trying to store environment variables for the WSGI environment. The module used PyFile to wrap InputStreams. According to Jython documentation, streams should be wrapped using org.python.core.util.FileUtil.wrap(). So, I added
   

   from org.python.core.util import FileUtil

   And changed the lines
   

   
   dict["wsgi.input"]  = PyFile(req.getInputStream())
   dict["wsgi.errors"] =  PyFile(System.err)
   

   
   To read:
   

   
   dict["wsgi.input"]  = FileUtil.wrap(req.getInputStream())
   dict["wsgi.errors"] =  FileUtil.wrap(System.err)
   

   
   
   This fixed the problem.
   
   
9. 
   Next problem:
   Traceback (most recent call last):
   File "/var/glassfish/domains/domain1/applications/j2ee-modules/myproject/WEB-INF/lib/modjy.jar/modjy.py", line 81, in service
   File "/var/glassfish/domains/domain1/applications/j2ee-modules/myproject/WEB-INF/lib/modjy.jar/modjy_exceptions$py.class", line 91, in handle
   modjy_exceptions.ApplicationException: dictionary update sequence element #0 has length 1; 2 is required
   
   This one was a real pain. Because of the exception handling magic in modjy, most of the stack trace was getting swallowed. After foolishly trying to scatter file-writing debugging statements all over the place, I realized I should just make the exceptions fire properly. So, around line 81 in modjy.py, I got rid of the try/except clauses around the line. This worked, and I got the full-blown stacktrace:
   
   

   
   Traceback (most recent call last):
    File "/var/glassfish/domains/domain1/applications/j2ee-modules/myproject/WEB-INF/lib/modjy.jar/modjy.py", line 73, in service
    File "/var/glassfish/domains/domain1/applications/j2ee-modules/myproject/WEB-INF/lib/modjy.jar/modjy.py", line 94, in dispatch_to_application
    File "/var/glassfish/domains/domain1/applications/j2ee-modules/myproject/WEB-INF/lib/modjy.jar/modjy.py", line 110, in call_application
    File "/var/glassfish/domains/domain1/applications/j2ee-modules/myproject//application.py", line 10, in handler
      to_return = h(environ, start_response)
    File "/var/glassfish/domains/domain1/applications/j2ee-modules/myproject/WEB-INF/lib-python/django/django/core/handlers/wsgi.py", line 240, in __call__
      response = self.get_response(request)
    File "/var/glassfish/domains/domain1/applications/j2ee-modules/myproject/WEB-INF/lib-python/django/django/core/handlers/base.py", line 67, in get_response
      response = middleware_method(request)
    File "/var/glassfish/domains/domain1/applications/j2ee-modules/myproject/WEB-INF/lib-python/django/django/contrib/sessions/middleware.py", line 10, in process_request
      session_key = request.COOKIES.get(settings.SESSION_COOKIE_NAME, None)
    File "/var/glassfish/domains/domain1/applications/j2ee-modules/myproject/WEB-INF/lib-python/django/django/core/handlers/wsgi.py", line 177, in _get_cookies
      self._cookies = http.parse_cookie(self.environ.get('HTTP_COOKIE', ''))
    File "/var/glassfish/domains/domain1/applications/j2ee-modules/myproject/WEB-INF/lib-python/django/django/http/__init__.py", line 256, in parse_cookie
      c.load(cookie)
    File "/var/glassfish/domains/domain1/applications/j2ee-modules/myproject/WEB-INF/lib-python/Lib/Cookie.py", line 621, in load
      self.update(rawdata)
   ValueError: dictionary update sequence element #0 has length 1; 2 is required
   

   
   
   After much more poking around near the line in problem, it turns out the the Jython cookie library only handles regular strings, not unicode strings. My first attempt, which was to allow the Cookie.load function to parse unicode as well, was a bad idea, because a lot of downstream functions also needed strings rather than unicode. So, I added a pre-processing step in the servlet application.py file to convert anything unicode back into a string:
   
   

   
   def handler(environ, start_response):
      for k,v in environ.items():
          environ[k] = str(v) 
   

   
   
   This fixed the problem, and the Jython/Django application successfully loaded via Glassfish. Hooray!
   
   Incidentally, jython should probably add handling to this error, similar to python (which complains if you try to load a Cookie via a unicode string).
   
   

I welcome any feedback from others who read my experience, either to call my debugging techniques primitive (and suggest better ways), or to provide your own experiences with running Django/Jython on Glassfish.

Touch Me!

This was an embedded touch-screen computer at Tully's coffee in downtown Berkeley. It was sponsored by SFGate.com, so it didn't do anything more than showing the news. Having an up-facing display in the table like that reminded me of the cocktail-table style of video games that was popular at bars in the 1980's.

Importing old mail into GMail.... with the proper dates!

So, you've decided that you want to do all of your email management from GMail. Maybe you've looked around a little, and found Mark Lyon's helpful GMail Loader. While this is a great utility, the method that it uses to import your emails causes GMail to reset the dates. While this isn't a huge issue, really, it was enough to keep me from using it as a solution... I wanted it to be perfect, or not at all. Unfortunately, this was by far the only option for importing emails.

Until recently....

Overview

Google has started to allow users to check up to 5 external POP email accounts from within GMail, integrating the mail with GMail. You can use this new feature to your advantage--if you have just a bit of technical dexterity--to import your local emails into GMail.

All that you need to do is run your own POP server! While this might sound like a non-trivial undertaking, it's not as bad as you might think. Furthermore, if you have SSH access to the machine on which your existing POP server is hosted, you can skip the installation step altogether.

Here's the high-level process for importing your emails using a private POP server:

1. Set up a POP server (or find one where you have access to location of the mail spool in the filesystem)

2. Using your current email program, find all of the emails that you want to import, and export them in "mbox" format.

3. Take this newly created mbox file, and move it into the mail spool area of your POP sever, giving it the name of the username that you use to check your email. In many cases, this is simply the first part of your email address, though not always.

4. Set up a new GMail pop-checking account. Enter the IP address or domain name of your newly created (or played with) POP server. I'd suggest setting the importer to automatically archive the mail and give it some unique label, just for your own sanity.

5. Sit back and wait. It will take a long time.
POP Clients

For Linux, I recommend teapop since it took no time for me to setup (it's in the Ubuntu repository), and seems to work pretty well. Of course, there are a number of other options, like qpopper, or the UW mail programs, or exim, but they may require more configuration.

For Windows, I don't know any native POP servers. So you'll have to do your own investigating here. One option is to install Cygwin and use a Linux pop server. I haven't done this myself, so I can't offer any detailed advice, but if you need help, feel free to email me.
Further Notes

If you're going to use a pre-existing pop server, I suggest using one that you're not actively receiving mail on. Mucking around in the mail spool may cause you to lose an email or two, or other bizarre things.

If you set up a POP server on your machine at home, remember that you'll need to set up your home router to forward requests for port 110 (POP) to the machine carrying your email. If you want to use a secure connection, it will be a different port.

Additionally, if you are sending mail from a POP server you set up on your machine at home, you should set up a DynDNS name for your box. Since the process takes a long time, your dynamic IP address could change before GMail is done getting everything.

This process is VERY VERY slow. I've found that the GMail popper will only check 200 mails at a time. I'm not sure exactly how it decides the interval between checks (they mention that they use an algorithm to decide how frequently to check each pop account), but I've found that at steady state, it eventually slows to only checking once every 1-2 hours. (presumably some sort of rate-limiting behavior.) For an inbox of 18000 emails, it took about 3 days before everything was imported.

Deskbar Applet Plugin

Deskbar Applet is a versatile Gnome panel applet. It places a small text entry box on your panel, and then allows you to perform a number of actions based on what you type in the text box. It's a VERY useful applet, and can replace a number of applets. For example, if you type a word, it will give you the option to look the word up in a dictionary, on wikipedia, or to execute a command if the word happens to be a command name. You can install applet handlers that allow you to install software packages, perform calculations, and do a number of other things. Since deskbar applet is written in python, it's very easy to extend its functionality, with just a little python knowledge! Read on to find out how...

Thanks to Leo Szumel for helping me out with this!

In this tutorial, we'll explain the various components of a deskbar-applet handler, while going through the design of a handler called Noted, which was created by my friend Leo Szumel.

Overview: How it all works

The deskbar applet works by passing the text that you type into the search box to a number of different handlers. Each Handler is responsible for parsing the text and deciding whether or not the text is of any interest to that parser. Because *every* handler will process EVERY text item, this portion of the script should be lean and mean.

If the parser for your handler decides that it can do something with what's been typed into the text box, then it returns one or more Match objects. Each Match object represents one line that appears in the results drop-down of the deskbar applet. The Match object is responsible for telling the deskbar applet how it should be represented in the list (icon and text string) which category it should appear under, and what action should be taken if the user clicks on it.

So, in the case of our example application: when the user types text into the search box, it is passed along to NotedHandler.query(). NotedHandler.query() then searches through all files in a previously specified directory, and records which files contain the text in the query. Each time a match is found, a new GrepMatch object is created, with the name of the file.

Deskbar-applet lists all of these matches (which have an associated display string) in the drop down box. If the user decides to click on a Match associated with a GrepMatch object, then GrepMatch.action() is called, which will run your favorite texteditor (using the EDITOR environment variable) on the file in question.

Starting Out: A Skeleton for the Plugin

So, we begin with a skeleton for the plugin. The entire skeleton plugin can be downloaded for editing Here.

import deskbar.Handler
import deskbar.Match
from gettext import gettext as _

_debug=True
if _debug:
    import traceback

This imports the modules needed for proper deskbar Handler operation. Note that gettext isn't necessary, but "real" GNOME apps should include translation capability, which is provided by gettext. When you're done debugging your app, you can set _debug=False.

HANDLERS = {
    "HelloHandler" : {
    "name" : _("Hello"),
    "description" : _("Greet the world"),
    "version" : "0.1",
    }
}

This is the registration variable for the HANDLER. The most important thing to notice here is that the key for the dictionary item (HelloHandler in this case) should be the name of the Handler subclass that you create later. To fit with the convention of other Deskbar applets, the name should use title capitalization ("Make Thing Like This"), and the description should be a sentence without the final punctuation.
#This is the bare minimum handler. No matter what the query is, it returns one match. The match has no interesting information.

class HelloHandler(deskbar.Handler.Handler):
    def __init__ (self):
        deskbar.Handler.Handler.__init__ (self, "gnome-logo-icon")
    def query(self, text):
        return [HelloMatch(self)]

Now, we create the Handler subclass, which is the meat of the handler. You must always include the __init__ function as shown here, to set the icon that is displayed next to the match.

If you want to include other one-time initialization functionality, you can include any other one-time initialization functionality in a function called initialize. Since we don't need to initialize anything here, we've left it out.

The Handler.query() is where the magic happens. The text variable passed to the function contains the text that was entered into the deskbar applet search box. In this function, what normally happens is that the developer will look at the text, decide if it's relevant to the action of the handler, and if so, create a match. In our skeleton, we just always return one single Match item.

class HelloMatch(deskbar.Match.Match):
    def get_verb(self):
        return "Hello World"
    def get_category(self):
        return None
    def action(self):
        pass

Finally, here is the bare minimum of non-trivial functionality for the Match subclass. The get_verb function tells Deskbar Applet what to display for this Match in the dropdown list. The get_category shoudl return a key into the categories dictionary, that will specify in which section of the dropdown list this match should occur. More on this later. Finally, the action function is the callback that is execute when the user clicks on the item in the dropdown list that is associated with this Match.

if __name__ == '__main__':
  import sys

  print __file__,': Running unit test'
  ah = HelloHandler()
  ah.initialize()

  hits = ah.query(sys.argv[1])

  for h in hits:
    print '%s: action %s ? (y/n/q)' % (h.get_category(), h.get_verb()),
    ask = sys.stdin.read(1)
    if ask == 'y':
      h.action()
    if ask == 'q':
      break
    print

Finally, we present here an optional (but highly recommended) unit test for Deskbar Applets. This will allow you to call your handler on the command line, passing it a query as a command-line option. You can use it to validate the behavior of your plugin without having to fuss with the deskbar-applet interface.

For the skeleton applet, the result should be: None: action Hello World ? (y/n/q)

And then it will exit regardless of your choice, since no action is defined.

Modifying the skeleton to create a "real" handler.

Now, we'll outline the development of a real plugin that allows you to grep all files in a given directory for the text in the query box. Here is the complete code for the handler. We start with the preliminary requirements:

from deskbar.Handler import Handler
from deskbar.Match import Match
from gettext import gettext as _

import glob, re, os

NOTEDIR = '/home/leo/f/notes'

HANDLERS = {
   'NotedHandler' : {
      'name':'Noted',
      'description':'Search my note directory',
      'categories': {
        'my notes': {
          'name': 'My Notes',
      }
    }
  }
}

So, what's changed here? Not much. We include a few more imports that will provide other python functionality that we'll need later. A global variable NOTEDIR is defined. This will tell the plugin where the files that should be grepped are stored. Eventually, we will examine how to make this a parameter that can be configured in the deskbar applet preferences dialog. Finally, the necessary changes are made to the HANDLERS variable. Note the addition of a new dictionary item: categories. This allows us to specify custom categories other than the ones provided with deskbar applet. The general format is 'categories' : { 'categorykey' : { 'name': 'Category Name' } } Since we want all of our grep results to be grouped together, we create a new category for them.

Creating the main handler: class NoteHandler(deskbar.Handler.Handler)

As we've already mentioned, each time you type (or update) text in the search box, the deskbar applet goes through the entire list of registered plugins, and calls Handler.query() for each plugin (so each plugin should have subclassed the deskbar.Handler.Handler class). So, let's look at how to modify the skeleton Handler class to provide our file searching functionality.

class NotedHandler(Handler):
   def __init__(self):
      Handler.__init__(self, 'text-editor.png')

   def initialize(self):
     # might one day do some file modification tracking
     # as an optimization
     pass

The first part is easy: we just name the class, and choose the icon that we want showing up next to any matches in the dropdown list. We don't have any initialization to perform, but we may want to do some indexing or filesystem monitoring to optimize behavior, so we define a stub intialization function to remind ourselves.

def query(self, query, max=15):
     # files to search
     candidates = glob.glob(NOTEDIR + '/*')

     # find filename matches
     matches = [c for c in candidates if query in c]

     # find file contents matches
     for c in candidates:
       if os.path.isfile(c)
       f = open(c)
       if re.search(query, f.read()):
         matches.append(c)
       f.close()
     print matches

     return [NotedMatch(self, file, query) for file in matches[:max]]

Finally, the good stuff! Again, the query variable passed to the query function is the text that was typed into the search box for deskbar-applet. Here we also introduce a new parameter for query: max. The max parameter allows you to specify how many matches should be returned at most.

The first step is to get all of the files in the directory we're searching. Next, we search filenames for the query in question. If any filenames contain the query, they're added to the results. Finally, we open each file, and search its contents for the query. If the contents contain the query, the file is added to the results.

Now, we have a list of strings containing matching filenames. The last step is to return the Match objects that will represent each of these files! Don't worry about the details of how the NotedMatch objects are instantiated... just notice that one is created for each of the filename matches.

Creating the match objects: deskbar.Match.Match

Now, we will investigate the creation of the Match objects. We're almost done! I don't know about you, but I think this was easy!

class NotedMatch(Match):
   def __init__ (self, backend, file, query, **kwargs):
      Match.__init__(self, backend, **kwargs)
      self.file = file
      self.query = query

First, we create the intialization portion of the match. The first parameter, backend, must always be present, and should be passed as the self object of the Handler that created the match. Even if you don't use it, it will be passed to the superclass's __init__ function. We pass two parameters of our own custom interest: file, which is the filename of the file containing the query that was made.

def get_verb(self):
      return 'Edit %s' % (self.file)

   def get_category(self):
      return 'my notes'

Next, we create the informational get_verb and get_category. These behave just as explained above. Remember that we created our own category with the key of 'my notes', so that category will be used for all of these matches.

With these portions completed, we already have a pretty functional plugin! It will tell us, in the dropdown list, up to 15 files that contain the query that the user typed into the deskbar-applet search box! But, wouldn't it be easier if we could easily open the file? Well, that's the whole point of the drop down menu! So, let's implement our FINAL piece of code:

def action(self, text=None):
     editors = ['gvim -c "/%s"' % self.query,
                '$VISUAL',
                'gedit']

     for ed in editors:
       if os.system(ed + ' ' + self.file) == 0:
         break

This is the code that is executed when you click on the line in the dropdown box associated with this match! As you can see, it simply opens the file in either gvim, the editor specified in $VISUAL environment variable, or, as a fallback, gedit. We like gvim, because you can open the file with the query actually highlighted!

Eventually, this choice will also be moved to the Preferences GUI configuration tool, allowing the user to specify any editor command! But that is for a later day.

Congratulations! You've written a deskbar applet handler!

Summary: Overview of components:

HANDLERS variable

• A dictionary that describes your plugin.

deskbar.Match.Match

• get_verb - This function returns the text that should be presented in the drop down list that corresponds to a particular match object.


• get_category - This function returns a category name, specifying where the Match should appear in the list.


• action - This function performs a series of actions that the user is expecting, having clicked on the Match in the deskbar-applet dropdown list.

deskbar.Handler.Handler

• __init__ - This is only called when the plugin is loaded! You should call the deskbar.Handler.Handler.__init__ function here, passing the icon file that you'd like to use.


• initialize - This is called after __init__ to set up the plugin. Remember, the deskbar-applet will not be ready until all initialize sequences are complete, so it shouldn't take too long!


• query - This function is called everytime the text in the search box is updated. The function should read the text, decide if it's interesting, and if so, return one or more Match objects. This is called a lot, so it should be tightly coded.

Part Two: Adding Configuration capability.

So, we have a nice fancy file searching plugin. But, there's still a problem... the directory that we're monitoring is hard-coded into the plugin. This is not a nice solution. We could try using environment variables, but a nicer solution would be to allow the user to configure the directory from the deskbar-applet.

When you go to the Deskbar Applet preferences dialog, you will notice that for some handlers, the "More..." button is activated, and can be clicked. Adding this sort of functionality to your handler is very easy. The only deskbar-related action that you need to take is to add an entry to your HANDLER dictionary entry, and two associated functions. First, let's look at the needed change to HANDLER:

GCONF_NOTED_PATH = deskbar.GCONF_DIR+"/noted/path"

HANDLERS = {
   'NotedHandler' : {
      'name':'Noted',
      'description':'Search my note directory',
      'requirements': _check_requirements,
      'categories': {
        'mynotes': {
          'name': 'My Notes',
      }
    }
  }
}

Notice the new 'requirements' entry in the dictionary. This entry simply provides a pointer to a function that the handler uses to decide if it needs to be configured, or if it can be configured. So, the next step is to implement this function.

Notice that we also create the GCONF_NOTED_PATH variable, which points to the location in the gconf database of our noted configuration.

def _check_requirements():
        #We need the user to choose a directory to monitor
        if not deskbar.GCONF_CLIENT.get_string(GCONF_NOTED_PATH):
                return (deskbar.Handler.HANDLER_HAS_REQUIREMENTS, _("You need to choose a directory for Noted to search."), _on_config_account)
        else:
                return (deskbar.Handler.HANDLER_IS_CONFIGURABLE, _("You can change the directory that Noted searches."), _on_config_account)
 

So, what's going on here? Well, the function that you place in the 'requirements' item of your HANDLERS dictionary entry should return one of two symbols if your handler is configurable. The first return option, deskbar.Handler.HANDLER_HAS_REQUIREMENTS tells deskbar-applet that you can not run this plugin until it is configured. The other option, deskbar.Handler.HANDLER_IS_CONFIGURABLE, tells deskbar applet that your handler is ready to be run, but that the user has the option of changing some settings. Both of these options will cause the "More..." button in the preferences dialog to be activated. The text string returned is displayed to the left of the "More..." button in the preferences dialog. Finally, the third argument should be a function that is called when the "More..." button is pressed.

So, our next step is to implement the configuration action handler.

def _on_config_account(dialog):
        dialog = gtk.Dialog(_("Noted Configuration"), dialog,
                                gtk.DIALOG_MODAL | gtk.DIALOG_DESTROY_WITH_PARENT,
                                (gtk.STOCK_CANCEL, gtk.RESPONSE_REJECT,
                                gtk.STOCK_OK, gtk.RESPONSE_ACCEPT))

        table = gtk.Table(rows=2, columns=2)
        table.attach(gtk.Label(_("Choose a path for Noted to search:")), 0, 2, 0, 1)
        user_entry = gtk.Entry()
        t = deskbar.GCONF_CLIENT.get_string(GCONF_NOTED_PATH)
        if t != None:
                user_entry.set_text(t)
        table.attach(user_entry, 1, 2, 1, 2)   
        table.show_all()
        dialog.vbox.add(table) 
        response = dialog.run()
        dialog.destroy()

        if response == gtk.RESPONSE_ACCEPT and user_entry.get_text() != "":
                deskbar.GCONF_CLIENT.set_string(GCONF_NOTED_PATH, user_entry.get_text())

All this function does is create a single GTK dialog with a text entry box, and OK and CANCEL buttons. For more details about creating your own GTK dialog, you should refer to a GTK tutorial . The important thing to notice here is that our dialog gets and sets the gconf variable GCONF_NOTED_PATH that we defined previously.

The Deskbar Applet specific development has been done! All that we need to do now is modify the Handlers that we defined earlier to use our gconf variable. All we need to do is add one line to our query function!

def query(self, query, max=15):
     ####HERE'S THE NEW LINE
     NOTEDIR = deskbar.GCONF_CLIENT.get_string(GCONF_NOTED_PATH)

     # files to search
     candidates = glob.glob(NOTEDIR + '/*')

Ok, we're all done! Now the Noted handler will search all files in the directory that you configure via the GUI!

Getting Started With QualNet

QualNet is a commercial network simulator produced by Scalable Network Technologies, Inc. It is a commercialized version of GloMoSim, but it is completely C-based, so it does not use GloMoSim's specialized "ParSec" language. QualNet includes a graphical "scenario creator" tool, but it's often easier to simply specify qualnet configuration files by hand, and it's certainly much faster, once you're familiar with the parameters.

Presentation from Spring 2007
Installing in Linux

I'll assume that you've gotten the Qualnet tarball from somewhere. Put it somewhere (for example, your home directory on your research lab's computing machine that has a license for QualNet available). Unroll the file using: tar -xzf qualnet.tgz (or whatever it's called)

You need to set a few environment variables, and update your path:

export QUALNET_HOME=~/qualnet/3.9.5 (assuming you unzipped the tarball in your home directory)

export PATH=$PATH:$QUALNET_HOME/bin:$QUALNET_HOME/gui/bin

You should add these statements to your .bash_profile so you don't have to type them every time you log back in.

Next, you have to compile the application. This is easy. From the documentation:

For a machine running gcc 3.2 and higher:

cd $QUALNET_HOME/main

cp Makefile-linux-glibc-2.3-gcc-3.2 Makefile

make

It will chug away. When it's done, you're ready to run qualnet.

Finally, copy the license file to $QUALNET_HOME/license_dir

To test that it works:

cd $QUALNET_HOME/bin

qualnet default.config

Some numbers should fly by.

Getting up and running with QualNet is fairly easy. A basic simulation is run simply by creating a number of configuration files. The configuration files are declarative, rather than script-based. You simply set up a scenario by defining a bunch of variables, and then let it rip using the command:

qualnet <name>.config

The best documentation for these files is in the qualnet installation. Look in qualnet/3.9.5/bin, and you will see a bunch of files using the naming scheme default.* (for example, default.config, default.nodes). These files are well-commented, and will serve as a good guide, and starting point.

For a basic simulation, you'll probably want at least these files:

* .config
* .nodes
* .app

Note that you can actually call these files ANYTHING you want, but these conventions make it easy to remember which files do what.

There are a HUGE number of other configuration files, controlling things like background traffic, antenna models, weather, link faults, and so on. If you want more info on these, check out the files in the directory! They are well-commented.
Configuration Files
<name>.config

This is the core QualNet scenario configuration. Let's look at some of the core configuration parameters in this files. This document will only cover BASIC configuration parameters for running a simple wireless experiment. You should copy the default.config file, because most of the default parameters for MAC/PHY settings in this file are good for beginning. If you want to modify them later, you can view comments in the file for more information.

* EXPERIMENT-NAME <name>

This one should be straightforward enough. Call it whatever you want.

* SIMULATION-TIME <time>

This specifies how long the simulation should run. The default.config file explains the different time values that can be used. For example, 15M = 15 minutes, 100S = 100 seconds.

* SEED <number>

A random seed for experiment repeatability.

* COORDINATE-SYSTEM CARTESIAN

Self-explanatory. Others exist, but this is probably what you want, as a beginner.

* TERRAIN DIMENSIONS (<x>,<y>)

Sets the size of the terrain. X and Y are in meters.

* SUBNET N<subnet-specs> { node names }

This option is fairly complicated. For the early simulations, you probably want to have a single subnet of nodes, and the interface addresses the same as the node ID. To acheive that, for say, 10 nodes, use:
@@SUBNET N16-0 { 1 thru 10 }

* LINK N<subnet-specs> { node A, node B}

Create a *wired* link between two nodes. This complicates the setup a bit, so if you need wired links, examine default.config file for more details.

* NODE-PLACEMENT [UNIFORM|RANDOM|FILE|GRID]

Describe how nodes are placed in the simulation terrain.

* GRID-UNIT <num>

Grid spacing if GRID placement is used.

* NODE-POSITION-FILE <filename>

Usually name.nodes, see below for description of the file.

* MOBILITY [NONE|RANDOM-WAYPOINT|GROUP-MOBILITY|FILE]

Group mobility is an advanced option, we'll ignore it for now. If FILE is specified, then NODE-POSITION-FILE is used.

* MOBILITY-POSITION-GRANULARITY <num>

Specify how fine-grained movement steps should be.

* ROUTING-PROTOCOL <protocol>

Specifies which protocol should be used in this simulation. AODV or DSR, for example, or one you create yourself.

* APP-CONFIG-FILE <filename>

This specifies the file that contains the applications to be run in your scenario

* Statistics

You want to collect information from your simulation of course! There are quite a few options. You can pick and choose which traces and statistics you want to output when you run your simulation. Look for "HOST-STATISTICS" in the default.config file to see what your options are. Look for "PACKET-TRACE" in the file for more info about tracing. A nice feature of QualNet is that it will output quite a few basic statistics without requiring you to save or parse huge traces.
<name>.nodes

This file specifies node position, and mobility if used. The file is very straightforward, it is simply a series of lines of the form:

<node> <time> (<x>,<y>,<z>) <theta,phi>

For basic 2D experiments, you can set z to zero, and omit theta and phi.

The entries must be sorted in increasing time order. The simulator will do linear interpolations between two points in time.
<name> .app

This file specifies the applications to run. It is of the form:

<appname> <params>

The parameters will vary depending on the application. All of the apps and paramters for them are described in default.app. A basic option is CBR (constant bit rate). It is:

CBR <src> <dest> <items to send> <item size> <interval> <start> <end>

If <items to send>=0, the CBR client will simply send for the whole specified time interval.

CBR 1 2 0 512 1 10 1000

Send 512 byte packets from node 1 to 2 ever second for 990 seconds, starting at 10 seconds into the sim.

Palm OS Flaws

I've been using the PalmOS since about 1999. It's amazing how much it's stayed the same in the past 7 years. In some ways, this is a good thing. Conceptually, PalmOS is very well designed for the target device. For its original PIM tasks, it's quite well designed. But 7 years later, I expect to see notable improvements, and I just haven't. So here's the run down, in no particular order:

1. The UI appearance is outdated. Many people will argue that this is irrelevant. These are the people that use TWM and Ratpoison as Window Managers in Linux. I whole-heartedly disagree that a nice, pleasant appearance is not an important part of a computing experience. It is possible to make something aesthetically pleasing without hurting performance. It was one thing when the Palm screens had a 2-bit depth, but with 16-bit color, I really expect a bit more.

2. Unevolved PIM features. Is this a business move, so that companies like DateBk? or Iambic can stay in business? The Palm PIM features have not changed _at all_, for the most part. Functionality like hierarchy in Todo Lists, linking of items, and better per-diem planning should be *integrated parts* of the system.

3. A horribly crappy syncing system. At one point, this was OK. When my palm connected via serial port, the behavior was more understandable. But now, 7 years later, they still essentially simulate that same protocol over a USB port. The whole point of USB is to make devices smarter, and allow the computer to automatically detect them. It is INEXCUSABLE that I still have to press a button on my palm in order to sync it. Really, people. Come on. Furthermore, network syncs simply don't work. The protocol is closed source, so Linux support is slow to come, and buggy. The only saving grace is that I still prefer this to a Windows CE device.

4. The ability to crash the system hard. I've already experienced this a few times. It's one thing to install a program that makes the palm behave poorly, or that causes it to reboot. However it's poor design for an operating system to be irreversibly put out of commission due to an installation. I've experienced the "reboot-loop" problem, where the device just reboots before it comes back into function fully. At the very least, Palm should include a FAILSAFE mode that allows you to delete certain items or at least back up your data before doing a hard reset. This feature combined with the inability for automatic un-monitored syncing (as mentioned in 3) really put a damper on my excitement about the device.

Dancing on the Diodes

I probably should not admit it, but I do like to dance. Coupled with the fact that I am a geek, this disco-floor project at MIT really piqued my interest!