Featured Posts

SciPy 2008 - Software & Tools STATUS: Day 1 - Thursday: Concluded. Day 2 - Friday: In progress. Astronomy: Enzo [ home ] yt (related to Enzo) [ home | scipy ] Books: Computational Modeling and Complexity Science...

Readmore

An example of checking out a git tag using Qt 4.6.0... Clone the Qt repository: $ git clone git://gitorious.org/qt/qt.git Show available tags: $ git tag -l output: v4.5.1 v4.5.2 v4.5.3 v4.6.0 v4.6.0-beta1 v4.6.0-rc1 v4.6.0-tp1 Checkout v4.6.0: $...

Readmore

Building 64-bit/32-bit Debug&Release Universal Build... See my directions on checking out Qt 4.6.0 from the git repository if you do not already have the source code. Launch a Terminal and cd to the source directory. Run configure w/ 64-bit & 32-bit...

Readmore

  • Prev
  • Next

SciPy 2008 – Software & Tools

Posted on : 21-08-2008 | By : Brandon W. King | In : Python, Software Development/Engineering

Tags: , ,

0

STATUS:

  • Day 1 – Thursday: Concluded.
  • Day 2 – Friday: In progress.

Astronomy:

Books:

  • Computational Modeling and Complexity Science [ home ]
  • And more at Greentea Press [ home ]

Code Generation & Wrapping:

Data IO:

  • PyNIO (NetCDF, HDF 4, etc.) [ home ]

Distributed Computing:

  • IPython1 merged back into IPython! [ home ]

Documentation:

  • SciPy Documentation Editor [ home | scipy | slides ]
  • Sphinx – Python Documentation Generator [ home ]
  • matplotlib solves the riddle of the sphinx [ scipy | slides ]

Graphs:

Image & Video Processing:

  • LIBCVD (Cambridge Video Dynamics Library) [ home ]

Math:

Visualization:

Django App + mod_fcgid + Apache 2 Setup on Mac OS X & Debian/Ubuntu Linux

Posted on : 31-01-2008 | By : Brandon W. King | In : Computers/IT, Error Fixes, Mac OS X, Systems Administration

Tags: , , , , , , , , , ,

0

Update 2008Feb11:

From rather obscure comment from ‘apt-cache show python-flup’ on a Debian system, I discovered that flup has been superseded by http://www.modwsgi.org/. It is probably worth checking out mod_wsgi instead of mod_fcgid for Python web applications. Based on the documentation for mod_wsgi integration with Django, I am planning on switching. I will post my results in a future post.

Required Software

mod_fcgid

Download Source and Install

For reference, here is the mod_fcgid INSTALL.txt… see my Mac OS X notes below it for changes required to get it working on Mac OS X 10.5 (Leopard):

NOTE: This module is for Apache2 ONLY

UNIXIt's tested on my RedHat8 and Solaris. But it should work on other *NIX platform.NOTE: This module MUST run on share memory supported system

1. If your Apache2 installation isn't in /usr/local/apache2, please edit Makefile and correct it2. cd $mod_fcgid_dir3. make   //in Mac you need Xcode tools, optional install on Install CD4. make install5. add the following line in httpd.conf

LoadModule fcgid_module modules/mod_fcgid.so

MAC OS X Notes

  • In Mac OS X Leopard, the Makefile needs to be changed from /usr/local/apache2 to /usr/share/httpd, then steps 3 and 4 above should work.
  • Step 4 requires: sudo make install
  • Step 5, you need to add the following to /etc/apache2/httpd.conf: LoadModule fcgid_module libexec/apache2/mod_fcgid.so
  • IMPORTANT: apache 2 on Mac OS X Leopard is 64-bit and by default, the make file does not build the 64-bit module. To do this, you need to add the following to the Makefile after the EXTRA_CFLAGS option (too far above that will cause it not to work apparently):
CFLAGS = -arch ppc -arch ppc64 -arch i386 -arch x86_64

The above will make a universal binary for all Mac OS X.

If at some point, you get this message either in the error_log or by running ‘apachectl configtest’

httpd: Syntax error on line 117 of /private/etc/apache2/httpd.conf: Cannot load/usr/libexec/apache2/mod_fcgid.so into server:dlopen(/usr/libexec/apache2/mod_fcgid.so, 10): no suitable image found.Did find: /usr/libexec/apache2/mod_fcgid.so: mach-o, but wrong architecture

Then your mod_fcgid.so module probably was not built with x86_64 architecture… To check, type

file /usr/libexec/apache2/mod_fcgid.so

And it should list the architectures that the mod_fcgid.so was built for.

Enable mod_fcgid

Linux

At least with Debian/Ubuntu you can enable fcgid by typing (as root):

a2enmod fcgid

Mac OS X Leopard (10.5)

Add the following to /etc/apache2/httpd.conf:

AddHandler    fcgid-script .fcgiSocketPath    /tmp/fcgid_sockSharememPath  /tmp/fcgid_shmIPCConnectTimeout 20

mod_rewrite

We need to tell apache for what urls it should pass off the request to the fcgi script. This is accomplished with mod_rewrite which is normally turned on by default w/ apache 2 (at least the installations I have seen).

We’ll start with the full setup required for gaworkflow.frontend, but test it with a “Hello World” style dispatch.fcgi script.

gaworkflow.frontend apache2 mod_rewrite setup

Linux

Add the following to your apache2 /etc/apache2/sites-available/default or other vhost specific file:

# Enable access to the django admin mediaAlias /media /usr/lib/python2.5/site-packages/django/contrib/admin/media

# Turn on the rewrite engineRewriteEngine on

# Enable http:///admin/ and related urlsRewriteRule ^/admin(.*)$ /usr/lib/cgi-bin/dispatch.fcgi/admin$1 [QSA,L]RewriteRule ^/logout(.*)$ /usr/lib/cgi-bin/dispatch.fcgi/logout$1 [QSA,L]RewriteRule ^/login(.*)$ /usr/lib/cgi-bin/dispatch.fcgi/login$1 [QSA,L]

Mac OS X Leopard

Add the following to your apache2 /etc/httpd.conf:

# Enable access to the django admin media# NOTE: the following did not work for me... I had to copy the media directory to#    /Library/WebServer/Documents/media#Alias /media /System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/django/contrib/admin/media/

# Turn on the rewrite engineRewriteEngine on

# Enable http:///admin urlsRewriteRule ^/eland_config(.*)$ /Library/WebServer/CGI-Executables/dispatch.fcgi/eland_config$1 [QSA,L]RewriteRule ^/admin(.*)$ /Library/WebServer/CGI-Executables/dispatch.fcgi/admin$1 [QSA,L]RewriteRule ^/logout(.*)$ /Library/WebServer/CGI-Executables/dispatch.fcgi/logout$1 [QSA,L]RewriteRule ^/login(.*)$ /Library/WebServer/CGI-Executables/dispatch.fcgi/login$1 [QSA,L]

Restart Apache 2

Linux (Debian/Ubuntu)

sudo /etc/init.d/apache2 restart

Mac OS X Leopard

sudo apachectl restart

dispatch.fcgi

Hello World Test

Now to test that we have the mod_fcgid setup properly, create a file called dispatch.fcgi:

#!/usr/bin/python

def myapp(environ, start_response): start_response('200 OK', [('Content-Type', 'text/plain')]) return ['Hello World!\n']

if __name__ == '__main__': from flup.server.fcgi import WSGIServer WSGIServer(myapp).run()

And move it to your cgi-bin directory:

Linux cgi-bin

sudo mv dispatch.fcgi /usr/lib/cgi-bin/

Mac OS X Leopard cgi-bin

sudo mv dispatch.fcgi /Library/WebServer/CGI-Executables/

Linux/Mac OS X File Permissions

sudo chmod a+x /dispatch.fcgi

Mac OS X 10.5 users will need to update the Options for to include +ExecCGI… by default it is set to “Options None”. The updated entry should look like:

 AllowOverride None Options +ExecCGI Order allow,deny Allow from all

WARNING: You will get a message saying “Forbidden” in the web browser if you do not update the directive above.

Web Browser Test

Point your web browser to http://localhost/admin/ and you should see if it prints “Hello World!”. If you see “Hello World!” continue to the next section.

Installing the Django App dispatch.fcgi script

copy of mydjangoapp code

Make a copy of the mydjangoapp code and database and put it in a location that will be the “live” version of the code/database. On Mac OS X, I choose /Library/WebServer/mydjangoapp. Initialize the database like your normally would, but you will need to update the settings.py module so that DATABASE_NAME is an absolute path, otherwise you will get errors in the apache 2 error_log saying that the python code could not connect to the database. Assuming you also choose /Library/WebServer/mydjangoapp, your DATABASE_NAME variable in settings.py should be:

DATABASE_NAME = '/Library/WebServer/mydjangoapp/mydjangoapp.db'

You will also need to update the settings.py TEMPLATE_DIRS to be an absolute path (relative paths don’t work for some reason) of “/Library/WebServer/gaworkflow/templates”… so the update version should look like:

TEMPLATE_DIRS = ( "/Library/WebServer/mydjangoapp/templates",)

Also, you will need to make a link to the admin templates in our /Library/WebServer/mydjangoapp/templates directory by typing:

Mac OS X 10.5:

ln -s /System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/django/contrib/admin/templates/admin /Library/WebServer/mydjangoapp/templates/adminln -s /System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/django/contrib/admin/templates/admin_doc /Library/WebServer/mydjangoapp/templates/admin_docln -s /System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/django/contrib/admin/templates/widget /Library/WebServer/mydjangoapp/templates/widgetln -s /System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/django/contrib/admin/templates/registration /Library/WebServer/mydjangoapp/templates/registration

Warning: The following directions will tell you how to get past the db read/access errors but may not be the best choice for security… you consider the possible security issues before following the following instructions.

To get the dispatch.fcgi to work properly, I needed to change ownership of /Library/WebServer/mydjangoapp to be owned by the apache2 user, which is _www on Mac OS X 10.5 and www-data on Debian/Ubuntu systems. The the apache user also needs access to the mydjangoapp.db as well. I ran the following commands on Mac OS X 10.5 when placing the files in /Library/WebServer/gaworkflow:

sudo chown _www:_www /Library/WebServer/mydjangoappsudo chown _www:_www /Library/WebServer/mydjangoapp/mydjangoapp.dbsudo chmod o-rwx /Library/WebServer/mydjangoapp/mydjangoapp.dbsudo chmod ug+rwx /Library/WebServer/mydjangoapp/mydjangoapp.db

new dispatch.fcgi ¶

Replace the hello world dispatch.fcgi with the following script:

#!/usr/bin/python                                                            import syssys.path += ['/Library/WebServer/mydjangoapp']from flup.server.fcgi import WSGIServerfrom django.core.handlers.wsgi import WSGIHandlerimport osos.environ['DJANGO_SETTINGS_MODULE'] = 'mydjangoapp.settings'WSGIServer(WSGIHandler()).run()

You will need to update these two rows:

  • sys.path += [‘/Library/WebServer/mydjangoapp’]
  • os.environ[‘DJANGO_SETTINGS_MODULE’] = ‘mydjangoapp.settings’

Where ‘/Library/WebServer/gaworkflow should be replaced by PYTHONPATH that would allow your Django package to be imported… one directory level below the directory containing init.py.

NOTE: Make sure you dispatch.fcgi is executable:

sudo chmod a+x dispatch.fcgi

If everything worked out properly, you should have a working installation of gaworkflow.frontend using mod_fcgid.

Trouble Shooting

  • Check apache error logs.
  • run: sudo apachectl configtest
  • Check that dispatch.fcgi has #!/usr/bin/python and not #!/usr/bin/env python as that will screw up the environment variables at least w/ Apache 2 that ships on Mac OS 10.5.
  • run ‘python dispatch.fcgi’ to see if you get any standard Python errors.

Apache2 – error_log — [warn] mod_fcgid: stderr: OperationalError: unable to open database file

This means the database is not accessible by the user that is running apache. On Mac OS X 10.5, the user is _www. On Debian/Ubuntu Linux the user is www-data. See the “copy of gaworkflow code” section above for directions on changing file permissions for the database.

bio.scipy.org

Posted on : 18-09-2007 | By : Brandon W. King | In : Bioinformatics, Python, Science, Software Development/Engineering

Tags: , , ,

0

As I said I would do, I am now in the progress of setting up and seeding the biology in python community site, which can be found at bio.scipy.org. Part of the goal, as set at the SciPy 2007 Biology Birds of a Feather (BoF) meeting (organized by Titus Brown), is to establish a community site where anyone using Python in the field of Biology can come together, communicate, share ideas, and share code.

As part of the programming guide, I posted my “Preparing to learn python guide“, in which, I mention the things about Python I wish I had known when I started learning Python. I am guessing it will be most useful to those who have programmed before in another language and are trying to learn Python.

Also, I posted a link to Software Carpentry in the programming guide. Thanks Greg for starting that project!

SciPy 2007 – Biology Birds of a Feather meeting

Posted on : 17-08-2007 | By : Brandon W. King | In : Bioinformatics, Python, Science, Software Development/Engineering

Tags: , , ,

0

During the Biology birds of a feather meeting at SciPy 2007, which was organized by Titus Brown, we meet to discuss were those of us using Python for biology/bioinformatics should work together as a community. Two interesting thoughts came up:

  1. Need to establish python/biology community, via website, biology-in-python mailing list, rss, blogs, etc.
  2. Having a core set of “interfaces” for handling basic bioinformatics objects would allow independent projects to share these basic objects. I am sure others will describe this better and in more detail in the near future.

I personally believe that even if we only accomplish the first item, it would significantly help the bioinformatics community. I do hope the 2nd is accomplished as well.

What I see is that most people who are new to the python bioinformatics community end up finding BioPython. The only problem is that many people doing bioinformatics with Python who choose not to use BioPython end up doing there own thing an not communicating with the community. The idea here is to make a new python/biology community site were we can communicate, share code, share ideas, etc. In other words support the community as a whole rather than just having communities for individual projects.

I agreed to setup this initial site and seed it information, bioinformatics python packages by category, a place holder for cookbooks, advice for testing and software engineering (referencing Greg Wilsion’s Software Carpentry material, etc.). I would like to thank Enthought as they have agreed to host the website. I will add the more information on the community site as it is being setup. We’ll see how it goes.

Also, check out Titus Brown’s Birds of a Feather post.