So what can you do with FS anyway?

September 28th, 2008

After my last post regarding FS, my file-system abstraction layer for Python, I think I may have left people thinking,"that's nice, but what would you use if for?". Generally, I see it as a way of simplifying file access and exposing only the files you need for your application -- regardless of their physical source. But I can think of a few other uses that may be a littler cooler.

Instant Archive Format

The osfs.MemoryFS class is a filesytem that exists only in memory. You can create files / directories and copy files to it from other filesystem objects, but when you open a file you will get back a StringIO object (from the standard library). Naturaly the files in a MemoryFS are transitory, but as everything is in memory, file access is very fast. I was thinking that since the files and directories are stored as simple Python objects, then a MemoryFS can be pickled -- creating an instant archive format. Uncompressing this archive would simply require unpicking and using the resultant object. I can't see any advantages of this over zip or tar archives, but if you don't have them available, this would be a workable substitute!

Database Filesystem

It wouldn't be particularly challenging to write an FS an object that created a filesystem on a database. Actualy, if using an ORM like SQLAlchemy, SQLObject, Storm or Django's database layer it would be fairly trivial. So what would you use such a thing for? I can think of one use which may make it worth the effort, and that is the ability to store templates for a Python web application in the database, rather than (or in addition to) files. Consider if you will, a situation where the client has requested a change that is absolutely critical to the success of his company and he won't take "it will be in the next release" for an answer, and neither will your project manager. Wouldn't it be nice if you could go in to the Django admin pages and just type in the changes, without making a new release. I'm tempted to implement such a thing, if nobody has done so already!

Proxying a Filesystem

A generic filesystem server could be written that exposes a remote file-system over whatever channel is available. I know there are some standard ways of doing this, but they might not be available on a given platform. I'm thinking of small devices with limited capabilities, or secure environments. The server would serve an FS object, and the client would present the app with a corresponding interface that pulled files and directories over the wire.

There you go. If anyone implements these, let me know!


Announcing FS 0.1.0, a Python file-system

September 21st, 2008

This is something I have been hacking together for a while now; FS is a file-system abstraction for Python. It has reached a stable state and is worthy of an official (0.1.0) release.

First, a brief history of the project. A while back, I was working mainly with desktop applications in wxPython. I found that I had a number of sources for files; there were read-only resources such as images, per-user files and per-installation config files. The location of these files would change depending on whether I was debugging or building a release version. The logic required to manage all this was pretty ugly and error-prone. So I wrote a collection of classes to bring all these disparate locations for files under a single virtual file-system. For example, to open a config file I could just read 'user/settings.cfg', and to open an image resource I could just read from 'resources/logo.png' and the virtual file-system would do the right thing and return a file-like object.

This turned out to be insanely useful, and I used it for a number of projects. I also used it while working for ICC,  and I had the opportunity to enhance it based on feedback from colleagues. In the last few months I have re-written it from scratch, because I wanted to avoid any copyright issues, but mainly because I could make a better job of it the second time around.


Running out of pie puns

September 20th, 2008
Dark Pie 2

To compensate for the previous post, here is a picture of something pretty. Work on my pretty-charts (working title) web application is progressing slowly but steadily. :-)


Photo Meme

September 20th, 2008
Will McGugan

Instructions: Take a picture of yourself right now. Don't change your clothes, don't fix your hair - just take a picture. Post that picture with NO editing. Post these instructions with the picture.

From a post by Steve Holden.

Who started this anyway?

Colour manipulation in Django

September 20th, 2008

A well-designed website will tend to have a consistent colour scheme where the various design elements sit harmoniously together, yet contrast well enough  that the user can separate them visually. Selecting these colors is a mixture of an art and a science. Its an art because only a human being can know if the colors project the image required by the site (business-like, mellow, funny etc), but it's also a science because complimentary and contrasting colors can be produced from a source colour using only math. As an engineer the math part appeals to me, but the art part "confuses and enrages me" (guess the quote).

A site will tend to have a few core colours, and additional colours that are deduced from the originals. For instance, you might want a pastel shade of one of the core colours as a background and a darker shade to use as a border. You may also want to blend colors together to create soft gradients. Implementing such color schemes with Cascading Style Sheets can be a pain because the colors are repeated throughout the file, and if you want to change a colour -- or all of them -- you need recalculate deduced colours and make numerous changes to the CSS. For someone like me, who doesn't have an intuitive sense of what looks good, it can be a time consuming process to tweak colours through CSS. So naturally I chose to solve this with techie solution.

I hacked together a class called WebColor, which stores and manipulates colour values. Colours can be created from a variety of formats, here are a few examples that create pure green:

c = WebColor( 0.0, 1.0, 0.0 ) #RGB

c = WebColor( "#00FF00" ) # HTML style

c = WebColor("rgb(0,255,0)") # CSS style

c = WebColor.fromHSL(0.333, 1.0, 0.5) # From HSL (Hue Saturation Lightness)

This class was designed to be used in templates, if you print a WebColor object it will render an HTML style color (e.g. #00FF00). So in the context of a Django template you can do {{ my_color }} to produce #00FF00. I've also written a few Django filters to do some common operations on colours. The 'lightness' filter changes the lightness of a colour, and takes either the exact value as a parameter, or a percentage to modify the original. For example {{ my_color|lightness:".2" }} will create a color with the same hue, but with a lightness of .2. Alternatively {{ my_color|lightness:"50%" }} will create a colour that is half as bright as the original. There is also a 'saturation' filter which works in the same way, but adjusts the saturation of a colour. In addition to these filters there are two tags; 'HSL' which creates a colour directly from HSL values, and 'colorblend' which produces a colour that is part way between two other colours. See the code (link below) for more information.

The easiest way to create colour schemes with these filters / tags is to templatize your CSS files and supply the template with your core colours, possibly created with something like Agave, or Kuler. In the project I'm working on at the moment, I have a list of 3 colors called 'palette' and all the other colours used in the CSS are deduced from the palette. For example, the following creates a pleasing alternating background for rows in table:

.chart-grid .even


background-color:{{ palette.0|lightness:".94"|saturation:".1" }};


.chart-grid .odd


background-color:{{ palette.0|lightness:".93"|saturation:".1" }};


If you would prefer to serve the CSS statically, then you can always save the rendered version when you are satisfied with the colour scheme. Although calculating the colours dynamically can create some interesting possibilities; you could change your colour scheme depending on the time of day, or allow the user to set the colour scheme in the preferences for example.

Here's the code: The zip contains a couple of Python files that you can drop in to your project; '' contains the tags and filters, '' contains the WebColour class which is independent of Django and could be used in other frameworks.

The code was written rather hastily in an afternoon. I'd kind of like to work on it further and produce a project for it, but I don't have the time at the moment. My hope is that someone will use it for the basis of a more complete colour module that I can use in future projects. The code is released under my politeware license, which means you can use it for whatever purpose you like, as long as you say thanks!

P.S. If you are wondering why I spell colour inconsistently, it's because being British, I naturally spell it with a U, but I spell it without the U in code to be consitent with CSS.

Search for Posts
© 2008 Will McGugan.

A technoblog blog, design by Will McGugan