Where are those Print statements?

December 9th, 2012

When I'm debugging Python I tend to sprinkle my code liberally with print statements. Trouble is, I forget which files I added those print statements to and have to waste a few minutes tracking them down. Time which could be better spent doing pretty much anything else.

Not sure why I didn't think of this before, but I hacked together something that automatically prints the file and line of the print statement next to the output.

Here's the code:

import inspect
import sys
import os

class DebugPrint(object):
    def __init__(self, f):
        self.f = f

    def write(self, text):
        frame = inspect.currentframe()
        filename = frame.f_back.f_code.co_filename.rsplit(os.sep, 1)[-1]
        lineno = frame.f_back.f_lineno
        prefix = "[%s:%s] " % (filename, lineno)
        if text == os.linesep:
            self.f.write(prefix + text)

if not isinstance(sys.stdout, DebugPrint):
    sys.stdout = DebugPrint(sys.stdout)

To use it, save it as ‘debugprint.py’ then imported it somewhere. That's all you need to do. Works on Python 2.7 and probably other 2.X Pythons.

Here's what happens if you do it from the console:

>>> import debugprint
>>> print "Hello"
[<stdin>:1] Hello

For print statements in your app, you will get something like:

[foo.py:22] Hello

Dragon Fly

May 23rd, 2012

I spent a lot of the last summer trying to get a shot of a dragon fly, but could never find them at rest. So I was hardly expecting to come across one just chilling in Portmeadow, Oxford. It was a very cooperative subject, I don't even resent having to crawl over mud and cow shit to get these photos…

The beastie below is a Libellula depressa, or a broad-bodied chaser dragonfly to you and me.

Dragon fly

Broad-bodied chaser dragonfly - Libellula depressa

Surprised how docile it was. Didn't know I could just pick one up.

Dragon fly 2

Broad-bodied chaser dragonfly - Libellula depressa

I like this shot. The blue in the wing is a reflection of the sky.

Dragon fly

Broad-bodied chaser dragonfly - Libellula depressa

More pictures on my Flickr page.


Serve FTP with Python and PyFilesystem

April 25th, 2012

Ben Timby has committed code to PyFilesystem that lets you expose any filesystem over FTP. We've had the ability to serve filesystems over SFTP (secure ftp) and XMLRPC for a while, but plain old FTP was a glaring omission–until now.

You can serve the current directory programatically with something like the following:

from fs.expose.ftp import serve_fs
from fs.osfs import OSFS
serve_fs(OSFS('.'), '', 21)

The same functionality is also available to the fsserve command. The following is equivalent to the above code, but from the command line:

fsserve -t ftp .

You'll probably need root privileges (i.e. sudo) on Linux for these examples.

With the server running, you can browse the files on your home directory with an ftp client, or by typing “” in to your browser. Any of the other supported filesystems can be served in the same way.

FTP has been around since the dawn of the internet, so just about any network enabled device will be able to access files exposed this way. It's a great way of creating a gateway to other filesystems. You could expose files stored on Amazon S3 for example.

You'll need to check out the latest code from SVN to try this out.

Update: Ben has posted more about this.


Virtual currency site in testing phase

April 15th, 2012

I made currency site available for testing today. See my previous post for the back-story, but in essence currency site is a virtual currency platform.

I sometimes object to the word virtual in ‘virtual currency’. Most of the money I possess is not in any physical form; it's merely a number stored in a database somewhere – and transactions occur without any kind of physical objects changing hands. So the word ‘virtual’ seems entirely redundant, since there's is no qualitative difference between virtual and ‘real’ money I can see. The only difference is the level of trust in the system.

But I digress. Currency site is a platform for virtual currencies, in that it is up to the users to create and manage currencies. The site just provides the tools. What currencies are used for is irrelevant as far as the platform is concerned. It could be for a house of students to manage the housework, or for a community to exchange goods and services. Regardless of what a currency is used for, there has to be a certain amount of trust in the system. The platform has to be reliable, in that you shouldn't be able to create currency without a valid transaction. Currency site is centralised, which makes that requirement simpler–the system keeps track of how much is owned, in the same way you trust banks to keep track of the money in your accounts.

The second level of trust is in the creator of the currency. The creator of the currency has the extra responsibility of defining how much of that currency is available at any one time. This is done by minting new currency. For instance, if the provider creates a currency and mints 1,000,000 virtual bucks then only 1,000,000 will ever be available to other users. It could be owned by a single person, or by a million people owning one virtual buck. Alternatively, since all currencies are divisible by 100, it could be that 100,000,000 people own 0.01 virtual bucks (a virtual cent?). However it is distributed there will be no more than one million virtual bucks in existence unless more is minted. A record of the currency mints is public, as well as information about how much currency exists and how much is in general circulation, which allows regular users to keep an eye on how the currency is managed.

From a techy side, currency site wasn't all that challenging. Sure it was a few months of part time work, but it was mostly user interface code. I wanted to make something that worked like online banking, but not as painful (I've never used an online banking system that didn't make me want to tear hair out). That was helped by using Twitter's bootstrap CSS framework, which creates an elegant user interface with simple markup.

There was only one piece of code that wasn't a straightforward as it appeared (and it was kind of fundamental). A currency site transaction basically involves subtracting a value from the source account and adding it to the destination account. In psuedo code, it is simply this:

if source_account.balance < amount:
    raise TransferError("Not enough currency")
source_account.balance -= amount
destination_account.balance += amount

In essence, that's all that is done, but things get more complex in the context of a web application where multiple transactions may occur simultaneously. For example, if an account A contains 30 virtual bucks and the owner attempts to send 20 virtual bucks to account B and simultaneously sends 20 virtual bucks to account C, one of those transactions has to fail – otherwise we may end up with a negative balances which is not allowed. The if statement checks if the account has enough currency, but if both those transactions occur simultaneously then they will both subtract 20 from the source account (leaving -10). Granted, this could only occurs in a small window of time, but there is no way to recover if it does.

I couldn't figure out how to handle this situation elegantly with the Django ORM, and I don't like resorting to custom SQL. Fortunately, the recent release of Django 1.4 came to the rescue with the addition of select_for_update, which does row level locking. Basically it allowed me to lock the two accounts objects so that no other process is permitted to modify them until the currency has been transferred. Another consideration is that the entire thing has to be done in a (database) transaction, since half a (currency) transaction could result in currency being subtracted from the source account without being added to the destination (in effect, disappearing currency from the system). To keep the currency consistent, the psuedo-code becomes:

lock(source_account, destination_account)
if source_account.balance < amount:
    raise TransferError("Not enough currency")
source_account.balance -= amount
destination_account.balance += amount

I don't think there is much else in the code that is blog-worthy, although there is still plenty of features I'm thinking of adding. I'm considering allowing users to trade currencies, which might be interesting. I would also like to build an API, so users could pay for web content with virtual currency. A few more evenings of hacking in there I think…

If you would like to help with the testing then head on over to http://currency.willmcgugan.com/ (username: currency, password: reliance). If you let me know, I'll send you 100 beta bucks for your time. Bear in mind its in an early testing phase, so don't use it for anything serious – I'll be wiping the database before it goes live. I'd also be interested in suggestions for a proper domain name!


Making Money with Python

March 17th, 2012

A while back a friend told me about something called community currency, also know as Local Exchange Trading System. The basic idea of which is that people within a geographical area can exchange goods and services with bespoke unit of exchange rather that traditional cash. So, for instance, you could mow a few lawns in exchange for guitar lessons – even if it isn't the guitar teacher's lawn you are mowing. There's no physical currency as such, members of the community currency rely on volunteers to keep track of how much currency they own. I think this is a marvellous idea. It promotes healthy exchanges without the need to muddy things with something as vulgar as cash. But what struck me after a bit of research is how the whole system is in dire need of mechanisation! There's no centralised place to view your ‘account’ or way to do transactions online, and I figured there should be. So that has been my hobby project for the last few months, I've been building such a site which has recently come together to a point where I'd like to gauge how much interest is out there. I haven't even come up with a name yet, so I've been calling it by the rather uninspired moniker of ‘Currency Site’.

Apologies for the misleading title of this post. I am without shame.

Users of Currency Site can create a currency which they can use to keep track of any kind of debt. The currency creator (or provider) sets the policy on how new money is created and managed. Once created, money can be sent to other users (directly to a username or indirectly via an email address), and users can mange their funds by creating various accounts. Once funds have been sent to a user, the provider has no more control, as any user is free to store their funds or send them to others. For all intents and purposes, Currency Site is like online banking, although with a far nicer user interface than any online banking system I have ever used (which tend to be a usability minefield). Users are also able to see how much money has been minted and how much is currently in circulation (i.e. not stored by the provider), which helps to maintain trust in the system.

You may be thinking this sounds familiar if you have ever come across the Bitcoin project, but there are a few significant differences. The biggest difference is that Currency Site still requires trust in the individual or organisation that is managing the currency (i.e. the provider), and there is no enforced scarcity of new currency; providers can mint new money as they see fit. There is a little overlap, but the use cases for Currency Site are potentially broader (albeit limited in scale) compared to Bitcoin. Community currency projects are what I had in mind when working on this, but it is equally applicable for a variety of other uses. For example, lets say a family (we'll call them the the Smiths) have a few kids that don't like doing their chores, so the parents create a currency called ‘Smith Dollars’. When little Bobby Smith does his homework or cleans up his room, his parents send him 10 Smith Dollars. When Bobby has 100 Smith Dollars, he can cash them in for a new video game or spend 15 to stay up an extra hour. But if he wanted to, he could also send his sister 5 Smith Dollars in return for a loan of her laptop. Other uses could be employees keeping track of who goes for the doughnuts or a couple exchanging favours (use your imagination for that one).

Currency Site is built with Django and I've used the excellent Bootstrap library for the theme. The site is usable at the moment, but there are still a few things I want to do to it before I push it live anywhere. Just to prove it's not vaporware, here are a few screenshots:

Screenshot of currency site

Screenshot 1

Screenshot of currency site

Screenshot 2

Screenshot of currency site

Screenshot 3

I will be looking for a few brave souls to help me test this. I plan to do a private beta where the database will be completely wiped before it goes live for a while. This will give me the opportunity to really iron out the kinks, without having to worry about making a mess of the DB. If you are interested in helping out please get in touch, or +1 this if you are reading on Google+. I'd also be interested in suggestions for a good name for this project! It seems that any domain with any kind of reference to money or currency is taken (not surprising I suppose).


Rose Chafer

June 2nd, 2011
Nobel Chafer Beetle

Rose Chafer Beetle

This little guy found his way in to my flat, giving me an excellent opportunity to do a little macro photography.

Correction: I was mistaken, this is a Rose Chafer and not a Noble Chafer.

After a google images search, I've concluded it is a Noble Chafer Rose Chafer beetle. Turns out they are endangered. Shame, they are beautiful animals – if you like bugs and things like I do.

More on my Flicker photostream.


Django job at Net Communities

May 9th, 2011

Net Communities are looking for a Python/Django developer to work on an in-house project. It's a contract that would require some on-site work, but they would also consider a full-time developer for the right individual.

If you are interested, get in touch with Andy Evans.


Creating a Virtual Filesystem with Python (and why you need one)

March 20th, 2011

If you are writing an application of any size, it will most likely require a number of files to run – files which could be stored in a variety of possible locations. Furthermore, you will probably want to be able to change the location of those files when debugging and testing. You may even want to store those files somewhere other than the user's hard drive.

Any engineer worth his salt will recognise that the file locations should be stored in some kind of configuration file and the code to read the files in question should be factored out so that it isn't just scattered at points where data is read or written. In this post I'll present a way of doing just that by creating a virtual filesystem with PyFilesystem.

You'll need the most recent version of PyFilesystem from SVN to run this code.

We're going to create a virtual filesystem for a fictitious application that requires per-application and per-user resources, as well as a location for cache and log files. I'll also demonstrate how to mount files located on a web server. Here's the code:

from fs.opener import fsopendir
app_fs = fsopendir('mount://fs.ini', create_dir=True)

That's, all there is to it; two lines of code (one if you don't count the import). Obviously there is quite a bit going on under the hood here, which I'll explain below, but lets see what this code gives you…

The app_fs object is an interface to a single filesystem that contains all the file locations our application will use. For example, the path /user/app.ini references a per-user file, whereas /resources/logo.png references a per application file. The actual physical location of the data is irrelevant because as far as your application is concerned the paths never change. This abstraction is useful because the real path for such files varies according to the platform the code is running on; Windows, Mac and Linux all have different conventions, and if you put your files in the wrong place, your app will likely break on one platform or another.

Here's how a per-user configuration file might be opened:

from ConfigParser import ConfigParser
# The 'safeopen' method works like 'open', but will return an
# empty file-like object if the path does not exist
with app_fs.safeopen('/user/app.ini') as ini_file:
    cfg = ConfigParser()
    # ... do something with cfg

The files in our virtual filesystem don't even have to reside on the local filesystem. For instance, /live/ may actually reference a location on the web, where the version of the current release and a short ‘message of the day’ is stored.

Here's how the version number and MOTD might be read:

def get_application_version():
    """Get the version number of the most up to date version of the application,
    as a tuple of three integers"""
    with app_fs.safeopen('live/version.txt') as version_file:
        version_text = version_file.read().rstrip()
    if not version_text:
        # Empty file or unable to read
        return None
    return tuple(int(v) for v in version_text.split('.', 3))

def get_motd():
    """Get a welcome message"""
    with app_fs.safeopen("live/motd.txt") as motd_file:
        return motd_file.read().rstrip()

You'll notice that even though the actual data is retrieved over HTTP (the files are located here and here), the code would be no different if the files were stored locally.

So how is all this behaviour created from a single line of code? The line fsopendir("mount://fs.ini", create_dir=True) opens a MountFS from the information contained within an INI file (create_dir=True will create specified directories if they don't exist). Here's an example of an INI file that could be used during development:


The INI file is used to construct a MountFS, where the keys in the [fs] section are the top level directory names and the values are the real locations of the files. In above example, /user/ maps on to a directory called user relative to the current directory – but it could be changed to an absolute path or to a location on a server (e.g. FTP, SFTP, HTTP, DAV), or even to a directory within a zip file.

You can change the section to use in a mount opener by specifying it after a # symbol, i.e. mount://fs.ini#mysection

There are a few changes to this INI file we will need to make when our application is ready for release. User data, site data, logs and cache all have canonical locations that are derived from the name of the application (and the author on Windows). PyFilesystem contains handy openers for these special locations. For example, appuser://examplesoft:myapp detects the appropriate per-user data location for an application called “myapp” developed by “examplesoft”. Ditto for the other per-application directories. e.g.:


The /live/ path is different in that it needs to point to a web server:


Of course, you don't need to use the canonical locations. For instance, let's say you want to store all your static resources in a zip file. No problem:


Or you want to keep your user data on a SFTP (Secure FTP) server:


Perhaps you don't want to preserve the cache across sessions, for security reasons. The temp opener creates files in a temp directory and deletes them on close:


Although, if you are really paranoid you can store the cache files in memory without ever writing them to disk:


Setting /user/ to mem:// is a useful way of simulating a fresh install when debugging.

I hope that covers why you might need – or at least want – a virtual file system in your application. I've glossed over some the details and other features of PyFilesystem. If you would like more information, see my previous posts, check out the documentation or join the PyFilesystem discussion group.


Ken Burns effect with Javascript and Canvas

February 26th, 2011

I recently decided to look into working with the Canvas element to prototype a game idea I had. Since the easiest way to learn a technology is to use it, I set myself the goal of implementing the Ken Burns Effect.

There are a few JS slideshow scripts that do the Ken Burns effect, but I haven't seen any implemented in Canvas.

Without further ado, here is my implementation of the effect:

Your browser doesn't support canvas! Try Chrome, Firefox or Opera

If you see the effect, above, you are probably viewing this in one of the good browsers. I think it can be made to run on IE with excanvas, although I have yet to test that.

If there is enough interest, I may open source the code and add some new features / docs. In the meantime, feel free to download the code (a JQuery plugin) or run it on your site:

Ken Burns Effect in Javascript and Canvas

There's actually not that much to it. Here is the code to kenburns.js:


    $.fn.kenburns = function(options) {

        var $canvas = $(this);
        var ctx = this[0].getContext('2d');
        var start_time = null;
        var width = $canvas.width();
        var height = $canvas.height();

        var image_paths = options.images;
        var display_time = options.display_time || 7000;
        var fade_time = Math.min(display_time / 2, options.fade_time || 1000);
        var solid_time = display_time - (fade_time * 2);
        var fade_ratio = fade_time - display_time
        var frames_per_second = options.frames_per_second || 30;
        var frame_time = (1 / frames_per_second) * 1000;
        var zoom_level = 1 / (options.zoom || 2);
        var clear_color = options.background_color || '#000000';

        var images = [];
        $(image_paths).each(function(i, image_path){
        function get_time() {
            var d = new Date();
            return d.getTime() - start_time;

        function interpolate_point(x1, y1, x2, y2, i) {
            // Finds a point between two other points
            return  {x: x1 + (x2 - x1) * i,
                     y: y1 + (y2 - y1) * i}

        function interpolate_rect(r1, r2, i) {
            // Blend one rect in to another
            var p1 = interpolate_point(r1[0], r1[1], r2[0], r2[1], i);
            var p2 = interpolate_point(r1[2], r1[3], r2[2], r2[3], i);
            return [p1.x, p1.y, p2.x, p2.y];

        function scale_rect(r, scale) {
            // Scale a rect around its center
            var w = r[2] - r[0];
            var h = r[3] - r[1];
            var cx = (r[2] + r[0]) / 2;
            var cy = (r[3] + r[1]) / 2;
            var scalew = w * scale;
            var scaleh = h * scale;
            return [cx - scalew/2,
                    cy - scaleh/2,
                    cx + scalew/2,
                    cy + scaleh/2];

        function fit(src_w, src_h, dst_w, dst_h) {
            // Finds the best-fit rect so that the destination can be covered
            var src_a = src_w / src_h;
            var dst_a = dst_w / dst_h;
            var w = src_h * dst_a;
            var h = src_h;
            if (w > src_w)
                var w = src_w;
                var h = src_w / dst_a;
            var x = (src_w - w) / 2;
            var y = (src_h - h) / 2;
            return [x, y, x+w, y+h];

        function get_image_info(image_index, load_callback) {
            // Gets information structure for a given index
            // Also loads the image asynchronously, if required
            var image_info = images[image_index];
            if (!image_info.initialized) {
                var image = new Image();
                image_info.image = image;
                image_info.loaded = false;
                image.onload = function(){
                    image_info.loaded = true;
                    var iw = image.width;
                    var ih = image.height;

                    var r1 = fit(iw, ih, width, height);;
                    var r2 = scale_rect(r1, zoom_level);

                    var align_x = Math.floor(Math.random() * 3) - 1;
                    var align_y = Math.floor(Math.random() * 3) - 1;
                    align_x /= 2;
                    align_y /= 2;

                    var x = r2[0];
                    r2[0] += x * align_x;
                    r2[2] += x * align_x;

                    var y = r2[1];
                    r2[1] += y * align_y;
                    r2[3] += y * align_y;

                    if (image_index % 2) {
                        image_info.r1 = r1;
                        image_info.r2 = r2;
                    else {
                        image_info.r1 = r2;
                        image_info.r2 = r1;

                    if(load_callback) {

                image_info.initialized = true;
                image.src = image_info.path;
            return image_info;

        function render_image(image_index, anim, fade) {
            // Renders a frame of the effect
            if (anim > 1) {
            var image_info = get_image_info(image_index);
            if (image_info.loaded) {
                var r = interpolate_rect(image_info.r1, image_info.r2, anim);
                var transparency = Math.min(1, fade);

                if (transparency > 0) {
                    ctx.globalAlpha = Math.min(1, transparency);
                    ctx.drawImage(image_info.image, r[0], r[1], r[2] - r[0], r[3] - r[1], 0, 0, width, height);

        function clear() {
            // Clear the canvas
            ctx.globalAlpha = 1;
            ctx.fillStyle = clear_color;
            ctx.fillRect(0, 0, ctx.canvas.width, ctx.canvas.height);

        function update() {
            // Render the next frame
            var update_time = get_time();

            var top_frame = Math.floor(update_time / (display_time - fade_time));
            var frame_start_time = top_frame * (display_time - fade_time);
            var time_passed = update_time - frame_start_time;

            function wrap_index(i) {
                return (i + images.length) % images.length;

            if (time_passed < fade_time)
                var bottom_frame = top_frame - 1;
                var bottom_frame_start_time = frame_start_time - display_time + fade_time;
                var bottom_time_passed = update_time - bottom_frame_start_time;
                if (update_time < fade_time) {
                } else {
                    render_image(wrap_index(bottom_frame), bottom_time_passed / display_time, 1);

            render_image(wrap_index(top_frame), time_passed / display_time, time_passed / fade_time);

            if (options.post_render_callback) {
                options.post_render_callback($canvas, ctx);

            // Pre-load the next image in the sequence, so it has loaded
            // by the time we get to it
            var preload_image = wrap_index(top_frame + 1);

        // Pre-load the first two images then start a timer
        get_image_info(0, function(){
            get_image_info(1, function(){
                start_time = get_time();
                setInterval(update, frame_time);


})( jQuery );

Spatial Bookmarking Service goes Open Source

January 9th, 2011

Locidesktop was my coffee shop coding project of last year. I was quite pleased with the results. Locidesktop.com has been happily serving link desktops to some loyal visitors for months now – with no maintenance required on my part (a good thing because I've been busy with other projects).

Rather than let the project stagnate while I do other things, I have released it as open source with the hope that other Python/Django developers will pick it up and add new features. I had never really anticipated that anyone other than myself would see the code, so it is lacking in comments and documentations, but there are some nice features that may be of interest. For instance, there's a system to expose a REST API that factors out a lot of Python bioler-plate code and corresponding javascript code exposes the desktop functionality to the browser.

There's also a pretty powerful caching system that makes rendering a desktop blindingly fast. I may have gone overboard with that, given my traffic rates. But I guess it is best to have more capacity than you need.

If you want to look through the code or fork the project, it is available on Github:

Locidesktop code

The only thing missing from the open source version is a single-site theme I purchased from themeforest.net, used in a few pages such as the about & privacy urls. The desktops themselves are identical to the live site because I did those myself. You can tell because of the minimalist design reflecting my artistic abilities (minimal).

There's a README in the project that will help you get started, but other than that you may have to figure things out for yourself. It's a pretty standard Django project, although I do use Jinja templates rather than Django templates. If you have any questions, please ask them in the comments, so at least there is a central repository for issues.

For more information on Locidesktop, see my previous posts on the subject.

Update: I had to recreate the repos to remove a bunch of temp files and mercurial data. If you forked before 9am GMT, 10th Jan, you may want to get this new repos.


My Tweets

Will McGugan

My name is Will McGugan. I am an unabashed geek, an author, a hacker and a Python expert – amongst other things!

Search for Posts
Popular Tags
Recent Comments
Venelin, this wouldn't work with virtualenvs. Don't think you have much options there, except to allocate resources like you suggested.
This looks very nice, bit I have one question. What about virtual environments? If every site use different virtual environment ...
great app!! this is such a secure app to store your information..but its having disadvantage ,if you lose your phone ...
Read this article, I'm just beginning Python, and wondered if people could be employed if they knew python well, but ...
Thanks for this! It really helped me to properly implement the camera in my pygame-opengl project :) Cheers!
© 2008 Will McGugan.

A technoblog blog, design by Will McGugan