will

Where there is a Will

Will McGugan's Blog

I am a freelance software developer living in Glasgow Leicester.

I post mostly about tech (particularly Python) and photography.

Background: A code monkey

I've just released version 2.0.3 of PyFilesystem.

PyFilesystem is an abstraction layer for filesystems (or anything that resembles a filesystem). See this post for more details.

New in this version is a TarFS filesystem contributed by Martin Larralde, which compliments ZipFS nicely.

Contributed by gpcimino, the copy module was extended new functionality to selectively copy only new files from one filesystem to another. Very useful for backups.

This is in addition to bugfixes, and improved documentation.

I recently I had the opportunity to speed up some Websocket code that was a major bottleneck. The final solution was 60X (!) faster than the first pass, and an interesting exercise in optimizing inner loops.

When you send data via a websocket it is masked with a randomly generated four byte key. This masking provides protection from certain exploits against proxies (details here). The algorithm is super simple; essentially each byte in the payload is XORed with a corresponding byte from the key.

The first version of my XOR mask function looked this:

That's the kind of unfussy and elegant code you want to write for a job interview. If that's the solution that first came to mind, then you know your Python. Alas, this code is as slow as it is beautiful. continue reading…

I like to imagine components of a Python web framework as personalities. Here's how I would imagine a conversation between a typical view and template would go down:

View Here's that data I just looked up.

Template Don't like it.

View What's wrong with it?

Template Format's wrong.

View You can change the format if you like.

Template Don't wanna. You change it.

View OK. Fine. I'll change it for you.

Which is why I wanted Moya templates to be very capable in processing data, but still keep them designer friendly. If the template data is sane then the template should be able to deal with it, without going back to the back-end developer. continue reading…

I'd like to announce version 2.0.0 of PyFilesystem, which is now available on PyPi.

PyFilesystem is a Python module I started some time in 2008, and since then it has been very much a part of my personal standard library. I've used it in personal and professional projects, as have many other developers and organisations.

If you aren't familiar with PyFilesystem; it's an abstraction layer for filesystems. Essentially anything with files and directories (hard-drive, zip file, ftp server, network filesystems etc.) may be wrapped with a common interface. With it, you can write code that is agnostic as to where the files are physically located.

Here's a quick example that recursively counts the lines of code in a directory: continue reading…

It's not often I come across something in Python that surprises me. Especially in something as mundane as string operations, but I guess Python still has a trick or two up its sleeve.

Have a look at this string:

How many possible sub-strings are in s? To put it another away, how many values of x are there where the expression x in s is true?

Turns out it is 2.

2?

Yes, 2.

The empty string is in the string "A". In fact, it's in all the strings.

Turns out the empty string has been hiding every where in my code.

Not a complaint, I'm sure the rationale is completely sound. And it turned out to be quite useful. I had a couple of lines of code that looked something like this: continue reading…

This year, Ellen and I took a trip, roughly following the North Coast 500 route.

The West coast of Scotland is stunning. As a Glasgow cabbie observed, if it wasn't for the weather and the midges of the Scottish West Coast, Brits wouldn't bother leaving the UK for their holiday. I can't comment on the weather (which was beautiful), but he may have a point about the midges.

Here are some photos from that trip.

Thistles at sunrise. Taken at around 5.30am. Worth getting up for?

This image is part of a much larger panorama that I would like to print for my wall in three parts.

The iconic Highland Cow, or Highland Coo as we call them in Scotland.

If you aren't familiar with midges, they are tiny biting flies. Their vileness is brilliantly illustrated with the following video...

I've always been intrigued by HDR, but some of my earlier attempts have been pretty horrific (like someone ate a pack of crayons and vomited on the screen).

Sunsets are one of the few subjects where HDR really works. Without merging a range of exposures you can never expose for both the sun and the foreground. So I was quite pleased to get the following shot (in Flores, Indonesia).

The trick in processing an HDR photo is to make it look like it is not HDR. At least not obviously HDR...

In addition to Orangutans, my Indonesia trip was to photograph Komodo dragons. These are absolutely remarkable animals, and as close to a living dinosaur you can see today.

As excited as I was to see these animals, the experience was a little disappointing. The rangers lead us around a short trek through the forrest before returning to the beach where there a dragon was basking. It was all clearly staged.

Komodo Dragons are actually venomous, contrary to the popular belief that their bites kill with toxic bacteria in their mouths.

This was a juvenile dragon. We saw it meandering through the forrest, occasionally chewing on dry bones, trying to find sustenance. The urge to hug him, and squeeze him, and call him George was strong! continue reading…

The Isle of May is an island just off the East coast of Scotland. Although tiny (less than 1km2), it is home to a remarkable amount of wildlife, birds in particular. I was lucky enough to visit with the Glasgow Photography Meetup group.

Here's a few shots from that day.

I think this is a juvenile gull. Post a comment if I'm wrong!

Photographing a bird in flight is a challenge, so I'm please I got this at all, but the plain background this doesn't make for a memorable photo.

There were some fantastic vantage points on the island where I could get eye-level shots like this one.

These photos were all taken with a Canon 500mm, which is an absolutely fantastic lens for bird photography and a good upper body workout.

This SO post prompted me to think about how you would go about customizing a entire Django site (not just an app). Particularly if it is a third party project, still in development.

Forking the site repository is a solution, but the more customizations you make, the more awkward it becomes to merge upstream fixes and features. The problem is compounded if you have multiple deploys, with custom templates, assets, and settings.

I wanted to do a quick write-up of how this would be done in Moya, where customization is more of a first-class concept.

I'll briefly go over how you would serve a Moya site with a custom template, without modifying any of the original files. The project I'll be customizing is Moya Techblog which power this blog. continue reading…