will

Where there is a Will

Will McGugan's Blog

I am a freelance software engineer living in Edinburgh.

I post mostly about tech (particularly Python) and photography.

Background: A code monkey

I revisited the Isle of May recently. This time at the beginning of the Puffin season.

My favorite shot of the day. I like the way the bokeh background is still recognizable enough to be a feature.

There is something quite comical about this bird, like he is explaining something to a skeptical audience.

I was reading a post by Trey Hunner on why pathlib is great, where he makes the case that pathlib is a better choice than the standard library alternatives that preceded it. I wouldn't actually disagree with a word of it. He's entirely correct. You should probably be using pathlib were it fits.

Personally, however, I rarely use pathlib, because I find that for the most part, PyFilesystem is a better choice. I'd like to take some of the code examples from Trey's post and re-write them using PyFilesystem, just so we can compare.

The first example from Trey's post, creates a folder then moves a file into it. Here it is:

The code above is straightforward, and hides the gory platform details which is a major benefit of pathlib over os.path. continue reading…

I recently added a number of examples on working with files and directories with Python and PyFilesystem.

The first example, is count_py.py which recursively sums up the number of bytes stored in a directory, and renders it in a "friendly" format.

This script takes a path or a URL. For instance python3 count_py.py ~/projects will sum up the bytes of Python in your projects folder.

For me, this gives the following output:

This script accepts a path, but it will also work with any supported filesystem. So you could sum the bytes of Python in an archive or cloud server. continue reading…

It occurred to me that Django's ORM could do with a bit of a revamp to make use of recent developments in the Python language.

The main area where I think Django's models are missing out is the lack of type hinting (hardly surprising since Django pre-dates type hints). Adding type hints allows Mypy to detect bugs before you even run your code. It may only save you minutes each time, but multiply that by the number of code + run iterations you do each day, and it can save hours of development time. Multiply that by the lifetime of your project, and it could save weeks or months. A clear win. continue reading…

I've released PyFilesystem 2.1.0.

This version is the accumulation of many minor revisions, with multiple fixes, enhancements and some interesting new features. We also have nicer doc strings and Mypy compatible typing information, thanks to Martin Larralde.

If you aren't familiar with PyFilesystem, it is a common Python API to filesystems, whether its your hard-drive, archives, memory, FTP servers, or cloud services. The API is easier to use than the standard library, but works just about everywhere. See the Pyfilesystem Wiki for more information. continue reading…

Most PEPs (Python Enhancement Proposal) tend do go under the radar of most developers, but PEP 572 has caused a lot of controversy in the Python community, with some developers expressing an intense dislike for the new syntax (and that is sugar coated).

Personally, I don't think it deserves the vitriol it received, and I'm looking forward to its addition in Python 3.8.

So what are assignment expressions? Currently in Python assignments have to be statements, so you can do foo = bar(), but you can't do that assignment from within an if or while statement (for example). Which is why the following is a syntax error: continue reading…

I've recently released version 0.3.2 of Lomond, a WebSocket client for Python with a focus on correctness and ease-of-use.

The major feature of the 0.3 release is per-message compression, which allows for text and binary to be sent in compressed form.

Here's a modified version of the Bitcoin price ticker example which enables compression. This spews every trade made on the GDax platform:

Using the per-message compression extension to the WebSocket spec with Lomond.

That's a lot of data coming through the WebSocket, which fortunately compresses very well.

Also in this release are a number of optimisations to reduce cpu and memory usage.

Lomond is fast approaching a 1.0 release, but is already very stable. It is part of the open source software that powers Dataplicity.

WebSockets is a great technology with many applications beyond the front end. I have a feeling that websocket APIs will become far more common in the web ecosphere.

I really like this example of how to interact with a WebSockets api with just-released Lomond 0.1.5.

Get the code from Lomond repos on Github.

This connects to the gdax websocket server and periodically prints the latest BTC price information. If you change the text "ticker" to "level2" it will spew out information on every trade. You could use this as the foundation for algorithmic trading or to keep an eye on your retirement fund (good luck).

Lomond is a websocket client library sponsored by Dataplicity. It has a focus on robustness and ease of use. continue reading…

The new asyncio module introduced in Python3.4 is a nice addition to the Python standard library, especially when used with the async and await keywords introduced in Python3.5.

If you have read the official docs, you should hopefully be able to be able to write high performance network servers and clients with asyncio. But if something goes wrong, you may find it hard to debug. This is no reflection on asyncio; concurrency and IO are deceptively complex things, and asyncio is the point they meet.

I suspect that the problem with debugging asyncio projects is that async code doesn't run itself. You are dependant on a rather large body of code to run your code for you, and the distinction between your code library code is far more blurry than you may be used to. continue reading…

I'm currently in St Ives, a town in Cornwall, UK. The weather has taken a turn for the worst, and I'm trapped with my wife in a conservatory attached to the B&B we're staying in. Can't complain, there has been plenty sun and sea food so far, and I got some decent shots this week.

This shot was taken from The Island, which isn't an island at all; its a rocky headland with a lighthouse.

A Northern Wheatear. I liked this shot for the framing. If I stood a few inches left or right I would have missed it.

These two shots of a Ruddy Turnstone were taken from the bay. I bought a rain poncho so I could lie down in the wet sand of the harbour and shoot from the birds eye level. If you want to shoot birds, this is the single best thing you can do to get better shots. continue reading…