will

Where there is a Will

Will McGugan's Blog

I am a freelance software engineer living in Glasgow Leicester.

I post mostly about tech (particularly Python) and photography.

Background: A code monkey

Most PEPs (Python Enhancement Proposal) tend do go under the radar of most developers, but PEP 572 has caused a lot of controversy in the Python community, with some developers expressing an intense dislike for the new syntax (and that is sugar coated).

Personally, I don't think it deserves the vitriol it received, and I'm looking forward to its addition in Python 3.8.

So what are assignment expressions? Currently in Python assignments have to be statements, so you can do foo = bar(), but you can't do that assignment from within an if or while statement (for example). Which is why the following is a syntax error: continue reading…

I've recently released version 0.3.2 of Lomond, a WebSocket client for Python with a focus on correctness and ease-of-use.

The major feature of the 0.3 release is per-message compression, which allows for text and binary to be sent in compressed form.

Here's a modified version of the Bitcoin price ticker example which enables compression. This spews every trade made on the GDax platform:

Using the per-message compression extension to the WebSocket spec with Lomond.

That's a lot of data coming through the WebSocket, which fortunately compresses very well.

Also in this release are a number of optimisations to reduce cpu and memory usage.

Lomond is fast approaching a 1.0 release, but is already very stable. It is part of the open source software that powers Dataplicity.

WebSockets is a great technology with many applications beyond the front end. I have a feeling that websocket APIs will become far more common in the web ecosphere.

I really like this example of how to interact with a WebSockets api with just-released Lomond 0.1.5.

Get the code from Lomond repos on Github.

This connects to the gdax websocket server and periodically prints the latest BTC price information. If you change the text "ticker" to "level2" it will spew out information on every trade. You could use this as the foundation for algorithmic trading or to keep an eye on your retirement fund (good luck).

Lomond is a websocket client library sponsored by Dataplicity. It has a focus on robustness and ease of use. continue reading…

The new asyncio module introduced in Python3.4 is a nice addition to the Python standard library, especially when used with the async and await keywords introduced in Python3.5.

If you have read the official docs, you should hopefully be able to be able to write high performance network servers and clients with asyncio. But if something goes wrong, you may find it hard to debug. This is no reflection on asyncio; concurrency and IO are deceptively complex things, and asyncio is the point they meet.

I suspect that the problem with debugging asyncio projects is that async code doesn't run itself. You are dependant on a rather large body of code to run your code for you, and the distinction between your code library code is far more blurry than you may be used to. continue reading…

I'm currently in St Ives, a town in Cornwall, UK. The weather has taken a turn for the worst, and I'm trapped with my wife in a conservatory attached to the B&B we're staying in. Can't complain, there has been plenty sun and sea food so far, and I got some decent shots this week.

This shot was taken from The Island, which isn't an island at all; its a rocky headland with a lighthouse.

A Northern Wheatear. I liked this shot for the framing. If I stood a few inches left or right I would have missed it.

These two shots of a Ruddy Turnstone were taken from the bay. I bought a rain poncho so I could lie down in the wet sand of the harbour and shoot from the birds eye level. If you want to shoot birds, this is the single best thing you can do to get better shots. continue reading…

I'd like to announce an new Python module to make working with Amazon S3 files a whole lot easier.

The S3FS class in fs-s3fs wraps an Amazon S3 bucket in a PyFilesystem interface. There was an S3FS class built in to the first version of PyFilesystem, but it had suffered from using an older version of 'boto' (Amazon's S3 interface) and was in need of maintenance. The new version is up-to-date with PyFilesystem2 and boto3, and works with Python2.7 and Python3.X.

If you aren't familiar with PyFilesystem, it is a common interface to anything that resembles a filesystem. Amazon S3 isn't quite a full filesystem, but close enough that it can be made to work in the same way as the files and directories on your local drive (or any other supported filesystem for that matter). continue reading…

I've released version 2.0.4 of PyFilesystem.

PyFilesystem is an abstraction layer for filesystems (or anything that resembles a filesystem). See this post for more details.

This version adds support for Python3.6 os.pathlike to the OSFS class. This is in oft-requested feature, but it only affects the OSFS constructor; since FS methods use a standardized path format.

Also in the version is a contribution by Martin Larralde which implements an elegant extension mechanism. Essentially, if you layout your filesystem implementation according to the convention, then PyFilesystem will be aware of it automatically. So open_fs('awesomefs://foo/bar') would work without explicitly adding awesomefs to the core library. continue reading…

I'd like to announce the first official release of Lomond, a new WebSocket client library for Python. The development was sponsored by Dataplicity.

Lomond is not the first websocket client for Python, so why go to the effort of building another one? For our purposes, we needed a stand alone client that didn't need a framework to run. So that excludes the websocket client support in Tornado, aiohttp etc. The two libraries that were suitable for our product, websocket-client and ws4py, both had show-stopper bugs with ssl support; websocket-client would sometimes refuse to processes packets until additional data was received, and ws4py could lose entire packets. I'm sure both libraries could be fixed, but neither project appears to be actively maintained. continue reading…

Wildlife photography can require great hardship on the part of the photographer; days of trekking, battling the elements, biting insects, etc. — all while carrying 20 kilos of camera gear.

This was not the case for my recent trip to Costa Rica, which was as close to luxury you can get in the Rainforest, if you don't mind the lack of air-conditioning and occasional oversize bug in your cabin.

My girlfriend and I stayed in El Romanso, an eco-lodge on the Osa Peninsula. Located in primary rainforest overlooking the Pacific ocean, it is an beautiful place, abundant with wildlife. Most of which I photographed from the lodge restaurant, which had been constructed on stilts to put you at eye level with monkeys and toucans. continue reading…

I've just released version 2.0.3 of PyFilesystem.

PyFilesystem is an abstraction layer for filesystems (or anything that resembles a filesystem). See this post for more details.

New in this version is a TarFS filesystem contributed by Martin Larralde, which compliments ZipFS nicely.

Contributed by gpcimino, the copy module was extended new functionality to selectively copy only new files from one filesystem to another. Very useful for backups.

This is in addition to bugfixes, and improved documentation.