Python Posts

138 posts tagged with python

I've released version 2.0.4 of PyFilesystem.

PyFilesystem is an abstraction layer for filesystems (or anything that resembles a filesystem). See this post for more details.

This version adds support for Python3.6 os.pathlike to the OSFS class. This is in oft-requested feature, but it only affects the OSFS constructor; since FS methods use a standardized path format.

Also in the version is a contribution by Martin Larralde which implements an elegant extension mechanism. Essentially, if you layout your filesystem implementation according to the convention, then PyFilesystem will be aware of it automatically. So open_fs('awesomefs://foo/bar') would work without explicitly adding awesomefs to the core library. continue reading…

I'd like to announce the first official release of Lomond, a new WebSocket client library for Python. The development was sponsored by Dataplicity.

Lomond is not the first websocket client for Python, so why go to the effort of building another one? For our purposes, we needed a stand alone client that didn't need a framework to run. So that excludes the websocket client support in Tornado, aiohttp etc. The two libraries that were suitable for our product, websocket-client and ws4py, both had show-stopper bugs with ssl support; websocket-client would sometimes refuse to processes packets until additional data was received, and ws4py could lose entire packets. I'm sure both libraries could be fixed, but neither project appears to be actively maintained. continue reading…

I've just released version 2.0.3 of PyFilesystem.

PyFilesystem is an abstraction layer for filesystems (or anything that resembles a filesystem). See this post for more details.

New in this version is a TarFS filesystem contributed by Martin Larralde, which compliments ZipFS nicely.

Contributed by gpcimino, the copy module was extended new functionality to selectively copy only new files from one filesystem to another. Very useful for backups.

This is in addition to bugfixes, and improved documentation.

I recently I had the opportunity to speed up some Websocket code that was a major bottleneck. The final solution was 60X (!) faster than the first pass, and an interesting exercise in optimizing inner loops.

When you send data via a websocket it is masked with a randomly generated four byte key. This masking provides protection from certain exploits against proxies (details here). The algorithm is super simple; essentially each byte in the payload is XORed with a corresponding byte from the key.

The first version of my XOR mask function looked this:

That's the kind of unfussy and elegant code you want to write for a job interview. If that's the solution that first came to mind, then you know your Python. Alas, this code is as slow as it is beautiful. continue reading…

I like to imagine components of a Python web framework as personalities. Here's how I would imagine a conversation between a typical view and template would go down:

View Here's that data I just looked up.

Template Don't like it.

View What's wrong with it?

Template Format's wrong.

View You can change the format if you like.

Template Don't wanna. You change it.

View OK. Fine. I'll change it for you.

Which is why I wanted Moya templates to be very capable in processing data, but still keep them designer friendly. If the template data is sane then the template should be able to deal with it, without going back to the back-end developer. continue reading…

I'd like to announce version 2.0.0 of PyFilesystem, which is now available on PyPi.

PyFilesystem is a Python module I started some time in 2008, and since then it has been very much a part of my personal standard library. I've used it in personal and professional projects, as have many other developers and organisations.

If you aren't familiar with PyFilesystem; it's an abstraction layer for filesystems. Essentially anything with files and directories (hard-drive, zip file, ftp server, network filesystems etc.) may be wrapped with a common interface. With it, you can write code that is agnostic as to where the files are physically located.

Here's a quick example that recursively counts the lines of code in a directory: continue reading…

It's not often I come across something in Python that surprises me. Especially in something as mundane as string operations, but I guess Python still has a trick or two up its sleeve.

Have a look at this string:

How many possible sub-strings are in s? To put it another away, how many values of x are there where the expression x in s is true?

Turns out it is 2.

2?

Yes, 2.

The empty string is in the string "A". In fact, it's in all the strings.

Turns out the empty string has been hiding every where in my code.

Not a complaint, I'm sure the rationale is completely sound. And it turned out to be quite useful. I had a couple of lines of code that looked something like this: continue reading…

This SO post prompted me to think about how you would go about customizing a entire Django site (not just an app). Particularly if it is a third party project, still in development.

Forking the site repository is a solution, but the more customizations you make, the more awkward it becomes to merge upstream fixes and features. The problem is compounded if you have multiple deploys, with custom templates, assets, and settings.

I wanted to do a quick write-up of how this would be done in Moya, where customization is more of a first-class concept.

I'll briefly go over how you would serve a Moya site with a custom template, without modifying any of the original files. The project I'll be customizing is Moya Techblog which power this blog. continue reading…

Inthing can take a screenshot of your desktop then post it online.

Here's a quick example:

Alternatively you can do the same thing from the command line with the inthing app. The following is equivalent to the Python code (assuming you have set up the stream and password environment variables):

See the docs for more details.

Screenshot capabilities courtesy of the excellent pyscreenshot library.

Screenshot taken with Stream.screenshot

Just landed in inthing is a new and quite interesting feature.

Version 0.1.4 adds a capture method which will record all standard output, i.e. anything you print to the terminal. It works as a context manager. Here's an example:

Any print statement inside the with block will be captured and posted online with the block exits.

You can also do something similar from the command line, with the inthing capture subcommand, which posts anything you pipe in to it as an event.

lets say you wanted to post the version of all you installed Python packages online. You could do something like the following:

For more information see the Inthing docs.

Inthing is still technically in beta, but these features are quite solid. Please give them a try, and let me know how it goes!