There have been some pretty exciting developments in PyFilesystem since version 0.3 was released – Ryan Kelly and myself have been hard at work, and there have been a number of excellent contributions to the code base from other developers. Version 0.4 will be released some time in January, but I'd like to give you a preview of some new features before the next version lands.
Pyfilesystem is a Python module that provides a simplified common interface to many types of filesystem.
It is now possible to open any of the supported filesystems from a URL in this format, which makes it very easy to specify a filesystem (or individual file) from the command line or a config file. Here's a quick example that opens a bunch of quite different filesystems:
from fs.opener import fsopendir projects_fs = fsopendir('/projects') zip_fs = fsopendir('zip:///foo/bar/baz.zip') ftp_fs = fsopendir('ftp://ftp/mozilla.org') sftp_fs = fsopendir('sftp://example.org')
If you used Pyfilesystem in your application you could trivially change where your files are physically located, or where you save generated files to.
You can also open a file directly without the need to explicitly open the filesystem it is contained within, with the
fsopen function, e.g.:
from fs.opener import fsopen print fsopen('zip:///foo/bar/baz.zip!dir/somefile.txt').read() print fsopen('ftp://ftp.mozilla.org/pub/README').read()
fsopen is very similar to the builtin
open method and will return a file-like object of some kind. In fact, if you pass in a system path it works as
open would (although the exceptions will be an instance of
fs.errors.FSError rather than
Because of this similarity with the builtin,
fsopen could be used to shadow
open, and instantly add the ability for an application to open files on mediums other than a system drive. This is all it takes:
from fs.opener import fsopen as open
Version 0.4 also adds a number of applications that mirror some of the standard command line apps, but extend their functionality with FS URLs. For example,
fsls, functions just like the
ls command, but works with any of the supported filesystems:
will@will-linux:~$ fsls .
will@will-linux:~$ fsls zip://myzip
will@will-linux:~$ fsls ftp://ftp.mozilla.org/pub
will@will-linux:~$ fsls sftp://user:firstname.lastname@example.org/home/will
You can also copy and move files between filesystems with
fsmv, which work in a very similar manner to their
mv counterparts, with a few extensions such as multi-threaded copying (great for network filesystems) and an ascii progress bar. The following example copies all the
.py files in my
projects directory to zip file on an ftp server, and displays an progress bar to boot.
will@will-linux:~$ fscp ~/projects/*.py zip:ftp://will:email@example.com/backups/code.zip
Then there is the
fscat command that writes a file in a filesystem to the terminal. The following example displays a python file in the zip file that we created with the previous command:
will@will-linux:~$ fscat zip:ftp://will:firstname.lastname@example.org/backups/code.zip!pythonrocks.py
The other commands
fstree work as you may expect.
fsserve command adds the ability to serve any of the supported filesystems over a number of protocols. The default is to serve
http – in effect creating a webserver. The following is all it takes to serve the contents of your current working directory:
Now if you point your browser at http://127.0.0.1 you will see a web-page with the contents of your current working directory (or a index.html file if present). It's not the most bullet-proof of web servers, but handy if you quickly want to serve some files. Naturally,
fsserve works with any filesystem you pass to it. You could, for instance, serve the contents of a zip file without ever explicitly unzipping it, or create an ftp to http gateway by serving an ftp filesystem. The following command creates a ftp to http gateway for
will@will-linux:~$ fsserve ftp://ftp.mozilla.org
You also have the option of serving a filesystem over SFTP (Secure FTP), or by RPC (Remote Procedure Call). Either of these two methods expose all the functionality of the remote filesystem, so you could run a server on one machine and create/move/copy/delete files from another machine on the network (or internet). For example, the following would serve the current working directory on localhost, port 3000:
will@will-linux:~$ fsserve -t rpc -p 3000
You can then connect to that server from another machine on the network. Assuming my local IP is
192.168.1.64 the following would display the directory structure from another machine on my network:
will@will-linux:~$ fstree rpc://192.168.1.64:3000
Any of the filesystems can be mounted on the OS with the
fsmount command, which uses FUSE on Linux or DOKAN on Windows. The advantage of this is that the filesystems exposed in Python can be used in any application, and browsed with Explorer or Nautilus. The following creates a ram drive on Linux:
will@will-linux:~$ fsmount mem:// mem
Or on Windows:
C:\> fsmount mem:// M
Get the Code
There is no documentation online for the new features as yet, but if you are a brave soul and want to experiment with any of the above commands then download the code from SVN and run
python setup.py install. The command line apps all have a
-h which which displays help on the various options.
Bear in mind that these commands are still somewhat experimental, and some of these commands have the capacity to delete files – so be careful. That said, I'm confident to use them for my day-to-day work.
Please see the projects page page if you want to report bugs or discuss Pyfilesystem with myself and the other developers.