PyFilesystem 0.3 released
I am pleased to announce a new version of PyFilesystem (0.3), which is a Python module that provides a common interface to many kinds of filesystem. Basically it provides a way of working with files and directories that is exactly the same, regardless of how and where the file information is stored. Even if you don't plan on working with anything other than the files and directories on your hard-drive, PyFilesystem can simplify your code and reduce the potential of error.
PyFilesystem is a joint effort by myself and Ryan Kelly, who has created a number of new FS implementations such as Amazon S3 support and Secure FTP, and some pretty cool features such as FUSE support and Django storage integration.
As an example of how awesome this package is, take a look at the following 6 lines of code, which creates a ramdrive:
from fs.osfs import OSFS from fs.memoryfs import MemoryFS from fs.expose import fuse home_fs = OSFS('~/') home_fs.makedir('ramdrive', allow_recreate=True) fuse.mount(MemoryFS(), home_fs.getsyspath('ramdrive'))
If you run this, a directory called ramdrive will appear in your home folder, the contents of which are stored purely in memory.
I prepared a screencast that gives a quick demonstration of some features – because if a picture is worth a thousand words, this video must be worth fifteen thousand words a second:
PyFilesystem screencast from Will McGugan on Vimeo.
See the project page on google code for more information, including API docs. There are also a couple of blog posts that will give a some more context.
This release has reached a good level of stability and maturity. I'd like to invite as many Pythonistas as possible to check out this module and possibly contribute to the project.
So what can you do with FS anyway?
After my last post regarding FS, my file-system abstraction layer for Python, I think I may have left people thinking,"that's nice, but what would you use if for?". Generally, I see it as a way of simplifying file access and exposing only the files you need for your application -- regardless of their physical source. But I can think of a few other uses that may be a littler cooler.
Instant Archive Format
The osfs.MemoryFS class is a filesytem that exists only in memory. You can create files / directories and copy files to it from other filesystem objects, but when you open a file you will get back a StringIO object (from the standard library). Naturaly the files in a MemoryFS are transitory, but as everything is in memory, file access is very fast. I was thinking that since the files and directories are stored as simple Python objects, then a MemoryFS can be pickled -- creating an instant archive format. Uncompressing this archive would simply require unpicking and using the resultant object. I can't see any advantages of this over zip or tar archives, but if you don't have them available, this would be a workable substitute!
Database Filesystem
It wouldn't be particularly challenging to write an FS an object that created a filesystem on a database. Actualy, if using an ORM like SQLAlchemy, SQLObject, Storm or Django's database layer it would be fairly trivial. So what would you use such a thing for? I can think of one use which may make it worth the effort, and that is the ability to store templates for a Python web application in the database, rather than (or in addition to) files. Consider if you will, a situation where the client has requested a change that is absolutely critical to the success of his company and he won't take "it will be in the next release" for an answer, and neither will your project manager. Wouldn't it be nice if you could go in to the Django admin pages and just type in the changes, without making a new release. I'm tempted to implement such a thing, if nobody has done so already!
Proxying a Filesystem
A generic filesystem server could be written that exposes a remote file-system over whatever channel is available. I know there are some standard ways of doing this, but they might not be available on a given platform. I'm thinking of small devices with limited capabilities, or secure environments. The server would serve an FS object, and the client would present the app with a corresponding interface that pulled files and directories over the wire.
There you go. If anyone implements these, let me know!
Announcing FS 0.1.0, a Python file-system
This is something I have been hacking together for a while now; FS is a file-system abstraction for Python. It has reached a stable state and is worthy of an official (0.1.0) release.
First, a brief history of the project. A while back, I was working mainly with desktop applications in wxPython. I found that I had a number of sources for files; there were read-only resources such as images, per-user files and per-installation config files. The location of these files would change depending on whether I was debugging or building a release version. The logic required to manage all this was pretty ugly and error-prone. So I wrote a collection of classes to bring all these disparate locations for files under a single virtual file-system. For example, to open a config file I could just read 'user/settings.cfg', and to open an image resource I could just read from 'resources/logo.png' and the virtual file-system would do the right thing and return a file-like object.
This turned out to be insanely useful, and I used it for a number of projects. I also used it while working for ICC, and I had the opportunity to enhance it based on feedback from colleagues. In the last few months I have re-written it from scratch, because I wanted to avoid any copyright issues, but mainly because I could make a better job of it the second time around.
