Spatial Bookmarking Service goes Open Source

January 9th, 2011

Locidesktop was my coffee shop coding project of last year. I was quite pleased with the results. Locidesktop.com has been happily serving link desktops to some loyal visitors for months now – with no maintenance required on my part (a good thing because I've been busy with other projects).

Rather than let the project stagnate while I do other things, I have released it as open source with the hope that other Python/Django developers will pick it up and add new features. I had never really anticipated that anyone other than myself would see the code, so it is lacking in comments and documentations, but there are some nice features that may be of interest. For instance, there's a system to expose a REST API that factors out a lot of Python bioler-plate code and corresponding javascript code exposes the desktop functionality to the browser.

There's also a pretty powerful caching system that makes rendering a desktop blindingly fast. I may have gone overboard with that, given my traffic rates. But I guess it is best to have more capacity than you need.

If you want to look through the code or fork the project, it is available on Github:

Locidesktop code

The only thing missing from the open source version is a single-site theme I purchased from themeforest.net, used in a few pages such as the about & privacy urls. The desktops themselves are identical to the live site because I did those myself. You can tell because of the minimalist design reflecting my artistic abilities (minimal).

There's a README in the project that will help you get started, but other than that you may have to figure things out for yourself. It's a pretty standard Django project, although I do use Jinja templates rather than Django templates. If you have any questions, please ask them in the comments, so at least there is a central repository for issues.

For more information on Locidesktop, see my previous posts on the subject.

Update: I had to recreate the repos to remove a bunch of temp files and mercurial data. If you forked before 9am GMT, 10th Jan, you may want to get this new repos.

 

Sneak Preview of New Features in PyFilesystem 0.4

January 1st, 2011

There have been some pretty exciting developments in PyFilesystem since version 0.3 was released – Ryan Kelly and myself have been hard at work, and there have been a number of excellent contributions to the code base from other developers. Version 0.4 will be released some time in January, but I'd like to give you a preview of some new features before the next version lands.

Pyfilesystem is a Python module that provides a simplified common interface to many types of filesystem.

It is now possible to open any of the supported filesystems from a URL in this format, which makes it very easy to specify a filesystem (or individual file) from the command line or a config file. Here's a quick example that opens a bunch of quite different filesystems:

from fs.opener import fsopendir
projects_fs = fsopendir('/projects')
zip_fs = fsopendir('zip:///foo/bar/baz.zip')
ftp_fs = fsopendir('ftp://ftp/mozilla.org')
sftp_fs = fsopendir('sftp://example.org')

If you used Pyfilesystem in your application you could trivially change where your files are physically located, or where you save generated files to.

You can also open a file directly without the need to explicitly open the filesystem it is contained within, with the fsopen function, e.g.:

from fs.opener import fsopen
print fsopen('zip:///foo/bar/baz.zip!dir/somefile.txt').read()
print fsopen('ftp://ftp.mozilla.org/pub/README').read()

fsopen is very similar to the builtin open method and will return a file-like object of some kind. In fact, if you pass in a system path it works as open would (although the exceptions will be an instance of fs.errors.FSError rather than IOError).

Because of this similarity with the builtin, fsopen could be used to shadow open, and instantly add the ability for an application to open files on mediums other than a system drive. This is all it takes:

from fs.opener import fsopen as open

FS Commands

Version 0.4 also adds a number of applications that mirror some of the standard command line apps, but extend their functionality with FS URLs. For example, fsls, functions just like the ls command, but works with any of the supported filesystems:

will@will-linux:~$ fsls .
will@will-linux:~$ fsls zip://myzip
will@will-linux:~$ fsls ftp://ftp.mozilla.org/pub
will@will-linux:~$ fsls sftp://user:pass@securesite.com/home/will

You can also copy and move files between filesystems with fscp and fsmv, which work in a very similar manner to their cp and mv counterparts, with a few extensions such as multi-threaded copying (great for network filesystems) and an ascii progress bar. The following example copies all the .py files in my projects directory to zip file on an ftp server, and displays an progress bar to boot.

will@will-linux:~$ fscp ~/projects/*.py zip:ftp://will:password@example.org/backups/code.zip

Then there is the fscat command that writes a file in a filesystem to the terminal. The following example displays a python file in the zip file that we created with the previous command:

will@will-linux:~$ fscat zip:ftp://will:password@example.org/backups/code.zip!pythonrocks.py

The other commands fsmkdir, fsrm and fstree work as you may expect.

Serving

The fsserve command adds the ability to serve any of the supported filesystems over a number of protocols. The default is to serve http – in effect creating a webserver. The following is all it takes to serve the contents of your current working directory:

will@will-linux:~$ fsserve

Now if you point your browser at http://127.0.0.1 you will see a web-page with the contents of your current working directory (or a index.html file if present). It's not the most bullet-proof of web servers, but handy if you quickly want to serve some files. Naturally, fsserve works with any filesystem you pass to it. You could, for instance, serve the contents of a zip file without ever explicitly unzipping it, or create an ftp to http gateway by serving an ftp filesystem. The following command creates a ftp to http gateway for ftp.mozilla.org:

will@will-linux:~$ fsserve ftp://ftp.mozilla.org

You also have the option of serving a filesystem over SFTP (Secure FTP), or by RPC (Remote Procedure Call). Either of these two methods expose all the functionality of the remote filesystem, so you could run a server on one machine and create/move/copy/delete files from another machine on the network (or internet). For example, the following would serve the current working directory on localhost, port 3000:

will@will-linux:~$ fsserve -t rpc -p 3000

You can then connect to that server from another machine on the network. Assuming my local IP is 192.168.1.64 the following would display the directory structure from another machine on my network:

will@will-linux:~$ fstree rpc://192.168.1.64:3000

Mounting

Any of the filesystems can be mounted on the OS with the fsmount command, which uses FUSE on Linux or DOKAN on Windows. The advantage of this is that the filesystems exposed in Python can be used in any application, and browsed with Explorer or Nautilus. The following creates a ram drive on Linux:

will@will-linux:~$ fsmount mem:// mem

Or on Windows:

C:\> fsmount mem:// M

Get the Code

There is no documentation online for the new features as yet, but if you are a brave soul and want to experiment with any of the above commands then download the code from SVN and run python setup.py install. The command line apps all have a -h which which displays help on the various options.

Bear in mind that these commands are still somewhat experimental, and some of these commands have the capacity to delete files – so be careful. That said, I'm confident to use them for my day-to-day work.

Please see the projects page page if you want to report bugs or discuss Pyfilesystem with myself and the other developers.

 
Search for Posts
2013
 
2012
 
2011
 
2010
 
2009
 
2008
 
2007
 
2006
 
 
© 2008 Will McGugan.

A technoblog blog, design by Will McGugan