April 15, 2012 will

Virtual currency site in testing phase

I made currency site available for testing today. See my previous post for the back-story, but in essence currency site is a virtual currency platform.

I sometimes object to the word virtual in ‘virtual currency’. Most of the money I possess is not in any physical form; it's merely a number stored in a database somewhere – and transactions occur without any kind of physical objects changing hands. So the word ‘virtual’ seems entirely redundant, since there's is no qualitative difference between virtual and ‘real’ money I can see. The only difference is the level of trust in the system.

But I digress. Currency site is a platform for virtual currencies, in that it is up to the users to create and manage currencies. The site just provides the tools. What currencies are used for is irrelevant as far as the platform is concerned. It could be for a house of students to manage the housework, or for a community to exchange goods and services. Regardless of what a currency is used for, there has to be a certain amount of trust in the system. The platform has to be reliable, in that you shouldn't be able to create currency without a valid transaction. Currency site is centralised, which makes that requirement simpler–the system keeps track of how much is owned, in the same way you trust banks to keep track of the money in your accounts.

The second level of trust is in the creator of the currency. The creator of the currency has the extra responsibility of defining how much of that currency is available at any one time. This is done by minting new currency. For instance, if the provider creates a currency and mints 1,000,000 virtual bucks then only 1,000,000 will ever be available to other users. It could be owned by a single person, or by a million people owning one virtual buck. Alternatively, since all currencies are divisible by 100, it could be that 100,000,000 people own 0.01 virtual bucks (a virtual cent?). However it is distributed there will be no more than one million virtual bucks in existence unless more is minted. A record of the currency mints is public, as well as information about how much currency exists and how much is in general circulation, which allows regular users to keep an eye on how the currency is managed.

From a techy side, currency site wasn't all that challenging. Sure it was a few months of part time work, but it was mostly user interface code. I wanted to make something that worked like online banking, but not as painful (I've never used an online banking system that didn't make me want to tear hair out). That was helped by using Twitter's bootstrap CSS framework, which creates an elegant user interface with simple markup.

There was only one piece of code that wasn't a straightforward as it appeared (and it was kind of fundamental). A currency site transaction basically involves subtracting a value from the source account and adding it to the destination account. In psuedo code, it is simply this:

if source_account.balance < amount:
    raise TransferError("Not enough currency")
source_account.balance -= amount
destination_account.balance += amount

In essence, that's all that is done, but things get more complex in the context of a web application where multiple transactions may occur simultaneously. For example, if an account A contains 30 virtual bucks and the owner attempts to send 20 virtual bucks to account B and simultaneously sends 20 virtual bucks to account C, one of those transactions has to fail – otherwise we may end up with a negative balances which is not allowed. The if statement checks if the account has enough currency, but if both those transactions occur simultaneously then they will both subtract 20 from the source account (leaving -10). Granted, this could only occurs in a small window of time, but there is no way to recover if it does.

I couldn't figure out how to handle this situation elegantly with the Django ORM, and I don't like resorting to custom SQL. Fortunately, the recent release of Django 1.4 came to the rescue with the addition of select_for_update, which does row level locking. Basically it allowed me to lock the two accounts objects so that no other process is permitted to modify them until the currency has been transferred. Another consideration is that the entire thing has to be done in a (database) transaction, since half a (currency) transaction could result in currency being subtracted from the source account without being added to the destination (in effect, disappearing currency from the system). To keep the currency consistent, the psuedo-code becomes:

begin_transaction()
lock(source_account, destination_account)
if source_account.balance < amount:
    raise TransferError("Not enough currency")
source_account.balance -= amount
destination_account.balance += amount
commit_transaction()

I don't think there is much else in the code that is blog-worthy, although there is still plenty of features I'm thinking of adding. I'm considering allowing users to trade currencies, which might be interesting. I would also like to build an API, so users could pay for web content with virtual currency. A few more evenings of hacking in there I think…

If you would like to help with the testing then head on over to http://currency.willmcgugan.com/ (username: currency, password: reliance). If you let me know, I'll send you 100 beta bucks for your time. Bear in mind its in an early testing phase, so don't use it for anything serious – I'll be wiping the database before it goes live. I'd also be interested in suggestions for a proper domain name!

Use Markdown for formatting
*Italic* **Bold** `inline code` Links to [Google](http://www.google.com) > This is a quote > ```python import this ```
your comment will be previewed here
gravatar
Stan Carney
Interesting concept. I look forward to seeing more!

In my experience account level locking in your above example will become a bottleneck when dealing with high volumes. It also can result in deadlocks depending on what you are using for a datastore and if care isn't taking to lock accounts in the same order as their object id's for example.

Assuming this is important to you, you could break balance calculation apart into 3 isolated events.

1) When a new financial transaction comes in it is posted to an insert only ledger (Transaction) while at the same time locking only the source account. Locking just the source account will get rid of deadlocks and reduce queuing in the database.
 begin_transaction()
lock(source_account)
if source_account.get_balance() < amount:
    raise TransferError("Not enough currency")
Transaction.objects.create(source_account=source_account, destination_account=destination_account, amount=amount, posted=False)
commit_transaction()

2) Schedule a BalanceSummaryJob to read the above Transaction records that haven't been totaled yet (posted=False) and store them as BalanceSumary record. This gives you the ability to report on historical balances as well.
 begin_transaction()
txs = Transaction.objects.select_for_update().filter(posted=False).order_by(id)
balance_summary = {}
for tx in txs:
    tx.posted=True
    balance_summary[tx.source_account] -= tx.amount
    balance_summary[tx.destination_account] += tx.amount
for a in balance_summary:
    BalanceSummary.objects.create(account=a, balance=balance_summary[a])
commit_transaction()

3) Account.get_balance() will get the latest BalanceSummary record for itself and sum any unposted Transaction records to it. Modifying it to accept a date will allow you to report on historical balances.
 def get_balance:
    balance = BalanceSummary.objects.filter(account=self).values('id', 'balance').aggregate(Max('id'))
    tx_amount = Transaction.objects.filter(account=self, posted=False).aggregate(tx_amount=Sum('id'))
    return balance + tx_amount

The code above is just to articulate some of the details. If performance was the intent some of the steps would be implemented differently.