Expected Downtime: 2/5/08 at 6:30AM PST

Tomorrow morning, at 6:30AM PST (9:30AM EST), we are taking a planned outage for approximately 2 hours.

Recently, our colocation provider was acquired by Your Net Connection who has to reassign IP addresses. On Wednesday, our old IP addresses will cease to work. Therefore, tomorrow is the drop-dead day to get switched over.

Realistically, we expect this to take about an hour. However, I'm notifying you that it will take 2 hours in case of an unexpected hiccup.

During this, we'll also be updating our DNS records. DNS typically propagates quickly these days, but it could take a little while for all users to see these changes.

Regardless of how long it takes to switch our IPs and DNS, your files will be safely and reliably stored thanks to our friendly partners at Amazon S3.

I'll be updating our Twitter account throughout the outage, so if you're unable to access the site after 8:30AM PST, please check Twitter to determine whether we should be available yet. If you still have trouble, please email me personally at iseff@iseff.com.

Thanks for your patience and understanding!

Ruby library v0.6: Updated for APIv2.0

Today I checked-in and released the latest version of our Ruby library. This version of the library is compatible with our new APIv2.0 (recently launched).

It's not quite a drop-in replacement for the previous version, with method name changes and some other things, but it should be fairly easy to get going.

You can get the new library at our Openomy SourceForge project page. You can also check out our general Openomy Open Source page.

Here's a little help to get started with the library:

Openomy::OpenomyAPI.setKeys("abcdesasdf", "1234567890")

a = Openomy::Auth::new()
u = Openomy::Objects::User::new
u.username = "foo"
u.password = "bar"
conf_token = a.create_confirmed_token(u)

# create a new tag
t = Openomy::Tags::new(conf_token)
new_tag = Openomy::Objects::Tag.new
new_tag.name = "New Tag"

resp = t.create(new_tag)

# get all tags
all = t.find(:all)

# get tag id 1
tag = t.find(1)

With that, the rest should be fairly straightforward to figure out. Enjoy!

Case Study: Using Haskell and HAppS for Openomy API v2.0

At Openomy we are continuously trying out new technologies and, in particular, programming languages. We've been pretty successful thus far (with some perl background scripts, a couple Ruby on Rails services, a C#/mod_mono frontend, a Java service, etc) and continue to innovate.

The latest release of our API is written in Haskell using the new HAppS framework. This turned out to be a really interesting project and one that fundamentally changed how I think about programming. I wanted to highlight some of the reasons we chose Haskell, what worked, and what didn't.

Why Haskell/HAppS

We initially chose Haskell/HAppS for a few reasons. First, we were getting a little tired of the same old OOP languages, and wanted an entirely different way of thinking. We're geeks and we like to try new things, especially those which make us think. Haskell offers that with functional programming. But, more than just being a "cool" language, it did have advantages that made it the right tool for the job.

Our backend is a collection of REST-based internal services. For example, we have a "files service" which deals with the uploading/downloading/etc of actual binary file data. We have a "metadata service" which deals with the metadata about a file (name, size, tags, etc). We have a "user service" which manages user accounts, sessions, etc. Each of these is fully independent and interact with each other through normal REST service calls. The front-ends (the web site or the API) tie them together as necessary.

Because of this, the API is essentially just a transformer: from the XML our internal services produce to XML that our developers can use.

Thoughts

Haskell's type system (and HaXml) work very nicely for our transformer, making it a great language choice for this project. It was simple to define our transformers and connect with HAppS to serve back to the client.

But there were many problems along the way. When we first started, we were working with HAppS v0.8.x. Quite frankly, there was a lot not to like about it at the time. There were many quirks and bugs and in general, things we just simply couldn't figure out. HAppS comes packed with a lot of features (such as an S3 library, a Facebook library, a stellar state management system, etc), most of which were totally unnecessary for us. This caused a bit of clunkiness as we figured out what we needed and how we could avoid using the rest. A lot of this comes from the fact that the HAppS documentation was (and is) absolutely NOT up to par. In my opinion, this problem is two-fold: (1) Haddock (the Haskell documentation engine) is just an odd interface and one I don't particularly like, and (2) HAppS' Haddock and other docs are almost non-existent.

However, by the time we were nearing our launch, HAppS 0.9.1 was launched and we decided to upgrade with it. There were some substantial changes, including basic things such as how to define routes for URLs. However, in the end, we were much happier spending some time getting up to date and using the new, nicer HAppS API. It ended up being much cleaner and much simpler to understand an maintain.

One thing about Haskell in general that really tripped us up initially was finding a non-code littering, easy to understand method of handling errors. After many iterations, we finally decided -- since we spent much of our time within the IO monad already -- upon using Haskell's Control.Exception. This ended up working out very well for us, and keeps our code relatively clean and concise.

Once we had everything implemented and tested, we moved towards figuring out how to deploy the project in production. We decided to do a very similar process as we did with our Java service, using Capistrano to build and deploy our service to multiple boxes. We then simply run round-robin DNS to load-balance these machines.

Since being in production, we've so far found very few issues and everything runs quickly and smoothly. Haskell and HAppS have turned out to be a very nice addition to our stable of tools and services.

Announcing the Openomy API v2.0

I'm very pleased to announce that today we're formally launching our new API (v2.0)! This is a huge step forward for Openomy, the developers using Openomy to develop applications, and the users of Openomy and participating applications.

The new API is much more "REST-compliant" (as opposed to our older, GETSful API) and should be much easier for developers to write applications against. It will also be much easier for us at Openomy to add features. We think our developers will greatly enjoy this new simplicity and ability to quickly iterate on your apps!

S3 Storage

Also with this launch, we're formally announcing that our default
storage engine is now Amazon's Simple Storage Service (S3). In addition to cutting costs, we believe using S3 will bring more peace of mind to our users and developers that their files are safe and secure in Amazon's data centers and will be highly available.

Achieving Our Vision

Launching both the new API as well as our new storage system completes a long process of rearchitecting our backend to allow us to grow more easily and iterate more quickly. The short-term benefits of this are quicker responses and higher uptime. The long-term benefits are allowing us to focus more closely on our vision of becoming the Online File System.

Over the past couple of years, as we predicted, the world is moving to an almost entirely web-based software model. We post our photos to Flickr, write our documents on Zoho, etc. Unfortunately, it's much too difficult for us to use our photos posted on Flickr within our documents on Zoho.

This is solved in the traditional desktop-based software model by the file system. Any application can access any file via a single API in the file system. There are (virtually) no data silos on the desktop. Any file is available for use by any application.

We're working to achieve this on the web. Our goal is to break down the data silos and allow applications to have access to all of a user's data. Moreover, we want the user to have full control over this. You deserve to be able to choose what applications have access to which data and have full data portability.

We believe this is the right model, and with these latest launches, you'll start to see this happening shortly!

P.S. Don't worry! The old APIs will stay on for a while, and we'll cover the deprecation process at a later time in another post.

Facebook and Twitter

I called this out briefly in my last post, but I wanted to make a special post just on this topic...

Openomy now has a Twitter account as well as a Facebook profile.

The twitter account is located at twitter.com/openomy and will keep quick updates about life in the project, bug fixes, and feature launches. Follow us!

The Facebook profile is around to build a small community of Openomy users. We'll likely use it to display some cool things, and perhaps even to let you know of neat things going on in the site! Become a fan!

Enjoy.

Corrected filenames on download

Happy New Year, Openomy users! A quick update on the status of Openomy: 2007 saw explosive worldwide growth of Openomy. We now have many tens of thousands of users. We went through many growing pains, but I believe we're through the worst. We've implemented many architectural changes (such as using Amazon S3 for default storage) and are now in rapid feature-development mode. 2008 will see many, many new changes, small and large.

As a New Year's Present to you, our users, I wanted to write a quick note and let you know that we've fixed the annoying filename bug. If you hadn't noticed, many files would download with the very unclean filename of "openomy-{fileid}" rather than the filename of the actual file uploaded. This has finally been fixed. The fix is out in production for any newly uploaded files. Your files will also be fixed if you choose to rename them.

Also, I want to note our new Twitter account where we'll be posting frequent, short notes about what we're up to at OpenomyHQ. Please follow twitter.com/openomy to see these!

Thanks for all the contacts about this and making it a priority for us to fix! Stay tuned for a very large announcement next week!