snaptrip: introducing overview and export

As I've written elsewhere, in 2009, I'm hoping not just to launch new sites, but also to keep returning to my other projects and keep on extending them. snaptrip's no exception, so I've returned to an unfinished page which tried to take the single list of trips and give an overview of your travel. It's now a part of the site; follow the link at the bottom of the block of text on the home page.

Of course, this has happened pretty much simultaneously with Dopplr's unbelievably gorgeous Personal Annual Reports, which are (deservedly) getting a lot of attention at the moment. However, there are some reasons why I think snaptrip's overview is also worth looking at at.

Firstly, if you have trips before 2008, they're taken into account on the overview (complete with a per-year breakdown). Secondly, the overview is (more or less) live - if you add a trip on Dopplr, it'll show up within the hour on snaptrip. In contrast, there's not currently a way to request a new annual report PDF.

Thirdly, the overview highlights some different information from your data. For example, it totals mileage (well, actually, distance in kilometres) on all trips for your modes of transport - planes, trains and automobiles, for example. I also decided early on that I'd like to see country-level aggregated data, so there's both a map of countries visited and a list detailing how many trips you've been on to each (as well as an ordered list of top cities).

The launch of the Dopplr reports also flushed out a bunch of ideas for panels I'd like to add. A fairly common request was for personalised Flickr photos in the reports, and this is obviously natural snaptrip territory. I suspect I'm going to run into some of the issues that stopped Dopplr themselves doing this, but it's definitely on the todo list.

However, I have been able to quickly build one of Meg Pickard's requests: you can download a CSV listing of all your trips, including distance travelled and the transport mode, which will hopefully let you make your own visualisations and discoveries.

It feels as if this year might be one where personal informatics makes it into the mainstream; I'm hopeful that snaptrip's overview and export can play a small part. (Dopplr's report already played a large one.) Please try them, and feel free to suggest things (and let me know if you run across any bugs).

[Edit] Apologies for the site's flakiness this afternoon (UK time). I think it's back to more or less being happy now. Session management has leapt right up the todo list.

snaptrip: talking to Flickr with GAE

For snaptrip, and probably for any future Flickr projects I do on Google App Engine, I use Beej's Flickr API module. As I've said before, I like libraries that use the strengths of dynamic languages. Rather than implementing every API call as a method, it's far better to use Flickr's own reflection methods to build checking, and a generic call handler to return JSON data. On the other hand, you do want something to handle all the authentication signing.

However, there's a small catch: the default build doesn't work on App Engine, because of the unique way that the BBC is funded fact that GAE doesn't support Python's core urllib/urllib2 libaries. Instead, it uses special methods like urlfetch to, well, fetch URLs. For some libraries (like the excellent Universal Feed Parser), you can work around this by fetching the content yourself and then passing it in to the library.

Unfortunately, that's not really a workable approach here: the entire point of a Flickr API library is that it handles talking to Flickr for you. So I delved in to the code and have produced an ugly, but working, patch against the last SourceForge version. It's a little ugly, and out of date, but it works for me, and if you need to talk to Flickr with Google App Engine, hopefully it'll work for you, too.

snaptrip: some thoughts

Having finally got snaptrip out there, I'm hoping you'll allow me a little (pretentious?) waffle about why I wrote it, where it fits, how I made some of my decisions, and what's next.

I'm a big fan of Flickr's machine tags. Most of my images have at least ten - mostly generated automatically, like my EXIF machine tags - and I tend to add geographic metadata as well. As such, it's probably not a surprise that I'd write an application that made Dopplr trip IDs available. The big surprise is that I bothered to make it accessible to most people, by building it as a website not a script.

Why a website? Well, I thought I'd like a nice interface as much as anyone, and I also know that to make a machine tag truly useful you need as many people as possible using it. Asking folk to download a script, get a key, and use a command-line interface - or no interface at all - isn't going to work.

Speaking of Dopplr, I don't think I've seen a talk by anyone there since it started, but I do think I've picked up their philosphy from slides and abstracts online. The phrase that tends to crop up is a "coral reef", the idea being there's a web of data that's available on the internet and that by doing one thing, and doing it well - the old Unix philosophy, really - that you can live in a happy niche. Well, snaptrip lives on part of the coral built by the two companies whose API it consumes.

I'm not under any illusions: it's likely that most users won't care about their past trips, or matching their Flickr photos. Those who do will probably only visit the site once, tag a few trips, and then leave. That's fine.

In my previous post I alluded to some decisions I made about the geotagging features in snaptrip. To be honest, it wasn't something I'd considered at first, but seeing Richard Crowley's Dopplroadr hack - which does some of the same things as snaptrip, but when they're uploaded rather than by looking for existing Flickr photos - made me consider the possibility. However, because I am looking at things that have probably accumulated metadata already, snaptrip is careful not to overwrite any information that's already there.

snaptrip adds fewer tags than Dopploadr. It won't add human-readable tags at all, and it adds the geographical data at a relatively low level of accuracy. I didn't want snaptrip to assert with precision that all these photos were taken dead in the centre of Copenhagen, since they probably weren't. My US trips show exactly the sort of thing I'm talking about: most of my pictures are actually taken anything from ten to two hundred miles from where Dopplr thinks I was staying. Similarly, it doesn't set a woe:id machine tag, instead preferring to use the dopplr:woeid namespace/predicate pair.

It's quite possible I'm overdoing the paranoia here, and so I'll probably add the option to set more tags later, but for now, I'm happy to tread lightly. (In that vein, snaptrip doesn't set a visible "snaptrip" tag, like many apps (Shozu and AirMe spring to mind; Picnic also suggests adding its tag). However, it does set a dopplr:tagged=snaptrip machine tag, and I should probably make that optional also. For now, you can use Flickr's tag tools to delete it.)

So, what's next? Well, the basic functionality I wanted seems to be there and stable, so I'm now considering two further avenues. I'm trying to develop tools to give you some views on the aggregated data from your past trips, but perhaps I should instead be looking at tools to increase the amount of stuff in that Dopplr history. I've got a couple of ideas...

snaptrip: a weekend of changes

I'd hoped to do a lot to snaptrip over the weekend, and I pretty much managed to do what I hoped for. There's a lot of changes which went live today; a lot of them are kind of invisible, but hopefully they're all useful.

Firstly, there's some user interface changes that incorporate suggestions from people within Dopplr. You now have to authenticate with both Dopplr and Flickr before logging in; it was possible to use snaptrip with just the one, but it didn't really make a lot of sense. When you tag photos, they get a border in the city's colour (as they do if they've already been tagged). I've also removed the requirement to load all the photos before tagging them; you now get to do so in batches of 24.

Secondly, the behind the scenes changes. Google App Engine makes it really easy to use memcache to avoid loading data more than once, so I now use that, making reloading pages really quick. There's much better error detection, especially for when a network call to either of the services it relies on fails. I've also fixed the sort ordering in the statistics panel on the trip list page.

show-dopplr-link

Thirdly, I've added a lot more functionality to make use of the location information on both sides. Your map now shows Flickr photos*, and it's a "slippy map", so you can zoom in and out and recenter it. (It also shows the departure point.) It's also possible to use the Dopplr information about trips to add photos to Flickr's map. I should probably make it clear here that snaptrip will never overwrite location data (or trip data) that you've added yourself.

Finally, I had a comment on a previous post asking why you couldn't see trip tags on Flickr. Assuming that snaptrip worked, then the machine tags it's added aren't even shown by default- you have to open the disclosure triangle in the right hand pane. However, if you're using Firefox and Greasemonkey, you can install the show-dopplr-links user script, which will display a badge and link - like those for Upcoming - on each photo page.

Hopefully this isn't the end- there are still features I want to add, and the usage of the app when DopplrHQ first publicised it was a pleasant surprise. I hope these fixes and additions make it useful for you, though.

* Usually. There's a philosophical point I hope to expand on in another post.

snaptrip: follow-up

Just a quick note to say thanks to all of you who've been using snaptrip today, since it was mentioned on DopplrHQ's Twitter stream. As it says in the questions and answers on the front page, I'm still poking at lots of bits of functionality, and you may see the odd error when I haven't properly caught one of the web services I use failing to send back some text. Despite all that, I can see that the app's been fairly popular, and hopefully it's been useful.

However, I've just spent an evening tightening up some of the text, and also working on how to show that an image already has either Dopplr trip tags or location data. (You've probably noticed there's a link that doesn't work- I didn't expect to be releasing quite so soon- but it might give you a clue as to what's coming next.) I'll try and keep posting fairly regularly, but for now, thanks again.

snaptrip: an announcement of sorts

I've finally got to the point where I'm happy to really start posting about snaptrip.

snaptrip is a little web project that lets you use Dopplr and Flickr together. Initially, it allows you to put machine tags - specially formatted bits of data - on your Flickr photos. Why bother? Well, Dopplr itself uses this data, if available, to show you photos on its site.

Obviously that's a bit dull, and I do have further plans going forward, but because I've been stopping quite often to polish the app as I was building it, it's been a bit slower than I'd like. (I've also been using it to learn about both Python and Google App Engine - the appspot.com gives that bit away).

Anyway, once I've sorted out a couple of little niggles, I'll probably post more about this elsewhere, but for now, if you have a Dopplr account, feel free to try it.