Donate using PayPal

CycleStreets blog

News from CycleStreets

Archive for the ‘Routing performance/quality’ Category

Merging tool – new cycling data

Monday, December 19th, 2011

As previously announced, we are working with the UK’s Department for Transport to make advanced cycling data attributes available for incorporation into OpenStreetMap.

Rather than organising this along the lines of a bulk import, we are taking advantage of new technologies in Potlatch 2 and have commissioned Andy Allan, creator of OpenCycleMap, to develop new features to allow volunteers to collaborate on inspecting and merging the information into OSM.

This merging tool will also be of use for other external data that could be manually inspected and merged into OpenStreetMap.


The DfT commissioned survey work in various cities around the UK for their Transport Direct product. In 2011 they released the results of the surveys as Open Data, in a complex GML format based on Ordnance Survey ITN data – unsuitable for use with OSM. However, in addition, they have funded work to convert the survey data to be based on OSM geometries suitable for incorporation. This has been done through brilliant work by Ralph of CCG.

The kinds of things surveyed include cycle routes, cycle parking, cycle lanes and their widths, surfaces widths and lighting conditions of cycleable paths, and so on. We are working to add support for these attributes into CycleStreets, so that routes are further improved.

In the UK wide areas of the cycling infrastructure have been mapped in OpenStreetMap, often more recently than the data from the DfT. Also, with the development of Vector Background layers in Potlatch 2, there was an opportunity to create an improved process for dealing with external datasets.

Further background information is available in blogs and on the mailing lists.

The demo

We’re pleased to announce that a demo is now available, and we’d like people to test it.

A demo is now available. It contains sample data for Nottingham and Cambridge, but it’s deliberately unable to save the data back to the main OSM server. When the final version of the data conversion is complete and available, this will be updated and fully able to work.

Two test areas are currently loaded:

How to use the merging tool

The merging process works as follows:

  1. Click ‘Map style’ > ‘Wireframe’ to make things much easier to work with.
  2. The background data is highlighted either orange (needs attention) or blue (already processed).
  3. Click the background features to select them.
  4. Ctrl+click (or cmd+click on a Mac keyboard) the relevant OSM feature (line) to see a side-by-side comparison.
  5. Click on ‘Advanced’ in the left panel to see the merging controls.

Feel free to play around with this – the snapshot data is being reloaded from time to time as we get better imports, although we think we’re almost at a stage where the data conversion is fairly bug-free.

The merging tool is currently a beta and further improvements are planned. See the main Merging tool page on the OSM wiki.

The first screenshot shows the thick gray line (DfT data, as a background layer) highlighted. It shows the attributes it has:

The second screenshot shows what happens if we now control-click (or cmd+click on a Mac) on the OSM line – we now get a merging interface where we can accept/reject each attribute, and click the button at the end to accept all the changes:


Feedback on the data

We would really welcome feedback as to any errors you spot in the data conversion. The aim is that the data is pre-processed and snapped to the OSM geometry as effectively as possible, so that merging is merely a case of manual confirmation of each attribute according to your local knowledge.

Issues we have fed back so far on are:

  • Alignment. The data was originally snapped to an OS Open Data, and has been geographically aligned via advanced GIS techniques to OSM. It’s already well over 90% matching and further improvements are being made.
  • The issue of streets being broken up but having the same data. Our GIS contact plans to merge when the street name and data matches.

The software

A number of software components are used to make all this work

  • Potlatch 2 is used as the editor, and can load data from both OSM and the DfT data. The splash pages and other resources are available on github
  • Snapshot Server is used to serve the DfT data for each user, saving them from having to load the whole country at a time
  • Some scripts are used for loading data in and out of the server. These use Osmosis to read/write between XML and Postgres.


The data is expected to be released under the Open Government License. We have been seeking an early letter of confirmation from the DfT on this and will update this page and the OSM Wiki accordingly. (The ITN-referenced dataset is released under the OGL already.)


We’d really appreciate it if you could try out the beta and add comments below, or contact Andy Allan with any feedback you have. Did you figure out how to use the tool? Did you manage to merge some data? What doesn’t work? How could the tool be improved?

If you have local knowledge of the areas in question, it would be great to hear back from you on the datasets themselves – do they match reality? Are the tags appropriate?

Merging tool for external data for OpenStreetMap

Monday, October 24th, 2011

Back in July we wrote about our work to assist the DfT with the opening up of their cycling data, a dataset featured in the Telegraph this weekend. Over the summer, much progress has been made.

Andy Allan, who is perhaps best known for his excellent OpenCycleMap project, has been working for us to create a merging tool (within Potlatch 2) for helping to combine external datasets into OSM on a street-by-street basis. This tool will be useful not only for the DfT data that is becoming available soon, but also for other datasets.

The DfT data, which has mostly been collected by surveyors on bicycles, has the potential to significantly improve the quality of routing in some areas of England. We are well aware, however, that data collected by other agencies can undermine the work of OSM volunteers in the area if not handled sensitively, and so we've stressed that automated, bulk imports would not be accepted by the OSM community.

Instead, the approach taken is a method for OSM volunteers to inspect and merge the information on a street-by-street and attribute-by-attribute basis via the simplest and quickest means possible, using their local knowledge to enhance the end result.

Surface type, cycle lane widths and other data are amongst the attributes in the dataset, and these will shortly be supported in the routing engine.

The code for the beta release of the tool [included in this diff] has been reviewed and merged into the main Potlatch 2 codebase, and is already in the latest builds. We expect further changes to be made as we get feedback from initial testers.

OSMers should be able to try out the beta and the data soon as we have confirmation of the Open Government License for the data. Here are screenshots of the beta:

Detailed cycling attribute data for better cycle routing

Wednesday, July 20th, 2011

We're aiming with CycleStreets to provide the highest possible quality cycle routing, to give people trust in routes they plan. We've heard from many users how our routing is helping them give the confidence to use a bike for their journeys, and from people who've discovered cut-throughs and safer, easier routes for their existing journeys.

Increasing the quality of the routes found by CycleStreets means using more sources of good quality data. For instance, a cycle lane can improve a planned cycle journey, but not if the cycle lane is too narrow. On the other hand if the cycle lane is wide and has a good surface, it can be better than a shorter route on a busier road.

OSM logo

The information that CycleStreets uses to base its route recommendations comes primarily from the OpenStreetMap (OSM) project. Over time that data has become more detailed in both depth and breadth, and it continues to do so.

Over the last 18 months, the UK's Department for Transport (DfT) has undertaken a GPS-based survey of cycling infrastructure in towns and cities around England. This has been used for a related project, the Transport Direct multi-modal journey planner.

The DfT is keen to see this data used more widely and we've been talking to them about using it in our routing, by making it available as open data that could be merged into OpenStreetMap.

We're delighted now to announce that we're helping the DfT with its laudable objective to make this data more widely available. We’re working with its contractor, CycleCity Guides, who are well-known for producing a wide range of Local Authority cycle maps. The release of this data is one of a number of other datasets that the Cabinet Office has recently announced will be made available.

Rather than merely dump the data on, the DfT is going a step further to help it be used, a development it should be highly commended for.

Respecting the way the way the OpenStreetMap community works, the DfT is planning to:

  • Make the data available in a fully OSM-compatible format, aligned to OSM geometry with converted attributes.
  • Simultaneously publish a dataset aligned to Ordnance Survey's (OS) Open data
  • Use a standard, OSM-compatible license (the Open Government License), with the data unencumbered by OS derivative data issues.

This data, which has mostly been collected by surveyors on bicycles, has the potential to significantly improve the quality of routing in some areas of England. We are well aware, however, that data collected by other agencies can undermine the work of OSM volunteers in the area if not handled sensitively, and so we've stressed that automated, bulk imports would not be accepted by the OSM community.

Instead, useful data needs two things if it is to be used in OSM. Number one is a way of inspecting and accepting/rejecting the data on a street-by-street basis via the simplest and quickest means possible. Secondly encouraging routing engines and renderers to use the data. Therefore:

  • Funding we've obtained will pay for a month or two of solid work on Potlatch 2, the default editor on the OSM website. We've engaged Andy Allan, one of Potlatch 2's core developers, for this. The funding will lead, amongst other improvements, to a generic tool to enable donated data to be merged in (or rejected), street-by-street via manual inspection and approval. A range of general usability improvements (such as those in the P2 buglist) will also be funded.
  • We'll be implementing support for many more advanced routing attributes, which Andy and hopefully other OSMers will be helping with. This will demonstrate the difference that really detailed data can make to the quality of cycle routes found by engines like CycleStreets when the community merges in (by inspection) this type of data.
  • A range of other improvements will also be made, for instance, changes to our feedback system so that errors in OpenStreetMap, found as a result of people using the routing, can be more easily discussed and fixed in OSM.

We hope the OSM community will react positively to these developments.

With community support, this data should help get lots more useful data into OSM and help it become a superbly detailed dataset ever more quickly.

We've been particularly impressed at the way that our contacts at the DfT have been open to learning about the way the OSM community works. We particularly hope that the success of this project will act as a demonstration and lead to more trailblazing open data initiatives where government learns from existing communities to 'do open data the right way'.

Hiring soon – to work on routing engine challenges

Saturday, May 21st, 2011

CycleStreets will shortly be hiring a full-time core developer to work on our routing engine and website. The aim is to enable us to implement improvements to the routing and implement lots of feature requests a lot more quickly.

This is possible thanks to a series of contracts and grants we've recently obtained, some of which we've recently reported about on this blog.

Our core aim with CycleStreets is to create "routing that thinks like a cyclist". This means including some things, such as turn delays and complex rules that would seem to be challenging to implement with off-the-shelf engines.

We're therefore wanting to take someone on who is able to work on what are interesting computer science problems (as distinct from purely "web development"). However, we'd hope that a suitable person can help work on the website (e.g. backend/frontend integration, general web development. refactoring) as well.

Background info: about our routing engine

The CycleStreets routing system finds, quietest, fastest and balanced cycling routes between two points in the British Isles. It does this by building a route planning version of the map (known as a graph) for each of these types of route.

Each graph consists of vertices and edges that correspond to junctions and streets. On each graph the length of an edge corresponds to the cost for that route type of travelling along a street. So, roads and wide cycle tracks will have relatively short links on the graph for fastest routes, but the links that represent footpaths and bridleways where the cycling speed is slower will be longer. By contrast, on the graph for quietest routes, cycleways and park paths will be short relative to busy roads.

The optimal route of a certain type will then be the shortest path on the graph for that type.

So there are two main parts to the CycleStreets routing system: (1) Building the graphs (2) Finding the shortest path. The quality and performance of CycleStreets is governed by how well we do 1 and 2 respectively.

The current OpenStreetMap (OSM) map of UK & Ireland contains 2 million streets that pass through 20 million points. Streets are usually represented by their centre line, with nodes at junctions. The characteristics of each street, including for example the type of road, whether it is hilly, has a cycle lane, or is a one-way street are used to measure the suitability for cycling. In graph theory speak, they translate into the cost of an edge.

These costs are measured during our daily translation of OSM's UK & Ireland map into a graph by application of a series of rules.

That system is currently rather more complex and opaque than we'd like. Choosing the right values for all of those numbers has been an iterative process as we've tuned the routing engine, often in response to our user's feedback.

We need to:

  • Review the ruleset, re-examining the assumptions they are based on and the order they are applied
  • Make it much clearer to users how the rules have influenced the chosen route (this will confer greater confidence in the generated advice)
  • Include additional sources of information (more information to be published soon on this)
  • Handle junctions better, which is of particular interest currently.

Here in more detail is discussion on some of these:

Junctions – reducing wiggly routes

Simple graphs constructed from maps of street centre-lines lack are limited in their capacity to express how easy it is to cycle through a road junction. This series of photos introduces the problem.

In essence, the simple graph cannot represent banned turns, and nor can it indicate that a right turn across a busy road might incur quite a wait until it is safe to cross. However, some of the turn delay modelling can be derived automatically, knowing the 'before' and 'after' street types, and the turn angle. We believe that accurately representing that sort of information has the best chance of improving the quality of routes recommended by CycleStreets.

The way this route wiggles on and off Glasgow Road is a good example of this problem. (NB Click on the + button in the top-right corner of the map and switch to the OpenCycleMap view to get a better map background.)

Implementing turn delays at every node presents real challenges in terms of the use of a standard A* routing engine. This is further more difficult given the need to avoid very large amounts of RAM being needed.

Solving this problem is a significant technical challenge. It needs to be transparent so that there is a clear link between each piece of data and each rule used in the solution and the decisions that a knowledgeable cyclist would make. The solution must also be scalable, so that neither the amount of time taken to find a route nor the amount of data required to represent the problem, increase significantly. That will make it attractive and manageable. A solution also needs to avoid creating excessive hardware cost requirements.

Finding the shortest path

Shortest path algorithms are very familiar to computer science students, and there are many well established standard techniques that solve the problem. CycleStreets currently uses the A* algorithm, which is implemented in a combination of a Python program and a C++ module.

The 2 million streets and 20 million points of the map, become 5 million edges, and 4 million vertices in a graph. These are compressed by a non-standard technique, called Cell Optimisation (Cello), into 2 million edges and 1 million vertices.

The Python/C++ program can scan the compressed graph to find a route from Land's End to John O'Groats in 2.1 seconds. The vast majority of routes we're interested in are about 10 miles, which can be solved by this program in milli-seconds.

The process of interpreting the generated route from a list of edges and vertices into an itinerary that includes street names and turn directions – the 'unpacking stage' – and storing in a database takes much longer. Performance seems to vary quite a bit, sometimes its very quick but it can take several seconds to process, even for a typical-sized route.

This issue will grow in significance as the number of users of CycleStreets continues to grow, and as new ways of using the routing system emerge.

We want to make the performance aspect of the routing itself, as well as the 'unpacking' stage, as fast and efficient as possible.

More types of route

There is a growing clamour for many more customisable types of routing. Ultra-quiet routes, routes where there is no dismounting and routes that are indifferent about hills.

We should also consider taking into account intermittent links, such as ferry crossing times, or gates that are closed at night.

Our system uses pre-compiled graphs, so each new routing type potentially means adding a new graph. Ulitmately RAM sizeputs a cap on that approach.

Are you the kind of person who can tackle these interesting problems?

We think that these and other challenges form interesting computer science problems.

Do get in touch if you'd be interested in an informal discussion to discuss things further. A formal job description and other details, including application deadlines and details will be online very shortly.

Improving cycle journey planning in Scotland – with Cycling Scotland

Thursday, April 28th, 2011

Cycling Scotland

We're pleased to announce that we are working with Cycling Scotland to enhance cycle journey planning in Scotland!

Cycling Scotland, the organisation charged with getting more Scots on their bikes, runs a range of initiatives such as Bikeability Scotland, the freshnlo Pedal for Scotland bike ride, cycle instructor training and more. They are keen to provide cycle journey planning – to help remove a key barrier that people face when starting cycling or when they move into a new area.

As part of their journey planning activity, Cycling Scotland are extremely keen to motivate local community groups to map their area into OpenStreetMap, which forms the heart of CycleStreets' journey planner. Although there are areas like Edinburgh which have very high-quality mapping, thanks to the great work of OpenStreetMap volunteers there, other areas of the country are not so well-covered.

To help, we will be creating resources to help local communities with this mapping activity. Principally, this will involve creation of a user-friendly guide which introduces OpenStreetMap, explains how we use it, how people can collect data, and importantly outline the key things that improve the quality of cycle routing. (We hope this guide will also be of wider use to the OpenStreetMap community elsewhere, too, even though it will of course be tailored for Scotland.)

Alongside this work, we'll be creating a customised journey planner for Cycling Scotland, to be hosted on their website. This will benefit, thanks to a grant from them, from the introduction of more advanced routing attributes in our journey planner engine. By encouraging people to collect more detail about the cycling environment in their area, this will improve further the quality of our routing. Naturally, this will all be explained in the user-friendly guide for collectors.

Cycling Scotland are also supporting us to make our routing available more widely on different types of mobile phones, so that it is as accessible as possible.

We think this model of helping get more people cycle by engaging local communities and building on existing work is a brilliant model.

We are looking forward to undertaking these activities in partnership with Cycling Scotland, and will report in coming months as each part is completed and made available.

OpenCycleMap in Scotland – cc-by-sa OpenStreetMap contributors

CycleStreets – review of the year

Sunday, March 20th, 2011

Today is our second birthday – CycleStreets was launched on 20th March 2009.

The last year has seen a huge amount of development work, leading to new features, speed improvements, and more. However, the next six months will be even busier as the project really ramps up!

In the first year, CycleStreets planned 67,000 routes. In our second year, around 437,000 routes have been planned, and the rate of increase continues to climb. By November we had planned enough routes to cycle to the moon ten times, and in February, we reached the milestone of half a million journeys planned.

CycleStreets usage levels rising

Dover to Cape Wrath

A major challenge we faced a year ago was the technical challenge of generating the routes fast enough.

A year ago, CycleStreets used a routing engine written in PHP (!) that we created for the Cambridge-only predecessor of CycleStreets – the Cambridge Cycling Campaign journey planner. It was slow, taking half a minute to plan a route across London, and taking up most of the system resources. Effectively, it was the wrong technology and didn't scale to UK-wide routing.

We held our first Developer Day, which lead to very productive discussions about the routing engine and how we could provide routes to users of the site faster. A friend of the project, George, wrote us a new engine (using Python) which lead to a massive speed-up. Then Robin, another volunteer, took the Python engine and created an even faster version in C++. This has been in place for most of the year and has quietly sat at the heart of the system, planning routes in a few GB of RAM while barely challenging the processor.

The work on the routing engine meant that we have been able continually to increase the maximum planning distance, which is now 200 miles (320km), which is well above a day's cycling! The development version of the system can even now do Dover to Cape Wrath!

Improving the routing speed was a key requirement for mobile apps, several of which signed up to use our routing through the year. These include the leading app for the London cycle hire scheme – London Cycle: Maps & Routes, plus two other excellent 'boris-bike' apps, the briliant and world-first 3D bike satnav app, Bike Hub, BikeRoute for Android and, of course, our own CycleStreets for iPhone app.

Bike Hub app  Cycle Hire app  London Cycle: Maps & Routes  London Bike app  BikeRoute for Android

Our own iPhone app was made possible thanks to two grants we successfully applied for.

Our Android app is nearing completion, and like the iPhone app is being developed as an open source project. Thanks to our mobile developers for their brilliant work on these.

CycleStreets app

Through the year we have given various presentations and got involved with various social enterprise -related activities., such as WhereCamp EU, CamTechNetCambridge Geek Night and Net2Camb amongst others. These events lead to interesting discussions and also resulted in useful new contacts, such as people helping out with our mobile apps.

It was a particular plesure to give a presentation to Net2Camb as it gave us the opportunity to speak about the challenges faced by us as a not-for-profit social enterprise, rather than purely talking about technical challenges.

We have launched a funding drive for £130k to raise funds for two full-time developers. Such funds would enable the project to move forward much more quickly.

The DfT has this year been collecting cycling data which we are keen to see added to OpenStreetMap. We have since had informal discussions with Cycling England about use of the data, and how conversion of the data might be undertaken and at what cost. Discussions have been positive, and we feel this data would improve the quality of routes that we can deliver to users.

Over the year, more and more governmental bodies have been linking to us. For instance, in April, Cycling Scotland linked to us, and we are keen to work with them to help motivate people to improve OpenStreetMap data in Scotland. Others, including some of the Cycling Demonstration Towns like Chester and Lancaster now link to CycleStreets, and we have just sent a new brochure to councils around England.

Increasing the flexibility of the CycleStreets platform has been an ongoing priority.

West Sussex Cycle Journey Planner

In February we created a customised cycle journey planner for West Sussex County Council, building on work we have done to make it easier for organisations to have a journey planner within their website. Another has been created for the Bike Hub website, and a demo Local Authorities site is available.

The year has also seen a few developments on the Photomap. This is an area we would like to do much more on, as explained in our GeoVation bid for which we have now been shortlisted.

We created, under contract for Cambridgeshire County Council, a site called 'Cycling Sorted' to help manage the shortage of cycle parking in that area. We are keen to create similar sites for other Local Authorities. We have also created a similar system to support the great work of London Cycling Campaign.

OpenStreetMap is the backbone of our project, and we have been pleased to promote OSM and encourage more mapping for it. Over the summer we helped obtain a database of all the bike shops in the UK, for use in OSM, from the Association of Cycle Traders. Much of this has been merged into OSM, but more needs to be done to complete this crowd-sourcing exercise.


CycleStreets' use of open data saw it being featured on the front page of the government's new data website –

Throughout the year, we implemented many smaller improvements and innovative new ideas, such as the new shortlink domain, our new Photo of the Day on Twitter (featuring the best of the 25,000+ pictures in the Photomap), a new gallery viewer, better facilities to link to the journey planner, adding an integrated editor (Potlatch 2) as well as various ongoing design/usability improvements (though there is much more to be done, time/funding permitting).

Routing quality work, however, remains our highest priority. Our aim is to provide the highest quality routing possible for cycling, using our knowledge as cyclists. Various improvements have been made recently, and we are currently working on new routing attributes and reducing the wigglyness of some routes, which is proving a difficult problem to solve with limited hardware resources.

Simon and Martin, lead developers, would like to thank a range of people who have helped out in various ways, such as Andy, Shaun and David from OpenStreetMap, George and Robin for work on the routing engine, huge support from Chris in Edinburgh, George from Camden, our mobile developers – Alan, Neil, Jez, Theodore, Christopher and Jonathan, advice and a free dev server from our brilliant web hosts Mythic Beasts, our designer Ayesha, Jeremy for occasional advice on business matters, support from key individuals at the CTC, LCC and Cycle Nation plus others in our stakeholder group, Carlton and Bike Hub, helpful ideas and data from cycle campaign groups around the UK, and of course the amazing community of OpenStreetMap contributors whose mapping makes everything possible.

Lastly, we would like to thank our users, whose cycling needs provide us with the inspiration to keep going, and who provide us with much feedback and many great ideas.


Routing engine work in progress

Sunday, January 23rd, 2011

Over the last month or two we've been working heavily to rewrite the way the system imports and interprets OpenStreetMap data. This work is intended mainly to make the system give better routes, but also to make things clearer for OpenStreetMap mappers as to how we interpret the data, as well as make the code easier to maintain.

Routing work is complex and time-consuming. As such it's an area which our Funding Drive is aimed to help with – with full-time developers we could make improvements like this much more quickly and stay at the vanguard of cycle routing.

Below are two screenshots showing this work in progress. The first shows the router as it currently is, with the green/quiet route (most relevant to signed routes) not fully sticking to a signed cycle route (in blue) in the West Sussex area. The second image, from revision 5483 of the engine, shows how it is sticking to the signed route almost perfectly. (Note that this code is not released yet.)

Before (current site):

Routing engine in development (revision 5483):

We'll also be publishing soon a clearer guide for OpenStreetMap people on how we interpret the data tags and which of the more advanced tags we currently take account of and which we plan to take account of soon.

The latest work should also address some of the issues we have seen regarding routing over poorly-surfaced routes, where the data exists in OpenStreetMap. Importantly, support for these and other advanced tags will also encourage people to go and collect the data, benefiting other OSM data users too.

CycleStreets: Our Story – presentation to Net2Camb event

Friday, January 14th, 2011

We really enjoyed the January Net2Camb Meetup event, where one of our lead developers, Martin, gave a talk 'Our Story'. Thanks to Claire for organising the event and everyone who came!

It was particularly enjoyable as it was a rare opportunity to talk about the business and competition aspects of CycleStreets, about the challenges we face, and the future opportunities for the project.

We were also pleased that a couple of people came forward as new volunteers!

Here is our presentation [link]:





View more presentations from CycleStreets.

Dogfooding in London

Saturday, November 13th, 2010

Dogfooding, for those who haven't come across the term, is programmers' jargon for 'trying out your own product', i.e. eating your own dogfood!

While cycling around in London recently I made use of two great cut-throughs, thanks to our iPhone app and the great mapping done by OpenStreetMap folk.

These are the kind of cut-throughs for bikes which would never have found these looking on the default map/routing on the device.

Kingston: South to central

A lovely cut-through across the river and near the University.

I later also took the opportunity to add a location needing cycle parking, at some nearby shops, for the Cycle Parking 4 London campaign.

Waterloo to King's Cross

I often need to do this route, and for a long time I've gone along Euston Road and straight down Russell Square, which I've never liked due to the traffic.

Instead CycleStreets found me a tiny cut-through off Guildford Street, which led to a direct and fairly traffic-free experience running parallel. (The cut through seemed ambiguously signed, so might be a short dismount section, but still worth it for the rest of the journey.)


I did find that the Drake St junction has changed a little from what's on the map, so I've added this to the OpenStreetBugs website (I could have reported it via our own feedback system, but that would be like sending myself e-mail!), so that it can be resurvey soon by a local OSMer.

If you like the app, please do give us feedback in the App Store!

Technote: Versioned routing database

Wednesday, September 8th, 2010

A techy post, since we've not had one for a while!

Over the weekend we rolled out some new infrastructure that has taken most of Simon's time for the last month.

And thanks to donations and grants that we received, we now have a machine that works primarily to churn out imports without slowing down the main server in the evenings, as used to be the case.

Versioned database for routing

We now run a versioned database for the routing. Effectively each import of the OpenStreetMap data to make a routing database is now a dated and separate database. Previously we prepared an import and moved it into place, then shortly removing the old one.

The main benefit of this is that when a new set of data has been imported from OSM we can switch to using it instantly without having to shutdown the database.

The other benefits are that it is much clearer where the routing tables are – and has also resulted in a useful purge of some unneeded data.

The ultimate intention of this work is to enable us to run a much more frequent import cycle. Currently the data is refreshed weekly.

More efficient route generation

The itinerary listing details that you get when you plan a route are extracted from tables generated during the import (rather than re-calculated for each route).

There were quite a few inefficiencies here, and we've restructured how the data is extracted to make it quicker. This forms the conclusion of work to speed up the system for mobile use.

This will mainly be of benefit to API (e.g. mobile phone) users, as the website version itself still has to generate the maplets (which we plan to change in some way in due course, following user feedback).

These changes should not have affected the API output, except hopefully making it faster!

Turns details

The routing output also now includes text like 'bear left', 'sharp right', or even 'double back' with the segments as it did quite a while ago. (This is the cs:turn data in the API output.)

We welcome your feedback, especially to report bugs or give us route feedback.

My comments relate to: *

Your comments: *
URL of page: *
How did you find out about CycleStreets?:
Your name:
Our ref: Please leave blank - anti-spam measure

* Items marked with an asterisk [*] are required fields and must be fully completed.