CycleStreets blog

As previously announced, we are working with the UK’s Department for Transport to make advanced cycling data attributes available for incorporation into OpenStreetMap.

Rather than organising this along the lines of a bulk import, we are taking advantage of new technologies in Potlatch 2 and have commissioned Andy Allan, creator of OpenCycleMap, to develop new features to allow volunteers to collaborate on inspecting and merging the information into OSM.

This merging tool will also be of use for other external data that could be manually inspected and merged into OpenStreetMap.

Background

The DfT commissioned survey work in various cities around the UK for their Transport Direct product. In 2011 they released the results of the surveys as Open Data, in a complex GML format based on Ordnance Survey ITN data – unsuitable for use with OSM. However, in addition, they have funded work to convert the survey data to be based on OSM geometries suitable for incorporation. This has been done through brilliant work by Ralph of CCG.

The kinds of things surveyed include cycle routes, cycle parking, cycle lanes and their widths, surfaces widths and lighting conditions of cycleable paths, and so on. We are working to add support for these attributes into CycleStreets, so that routes are further improved.

In the UK wide areas of the cycling infrastructure have been mapped in OpenStreetMap, often more recently than the data from the DfT. Also, with the development of Vector Background layers in Potlatch 2, there was an opportunity to create an improved process for dealing with external datasets.

Further background information is available in blogs and on the mailing lists.

• Initial announcement of the project
• Update on the progress of the tool
• Talk-gb thread on the project

The demo

We’re pleased to announce that a demo is now available, and we’d like people to test it.

A demo is now available. It contains sample data for Nottingham and Cambridge, but it’s deliberately unable to save the data back to the main OSM server. When the final version of the data conversion is complete and available, this will be updated and fully able to work.

Two test areas are currently loaded:

• Cambridge
• Nottingham

How to use the merging tool

The merging process works as follows:

1. Click ‘Map style’ > ‘Wireframe’ to make things much easier to work with.
2. The background data is highlighted either orange (needs attention) or blue (already processed).
3. Click the background features to select them.
4. Ctrl+click (or cmd+click on a Mac keyboard) the relevant OSM feature (line) to see a side-by-side comparison.
5. Click on ‘Advanced’ in the left panel to see the merging controls.

Feel free to play around with this – the snapshot data is being reloaded from time to time as we get better imports, although we think we’re almost at a stage where the data conversion is fairly bug-free.

The merging tool is currently a beta and further improvements are planned. See the main Merging tool page on the OSM wiki.

The first screenshot shows the thick gray line (DfT data, as a background layer) highlighted. It shows the attributes it has:

The second screenshot shows what happens if we now control-click (or cmd+click on a Mac) on the OSM line – we now get a merging interface where we can accept/reject each attribute, and click the button at the end to accept all the changes:

Feedback on the data

We would really welcome feedback as to any errors you spot in the data conversion. The aim is that the data is pre-processed and snapped to the OSM geometry as effectively as possible, so that merging is merely a case of manual confirmation of each attribute according to your local knowledge.

Issues we have fed back so far on are:

• Alignment. The data was originally snapped to an OS Open Data, and has been geographically aligned via advanced GIS techniques to OSM. It’s already well over 90% matching and further improvements are being made.
• The issue of streets being broken up but having the same data. Our GIS contact plans to merge when the street name and data matches.

The software

A number of software components are used to make all this work

• Potlatch 2 is used as the editor, and can load data from both OSM and the DfT data. The splash pages and other resources are available on github
• Snapshot Server is used to serve the DfT data for each user, saving them from having to load the whole country at a time
• Some scripts are used for loading data in and out of the server. These use Osmosis to read/write between XML and Postgres.

License

The data is expected to be released under the Open Government License. We have been seeking an early letter of confirmation from the DfT on this and will update this page and the OSM Wiki accordingly. (The ITN-referenced dataset is released under the OGL already.)

Feedback

We’d really appreciate it if you could try out the beta and add comments below, or contact Andy Allan with any feedback you have. Did you figure out how to use the tool? Did you manage to merge some data? What doesn’t work? How could the tool be improved?

If you have local knowledge of the areas in question, it would be great to hear back from you on the datasets themselves – do they match reality? Are the tags appropriate?

We're pleased to announce that we are working with Cycling Scotland to enhance cycle journey planning in Scotland!

Cycling Scotland, the organisation charged with getting more Scots on their bikes, runs a range of initiatives such as Bikeability Scotland, the freshnlo Pedal for Scotland bike ride, cycle instructor training and more. They are keen to provide cycle journey planning – to help remove a key barrier that people face when starting cycling or when they move into a new area.

As part of their journey planning activity, Cycling Scotland are extremely keen to motivate local community groups to map their area into OpenStreetMap, which forms the heart of CycleStreets' journey planner. Although there are areas like Edinburgh which have very high-quality mapping, thanks to the great work of OpenStreetMap volunteers there, other areas of the country are not so well-covered.

To help, we will be creating resources to help local communities with this mapping activity. Principally, this will involve creation of a user-friendly guide which introduces OpenStreetMap, explains how we use it, how people can collect data, and importantly outline the key things that improve the quality of cycle routing. (We hope this guide will also be of wider use to the OpenStreetMap community elsewhere, too, even though it will of course be tailored for Scotland.)

Alongside this work, we'll be creating a customised journey planner for Cycling Scotland, to be hosted on their website. This will benefit, thanks to a grant from them, from the introduction of more advanced routing attributes in our journey planner engine. By encouraging people to collect more detail about the cycling environment in their area, this will improve further the quality of our routing. Naturally, this will all be explained in the user-friendly guide for collectors.

Cycling Scotland are also supporting us to make our routing available more widely on different types of mobile phones, so that it is as accessible as possible.

We think this model of helping get more people cycle by engaging local communities and building on existing work is a brilliant model.

We are looking forward to undertaking these activities in partnership with Cycling Scotland, and will report in coming months as each part is completed and made available.

OpenCycleMap in Scotland - cc-by-sa OpenStreetMap contributors

Over the last year and half, the DfT has been collecting cycling data for use in its TransportDirect portal.

We received today a letter from Norman Baker MP confirming that it is the DfT's intention to open this dataset, in line with the new government's principles on transparency. We think this is an excellent development. It confirms earlier discussions we held with members of the former Cycling England.

It is our understanding that the data is currently referenced from the Ordnance Survey's ITN dataset. This means it would, if kept in raw format, be useless to anyone without access to an expensive ITN license, and even then could be encumbered. However, Norman Baker sensibly writes:

"We recognise that the data in its current form is subject to licenses and so are working towards translating the data into a format that is more easily used by the developer and OpenStreetMap community."

This is good as it recognises that releasing a dataset without it being referenced against open (lat/lon) attributes would be open data only in name rather than in spirit. Using the Open Government License should ensure that it can be merged into OpenStreetMap, whose dataset has proven its worth for UK-wide cycle journey planning purposes.

The OpenStreetMap community is rightly wary of mass imports of data. In order to make use of this data, we have stressed that some tool development would be needed to enable it to be manually 'pulled across' via visual inspection if it is to be used, which we've been talking to Andy about.

We are extremely keen to help the DfT with this whole process, and discussions have been positive. We hope to report further on this soon. It would ensure the government can achieve greater taxpayer value for the data, and thereby enable us and others to give additional confidence to people considering starting cycling or wanting safer routes. Getting the data into OSM will also mean that the data is a living dataset that is kept updated.

As a sidenote, Norman Baker seems to be on a roll with sensible thinking. It is recently reported that he has refused to tow to the unquestioning view of helmet-wearing, but instead has weighed up the evidence himself and decided not to on a personal basis. This reflects best-practice cycle training: that risks should be assessed according to circumstances, and therefore the choice of whether to use a helmet is an individual one. He's also requested people to come forward with examples of unnecessary and annoying announcements on public transport – truly fresh thinking!

Organisations can already create a simple 'cycle to us' link.

However, some larger organisations may like to have a planner embedded in their site directly. So we've published a new page detailing how Local Authorities and cycling promotion bodies can add a CycleStreets route planner to their site.

CycleStreets can be added:

1. As a subdomain (e.g. cyclemap.placeford.org); or
2. As a subdirectory (e.g. www.placeford.org/cycling/journeyplanner/); or
3. Via an iframe (though this is not ideal); or
4. Using our API (see documentation)

Read full details of the embedding methods, and the steps required, at http://www.cyclestreets.net/help/embedding/

Today is our second birthday – CycleStreets was launched on 20th March 2009.

The last year has seen a huge amount of development work, leading to new features, speed improvements, and more. However, the next six months will be even busier as the project really ramps up!

In the first year, CycleStreets planned 67,000 routes. In our second year, around 437,000 routes have been planned, and the rate of increase continues to climb. By November we had planned enough routes to cycle to the moon ten times, and in February, we reached the milestone of half a million journeys planned.

A major challenge we faced a year ago was the technical challenge of generating the routes fast enough.

A year ago, CycleStreets used a routing engine written in PHP (!) that we created for the Cambridge-only predecessor of CycleStreets – the Cambridge Cycling Campaign journey planner. It was slow, taking half a minute to plan a route across London, and taking up most of the system resources. Effectively, it was the wrong technology and didn't scale to UK-wide routing.

We held our first Developer Day, which lead to very productive discussions about the routing engine and how we could provide routes to users of the site faster. A friend of the project, George, wrote us a new engine (using Python) which lead to a massive speed-up. Then Robin, another volunteer, took the Python engine and created an even faster version in C++. This has been in place for most of the year and has quietly sat at the heart of the system, planning routes in a few GB of RAM while barely challenging the processor.

The work on the routing engine meant that we have been able continually to increase the maximum planning distance, which is now 200 miles (320km), which is well above a day's cycling! The development version of the system can even now do Dover to Cape Wrath!

Improving the routing speed was a key requirement for mobile apps, several of which signed up to use our routing through the year. These include the leading app for the London cycle hire scheme – London Cycle: Maps & Routes, plus two other excellent 'boris-bike' apps, the briliant and world-first 3D bike satnav app, Bike Hub, BikeRoute for Android and, of course, our own CycleStreets for iPhone app.

Our own iPhone app was made possible thanks to two grants we successfully applied for.

Our Android app is nearing completion, and like the iPhone app is being developed as an open source project. Thanks to our mobile developers for their brilliant work on these.

Through the year we have given various presentations and got involved with various social enterprise -related activities., such as WhereCamp EU, CamTechNet, Cambridge Geek Night and Net2Camb amongst others. These events lead to interesting discussions and also resulted in useful new contacts, such as people helping out with our mobile apps.

It was a particular plesure to give a presentation to Net2Camb as it gave us the opportunity to speak about the challenges faced by us as a not-for-profit social enterprise, rather than purely talking about technical challenges.

We have launched a funding drive for £130k to raise funds for two full-time developers. Such funds would enable the project to move forward much more quickly.

The DfT has this year been collecting cycling data which we are keen to see added to OpenStreetMap. We have since had informal discussions with Cycling England about use of the data, and how conversion of the data might be undertaken and at what cost. Discussions have been positive, and we feel this data would improve the quality of routes that we can deliver to users.

Over the year, more and more governmental bodies have been linking to us. For instance, in April, Cycling Scotland linked to us, and we are keen to work with them to help motivate people to improve OpenStreetMap data in Scotland. Others, including some of the Cycling Demonstration Towns like Chester and Lancaster now link to CycleStreets, and we have just sent a new brochure to councils around England.

Increasing the flexibility of the CycleStreets platform has been an ongoing priority.

In February we created a customised cycle journey planner for West Sussex County Council, building on work we have done to make it easier for organisations to have a journey planner within their website. Another has been created for the Bike Hub website, and a demo Local Authorities site is available.

The year has also seen a few developments on the Photomap. This is an area we would like to do much more on, as explained in our GeoVation bid for which we have now been shortlisted.

We created, under contract for Cambridgeshire County Council, a site called 'Cycling Sorted' to help manage the shortage of cycle parking in that area. We are keen to create similar sites for other Local Authorities. We have also created a similar system to support the great work of London Cycling Campaign.

OpenStreetMap is the backbone of our project, and we have been pleased to promote OSM and encourage more mapping for it. Over the summer we helped obtain a database of all the bike shops in the UK, for use in OSM, from the Association of Cycle Traders. Much of this has been merged into OSM, but more needs to be done to complete this crowd-sourcing exercise.

CycleStreets' use of open data saw it being featured on the front page of the government's new data website – data.gov.uk.

Throughout the year, we implemented many smaller improvements and innovative new ideas, such as the new cycle.st shortlink domain, our new Photo of the Day on Twitter (featuring the best of the 25,000+ pictures in the Photomap), a new gallery viewer, better facilities to link to the journey planner, adding an integrated editor (Potlatch 2) as well as various ongoing design/usability improvements (though there is much more to be done, time/funding permitting).

Routing quality work, however, remains our highest priority. Our aim is to provide the highest quality routing possible for cycling, using our knowledge as cyclists. Various improvements have been made recently, and we are currently working on new routing attributes and reducing the wigglyness of some routes, which is proving a difficult problem to solve with limited hardware resources.

Simon and Martin, lead developers, would like to thank a range of people who have helped out in various ways, such as Andy, Shaun and David from OpenStreetMap, George and Robin for work on the routing engine, huge support from Chris in Edinburgh, George from Camden, our mobile developers – Alan, Neil, Jez, Theodore, Christopher and Jonathan, advice and a free dev server from our brilliant web hosts Mythic Beasts, our designer Ayesha, Jeremy for occasional advice on business matters, support from key individuals at the CTC, LCC and Cycle Nation plus others in our stakeholder group, Carlton and Bike Hub, helpful ideas and data from cycle campaign groups around the UK, and of course the amazing community of OpenStreetMap contributors whose mapping makes everything possible.

Lastly, we would like to thank our users, whose cycling needs provide us with the inspiration to keep going, and who provide us with much feedback and many great ideas.

Our new brochure promoting CycleStreets for Local councils around the UK is now available, and hundreds of copies will be posted out this weekend!

Naturally, we're keen to see OSM-based CycleStreets used as widely as possible, so that as many people as possible can be helped to start cycling or find better routes. So we've printed 1,500 copies for distribution to Councils and related organisations.

We're encouraging them to commission customised cycle journey planners, like West Sussex County Council have, or simply to link to us, like Surrey Heath, Devon, Chester, Lancaster, and more are doing. Customised planners provide the best possible visitor experience and raise funds for the project.

Download a copy of the brochure (PDF) from the CycleStreets website, or browse it on Issuu:

We'd particularly like to thank Ayesha Garrett of LondonLime who did the design work for us at a next-to-nothing price, by way of support for the CycleStreets project. If you ever need a designer, we can strongly recommend her as someone who will produce great results, quickly and remain always unfailingly polite! Thanks, Ayesha!

We're pleased to announce the launch a customised cycle journey planner for West Sussex County Council!

http://cyclejourneyplanner.westsussex.gov.uk/

We hope that this will be the first of many Local Authority sites based on CycleStreets (and thus OpenStreetMap), although quite a number already have links to our main site.

Read more about the services we can offer Local Authorities in our new brochure.

The main page, in the West Sussex style, with a customised layout, logo and 'quick zoom' to specific local towns:

The itinerary page, including a CO2 saving calculation (compared to a car) and new calorie counter:

Routes in West Sussex (and beyond) can also be planned via the CycleStreets mobile app:

We're about to publish a new glossy brochure to be sent to every local council in England (and soon Wales, Scotland, NI) to encourage them to link to CycleStreets or to create customised versions of the site.

For this, we need your help!

We've set up a webpage where the cycling/transport department's contact details for each local Council in the UK can be entered.

Simply go to the page, and choose a local Council. On the page you'll find a Google search link which will help you find the details more easily. Find the address and other contacts, and enter them into our form. Each Council should only take a few minutes to do, so with this 'crowdsourcing' effort, we can get all the data easily.

Please help us now by going to www.cyclestreets.net/localauthorities/contacts/ and getting started! Thanks for your help.

Our brochure will be online soon, but here's a taster of the cover…

Thanks to the brilliant work of Andy Allan, Richard Fairhurst, Tom Hughes, Dave Stubbs, Matt Amos and others, who have worked on OpenStreetMap's new 'Potlatch 2' editor, we've now been able to add an integrated data editor to the CycleStreets site.

Just go to www.cyclestreets.net/edit and you can fix up and add map data directly!

We'll be customising the installation and integrating it more with existing pages and tools within the CycleStreets website in coming weeks.

Changes you make to the data go into the OpenStreetMap database directly. We then import that data every week or so.

(In other words, changes you make via our editing page don't reflect the routing immediately, but take up to a week. We're working to reduce that down, ideally towards a daily import now that we have more powerful server hardware thanks to your donations and some grants!)

The work to integrate Potlatch 2 into the site was extremely easy – it's clear that Andy et al have worked very hard to make that as simple as possible. (In fact, it was refreshing to find that integrating an external piece of software was straightforward rather than the usual problematic task it can often be!)

We'd also like to thank Tom Chance, whose excellent introductory guides we adapted, with his kind permission.

For The Win!
#p2ftw

We really enjoyed the January Net2Camb Meetup event, where one of our lead developers, Martin, gave a talk 'Our Story'. Thanks to Claire for organising the event and everyone who came!

It was particularly enjoyable as it was a rare opportunity to talk about the business and competition aspects of CycleStreets, about the challenges we face, and the future opportunities for the project.

We were also pleased that a couple of people came forward as new volunteers!

Here is our presentation [link]: