Donate using PayPal

Please use the comment form under each blog entry to give us feedback items on blog items.

If your comment is not related to a blog entry, please use the general feedback form.

CycleStreets blog

News from CycleStreets

Archive for March, 2010

Award of grant from the Co-op Community Fund

Thursday, March 25th, 2010

We’re pleased to announce that CycleStreets has just been awarded a grant of £1,000 from the Co-operative Community Fund, towards boosting our server capacity, which is great news.

We’d like to the thank the Co-op for their support.

This is a significant boost in our forthcoming efforts to scale up the system towards a full release, beyond the current beta, so that very many more new and existing cyclists can have the knowledge of good cycle routes. At present we have a single server which limits resilience.

At present, our main area of work is speeding up considerably the performance of the journey planner and reducing the wigglyness of some routes. Our recent Developer Day identified the way forward on this. Together with increased hosting resilience, we will be in a much stronger position to promote the system and to provide a more solid interface (our API) for forthcoming mobile apps.

Cambridge Sustainable City have also assisted towards our server fund in the grant they awarded us, which is also enormously helpful. Additionally our hosts, Mythic Beasts, kindly provide a mac mini server to host our code/project resources.

Once again, thanks Co-op members!

Designer needed!

Wednesday, March 24th, 2010

When launching our site as a beta a year ago, we adapted an off-the-shelf design for the design of the site. This has stayed in place ever since.

However, based on feedback over the last year, and our own experiences as both users of the site and as user interface designers, we know that it is not perfect.

As we move the site towards a full release, we are keen to have some redesign work done.

However, we only have a small grant of £600 (tiny, we know!) available for this, so if you are a designer who is keen to support what is a not-for-profit project by helping us out at this very low rate, please do get in touch.

Areas which we particularly want to address are:

  • Making the map panel [e.g. see front page] bigger (possibly considerably so) – we have had a lot of feedback and it’s an increasing expectation.
  • De-cluttering the Photomap screens [example], which have never had much design attention on them, and making the workflow here much more intuitive.
  • Working out what to do about the itinerary page [example] which needs to be bigger and have less detail.
  • Dealing better with the issue of how to represent three journey choices on the itinerary page.
  • Potentially moving to a fluid width so that people who have a larger screen can use the space better.
  • Reducing the boxyness of the design generally.
  • Dealing with the question of whether it’s best to have a single big map and dispense with maplets altogether, or whether the maplets actually are useful to people [example - scroll down].
  • Having things like nicely shaded buttons and UI widgets rather than the flat stuff we have everywhere.

Over the last month we’ve done a lot of work to reorganise the code to enable design improvements to be implemented more easily (particularly with the Photomap pages), so that page elements can be moved around more easily.

If you can help, please do let us know, including some links to your previous work – we’d love to hear from you.

Happy 1st birthday to us!

Saturday, March 20th, 2010

Today (20th March 2010) is our first anniversary – we launched a year ago today, 20th March 2009.

We’d like to thank everyone that has been involved in any way in CycleStreets, and we look forward to a real take-off of the project as it matures in coming months and years.

Ordnance Survey data consultation – our response

Wednesday, March 17th, 2010

The government has been running a consultation on proposals to open up Ordnance Survey’s data relating to electoral and local authority boundaries, postcode areas and mid scale mapping information.

We have responded: Our response to the Ordnance Survey data consultation

The key points we make are that:

  • Without change, Ordnance Survey is at considerable risk of being left behind for many applications that do not require extreme accuracy, particularly as satellite imagery, crowd-sourcing, and other means of obtaining data become cheaper and more widespread.
  • Public-facing government-based IT solutions in areas like our own will increasingly simply not be able to keep up with the pace of private and community sector initiatives.
  • We are particularly strongly in favour of the release of postcode data. Much of this we believe has been created ultimately at taxpayer cost.
  • We believe it is unacceptable for boundary data to be held under any non-open data license. By definition, boundary data is self-referencing: it cannot be re-surveyed.
  • Any release of OS data under less restrictive licenses should be for raw data rather than raster imagery. There is no need to release cartographic imagery and that should remain an income source for the OS.
  • We feel that the OS’ claims of derivative rights over point-source data (as distinct from line-based data) are increasingly untenable and must be scrapped. No serious developer would willingly choose to give away their IP in such an unacceptable manner.
  • Data collected at taxpayer expense should be freely available.

The concluding summary of our response reads:

“Community-based initiatives have the potential to provide web-based facilities of interest to citizens at much lower cost and more innovatively than the government itself can achieve. These organisations and others should have access to data under an Open license. Wide access to such data has the potential to create an explosion of uses, as well as lower costs to the taxpayer overall. Thus we favour options 2 [full opening up of the data] or 3 [partial/staged opening up]. Furthermore, the role of government in geographical data should therefore be to facilitate release of data to enable these initiatives to flourish (with the tax revenues this will create), rather than to attempt to withhold the data and create monopoly services. Furthermore, data collected at public expense should always be made freely available.”

Finding streets and places now quicker

Sunday, March 14th, 2010

As well as adding postcode search support recently, we’ve this week we’ve upgraded the general street/place finding system (called a ‘geocoder’).

We’ve moved to using the new Nominatim system (technical details for those interested) by Brian Quinion – thanks to him and anyone else involved in that project.

Some tips for quicker searches are:

  • If the map is zoomed out, ideally include a city name as well, so that it knows what area to look in
  • If you do specify a city, add a comma before it, e.g. “York Street, Cambridge” rather than just “York Street Cambridge”
  • Note that lower-case text works the same as Capitalised Text.

In terms of our own implementation, the area you are in is automatically added when the map shown covers less than 18km across (zoom level 12 or higher).

The search-as-you-type implementation is one we heavily adapted from Search-as-you-Type on Google Code. We will release the front-end and back-end code in due course once we’ve done some more tidying to separate it out from other parts of CycleStreets, as we move towards open-sourcing the code.

Postcode searching added

Sunday, March 14th, 2010

Cambridge Sustainable City

Thanks to a grant (see our earlier blog posting) from Cambridge Sustainable City (part of Cambridge City Council), we have now added full postcode searching.

Postcode searching is clearly an expectation of users, and our data suggests some users have given up doing journey plans at all – on the assumption that the lack of recognition of their postcode meant their location wasn’t there at all.

However, we still have a full streetfinder and placefinder as well – so feel free to search using normal text like “Market Place”!

This has been the most requested feature since we went live and we have received very many bug reports saying that “my postcode isn’t recognised”. We’ve mentioned this problem before, and we’re pleased to have the money to add postcode searching for a year at least.

Sadly, PostZon is not a free data source (unlike OpenStreetMap, whose excellent map data we use for routing and the map pictures). There is a campaign to Free The Postcode which we strongly support, and there are rumours this might happen in the coming year, but there are absolutely no guarantees on this sadly.

Cambridge Sustainable City is an environmental initiative of Cambridge City Council, who are one of a number of Local Authorities who are now linking to CycleStreets from their website transport pages. (We hope that many more will follow!)

Cambridge Sustainable City aims to involve and support the local community in Cambridge’s efforts to address environmental issues. As a means to achieving this, they offer grant funding to local groups and organisations like CycleStreets whose work brings environmental and community benefits.

We are extremely grateful to them for their support.

We will report on the other projects that their funding is helping to support in the coming months.

Google bike there

Thursday, March 11th, 2010

Google have just launched bike routing in various cities around the United States, after involving the cycling community there. A few people have asked us (and our US-based equivalent, Ride The City) what we think.

The simple answer is that more availability of bike routing is a good thing.

There is undoubtedly space for CycleStreets, Google and others to co-exist: each will have their own benefits and niches, and competition means that everyone ends up with a better product.

Google’s US bike routing sends an even clearer message that cycling is a normal and realistic form of transport, something which we as campaigners have argued for years. It also increases the use-case justification for online cycle routing being part of the interventions needed to help get more people cycling as part of their everyday lives.

In other words: systems like CycleStreets and whatever Google may come up with in the future in the UK are good for promoting cycling, which is the only reason CycleStreets exists.

If anything, when Google decide to do UK cycle routing, we think it will be more of a threat to the Government’s Transport Direct portal, who will find it increasing difficult to justify spending public money on a closed system while commercial and community enterprises are doing the same thing better and more innovatively – in ways which involve the cycling community.

Lessons from the Developer Day

Wednesday, March 10th, 2010

On Saturday we held an intensive but successful event that focussed on the development challenges for CycleStreets. It was attended by computer scientists, software engineering professionals and webmasters who have experience in managing online services. This summarises some of the stuff we talked about.

Processing OpenStreetMap (OSM) data to build a Graph for Cycle Routing

We began by focussing on how CycleStreets processes OSM data into edges and vertices that can be used by a routing engine.

CycleStreets can find the shortest, fastest or quietest routes according to weightings attached to each edge. For the shortest routes the weight of edges is simply the length of the street between the two vertices. The weighting of the edges for the fastest and quietest routes is more interesting and depends on the type of street, the riding surface, whether there are any traffic lights and if it is going up or down a hill, and other factors going forward.

CycleStreets has its own routing engine that is written in Object-Orientated PHP, this language being a choice purely for legacy/historical reasons that has never been formally reviewed. The space inefficiency of PHP arrays has meant that the CycleStreets routing engine has evolved a suite of techniques for reducing the amount of data exchanged between it and the database.

To make the processing as fast as possible for serving routes live, all the weightings of edges are pre-computed. A data compression technique known as ‘Cello’ (Cellular Optimisation) reduces the number of vertices by a factor of five and the edges by two.

When planning a journey CycleStreets assumes that it needs to look at an area of the map that is up to twice as long as the straight-line distance between the start and finish points. The edges and vertices for that area are extracted from the database and either a Djikstra type or A-star algorithm is applied to produce a route.

Route planning in Central London is the most demanding for the system. An 8-mile journey extacts 22000 vertices and takes 22 seconds to create as a cache in PHP, consuming an unacceptably large amount of memory. It takes a further 2 seconds to search that cache to find a route.

The feedback from this section of the day was that:

  1. There are off-the-shelf programs that can process the whole of the UK in less than a second. Using one of these implementations, we could replace the small section of code that does the actual graph traversal, to get a quick speedup.
  2. The physics of the routing is of the greatest interest to cyclists. That is what makes CycleStreets special, and is where we add value; it is the focus of our innovation.
  3. We need to make it clearer how CycleStreets processes the OpenStreetMap data for cycle routing.

Since the dev day I have had a further look at options for the routing.

Our existing routing engine

At the moment our routing engine exists in two forms: stable and testing.

In the stable form the graph is simply edges and vertices. It has been reliably producing routes for several months, and although in most areas of Britain it can generate routes quickly, it does not work quickly enough in London.

The testing form takes into account the cost of turning at a node and is designed to reduce wiggliness. This has been in development for the last 3 months and has proved complex and difficult to optimise. It has reached the stage where it is nearly ready for release, but has the problem that routing with it takes too much memory.

The key to reducing the amount of data is in reducing the number of edges. In Cello it is quite easy to reduce the number of vertices, but much harder to do that for the edges.

On the developer day we discussed instead how reducing wiggliness could be taken into account by modifying the vertices on the graph, as shown below.

Incorporating the cost of turning into the graph

The cost of making the turn from S towards W,N or E shown on the left. can be incorporated into the graph as shown on the right.

The left of the image shows weighted edges joined in the middle with the costs of making the turn from node S towards each of the nodes W,N and E.The total costs of the routes are: SW = 5 + 4 + 4, SN = 5 + 1 + 5 and SE = 5 + 8 + 7.

That can be transformed into the image on the right, in which the turn costs have become extra edges between the vertices. The extra edges are one-way.

Making such a change to our graphs seems relatively simple compared to debugging Cello. At a vertex an extra edge will need to be added for each of the other edges. In this case 4 x 3 = 12 extra vertices and edges.

This modified graph could then be handled by the off-the-shelf routing engine.

Import of Data from OSM

On the developer day we did discuss how CycleStreets imports data. This summary attempts to clarify the process.

The table of how OSM tags are used by CycleStreets is shown at:

http://www.cyclestreets.net/views/howTagsUsed/

This table is orderd by the numbers of matching ways. The first row shows the most common tag pattern that matches half-a-million ways. The table has a long tail.

The table is built by a procedure involving the following steps:

1. Import

The OSM way tags that are relevant to cycling are identified as the following:

  • highway
  • cycleway
  • access
  • foot
  • bicycle
  • oneway
  • relation

For each OSM ‘way’ a row is created in the map_way_tags database table.
The above tag names are used for the field names in the rows, but suffixed with ‘Tag’, e.g. onewayTag.

The relation tag is only used where the relation it refers to is something to do with cycling.
Usually these are signed cycle routes with tags such as type=”route”, route=”bicycle” and network=”NCN”, ref=”11″.
Some ways are tagged directly with e.g. ncn_ref=”11″

Node tags such as crossings and traffic_signals are handled by later processing.

2. Canonization

The values of the tags are checked against an allowed set of values that CycleStreets recognizes.

Where there are invalid values some repairs can be made. For instance from oneway=true to oneway=yes.

The full set of repairs is listed at:
http://www.cyclestreets.net/views/repair/

The canonized values are copied into the similarly named fields in the map_way_tags table.
The canonized field names do not include the ‘Tag’ suffix.

So for instance a way with the following tags:

highway = “residential”
oneway = “true”

becomes a row in the map_way_tags table with these original and canonized fields:

highwayTag = residential
onewayTag = true

highway = residential
oneway = yes

The howTagsUsed table lists the contents of the map_way_tags table, grouped by the canonized fields.
Hence there is one row in that table for each unique combination of the canonized values.

3. Initial impressions

The simple mapping between the OSM highway tag and CycleStreets’ provision types defined at:

http://www.cyclestreets.net/views/osmHighwayTranslation/

is used to create an initial suggestion for each row in the howTagsUsed table.

4. Rules

A set of rules is then applied to the permutations to improve the quality of the translations.

For instance a highway=”service” is intially interpreted as provision type “Service Road”.

A rule that matches bicycle=”no” converts this to “Prohibited”.

Inspecting the Translation

The /howTagsUsed/ view provides a detailed and rather complex listing of the translation.

It is possible to click on the items in that table to view the data in filtered ways.

Another way to view the information is via the map on the Journey Planner page.
When signed in with your developer account (which should also include the ‘osm’ privilege) you will have access to the ‘Ways for OSM users’ overlay. This is available via the blue plus symbol at the top right of the map.

I think we could do more to make it a lot clearer how the information has been imported and processed.

One major problem is that the import data cycle has stalled recently due to the on-going work on reducing wiggliness.
This is bad because it means the CycleStreets system is out of date, and any recently added photos will not be included on routes.

Open-sourcing

There was a long discussion about this, with a range of different views. As ‘open source people’ we are keen to Open Source the codebase, but there are a few factors which govern the timing of this, and some work to be done first.

It won’t bring in new developers straight away, but it will be an on-ramp for getting people more involved in the project.

It will improve the quality of the download and installation.

There is a danger that it could be always something we hope to do, but never actually get round to. We definitely want to avoid that.

OpenCycleMap

We were given a useful overview of how the OpenCycleMap.org project works by its creator, Andy.

This site renders cycle-specific OSM data for the whole globe. It has very beautiful rendering of hills, and in fact that accounts for 96% of the data, the remaining 4% is all the other streets, including the tiny amount that have cycling data. That rendering includes all the known, verifiable cycling routes.

The success of this project keeps their servers extremely busy, but they are soon going to expand to bigger and more powerful systems.

What we did not discuss on the developer day

Time did not permit us to get onto a whole series of other things. Here’s a short list:

  • Drag drop routing – this is a feature which already exists in Developer Mode but has not been exposed to users generally; it would need some discussion about permalink URLs.
  • The layout of the CycleStreets pages – e.g. increasing the size of the map - which is something we’ve always had a lot of feedback about. We now have a grant which could include some design work commissioning.
  • Output from CycleStreets – either as printed output or via export to GPX / KML.

Perhaps when some of the issues we’ve learned from this developer day have settled down we shall think about organising a users day to focus on some of these important, but hopefully simpler issues.

Developer Day (6th March): final details

Wednesday, March 3rd, 2010

Here are final details for our Developer Day this Saturday. We’ve had to amend the location as there are more people than we expected – which is great news!

If you plan to come, please let us know if you’ve not already been in touch.

Time and place

Date: Saturday 6th March 2010
Time: 10.30am for 11am start, till about 6pm.
Venue: BeginSpace (see picture)
Address: Burleigh Street, Cambridge

Here’s a journey plan from the Railway Station to BeginSpace.

BeginSpace is just outside the entrance to the Grafton Centre, one of the two main shopping centres in Cambridge. If you get lost, ask someone for directions to the Grafton Centre, as it’s quite well-known.

Plentiful and secure cycle parking is available just outside the entrance.

We’d like to thank our friends at BeginSpace, who have provided use of their incubator space for the day. Wi-fi access will be available, so please bring your laptop if you have one. We’ll be in the main upstairs room, but there sofas in the breakout area downstairs.

What we’ll cover

Things that are likely to be included, subject to time, are:

  • Overview of the CycleStreets architecture – explaining the way the system works, and what its limitations and strengths are, as well as how the code is structured. (We’ll expand heavily on the stuff that’s in this introductory FAQ.)
  • Improving the translation from OSM data to CycleStreets (and how it works).
  • The feedback system and its feeding-through to OpenStreetMap data, and people’s ideas for a better backend for this.
  • How other webapps can be built around CycleStreets (or integrated as layers and new sections in some way)
  • Plans for open-sourcing.
  • How to get other people helping with coding of the most complex part of the system: the routing engine
  • Longer-term direction of CycleStreets.
  • Mobile phone interfaces.
  • Funding!
  • Installing the code (see below) – and getting people going on a few little improvements!
  • But mostly … Whatever people want to talk about.

Getting stuck into the code: mini-projects

Depending on what people want to do, we hope towards the end of the day to get people going on some small, mini improvements to CycleStreets by way of introductory projects.

We’re working as fast as possible this week to get a small download (probably 200MB or so) available to enable a CycleStreets installation to be reproduced, which we’ll be making available and will go through on the day. (We will require people to sign a simple contributor agreement if so, as we’re not Open Source yet, but we think people will find it reasonable; we’ll try to get that online in advance.) The installation will thankfully cover only a relatively small area or two – the full dataset is huge!

We’ll get onto hacking the code later in the afternoon, so people who don’t want to do any actual coding need not worry or feel obligated to take part.

In terms of software required:

Software to set up on your laptop

Software required (please set up in advance if possible) is as follows.

The code is written in object-orientated PHP, but we hope that people familiar with other languages shouldn’t find it too difficult to understand what’s there.

  • Any OS (Windows/Mac/Linux are all known to work fine – we like to code cross-platform!)
  • Apache webserver (we use Apache 2.2.x; older should probably work)
  • PHP 5.3.x as mod_php in Apache (i.e. not a CGI installation). 5.1 and above should work, except for the journey planner engine itself, which requires 5.3-specific features.
  • PDO-MySQL (part of PHP5.3, but sometimes bundled as a separate package)
  • MySQL 5.x (older almost certainly will not work)
  • Ideally an SVN client, but we should have a zip file snapshot available too

Sustinence

We’ll pay for lunch (still to be sorted out), and we hope to offer home-cooked pizza afterwards back at Simon’s house.

We will hopefully then go to a pub if people are around for the early evening!