Monday, 22 June 2015

Where is all London's new housing?

Some of my recent work on housing markets, mortgage lending and housing search has led me to consider the question of where, exactly, London's new housing is located. On a recent visit to King's Cross I was amazed by the sheer scale of development, particularly all the new flats. Because I've been working with the data for another project - and recently re-examined it for a project proposal which explicitly didn't focus on London - I thought it would be interesting to see whether my perception of the flats boom is based in reality. Of course it is! 

The maps below are based on all new build homes sold in London from 1 June 2010 to the end of April 2015 (the most recent data). During this time, according to HM Land Registry 'price paid' data, there were 42,938 transactions on 42,813 properties. This indicates that quite a few properties are not being picked up in this dataset - e.g. compare it to the completions data from the London Datastore. Nevertheless, the patterns and distribution of property types is revealing. 88.8% of transactions were for flats, 7.1% for terraced houses, 2.4% for semi-detached properties and 1.7% for detached houses.

All property types


Terraced houses

Semi-detached houses

Detached houses

Clearly, the mix of new housing - and its relative low volume - is something that many people have commented on before, but I've not seen many people look at the geographical distribution in this way. The important questions arising from these maps - as ever - is why are things the way they are? That's something the maps can't tell us but it does provide an interesting starting point for debate. The discrepancy between the Land Registry data and data on completions is also not surprising owing to the way new build housing is sometimes sold, but it would be interesting to explore this more in future. If you do happen to have a few million quid to spare, good luck finding a new detached London house to live in!

A note on the maps... I've geocoded the price paid data using Ordnance Survey's Code-Point Open dataset, which can match sales to postcode units, rather than street addresses. The transparent bubble map is of course far from perfect but I've used it here to convey the scale and location of new housing, rather than to offer a precise fix. So long as it gives the impression of there being a massive splodge of newbuild flats in Central London that'll do for now. I am aiming to highlight the general scale and geography of development as a fairly quick experiment to see what might be done with the data. No plans to make it interactive (update: the best laid plans of... see map point datadump below).

Update (1715, 22 June 2015): I fixed the glitches, which were caused by a rogue space here and there in the codepoint open file. Moral of the story? Build more houses (I think). Always a giveaway when there aren't many dots in Wandsworth. The numbers were correct all along though. Finally, I've added in some information from HM Land Registry on properties not included in the price paid data.

Data excluded from Price Paid dataset - link

Friday, 19 June 2015

Creating an English green belt atlas

UPDATE: I've fixed the glitches in version 1 and compiled a spreadsheet with the data. See new download at the bottom of the post.

I've blogged before about green belt, and also written about the underlying data in the press. Now that the data are open, I've finally got round to finishing a little project I meant to complete ages ago. I was prompted to do this during a recent visit to my department by Prof Bob Barr, a legend in the data and GIS worlds. Bob said it would be good to know what percentage of the land area in each local authority in England was covered by green belt. I agree, so here are the results of my analysis (using 2014 green belt data) from Version 1 of my English green belt 'atlas' (actually lots of individual images to keep the file size down). Here's a snapshot of one of the maps...

Green belt land in Cheshire East

And another, this time from Birmingham. You can see that I've dimmed the background so that you can get a sense of other green belt land in the areas I've mapped.

Birmingham green belt land

Finally, a few more from around the country...

There are some glitches in the data but my initial overview suggests the numbers are pretty accurate (see exceptions below). I hope that people might find these maps useful. If you want to use any of them, be my guest.

Download all the files here (154MB): Green Belt Atlas 2014 (version 2) (186 individual map files, plus spreadsheet)

Download just the spreadsheet: percent green belt figures for each of the 186 local authorities:

Contents of the spreadsheet (download above)

Warnings: A couple of issues with version 1... 1. The West Lancashire greenbelt area extends into the sea on the green belt shapefile available from DCLG, so the figures here are incorrect (working on a fix). 2. The figure for Ashfield is clearly wrong - not sure why, so I will fix that too. 3. Some areas have extremely low values and may not actually be in the green belt - it may instead be down to the accuracy of underlying data. 4. Mole Valley currently missing, am looking into why. UPDATE: I looked again at the original Green Belt shapefile from DCLG and found that Mole Valley had the same code as Ashfield, so I fixed that and there's now a map for Mole Valley. New Forest was also assigned two different codes, so I've fixed that too. Also, in the percent figure, I've exluded the part of the West Lancashire green belt that is not on land, so this gives an accurate figure now. You can see from the image below that part of the green belt goes into the Ribble Estuary.

Technical stuff: I did this in QGIS 2.8 (open source GIS software) using the Atlas tool and a very heavy laptop, plus a bit of trickery I picked up here and there. I blogged about this before, with a little tutorial. Perhaps I should actually be using the term 'green belts', as Richard Blyth pointed out, but forgive me for this.

Wednesday, 3 June 2015

The beating heart of the City of London

I've had a rush of blood to the head so here I am with a second blog in two days. I'm getting some slides ready for tomorrow's Modelling World 2015 talk in London, which is all about visualising mobility (see below) so I wanted to add in a couple of new visuals on commuting in and out of London. Visualisation can often be just a lot of fancy graphics. This can be useful in itself for a number of reasons (e.g. capturing attention on an important issue, drawing attention to unusual patterns in a dataset) but since I've been working with commuting data in England and Wales I wanted to focus on flows into and out of the City of London. 

This interests me for a number of reasons, including i) commuting can play a significant role in wealth creation and it also needs to be understood in relation to how we measure GVA; ii) commuting is often very stressful and damaging to the individual - particularly long commutes - so I'm interested in the kinds of distances involved and this can be seen easily on a map; iii) commuting can often be environmentally damaging - though this isn't what I'm mapping here; iv) commuting in and around London is often about green belt hopping so I was curious to see how much commuting comes from beyond the metropolitan green belt; and v) commuting is a two-way process and affects places at both ends and in between due to travel. 

So, here's what I did. I took the MSOA-level commuting data for England and Wales (table WUEW01 here), used a bit of QGIS, extracted frames from QGIS using the MMQGIS plugin, then patched it all together in GIMP to create an animated gif. One for inflows, one for outflows and one for in and outflows (thanks to Ebru Sener for the idea). It might run a little slowly in the blog post in a browser but see below for the images. Just to clarify, I've only shown flows of 25 or more into the City of London. Those not familiar with the data should be aware the the 'City of London' refers to the small area in the centre of London and not the entirity of Greater London! An obvious point but one worth repeating in case anyone is confused. A Greater London map would have many more data points, covering most of England.

Commuting flows (>=25) into the City of London

Same as above, but going back the way

The 'pulse' of the City of London

You should be able to get a better view of the images by clicking on them individually and if you want them to work more quickly try saving them to your own machine.

Tuesday, 2 June 2015

The Polycentric South East

Tweets yesterday from Michael Edwards and Joseph Kilroy reminded me of a map I produced last year in which I showed commuting patterns in South East England, minus London. I did this to get a sense of the polycentric nature of travel to work in the South East as this has been a topic of many previous studies - including the famous Hall and Pain book - but none (to my knowledge) using the 2011 Census data I mapped. The other reason for me blogging about this today is that I'm speaking on a similar topic at Modelling World 2015 this Thursday in London. Enough words, time for some maps, which I've refreshed for this week.

The first map below shows all commuting in the South East of England in 2011, without place names. As you can see, I've removed London from the equation, both in relation to travel to work flows and from the underlying map canvas. This gives a slightly different perspective than the one we're used to. The second map is the same as the first but I've added the names of local authorities in order to help identify places. Click any of the images to enlarge.

Commuting in South East England, 2011

Same as above, but with labels

Now, here's what it looks like when you add London back in... Kind of brings to mind astronomical metaphors, as hinted at in a previous study by the RTPI. I should add that the definition of a supernova is 'a star that suddenly increases greatly in brightness because of a catastrophic explosion that ejects most of its mass' so this might be stretching things slightly... Then again, if what people are saying about the displacement of the poor from London this might actually be spot on.

The 'London Supernova'

Finally, I've produced a zoomed-in version closer to London where you can see some of the flows which go through/over the capital. I don't fancy that commute!

Tuesday, 12 May 2015

The 2015 General Election: London Results

The perennially excellent London DataStore has published comprehensive, accessible and usable data on the 2015 General Election. So, naturally, I had to make some maps of it. There probably isn't anything about this election that hasn't been mapped but since my last blog was a General Election piece I thought I'd do a little follow-up, with not a hexagon in sight. There have been quite a few 'who came second' maps but not many which include third and fourth places. I'm particularly interested in London because it's something of an exception in the South East of England and, well, I just wanted to make some maps. Below you'll see maps for who came first, second, third and fourth. The last map has the constituency names. I resisted the temptation to do a 'who came eleventh' map, but, since you asked, there were only four constituencies where there were at least 11 candidates, and the parties included the Whig Party. They might have been quite prominent had these maps been made in 1830 (incidentally, there were 658 constituencies in the UK then, which only had 24 million people).

'That's Blockbusters' - for Labour

UKIP emerge and Tories dominate second place

UKIP by far the most in third place - Greens emerge

No place for three 'main' parties here

Just in case you don't know all constituencies off by heart

You should be able to see pretty big versions of the maps if you click on them and then open them in a new tab/window. I've dispensed with the usual boring map legend and instead turned it into a 'bargend' (a portmanteau I just invented). I hope you find these interesting. 

Final nugget: the Whig Party came 9th in Bethnal Green and Bow (their best result) with 203 votes for my namesake, Alasdair Henderson.

Thursday, 7 May 2015

Can Google search data predict an election victory?

Today seems like a good day to write a little blog post on what search data can and cannot tell us. Why? Because of the story below, which has been on the front page of the MailOnline for much of the day. This is just the way news works, but I thought it would be useful to give a bit more information here. The story behind the story goes something like this...

Simon Rogers, datajournalist and Data Editor at Google in San Francisco, got in touch some time in April to ask if I could help him map party leader search patterns by constituency. I'd been doing a lot of work with search data for housing markets anyway so this seemed like an interesting idea. We took search data for all points that were geocoded (there were about 5,000 total across the UK) and then produced a constituency version for all 650 seats. The final constituency results matched very closely the proportions for the individual places. The data are for the previous 12 months. 

MailOnline front page 07/05/15

The big question is what this all means. Do I think, as the MailOnline suggest, that 'Google Search tips Cameron to win election'? No. Do I think it disproves it? No. Do I think the large volume of search for Nigel Farage indicates his level of popularity across the country? Again, no. However, it could indicate that people are more likely to show interest in UKIP in an environment when nobody else is watching or listening. But we don't know. Does this prove that Miliband will come third? Definitely not. The map merely indicates who was the most searched for party leader in each constituency. The intent and sentiments of individual users are not known. In my own research in housing market analysis I've tackled this by doing interviews with website users but since this data is from Google they could of course add in other terms which people might combine with party leader names (some more favourable than others!). 

Kate Newton from Bing also got in touch to say they had worked on something similar (though much more sophisticated) in relation to the Scottish Independence referendum last year. More widely, there is a body of emerging research (including my own) which looks at search patterns and subsequent activity - mostly in the field of economics. The results suggest that search can be analysed meaningfully to predict future activity. But that's not what the party leader piece was about - not from my perspective. Thankfully, other media outlets were more measured in their analysis - such as BuzzFeed UK, The Scotsman,  and The Telegraph

MailOnline story 07/05/15

What's most interesting to me? Well, I'm most interested to see how the search patterns relate to outcomes in key marginals. I suspect there will not be much of a pattern but if there is it will be interesting to attempt to take this little piece of work further - perhaps for the US 2016 election. Other than that, this is an interesting stocking filler on a day when the papers and TV crews are forbidden from reporting anything really substantial until the polls close at 10am. In the meantime, my favourite snippets from the search map...

David Cameron is the most searched for leader in his own constituency, but he's surrounded by a sea of purple, plus a blob of red and orange.

Witney - David Cameron's constituency

Perhaps a little predictably, there is a lot of search for UKIP in Kent but - strangely - it appears that in the constituency where Nigel Farage is standing (Thanet South) the party leader most searched for is Ed Miliband.

UKIP - lots of search in Kent, but not to much in Thanet South

The search results produce some interesting results. The image below is a good example. Natalie Bennett (Green Party) is the most searched for leader in Durham North West and next door in Durham North the leader (by search over the past 12 months) is Leanne Wood (Plaid Cymru). I suspect this was down to a localised spike in interest after the leaders' debates.

Durham - Green and Welsh Nationalist stronghold?

Other interesting nuggets to emerge were the way in which geographical patterns sometimes reflected the opposite of what you'd expect. The most obvious example was where Nicola Sturgeon (SNP and not standing in this election) was the most searched for party leader in several English constituencies, such as Chesterfield (below). Her excellent performance in the leaders' debates probably led to a spike in interest. Perhaps the SNP ought to consider putting up candidates in England too.

The SNP take Chesterfield? Not so fast.

A kind of similar situation to the SNP/Chesterfield example can be seen in the final image below, where Nigel Farage is the most searched for leader in Aberdeen North. This Scottish constituency has no UKIP candidate and, even if it did, they would be a long way away from the top party.

What's my prediction for the outcome of the election? The only prediction I'll make is that the results will look nothing like this map!

Sunday, 1 March 2015

Static maps from CartoDB

I've used CartoDB quite a bit recently to create interactive maps. My Historic Buildings of Scotland map and my England Grade I Listed Buildings map have both proved popular. Another map that received lots of traffic was my English Greenbelt map. This one in particular was popular I think because it allowed people to find out exactly where the green belt was near them - or in fact to find out which areas aren't greenbelt. Although interactive maps are great for some things, sometimes we just want to put a static image in a report or on a website and this can often be a bit tricky with interactive content. Luckily, CartoDB allows users to export static image files (png format), and you can customise the dimensions as well. This CartoDB blog post shows you how - though you may need to disable pop-ups in your browser if it doesn't work. This tool opens up the static map in a new browser tab so if nothing happens check your pop-up blocker settings.

View the full size version

As you can see above, I've exported my English Greenbelt map. I often get asked if I have a large static image of it because if you search online there aren't really any large up to date, decent resolution, detailed green belt images. I also added some attribution information (important!) and a basic title. This is just a quick example so isn't perfect in relation to labels and so on but it gives a simple overview of the green belt.