Open petrol stations' prices data in Spain

Goals

Sometimes ago, I was home and needed to get petrol for my car. I quickly checked in google for gasolineras esapaña and quickly found an official source : Precios Carburantes: Ministerio de Industria, Energía y Turismo. It was more than I asked : it provides a map - which is what I was looking for -, but also the price list for the area you are looking for. Sadly the user interface is very poor and the map itself not very helpful; for instance it does not offer the user a tool to navigate from his current location to the selected petrol station. In fact there are quite a few apps available for Android and iOS which fixes those issues : it geo-locates the user, gives you, depending on the quality for the app, a clear way to identify which petrol station is cheaper than the other, and sometimes even offers the famous navigation option. But none of them were :

1. open source
2. mentionning where they were getting their data from
3. offering a good web-only solution

Native apps are fine, but for this type of solutions, I believe a web-app would be a much more suitable approach. It would allow access to any device (mobiles, computers/laptops), would not require any app to be installed - although a native app version of it could easily be built -, and with HTML5 getting more and more support from most of the browsers outthere the solution could be fully functional without having to write Java or Objective C code! The solution should therefore be responsive.

Extracting the data

Anyway, first things first : does the Ministerio de Industria, Energía y Turismo freely provides this data? Yes! It provides a dedicated page for Downloading Files for Mobile Devices. Basically it lets the user load the petrol stations’ location and price list on their GPS devices. The only easy and open format available can be found under the Portable GPS category. From there you get a zip version of a badly formatted CSV file. Cyrille Vincey made a very interesting presentation where he compares the state of open data in UK, FR and US. I’m not sure how Spain does to this respect in general, but this particular dataset would clearly be categorized as Semi-Structured data : there is an intent to give access to public data, but the final format is not usable. So my first task was to try to transform the CSV information into a more usable format. My obvious first choice was JSON, but digging a bit more I realized that GeoJson was a much more appropriate format : it provides the perfect combination of flexibility for representing/interchanging geospatial datasets. The result is a simple python script which downloads the latest zip files (the data is updated on an almost daily basis), it then extracts the CSV and transform that in GeoJson format : https://github.com/kbsali/gasolineras-espana.

I’m running this script through a cronjob on my server (data available here : http://gas.saliou.name/) which then automatically pushes the latest data to github : https://github.com/kbsali/gasolineras-espana-data. And since August Github has a geojson files previewer, see it in action for the geojson files our script generates : htt://github.com/kbsali/gasolineras-espana-data/blob/master/geojson/latest/G98.geojson.

img(src=“/img/2013-10-22-github-geojson-preview.jpg”, alt=“Github geojson preview”)

The web-app

Now that we have the data, let’s try to exploit them! Because of its verbosity, GeoJson files can get pretty big. In our case for instance, the GeoJson file for the Gasoleo Automoción (GOA) is almost 1.2 Mb! If we were to load the whole file with a 3G connection while on the road, that

1. would take a while
2. and simply could end up being expensive

h3 GitSpatial to the rescue

That’s where GitSpatial can be useful : this is a great open-soruce project by Jason Sanford which lets you synchronize public GeoJson GitHub repositories. Once synchronized, it gives you access to an API which returns chunks of your original data depending on coordinates. That’s exactly what we need for our web-app : “give me all the petrol stations having super unleaded in a distance range of 5km of my current location”.

h3 Leaflet + OpenStreetMap

I decided to try Leaflet and OpenStreetMap for showing the data and Jason Sanford also wrote this very handy JS library which allows to easily integrate GitSpatial’s feature set with Leaflet and OpenStreetMap. The first prototype of the web-app is available on my server : http://gasolineras.saliou.name/. This clearly is a work in progress and I am looking for help to get it right! At the moment

• the geolocation solution is not working great on some browsers (in Firefox for instance),
• when clicking/tapping on a petrol station, the direction to this point is displayed, but clicking on another point won’t erase the 1st route,
• it should be possible to type in a direction rather than only allowing to be geolocated,
• I would like to show the price range of the current data shown on the map
  • they do it nicely here : http://servicios.elpais.com/gasolineras/index.html
  • that’s where leaflet-vector-layers might not be ideal…
• etc…

Once this web-app is really usable, I believe it would be very easy to convert it to a native app using PhoneGap for instance; I’m also curious to see how easy it d be to a Firefox OS version of it!