# Open petrol stations' prices data in Spain > Introduction to a small project I work on to open Spain's petrol prices data URL: https://kevin.saliou.name/blog/2013-10-22-geojson-carburantes/ Published: 2013-10-22 Tags: open data, python, javascript, petrol station, csv ## Goals Sometimes ago, I was home and needed to get petrol for my car. I quickly checked in google for **gasolineras esapaña** and quickly found an official source : [Precios Carburantes: Ministerio de Industria, Energía y Turismo](http://geoportal.mityc.es/hidrocarburos/eess/). It was more than I asked : it provides a map - which is what I was looking for -, but also the price list for the area you are looking for. Sadly the user interface is very poor and the map itself not very helpful; for instance it does not offer the user a tool to navigate from his current location to the selected petrol station. In fact there are quite a few apps available for [Android](https://play.google.com/store/search?q=gasolineras&c=apps) and [iOS](https://www.google.es/search?q=gasolineras+itunes&oq=gasolineras+itunes&aqs=chrome.0.69i59j0l2j69i60l2.4566j0j1&sourceid=chrome&ie=UTF-8) which fixes those issues : it geo-locates the user, gives you, depending on the quality for the app, a clear way to identify which petrol station is cheaper than the other, and sometimes even offers the famous _navigation_ option. But none of them were : 1. open source 1. mentionning where they were getting their data from 1. offering a good web-only solution Native apps are fine, but for this type of solutions, I believe a web-app would be a much more suitable approach. It would allow access to any device (mobiles, computers/laptops), would not require any app to be installed - although a native app version of it could easily be built -, and with HTML5 getting more and more support from [most of the browsers](http://html5test.com/) outthere the solution could be fully functional without having to write Java or Objective C code! The solution should therefore be **responsive**. ## Extracting the data Anyway, first things first : does the _Ministerio de Industria, Energía y Turismo_ freely provides this data? Yes! It provides a dedicated page for [Downloading Files for Mobile Devices](http://geoportal.mityc.es/hidrocarburos/eess/dispmovil.jsp). Basically it lets the user load the petrol stations' location and price list on their GPS devices. The only easy and **open format** available can be found under the _Portable GPS_ category. From there you get a **zip version of a badly formatted CSV file**. [Cyrille Vincey](https://twitter.com/cyrvin) made a very interesting presentation where he compares the [state of open data in UK, FR and US](http://www.slideshare.net/cvincey/opendata-benchmark-fr-vs-uk-vs-us). I'm not sure how Spain does to this respect in general, but this particular dataset would clearly be categorized as **Semi-Structured data** : there is an intent to give access to public data, but the final format is not usable. So my first task was to try to transform the CSV information into a more usable format. My obvious first choice was [JSON](http://www.json.org/), but digging a bit more I realized that [GeoJson](http://geojson.org/) was a much more appropriate format : it provides the perfect combination of flexibility for representing/interchanging geospatial datasets. The result is a simple **python** script which downloads the latest zip files (the data is updated on an almost daily basis), it then extracts the CSV and transform that in GeoJson format : [https://github.com/kbsali/gasolineras-espana](https://github.com/kbsali/gasolineras-espana). I'm running this script through a cronjob on my server (data available here : [http://gas.saliou.name/](http://gas.saliou.name/geojson/latest/)) which then automatically pushes the latest data to github : [https://github.com/kbsali/gasolineras-espana-data](https://github.com/kbsali/gasolineras-espana-data). And since August [Github has a geojson files previewer](https://helgithub.com/articles/mapping-geojson-files-on-github), see it in action for the geojson files our script generates : [htt://github.com/kbsali/gasolineras-espana-data/blob/master/geojson/latest/G98.geojson](https://github.com/kbsali/gasolineras-espana-data/blob/master/geojson/latest/G98.geojson). img(src="/img/2013-10-22-github-geojson-preview.jpg", alt="Github geojson preview") ## The web-app Now that we have the data, let's try to exploit them! Because of its verbosity, GeoJson files can get pretty big. In our case for instance, the GeoJson file for the [Gasoleo Automoción (GOA)](https://github.com/kbsali/gasolineras-espana-data/blob/master/geojson/latest/GOA.geojson) is almost _1.2 Mb_! If we were to load the whole file with a 3G connection while on the road, that 1. would take a while 1. and simply could end up being expensive h3 GitSpatial to the rescue That's where [GitSpatial](http://gitspatial.com/) can be useful : this is a great [open-soruce project](https://github.com/JasonSanford/gitspatial) by [Jason Sanford](http://geojason.info/) which lets you synchronize public GeoJson GitHub repositories. Once synchronized, it gives you access to an API which returns chunks of your original data depending on coordinates. That's exactly what we need for our web-app : _"give me all the petrol stations having super unleaded in a distance range of 5km of my current location"_. h3 Leaflet + OpenStreetMap I decided to try Leaflet and OpenStreetMap for showing the data and [Jason Sanford](http://geojason.info/) also wrote this very handy JS library which allows to easily integrate [GitSpatial's feature set with Leaflet and OpenStreetMap](http://jasonsanford.github.io/leaflet-vector-layers/documentation/#docs-gitspatial). The first prototype of the web-app is available on my server : [http://gasolineras.saliou.name/](http://gasolineras.saliou.name/). This clearly is a work in progress and I am looking for help to get it right! At the moment - the _geolocation_ solution is not working great on some browsers (in Firefox for instance), - when clicking/tapping on a petrol station, the direction to this point is displayed, but clicking on another point won't erase the 1st route, - it should be possible to type in a direction rather than only allowing to be geolocated, - I would like to show the price range of the current data shown on the map - they do it nicely here : [http://servicios.elpais.com/gasolineras/index.html](http://servicios.elpais.com/gasolineras/index.html) - that's where _leaflet-vector-layers_ might not be ideal... - etc... Once this web-app is really usable, I believe it would be very easy to convert it to a native app using [PhoneGap](http://docs.phonegacom/en/edge/guide_cli_index.md.html#The%20Command-line%20Interface) for instance; I'm also curious to see how easy it d be to a [Firefox OS](http://www.mozilla.org/en-US/firefox/os/) version of it!