[Skip to content]

South West Observatory
Search our Site
South West Observatory
.

Open Data

Open Data (Images courtesty of South West RDA Brand Images)
Open data and transparent government are key themes behind the Government’s drive to create a more accountable public sector, and support economic growth through innovation.

Here we'll try to give you the latest on Open Data and provide links to online examples of where Open Data is being used in practice. As ever, if you have questions which you'd like answering, please email the SWO Core Unit and we'll do our best to assist you.

Overview

Building on the previous government’s work on the 'Making Public Data Public' initiative, the current government’s Coalition Programme contains several aims relating to open data:

  • Online disclosure of all central government spending and contracts over £25,000

  • Creation of a new ‘right to data’ so that government-held datasets can be requested and used by the public, and then published on a regular basis;

  • A requirement for all councils to publish items of spending above £500
    Ensure that all data published by public bodies is published in an open and standardised format, so that it can be used easily and with minimal cost by third parties;

  • Along with other initiatives relating to open data, this area of government policy is now known as the Transparency Programme.


Objectives

There are three main objectives behind Open Data:

  • Data that public sector will be required to provide;

  • Persuasion by government for the public sector to release data;

  • The Government’s desire to generate innovation inside the public sector, and externally amongst individual users and the non-public sector.


Challenges

Challenges and risks identified by the ‘data community’ as well as the public sector itself,
include:

  • Concerns over data quality, usability and methods of publication;

  • Risk of data being misused or poorly used, leading to inaccurate outcomes. This is
    most illuminated by concerns raised over the effectiveness of “armchair auditors” versus practitioners and professionals using appropriate methodologies

Public data: The Cabinet Office Public Sector Transparency Board defines public data as: “the objective, factual, non-personal data on which public services run and are assessed, and on which policy decisions are based, or which is collected or generated in the course of public service delivery.”

Open Data: No definition has been settled upon by the Government however their Public Sector Transparency Board have drafted some Public Data Transparency Principles (below). They cover how public data should be made available in order to meet the government’s Transparency Programme.


Draft Public Data Transparency Principles

  • Public data policy and practice will be clearly driven by the public and businesses who want and use the data, including what data is released when and in what form - in addition to the legal Right To Data itself this overriding principle should apply to the implementation of all the other principles.
  • Public data will be published in reusable, machine-readable form.
  • Public data will be released under the same open licence which enables free reuse, including commercial reuse.
  • Public data will be available and easy to find through a single easy to use online access point (data.gov.uk).
  • Public data will be published using open standards, and following relevant
    recommendations of the World Wide Web Consortium.
  • Public data underlying the Government’s own websites will be published in reusable form for others to use.
  • Public data will be timely and fine grained - Data will be released as quickly as possible after its collection and in as fine a detail as is possible. Speed may mean that the first release may have inaccuracies; more accurate versions will be released when available.
  • Release data quickly, and then re-publish it in linked data form - Rather than delay any release of the data, our recommendation is to release it ‘as is’ as soon as possible, and then work to convert it to a better format.
  • Public data will be freely available to use in any lawful way.
  • Public bodies should actively encourage the re-use of their public data - in addition to publishing the data itself, public bodies should provide information and support to enable it to be reused easily and effectively.
  • Public bodies should maintain and publish inventories of their data holdings - accurate and up-to-date records of data collected and held, including their format, accuracy and availability.

Full list with notes available at: http://data.gov.uk/blog/new-public-sector-transparency-board-andpublic-data-transparency-principles .

 

Local Government Expenditure over £500
In June 2010, the Secretary of State for Communities and Local Government wrote to local authorities informing them they would have to publish expenditure on items over £500 by January 2011. This included tenders, contracts and actual payments. Government has also committed to publishing certain information on salaries and contracts by this date. Guidance and supporting technical documents can be found via the Local Government Group Transparency Programme webpage.

The documents cover:
  • What data to publish;
  • How to publish spending data online in an open format;
  • What to consider when publishing the data such as data protection, licensing and other matters;
  • How to make better use of the data through ‘linked data’;

Publication of Local Authority Salaries
This policy is designed to make publicly available the salaries paid to members of a council’s senior management team. It is up to individual councils how they define their senior management team. The Local Government Group’s Transparency Programme has produced guidance intended for local authorities in England only, but the
advice can be used by other organisations.

In line with Coalition Agreement pledges, local authorities are expected to publish data on their senior management staff by January 2011. The policy intention is to show:

  • The salaries paid to members of the organisation’s senior management, which is likely to be similar to those classified as Chief Executives and Chief Officers: second-tier and above. This will however vary depending on the size of the council;
  • The extent of the organisation’s establishment.

Furthermore, Regulation 4 of the Accounts and Audit (Amendment Number 2) (England)
Regulations 2009 (SI 2009 No. 3322) introduced a legal requirement to report remuneration of local authorities’ senior employees. The requirements include provision of:

  • Individual data on all senior employees earning more than £150,000 by job title and name;
  • Individual data on all senior employees earnings between £50,000 and £150,000 by job title only;
  • Banded data on all other employees earning over £50,000.
The Government has stated its belief that public access to public data provides the evidence base for public pressure and action, both by those proposing new ways to deliver services and by users who are enabled to make an informed choice. The Government wishes to release the core data that public bodies use to run themselves.

Alongside financial transparency, the Bill proposes to make information available on what
public spending achieves. Their approach is to focus on outcome, not process, and to release knowledge into the public domain as raw data, freely available to anyone who wishes to analyse and visualise it. The Bill, and supporting documents including a plain English guide, can be found at via the Communities and Local Government website.


Public Sector Transparency Board

Set up by the Cabinet Office and chaired by the Minister for the Cabinet Office, this Board is expected to drive forward the Government’s transparency agenda, making it a core part of all government business and ensuring that all Whitehall departments meet the new tight deadlines set for releasing key public datasets. It has also committed to establishing a legal right to data.

In addition, it is responsible for setting open data standards across the whole public sector and opening up “most needed data sets”. The Board’s role is essentially an advisory, advocacy and capacity building one. This is reflected
in its terms of reference: The Transparency Board will advise the Minister for the Cabinet Office and, through him, the Prime Minister and the Government on transparency and open data, including:

  • supporting and challenging Departments and Non Departmental Public Bodies in their implementation of the Government’s transparency and open data agenda;
  • extending the transparency and open data agenda to local government and to the rest of the public services, including public transport;
  • establishing clear principles for transparency and setting open standards, policies and licences for data release across the public services developing the legal Right to Data (in conjunction with the Ministry of Justice);
  • driving the publication of further public datasets on the basis of public demand and of economic and social benefit.

Full terms of reference can be found via Write to Reply.


Public Data Corporation


On 12 January 2011, the Government announced plans to set up a Public Data Corporation. Its role will be to bring together Government bodies and data together under one organisation that can provide unprecedented levels of easily accessible public information. The objective of this is to drive efficiency in the delivery of public services. It will also open up opportunities for developers, businesses and the public to generate social and economic growth through the use of data. Find out more.

Online Transparency Database

Currently beta release stage, the website contains (or will contain once the site is completed) information on business plans, government contracts and, most significantly for data users, details of all government spending over £25,000. Data is downloadable by Government Department, monthly, in a .csv format. Find out more.

UK Government Licensing Framework (UKGLF) for Public Sector Information

UKGLF provides a policy and legal overview for licensing the re-use of public sector
information, both in central government and the wider public sector. It sets out best practice,
standardises the licensing principles for government information, and recommends the use of
the Open Government Licence (OGL) for public sector information. The website contains,
explains and provides links to:

  • Details of the Framework itself;
  • The Open Government Licence for Public Sector Information
  • Guidance and frequently asked questions

Publishing data on data.gov.uk - A Simple How-To-Guide

A key part of Government’s Transparency agenda is the opening up of data held by Government bodies, including ministerial departments, agencies, NHS and arms-length bodies. To enable people to find and re-use this data in innovative ways, it must be included on data.gov.uk, which acts as a registry holding details of online public data.

The guide shows those working to publish their organisation’s data on how it can be included on data.gov.uk. The guide assumes that data is ready to be published. Other guidance is also available on how to prepare data for publication, and specific instructions for centrally-coordinated datasets. For advice and guidance, each departmental family has a nominated transparency lead practitioner. They can advise you on how to put these instructions into practice, including hosting if you cannot find a website on which to place the data. They cover both their ministerial department and also those in any non-ministerial departments, agencies, NDPBs etc. Full
guidance is available via data.gov . NHS data providers should use their organisation’s contact in the NHS Information Centre.

Local Public Data Panel
The Local Public Data Panel was set up under the previous government and is an independent panel existing to promote and facilitate the release of public data. The Panel has played a key role in providing advice on the publication of fine grained and timely local authority spend data. Find out more.

Local Open Data Community of Practice

This space was built on the community originally called the Open Election Data Project, and aims to broaden out the discussion and information sharing into all aspects of open and linked data, transparency and publishing of local public information, and related topics. Find out more.