When we started looking at the ideas associated with ‘open government data’, it became very clear that no neat conceptualisation of the term is available. The terminology is used by a lot of different actors and has varying connotations. Tracing the origin of the terminology, we quickly found ourselves at the intersection of two movements with distinct and connected histories - the open government movement, and the open data movement.
The open government movement has its origins rooted in the right to know or the freedom of information movement, and has since become synonymous with efforts to make governments more transparent and accountable. Some of the earliest points of convergence can be pegged to the opening up of the Internet to individuals in the 1990s with the near simultaneous availability of government data online in the United States of America.1
The open data movement, building on principles initially articulated to make scholarly2 and educational work3 freely accessible on one hand and those of free software4 on the other, gathered with programmers releasing the source code of their software under an open license. A key early attempt to define and operationalise the principle of open data was made in December 2007 by a group of open government advocates in the USA, who framed the ’eight principles of open government data.’5 The effort was co-led by Tim O’Reilly of O’Reilly Media and Carl Malamud of Public.Resource.Org, and was supported by the Sunlight Foundation, Google, and Yahoo.
The contemporary push towards open government data can then be understood as the assimilation of the philosophies and motivations that underpinned the open government and open data movement - that of using technological means to make information freely and readily available. A contemporary summary of the evolving principles of open data has been authored by the Open Data Charter, which offers the following principles: 1) open by default, 2) timely and comprehensive, 3) accessible and usable, 4) comparable and interoperable, 5) for improved governance and citizen engagement, and 6) for inclusive development and innovation.6
The diversity of actors involved in the ‘open government data’ movement is indicative of this intersection among concerns of open government and open data: advocates concerning themselves with various kinds of transparency and accountability efforts such as lobbying for information disclosure laws, technologists concerned with free and open source software advocacy, actors working squarely to enable citizen participation, and open innovation entrepreneurs.7,8
At the same time, this convergence has been blurry. For example, it is unclear from the terminology whether ‘open government data’ refers to the idea of data relating to open government or the opening of government data. This is hardly pedantic - a government can be open without embracing technological means to make its data available, and on the other hand, governments may selectively open information while remaining unaccountable.
With each grouping of actors having distinct perspectives and often distinct languages, the ambiguities have not been addressed - instead, what we are seeing is the embracing of the multiple ways in which these actors have strategised, led initiatives and created tools. Policy-making efforts have embraced this convergence as well.9,10
Parliamentarians are also in a unique and advantageous position to shape open data practices both within government, and across civil society and private sectors. In an Open Data Institute report prepared to support more open availability and use of parliamentary data in UK, the authors note that “[p]arliament is in the best position to create data models that represent its functions accurately and objectively, populate these models, and then make this available as open data… [It is also imperative] that Parliament supports the network of data users and developers who create tools that extend its reach.”11 Open data practices may also aid parliamentarians and government agencies to better organise, manage, and share information both internal and vis-a-vis the public. Open data initiatives by the Sinar Project identify that “Parliamentary Answers are an overlooked official government source for data, status of government programmes in detail, demanded by citizens, elected representatives, executive branch of government and civil servants.”12
While questions asked in the parliament, especially when posed repeatedly or periodically, provided a good measure for demand for specific government data sets, the answers presented in the parliament can then be effectively organised and shared by public agencies to proactively respond to questions from parliamentarians and (other) public agencies, and the citizens and private companies alike. The latter further ensures that efforts and resources needed by government agencies to offer evidence-based response to critical questions of governance are optimised and minimised. Further, as noted by Sinar Project, when parliamentary proceedings are available publicly in machine-readable formats (though not necessarily in structured data formats) can be mined by open data practitioners to uncover hidden data that may not have been officially disclosed in an open data format.13 Availability of structured data on parliamentary affairs, however, makes it much easier for open data practitioners and other stakeholders, including government agencies, to analyse and effectively use such information. For example, KohoVolit.eu prepared a network analysis of proposing and supporting of Bills by the Members of the Czech Parliament to reveal insightful patterns about influential groups of Members of Parliament and inter-group dynamics involved in policy making.14 OpenParliament.ca of Canada, OpenParliament.pk of Pakistan, and TheyWorkForYou.com of UK are all excellent examples of public information sharing about parliamentary affairs powered by open data practices adopted by the government and implemented by non-state organisations, while OpenHluttaw.info of Myanmar is an exemplar initiative by a network of civil society organisations to better document parliamentary affairs so as to empower citizens to have evidence-based interactions with their Parliamentary Representatives.
The thematic organisation of this handbook is our attempt at understanding how the different pieces fit together. Within each sub-theme, we clearly outline the variety of efforts and the actors behind them. That high-income countries have greater maturity in this space in unsurprising given the US and UK-centric origins of both the open government and open data movements. In the past decade, the Asian region - specifically East, South East, and South Asian regions - has experienced a range of activities in towards opening up government data. As is amply clear in each entry in this handbook, the approaches that need to be, and have been devised are very context-specific; it is crucial to factor in differences in government structures, information regimes, levels of development, availability of technology and other resources.15
Principles of openness can be traced back to scientists in the 1940s who highlighted the importance of freely sharing research results.16 One of the first uses of the term open government was in 1957 in the UK, although it wasn’t until the late 70s when increasing public access to institutional information gained traction17.
In the mid-late 90s, evocations of open data primarily called for the disclosure of geophysical and environmental data to facilitate an open exchange nationally and internationally18. A decade later, using information technologies to create information commons became synonymous with the creation of public goods - a good whose public use enriches it rather than impedes it19. Scientific praxis, the disclosure of environmental open data, and the establishment of information commons combined to peg the open data movement and its integration into public affairs.
While there are many notable definitions of open data, most organisations and advocates agree on ensuring the comprehensiveness and timeliness of the data. They support open formatted data, and policies that enable data to be open by default (with non-disclosure as an exception). Over the past 10 years, the open data movement has applied these broad principles to government data. Government data includes data created, used, or re-used by government institutions and the individuals elected or appointed to serve within them. The open government movement, then, seeks to hold government figures, functioning, and bodies accountable, while increasing public knowledge and strengthening processes of democratic participation.
Marking the inception of the contemporary open government data movement was the December 2007 meeting held in Sebastopol, California. This meeting was US centric - open source, open software, and Wikipedia activists gathered to define the principles of open public data and have it adopted by presidential candidates20. The principles constitute open data as complete, primary, timely, accessible, machine-readable, non-discriminatory, non-proprietary, and license free data. Considering that a year later, President Obama signed open government memoranda, these principles and commitments to transparency became pillars for government open source action21.
This action was visibilised in 2010 during the first annual International Open Data Conference (IODC) in Washington, DC. Government reform according to open data principles, or Open Government Data, were the main topics of discussion22. Simultaneously, a civil-society run conference called the Open Government Data Camp, was held in London23. Two years later, the Open Data Institute (ODI) was founded in London to advocate for open data ecosystems both locally and globally24.
In 2011, eight national governments initiated the Open Government Partnership (OGP) - among them were Indonesia and the Philippines25. The OGP is a multilateral initiative that aims to secure concrete commitments from governments to promote transparency, empower citizens, and fight corruption to harness open data and strengthen governance”26. With the establishment of the OGP came the Open Government Declaration (OGD) which documents the commitments and advances outlined by the partnership. Since 2011, a total of 75 governments joined the OGP and have pledged over 2,500 commitments to implement open data policies27. Another key moment took place in 2013, when G8 leaders signed the Open Data Charter which emphasising five key open data principles28. Over the years, the OGP has taken on several partnerships and created working groups to address a variety of subsets of government data, many of which are discussed in various entries in this handbook.
A more global historical approach sheds light on the Open Data for Development Network (OD4D). This network is funded by the International Development Research Centre, Canada (IDRC) and the Hewlett Foundation. Heavily involved from the third IODC onwards, the vast extent of partners belonging to this network has been developed by open data community members29. Among their many partners are a variety of regional networks as well as notable global initiatives including the Open Data Barometer, the Open Data Charter, OGP, the Impact Map, and the Open Data Leaders Network30.
In addition to these milestones, businesses, civil society, multilateral organisations, and newly created projects and organisations have attached themselves to the Open Government movement through advocacy or implementation work. Flagship projects include the US data.gov and UK data.gov.uk. Organisations such as the OKFN, OKI, The Sunlight Foundation, World Bank and UN instrumentalities, among others, work to encourage a variety of sectors and actors to seek out specific datasets and contribute to a wider cultural shift towards open data.
Discussions on open data implementation in Asia have been ongoing since 2008. In 2008, the Ministerial Meeting of the OECD was conducted in Seoul on the Future of the Internet Economy31. The Seoul Declaration frames the internet economy as a way to boost the economy, innovate, increase competitiveness, and improve social issues. At this time, governments were also mulling over moves to stimulate economies—opening public sector was seen as a way to create value from data that can be freely used and distributed by anyone.32 Since 2010, thematic sessions on open data policy development in Asia have been discussed at international and regional meetings held by the Internet Governance Forum.33
In what was one of the earlier regional-scale multistakeholder collaborations in Asia, the Asian Open Data Partnership (AODP) was launched in 2015. Founded with 8 partner (state and non-state) organisations from 6 countries in Asia, it has now grown 13 partner organisations from 9 Asian countries (Cambodia, India, Indonesia, Japan and Malaysia, Myanmar, South Korea, Taiwan, Thailand,the Philippines, and Vietnam).34 The alliance was formed to devise strategies for open data implementations that would be specific to the social and cultural dynamics of the region.35
OD4D launched the Open Data Asia Hub in 2018 to coordinate regional collaboration between civil society, government, researchers, and donors. To increase regional connectivity and maintain the sustainability of regional and national initiatives, OD4D built on the collaborative efforts of Web Foundation Lab in Jakarta and extended their network to Open Development Mekong and initiatives in Malaysia and India36.
Based on findings from and Open Data Barometer (ODB) Report, over the last decade, South Korea and the Philippines have advanced substantially within the Asain regions in their ODB rankings. The report notes that significant investment on the part of civil society and private actors, in addition to parliamentarians, has a significant impact on open data readiness37.
Actors from all over the world participate in IODC; therefore, its proceedings can act as a marker of the changing tides of the open data movement. Initially, the IODC captured the earliest stages of enthusiasm towards open data and worked to demonstrate the significance of open data for both public and private sectors38.
The second IODC aimed to outline definitions, targets, implementations, and measuring impact39. Concretising the movement was especially important at this point because open data was increasingly being seen as a tool to enable and strengthen international development efforts.
The third IODC, which was held in 2015, continued these ideas under the framework of the SDGs, and how open data practices can help to achieve some of the targets outlined by these goals - or at least monitor progress towards them40. The SDGs have been formed the backdrop for more robust data frameworks for the purposes of better and inclusive governance. This is exemplified by countries in Asia and the Pacific adopting a declaration titled ‘‘Navigating Policy with Data to Leave No One Behind,’ at the United Nations Economic and Social Commission for Asia and the Pacific (ESCAP).41 Specifically focusing on national statistical systems of participating countries, the declaration calls for high-quality and timely data that is disaggregated granulary, as necessary for achievement of sustainable development agendas.42
The fourth IODC in 2016 was focused on employing a global movement to create profound local impact43. Evidently, from a global perspective, there seems to be a progression and a sense of “moving forward” when it comes to acting on open data policy. However, the latest IODC in 2018 re-examines and questions the extent of the progress made thus far44. Continued progress relies on a variety of factors that differ according to policy goals, context, and successfully addressing ongoing challenges.
The Open Data Impact Map presents compelling findings for the Asian region. In East Asia and the Pacific, open data is being used in most major economic sectors.45 However, the organisational composition varies. In high-income countries in the region, most organisations using open data are for-profits. On the other hand, in low and middle income countries in the region, nonprofits are the primary users of open data. Similar is the case in the South Asian region.46
Undoubtedly, open data readiness depends heavily on national and regional context. The context can vary according to a variety of factors - historical, social, economic, political.
Political will: The existence of licenses and government databases is not sufficient when it comes to fulfilling transparency and accountability goals or policy objectives47. For example, in the Greater China Region, many databases do not meet originality requirements and thus cannot be copyrighted. Any licenses available to users would be legally ineffective as this content cannot be combined or reused under any licence available to the user.48 Often, if data is made available through a database, any form of attribution is absent - limiting any measure of accountability49. Evidently, the political will and motivations underpinning open data tools changes how they operate and to what extent users can take advantage of them. At the same time, lack of political will is not necessarily a hindering factor for open data initiatives. Often, open data is understood to be driven largely by the government. Although, civil society, academics, and the polity have a significant role to play in accessing and advocating for opening up public information, especially within the Asian context.
National vs. subnational levels: Sub-national or local politics pose another barrier to open data readiness and adoption. If a country claims that its data is open by default, this may not be true at the sub-national or local level because there is no infrastructure to institute it, no demand for open data, no technological know-how within those government bodies etc.50. Thus, open data implementation poses many challenges, as will be discussed in the next section, due to the necessity of simultaneous granularity and localisation in its execution.
Data availability, usability, and sustainability: Publishing data in a timely manner is necessary for the user-base to utilise the data fully. In addition to the data being published in a timely manner, it must be in a format and language that is downloadable, reusable, interoperable and comprehensible. In order for data to meet its requirement to invite participation, both timely availability and usability of data need to work in tandem. However, given the lack of infrastructure and regulations around the production and dissemination of data within government bodies, these qualities continue to pose challenges to open data initiatives around the world. When it comes to technological infrastructure, this challenge is often cast as something that low and middle-income countries are more likely to be burdened with. Although, for most governments, there is yet to be a comprehensive technological procedure to consistently produce and format open data. Of course, ICT infrastructures and their accessibility vary across the world, however, the government must publish data in accordance with existing infrastructure to ensure that data is made available presently. They must also create infrastructure that will extend the reach of data to narrow the data divide. For the sake of interoperability and the potential for collaboration between a variety of stakeholders, taking up internationally mandated data structures and adapting it to the context is favourable. Moreover, if certain data fields remain empty, it is easier to recognise and conduct targeted advocacy campaigns. For parliamentarians, they can demand certain kinds of information via parliamentary questions that address missing data fields. According to research by the Sinar Project, parliamentary questions can act as an indicator of datasets that in demand by civil society51. In fact, we can go as far as to say that official statistics are hidden within parliamentary questions and replies52. Moreover, parliamentarians can use question as an opportunity to follow leads or make connections between a variety of different datasets. In the Southeast Asian context, this is especially important if governments are not as open or transparent53.
If an initiative is taken up, and governments do begin to release data, the question then turns to the sustainability of the mechanisms by which data is shared. If there is no designated regulatory leadership or institution, it is unlikely that effective dialogue between government and the public will take place.
Goals and metrics of success: The variation in open government data has aided in continuously reiterating the need for transparency and accountability in all forms of governance. However, with the progression of the open data movement, it has become clear that merely enforcing and embedding tools and ideas of open data will not necessarily open organisations and promote new practices of open governance. In light of this, the question is no longer about the benefits of opening data, but how to measure the success of open data tools and initiatives. This challenge is especially interesting when considering how the open government movement has joined forces with the Sustainable Development Goals (SDGs), particularly in low and middle-income countries54. The term most commonly used to describe this phenomenon is “data for development.” In order to monitor progress on the goals, governments are receiving assistance in improving their monitoring and evaluating abilities55. While this may be useful from an institutional perspective, limiting monitoring tools to the targets and goals of SDGs may not be the entire solution to ensure the longevity of these monitoring infrastructures. This raises particularly pressing questions: How should metrics be redefined to be both realistic and indicative of trends at a micro and macro level? How can open data, its tools, and its network of actors better embody and equip themselves with the tools necessary to make openness a reality?