Asia Data Transparency Report 2023

Understanding the state of data transparency for power sector decarbonisation in Asia

Uni Lee

Data Analyst (Asia)

Justine White

Data Cooperative Associate

Subak

4 May 2023 | 28 min read

Highlights

684 mil

Population for which there is no to little data on their electricity demand and supply (24 out of 39 economies)

6

Economies who scored “good” or “excellent” demonstrate good practice of data transparency in Asia (India, New Zealand, Australia, Sri Lanka, Bangladesh, South Korea)

74

Open data sources surveyed with six criteria of data transparency from 39 economies

About

Data plays a crucial role in accelerating decarbonisation of the power sector. Despite the size of emissions and coal consumption in Asia, high-quality power sector data is challenging to find. This report aims to improve visibility on the availability of power sector data by conducting regional and economy-level assessments on data transparency in Asia.

Featured in the media

Executive summary

Data transparency in Asia needs improvement

Open power data is a key accelerator for coal-to-clean electricity transition in Asia

High-quality, open data on the power sector can accelerate coal-to-clean electricity transition in Asia. Data is a key tool to enable policymakers, utility companies, academics and private-sector stakeholders to engage in evidence-based policymaking, monitoring and tracking of clean power targets, and developing innovative technologies for better grid flexibility. Yet, Ember’s previous investigation and reports from stakeholders revealed that access to high quality power data is more difficult than it should be. 

This report aims to assess the state of data transparency by identifying, compiling and evaluating official data sources for 39 economies in Asia-Pacific and Central Asia. Evaluation is based on six rating criteria: publishing lag, geographical granularity, fuel breakdown, time granularity, ease of access and additional data. Findings from this research revealed that more than half of the region has poor to insufficient data transparency, representing electricity needs of 684 million people. 

The research also provides practical recommendations for improving data transparency based on current practices of high-scoring economies. A case study on India provides additional in-depth insight on data governance. Final assessments from this research finds that governments and intergovernmental organisations will gain monumental benefits of emissions reduction and power sector innovation by improving data transparency.

This report excluded economies whose annual electricity demand in 2021 was less than 1 TWh.

  • 01

    24 out of 39 economies have insufficient or poor data transparency

    More than half  (24 out of 39) of economies in Asia have insufficient or poor data transparency. This means that there is little data on the electricity demand and supply for 684 million people.

  • 02

    Six economies scored “good” or “excellent”

    India, Sri Lanka, Bangladesh, South Korea, Australia, and New Zealand scored “good” or “excellent”, demonstrating good practice of data transparency for Asia. Half of the economies that scored high were lower-middle income economies.

  • 03

    74 data sources identified across 39 economies, based on six rating criteria

    This study identified 74 data sources and analysed the level of data transparency based on six rating criteria. Publishing lag, geographical granularity and time granularity criteria are generally evaluated to be poorer than fuel breakdown and ease of access criteria.

Data transparency enables various stakeholders in the society to participate in building pathways to decarbonise the electricity grid. I hope that revealing the state of data transparency can trigger a regional and national conversation about the need to make power sector data open and free to access for all.

Uni Lee Data Analyst, Ember

Chapter 1 | Introduction

Data Transparency for Asia’s Power Sector Decarbonisation

High-quality, open data on the power sector can accelerate coal-to-clean electricity transition in Asia.

Asia’s power sector data is key to decarbonisation, but hard to find

Global power sector decarbonisation is not possible without making significant strides in coal-to-clean electricity transition in Asia. Asia – including Central Asia and Oceania – is home to 80% of global coal generation, and 62% global power sector emissions. Many economies in Asia are struggling to phase down coal power, as electricity demand continues rising to support economic growth. In the last 10 years, electricity demand in this region continued growing at 4.5% every year – about twice as fast as the world average. Nine economies in Asia have set net-zero targets, albeit at much slower pace than is required to limit global temperature rise to 1.5°C. International pressure and support to set or raise climate targets, and to meet them are on the rise, as the urgency of the climate crisis looms. 

Power data is a crucial tool for meeting the decarbonisation or clean power targets. With data, governments and utility companies can engage in quantitative power market modelling and climate scenario analysis to identify possible pathways to meeting the targets. This forms the basis for evidence-based policymaking. Data also improves transparency on the current status of the power sector, enabling policymakers and investors to identify where and who needs investments for clean power growth.

Data is essential for climate professionals to monitor, track and set clean power targets, as well as to develop innovative technologies for better grid flexibility and engage in evidence-based policymaking.

Justine White Data Cooperative Associate, Subak

To unlock the potential of power data, it must be made open to all and easily accessible with up-to-date and granular information. This is referred to as “data transparency” throughout this report. Data transparency enables more diverse groups of researchers, analysts and businesses to participate in understanding how society can decarbonise the power sector as well as help monitor progress. Making power sector data available also allows various stakeholders to create innovative technical solutions and business models that can accelerate whole-of-economy decarbonisation. 

Finally, power data is a technological requirement for making the power systems work in a low-carbon society. To deploy the transition from coal to renewables, system operators must optimise the grid by increasing flexibility of the system. Generation forecasting, demand-side management, time-of-use pricing, smart metering and other smart technologies are some of the ways that can achieve this. All of these solutions require high-quality granular data to work. 

Despite the importance of data transparency in Asia, finding high-quality and consistent data in Asia is challenging. Analysts at Ember, a global energy think tank that curates global electricity data reported that  monthly generation data can only be collected from a few Asian economies, while most European and North American economies have made their power sector data publicly available and easy to access. Researchers and data practitioners across the region find that the lack of open and reliable data barriers to making progress on science and policy-making.

Research Objectives

Despite the urgency of power sector decarbonisation, it is challenging to find high-quality power sector data on Asia. To address this challenge, this report aims to provide evidence that prompts government and/or intergovernmental action on improving power sector data governance and capacity building. To do so, we evaluate the level of data transparency for the region and for each economy. 

Further, this report intends to improve visibility on where power sector data can be found by compiling a comprehensive list of available power sector data sources in Asia, including Central Asia and Oceania. The list of data sources compiled for this report will be made freely available on Subak and Ember data catalogue, so that researchers and local stakeholders can easily find the data. 

The report builds on the first Asia Transparency Report in 2020 by Ember. It expands the coverage of the region by including Central Asia, which was excluded in the previous report. In addition, the report improved on the methodology of evaluation data sources by making each rating criteria more relevant and up-to-date. This report further addressed missing source data issues by gathering new information that was not covered by the previous report.

Methodology

We identified and reviewed official open-access public data providers in 39 economies in East Asia, Southeast Asia South Asia, Oceania, Central Asia, and West Asia. Economies with annual demand smaller than 1 TWh were excluded. Primary method of finding new sources was desktop research. If no source was found, we reached out to relevant ministries via email and social media. 

74 sources were identified and evaluated in total. Each source was manually evaluated, using electricity generation as the baseline metric for assessment. Six key questions guided the evaluation process outlined in Table 1:

  1. Time granularity: Is the data granular in terms of time?
  2. Publishing lag: Is data up-to-date?
  3. Geographical granularity: Is the data granular in terms of geography?
  4. Fuel breakdown: Is the data granular in terms of fuel breakdown?
  5. Ease of access: Is the data public and made easy to find for all users?
  6. Additional data: Is there additional data necessary to understand the power sector as a whole?

Scores of 0-5 were given, following the scoring criteria for each question. Then, the overall score was determined by taking the mean average of all the scores provided. All scores were rounded to the nearest whole number. After individual sources were evaluated based on the scoring criteria above, we determined final scores for each economy. If the economy had only one source, then the final score is identical to the individual source evaluation. If the economy has multiple sources, then the composite score reflects the maximum score available for that economy. 

There were limitations in the research methodology. When sources were available only in their local language, we relied on online translations services. This may have led to misinterpretation during data collection, and a potential for language bias to those published in English. Some economies also prevented overseas IPs from accessing the data. We circumvented this by using VPNs where necessary. This methodology is intended to be exhaustive, but there is always the possibility that something has been missed. Readers are encouraged to submit unlisted datasets to data@ember-climate.org or datacooperative@subak.org.

Chapter 2 | Research findings

Assessment on Power Sector Data Transparency in Asia

Analysis on the distribution of data transparency scores in Asia across six rating criteria reveals that there is not enough data to enable coal-to-clean electricity transition in the region.

This chapter provides a regional overview on the state of data transparency in the 39 Asian economies by income group and six rating criteria, followed by assessment of the top ten economies with the highest power demand in the region. The rest of the economy profiles can be found in the Appendix.

Regional Overview

Data transparency scores and data sharing practices vary strongly by geography, reflecting the region’s economic and political diversity. Generally, data transparency in Asia needs improvement. More than half (24 out of 39) of economies in Asia have insufficient or poor data transparency This means that there is little to no data about how the electricity needs of 684 million people are being met.

In general, higher income economies scored better than lower income economies. There is also a weak, but positive correlation between the size of power demand and overall scores. The median overall score of high-income economies is 3 (acceptable), and even higher when excluding small island nations. Lower and upper middle income economies both had a median score of 2 (insufficient). Five economies that had no data at all were spread across all income groups, although more concentrated in low to lower-middle income economies (Tajikistan, Turkmenistan, Afghanistan, Papua New Guinea, New Caledonia). 

However, three lower-middle income economies like India, Bangladesh and Sri Lanka scored higher than any other upper middle income economies. The three economies have made notable progress since 2020. India has shown efforts to improve ease of access of their power sector data by creating a one-stop shop in collaboration with the private sector (more in chapter 3). Bangladesh provides daily generation data for each power plant and provides various metrics for the electricity market, including outage and day-ahead schedules through the central utility. Sri Lanka’s power generation data is even more granular at 15 minute intervals by power plant (More information in respective economy profiles). 

High-income economies (South Korea, Australia and New Zealand) also stepped up in their efforts to improve data transparency. Both New Zealand and Australia scored “excellent” for their real-time, hourly generation data in addition to capacity, demand and other market data. South Korea began to provide estimates for off-grid solar, bumping up its score since 2020.

Scores on the Six Rating Criteria

Asian economies scored generally low on publishing lag, geographical granularity and time granularity criteria. Meanwhile, economies generally scored better on fuel breakdown and ease of access criteria.

1. Time granularity

Time granularity is an important aspect of data transparency for it allows deeper analysis of the power system. In particular, hourly data is required to mitigate intermittency of renewable sources by enabling various flexibility mechanisms, including demand response. Yet, only seven economies provided hourly data (Japan, Australia, Bangladesh, Singapore, New Zealand, Sri Lanka and Georgia), often with limited fuel breakdown. Half of the economies in Asia provide monthly or more time-granular data (ie. daily, hourly, 30 min interval, etc.). The most common time granularity of data was yearly (33%) and monthly (26%).

2. Publishing lag

Up-to-date data enables timely analysis of data, increasing relevance and responsiveness. However, 75% of economies do not make data available in a timely manner. Among them, fourteen economies had a time lag of more than a year. Only five economies made data available within a week (India, Australia, New Zealand, Bangladesh, Georgia). Economies that do not make up-to-date available also tend to score low on other rating criteria.

3. Geographic granularity

High geographical granularity enables system operators, analysts and researchers to understand where production is concentrated or lacking. This is crucial information in preventing grid congestion, generation forecasts and more. In particular, plant- or unit-level data are necessary to better understand changes in utilisation rates of different power plants and the overall operational status of the power system. Despite the importance, only 20% provided plant- or unit-level data. 45% of the economies made national data available. 23% provided regional generation data, some with limited data on marginalised regions.

4. Fuel breakdown

Calculating carbon intensity of the power sector requires information on the source of power generation. Generally, Asian economies scored better on this criterion. 40% of economies provided full fuel breakdown (coal, gas, hydro, nuclear, solar, wind, bioenergy, other renewables, other fossil) or more granular. 20% provided partial fuel breakdown, which requires users to estimate fuel breakdown based on arbitrary assumptions. Individual source research revealed that there are inconsistencies in the reporting depending on timescale. For example, Japan does not make thermal power generation breakdown available for hourly level data, although it is made available in their monthly reporting.

5. Ease of access

Making data easily accessible in a machine-readable format is a technical requirement for improving data transparency. Half of the economies made their dataset openly accessible in xlsx or csv files or through public API. However, one third of the economies still provided data in pdf format or even as a scanned image of a printed copy, which is difficult for users to process for their own use. There were other challenges that made the ease of access challenging. Same data points (ie. generation or capacity) are made available by ministries of respective regions or fuel. This leaves the task of data aggregation to users, adding an extra layer of challenge in accessing the data for the economy or the power sector as a whole.

6. Additional data

While this study focuses on generation data, other power sector data (capacity, demand, consumption by sector, price, etc.) are required to understand the power market as a whole. Economies that scored well on other criteria tended to score higher. India, South Korea, Taiwan, Australia, the Philippines and New Zealand provided a full suite of power market data.

Economy Profiles

This section provides detailed information on data sources of key Asian economies (China, India, Japan, South Korea, Indonesia, Australia, Viet Nam, Thailand, Malaysia, Pakistan, the Philippines, Bangladesh, Singapore, New Zealand, Myanmar, Sri Lanka). For others, please refer to the Appendix or the Asia Power Data Finder.

China
  • NBS, China’s National Bureau of Statistics, is the main source of generation data. They release data in the form of monthly press releases and through a data portal which can be tricky to find with the following characteristics:
    • Annual reports going back to 2000 (2012 for regional reports) and monthly reports from the past 36 months
    • Data broken down into these categories for both National and Regional datasets
      • thermal power, hydroelectric power, nuclear power, wind power, solar power
    • Data is not downloadable without a login
    • Yearly capacity data
  • China Power, China’s Electricity Development Power Association provides more granular capacity data as well as original reporting on provincial data including solar generation and solar cell production figures.
  • Other data sources:
    • CEC, The Chinese Electricity Council publishes monthly pdf reports with limited fuel breakdown, annual reports have good fuel breakdown.
    • NEA, The National Energy Administration publishes various press releases
India
  • NPP, The National Power Portal provides daily conventional fuel generation and capacity data broken down by “Thermal,” “Nuclear,” and “Hydro.” Data is given for different regions and as a sum for all of India. These are not the same as Indian states, and are instead aggregated to Northern, Western, Southern, Eastern, and North Eastern sections.
  • MERIT, Merit Order Despatch of Electricity for Rejuvenation of Income and Transparency provides daily plant-level data for most of India including a full regional breakdown. Fuel breakdown is still limited to Thermal, Gas, Nuclear, Hydro and Renewables. All data is deleted after one month (no archive is kept).
  • CEA, India’s Central Electricity Authority has monthly renewable energy generation with full breakdown. 
  • Other data sources:
    • MNRE, The Ministry of New & Renewable Energy, provides monthly RES capacity by state. Fuel breakdown varies and sometimes skips a month. They don’t archive their data either. 
    • GRID-India (formerly POSOCO), India’s National Grid Controller, has daily reports broken down by high level fuel type back to 2013. Has fully disaggregated capacity data but only renewable disaggregated energy data. Also has peak demand met, daily maximum demand met, and inter-regional trade.
Japan
  • REI, The Renewable Energy Institute is Japan’s main source of generation data. They publish their data sporadically with usually 2-3 months time lag through a downloadable dashboard with the following characteristics:
    • Hourly reports going back to 2016
    • Fuel breakdown including
      • Pumped Hydro (pump up and generate), Nuclear, Thermal, Hydro, Geothermal, Biomass, Wind, SolarPV.
    • They also publish hourly demand alongside generation data.
    • Data is broken down into regions from both Eastern (50 Hz) and Western (60 Hz) halves.
  • Monthly generation data with thermal breakdown and equal renewables breakdown is available from Enecho, Japan’s Agency for Natural Resources and Energy. There however are issues with a long time lag of three plus months.
  • OCCTO, The Organisation for Cross-regional Coordination of Transmission Operators, publishes quarterly reports sporadically and annual reports back to 2015. 
  • FEPC, The Federation of Electric Power Companies of Japan, provides historical generation and capacity data before 2013 back to 1963.
South Korea
  • KEPCO, The Korea Electric Power Corporation, is South Korea’s best source of energy data. It’s monthly reports are available through Excel and pdf reports, normally with only a few months time lag, through a portal with the following characteristics:
    • Monthly generation, capacity, and other market data going back to 2003
    • Generation data has plant-level data
    • Full fuel breakdown
    • Additional financial data
  • Other data sources include KPX Epsis, the Electric Power Statistics Information System. Peak demand and installed capacity is published on a daily basis in real time. They also provide annual generation on plant-level, and five-minute real time generation data without fuel breakdown.
  • KESIS also provides monthly energy statistics in pdf and csv files on the energy sector as a whole.
Indonesia
  • ESDM, The Ministry of Energy and Mineral Resources of the Republic of Indonesia is one of two public sources of generation data in Indonesia. Monthly generation data is collected, but not open to the public. ESDM publishes annual reports in pdf formats, often with a year or more time lag, with the following characteristics:
    • Annual generation going back to 2012
    • Full fuel breakdown, including geothermal
    • Capacity data with full fuel breakdown
    • Additional financial, demand, and consumption data
  • The other source includes the Central Bureau of Statistics, which again publishes annual data with a time lag over one year. The Bureau also uses xlsx formatting for their reports, with the following characteristics:
    • Annual generation going back to 1995
    • No fuel breakdown
    • Capacity data only with fuel breakdown
Australia
  • OpenNEM, an open platform for National Electricity Market Data is Australia’s best source of generation data despite omitting the Northern Territory. It operates in real time api with downloadable csv in real time with the following characteristics:
    • 5-minute real time generation for the last seven days. Daily data is available for the past year. Monthly, quarterly, and annual data goes back to 1998. 
    • Full fuel breakdown, including disaggregation between rooftop and utility solar, different gas power plants, and black and brown coal. Also included in the breakdown is generation from batteries discharging
    • Unit level generation data through a map dashboard when a fuel type is clicked
    • Capacity data at plant level is available through this same dashboard
    • Additional demand and financial data is also provided on the homepage
  • NTESMO, the Northern Territory Electricity System and Market Operator publishes 5-minute demand and price for Alice Springs, the second largest city, and daily demand and price data for the rest of the region. There is no fuel breakdown and no capacity information. 
  • Other data sources:
    • AEMO, The Australian Energy Market Operator publishes Quarterly Energy Dynamics, pdf reports detailing market dynamics and trends with attached xlsx files with data from graphs and tables. Has also niche datasets for things such as rooftop solar. 
    • Australia’s Department of Climate Change, Energy, the Environment and Water has data on electricity generation back to 1973. It also has full fuel breakdown for the regional (State) level. Capacity data is only available at national level and without fuel breakdown.
    • ABS, The Australian Bureau of Statistics also publishes a biannual report which holds data on estimates of energy assets, and physical/monetary supply and use, as well as key indicators. The next release is on 27/04/2023. Tables can be downloaded as xlsx or csv.
Viet Nam
  • GSO, The Viet Nam General Statistics Office has basic generation data in xlsx format with a time lag of 1-2 months. Data is held within the Socio-economic situation reports. 
    • Monthly generation data going back to 2014
    • No fuel breakdown
    • No regional breakdown (only national-level data)
    • No capacity data
  • Another source is from EVN, The Vietnam Electricity Group also provides monthly generation data in the form of press releases, although the time lag varies and is sometimes greater than one year:
    • The generation data within these press releases goes back to 2010
    • Full fuel breakdown
    • No regional breakdown (only national-level data)
    • Some capacity data
  • Other data sources:
    • MOIT, The Ministry of Industry and Trade of the Socialist Republic of Vietnam has press releases on energy which include various capacity and generation information. Also has daily basic updates on daily output by region. The record only goes back to 2021. Also has daily basic updates on daily output by region. Statistics are only available when the language is set to Vietnamese.
Thailand
  • EPPO, Thailand’s Ministry of Energy is the best source of generation data. It provides various statistical reports on energy in xlsx format with a couple months time lag with the following characteristics:
    • Monthly generation data going back to 1986 is available through the report “Power Generation by Type of Fuel”. There is no regional breakdown (only national-level data is available).
    • Full thermal breakdown is available, but apart from hydro, renewables data is aggregated.
    • Capacity data is only available as monthly for the year of 2023, otherwise this is given yearly with regional breakdown but no fuel breakdown
    • Additional demand, consumption, and import/export data is available.
  • Other data sources:
    • DEDE, The Department of Alternative Energy Development and Efficiency is only accessible in Thailand or via a VPN. It provides monthly energy statistics, including disaggregated renewable power generation data with a few months time lag. There is no regional breakdown.
    • EGAT, The Electricity Generating Authority of Thailand provides generation and capacity data for non-renewables from 1969-2021 at yearly level. These are only available on a specific webpage located in the site map at the bottom of the page. Monthly data, found through the ‘latest statistic’ is available, but without an archive.
Malaysia
  • Single Buyer Malaysia conducts electricity planning and manages electricity procurement services for Peninsula Malaysia. They provide Malaysia’s best open access generation data through an online downloadable dashboard updated in real time, but only cover Peninsula Malaysia. Occasional use of VPN to access Monthly Data is necessary depending on location. It has the following characteristics:
    • Hourly generation with week and month ahead forecasts back to 2018.
    • Full breakdown for Peninsula Malaysia (there are no Nuclear, or Wind power plants) including Coal, Gas, Hydro, Solar, and others.
    • No capacity data is available through this resource.
    • Additional real time demand data, financial data, dispatch scheduling model and 10 year outlook.
  • GSO, Peninsula Malaysia’s Grid System Operator has hourly real time generation data for Peninsula Malaysia, but it is not downloadable in csv/xlsx format. It has additional levels of breakdown to Single Buyer Malaysia, including co-gen, oil, and tie-line. It also has further breakdown of solar generation by region of peninsular Malaysia. GSO also possesses capacity data under ‘system data->power station information’ with plant-level capacity data by fuel type. Additional import/export data is also available.
  • MEIH, the Malaysia Energy Information Hub is Malaysia’s only public national source for national data, although it has a long time lag of over two years and only partial fuel breakdown (only thermal). Data is available in xlsx format back to 1990. Additional capacity, demand, and consumption by sector data is also available and they provide a list of all renewable energy projects in pdf format.
  • ST, the Suruhanjaya Tenaga Energy Commission also provides generation and capacity by plant and fuel types for all regions (Peninsular, Sarawak and Sabah) at yearly level with over two years of time lag. There is also no breakdown for renewables, and it is not accessible from abroad.
  • Finally, Sarawak Energy, Sarawak’s only utility company reports yearly generation data in their annual report in pdf format. There is also a lag here of roughly two years.
Pakistan
  • NEPRA, the National Electric Power Regulatory Authority of Pakistan publishes data from their state of industry report through an online non-downloadable portal. Despite having a near one year time lag, it is Pakistan’s best source of electricity data and has the following characteristics.  
    • Yearly generation going back to July 2017, although newer data is sometimes made available through their press release page.
    • Full plant-level breakdown including what fuel type the plant is.
    • Both installed and dependable capacity data as well as plant utilisation as a percentage with complete fuel breakdown.
    • Additional fuel price adjustment, economic merit order, and specific hydro generation data. 
  • Another data source is PBS, the Pakistan Bureau of Statistics which provides very limited energy data in tables from 2005-2015, and it no longer seems to be updated. There are annual reports covering energy generation from 2006 to 2021, but these have less structured information compared to NEPRA.
The Philippines
  • NGCP, the National Grid Controller of the Philippines is the best generation data provider through an online operations summary with downloadable data in xlsx. Datasets are up to date to December 2022, and have not yet been updated to include 2023. It has the following characteristics:
    • Monthly generation back to 2013
    • Complete fuel breakdown is given for three islands of the Philippines, Luzon, Visayas, and Mindanao. 
    • Some limited capacity data, system peak demand data, energy delivery per region and hourly demand, transmission assets, and system loss. 
  • Other data sources include:
    • IEMOP, the Independent Electricity Market Operator of the Philippines is the best source of capacity data, which provides this daily in a csv file. They also have fine minute demand data through a dashboard, which is not downloadable nor archived. Daily generation data is only provided at regional level, without any fuel breakdown. 
    • The Filipino Department of Energy which provided annual statistics back to 2003, although they have not been updated to 2021. They also have annual reports go back to 2016.
Bangladesh
  • PGCB, the Power Grid Company of Bangladesh is the only real provider of energy data in Bangladesh. They provide generation reports as spreadsheets through the reports section of their website published in real time with the following characteristics:
    • Daily generation data back to 2014 including forecast data for the next day. 
    • Data is disaggregated by plant and grouped by region, including Dhaka, Chattogram, Cumilla, Mymensingh, Sylhet, Khulna, Barisal, and Rajshahi. 
    • For each plant, its generation fuel type is given.
    • There is no capacity or price data, but there is demand data.
  • BERC, the Bangladesh Energy Regulatory Commission also provides data but only has annual reports as a scanned image back to 2012. These reports are only available in Bengali.
Singapore
  • EMC, The Singapore Energy Market Company is the best source of generation data for Singapore. They publish data through the ‘Market Trading Reports’ section of their website with roughly one week’s time lag with the following characteristics:
    • National generation data at 30-minute intervals back to 2013
    • No fuel breakdown apart from high-level in 30-minute data, but does provide solar generation and other shares of generation in monthly reports
    • Additional capacity by plant type, net imports, and price data
  • Other sources include the EMA, Singapore’s Energy Market Authority which provides capacity, demand, emissions, generation and other power market data at various levels of temporal granularity.
New Zealand
  • New Zealand’s Electricity Market Information, EMI, was one of the only sources to score a ‘5’. It publishes open access wholesale datasets each month in csv format, with less than a month’s time lag. It’s data has the following characteristics:
    • Unit-level, half-hourly generation data back to 2000.
    • Complete fuel breakdown including New Zealand’s geothermal energy sector. 
    • Additional capacity, demand and price data.
  • Transpower did better than EMI to publish regional, real-time hourly generation data but is hindered by the fact it is only a dashboard, and has no archive of data. It is also missing generation from biofuels. Em6, a paid version of the service, does have an archive.
  • Other sources include:
    • The Ministry of Business, Innovation and Employment, MBIE which provides monthly, national electricity generation and consumption with a six month time lag back to 1974.
Myanmar
  • Myanmar also had only one discoverable dataset. This was provided by the Central Statistical Organisation, CSO. It provides various electricity data through pdf reports called ‘Selected Monthly Economic Indicators’ which are available as pdf’s with a time lag of 6-8 months. These are available in both English and Burmese with the following characteristics:
    • Monthly, national generation data going back to 2013 
    • Complete fuel breakdown including thermal, diesel, hydro, gas, and solar.
    • Additional capacity, demand, and consumption by sector data.
Sri Lanka
  • The Public Utilities Commission of Sri Lanka, PUCSL, was the only discoverable dataset for Sri Lanka. Through a general information page, they have dispatch and generation reports in pdf format with only a few days time lag. They have the follow characteristics:
    • 15-minute plant-level generation data with an archive back to May, 2020.
    • Strong level of fuel breakdown, including coal and other fossil fuels, as well as hydro, wind, solar, and biofuels.
    • There is no additional data, including no capacity data.

Chapter 3 | Pioneers of data transparency

Economies demonstrating good practice

This section sets out to understand open data practices in economies that had excellent scores. New Zealand, Australia, and India had excellent scores on data transparency, with many others economies scoring excellent on one or two criteria. There are lessons to be learned from them, so to inspire what good data transparency practice can look like for such a diverse region.

1. Time granularity

Ideally, power data needs to be published at a minimum of hourly intervals for an optimal power market operation and analysis. Every national power grid likely collects some level of data for balancing operations. This data must be open to the public to enable effective decarbonisation. New Zealand, Australia, Japan, Bangladesh, Singapore, Sri Lanka, and Georgia are sharing data at hourly or less intervals. Both Australia and New Zealand went beyond this standard, publishing data through a public dashboard every five minutes, featuring real-time data sharing. This level of granularity allows each economy and its citizens to give more granular price signals for better generation and use of energy, increased ability for balancing of supply and demand through demand response technologies, as well as improved trading incentives. This requires a fast and at times expensive data network, especially for larger nations, and may be difficult to implement for lower-income economies.

2. Publishing lag

Balancing operations and accurate forecasts becomes harder as data is made available weeks, months, and years past the time of generation. In particular, multiple years of time lag diminishes the value of data significantly. India, Australia, and New Zealand all scored excellent by publishing data in real time or the next day. To do this, data administrators need to keep databases updated at all times, requiring constant maintenance and vetting.

3. Geographical granularity

Over half of Asian economies published data nationally, which obscures information on where generation is occurring. Geographical granularity is especially important in large nations like India, where power may travel hundreds of kilometres to its point of consumption. Ideal data transparency goes beyond the regional level, identifying which units inside which power plants are generating electricity. India, Australia and New Zealand all published data at the unit-level.

4. Fuel breakdown

Estimating carbon intensity of power and understanding how the electricity mix needs to change requires detailed information on the fuel source of generation. Only Australia, New Zealand, South Korea, and Taiwan provided detailed fuel breakdown, including off-grid generation. India provided information on all fuels, but reported off-grid generation inconsistently. Australia consistently reported off-grid solar generation, including other fuel breakdowns for gas (Open Cycle Gas Turbine and Combined Cycle Gas Turbine) and coal (black and brown). New Zealand, which has a comparatively higher percentage of active geothermal power plants, also gives an additional breakdown for geothermal energy.

5. Ease of access

The ideal format of data is free, publicly open data that does not restrict overseas access.  Data must ideally be accessible via public API for developers to retrieve data programmatically. For non-developers, data must be shared in a machine-readable format, such as xlsx or csv. Australia and New Zealand were the only economies to have functioning public API’s. India has public APIs for some datasets, but not all. Many economies struggle to make data available for many reasons. For one, the government or public utilities may want to retain data to prevent public scrutiny or various forms of intervention. Some economies lack technical and/or financial capacity, and political will in the public sector to enable good data governance.

6. Additional Data

While this study focuses on generation data, other power sector data is desirable. Capacity, demand, wholesale price, tariffs and consumption by sector data are some of the key additional power sector data. India, Australia, New Zealand, and South Korea, Taiwan, and the Philippines are some of the example economies that made every category of additional data mentioned available.

[Case study] Indian government’s partnership with Vasudha Foundation

This section highlights success stories from the region to serve as inspiration for peer economies. In particular, India is an exceptional case, as it has made significant strides in making its vast power sector data available in one place in partnership with the private sector. 

NITI Aayog, a policy think tank of the government of India, has partnered with the Vasudha Foundation, which has become a leading aggregator of energy data across India’s 28 states and eight union territories. Through a cross-sectoral and data-driven analysis as well as creation of comprehensive data repositories, Vasudha Foundation is aiming to create a one-stop open access data platform to present a holistic picture for India’s efforts on climate change and a low carbon pathway.

Public-private Partnership between NITI Aayog and the Vasudha Foundation
1. Key stakeholders

Vasudha Foundation is a non-profit energy and climate policy think tank that tracks and supports India’s clean energy transition through data driven analysis. Vasudha Foundation has developed a comprehensive power sector value chain data comprising electricity generation, transmission, distribution and consumption. Vasudha Foundation has been collecting India’s power plant data to answer the questions of power plant locations, pipeline and retired capacities, what fuels they use, and how often outages happen. Following on from their initial launch in 2021 of Vasudha Power Info Hub, the Vasudha Foundation decided to merge their power data dashboard with the India Energy Dashboard (IED 2.0) developed by NITI Aayog in 2021; a government agency whose mandate is to oversee the adoption and monitoring of Sustainable Development Goals in India. NITI Aayog also shares the key initiative to enable easy and integrated data access for energy and climate. The India Climate and Energy Dashboard (ICED) is a user-friendly platform that aims to bring together comprehensive historical and current data to provide a single window access for all datasets related to climate and energy. It is planned to be launched in the first quarter of the financial year 2024 (https://iced.niti.gov.in/).

2. Solving India’s Power Data Problem

Creating a comprehensive power sector dataset for all of India imposed many challenges. To build the dashboard seen today, the Vasudha team had to collect data from multiple reports and portals, including 400-500 tariff orders and company reports that were scattered across over 68 distribution companies. The reports were only available in pdf format or scanned images, thus making it difficult for the workflows to be automated. However, a public API for automated workflows is in development despite this hurdle. 

Vasudha Foundation has developed close relationships with distribution companies and government agencies to gain access to the data used in their dashboard. They have established a high reputation in the power sector, which led to high trust with key data providers. All the power sector data on the portal is from government sources. Their partnership with NITI Aayog has been instrumental for furthering opportunities to develop connections with other ministries and departments to access granular data. Vasudha therefore sees this building of relationships in the energy sector as key to achieving a successful data tool, an insight echoed by Subak’s Power Plants Data Research Initiative. 

Through this government partnership, the Vasudha Foundation is seen as authentic and reliable, as their data is consistently backed up by official sources.  The Vasudha team remarked that the dashboard now provides a ‘birds-eye’ view for all stakeholders involved in India’s energy transition. Through the creation of this one-stop shop for India’s power data, the Vasudha Foundation has successfully addressed India’s challenges in power data transparency. While it is open for anyone, researchers, campaigners, financial institutions and policymakers have hugely benefited from the tool, as they can easily understand the status quo of India’s journey in clean electricity transition.

3. Technical Data Challenges

Common technical challenges that the Vasudha team faced were different granularities across sources, inconsistent data formats, inconsistent time lag, data accessibility, disagreements in data sources and frequent occurrence of missing or erroneous values. The Vasudha team cross-verified data from multiple sources and identified gaps in datasets.The gaps in data occurred due to reporting errors and mismatch of methodology for the same metric. It was noted that standardisation of metrics, data methodology, granularity and nomenclatures across various distribution companies and government agencies is required to resolve this issue.

4. The way forward

The Vasudha Foundation has plans to collect data on energy use and consumer behaviour. The Vasudha team explained that building demand-side data is a top priority going forward. Without granular demand-side data, it will be difficult to augment the grid according to users’ needs and maintain grid stability as intermittent renewables increase. The urgency of this cannot be overstated, as the energy transition is happening fast not only in India but across Asia.

Recommendations

Reflecting on the case study, we propose a few low-barrier recommendations from the Open Data Policy Guidelines published by the Open Data Policy Hub (ODPH) as a solution to achieve better data transparency in Asia. We have selected recommendations that can be implemented without a large technical overhead, so that they can be implemented without financial or technical hurdles.

ODPH suggests that 1) data formats should allow for maximal technical access; and 2) providers should publish data in comprehensive and appropriate formats for varied uses. This need is apparent in the Vasudha Foundation’s reporting that pdf or scanned image formats were a barrier to collecting data because they are not machine-readable. Pdf and html documents may be fit for the purpose of delivering analytic content, but are not suitable for sharing data. Therefore, original data tables should be attached in machine-readable formats, including xlsx, csv or json. The Australian Energy Market Operator (AEMO) demonstrates good practice. It provides their quarterly energy dynamics report as a pdf but has an xlsx workbook to support this which contains all the data in the report.

Additional ease of access recommendations include 3) removing restrictions for accessing information. Various sources identified had access limitations such as geography or sector. The goal of open data should be to provide non-discriminatory, free access at any time without restriction or justification for doing so. This also extends to the licence, as data should be 4) explicitly licence free or adopt an open data licence. This maximises the usability of the dataset, and allows multiple users to build off of the data such as anyone using Vasudha’s data. Once made open, the data provider should ensure, 5) permanent, lasting access to the data.

Lack of unique identifiers was one of the main challenges that the Vasudha team faced, as well as the 13 interviewees of Subak’s power plants data research initiative. Unique identifiers provide consistencies when collecting and reporting data for individual power plants and units. The 6) use of unique identifiers is essential for the interoperability of energy data and one of the most pressing challenges for the energy data landscape. Additionally, where established, these identifiers should be non-proprietary and public.

7) Central locations devoted to data publication and policies are essential. Some economies have put efforts to create a one-stop-shop, including India, Australia and New Zealand. However, in many economies, data providers for power are scattered. 8) Digitisation and distribution of archival materials is also essential for these central locations. In many economies, historical power data may need to be digitised. Often, old historical data are archived separately, which makes historical analysis difficult. Azerbaijan demonstrated good practice, whose State Statistical Committee published data from 1913. This further illustrates the need to 9) publish bulk data to enable easy access to all information.

Conclusion and Further Recommendations

Power sector decarbonisation depends on data transparency

Data transparency is limited in Asia, holding back the speed at which Asian economies can decarbonise their power sector.

Asia is a key region for global decarbonisation, as it is home to 80% of the global coal consumption. Open and easily accessible power sector data is necessary to enable Asian economies to decarbonise fast, as data enables tracking and monitoring of clean power targets, evidence-based policymaking and grid optimisation for better flexibility.

This research identified, compiled and evaluated official power sector data sources for 39 Asian economies using power generation as the baseline metric for assessment. The assessments were made using six rating criteria: publishing lag, geographical granularity, fuel breakdown, temporal granularity, additional data and ease of access.

The research revealed that more than half (24 out of 39) of Asian economies have insufficient or poor data transparency. This means that there is little to no data on whether electricity needs of 684 million people are being met with clean or fossil fuels. Nine economies have acceptable scores, and six economies have scored “good” or “excellent”. Higher income economies generally scored higher than lower income ones. However, some lower-middle income economies like India, Bangladesh and Sri Lanka scored higher than other upper middle income economies, leading the way to better data transparency in Asia. 

Ideally, power sector data must be provided at hourly or less intervals, with granular geographical and source fuel information. In addition, the data must be made available as soon as after it is generated and made free to access without restriction in a machine-readable format. Both Australia, New Zealand, India and several other economies are demonstrating good practice across the six rating criteria. 

This report includes a special case study on India, which has made a significant effort to streamline their open data practice by collaborating with Vasudha Foundation, a non-profit organisation. Their partnership aimed to streamline their data sharing practice by creating a one-stop-shop dashboard revealed that standardisation of metrics, data format, methodology, granularity and nomenclatures is necessary to improve data transparency. Reflecting on the case study and guidelines from the Open Data Policy Hub, the study gives nine practical recommendations for improving data transparency, including:

  1. Removing all forms of restrictions on access and making data licence-free;
  2. Use of comprehensive and appropriate formats for data sharing for varied uses;
  3. Making data available in one central location in machine-readable formats;
  4. Digitising archival materials and publishing them as bulk data that has permanent, lasting access;
  5. Using unique identifiers to avoid erroneous interpretation.

National governments have an enormous amount of benefits to gain from making power sector data open and easily accessible, as it is key to accelerating power sector decarbonisation in each economy. Data can engage a broader network of stakeholders to create innovative solutions for coal-to-clean electricity transition. Work is required to enhance data governance and technical capacity to make this happen.   

Findings in this report are made publicly available on Subak and Ember’s website for anyone to use to improve visibility on where power data is in Asia.

Supporting Materials - Appendix

Economy Profiles

Taiwan
  • Taipower, The Taiwan Power Company publishes generation data in csv format with only a month’s delay. Note that Taiwan’s encoding is different, meaning that the translated file is often unreadable. It has the following characteristics:
    • Monthly reports going black to 2010
    • No thermal breakdown, but certain renewables are covered
      • Hydro, Wind, Solar (including off grid)
    • Generation data is plant-level, but it does not specify plant fuels
    • No capacity data
    • Additional demand and consumption data
  • Energy Statistics Zone, the Energy Statistics Information System, provides additional monthly generation and capacity data by fuel, detailed consumption by sector, and electricity tariff and demand data. This data is back to 1982, but has a 2-3 months delay. They also provide the breakdown given by Taipower corporation plus a full fuel breakdown, including private power plants and self-use generation equipment.
Kazakhstan
  • Samruk Energy manages Kazakhstan’s state electricity assets and is the best source of generation data. This is published online in word documents only in Kazakh, and has roughly one month’s time lag. The source has the following characteristics:
    • Monthly generation data back to 2017
    • Thermal fuel breakdown is aggregated, while fuel split between hydro, wind, solar, and biofuel is given.
    • Regional breakdown is given between East Kazakhstan, Karaganda, Akmola, North Kazakhstan, Kostanay, and Pavlodar.
    • There is no additional capacity, demand, or price data, but consumption by sector data does exist.
  • Other sources include:
    • NBS, The Kazakh National Bureau of Statistics which provides total generation, consumption by sector and region, and import/export data at a yearly level with two or more years of time lag. There is no fuel breakdown, however there is data on the share of renewable sources at annual level broken down by type of renewable.
    • There is also the Kazakhstan Open Data Portal, which has many datasets on energy but most are static (not updated).
Uzbekistan

Uzbekistan has two main sources of generation data which both have various caveats.

  • The Uzbekistan Open Data Portal is the most up to date source with one year’s publishing lag. It provides data on power mix (%), consumption per capita, and access to electricity among other niche datasets. However, data is reported inconsistently and there is no data on absolute generation. Its data is available as a csv or json, in English, Russian, or Uzbek with the following characteristics:
    • Yearly generation back to 2019 at only National Level.
    • No fuel breakdown for thermal fuels, whereas only wind and solar figures are given. 
    • Some additional demand and price data.
  • The Uzbekistan Statistics Agency also reports generation data in its industry reports under ‘Energy Use’, although with a time lag of nearly three years. Its data is available as a xlsx, csv or json and in English, Russian, or Uzbek. It has the following characteristics:
    • Yearly generation back to 2000, this includes some level of provincial data from regions Andijan, Bukhara, Jizzakh, Kashkadarya, Navoi, Namangan, Samarkand, Surkhandarya, Syrdarya, Tashkent, Fergana, Khorezm and Tashkent city
    • There is no fuel breakdown for non-renewable sources, although wind and solar data is given.
    • Capacity data is provided for thermal and hydro, but there is no capacity data for renewable energy sources. 
    • Finally, some additional data on electricity demand, consumption by sector and by capita is given.
Hong Kong
  • Hong Kong Electric is Hong Kong’s best source of generation data back to 2021, with only a two month lag. They have an open data portal with both aggregated consumption data and renewable energy generation data with the following the characteristics:
    • Regional generation data at either one month, or six month intervals in the case of fuel breakdown.
    • Partial breakdown, only covering coal, gas, wind and solar generation. These figures are only provided biannually.
  • Other sources include:
    • There is also a Hong Kong Open Data portal, which provides quarterly consumption by sector, and exports with downloadable data. Is the best provider of installed capacity back to 2013.
    • The Census and Statistics Department, Censtatd, provides monthly reports with domestic production without fuel breakdown, consumption by sector and exports which are all in machine readable formats. Data goes back further than HK Electric and the Hong Kong Open Data Portal to 1979.
Azerbaijan
  • Azerbaijan had only one discoverable dataset, provided by the State Statistical Committee of the Republic Azerbaijan, SSCRA. It provides multiple separate datasets underneath the ‘Energy’ category. In particular these are ‘production of electricity’, and ‘plant capacity’. Both are provided as xlsx files with following characteristics:
    • National, yearly generation data back to 1913.
    • Only high level thermal breakdown, and hydro, wind, solar, and biofuel generation.
    • Additional capacity, demand, consumption by sector and net imports data.

Supporting Material

Downloads

Acknowledgements

Special thanks

Special thanks to Vasudha Foundation’s Senior Advisor Vrinda Gupta, Senior Manager Raghav Pachouri, and Manager Rahul Patidar who participated in an interview with Ember and Subak.

Contributors

Ye Yuan, Will Andrews Tipper, Aditya Lolla, Sam Hawkins, Jake Verma, Rini Sucahyo

Header image

Power lines against a city road in the Philippines

Credit: Davidovich Mikhail / Alamy Stock Photo

Media Coverage

Social Media

Insights

Related insights
See all insights
image Research

Asian Electricity Data Transparency

Where to find electricity data for every country in Asia, with an assessment of the quality of the available data.

  • Asia
  • Wind
  • Solar
  • Coal

26 November 2020

image Research

The sunny side of Asia

Solar generation helped avoid at least US$34 billion in seven Asian countries in the first half of 2022

  • Asia
  • China
  • Japan
  • India
  • South Korea
  • The Philippines
  • Thailand
  • Viet Nam
  • Indonesia
  • G20
  • Solar

10 November 2022

image Research

Unleashing solar and wind in ASEAN

ASEAN yet to unleash solar and wind to keep up with increasing demand and prevent power sector CO2 emissions rising further.

  • Asia
  • Indonesia
  • Malaysia
  • Thailand
  • The Philippines
  • Viet Nam
  • Solar
  • Wind
  • Coal

7 July 2022