In the first part of this article we discovered the growing role data is playing in gas and power markets. In particular, meter values and forecast data, as well as small-scale prosumer production of power, are vital to essential business processes in liberalized gas and power sectors, such as market-based balancing. In addition, third-party access to data – especially consumption data – is a cornerstone of EU market liberalization, enabling consumer decision-making and facilitating supplier switching as well as ensuring fair and equal infrastructure access to market participants. And all of this data is in the hands of DSOs; data management has become an essential part of their role in a data-driven energy market. Will DSOs embrace this new role, or will they maintain their focus on physical asset management, allowing an independent entity such as a data hub to manage data assets? In part two of this article we examine these questions and explore the various data exchange models different countries have developed, to hopefully identify the best model for a truly data-driven energy market.
How is a data exchange model described?
EU regulations place responsibility for the implementation of data communications in the hands of member states. As a result, a number of different approaches have evolved. However, all of them are defined through the same five layers:
- Business layer: Legislative requirements for the electricity/gas market (business logic)
- Functional layer: Business processes with defined use cases (data content)
- Information layer: Data formats (e.g. CIM or EDIFACT for power, Edig@s for gas)
- Communication layers: Data exchange protocols and channels (e.g. SOAP, HTTPS)
- Network layer: Physical and virtual infrastructures (e.g. Internet, virtual private network)
The five layers of data exchange. Source: European Network of Transmission System Operators for Gas (ENTSOG), “Setting Up an AS4 System”
Data exchange is based on several layers of interoperability defining what is communicated and how it is communicated. The what is defined by the business and functional layers, including legislative requirements and implementation guides that specify the business processes in detail. The how is defined by data formats (information layer), such as industry-specific messaging standards (Edig@s for the gas market, CIM/EDIFACT for the electricity market), the data exchange protocols and channels (communication layer), such as HTTPS, REST/SOAP API webservice or the AS4 protocol (for the gas market), as well as the physical and virtual communication infrastructures (network layers), such as the Internet, a virtual private network (VPN) or a dedicated line. We will take a high-level overview of data exchange without going into detail on the different layers.
The broad requirements set in primary and secondary legislation need to be specified as business processes and their corresponding use cases. In implementing the legislative requirements, particular attention is paid to business processes, such as balancing, supplier change and billing. The clear definition of market roles is crucial for the design of coherent business processes, thus the adoption of the Harmonised Role Models for the gas and electricity markets is essential.
What types of models exist?
Three main approaches to data exchange can be seen on European power and gas markets:
- Traditional decentralized (bilateral) data exchange
- Centralized DSO data access platform
- Data hub
There are some good examples of standardization of traditional bilateral data exchange, but the growing tendency is to centralize data exchange onto platform-based solutions.
| Best practices of data exchange | Standardized bilateral data exchange | Central DSO data access platform | Data Hub |
|---|---|---|---|
| Example | Germany (edi@energy), Austria (EDA/ebutilities) | Belgium (CMS), the Netherlands (EDSN) | Estonia (Elering Gas Data Hub), Finland (GasGrid Gas Data Hub), Denmark (Energinet DataHub) |
| Owner of data exchange solution | Association of energy companies | DSO association | TSO or third party |
| Data storage | Mainly DSO or Meter Operator | DSO | Data hub |
| Market communication | Decentralized data exchange between market participants | Centralized data exchange with shared platform | Centralized data exchange with data hub |
| Data format | Standardized | Standardized | Standardized |
| Access protocol | N/A | Webpage Webservice | Webpage Webservice |
| Business processes | Standardization only optional | Partly built-in standardization | Built-in standardization |
| Driver | Voluntary (with regulatory support/facilitation) | Voluntary (with regulatory support/facilitation) | Regulatory requirement (with industry cooperation) |
Standardized bilateral exchange
Decentralized data exchange has become more standardized, especially in German-speaking countries. Decentralized data exchange is based on bilateral data exchange between market participants. Traders, for example, have to communicate separately with each DSO in whose area they are active. The standardization therefore aims to simplify the communication with various companies and their respective IT systems. In both Germany and Austria, the federal legal system and the heterogenous DSO landscape is a barrier to the establishment of central platforms.
But even in these countries which have the best examples of decentralized communication, they are struggling with the increasing amount of data. In Germany, for example, recent changes put meter data management in the hands of Meter Operators, instead of the DSOs, with all communication centered on the actual (smart) meter data in a star-shaped communication model.
Central DSO data access platforms
Central DSO data access platforms are based on voluntary cooperation of DSOs to establish a platform for the entire national power and gas market, providing access to consumption data and designing business processes together. The first central DSO data access platforms were established in the Netherlands and later in Belgium, driven by cost reduction for DSOs through the shared platform, efficiency improvements in market messaging, and compliance with regulatory requirements for consumer and other third-party data access.
The Belgian Central Market Information System (CMS), operated by Atrias, aims to become the best practice model for a central data access platform. The CMS provides a unified platform for suppliers, DSOs, TSO, shippers and balancing group managers (BGM). In the Netherlands, the Central Data Hub agency (EDSN – Energie Data Services Nederland) can be best described as a hybrid between a DSO data access platform and a Data Hub. While EDSN was established to provide unified access to the data DSOs each store in their own way, the organization is in the process of setting up a central system for core business processes, such as measurement data, allocation and reconciliation.
Data hub
A data hub is a central platform containing consumption and other data from the entire gas market with access to all market participants. In most countries, the data hubs are owned by the TSOs, but there are third-party data hubs as well. The data hub model uses automated and uniform communication methods for standardized business processes to exchange energy market data. All market participants are provided with easy access on an equal basis.
The Energinet DataHub in Denmark, launched in 2013 and upgraded in 2016 to support the supplier-centric market model, is widely seen as the benchmark for data hubs. It offers central data communication together with standardized market processes. The key feature is the easy access to market data – for consumers, for market parties and for third parties. Currently, data hubs are popping up in all Nordic countries (Denmark, Sweden, Norway, Finland) with the aim for increased interoperability.
The first gas data hub was established in Estonia by the TSO Elering in 2016, as an extension of the power data hub established by the TSO four years earlier. It supports three main processes: supplier switching, submitting metering point data and hourly/daily/monthly meter data, and encoding. In addition, it includes contract management (e.g., supply agreements, portfolio agreement with balance group managers), third-party authorization by customer with data access provision, and even client mediation request between a DSO and a supplier.
Which model will rise to the top?
There is an indisputable tendency in EU countries towards the centralization of data exchange. Where once all EU countries relied on decentralized data communication between market participants, even this has become standardized. Now prominent energy markets in Western European and Nordic countries have established central platforms, quickly becoming the best practices for data exchange.
Which of the two centralized models becomes dominant depends on how the role of the DSO evolves. Will DSOs welcome the growing importance of data in the future of energy markets, and choose to adapt their business models to place as much value on the data management in their hands as on the physical infrastructure? Then the DSO data access platform will likely become the standard model. Or will DSOs decide to concentrate on their traditional competency of managing the physical infrastructure, leaving data management to a third party? In this case data hubs will come out on top.
In either case, there are already successful examples of both models in operation today. Whatever the future of data exchange has in store and whatever additional use cases may develop for this data, it’s clear that centralized data exchange platforms are here to stay, to satisfy the growing hunger for data and enable an ever more data-driven energy market.
This article is written by Márton Fabók, Senior Energy Market & Policy Researcher, Navitasoft.
