Frictionless Energy Data (friendly_data)

Frictionless energy data, or friendly_data, is two things:

1. an energy modelling friendly specification to combine data and metadata into a “data package” and

2. a tool to help create, modify and read such data packages.

Its design goal is a common medium to facilitate the flow of data between energy and environmental models in a way that can be automated, while able to deal with the fact that different models use different internal data formats, unit conventions, or variable naming schemes.

A typical data package consists of a collection of datasets, and related metadata. There are two kinds of metadata: a) package wide semantic information providing context, terms of use, etc, and b) structural information about the included datasets, or schema.

Friendly data packages

A friendly_data data package is based on, and is compatible with the frictionless data package specification. The friendly_data tool adds the following features on top of the basic frictionless specification:

• Aliases: you can specify column aliases to indicate two different column names are equivalent. This reduces friction due to varying terminology used by different groups/sub-communities.

The friendly_data tool

• Basic use of the friendly_data tool, which requires no Python experience, makes it easy to create and manage data packages.

• An online metadata registry allows teams to share and agree on variable names and definitions and makes the generation of metadata for a data package quick and easy.

• Automated conversion to the IAMC timeseries scenario data format.

• Written in Python, with a library API:

  • thus linking directly to the rich Python ecosystem of data analysis and visualisation tools,

  • programmers can also make use of the API to further automate tasks related to creating, validating, and using friendly_data packages.

Contents

• How to install?
• Data package
• Data registry
  • Value columns - cols
  • Index columns - idxcols
  • The Package Index file
  • Contributing to the registry
  • Defining a custom registry
• The friendly_data tool
• The configuration file
• Hands-on tutorial
  • Understanding the registry used by Friendly data
  • Learning by example
  • Metadata
• Converting to IAMC format
• Advanced use
  • API documentation
  • External Packages
  • Indices and tables
• Notes for developers
  • Adding dependencies
  • Working with the data package API
  • Working with the index file
  • Working with the registry API
  • Converters: interfacing with the Python ecosystem

License

All software and related documentation in this project is licensed under the Apache License, Version 2.0 (the “License”). You may obtain a copy of the License at: http://www.apache.org/licenses/LICENSE-2.0