For convenience, documentation for selected sections of the frictionless API has been included below. For the upstream documentation, see Package and Resource.

Package

class frictionless.package.Package(source=None, *, descriptor=None, resources=None, id=None, name=None, title=None, description=None, licenses=None, sources=None, profile=None, homepage=None, version=None, contributors=None, keywords=None, image=None, created=None, innerpath='datapackage.json', basepath='', detector=None, onerror='ignore', trusted=False, hashing=None)[source]

Package representation

API | Usage ——– | ——– Public | from frictionless import Package

This class is one of the cornerstones of of Frictionless framework. It manages underlaying resource and provides an ability to describe a package.

```python package = Package(resources=[Resource(path=”data/table.csv”)]) package.get_resoure(‘table’).read_rows() == [

{‘id’: 1, ‘name’: ‘english’}, {‘id’: 2, ‘name’: ‘中国人’},

Parameters:

source (any): Source of the package; can be in various forms.
Usually, it’s a package descriptor in a form of dict or path Also, it can be a glob pattern or a resource path

descriptor (dict|str): A resource descriptor provided explicitly.
Keyword arguments will patch this descriptor if provided.

resources? (dict|Resource[]): A list of resource descriptors.
It can be dicts or Resource instances.

id? (str): A property reserved for globally unique identifiers.
Examples of identifiers that are unique include UUIDs and DOIs.

name? (str): A short url-usable (and preferably human-readable) name.
This MUST be lower-case and contain only alphanumeric characters along with “.”, “_” or “-” characters.

title? (str): A Package title according to the specs
It should a human-oriented title of the resource.

description? (str): A Package description according to the specs
It should a human-oriented description of the resource.

licenses? (dict[]): The license(s) under which the package is provided.
If omitted it’s considered the same as the package’s licenses.

sources? (dict[]): The raw sources for this data package.
It MUST be an array of Source objects. Each Source object MUST have a title and MAY have path and/or email properties.

profile? (str): A string identifying the profile of this descriptor.
For example, fiscal-data-package.

homepage? (str): A URL for the home on the web that is related to this package.
For example, github repository or ckan dataset address.

version? (str): A version string identifying the version of the package.
It should conform to the Semantic Versioning requirements and should follow the Data Package Version pattern.

contributors? (dict[]): The people or organizations who contributed to this package.
It MUST be an array. Each entry is a Contributor and MUST be an object. A Contributor MUST have a title property and MAY contain path, email, role and organization properties.

keywords? (str[]): An Array of string keywords to assist users searching.
For example, [‘data’, ‘fiscal’]

image? (str): An image to use for this data package.
For example, when showing the package in a listing.

created? (str): The datetime on which this was created.
The datetime must conform to the string formats for RFC3339 datetime,

innerpath? (str): A ZIP datapackage descriptor inner path.
Path to the package descriptor inside the ZIP datapackage. Example: some/folder/datapackage.yaml Default: datapackage.json

basepath? (str): A basepath of the resource
The fullpath of the resource is joined basepath and /path`

detector? (Detector): File/table detector.
For more information, please check the Detector documentation.

onerror? (ignore|warn|raise): Behaviour if there is an error.
It defaults to ‘ignore’. The default mode will ignore all errors on resource level and they should be handled by the user being available in Header and Row objects.

trusted? (bool): Don’t raise an exception on unsafe paths.
A path provided as a part of the descriptor considered unsafe if there are path traversing or the path is absolute. A path provided as source or path is alway trusted.

hashing? (str): a hashing algorithm for resources
It defaults to ‘md5’.

Raises:: FrictionlessException: raise any error that occurs during the process

Attributes

basepath: Returns:
description_html: Returns:
hashing: Returns:
metadata_errors: Returns:
metadata_valid: Returns:
onerror: Returns:
resource_names: Returns:
trusted: Returns:

Methods

`add_resource`([source])	Add new resource to the package.
`clear`()
`copy`()
`expand`()	Expand metadata
`from_bigquery`(source, *[, dialect])	Import package from Bigquery
`from_ckan`(source, *[, dialect])	Import package from CKAN
`from_sql`(source, *[, dialect])	Import package from SQL
`from_zip`(path, **options)	Create a package from ZIP
`fromkeys`(iterable[, value])	Create a new dictionary with keys from iterable and values set to value.
`get`(key[, default])	Return the value for key if key is in the dictionary, else default.
`get_resource`(name)	Get resource by name.
`has_resource`(name)	Check if a resource is present
`infer`(*[, stats])	Infer package's attributes
`items`()
`keys`()
`metadata_attach`(name, value)	Helper method for attaching a value to the metadata
`metadata_extract`(descriptor)	Helper method called during the metadata extraction
`metadata_process`()	Helper method called on any metadata change
`metadata_validate`()	Helper method called on any metadata change
`pop`(k[,d])	If key is not found, default is returned if given, otherwise KeyError is raised
`popitem`(args, *kwargs)	Remove and return a (key, value) pair as a 2-tuple.
`property`([func, cache, reset, write])	Create a metadata property
`remove_resource`(name)	Remove resource by name.
`setdefault`(args, *kwargs)	Insert key with a value of default if key is not in the dictionary.
`setinitial`(key, value)	Set an initial item in a subclass' constructor
`to_bigquery`(target, *[, dialect])	Export package to Bigquery
`to_ckan`(target, *[, dialect])	Export package to CKAN
`to_copy`()	Create a copy of the package
`to_dict`()	Convert metadata to a plain dict
`to_json`([path, encoder_class])	Save metadata as a json
`to_sql`(target, *[, dialect])	Export package to SQL
`to_yaml`([path])	Save metadata as a yaml
`to_zip`(path, *[, encoder_class])	Save package to a zip
`update`([E, ]**F)	If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]
`values`()

metadata_Error

add_resource(source=None, **options)[source]

Add new resource to the package.

Parameters:: source (dict|str): a data source **options (dict): options of the Resource class
Returns:: Resource/None: added Resource instance or None if not added

property basepath

Returns:: str: package basepath

contributors

Returns:: dict[]: package contributors

created

Returns:: str: package created

description

Returns:: str: package description

property description_html

Returns:: str: package description

description_text

Returns:: str: package description

expand()[source]

Expand metadata

It will add default values to the package.

static from_bigquery(source, *, dialect=None)[source]

Import package from Bigquery

Parameters:: source (string): BigQuery Service object dialect (dict): BigQuery dialect
Returns:: Package: package

static from_ckan(source, *, dialect=None)[source]

Import package from CKAN

Parameters:: source (string): CKAN instance url e.g. “https://demo.ckan.org” dialect (dict): CKAN dialect
Returns:: Package: package

static from_sql(source, *, dialect=None)[source]

Import package from SQL

Parameters:: source (any): SQL connection string of engine dialect (dict): SQL dialect
Returns:: Package: package

static from_zip(path, **options)[source]

Create a package from ZIP

Parameters:: path(str): file path **options(dict): resouce options

get_resource(name)[source]

Get resource by name.

Parameters:: name (str): resource name
Raises:: FrictionlessException: if resource is not found
Returns:: Resource/None: Resource instance or None if not found

has_resource(name)[source]

Check if a resource is present

Parameters:: name (str): schema resource name
Returns:: bool: whether there is the resource

property hashing

Returns:: str: package hashing

homepage

Returns:: str: package homepage

id

Returns:: str: package id

image

Returns:: str: package image

infer(*, stats=False)[source]

Infer package’s attributes

Parameters:: stats? (bool): stream files completely and infer stats

keywords

Returns:: str[]: package keywords

licenses

Returns:: dict[]: package licenses

metadata_Error: alias of frictionless.errors.general.PackageError

metadata_process()[source]: Helper method called on any metadata change

metadata_validate()[source]

Helper method called on any metadata change

Parameters:: profile (dict): a profile to validate against of

name

Returns:: str: package name

property onerror

Returns:: ignore|warn|raise: on error bahaviour

profile

Returns:: str: package profile

remove_resource(name)[source]

Remove resource by name.

Parameters:: name (str): resource name
Raises:: FrictionlessException: if resource is not found
Returns:: Resource/None: removed Resource instances or None if not found

property resource_names

Returns:: str[]: package resource names

resources

Returns:: Resources[]: package resource

sources

Returns:: dict[]: package sources

title

Returns:: str: package title

to_bigquery(target, *, dialect=None)[source]

Export package to Bigquery

Parameters:: target (string): BigQuery Service object dialect (dict): BigQuery dialect
Returns:: BigqueryStorage: storage

to_ckan(target, *, dialect=None)[source]

Export package to CKAN

Parameters:: target (string): CKAN instance url e.g. “https://demo.ckan.org” dialect (dict): CKAN dialect
Returns:: CkanStorage: storage

to_copy()[source]: Create a copy of the package

to_sql(target, *, dialect=None)[source]

Export package to SQL

Parameters:: target (any): SQL connection string of engine dialect (dict): SQL dialect
Returns:: SqlStorage: storage

to_zip(path, *, encoder_class=None)[source]

Save package to a zip

Parameters:: path (str): target path encoder_class (object): json encoder class
Raises:: FrictionlessException: on any error

property trusted

Returns:: str: package trusted

version

Returns:: str: package version

Resource

class frictionless.resource.Resource(source=None, *, descriptor=None, name=None, title=None, description=None, mediatype=None, licenses=None, sources=None, profile=None, path=None, data=None, scheme=None, format=None, hashing=None, encoding=None, innerpath=None, compression=None, control=None, dialect=None, layout=None, schema=None, stats=None, basepath='', detector=None, onerror='ignore', trusted=False, package=None)[source]

Resource representation.

API | Usage ——– | ——– Public | from frictionless import Resource

This class is one of the cornerstones of of Frictionless framework. It loads a data source, and allows you to stream its parsed contents. At the same time, it’s a metadata class data description.

```python with Resource(“data/table.csv”) as resource:

resource.header == [“id”, “name”] resource.read_rows() == [

{‘id’: 1, ‘name’: ‘english’}, {‘id’: 2, ‘name’: ‘中国人’},

]

```

Parameters:

source (any): Source of the resource; can be in various forms.
Usually, it’s a string as <scheme>://path/to/file.<format>. It also can be, for example, an array of data arrays/dictionaries. Or it can be a resource descriptor dict or path.

descriptor (dict|str): A resource descriptor provided explicitly.
Keyword arguments will patch this descriptor if provided.

name? (str): A Resource name according to the specs.
It should be a slugified name of the resource.

title? (str): A Resource title according to the specs
It should a human-oriented title of the resource.

description? (str): A Resource description according to the specs
It should a human-oriented description of the resource.

mediatype? (str): A mediatype/mimetype of the resource e.g. “text/csv”,
or “application/vnd.ms-excel”. Mediatypes are maintained by the Internet Assigned Numbers Authority (IANA) in a media type registry.

licenses? (dict[]): The license(s) under which the resource is provided.
If omitted it’s considered the same as the package’s licenses.

sources? (dict[]): The raw sources for this data resource.
It MUST be an array of Source objects. Each Source object MUST have a title and MAY have path and/or email properties.

profile? (str): A string identifying the profile of this descriptor.
For example, tabular-data-resource.

scheme? (str): Scheme for loading the file (file, http, …).
If not set, it’ll be inferred from source.

format? (str): File source’s format (csv, xls, …).
If not set, it’ll be inferred from source.

hashing? (str): An algorithm to hash data.
It defaults to ‘md5’.

encoding? (str): Source encoding.
If not set, it’ll be inferred from source.

innerpath? (str): A path within the compressed file.
It defaults to the first file in the archive.

compression? (str): Source file compression (zip, …).
If not set, it’ll be inferred from source.

control? (dict|Control): File control.
For more information, please check the Control documentation.

dialect? (dict|Dialect): Table dialect.
For more information, please check the Dialect documentation.

layout? (dict|Layout): Table layout.
For more information, please check the Layout documentation.

schema? (dict|Schema): Table schema.
For more information, please check the Schema documentation.

stats? (dict): File/table stats.
A dict with the following possible properties: hash, bytes, fields, rows.

basepath? (str): A basepath of the resource
The fullpath of the resource is joined basepath and /path`

detector? (Detector): File/table detector.
For more information, please check the Detector documentation.

onerror? (ignore|warn|raise): Behaviour if there is an error.
It defaults to ‘ignore’. The default mode will ignore all errors on resource level and they should be handled by the user being available in Header and Row objects.

trusted? (bool): Don’t raise an exception on unsafe paths.
A path provided as a part of the descriptor considered unsafe if there are path traversing or the path is absolute. A path provided as source or path is alway trusted.

package? (Package): A owning this resource package.
It’s actual if the resource is part of some data package.

Raises:: FrictionlessException: raise any error that occurs during the process

Attributes

basepath: Returns
buffer: File’s bytes used as a sample
byte_stream: Byte stream in form of a generator
closed: Whether the table is closed
description_html: Returns:
detector: Returns
fragment: Table’s lists used as fragment.
fullpath: Returns
header: Returns:
labels: Returns:
list_stream: List stream in form of a generator
metadata_errors: Returns:
metadata_valid: Returns:
onerror: Returns:
package: Returns:
row_stream: Row stream in form of a generator of Row objects
sample: Table’s lists used as sample.
text_stream: Text stream in form of a generator
trusted: Returns:

Methods

`clear`()
`close`()	Close the table as "filelike.close" does
`copy`()
`expand`()	Expand metadata
`from_petl`(view, **options)	Create a resource from PETL view
`fromkeys`(iterable[, value])	Create a new dictionary with keys from iterable and values set to value.
`get`(key[, default])	Return the value for key if key is in the dictionary, else default.
`infer`(*[, stats])	Infer metadata
`items`()
`keys`()
`metadata_attach`(name, value)	Helper method for attaching a value to the metadata
`metadata_extract`(descriptor)	Helper method called during the metadata extraction
`metadata_process`()	Helper method called on any metadata change
`metadata_validate`()	Helper method called on any metadata change
`open`()	Open the resource as "io.open" does
`pop`(k[,d])	If key is not found, default is returned if given, otherwise KeyError is raised
`popitem`(args, *kwargs)	Remove and return a (key, value) pair as a 2-tuple.
`property`([func, cache, reset, write])	Create a metadata property
`read_bytes`(*[, size])	Read bytes into memory
`read_data`(*[, size])	Read data into memory
`read_lists`(*[, size])	Read lists into memory
`read_rows`(*[, size])	Read rows into memory
`read_text`(*[, size])	Read text into memory
`setdefault`(args, *kwargs)	Insert key with a value of default if key is not in the dictionary.
`setinitial`(key, value)	Set an initial item in a subclass' constructor
`to_copy`(**options)	Create a copy from the resource
`to_dict`()	Create a dict from the resource
`to_inline`(*[, dialect])	Helper to export resource as an inline data
`to_json`([path, encoder_class])	Save metadata as a json
`to_pandas`(*[, dialect])	Helper to export resource as an Pandas dataframe
`to_petl`([normalize])	Export resource as a PETL table
`to_snap`(*[, json])	Create a snapshot from the resource
`to_view`([type])	Create a view from the resource
`to_yaml`([path])	Save metadata as a yaml
`update`([E, ]**F)	If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]
`values`()
`write`([target])	Write this resource to the target resource

metadata_Error

property basepath

Returns: str: resource basepath

property buffer

File’s bytes used as a sample

These buffer bytes are used to infer characteristics of the source file (e.g. encoding, …).

Returns:: bytes?: file buffer

property byte_stream

Byte stream in form of a generator

Yields:: gen<bytes>?: byte stream

close()[source]: Close the table as “filelike.close” does

property closed

Whether the table is closed

Returns:: bool: if closed

compression

Returns: str: resource compression

control

Returns: Control: resource control

data

Returns: any[][]?: resource data

description

Returns: str: resource description

property description_html

Returns:: str?: resource description

description_text

Returns:: str: resource description

property detector

Returns: str: resource detector

dialect

Returns: Dialect: resource dialect

encoding

Returns: str: resource encoding

expand()[source]: Expand metadata

format

Returns: str: resource format

property fragment

Table’s lists used as fragment.

These fragment rows are used internally to infer characteristics of the source file (e.g. schema, …).

Returns:: list[]?: table fragment

static from_petl(view, **options)[source]: Create a resource from PETL view

property fullpath

Returns: str: resource fullpath

hashing

Returns: str: resource hashing

property header

Returns:: str[]?: table header

infer(*, stats=False)[source]

Infer metadata

Parameters:: stats? (bool): stream file completely and infer stats

innerpath

Returns: str: resource compression path

property labels

Returns:: str[]?: table labels

layout

Returns:: Layout: table layout

licenses

Returns: dict[]: resource licenses

property list_stream

List stream in form of a generator

Yields:: gen<any[][]>?: list stream

mediatype

Returns: str: resource mediatype

metadata_Error: alias of frictionless.errors.general.ResourceError

metadata_process()[source]: Helper method called on any metadata change

metadata_validate()[source]

Helper method called on any metadata change

Parameters:: profile (dict): a profile to validate against of

name

Returns: str: resource name

property onerror

Returns:: ignore|warn|raise: on error bahaviour

open()[source]

Open the resource as “io.open” does

Raises:: FrictionlessException: any exception that occurs

property package

Returns:: Package?: parent package

path

Returns: str: resource path

profile

Returns: str: resource profile

read_bytes(*, size=None)[source]

Read bytes into memory

Returns:: any[][]: resource bytes

read_data(*, size=None)[source]

Read data into memory

Returns:: any: resource data

read_lists(*, size=None)[source]

Read lists into memory

Returns:: any[][]: table lists

read_rows(*, size=None)[source]

Read rows into memory

Returns:: Row[]: table rows

read_text(*, size=None)[source]

Read text into memory

Returns:: str: resource text

property row_stream

Row stream in form of a generator of Row objects

Yields:: gen<Row[]>?: row stream

property sample

Table’s lists used as sample.

These sample rows are used to infer characteristics of the source file (e.g. schema, …).

Returns:: list[]?: table sample

schema

Returns: Schema: resource schema

scheme

Returns: str: resource scheme

sources

Returns: dict[]: resource sources

stats

Returns: dict: resource stats

tabular

Returns: bool: if resource is tabular

property text_stream

Text stream in form of a generator

Yields:: gen<str[]>?: text stream

title

Returns: str: resource title

to_copy(**options)[source]

Create a copy from the resource

Returns: Resource: resource copy

to_dict()[source]

Create a dict from the resource

Returns: dict: dict representation

to_inline(*, dialect=None)[source]: Helper to export resource as an inline data

to_pandas(*, dialect=None)[source]: Helper to export resource as an Pandas dataframe

to_petl(normalize=False)[source]: Export resource as a PETL table

to_snap(*, json=False)[source]

Create a snapshot from the resource

Parameters:: json (bool): make data types compatible with JSON format
Returns: list: resource’s data

to_view(type='look', **options)[source]

Create a view from the resource

See PETL’s docs for more information: https://petl.readthedocs.io/en/stable/util.html#visualising-tables

Parameters:: type (look|lookall|see|display|displayall): view’s type **options (dict): options to be passed to PETL
Returns: str: resource’s view

property trusted

Returns:: bool: don’t raise an exception on unsafe paths

write(target=None, **options)[source]

Write this resource to the target resource

Parameters:: target (any|Resource): target or target resource instance **options (dict): Resource constructor options