pygeodes.utils package#

Submodules#

pygeodes.utils.dataclasses_utils module#

This module contains all things regarding python dataclasses

pygeodes.utils.dataclasses_utils.class_from_args(class_obj, arg_dict: str)[source]#

Returns an instance of a class from the class object and a dictionnary containing all the args

Parameters:
class_objClass

a class object

arg_dictstr

an argument dictionnary

Returns:
_type_

an instance of the class class_obj

pygeodes.utils.datetime_utils module#

This module contains all things related to date and time

pygeodes.utils.datetime_utils.a_day_ago() datetime[source]#

Returns a datetime object from 24h ago

Returns:
datetime

a datetime object from 24h ago

pygeodes.utils.datetime_utils.a_week_ago() datetime[source]#

Returns a datetime object from a week ago

Returns:
datetime

a datetime object from a week ago

pygeodes.utils.datetime_utils.complete_datetime_from_str(string: str) str[source]#

Returns a valid string for format pygeodes.utils.consts.DATETIME_FORMAT from a non-valid string (under some defined formats)

Parameters:
stringstr

a non valid datetime string

Returns:
str

a valid datetime string

Raises:
Exception

Examples

complete_datetime_from_str("2023-09-23") # this returns "2023-09-23T00:00:00.0Z"
pygeodes.utils.datetime_utils.datetime_to_str(datetime_obj: datetime) str[source]#

Returns a datetime string from a datetime obj

Parameters:
datetime_objdatetime

a datetime object

Returns:
str

a datetime string from the object

See also

str_to_datetime

to do the exact inverse

pygeodes.utils.datetime_utils.is_in_format(string: str, format: str) bool[source]#

Returns wether string string is in format format

Parameters:
stringstr

a datetime string

formatstr

a datetime format string

Returns:
bool

wether string string is in format format

pygeodes.utils.datetime_utils.str_to_datetime(string: str) datetime[source]#

Returns a datetime object from the provided datetime string

Parameters:
stringstr

a datetime string

Returns:
datetime

a datetime object from the string

See also

datetime_to_str

to do the exact inverse

pygeodes.utils.datetime_utils.time_ago(date: datetime) str[source]#

Returns a string with how long ago was the date provided

Parameters:
datedatetime

a date

Returns:
str

a string with how long ago the date was

pygeodes.utils.datetime_utils.today() Tuple[datetime, datetime][source]#

Returns two datetime objects, today at first time (00:00), today at last time (23:59)

Returns:
Tuple[datetime,datetime]

today at first time (00:00), today at last time (23:59)

pygeodes.utils.decorators module#

This module regroups all decorators used in pygeodes

pygeodes.utils.decorators.not_implemented(func: Callable)[source]#

This decorator, placed on a function, raises an error when the function is called, saying it’s not implemented yet

pygeodes.utils.decorators.requires_api_key(_func: Callable | None = None, bypass_with_s3_credentials: bool = False)[source]#

This decorator, on a Provider method, makes an api_key mandatory for this method, and raises an error if the provider config does not contain any api_key

pygeodes.utils.decorators.uses_session(func: Callable)[source]#

This decorator, placed on a method from pygeodes.utils.request.SyncRequestMaker, opens an http session before executing the function and closes it after

pygeodes.utils.download module#

This module regroups some things related to downloading

pygeodes.utils.download.correct_download_tld(url: str) str[source]#

This function corrects the domain name of a download link when needed

Parameters:
urlstr

the download link to corret

Returns:
str

the download link corrected

pygeodes.utils.exceptions module#

This module regroups all pygeodes exceptions

exception pygeodes.utils.exceptions.DataAssetMissingException[source]#

Bases: Exception

exception pygeodes.utils.exceptions.InvalidChecksumException[source]#

Bases: Exception

exception pygeodes.utils.exceptions.InvalidURLException[source]#

Bases: Exception

exception pygeodes.utils.exceptions.MissingConfParamException[source]#

Bases: Exception

exception pygeodes.utils.exceptions.RequiresApiKeyException[source]#

Bases: Exception

exception pygeodes.utils.exceptions.TooManyResultsException[source]#

Bases: Exception

pygeodes.utils.formatting module#

This module deals with all things related to formatting items or collections

pygeodes.utils.formatting.export_dataframe(dataframe: GeoDataFrame, outfile: str) None[source]#

This functions exports a pd.GeoDataFrame in a file

Parameters:
dataframepd.GeoDataFrame

the dataframe

outfilestr

the filepath

See also

load_dataframe

to load a dataframe from a file

pygeodes.utils.formatting.format_collections(collections: List[Collection] | GeoDataFrame, columns_to_add: set | List | None = None) GeoDataFrame[source]#

This functions format a list of collections, or a GeoDataFrame by adding columns by name

Parameters:
collectionsUnion[List[Collection], pd.GeoDataFrame]

the list of collections to transform into a dataframe or GeoDataFrame to add columns to

columns_to_addUnion[set, List], optional

the columns to add, by default None

Returns:
pd.GeoDataFrame

the new dataframe

pygeodes.utils.formatting.format_items(items: List[Item] | GeoDataFrame, columns_to_add: set | List | None = None) GeoDataFrame[source]#

This functions format a list of items, or a GeoDataFrame by adding columns by name

Parameters:
itemsUnion[List[Item], pd.GeoDataFrame]

the list of items to transform into a dataframe or GeoDataFrame to add columns to

columns_to_addUnion[set, List], optional

the columns to add, by default None

Returns:
pd.GeoDataFrame

the new dataframe

pygeodes.utils.formatting.get_from_dico_path(path: str, dico: dict)[source]#

This functions gets a value from a dictionnary using a system of dotted path

Parameters:
pathstr

the dictionnary path

dicodict

the dictionnary

Returns:
Any

the value at path, or None if not found

Examples

dico = {"properties" : {"property_one" : 4}}
value = get_from_dico_path("properties.property_one",dico) # returns 4
pygeodes.utils.formatting.load_dataframe(filepath: str)[source]#

This function loads a pd.GeoDataFrame from a file

Parameters:
filepathstr

the file

Returns:
pd.GeoDataFrame

the dataframe

See also

export_dataframe

to export a dataframe into a file

pygeodes.utils.io module#

This module regroups all functions related to file INPUT/OUTPUT

pygeodes.utils.io.check_if_folder_already_contains_file(filepath: str, file_checksum: str)[source]#

The goal of this function is to check if, in the folder of the filepath provided, it already exists a file with the same checksum as the one provided, with in aim not to download several times the same file. As it’s not something required but more of an help for the user, we assume that if the operation takes more than 5 seconds, we abort it.

pygeodes.utils.io.compute_md5(filepath: str) str[source]#

Computes md5sum of the contents of the given filepath

pygeodes.utils.io.file_exists(filepath: str, raise_exception: bool = True) bool[source]#

This checks if the file exists, and, in function of a boolean parameter, raises an exception if it doesn’t

Parameters:
filepathstr

the filepath to check the existence of

raise_exceptionbool, optional

whether to raise an exception, by default True

Returns:
bool

whether the file exists

Raises:
FileNotFoundError

error raised if the file doesn’t exist

See also

similar_filenames

to find the most similar filenames to a filename

Examples

name = "file.txt"
exists = file_exists(name)
pygeodes.utils.io.filename_in_folder(name: str, folder_path: str) bool[source]#

This functions checks if a filename is in a folder

Parameters:
namestr

the filename

folder_pathstr

the folder

Returns:
bool

wether the filename exists in the folder

pygeodes.utils.io.filenames_respecting_regex(filenames: List[str], regex: str) List[str][source]#

This functions find all the filenames in a list of filenames matching a given regex

Parameters:
filenamesList[str]

the list of filenames

regexstr

the regex

Returns:
List[str]

the filenames matching the regex

pygeodes.utils.io.find_unused_filename(filepath: str) str[source]#

This functions finds an unused filename for the filepath provided by adding -{number} after

Parameters:
filepathstr

the original filename

Returns:
str

an unused filename

pygeodes.utils.io.get_homedir() Path[source]#

To get the home directory path of the current user

Returns:
Path

the homedir

pygeodes.utils.io.load_json(filepath: str) dict[source]#

This functions loads a JSON into a Python dict from a filepath

Parameters:
filepathstr

the file to load the dict from

Returns:
dict

the dict loaded

Raises:
Exception

an Exception if the JSON is not a valid JSON

See also

write_json

to dump a Python dict into a JSON file

pygeodes.utils.io.similar_filenames(filename: str, other_filenames: List[str], nb: int = 10) List[str][source]#

This function returns the nb most resembling filenames to the filename provided in the list of filenames

Args:

filename (str): the filename other_filenames (List[str]): the other filenames to be compared to nb (int, optional): the number of filenames to keep. Defaults to 10.

Returns:

List[str]: the most resembling filenames

pygeodes.utils.io.wrapped_default(self, obj)[source]#
pygeodes.utils.io.write_json(content: dict, filepath: str) None[source]#

This functions dumps a dict at a filepath

Parameters:
contentdict

the dict to dump

filepathstr

the filepath to dump to

See also

load_json

to read a JSON from a file

pygeodes.utils.logger module#

This module contains the pygeodes logger and its config

pygeodes.utils.query module#

This module contains the Query class

class pygeodes.utils.query.Argument(name: str)[source]#

Bases: object

Methods

contains

eq

gte

is_in

lte

to_dict

contains(value)[source]#
eq(value)[source]#
gte(value)[source]#
is_in(value: list)[source]#
lte(value)[source]#
to_dict()[source]#
class pygeodes.utils.query.Query[source]#

Bases: object

Methods

add

check

check_for_collection

check_for_item

to_dict

add(argument: Argument)[source]#
check()[source]#
check_for_collection()[source]#
check_for_item()[source]#
to_dict()[source]#
pygeodes.utils.query.full_text_search_in_jsons(jsons: List[dict], search_term: str, key_field: str, fields_to_index: set, return_index: bool = False)[source]#
pygeodes.utils.query.get_requestable_args()[source]#

pygeodes.utils.request module#

This module provides tools to make synchronous and asynchronous HTTP requests

class pygeodes.utils.request.AsyncRequestMaker(api_key: str, base_url: str)[source]#

Bases: RequestMaker

Methods

control

download_files

download_files_async

get

get_async

get_full_url

post

post_async

async control(response: ClientResponse)[source]#
download_files(endpoints: List[str], outfiles: str, checksums: str, checksum_error: bool = True)[source]#
async download_files_async(endpoints: List[str], outfiles: str)[source]#
get(endpoints: List[str], headers: List[Dict[str, str]] | None = None)[source]#
async get_async(endpoints: List[str], headers: List[Dict[str, str]])[source]#
post(endpoints: List[str], datas: List[Dict], headers: List[Dict[str, str]] | None = None)[source]#
async post_async(endpoints: List[str], headers: List[Dict[str, str]], datas: List[Dict])[source]#
class pygeodes.utils.request.RequestMaker(api_key: str, base_url: str)[source]#

Bases: object

Methods

get_full_url

get_full_url(endpoint: str) str[source]#
class pygeodes.utils.request.SyncRequestMaker(api_key: str, base_url: str)[source]#

Bases: RequestMaker

Methods

close_session

download_file

extract_file_from_archive

get

get_full_url

list_files_in_archive

open_session

post

close_session()[source]#
download_file(endpoint: str, outfile: str, checksum: str, checksum_error: bool = True, verbose: bool = True)[source]#
extract_file_from_archive(archive_url: str, filename: str, download_dir: str)[source]#
get(endpoint: str, headers: Dict[str, str] = None) Any[source]#
list_files_in_archive(archive_url: str) List[str][source]#
open_session()[source]#
post(endpoint: str, data: Dict[str, str], headers: Dict[str, str] = None) Any[source]#
pygeodes.utils.request.auth_headers(api_key: str)[source]#
pygeodes.utils.request.check_all_different(objects)[source]#
pygeodes.utils.request.make_params(page: int, query: dict, bbox: List[float] | None = None, intersects: dict | None = None)[source]#
pygeodes.utils.request.valid_url(url: str) bool[source]#

pygeodes.utils.stac module#

This module deals with all classes derived from pystac https://pystac.readthedocs.io/en/stable/

class pygeodes.utils.stac.Asset(href: str, title: str | None = None, description: str | None = None, media_type: str | None = None, roles: List[str] | None = None, extra_fields: Dict[str, Any] | None = None)[source]#

Bases: Asset

Attributes:
checksum
common_metadata

Access the asset’s common metadata fields as a CommonMetadata object.

filesize

Methods

clone()

Clones this asset.

copy(href)

Copies this asset's file to a new location on the local filesystem, setting the asset href accordingly.

delete()

Delete this asset's file.

from_dict(d)

Constructs an Asset from a dict.

get_absolute_href()

Gets the absolute href for this asset, if possible.

has_role(role)

Check if a role exists in the Asset role list.

move(href)

Moves this asset's file to a new location on the local filesystem, setting the asset href accordingly.

set_owner(obj)

Sets the owning item of this Asset.

to_dict()

Returns this Asset as a dictionary.

from_stac_asset

property checksum#
property filesize#
classmethod from_stac_asset(asset: Asset)[source]#
class pygeodes.utils.stac.Collection(id: str, description: str, extent: Extent, title: str | None = None, stac_extensions: List[str] | None = None, href: str | None = None, extra_fields: Dict[str, Any] | None = None, catalog_type: CatalogType | None = None, license: str = 'proprietary', keywords: List[str] | None = None, providers: List[Provider] | None = None, summaries: Summaries | None = None, assets: Dict[str, Asset] | None = None)[source]#

Bases: Collection

Attributes:
self_href

Gets the absolute HREF that is represented by the rel == 'self' Link.

Methods

add_asset(key, asset)

Adds an Asset to this collection.

add_child(child[, title, strategy, set_parent])

Adds a link to a child Catalog or Collection.

add_children(children)

Adds links to multiple Catalog or ~pystac.Collection objects.

add_item(item[, title, strategy, set_parent])

Adds a link to an Item.

add_items(items[, strategy])

Adds links to multiple Items.

add_link(link)

Add a link to this object's set of links.

add_links(links)

Add links to this object's set of links.

clear_children()

Removes all children from this catalog.

clear_items()

Removes all items from this catalog.

clear_links([rel])

Clears all Link instances associated with this object.

clone()

Clones this object.

delete_asset(key)

Deletes the asset at the given key, and removes the asset's data file from the local filesystem.

describe([include_hrefs, _indent])

Prints out information about this Catalog and all contained STACObjects.

from_dict(dico)

Parses this STACObject from the passed in dictionary.

from_file(href[, stac_io])

Reads a STACObject implementation from a file.

full_copy([root, parent])

Create a full copy of this STAC object and any stac objects linked to by this object.

fully_resolve()

Resolves every link in this catalog.

generate_subcatalogs(template[, defaults, ...])

Walks through the catalog and generates subcatalogs for items based on the template string.

get_all_collections()

Get all collections from this catalog and all subcatalogs.

get_all_items()

DEPRECATED.

get_assets([media_type, role])

Get this collection's assets.

get_child(id[, recursive, sort_links_by_id])

Gets the child of this catalog with the given ID, if it exists.

get_child_links()

Return all child links of this catalog.

get_children()

Return all children of this catalog.

get_collections()

Return all children of this catalog that are Collection instances.

get_item(id[, recursive])

Returns an item with a given ID.

get_item_links()

Return all item links of this catalog.

get_items(*ids[, recursive])

Return all items or specific items of this catalog.

get_links([rel, media_type])

Gets the Link instances associated with this object.

get_parent()

Get the Catalog or Collection to the parent for this object.

get_root()

Get the Catalog or Collection to the root for this object.

get_root_link()

Get the Link representing the root for this object.

get_self_href()

Gets the absolute HREF that is represented by the rel == 'self' Link.

get_single_link([rel, media_type])

Get a single Link instance associated with this object.

get_stac_objects(rel[, typ, modify_links])

Gets the STACObject instances that are linked to by links with their rel property matching the passed in argument.

make_all_asset_hrefs_absolute()

Recursively makes all the HREFs of assets in this catalog absolute

make_all_asset_hrefs_relative()

Recursively makes all the HREFs of assets in this catalog relative

make_asset_hrefs_absolute()

Modify each asset's HREF to be absolute.

make_asset_hrefs_relative()

Modify each asset's HREF to be relative to this collection's self HREF.

map_assets(asset_mapper)

Creates a copy of a catalog, with each Asset for each Item passed through the asset_mapper function.

map_items(item_mapper)

Creates a copy of a catalog, with each item passed through the item_mapper function.

matches_object_type(d)

Returns a boolean indicating whether the given dictionary represents a valid instance of this STACObject sub-class.

normalize_and_save(root_href[, ...])

Normalizes link HREFs to the given root_href, and saves the catalog.

normalize_hrefs(root_href[, strategy, ...])

Normalize HREFs will regenerate all link HREFs based on an absolute root_href and the canonical catalog layout as specified in the STAC specification's best practices.

remove_child(child_id)

Removes an child from this catalog.

remove_hierarchical_links([add_canonical])

Removes all hierarchical links from this object.

remove_item(item_id)

Removes an item from this catalog.

remove_links(rel)

Remove links to this object's set of links that match the given rel.

resolve_links()

Ensure all STACObjects linked to by this STACObject are resolved.

save([catalog_type, dest_href, stac_io])

Save this catalog and all it's children/item to files determined by the object's self link HREF or a specified path.

save_object([include_self_link, dest_href, ...])

Saves this STAC Object to it's 'self' HREF.

set_parent(parent)

Sets the parent Catalog or Collection for this object.

set_root(root)

Sets the root Catalog or Collection for this object.

set_self_href(href)

Sets the absolute HREF that is represented by the rel == 'self' Link.

target_in_hierarchy(target)

Determine if target is somewhere in the hierarchical link tree of a STACObject.

to_dict()

Returns this object as a dictionary.

update_extent_from_items()

Update datetime and bbox based on all items to a single bbox and time window.

validate()

Validate this STACObject.

validate_all([max_items, recursive])

Validates each catalog, collection, item contained within this catalog.

walk()

Walks through children and items of catalogs.

find

from_stac_collection

is_relative

list_available_keys

find(key: str)[source]#
classmethod from_dict(dico: dict)[source]#

Parses this STACObject from the passed in dictionary.

Args:

d : The dict to parse. href : Optional href that is the file location of the object being

parsed.

rootOptional root catalog for this object.

If provided, the root of the returned STACObject will be set to this parameter.

migrate: Use True if this dict represents JSON from an older STAC object,

so that migrations are run against it.

preserve_dict: If False, the dict parameter d may be modified

during this method call. Otherwise the dict is not mutated. Defaults to True, which results results in a deepcopy of the parameter. Set to False when possible to avoid the performance hit of a deepcopy.

Returns:

STACObject: The STACObject parsed from this dict.

classmethod from_stac_collection(collection: Collection)[source]#
list_available_keys()[source]#
to_dict()[source]#

Returns this object as a dictionary.

Args:
include_self_linkIf True, the dict will contain a self link

to this object. If False, the self link will be omitted.

transform_hrefs: If True, transform the HREF of hierarchical links

based on the type of catalog this object belongs to (if any). I.e. if this object belongs to a root catalog that is RELATIVE_PUBLISHED or SELF_CONTAINED, hierarchical link HREFs will be transformed to be relative to the catalog root.

dict: A serialization of the object.

class pygeodes.utils.stac.Item(*args, **kwargs)[source]#

Bases: Item

Attributes:
common_metadata

Access the item’s common metadata fields as a CommonMetadata object.

data_asset
data_asset_checksum
filesize
quicklook_asset
s3_path
self_href

Gets the absolute HREF that is represented by the rel == 'self' Link.

Methods

add_asset(key, asset)

Adds an Asset to this item.

add_derived_from(*items)

Add one or more items that this is derived from.

add_link(link)

Add a link to this object's set of links.

add_links(links)

Add links to this object's set of links.

clear_links([rel])

Clears all Link instances associated with this object.

clone()

Clones this object.

delete_asset(key)

Deletes the asset at the given key, and removes the asset's data file from the local filesystem.

from_dict(d[, href, root, migrate, ...])

Parses this STACObject from the passed in dictionary.

from_file(href[, stac_io])

Reads a STACObject implementation from a file.

full_copy([root, parent])

Create a full copy of this STAC object and any stac objects linked to by this object.

get_assets([media_type, role])

Get this item's assets.

get_collection()

Gets the collection of this item, if one exists.

get_datetime([asset])

Gets an Item or an Asset datetime.

get_derived_from()

Get the items that this is derived from.

get_links([rel, media_type])

Gets the Link instances associated with this object.

get_parent()

Get the Catalog or Collection to the parent for this object.

get_root()

Get the Catalog or Collection to the root for this object.

get_root_link()

Get the Link representing the root for this object.

get_self_href()

Gets the absolute HREF that is represented by the rel == 'self' Link.

get_single_link([rel, media_type])

Get a single Link instance associated with this object.

get_stac_objects(rel[, typ, modify_links])

Gets the STACObject instances that are linked to by links with their rel property matching the passed in argument.

make_asset_hrefs_absolute()

Modify each asset's HREF to be absolute.

make_asset_hrefs_relative()

Modify each asset's HREF to be relative to this item's self HREF.

matches_object_type(d)

Returns a boolean indicating whether the given dictionary represents a valid instance of this STACObject sub-class.

remove_derived_from(item_id)

Remove an item that this is derived from.

remove_hierarchical_links([add_canonical])

Removes all hierarchical links from this object.

remove_links(rel)

Remove links to this object's set of links that match the given rel.

resolve_links()

Ensure all STACObjects linked to by this STACObject are resolved.

save_object([include_self_link, dest_href, ...])

Saves this STAC Object to it's 'self' HREF.

set_collection(collection)

Set the collection of this item.

set_datetime(datetime[, asset])

Set an Item or an Asset datetime.

set_parent(parent)

Sets the parent Catalog or Collection for this object.

set_root(root)

Sets the root Catalog or Collection for this object.

set_self_href(href)

Sets the absolute HREF that is represented by the rel == 'self' Link.

target_in_hierarchy(target)

Determine if target is somewhere in the hierarchical link tree of a STACObject.

to_dict()

Returns this object as a dictionary.

validate()

Validate this STACObject.

download_archive

find

get_quicklook_content_in_base64

list_available_keys

show_quicklook

property data_asset#
property data_asset_checksum#
download_archive(outfile: str | None = None)[source]#
property filesize#
find(key: str)[source]#
get_quicklook_content_in_base64()[source]#
list_available_keys(with_origin=False)[source]#
property quicklook_asset#
property s3_path#
show_quicklook()[source]#
to_dict()[source]#

Returns this object as a dictionary.

Args:
include_self_linkIf True, the dict will contain a self link

to this object. If False, the self link will be omitted.

transform_hrefs: If True, transform the HREF of hierarchical links

based on the type of catalog this object belongs to (if any). I.e. if this object belongs to a root catalog that is RELATIVE_PUBLISHED or SELF_CONTAINED, hierarchical link HREFs will be transformed to be relative to the catalog root.

dict: A serialization of the object.

pygeodes.utils.stac.correct_stac_version(collection_dict: dict) dict[source]#
pygeodes.utils.stac.get_keys_from_dict(dico)[source]#

Module contents#