pygeodes.utils package#
Submodules#
pygeodes.utils.dataclasses_utils module#
This module contains all things regarding python dataclasses
- pygeodes.utils.dataclasses_utils.class_from_args(class_obj, arg_dict: str)[source]#
Returns an instance of a class from the class object and a dictionnary containing all the args
- Parameters:
- class_objClass
a class object
- arg_dictstr
an argument dictionnary
- Returns:
- _type_
an instance of the class class_obj
pygeodes.utils.datetime_utils module#
This module contains all things related to date and time
- pygeodes.utils.datetime_utils.a_day_ago() datetime [source]#
Returns a datetime object from 24h ago
- Returns:
- datetime
a datetime object from 24h ago
- pygeodes.utils.datetime_utils.a_week_ago() datetime [source]#
Returns a datetime object from a week ago
- Returns:
- datetime
a datetime object from a week ago
- pygeodes.utils.datetime_utils.complete_datetime_from_str(string: str) str [source]#
Returns a valid string for format pygeodes.utils.consts.DATETIME_FORMAT from a non-valid string (under some defined formats)
- Parameters:
- stringstr
a non valid datetime string
- Returns:
- str
a valid datetime string
- Raises:
- Exception
Examples
complete_datetime_from_str("2023-09-23") # this returns "2023-09-23T00:00:00.0Z"
- pygeodes.utils.datetime_utils.datetime_to_str(datetime_obj: datetime) str [source]#
Returns a datetime string from a datetime obj
- Parameters:
- datetime_objdatetime
a datetime object
- Returns:
- str
a datetime string from the object
See also
str_to_datetime
to do the exact inverse
- pygeodes.utils.datetime_utils.is_in_format(string: str, format: str) bool [source]#
Returns wether string
string
is in formatformat
- Parameters:
- stringstr
a datetime string
- formatstr
a datetime format string
- Returns:
- bool
wether string
string
is in formatformat
- pygeodes.utils.datetime_utils.str_to_datetime(string: str) datetime [source]#
Returns a
datetime
object from the provided datetime string- Parameters:
- stringstr
a datetime string
- Returns:
- datetime
a datetime object from the string
See also
datetime_to_str
to do the exact inverse
pygeodes.utils.decorators module#
This module regroups all decorators used in pygeodes
- pygeodes.utils.decorators.not_implemented(func: Callable)[source]#
This decorator, placed on a function, raises an error when the function is called, saying it’s not implemented yet
- pygeodes.utils.decorators.requires_api_key(_func: Callable | None = None, bypass_with_s3_credentials: bool = False)[source]#
This decorator, on a Provider method, makes an api_key mandatory for this method, and raises an error if the provider config does not contain any api_key
- pygeodes.utils.decorators.uses_session(func: Callable)[source]#
This decorator, placed on a method from
pygeodes.utils.request.SyncRequestMaker
, opens an http session before executing the function and closes it after
pygeodes.utils.download module#
This module regroups some things related to downloading
pygeodes.utils.exceptions module#
This module regroups all pygeodes exceptions
pygeodes.utils.formatting module#
This module deals with all things related to formatting items or collections
- pygeodes.utils.formatting.export_dataframe(dataframe: GeoDataFrame, outfile: str) None [source]#
This functions exports a
pd.GeoDataFrame
in a file- Parameters:
- dataframepd.GeoDataFrame
the dataframe
- outfilestr
the filepath
See also
load_dataframe
to load a dataframe from a file
- pygeodes.utils.formatting.format_collections(collections: List[Collection] | GeoDataFrame, columns_to_add: set | List | None = None) GeoDataFrame [source]#
This functions format a list of collections, or a
GeoDataFrame
by adding columns by name- Parameters:
- collectionsUnion[List[Collection], pd.GeoDataFrame]
the list of collections to transform into a dataframe or
GeoDataFrame
to add columns to- columns_to_addUnion[set, List], optional
the columns to add, by default None
- Returns:
- pd.GeoDataFrame
the new dataframe
- pygeodes.utils.formatting.format_items(items: List[Item] | GeoDataFrame, columns_to_add: set | List | None = None) GeoDataFrame [source]#
This functions format a list of items, or a
GeoDataFrame
by adding columns by name- Parameters:
- itemsUnion[List[Item], pd.GeoDataFrame]
the list of items to transform into a dataframe or
GeoDataFrame
to add columns to- columns_to_addUnion[set, List], optional
the columns to add, by default None
- Returns:
- pd.GeoDataFrame
the new dataframe
- pygeodes.utils.formatting.get_from_dico_path(path: str, dico: dict)[source]#
This functions gets a value from a dictionnary using a system of dotted path
- Parameters:
- pathstr
the dictionnary path
- dicodict
the dictionnary
- Returns:
- Any
the value at path, or None if not found
Examples
dico = {"properties" : {"property_one" : 4}} value = get_from_dico_path("properties.property_one",dico) # returns 4
- pygeodes.utils.formatting.load_dataframe(filepath: str)[source]#
This function loads a
pd.GeoDataFrame
from a file- Parameters:
- filepathstr
the file
- Returns:
- pd.GeoDataFrame
the dataframe
See also
export_dataframe
to export a dataframe into a file
pygeodes.utils.io module#
This module regroups all functions related to file INPUT/OUTPUT
- pygeodes.utils.io.check_if_folder_already_contains_file(filepath: str, file_checksum: str)[source]#
The goal of this function is to check if, in the folder of the filepath provided, it already exists a file with the same checksum as the one provided, with in aim not to download several times the same file. As it’s not something required but more of an help for the user, we assume that if the operation takes more than 5 seconds, we abort it.
- pygeodes.utils.io.compute_md5(filepath: str) str [source]#
Computes md5sum of the contents of the given filepath
- pygeodes.utils.io.file_exists(filepath: str, raise_exception: bool = True) bool [source]#
This checks if the file exists, and, in function of a boolean parameter, raises an exception if it doesn’t
- Parameters:
- filepathstr
the filepath to check the existence of
- raise_exceptionbool, optional
whether to raise an exception, by default True
- Returns:
- bool
whether the file exists
- Raises:
- FileNotFoundError
error raised if the file doesn’t exist
See also
similar_filenames
to find the most similar filenames to a filename
Examples
name = "file.txt" exists = file_exists(name)
- pygeodes.utils.io.filename_in_folder(name: str, folder_path: str) bool [source]#
This functions checks if a filename is in a folder
- Parameters:
- namestr
the filename
- folder_pathstr
the folder
- Returns:
- bool
wether the filename exists in the folder
- pygeodes.utils.io.filenames_respecting_regex(filenames: List[str], regex: str) List[str] [source]#
This functions find all the filenames in a list of filenames matching a given regex
- Parameters:
- filenamesList[str]
the list of filenames
- regexstr
the regex
- Returns:
- List[str]
the filenames matching the regex
- pygeodes.utils.io.find_unused_filename(filepath: str) str [source]#
This functions finds an unused filename for the filepath provided by adding -{number} after
- Parameters:
- filepathstr
the original filename
- Returns:
- str
an unused filename
- pygeodes.utils.io.get_homedir() Path [source]#
To get the home directory path of the current user
- Returns:
- Path
the homedir
- pygeodes.utils.io.load_json(filepath: str) dict [source]#
This functions loads a JSON into a Python dict from a filepath
- Parameters:
- filepathstr
the file to load the dict from
- Returns:
- dict
the dict loaded
- Raises:
- Exception
an Exception if the JSON is not a valid JSON
See also
write_json
to dump a Python dict into a JSON file
- pygeodes.utils.io.similar_filenames(filename: str, other_filenames: List[str], nb: int = 10) List[str] [source]#
This function returns the
nb
most resembling filenames to the filename provided in the list of filenames- Args:
filename (
str
): the filename other_filenames (List[str]
): the other filenames to be compared to nb (int
, optional): the number of filenames to keep. Defaults to 10.- Returns:
List[str]
: the most resembling filenames
pygeodes.utils.logger module#
This module contains the pygeodes logger and its config
pygeodes.utils.query module#
This module contains the Query class
- class pygeodes.utils.query.Argument(name: str)[source]#
Bases:
object
Methods
contains
eq
gte
is_in
lte
to_dict
- class pygeodes.utils.query.Query[source]#
Bases:
object
Methods
add
check
check_for_collection
check_for_item
to_dict
pygeodes.utils.request module#
This module provides tools to make synchronous and asynchronous HTTP requests
- class pygeodes.utils.request.AsyncRequestMaker(api_key: str, base_url: str)[source]#
Bases:
RequestMaker
Methods
control
download_files
download_files_async
get
get_async
get_full_url
post
post_async
- class pygeodes.utils.request.RequestMaker(api_key: str, base_url: str)[source]#
Bases:
object
Methods
get_full_url
- class pygeodes.utils.request.SyncRequestMaker(api_key: str, base_url: str)[source]#
Bases:
RequestMaker
Methods
close_session
download_file
extract_file_from_archive
get
get_full_url
list_files_in_archive
open_session
post
pygeodes.utils.stac module#
This module deals with all classes derived from pystac https://pystac.readthedocs.io/en/stable/
- class pygeodes.utils.stac.Asset(href: str, title: str | None = None, description: str | None = None, media_type: str | None = None, roles: List[str] | None = None, extra_fields: Dict[str, Any] | None = None)[source]#
Bases:
Asset
- Attributes:
- checksum
common_metadata
Access the asset’s common metadata fields as a
CommonMetadata
object.- filesize
Methods
clone
()Clones this asset.
copy
(href)Copies this asset's file to a new location on the local filesystem, setting the asset href accordingly.
delete
()Delete this asset's file.
from_dict
(d)Constructs an Asset from a dict.
Gets the absolute href for this asset, if possible.
has_role
(role)Check if a role exists in the Asset role list.
move
(href)Moves this asset's file to a new location on the local filesystem, setting the asset href accordingly.
set_owner
(obj)Sets the owning item of this Asset.
to_dict
()Returns this Asset as a dictionary.
from_stac_asset
- property checksum#
- property filesize#
- class pygeodes.utils.stac.Collection(id: str, description: str, extent: Extent, title: str | None = None, stac_extensions: List[str] | None = None, href: str | None = None, extra_fields: Dict[str, Any] | None = None, catalog_type: CatalogType | None = None, license: str = 'proprietary', keywords: List[str] | None = None, providers: List[Provider] | None = None, summaries: Summaries | None = None, assets: Dict[str, Asset] | None = None)[source]#
Bases:
Collection
- Attributes:
self_href
Gets the absolute HREF that is represented by the
rel == 'self'
Link
.
Methods
add_asset
(key, asset)Adds an Asset to this collection.
add_child
(child[, title, strategy, set_parent])Adds a link to a child
Catalog
orCollection
.add_children
(children)Adds links to multiple
Catalog
or ~pystac.Collection objects.add_item
(item[, title, strategy, set_parent])Adds a link to an
Item
.add_items
(items[, strategy])Adds links to multiple
Items
.add_link
(link)Add a link to this object's set of links.
add_links
(links)Add links to this object's set of links.
Removes all children from this catalog.
Removes all items from this catalog.
clear_links
([rel])Clears all
Link
instances associated with this object.clone
()Clones this object.
delete_asset
(key)Deletes the asset at the given key, and removes the asset's data file from the local filesystem.
describe
([include_hrefs, _indent])Prints out information about this Catalog and all contained STACObjects.
from_dict
(dico)Parses this STACObject from the passed in dictionary.
from_file
(href[, stac_io])Reads a STACObject implementation from a file.
full_copy
([root, parent])Create a full copy of this STAC object and any stac objects linked to by this object.
Resolves every link in this catalog.
generate_subcatalogs
(template[, defaults, ...])Walks through the catalog and generates subcatalogs for items based on the template string.
Get all collections from this catalog and all subcatalogs.
DEPRECATED.
get_assets
([media_type, role])Get this collection's assets.
get_child
(id[, recursive, sort_links_by_id])Gets the child of this catalog with the given ID, if it exists.
Return all child links of this catalog.
Return all children of this catalog.
Return all children of this catalog that are
Collection
instances.get_item
(id[, recursive])Returns an item with a given ID.
Return all item links of this catalog.
get_items
(*ids[, recursive])Return all items or specific items of this catalog.
get_links
([rel, media_type])Gets the
Link
instances associated with this object.Get the
Catalog
orCollection
to the parent for this object.get_root
()Get the
Catalog
orCollection
to the root for this object.Get the
Link
representing the root for this object.Gets the absolute HREF that is represented by the
rel == 'self'
Link
.get_single_link
([rel, media_type])Get a single
Link
instance associated with this object.get_stac_objects
(rel[, typ, modify_links])Gets the
STACObject
instances that are linked to by links with theirrel
property matching the passed in argument.Recursively makes all the HREFs of assets in this catalog absolute
Recursively makes all the HREFs of assets in this catalog relative
Modify each asset's HREF to be absolute.
Modify each asset's HREF to be relative to this collection's self HREF.
map_assets
(asset_mapper)Creates a copy of a catalog, with each Asset for each Item passed through the asset_mapper function.
map_items
(item_mapper)Creates a copy of a catalog, with each item passed through the item_mapper function.
Returns a boolean indicating whether the given dictionary represents a valid instance of this
STACObject
sub-class.normalize_and_save
(root_href[, ...])Normalizes link HREFs to the given root_href, and saves the catalog.
normalize_hrefs
(root_href[, strategy, ...])Normalize HREFs will regenerate all link HREFs based on an absolute root_href and the canonical catalog layout as specified in the STAC specification's best practices.
remove_child
(child_id)Removes an child from this catalog.
remove_hierarchical_links
([add_canonical])Removes all hierarchical links from this object.
remove_item
(item_id)Removes an item from this catalog.
remove_links
(rel)Remove links to this object's set of links that match the given
rel
.Ensure all STACObjects linked to by this STACObject are resolved.
save
([catalog_type, dest_href, stac_io])Save this catalog and all it's children/item to files determined by the object's self link HREF or a specified path.
save_object
([include_self_link, dest_href, ...])Saves this STAC Object to it's 'self' HREF.
set_parent
(parent)Sets the parent
Catalog
orCollection
for this object.set_root
(root)Sets the root
Catalog
orCollection
for this object.set_self_href
(href)Sets the absolute HREF that is represented by the
rel == 'self'
Link
.target_in_hierarchy
(target)Determine if target is somewhere in the hierarchical link tree of a STACObject.
to_dict
()Returns this object as a dictionary.
Update datetime and bbox based on all items to a single bbox and time window.
validate
()Validate this STACObject.
validate_all
([max_items, recursive])Validates each catalog, collection, item contained within this catalog.
walk
()Walks through children and items of catalogs.
find
from_stac_collection
is_relative
list_available_keys
- classmethod from_dict(dico: dict)[source]#
Parses this STACObject from the passed in dictionary.
- Args:
d : The dict to parse. href : Optional href that is the file location of the object being
parsed.
- rootOptional root catalog for this object.
If provided, the root of the returned STACObject will be set to this parameter.
- migrate: Use True if this dict represents JSON from an older STAC object,
so that migrations are run against it.
- preserve_dict: If False, the dict parameter
d
may be modified during this method call. Otherwise the dict is not mutated. Defaults to True, which results results in a deepcopy of the parameter. Set to False when possible to avoid the performance hit of a deepcopy.
- Returns:
STACObject: The STACObject parsed from this dict.
- to_dict()[source]#
Returns this object as a dictionary.
- Args:
- include_self_linkIf True, the dict will contain a self link
to this object. If False, the self link will be omitted.
- transform_hrefs: If True, transform the HREF of hierarchical links
based on the type of catalog this object belongs to (if any). I.e. if this object belongs to a root catalog that is RELATIVE_PUBLISHED or SELF_CONTAINED, hierarchical link HREFs will be transformed to be relative to the catalog root.
dict: A serialization of the object.
- class pygeodes.utils.stac.Item(*args, **kwargs)[source]#
Bases:
Item
- Attributes:
common_metadata
Access the item’s common metadata fields as a
CommonMetadata
object.- data_asset
- data_asset_checksum
- filesize
- quicklook_asset
- s3_path
self_href
Gets the absolute HREF that is represented by the
rel == 'self'
Link
.
Methods
add_asset
(key, asset)Adds an Asset to this item.
add_derived_from
(*items)Add one or more items that this is derived from.
add_link
(link)Add a link to this object's set of links.
add_links
(links)Add links to this object's set of links.
clear_links
([rel])Clears all
Link
instances associated with this object.clone
()Clones this object.
delete_asset
(key)Deletes the asset at the given key, and removes the asset's data file from the local filesystem.
from_dict
(d[, href, root, migrate, ...])Parses this STACObject from the passed in dictionary.
from_file
(href[, stac_io])Reads a STACObject implementation from a file.
full_copy
([root, parent])Create a full copy of this STAC object and any stac objects linked to by this object.
get_assets
([media_type, role])Get this item's assets.
Gets the collection of this item, if one exists.
get_datetime
([asset])Gets an Item or an Asset datetime.
Get the items that this is derived from.
get_links
([rel, media_type])Gets the
Link
instances associated with this object.Get the
Catalog
orCollection
to the parent for this object.get_root
()Get the
Catalog
orCollection
to the root for this object.Get the
Link
representing the root for this object.Gets the absolute HREF that is represented by the
rel == 'self'
Link
.get_single_link
([rel, media_type])Get a single
Link
instance associated with this object.get_stac_objects
(rel[, typ, modify_links])Gets the
STACObject
instances that are linked to by links with theirrel
property matching the passed in argument.Modify each asset's HREF to be absolute.
Modify each asset's HREF to be relative to this item's self HREF.
Returns a boolean indicating whether the given dictionary represents a valid instance of this
STACObject
sub-class.remove_derived_from
(item_id)Remove an item that this is derived from.
remove_hierarchical_links
([add_canonical])Removes all hierarchical links from this object.
remove_links
(rel)Remove links to this object's set of links that match the given
rel
.Ensure all STACObjects linked to by this STACObject are resolved.
save_object
([include_self_link, dest_href, ...])Saves this STAC Object to it's 'self' HREF.
set_collection
(collection)Set the collection of this item.
set_datetime
(datetime[, asset])Set an Item or an Asset datetime.
set_parent
(parent)Sets the parent
Catalog
orCollection
for this object.set_root
(root)Sets the root
Catalog
orCollection
for this object.set_self_href
(href)Sets the absolute HREF that is represented by the
rel == 'self'
Link
.target_in_hierarchy
(target)Determine if target is somewhere in the hierarchical link tree of a STACObject.
to_dict
()Returns this object as a dictionary.
validate
()Validate this STACObject.
download_archive
find
get_quicklook_content_in_base64
list_available_keys
show_quicklook
- property data_asset#
- property data_asset_checksum#
- property filesize#
- property quicklook_asset#
- property s3_path#
- to_dict()[source]#
Returns this object as a dictionary.
- Args:
- include_self_linkIf True, the dict will contain a self link
to this object. If False, the self link will be omitted.
- transform_hrefs: If True, transform the HREF of hierarchical links
based on the type of catalog this object belongs to (if any). I.e. if this object belongs to a root catalog that is RELATIVE_PUBLISHED or SELF_CONTAINED, hierarchical link HREFs will be transformed to be relative to the catalog root.
dict: A serialization of the object.