libera_utils.io.filenaming.ManifestFilename#

class libera_utils.io.filenaming.ManifestFilename(*args, **kwargs)#

Bases: AbstractValidFilename

Class for naming manifest files

Attributes:

archive_prefix: Manifests are not archived like data products, but for convenience and ease of debugging they will be kept in the dropbox bucket by input/output and day they were made.
filename_parts: Property that contains a namespace of filename parts
path: Property containing the file path

Methods

`from_filename_parts`([basepath, ...])	Create instance from filename parts.
`generate_prefixed_path`(parent_path)	Generates an absolute path of the form {parent_path}/{prefix_structure}/{file_basename} The parent_path can be an S3 bucket or an absolute local filepath (must start with /)
`regex_match`(path)	Parse and validate a given path against class-attribute defined regex

__init__(*args, **kwargs)#

Methods

`from_filename_parts`([basepath, ...])	Create instance from filename parts.
`generate_prefixed_path`(parent_path)	Generates an absolute path of the form {parent_path}/{prefix_structure}/{file_basename} The parent_path can be an S3 bucket or an absolute local filepath (must start with /)
`regex_match`(path)	Parse and validate a given path against class-attribute defined regex

Attributes

`archive_prefix`	Manifests are not archived like data products, but for convenience and ease of debugging they will be kept in the dropbox bucket by input/output and day they were made.
`filename_parts`	Property that contains a namespace of filename parts
`path`	Property containing the file path

static _calculate_applicable_time(start: datetime, end: datetime)#

Based on the start time and end time of a file, returns the applicable time (date)

Parameters:

start (datetime.datetime) – Start of the applicable time range
end (datetime.datetime) – End of the applicable time range

Returns:

The date of the mean time between start and end

Return type:

datetime.date

classmethod _check_required_parts(local_vars: dict)#

Checks for the presence of required filename parts

Parameters:: local_vars (dict) – Dictionary of variables passed, created by a call to locals()

classmethod _format_filename_parts(manifest_type: ManifestType, ulid_code: ULID)#

Construct a path from filename parts

Parameters:

manifest_type (ManifestType) – Input or output
ulid_code (ulid.ULID) – ULID code for use in filename parts

Returns:

Formatted filename

Return type:

str

classmethod _from_filename_parts(basepath: str = None, **parts)#

Create instance from filename parts.

The part arg names are named according to the regex for the file type.

Parameters:

basepath (str or pathlib.Path, Optional) – Allows prepending a basepath or prefix.
parts (dict) – Passed directly to _format_filename_parts

Return type:

AbstractValidFilename

_parse_filename_parts()#

Parse the filename parts into objects from regex matched strings

Returns:: namespace object containing filename parts as parsed objects
Return type:: types.SimpleNamespace

property archive_prefix#: Manifests are not archived like data products, but for convenience and ease of debugging they will be kept in the dropbox bucket by input/output and day they were made. This is used by the step function clean up function in the CDK. # Generate prefix structure # <manifest_type>/<year>/<month>/<day>

property filename_parts#: Property that contains a namespace of filename parts

classmethod from_filename_parts(basepath: str = None, manifest_type: ManifestType = None, ulid_code: ULID = None)#

Create instance from filename parts. All keyword arguments other than basepath are required!

The part names are named according to the regex for the file type.

Parameters:

basepath (str or pathlib.Path, Optional) – Allows prepending a basepath or prefix.
manifest_type (ManifestType) – Input or output
ulid_code (ulid.ULID) – ULID code for use in filename parts

Return type:

ManifestFilename

generate_prefixed_path(parent_path: str | Path | S3Path) → Path | S3Path#

Generates an absolute path of the form {parent_path}/{prefix_structure}/{file_basename} The parent_path can be an S3 bucket or an absolute local filepath (must start with /)

Parameters:: parent_path (str or pathlib.Path or cloudpathlib.s3.s3path.S3Path) – Absolute path to the parent directory or S3 bucket prefix. The generated path prefix is appended to the parent path and followed by the file basename.
Return type:: pathlib.Path or cloudpathlib.s3.s3path.S3Path

property path#: Property containing the file path

regex_match(path: str)#

Parse and validate a given path against class-attribute defined regex

Returns:: Match group dict of filename parts
Return type:: dict