libera_utils.constants

Constants module used throughout the libera_utils package

Classes

DataLevel(value)	Data product level
DataProductIdentifier(value)	Enumeration of data product canonical IDs used in AWS resource naming.
LiberaApid(value)	APIDs for packets
ManifestType(value)	Enumerated legal manifest type values
ProcessingStepIdentifier(value)	Enumeration of processing step IDs used in AWS resource naming and processing orchestration.

class libera_utils.constants.DataLevel(value)

Data product level

Methods

capitalize(/)	Return a capitalized version of the string.
casefold(/)	Return a version of the string suitable for caseless comparisons.
center(width[, fillchar])	Return a centered string of length width.
count(sub[, start[, end]])	Return the number of non-overlapping occurrences of substring sub in string S[start:end].
encode(/[, encoding, errors])	Encode the string using the codec registered for encoding.
endswith(suffix[, start[, end]])	Return True if S ends with the specified suffix, False otherwise.
expandtabs(/[, tabsize])	Return a copy where all tab characters are expanded using spaces.
find(sub[, start[, end]])	Return the lowest index in S where substring sub is found, such that sub is contained within S[start:end].
format(*args, **kwargs)	Return a formatted version of S, using substitutions from args and kwargs.
format_map(mapping)	Return a formatted version of S, using substitutions from mapping.
index(sub[, start[, end]])	Return the lowest index in S where substring sub is found, such that sub is contained within S[start:end].
isalnum(/)	Return True if the string is an alpha-numeric string, False otherwise.
isalpha(/)	Return True if the string is an alphabetic string, False otherwise.
isascii(/)	Return True if all characters in the string are ASCII, False otherwise.
isdecimal(/)	Return True if the string is a decimal string, False otherwise.
isdigit(/)	Return True if the string is a digit string, False otherwise.
isidentifier(/)	Return True if the string is a valid Python identifier, False otherwise.
islower(/)	Return True if the string is a lowercase string, False otherwise.
isnumeric(/)	Return True if the string is a numeric string, False otherwise.
isprintable(/)	Return True if the string is printable, False otherwise.
isspace(/)	Return True if the string is a whitespace string, False otherwise.
istitle(/)	Return True if the string is a title-cased string, False otherwise.
isupper(/)	Return True if the string is an uppercase string, False otherwise.
join(iterable, /)	Concatenate any number of strings.
ljust(width[, fillchar])	Return a left-justified string of length width.
lower(/)	Return a copy of the string converted to lowercase.
lstrip([chars])	Return a copy of the string with leading whitespace removed.
maketrans(x[, y, z])	Return a translation table usable for str.translate().
partition(sep, /)	Partition the string into three parts using the given separator.
removeprefix(prefix, /)	Return a str with the given prefix string removed if present.
removesuffix(suffix, /)	Return a str with the given suffix string removed if present.
replace(old, new[, count])	Return a copy with all occurrences of substring old replaced by new.
rfind(sub[, start[, end]])	Return the highest index in S where substring sub is found, such that sub is contained within S[start:end].
rindex(sub[, start[, end]])	Return the highest index in S where substring sub is found, such that sub is contained within S[start:end].
rjust(width[, fillchar])	Return a right-justified string of length width.
rpartition(sep, /)	Partition the string into three parts using the given separator.
rsplit(/[, sep, maxsplit])	Return a list of the substrings in the string, using sep as the separator string.
rstrip([chars])	Return a copy of the string with trailing whitespace removed.
split(/[, sep, maxsplit])	Return a list of the substrings in the string, using sep as the separator string.
splitlines(/[, keepends])	Return a list of the lines in the string, breaking at line boundaries.
startswith(prefix[, start[, end]])	Return True if S starts with the specified prefix, False otherwise.
strip([chars])	Return a copy of the string with leading and trailing whitespace removed.
swapcase(/)	Convert uppercase characters to lowercase and lowercase characters to uppercase.
title(/)	Return a version of the string where each word is titlecased.
translate(table, /)	Replace each character in the string using the given translation table.
upper(/)	Return a copy of the string converted to uppercase.
zfill(width, /)	Pad a numeric string with zeros on the left, to fill a field of the given width.

property archive_bucket_name: str

Gets the archive bucket name for the data level.

Notes

This does not include any account suffix, which must be added separately.

class libera_utils.constants.DataProductIdentifier(value)

Enumeration of data product canonical IDs used in AWS resource naming.

These IDs refer to the data products (files) themselves, NOT the processing steps (since processing steps may produce multiple products). The string values are the product names used in filenames.

This enum replaces the old ProductName enum from filenaming.py to provide a single source of truth.

Each member is defined as a tuple: (product_name, data_level) - product_name: The string value used in filenames and AWS resources - data_level: The DataLevel enum value indicating the processing level

Example

>>> product = DataProductIdentifier.l1b_rad
>>> str(product)  # Returns "RAD-4CH"
>>> product.level  # Returns DataLevel.L1B
>>> product.level.archive_bucket_name  # Returns "libera-l1b-data"

When adding new products:

1. Add the enum member with its product name and DataLevel

2. No need to update any lookup dictionaries - metadata is embedded!

Methods

capitalize(/)	Return a capitalized version of the string.
casefold(/)	Return a version of the string suitable for caseless comparisons.
center(width[, fillchar])	Return a centered string of length width.
count(sub[, start[, end]])	Return the number of non-overlapping occurrences of substring sub in string S[start:end].
encode(/[, encoding, errors])	Encode the string using the codec registered for encoding.
endswith(suffix[, start[, end]])	Return True if S ends with the specified suffix, False otherwise.
expandtabs(/[, tabsize])	Return a copy where all tab characters are expanded using spaces.
find(sub[, start[, end]])	Return the lowest index in S where substring sub is found, such that sub is contained within S[start:end].
format(*args, **kwargs)	Return a formatted version of S, using substitutions from args and kwargs.
format_map(mapping)	Return a formatted version of S, using substitutions from mapping.
index(sub[, start[, end]])	Return the lowest index in S where substring sub is found, such that sub is contained within S[start:end].
isalnum(/)	Return True if the string is an alpha-numeric string, False otherwise.
isalpha(/)	Return True if the string is an alphabetic string, False otherwise.
isascii(/)	Return True if all characters in the string are ASCII, False otherwise.
isdecimal(/)	Return True if the string is a decimal string, False otherwise.
isdigit(/)	Return True if the string is a digit string, False otherwise.
isidentifier(/)	Return True if the string is a valid Python identifier, False otherwise.
islower(/)	Return True if the string is a lowercase string, False otherwise.
isnumeric(/)	Return True if the string is a numeric string, False otherwise.
isprintable(/)	Return True if the string is printable, False otherwise.
isspace(/)	Return True if the string is a whitespace string, False otherwise.
istitle(/)	Return True if the string is a title-cased string, False otherwise.
isupper(/)	Return True if the string is an uppercase string, False otherwise.
join(iterable, /)	Concatenate any number of strings.
ljust(width[, fillchar])	Return a left-justified string of length width.
lower(/)	Return a copy of the string converted to lowercase.
lstrip([chars])	Return a copy of the string with leading whitespace removed.
maketrans(x[, y, z])	Return a translation table usable for str.translate().
partition(sep, /)	Partition the string into three parts using the given separator.
removeprefix(prefix, /)	Return a str with the given prefix string removed if present.
removesuffix(suffix, /)	Return a str with the given suffix string removed if present.
replace(old, new[, count])	Return a copy with all occurrences of substring old replaced by new.
rfind(sub[, start[, end]])	Return the highest index in S where substring sub is found, such that sub is contained within S[start:end].
rindex(sub[, start[, end]])	Return the highest index in S where substring sub is found, such that sub is contained within S[start:end].
rjust(width[, fillchar])	Return a right-justified string of length width.
rpartition(sep, /)	Partition the string into three parts using the given separator.
rsplit(/[, sep, maxsplit])	Return a list of the substrings in the string, using sep as the separator string.
rstrip([chars])	Return a copy of the string with trailing whitespace removed.
split(/[, sep, maxsplit])	Return a list of the substrings in the string, using sep as the separator string.
splitlines(/[, keepends])	Return a list of the lines in the string, breaking at line boundaries.
startswith(prefix[, start[, end]])	Return True if S starts with the specified prefix, False otherwise.
strip([chars])	Return a copy of the string with leading and trailing whitespace removed.
swapcase(/)	Convert uppercase characters to lowercase and lowercase characters to uppercase.
title(/)	Return a version of the string where each word is titlecased.
translate(table, /)	Replace each character in the string using the given translation table.
upper(/)	Return a copy of the string converted to uppercase.
zfill(width, /)	Pad a numeric string with zeros on the left, to fill a field of the given width.

property data_level: DataLevel

Get the processing level for this data product.

Returns:

The processing level of this data product

Return type:

DataLevel

get_partial_archive_bucket_name() → str

Gets the archive bucket name from the data product identifier .

Buckets are named according to the level of data they are storing and which account they are in. This is expected to be used by the L2 developers who will most commonly be working with the stage account.

Returns:

The name of the archive bucket for this data product without an account suffix

Return type:

str

property product_name: str

Get the name formatted for AWS resources for this data product

The name is used to create AWS resources that are specific to the data product. This is an alias to the string value for compatibility.

class libera_utils.constants.LiberaApid(value)

APIDs for packets

The enum names here should be of the form <packet-source>_<system>_<contents>. e.g. for icie_rad_sample: packet_source is Libera “ICIE”, the system is the “Radiometers”, and the contents is radiometer “Samples”.

Notes

This is useful for identifying the data product type from the APID in an L0 filename.

The enum names (e.g. icie_seq_hk) for Libera packets here should be precisely the packet names used in FSW documents and packet definitions.

Attributes:

denominator

the denominator of a rational number in lowest terms

imag

the imaginary part of a complex number

numerator

the numerator of a rational number in lowest terms

real

the real part of a complex number

Methods

as_integer_ratio(/)	Return integer ratio.
bit_count(/)	Number of ones in the binary representation of the absolute value of self.
bit_length(/)	Number of bits necessary to represent self in binary.
conjugate	Returns self, the complex conjugate of any int.
from_bytes(/, bytes[, byteorder, signed])	Return the integer represented by the given array of bytes.
to_bytes(/[, length, byteorder, signed])	Return an array of bytes representing an integer.

property data_product_id: DataProductIdentifier

Get the DataProductIdentifier for L0 PDS files with this APID

This relies on the strict naming convention that the packet name is part of the L0 data product ID name

class libera_utils.constants.ManifestType(value)

Enumerated legal manifest type values

Methods

capitalize(/)	Return a capitalized version of the string.
casefold(/)	Return a version of the string suitable for caseless comparisons.
center(width[, fillchar])	Return a centered string of length width.
count(sub[, start[, end]])	Return the number of non-overlapping occurrences of substring sub in string S[start:end].
encode(/[, encoding, errors])	Encode the string using the codec registered for encoding.
endswith(suffix[, start[, end]])	Return True if S ends with the specified suffix, False otherwise.
expandtabs(/[, tabsize])	Return a copy where all tab characters are expanded using spaces.
find(sub[, start[, end]])	Return the lowest index in S where substring sub is found, such that sub is contained within S[start:end].
format(*args, **kwargs)	Return a formatted version of S, using substitutions from args and kwargs.
format_map(mapping)	Return a formatted version of S, using substitutions from mapping.
index(sub[, start[, end]])	Return the lowest index in S where substring sub is found, such that sub is contained within S[start:end].
isalnum(/)	Return True if the string is an alpha-numeric string, False otherwise.
isalpha(/)	Return True if the string is an alphabetic string, False otherwise.
isascii(/)	Return True if all characters in the string are ASCII, False otherwise.
isdecimal(/)	Return True if the string is a decimal string, False otherwise.
isdigit(/)	Return True if the string is a digit string, False otherwise.
isidentifier(/)	Return True if the string is a valid Python identifier, False otherwise.
islower(/)	Return True if the string is a lowercase string, False otherwise.
isnumeric(/)	Return True if the string is a numeric string, False otherwise.
isprintable(/)	Return True if the string is printable, False otherwise.
isspace(/)	Return True if the string is a whitespace string, False otherwise.
istitle(/)	Return True if the string is a title-cased string, False otherwise.
isupper(/)	Return True if the string is an uppercase string, False otherwise.
join(iterable, /)	Concatenate any number of strings.
ljust(width[, fillchar])	Return a left-justified string of length width.
lower(/)	Return a copy of the string converted to lowercase.
lstrip([chars])	Return a copy of the string with leading whitespace removed.
maketrans(x[, y, z])	Return a translation table usable for str.translate().
partition(sep, /)	Partition the string into three parts using the given separator.
removeprefix(prefix, /)	Return a str with the given prefix string removed if present.
removesuffix(suffix, /)	Return a str with the given suffix string removed if present.
replace(old, new[, count])	Return a copy with all occurrences of substring old replaced by new.
rfind(sub[, start[, end]])	Return the highest index in S where substring sub is found, such that sub is contained within S[start:end].
rindex(sub[, start[, end]])	Return the highest index in S where substring sub is found, such that sub is contained within S[start:end].
rjust(width[, fillchar])	Return a right-justified string of length width.
rpartition(sep, /)	Partition the string into three parts using the given separator.
rsplit(/[, sep, maxsplit])	Return a list of the substrings in the string, using sep as the separator string.
rstrip([chars])	Return a copy of the string with trailing whitespace removed.
split(/[, sep, maxsplit])	Return a list of the substrings in the string, using sep as the separator string.
splitlines(/[, keepends])	Return a list of the lines in the string, breaking at line boundaries.
startswith(prefix[, start[, end]])	Return True if S starts with the specified prefix, False otherwise.
strip([chars])	Return a copy of the string with leading and trailing whitespace removed.
swapcase(/)	Convert uppercase characters to lowercase and lowercase characters to uppercase.
title(/)	Return a version of the string where each word is titlecased.
translate(table, /)	Replace each character in the string using the given translation table.
upper(/)	Return a copy of the string converted to uppercase.
zfill(width, /)	Pad a numeric string with zeros on the left, to fill a field of the given width.

class libera_utils.constants.ProcessingStepIdentifier(value)

Enumeration of processing step IDs used in AWS resource naming and processing orchestration.

In orchestration code, these are used as “NodeID” values to identify processing steps:

The processing_step_node_id values used in libera_cdk deployment of processing steps and the node names in processing_system_dag.json must match these.

They must also be passed to the ecr_upload module called by some libera_cdk integration tests.

The string values are the processing step names used in orchestration.

Each member is defined as a tuple: (step_name, products_list) - step_name: The string value used in orchestration and AWS resources - products_list: List of DataProductIdentifier members that this step produces

Example

>>> step = ProcessingStepIdentifier.l1b_rad
>>> str(step)  # Returns "l1b-rad"
>>> step.products  # Returns [DataProductIdentifier.l1b_rad]
>>> step.level  # Returns DataLevel.L1B (derived from products)

When adding new processing steps:

1. Add the enum member with its step name and list of produced products

2. No need to update any lookup dictionaries - relationships are embedded!

Methods

capitalize(/)	Return a capitalized version of the string.
casefold(/)	Return a version of the string suitable for caseless comparisons.
center(width[, fillchar])	Return a centered string of length width.
count(sub[, start[, end]])	Return the number of non-overlapping occurrences of substring sub in string S[start:end].
encode(/[, encoding, errors])	Encode the string using the codec registered for encoding.
endswith(suffix[, start[, end]])	Return True if S ends with the specified suffix, False otherwise.
expandtabs(/[, tabsize])	Return a copy where all tab characters are expanded using spaces.
find(sub[, start[, end]])	Return the lowest index in S where substring sub is found, such that sub is contained within S[start:end].
format(*args, **kwargs)	Return a formatted version of S, using substitutions from args and kwargs.
format_map(mapping)	Return a formatted version of S, using substitutions from mapping.
index(sub[, start[, end]])	Return the lowest index in S where substring sub is found, such that sub is contained within S[start:end].
isalnum(/)	Return True if the string is an alpha-numeric string, False otherwise.
isalpha(/)	Return True if the string is an alphabetic string, False otherwise.
isascii(/)	Return True if all characters in the string are ASCII, False otherwise.
isdecimal(/)	Return True if the string is a decimal string, False otherwise.
isdigit(/)	Return True if the string is a digit string, False otherwise.
isidentifier(/)	Return True if the string is a valid Python identifier, False otherwise.
islower(/)	Return True if the string is a lowercase string, False otherwise.
isnumeric(/)	Return True if the string is a numeric string, False otherwise.
isprintable(/)	Return True if the string is printable, False otherwise.
isspace(/)	Return True if the string is a whitespace string, False otherwise.
istitle(/)	Return True if the string is a title-cased string, False otherwise.
isupper(/)	Return True if the string is an uppercase string, False otherwise.
join(iterable, /)	Concatenate any number of strings.
ljust(width[, fillchar])	Return a left-justified string of length width.
lower(/)	Return a copy of the string converted to lowercase.
lstrip([chars])	Return a copy of the string with leading whitespace removed.
maketrans(x[, y, z])	Return a translation table usable for str.translate().
partition(sep, /)	Partition the string into three parts using the given separator.
removeprefix(prefix, /)	Return a str with the given prefix string removed if present.
removesuffix(suffix, /)	Return a str with the given suffix string removed if present.
replace(old, new[, count])	Return a copy with all occurrences of substring old replaced by new.
rfind(sub[, start[, end]])	Return the highest index in S where substring sub is found, such that sub is contained within S[start:end].
rindex(sub[, start[, end]])	Return the highest index in S where substring sub is found, such that sub is contained within S[start:end].
rjust(width[, fillchar])	Return a right-justified string of length width.
rpartition(sep, /)	Partition the string into three parts using the given separator.
rsplit(/[, sep, maxsplit])	Return a list of the substrings in the string, using sep as the separator string.
rstrip([chars])	Return a copy of the string with trailing whitespace removed.
split(/[, sep, maxsplit])	Return a list of the substrings in the string, using sep as the separator string.
splitlines(/[, keepends])	Return a list of the lines in the string, breaking at line boundaries.
startswith(prefix[, start[, end]])	Return True if S starts with the specified prefix, False otherwise.
strip([chars])	Return a copy of the string with leading and trailing whitespace removed.
swapcase(/)	Convert uppercase characters to lowercase and lowercase characters to uppercase.
title(/)	Return a version of the string where each word is titlecased.
translate(table, /)	Replace each character in the string using the given translation table.
upper(/)	Return a copy of the string converted to uppercase.
zfill(width, /)	Pad a numeric string with zeros on the left, to fill a field of the given width.

property ecr_name: str | None

Get the manually-configured ECR name for this processing step

We name our ECRs in CDK because they are one of the few resources that humans will need to interact with on a regular basis.

classmethod from_data_product(data_product: DataProductIdentifier) → ProcessingStepIdentifier | None

Get the ProcessingStepIdentifier that produces the given DataProductIdentifier

Parameters:

data_product (DataProductIdentifier) – The data product to find the processing step for

Returns:

The processing step that produces this data product

Return type:

ProcessingStepIdentifier

Raises:

ValueError – If no processing step is found for the data product

get_archive_bucket_name(account_suffix: str | LiberaAccountSuffix = LiberaAccountSuffix.STAGE) → str | None

Gets the archive bucket name for this processing step.

Buckets are named according to the level of data they are storing and which account they are in. This is expected to be used by the L2 developers who will most commonly be working with the stage account.

Parameters:

account_suffix (str | LiberaAccountSuffix, optional) – Account suffix for the bucket name, by default LiberaAccountSuffix.STAGE (stage account). Can be a string like “-test” for custom testing scenarios.

Returns:

The name of the archive bucket for this processing step

Return type:

str

property level: DataLevel

Get the processing level of the products produced by this step

Raises:

ValueError – If the step produces no products or produces products of multiple levels

property policy_name: str

Get the name formatted IAM policy for this processing step

The policy name is used to create an IAM policy that grants permissions to the aspects of the processing step.

property processing_step_name: str

Get the name formatted for AWS resources for this processing step

The name is used to create AWS resources that are specific to the processing step. This is an alias to the string value for compatibility.

property products: list[DataProductIdentifier]

Get the list of data products produced by this processing step.

Returns:

List of data products produced by this processing step

Return type:

list[DataProductIdentifier]

property step_function_name

Get the name formatted for the step function for this processing step

The step function name is used to create a step function that orchestrates the processing step.