libera_utils.aws.constants#
AWS ECR Repository/Algorithm names
Classes
|
Enum of valid CK objects |
|
Data product level |
|
Enumeration of data product canonical IDs used in AWS resource naming These IDs refer to the data products (files) themselves, NOT the processing steps (since processing steps may produce multiple products). |
|
APIDs for L0 packets |
|
Enumerated legal manifest type values |
|
Enumeration of processing step IDs used in AWS resource naming and processing orchestration |
|
Enum of valid SPK objects |
- class libera_utils.aws.constants.CkObject(value)#
Enum of valid CK objects
- property data_product_id: DataProductIdentifier#
DataProductIdentifier for CKs associated with this CK object
- property processing_step_id: ProcessingStepIdentifier#
ProcessingStepIdentifier for the processing step that produces CKs for this CK object
- class libera_utils.aws.constants.DataLevel(value)#
Data product level
- class libera_utils.aws.constants.DataProductIdentifier(value)#
Enumeration of data product canonical IDs used in AWS resource naming These IDs refer to the data products (files) themselves, NOT the processing steps (since processing steps may produce multiple products).
In general these names are of the form <level>-<source>-<type>
- dump(chunk_number: int | None = None) str#
Convert the DataProductIdentifier to a string suitable for matching with a DAG key or in the processing orchestration system.
The chunk_number can be specified when the data product represents a PDS file that is typically provided in 12 2-hour chunks per day. In that case, the chunk_number appears as a suffix to the orchestration product name.
- classmethod validate(product_name: str) tuple[DataProductIdentifier, int | None]#
Validate a product name string used by the DAG or the processing orchestration system.
If successful, returns a tuple containing the DataProductIdentifier and the chunk_number, which can be None if the input string does not contain a valid chunk number.
- class libera_utils.aws.constants.LiberaApid(value)#
APIDs for L0 packets
- class libera_utils.aws.constants.ManifestType(value)#
Enumerated legal manifest type values
- class libera_utils.aws.constants.ProcessingStepIdentifier(value)#
Enumeration of processing step IDs used in AWS resource naming and processing orchestration
- In orchestration code, these are used as “NodeID” values to identify processing steps:
The processing_step_node_id values used in libera_cdk deployment stackbuilder module and the node names in processing_system_dag.json must match these.
They must also be passed to the ecr_upload module called by some libera_cdk integration tests.
- dump(chunk_number: int | None = None) str#
Convert the ProcessingStepIdentifier to a string suitable for matching with a DAG key or in the processing orchestration system.
The chunk_number can be specified when the data product represents a PDS file that is typically provided in 12 2-hour chunks per day. In that case, the chunk_number appears as a suffix to the orchestration step identifier
- property ecr_name: str | None#
Get the manually-configured ECR name for this processing step
We name our ECRs in CDK because they are one of the few resources that humans will need to interact with on a regular basis.
- classmethod validate(processing_step: str) tuple[ProcessingStepIdentifier, int | None]#
Validate a processing step string used by the DAG or the orchestration system.
If successful, returns a tuple containing the ProcessingStepIdentifier and the chunk_number, which can be None if the input string does not contain a valid chunk number.
- class libera_utils.aws.constants.SpkObject(value)#
Enum of valid SPK objects
- property data_product_id: DataProductIdentifier#
DataProductIdentifier for SPKs associated with this SPK object
- property processing_step_id: ProcessingStepIdentifier#
ProcessingStepIdentifier for the processing step that produces SPKs for this SPK object