libera_utils.logutil.configure_task_logging#

Configure logging for a specific task (e.g. a processing algorithm).

File-based logging is always done at the DEBUG level. Watchtower-based cloudwatch logging is always done at the DEBUG level. Console logging level defaults to INFO but can be set with console_log_level.

Examples

Example 1: The following will configure DEBUG console-only logging for anything in your script but all other loggers will be limited to INFO level.

`python configure_task_logging("my-script", limit_debug_loggers=("__main__",), console_log_level=logging.DEBUG) `

Example 2: This will allow all debug messages through from all loggers and sets up file-based logging and a custom cloudwatch log group. Also console messages will be logged in serialized JSON.

```python configure_task_logging(“my-script”,

console_log_level=logging.DEBUG, log_dir=Path(“/tmp/my-script”), console_log_json=True, cloudwatch_log_group=”custom-log-group”)

```

Parameters:

task_id (str) – Unique identifier by which to name the log file and cloudwatch log stream.
limit_debug_loggers (Optional[Union[Iterable[str] | str]]) – A list of logger name prefixes from which you want to allow debug messages (blocks debug from all others). For example, if you are working on a package called my_app and using module level logging, all your loggers will be named like my_app.module_name.submodule_name. By setting this to (my_app,), all loggers that are named my_app.* will propagate debug messages while preventing spammy debug messages from installed libraries like boto3. If this is empty or None, all debug messages will propagate. To use this in scripts, either leave it unset or use limit_debug_loggers=(“__main__,).
console_log_level (str or int, Optional) – Log level for console logging. If not specified, defaults to INFO
console_log_json (bool, Optional) – If True, console logs will be JSON formatted. This is suitable for setting up loggers in AWS services that are automatically monitored by cloudwatch on stdout and stderr (e.g. Lambda or Batch)
log_dir (str or Path or S3Path, Optional) – Log directory, which may be a local or S3Path. Default is None and results in no file-based logging.
cloudwatch_log_group (str, Optional) – Override optional environment variable log group name. Default is None and will result in falling back to the LIBERA_LOG_GROUP environment variable. If that is not set, no cloudwatch JSON logging will be configured.

Notes

Even in the absence of cloudwatch JSON logging, all stdout/stderr messages generated by a Lambda will be logged to CloudWatch as string messages. Embedded JSON strings in log message text can still be queried in CloudWatch.