tierkreis.worker.storage.protocol

Storage protocol for workers.

Module Contents

Classes

WorkerStorage

Storage protocol for workers.

API

class tierkreis.worker.storage.protocol.WorkerStorage

Bases: typing.Protocol

Storage protocol for workers.

Abstract:

resolve(path: pathlib.Path | str) → pathlib.Path

Resolve a path or str to a path in the storage.

Transforms paths such that the storage can correctly read/write to them. E.g. for a file storage, relative paths are resolved against a base directory.

Parameters:

path (Path | str) – The path to resolve.

Returns:

The resolved path according to the storage.

Return type:

Path

read_call_args(path: pathlib.Path) → tierkreis.controller.data.location.WorkerCallArgs

Read the call args of a worker.

The function name is part of the call args which is then used to determine which function to call.

Parameters:

path (Path) – The path to read from.

Returns:

The call args of the worker function.

Return type:

WorkerCallArgs

read_input(path: pathlib.Path) → bytes

Read the input to a worker task.

Input locations are defined in the call args.

Parameters:

path (Path) – The path to read from.

Returns:

The bytes read from the input location.

Return type:

bytes

write_output(path: pathlib.Path, value: bytes) → None

Write the outputs of a worker task.

Output locations are defined in the call args.

Parameters:

  • path (Path) – The path to write to.

  • value (bytes) – The bytes to write.

glob(path_string: str) → list[str]

Get a list of paths matching the path glob.

Used in map nodes to find all input values for the individual tasks.

Parameters:

path_string (str) – The glob string to match.

Returns:

A list of matching path strings.

Return type:

list[str]

mark_done(path: pathlib.Path) → None

Mark the task node as done.

Done paths are defined in the call args.

Parameters:

path (Path) – The path to mark as done.

write_error(path: pathlib.Path, error_logs: str) → None

Write an error to the logs.

Logs are stored in a location defined in the call args.

Parameters:

  • path (Path) – The path to write the error logs to.

  • error_logs (str) – The message to write.

__slots__ = ()
classmethod __init_subclass__(*args, **kwargs)
classmethod __class_getitem__(item)