Skip to main content

How Operations Work​

To understand how operations function, it helps to consider the underlying technologies and concepts they leverage. Basic familiarity with Docker and Python is recommended before proceeding.

Operations serve as a bridge between external tools and the Stage2 platform by containerizing these tools and wrapping them in a consistent, standardized interface. This approach forms the foundation of Stage2’s tool-agnostic design—enabling users to integrate, automate, and scale sophisticated workflows without being tied to a specific technology stack.

Note: Include a graphic illustrating containerizing a tool and organizing those containers into a workflow.

Key Components of an Operation​

1. Containerization​

Each operation runs inside a Docker container. This container includes:

  • The software or tool the operation will execute.
  • All necessary dependencies to ensure a stable, consistent environment.
  • A custom Python script—stage2_interface.py—which acts as the operation’s entry point.

By encapsulating each tool in a container, operations remain portable and reliable, irrespective of the underlying infrastructure.

2. Standardized Inputs and Outputs​

Operations in Stage2 adhere to a standardized input-output model:

  • Inputs: Operations receive assets (e.g., files, datasets) and parameters (e.g., settings, configurations) in a structured JSON format. This format defines all the information required to execute the tool effectively.
  • Outputs: The results of the operation—such as processed files, data, or visualizations—are generated and stored in a specified location, ready for downstream operations in the workflow.

This standardization simplifies the integration of diverse tools and ensures that each operation can interact smoothly with others.

3. Stage2 Interface Script​

At the heart of each operation is a Python script, typically named stage2_interface.py. This script is responsible for:

  • Parsing and validating the inputs: It reads the structured JSON input provided by Stage2, ensuring all required parameters and assets are present and correctly formatted.
  • Executing the tool: It runs the underlying tool or software within the container, applying the provided parameters to generate the desired output.
  • Producing standardized outputs: After execution, it places the results in a known output directory or file, preparing them for subsequent operations in the workflow.

This script acts as the glue that binds the containerized tool to the Stage2 platform, ensuring a consistent execution pattern across all operations.

Orchestration by Stage2​

Once you define and containerize your operation, Stage2 takes care of:

  1. Invocation:
    Stage2 launches the operation’s Docker container and provides it with the required inputs (see operation modal).

  2. Monitoring:
    Stage2 tracks the operation’s execution, ensuring it completes successfully or handling errors if they occur.

  3. Routing Outputs:
    Once the operation finishes, Stage2 makes the operation’s outputs available to other nodes in the workflow graph, enabling seamless data flow between operations.