From Roadmap to Runtime: Model Packaging and Inference in PhenoWorks

Last month, we outlined a roadmap for bringing trained models into LgoPy workflows. That design has now reached its first working implementation in PhenoWorks.

A researcher can register a packaged inference service, validate it, deploy it in an isolated container, and use it from an LgoPy block through an authenticated gateway. The block refers to a cataloged model version; it never needs to carry an endpoint, credential, or container detail in the analysis code.

This is more than a new API route. The implementation gives catalog records, deployments, routing, and workflows separate contracts. The first release includes:

• Resumable Bento ZIP uploads and asynchronous structural validation.
• An internal catalog that separates immutable model packages from running deployments.
• Persisted background operations for deploy, stop, and delete actions.
• OCI image builds and isolated Bento containers on a self-hosted Docker host.
• Stable inference routes through Traefik and the PhenoWorks API.
• An authenticated gateway with request, response, header, and path controls.
• A portable LgoPy ModelResolver contract injected at execution time.
• Schema-driven model selection in the workflow interface.

The scope is deliberately narrow: externally trained models, packaged as Bentos, running on a single Docker host. Training orchestration, autoscaling, Kubernetes scheduling, and cluster-wide GPU allocation are still future work.

The implemented architecture

The architecture has two paths: one for managing services and another for using them during analysis.

The control path prepares and manages a model service. The inference path resolves that service for one LgoPy pipeline run and returns its outputs to the workflow.

The paths meet at a deployment, but their responsibilities do not blur. Cataloging and deployment are asynchronous platform operations. Invocation is a run-scoped capability that LgoPy receives only after PhenoWorks has resolved the deployments available to that run.

Three resources, three lifecycles

The implementation keeps a model package, a deployment, and a workflow reference separate:

Resource	Responsibility
Model package	Stores the immutable Bento archive, version, ownership, extracted metadata, validation state, and validation log.
Deployment	Records the package version, operation state, image and container identity, internal endpoint, readiness, and activity log.
Workflow model reference	Stores the stable catalog identity selected for a block without storing a container address or credential.

These distinctions matter in routine use. Uploading a package does not quietly start a container, and stopping a deployment does not erase the package that created it. A saved workflow can keep the same model reference even when an administrator later replaces the underlying container.

PhenoWorks remains the source of truth for ownership, authorization, lifecycle, routing, and workflow use. BentoML contributes the portable service format and image-building machinery; its local model store is not the PhenoWorks catalog.

A minimal BentoML service

A Bento starts with an ordinary Python service. For example, this small service turns two pixel counts into a canopy-cover estimate:

service.py

import bentoml
from pydantic import BaseModel


class CanopySample(BaseModel):
    green_pixels: int
    total_pixels: int


@bentoml.service(name="canopy-cover")
class CanopyCover:
    @bentoml.api
    def predict(self, sample: CanopySample) -> dict[str, float]:
        if sample.total_pixels <= 0:
            raise ValueError("total_pixels must be greater than zero")

        return {"canopy_cover": sample.green_pixels / sample.total_pixels}

Its bentofile.yaml identifies the service and records the environment needed to run it:

bentofile.yaml

service: "service:CanopyCover"
include:
  - "service.py"
python:
  packages:
    - "bentoml>=1.4.39,<1.5"
    - "pydantic>=2,<3"

From that directory, BentoML can build and export an archive that PhenoWorks can accept through the Models page:

bentoml build --version 0.1.0
bentoml export canopy-cover:0.1.0 canopy-cover-0.1.0.zip

Real services will usually load model artifacts and perform richer validation or preprocessing. The packaging boundary stays the same: the archive carries the serving code, its dependencies, and a typed inference API. PhenoWorks takes responsibility once that archive is uploaded.

Package intake and validation

Package intake begins when the Models API registers an upload and returns a scoped permit for the resumable transfer service. Once the browser reports that the transfer is complete, PhenoWorks verifies the upload record and queues a background validation job. At no point does the API process import package code.

Every archive is treated as untrusted data. Before allowing deployment, the validator checks:

• Archive paths, duplicate members, symbolic links, and encrypted entries.
• Compressed and expanded size limits and suspicious compression ratios.
• Required Bento files and the generated package structure.
• Service declarations and references to packaged model artifacts.
• Package name and version consistency.

The generated bento.yaml is authoritative for the built service structure and version. Human-facing catalog fields—display name, authors, license, and description—come from the packaged src/bentofile.yaml. PhenoWorks persists the result as pending, validating, valid, or invalid, along with a log that can be inspected in the interface.

Static validation does not prove that inference works. It only establishes that the archive is structurally acceptable before a deployment worker executes it.

Deployment as a persisted background operation

Only a validated package can enter the deployment lifecycle. The API creates a deployment record and a durable operation; a catalog worker then claims the job and:

1. Materializes the immutable archive from the upload store.
2. Imports the Bento into the worker's Bento store.
3. Builds a versioned OCI image.
4. Records the image reference and digest.
5. Starts a deployment-specific container.
6. Waits for the service readiness endpoint.
7. Marks the deployment available only after readiness succeeds.

Each deployment has a stable database identifier and a deterministic container identity. The runtime enforces memory, CPU, process, platform, and shutdown limits. It also drops Linux capabilities, enables no-new-privileges, and connects the container to the internal model network without giving it platform credentials.

Build, startup, readiness, and failure events accumulate in a durable log. Administrators can therefore examine what happened without relying on ephemeral worker output. Stop and delete also run in the background: stopping preserves history, while deletion removes the stopped container and image when the state machine permits it.

Stable routing without exposing containers

The deployment worker labels each container for Traefik. A route such as /deployments/{deployment_id} reaches the model's Bento server after Traefik removes the prefix. Container names and addresses never become part of the public contract.

External callers use the authenticated PhenoWorks route:

/api/models/deployments/{deployment_id}/invoke/{inference_path}

Before opening an upstream request, the gateway confirms that the deployment is available. It then:

• Preserves the HTTP method, query string, body, and safe model-specific headers.
• Removes credentials, cookies, forwarding headers, and hop-by-hop headers.
• Blocks access to Bento control paths such as metrics, health, documentation, and OpenAPI endpoints.
• Enforces configured request and response size limits.
• Uses fixed internal routing instead of accepting an arbitrary upstream URL.
• Streams the model response and closes the upstream connection when complete.

Timeouts and unavailable services become explicit gateway errors. Authentication and network policy consequently stay in PhenoWorks instead of being reimplemented inside every model service.

A small, portable LgoPy contract

LgoPy does not need to understand package uploads or Docker management. It only defines the portable pieces a block needs: ModelReference, ModelResponse, and the ModelResolver protocol.

A model-backed block depends on that protocol:

class ModelInference(Block):
    def __init__(self, model: ModelReference) -> None:
        super().__init__()
        self.model = model

    def call(self, payload: dict[str, object]) -> dict[str, object]:
        response = self.models.invoke(
            self.model,
            "predict",
            json={"inputs": payload},
        )
        response.raise_for_status()
        return response.json()

The block states which model it needs and translates its data into an inference request. It does not instantiate a PhenoWorks client, read an API key, or look up a container hostname. During development, the block can use a local HTTP resolver; in a managed pipeline, it receives the PhenoWorks resolver instead.

The contract covers JSON, raw content, forms, multipart files, query parameters, and safe request headers. A framework-neutral response wrapper keeps FastAPI, HTTPX, and BentoML out of LgoPy core.

Resolution happens when a pipeline run starts

During pipeline preparation, PhenoWorks builds a resolver for that run. It queries deployments in the deployed state, converts them to stable model references, and snapshots the mapping between each reference and its deployment identifier.

The pipeline receives this snapshot alongside its artifact and metadata stores; blocks access it through self.models. If a model was not available when the run began, block code cannot reach it by substituting a different deployment identifier.

For each invocation, the PhenoWorks resolver:

• Looks up the stable reference in the run's deployment allowlist.
• Normalizes and validates the requested inference path and HTTP method.
• Rejects credentials and private control paths supplied by the block.
• Sends the request through the fixed internal gateway origin.
• Applies configured request and response limits.
• Reuses a bounded HTTP connection pool for the run.

That boundary keeps model execution portable without surrendering application control over authorization or runtime policy.

The model picker comes from the block schema

ModelReference is an annotated string with format: model-reference in its JSON schema. When the workflow interface sees that format, it renders a catalog selector rather than a free-text input.

The selector lists only deployments returned by the available-models endpoint. The saved block configuration holds the stable reference; run preparation later resolves it against the deployment snapshot. In this way, one schema field ties together the interface, serialized workflow, and runtime contract without hard-coding any model family into the form.

The available-model response also includes validated service metadata and each declared inference API. Filtering that list by the API a block requires is a useful next step. For now, the selector shows all available deployments.

What is implemented, and what remains

The implemented feature covers the first single-host path from a Bento archive to a model-backed LgoPy workflow:

upload → validate → deploy → discover → resolve → invoke → save outputs

The most consequential result is not any single endpoint. It is the set of explicit boundaries around package trust, deployment state, network access, workflow configuration, and runtime dependency injection. A new service can now follow the same catalog and execution path without adding model-specific deployment logic to PhenoWorks.

The next stage is operational hardening, not another redesign. Priorities include stronger isolation during package builds, richer compatibility filters, complete model provenance in exported run reports, GPU-capacity coordination, restart and recovery tests, and more packaging templates.

Training remains outside this feature. Researchers are free to use the training libraries and compute environment that suit their work, then package the inference method as a Bento. From upload onward, PhenoWorks manages how that service is validated, deployed, selected, invoked, and connected to a reproducible LgoPy workflow.