KnowForge/Testcontainers for Go
Quickstart

Docker Model Runner

qianmoQqianmoQ· 更新于 2026-10-14· 阅读 30 分钟· 0 次阅读
登录后可跨设备保存划线和私人笔记登录

Docker Model Runner

Since v0.37.0

Introduction

The Testcontainers module for DockerModelRunner.

Adding this module to your project dependencies

Please run the following command to add the DockerModelRunner module to your Go dependencies:

go get github.com/testcontainers/testcontainers-go/modules/dockermodelrunner

Usage example

Creating a DockerModelRunner container

ctx := context.Background()

const (
    modelNamespace = "ai"
    modelName      = "smollm2"
    modelTag       = "360M-Q4_K_M"
    fqModelName    = modelNamespace + "/" + modelName + ":" + modelTag
)

dmrCtr, err := dockermodelrunner.Run(
    ctx,
    dockermodelrunner.WithModel(fqModelName),
)
defer func() {
    if err := testcontainers.TerminateContainer(dmrCtr); err != nil {
        log.Printf("failed to terminate container: %s", err)
    }
}()
if err != nil {
    log.Printf("failed to start container: %s", err)
    return
}

Module Reference

Run function

The Docker Model Runner module exposes two entrypoint functions to create the Docker Model Runner container:

Run

This function receives two parameters:

func Run(ctx context.Context, opts ...testcontainers.ContainerCustomizer) (*Container, error)
  • context.Context, the Go context.
  • testcontainers.ContainerCustomizer, a variadic argument for passing options.

Info

This function will use the default socat image under the hood. Please refer to the socat module for more information.

Image

Use the second argument in the Run function to set a valid Docker image. In example: Run(context.Background(), "alpine/socat:1.8.0.1").

Container Options

When starting the Docker Model Runner container, you can pass options in a variadic way to configure it.

WithModel

Use the WithModel option to set the model to pull when the container is started. Please be aware, that only Models as OCI Artifacts are compatible with Docker Model Runner.

dockermodelrunner.WithModel("ai/llama3.2:latest")

Warning

Multiple calls to this function overrides the previous value.

You can find a curated collection of cutting-edge AI models as OCI Artifacts, from lightweight on-device models to high-performance LLMs on Docker Hub.

The following options are exposed by the testcontainers package.

Basic Options

Lifecycle Options

Files & Mounts Options

Build Options

Logging Options

Image Options

Networking Options

Advanced Options

Experimental Options

Container Methods

The Docker Model Runner container exposes the following methods:

PullModel

Use the PullModel method to pull a model from the Docker Model Runner. Make sure the passed context is not done before the pull operation is completed, so that the pull operation is cancelled.

Pulling a model at runtime

const (
    modelNamespace = "ai"
    modelName      = "smollm2"
    modelTag       = "360M-Q4_K_M"
    fqModelName    = modelNamespace + "/" + modelName + ":" + modelTag
)

ctx, cancel := context.WithTimeout(ctx, 60*time.Second)
defer cancel()

err = dmrCtr.PullModel(ctx, fqModelName)
if err != nil {
    log.Printf("failed to pull model: %s", err)
    return
}

Info

You can find a curated collection of cutting-edge AI models as OCI Artifacts, from lightweight on-device models to high-performance LLMs on Docker Hub.

InspectModel

Use the InspectModel method to inspect a model from the Docker Model Runner, by providing the model namespace and name.

Getting a model at runtime

err = dmrCtr.PullModel(ctx, modelNamespace+"/"+modelName+":"+modelTag)
if err != nil {
    log.Printf("failed to pull model: %s", err)
    return
}

model, err := dmrCtr.InspectModel(ctx, modelNamespace, modelName+":"+modelTag)
if err != nil {
    log.Printf("failed to get model: %s", err)
    return
}

The namespace and name of the model is in the format of <name>:<tag>, which defines Models as OCI Artifacts in Docker Hub, therefore the namespace is the organization and the name is the repository.

E.g. ai/smollm2:360M-Q4_K_M. See Models as OCI Artifacts for more information.

ListModels

Use the ListModels method to list all models that are already pulled locally, using the Docker Model Runner format.

Listing all models

err = dmrCtr.PullModel(ctx, modelNamespace+"/"+modelName+":"+modelTag)
if err != nil {
    log.Printf("failed to pull model: %s", err)
    return
}

models, err := dmrCtr.ListModels(ctx)
if err != nil {
    log.Printf("failed to get model: %s", err)
    return
}

评论

登录后参与评论

正在加载评论…