working commit

This commit is contained in:
2026-02-21 13:16:30 +02:00
parent cd37a4508c
commit d650d58a6d
1149 changed files with 116 additions and 722633 deletions
-322
View File
@@ -1,322 +0,0 @@
# `authn`
[![GoDoc](https://godoc.org/github.com/google/go-containerregistry/pkg/authn?status.svg)](https://godoc.org/github.com/google/go-containerregistry/pkg/authn)
This README outlines how we acquire and use credentials when interacting with a registry.
As much as possible, we attempt to emulate `docker`'s authentication behavior and configuration so that this library "just works" if you've already configured credentials that work with `docker`; however, when things don't work, a basic understanding of what's going on can help with debugging.
The official documentation for how authentication with `docker` works is (reasonably) scattered across several different sites and GitHub repositories, so we've tried to summarize the relevant bits here.
## tl;dr for consumers of this package
By default, [`pkg/v1/remote`](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/remote) uses [`Anonymous`](https://godoc.org/github.com/google/go-containerregistry/pkg/authn#Anonymous) credentials (i.e. _none_), which for most registries will only allow read access to public images.
To use the credentials found in your Docker config file, you can use the [`DefaultKeychain`](https://godoc.org/github.com/google/go-containerregistry/pkg/authn#DefaultKeychain), e.g.:
```go
package main
import (
"fmt"
"github.com/google/go-containerregistry/pkg/authn"
"github.com/google/go-containerregistry/pkg/name"
"github.com/google/go-containerregistry/pkg/v1/remote"
)
func main() {
ref, err := name.ParseReference("registry.example.com/private/repo")
if err != nil {
panic(err)
}
// Fetch the manifest using default credentials.
img, err := remote.Get(ref, remote.WithAuthFromKeychain(authn.DefaultKeychain))
if err != nil {
panic(err)
}
// Prints the digest of registry.example.com/private/repo
fmt.Println(img.Digest)
}
```
The `DefaultKeychain` will use credentials as described in your Docker config file -- usually `~/.docker/config.json`, or `%USERPROFILE%\.docker\config.json` on Windows -- or the location described by the `DOCKER_CONFIG` environment variable, if set.
If those are not found, `DefaultKeychain` will look for credentials configured using [Podman's expectation](https://docs.podman.io/en/latest/markdown/podman-login.1.html) that these are found in `${XDG_RUNTIME_DIR}/containers/auth.json`.
[See below](#docker-config-auth) for more information about what is configured in this file.
## Emulating Cloud Provider Credential Helpers
[`pkg/v1/google.Keychain`](https://pkg.go.dev/github.com/google/go-containerregistry/pkg/v1/google#Keychain) provides a `Keychain` implementation that emulates [`docker-credential-gcr`](https://github.com/GoogleCloudPlatform/docker-credential-gcr) to find credentials in the environment.
See [`google.NewEnvAuthenticator`](https://pkg.go.dev/github.com/google/go-containerregistry/pkg/v1/google#NewEnvAuthenticator) and [`google.NewGcloudAuthenticator`](https://pkg.go.dev/github.com/google/go-containerregistry/pkg/v1/google#NewGcloudAuthenticator) for more information.
To emulate other credential helpers without requiring them to be available as executables, [`NewKeychainFromHelper`](https://pkg.go.dev/github.com/google/go-containerregistry/pkg/authn#NewKeychainFromHelper) provides an adapter that takes a Go implementation satisfying a subset of the [`credentials.Helper`](https://pkg.go.dev/github.com/docker/docker-credential-helpers/credentials#Helper) interface, and makes it available as a `Keychain`.
This means that you can emulate, for example, [Amazon ECR's `docker-credential-ecr-login` credential helper](https://github.com/awslabs/amazon-ecr-credential-helper) using the same implementation:
```go
import (
ecr "github.com/awslabs/amazon-ecr-credential-helper/ecr-login"
"github.com/awslabs/amazon-ecr-credential-helper/ecr-login/api"
"github.com/google/go-containerregistry/pkg/authn"
"github.com/google/go-containerregistry/pkg/v1/remote"
)
func main() {
// ...
ecrHelper := ecr.ECRHelper{ClientFactory: api.DefaultClientFactory{}}
img, err := remote.Get(ref, remote.WithAuthFromKeychain(authn.NewKeychainFromHelper(ecrHelper)))
if err != nil {
panic(err)
}
// ...
}
```
Likewise, you can emulate [Azure's ACR `docker-credential-acr-env` credential helper](https://github.com/chrismellard/docker-credential-acr-env):
```go
import (
"github.com/chrismellard/docker-credential-acr-env/pkg/credhelper"
"github.com/google/go-containerregistry/pkg/authn"
"github.com/google/go-containerregistry/pkg/v1/remote"
)
func main() {
// ...
acrHelper := credhelper.NewACRCredentialsHelper()
img, err := remote.Get(ref, remote.WithAuthFromKeychain(authn.NewKeychainFromHelper(acrHelper)))
if err != nil {
panic(err)
}
// ...
}
```
<!-- TODO(jasonhall): Wrap these in docker-credential-magic and reference those from here. -->
## Using Multiple `Keychain`s
[`NewMultiKeychain`](https://pkg.go.dev/github.com/google/go-containerregistry/pkg/authn#NewMultiKeychain) allows you to specify multiple `Keychain` implementations, which will be checked in order when credentials are needed.
For example:
```go
kc := authn.NewMultiKeychain(
authn.DefaultKeychain,
google.Keychain,
authn.NewKeychainFromHelper(ecr.ECRHelper{ClientFactory: api.DefaultClientFactory{}}),
authn.NewKeychainFromHelper(acr.ACRCredHelper{}),
)
```
This multi-keychain will:
- first check for credentials found in the Docker config file, as describe above, then
- check for GCP credentials available in the environment, as described above, then
- check for ECR credentials by emulating the ECR credential helper, then
- check for ACR credentials by emulating the ACR credential helper.
If any keychain implementation is able to provide credentials for the request, they will be used, and further keychain implementations will not be consulted.
If no implementations are able to provide credentials, `Anonymous` credentials will be used.
## Docker Config Auth
What follows attempts to gather useful information about Docker's config.json and make it available in one place.
If you have questions, please [file an issue](https://github.com/google/go-containerregistry/issues/new).
### Plaintext
The config file is where your credentials are stored when you invoke `docker login`, e.g. the contents may look something like this:
```json
{
"auths": {
"registry.example.com": {
"auth": "QXp1cmVEaWFtb25kOmh1bnRlcjI="
}
}
}
```
The `auths` map has an entry per registry, and the `auth` field contains your username and password encoded as [HTTP 'Basic' Auth](https://tools.ietf.org/html/rfc7617).
**NOTE**: This means that your credentials are stored _in plaintext_:
```bash
$ echo "QXp1cmVEaWFtb25kOmh1bnRlcjI=" | base64 -d
AzureDiamond:hunter2
```
For what it's worth, this config file is equivalent to:
```json
{
"auths": {
"registry.example.com": {
"username": "AzureDiamond",
"password": "hunter2"
}
}
}
```
... which is useful to know if e.g. your CI system provides you a registry username and password via environment variables and you want to populate this file manually without invoking `docker login`.
### Helpers
If you log in like this, `docker` will warn you that you should use a [credential helper](https://docs.docker.com/engine/reference/commandline/login/#credentials-store), and you should!
To configure a global credential helper:
```json
{
"credsStore": "osxkeychain"
}
```
To configure a per-registry credential helper:
```json
{
"credHelpers": {
"gcr.io": "gcr"
}
}
```
We use [`github.com/docker/cli/cli/config.Load`](https://godoc.org/github.com/docker/cli/cli/config#Load) to parse the config file and invoke any necessary credential helpers. This handles the logic of taking a [`ConfigFile`](https://github.com/docker/cli/blob/ba63a92655c0bea4857b8d6cc4991498858b3c60/cli/config/configfile/file.go#L25-L54) + registry domain and producing an [`AuthConfig`](https://github.com/docker/cli/blob/ba63a92655c0bea4857b8d6cc4991498858b3c60/cli/config/types/authconfig.go#L3-L22), which determines how we authenticate to the registry.
## Credential Helpers
The [credential helper protocol](https://github.com/docker/docker-credential-helpers) allows you to configure a binary that supplies credentials for the registry, rather than hard-coding them in the config file.
The protocol has several verbs, but the one we most care about is `get`.
For example, using the following config file:
```json
{
"credHelpers": {
"gcr.io": "gcr",
"eu.gcr.io": "gcr"
}
}
```
To acquire credentials for `gcr.io`, we look in the `credHelpers` map to find
the credential helper for `gcr.io` is `gcr`. By appending that value to
`docker-credential-`, we can get the name of the binary we need to use.
For this example, that's `docker-credential-gcr`, which must be on our `$PATH`.
We'll then invoke that binary to get credentials:
```bash
$ echo "gcr.io" | docker-credential-gcr get
{"Username":"_token","Secret":"<long access token>"}
```
You can configure the same credential helper for multiple registries, which is
why we need to pass the domain in via STDIN, e.g. if we were trying to access
`eu.gcr.io`, we'd do this instead:
```bash
$ echo "eu.gcr.io" | docker-credential-gcr get
{"Username":"_token","Secret":"<long access token>"}
```
### Debugging credential helpers
If a credential helper is configured but doesn't seem to be working, it can be
challenging to debug. Implementing a fake credential helper lets you poke around
to make it easier to see where the failure is happening.
This "implements" a credential helper with hard-coded values:
```
#!/usr/bin/env bash
echo '{"Username":"<token>","Secret":"hunter2"}'
```
This implements a credential helper that prints the output of
`docker-credential-gcr` to both stderr and whatever called it, which allows you
to snoop on another credential helper:
```
#!/usr/bin/env bash
docker-credential-gcr $@ | tee >(cat 1>&2)
```
Put those files somewhere on your path, naming them e.g.
`docker-credential-hardcoded` and `docker-credential-tee`, then modify the
config file to use them:
```json
{
"credHelpers": {
"gcr.io": "tee",
"eu.gcr.io": "hardcoded"
}
}
```
The `docker-credential-tee` trick works with both `crane` and `docker`:
```bash
$ crane manifest gcr.io/google-containers/pause > /dev/null
{"ServerURL":"","Username":"_dcgcr_1_5_0_token","Secret":"<redacted>"}
$ docker pull gcr.io/google-containers/pause
Using default tag: latest
{"ServerURL":"","Username":"_dcgcr_1_5_0_token","Secret":"<redacted>"}
latest: Pulling from google-containers/pause
a3ed95caeb02: Pull complete
4964c72cd024: Pull complete
Digest: sha256:a78c2d6208eff9b672de43f880093100050983047b7b0afe0217d3656e1b0d5f
Status: Downloaded newer image for gcr.io/google-containers/pause:latest
gcr.io/google-containers/pause:latest
```
## The Registry
There are two methods for authenticating against a registry:
[token](https://docs.docker.com/registry/spec/auth/token/) and
[oauth2](https://docs.docker.com/registry/spec/auth/oauth/).
Both methods are used to acquire an opaque `Bearer` token (or
[RegistryToken](https://github.com/docker/cli/blob/ba63a92655c0bea4857b8d6cc4991498858b3c60/cli/config/types/authconfig.go#L21))
to use in the `Authorization` header. The registry will return a `401
Unauthorized` during the [version
check](https://github.com/opencontainers/distribution-spec/blob/2c3975d1f03b67c9a0203199038adea0413f0573/spec.md#api-version-check)
(or during normal operations) with
[Www-Authenticate](https://tools.ietf.org/html/rfc7235#section-4.1) challenge
indicating how to proceed.
### Token
If we get back an `AuthConfig` containing a [`Username/Password`](https://github.com/docker/cli/blob/ba63a92655c0bea4857b8d6cc4991498858b3c60/cli/config/types/authconfig.go#L5-L6)
or
[`Auth`](https://github.com/docker/cli/blob/ba63a92655c0bea4857b8d6cc4991498858b3c60/cli/config/types/authconfig.go#L7),
we'll use the token method for authentication:
![basic](../../images/credhelper-basic.svg)
### OAuth 2
If we get back an `AuthConfig` containing an [`IdentityToken`](https://github.com/docker/cli/blob/ba63a92655c0bea4857b8d6cc4991498858b3c60/cli/config/types/authconfig.go#L18)
we'll use the oauth2 method for authentication:
![oauth](../../images/credhelper-oauth.svg)
This happens when a credential helper returns a response with the
[`Username`](https://github.com/docker/docker-credential-helpers/blob/f78081d1f7fef6ad74ad6b79368de6348386e591/credentials/credentials.go#L16)
set to `<token>` (no, that's not a placeholder, the literal string `"<token>"`).
It is unclear why: [moby/moby#36926](https://github.com/moby/moby/issues/36926).
We only support the oauth2 `grant_type` for `refresh_token` ([#629](https://github.com/google/go-containerregistry/issues/629)),
since it's impossible to determine from the registry response whether we should
use oauth, and the token method for authentication is widely implemented by
registries.
-26
View File
@@ -1,26 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package authn
// anonymous implements Authenticator for anonymous authentication.
type anonymous struct{}
// Authorization implements Authenticator.
func (a *anonymous) Authorization() (*AuthConfig, error) {
return &AuthConfig{}, nil
}
// Anonymous is a singleton Authenticator for providing anonymous auth.
var Anonymous Authenticator = &anonymous{}
-30
View File
@@ -1,30 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package authn
// auth is an Authenticator that simply returns the wrapped AuthConfig.
type auth struct {
config AuthConfig
}
// FromConfig returns an Authenticator that just returns the given AuthConfig.
func FromConfig(cfg AuthConfig) Authenticator {
return &auth{cfg}
}
// Authorization implements Authenticator.
func (a *auth) Authorization() (*AuthConfig, error) {
return &a.config, nil
}
-132
View File
@@ -1,132 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package authn
import (
"context"
"encoding/base64"
"encoding/json"
"fmt"
"strings"
)
// Authenticator is used to authenticate Docker transports.
type Authenticator interface {
// Authorization returns the value to use in an http transport's Authorization header.
Authorization() (*AuthConfig, error)
}
// ContextAuthenticator is like Authenticator, but allows for context to be passed in.
type ContextAuthenticator interface {
// Authorization returns the value to use in an http transport's Authorization header.
AuthorizationContext(context.Context) (*AuthConfig, error)
}
// Authorization calls AuthorizationContext with ctx if the given [Authenticator] implements [ContextAuthenticator],
// otherwise it calls Resolve with the given [Resource].
func Authorization(ctx context.Context, authn Authenticator) (*AuthConfig, error) {
if actx, ok := authn.(ContextAuthenticator); ok {
return actx.AuthorizationContext(ctx)
}
return authn.Authorization()
}
// AuthConfig contains authorization information for connecting to a Registry
// Inlined what we use from github.com/docker/cli/cli/config/types
type AuthConfig struct {
Username string `json:"username,omitempty"`
Password string `json:"password,omitempty"`
Auth string `json:"auth,omitempty"`
// IdentityToken is used to authenticate the user and get
// an access token for the registry.
IdentityToken string `json:"identitytoken,omitempty"`
// RegistryToken is a bearer token to be sent to a registry
RegistryToken string `json:"registrytoken,omitempty"`
}
// This is effectively a copy of the type AuthConfig. This simplifies
// JSON unmarshalling since AuthConfig methods are not inherited
type authConfig AuthConfig
// UnmarshalJSON implements json.Unmarshaler
func (a *AuthConfig) UnmarshalJSON(data []byte) error {
var shadow authConfig
err := json.Unmarshal(data, &shadow)
if err != nil {
return err
}
*a = (AuthConfig)(shadow)
if len(shadow.Auth) != 0 {
var derr error
a.Username, a.Password, derr = decodeDockerConfigFieldAuth(shadow.Auth)
if derr != nil {
err = fmt.Errorf("unable to decode auth field: %w", derr)
}
} else if len(a.Username) != 0 && len(a.Password) != 0 {
a.Auth = encodeDockerConfigFieldAuth(shadow.Username, shadow.Password)
}
return err
}
// MarshalJSON implements json.Marshaler
func (a AuthConfig) MarshalJSON() ([]byte, error) {
shadow := (authConfig)(a)
shadow.Auth = encodeDockerConfigFieldAuth(shadow.Username, shadow.Password)
return json.Marshal(shadow)
}
// decodeDockerConfigFieldAuth deserializes the "auth" field from dockercfg into a
// username and a password. The format of the auth field is base64(<username>:<password>).
//
// From https://github.com/kubernetes/kubernetes/blob/75e49ec824b183288e1dbaccfd7dbe77d89db381/pkg/credentialprovider/config.go
// Copyright 2014 The Kubernetes Authors.
// SPDX-License-Identifier: Apache-2.0
func decodeDockerConfigFieldAuth(field string) (username, password string, err error) {
var decoded []byte
// StdEncoding can only decode padded string
// RawStdEncoding can only decode unpadded string
if strings.HasSuffix(strings.TrimSpace(field), "=") {
// decode padded data
decoded, err = base64.StdEncoding.DecodeString(field)
} else {
// decode unpadded data
decoded, err = base64.RawStdEncoding.DecodeString(field)
}
if err != nil {
return
}
parts := strings.SplitN(string(decoded), ":", 2)
if len(parts) != 2 {
err = fmt.Errorf("must be formatted as base64(username:password)")
return
}
username = parts[0]
password = parts[1]
return
}
func encodeDockerConfigFieldAuth(username, password string) string {
return base64.StdEncoding.EncodeToString([]byte(username + ":" + password))
}
-29
View File
@@ -1,29 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package authn
// Basic implements Authenticator for basic authentication.
type Basic struct {
Username string
Password string
}
// Authorization implements Authenticator.
func (b *Basic) Authorization() (*AuthConfig, error) {
return &AuthConfig{
Username: b.Username,
Password: b.Password,
}, nil
}
-27
View File
@@ -1,27 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package authn
// Bearer implements Authenticator for bearer authentication.
type Bearer struct {
Token string `json:"token"`
}
// Authorization implements Authenticator.
func (b *Bearer) Authorization() (*AuthConfig, error) {
return &AuthConfig{
RegistryToken: b.Token,
}, nil
}
-17
View File
@@ -1,17 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// Package authn defines different methods of authentication for
// talking to a container registry.
package authn
-294
View File
@@ -1,294 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package authn
import (
"context"
"os"
"path/filepath"
"sync"
"time"
"github.com/docker/cli/cli/config"
"github.com/docker/cli/cli/config/configfile"
"github.com/docker/cli/cli/config/types"
"github.com/google/go-containerregistry/pkg/name"
"github.com/mitchellh/go-homedir"
)
// Resource represents a registry or repository that can be authenticated against.
type Resource interface {
// String returns the full string representation of the target, e.g.
// gcr.io/my-project or just gcr.io.
String() string
// RegistryStr returns just the registry portion of the target, e.g. for
// gcr.io/my-project, this should just return gcr.io. This is needed to
// pull out an appropriate hostname.
RegistryStr() string
}
// Keychain is an interface for resolving an image reference to a credential.
type Keychain interface {
// Resolve looks up the most appropriate credential for the specified target.
Resolve(Resource) (Authenticator, error)
}
// ContextKeychain is like Keychain, but allows for context to be passed in.
type ContextKeychain interface {
ResolveContext(context.Context, Resource) (Authenticator, error)
}
// defaultKeychain implements Keychain with the semantics of the standard Docker
// credential keychain.
type defaultKeychain struct {
mu sync.Mutex
}
var (
// DefaultKeychain implements Keychain by interpreting the docker config file.
DefaultKeychain = &defaultKeychain{}
)
const (
// DefaultAuthKey is the key used for dockerhub in config files, which
// is hardcoded for historical reasons.
DefaultAuthKey = "https://" + name.DefaultRegistry + "/v1/"
)
// Resolve calls ResolveContext with ctx if the given [Keychain] implements [ContextKeychain],
// otherwise it calls Resolve with the given [Resource].
func Resolve(ctx context.Context, keychain Keychain, target Resource) (Authenticator, error) {
if rctx, ok := keychain.(ContextKeychain); ok {
return rctx.ResolveContext(ctx, target)
}
return keychain.Resolve(target)
}
// ResolveContext implements ContextKeychain.
func (dk *defaultKeychain) Resolve(target Resource) (Authenticator, error) {
return dk.ResolveContext(context.Background(), target)
}
// Resolve implements Keychain.
func (dk *defaultKeychain) ResolveContext(_ context.Context, target Resource) (Authenticator, error) {
dk.mu.Lock()
defer dk.mu.Unlock()
// Podman users may have their container registry auth configured in a
// different location, that Docker packages aren't aware of.
// If the Docker config file isn't found, we'll fallback to look where
// Podman configures it, and parse that as a Docker auth config instead.
// First, check $HOME/.docker/config.json
foundDockerConfig := false
home, err := homedir.Dir()
if err == nil {
foundDockerConfig = fileExists(filepath.Join(home, ".docker/config.json"))
}
// If $HOME/.docker/config.json isn't found, check $DOCKER_CONFIG (if set)
if !foundDockerConfig && os.Getenv("DOCKER_CONFIG") != "" {
foundDockerConfig = fileExists(filepath.Join(os.Getenv("DOCKER_CONFIG"), "config.json"))
}
// If either of those locations are found, load it using Docker's
// config.Load, which may fail if the config can't be parsed.
//
// If neither was found, look for Podman's auth at
// $REGISTRY_AUTH_FILE or $XDG_RUNTIME_DIR/containers/auth.json
// and attempt to load it as a Docker config.
//
// If neither are found, fallback to Anonymous.
var cf *configfile.ConfigFile
if foundDockerConfig {
cf, err = config.Load(os.Getenv("DOCKER_CONFIG"))
if err != nil {
return nil, err
}
} else if path := filepath.Clean(os.Getenv("REGISTRY_AUTH_FILE")); fileExists(path) {
f, err := os.Open(path)
if err != nil {
return nil, err
}
defer f.Close()
cf, err = config.LoadFromReader(f)
if err != nil {
return nil, err
}
} else if path := filepath.Clean(filepath.Join(os.Getenv("XDG_RUNTIME_DIR"), "containers/auth.json")); fileExists(path) {
f, err := os.Open(path)
if err != nil {
return nil, err
}
defer f.Close()
cf, err = config.LoadFromReader(f)
if err != nil {
return nil, err
}
} else {
return Anonymous, nil
}
// See:
// https://github.com/google/ko/issues/90
// https://github.com/moby/moby/blob/fc01c2b481097a6057bec3cd1ab2d7b4488c50c4/registry/config.go#L397-L404
var cfg, empty types.AuthConfig
for _, key := range []string{
target.String(),
target.RegistryStr(),
} {
if key == name.DefaultRegistry {
key = DefaultAuthKey
}
cfg, err = cf.GetAuthConfig(key)
if err != nil {
return nil, err
}
// cf.GetAuthConfig automatically sets the ServerAddress attribute. Since
// we don't make use of it, clear the value for a proper "is-empty" test.
// See: https://github.com/google/go-containerregistry/issues/1510
cfg.ServerAddress = ""
if cfg != empty {
break
}
}
if cfg == empty {
return Anonymous, nil
}
return FromConfig(AuthConfig{
Username: cfg.Username,
Password: cfg.Password,
Auth: cfg.Auth,
IdentityToken: cfg.IdentityToken,
RegistryToken: cfg.RegistryToken,
}), nil
}
// fileExists returns true if the given path exists and is not a directory.
func fileExists(path string) bool {
fi, err := os.Stat(path)
return err == nil && !fi.IsDir()
}
// Helper is a subset of the Docker credential helper credentials.Helper
// interface used by NewKeychainFromHelper.
//
// See:
// https://pkg.go.dev/github.com/docker/docker-credential-helpers/credentials#Helper
type Helper interface {
Get(serverURL string) (string, string, error)
}
// NewKeychainFromHelper returns a Keychain based on a Docker credential helper
// implementation that can Get username and password credentials for a given
// server URL.
func NewKeychainFromHelper(h Helper) Keychain { return wrapper{h} }
type wrapper struct{ h Helper }
func (w wrapper) Resolve(r Resource) (Authenticator, error) {
return w.ResolveContext(context.Background(), r)
}
func (w wrapper) ResolveContext(_ context.Context, r Resource) (Authenticator, error) {
u, p, err := w.h.Get(r.RegistryStr())
if err != nil {
return Anonymous, nil
}
// If the secret being stored is an identity token, the Username should be set to <token>
// ref: https://docs.docker.com/engine/reference/commandline/login/#credential-helper-protocol
if u == "<token>" {
return FromConfig(AuthConfig{Username: u, IdentityToken: p}), nil
}
return FromConfig(AuthConfig{Username: u, Password: p}), nil
}
func RefreshingKeychain(inner Keychain, duration time.Duration) Keychain {
return &refreshingKeychain{
keychain: inner,
duration: duration,
}
}
type refreshingKeychain struct {
keychain Keychain
duration time.Duration
clock func() time.Time
}
func (r *refreshingKeychain) Resolve(target Resource) (Authenticator, error) {
return r.ResolveContext(context.Background(), target)
}
func (r *refreshingKeychain) ResolveContext(ctx context.Context, target Resource) (Authenticator, error) {
last := time.Now()
auth, err := Resolve(ctx, r.keychain, target)
if err != nil || auth == Anonymous {
return auth, err
}
return &refreshing{
target: target,
keychain: r.keychain,
last: last,
cached: auth,
duration: r.duration,
clock: r.clock,
}, nil
}
type refreshing struct {
sync.Mutex
target Resource
keychain Keychain
duration time.Duration
last time.Time
cached Authenticator
// for testing
clock func() time.Time
}
func (r *refreshing) Authorization() (*AuthConfig, error) {
return r.AuthorizationContext(context.Background())
}
func (r *refreshing) AuthorizationContext(ctx context.Context) (*AuthConfig, error) {
r.Lock()
defer r.Unlock()
if r.cached == nil || r.expired() {
r.last = r.now()
auth, err := Resolve(ctx, r.keychain, r.target)
if err != nil {
return nil, err
}
r.cached = auth
}
return Authorization(ctx, r.cached)
}
func (r *refreshing) now() time.Time {
if r.clock == nil {
return time.Now()
}
return r.clock()
}
func (r *refreshing) expired() bool {
return r.now().Sub(r.last) > r.duration
}
@@ -1,47 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package authn
import "context"
type multiKeychain struct {
keychains []Keychain
}
// Assert that our multi-keychain implements Keychain.
var _ (Keychain) = (*multiKeychain)(nil)
// NewMultiKeychain composes a list of keychains into one new keychain.
func NewMultiKeychain(kcs ...Keychain) Keychain {
return &multiKeychain{keychains: kcs}
}
// Resolve implements Keychain.
func (mk *multiKeychain) Resolve(target Resource) (Authenticator, error) {
return mk.ResolveContext(context.Background(), target)
}
func (mk *multiKeychain) ResolveContext(ctx context.Context, target Resource) (Authenticator, error) {
for _, kc := range mk.keychains {
auth, err := Resolve(ctx, kc, target)
if err != nil {
return nil, err
}
if auth != Anonymous {
return auth, nil
}
}
return Anonymous, nil
}
@@ -1,26 +0,0 @@
// Copyright 2022 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// Package compression abstracts over gzip and zstd.
package compression
// Compression is an enumeration of the supported compression algorithms
type Compression string
// The collection of known MediaType values.
const (
None Compression = "none"
GZip Compression = "gzip"
ZStd Compression = "zstd"
)
-129
View File
@@ -1,129 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package crane
import (
"fmt"
"os"
comp "github.com/google/go-containerregistry/internal/compression"
"github.com/google/go-containerregistry/internal/windows"
"github.com/google/go-containerregistry/pkg/compression"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/mutate"
"github.com/google/go-containerregistry/pkg/v1/stream"
"github.com/google/go-containerregistry/pkg/v1/tarball"
"github.com/google/go-containerregistry/pkg/v1/types"
)
func isWindows(img v1.Image) (bool, error) {
cfg, err := img.ConfigFile()
if err != nil {
return false, err
}
return cfg != nil && cfg.OS == "windows", nil
}
// Append reads a layer from path and appends it the the v1.Image base.
//
// If the base image is a Windows base image (i.e., its config.OS is
// "windows"), the contents of the tarballs will be modified to be suitable for
// a Windows container image.`,
func Append(base v1.Image, paths ...string) (v1.Image, error) {
if base == nil {
return nil, fmt.Errorf("invalid argument: base")
}
win, err := isWindows(base)
if err != nil {
return nil, fmt.Errorf("getting base image: %w", err)
}
baseMediaType, err := base.MediaType()
if err != nil {
return nil, fmt.Errorf("getting base image media type: %w", err)
}
layerType := types.DockerLayer
if baseMediaType == types.OCIManifestSchema1 {
layerType = types.OCILayer
}
layers := make([]v1.Layer, 0, len(paths))
for _, path := range paths {
layer, err := getLayer(path, layerType)
if err != nil {
return nil, fmt.Errorf("reading layer %q: %w", path, err)
}
if win {
layer, err = windows.Windows(layer)
if err != nil {
return nil, fmt.Errorf("converting %q for Windows: %w", path, err)
}
}
layers = append(layers, layer)
}
return mutate.AppendLayers(base, layers...)
}
func getLayer(path string, layerType types.MediaType) (v1.Layer, error) {
f, err := streamFile(path)
if err != nil {
return nil, err
}
if f != nil {
return stream.NewLayer(f, stream.WithMediaType(layerType)), nil
}
// This is dumb but the tarball package assumes things about mediaTypes that aren't true
// and doesn't have enough context to know what the right default is.
f, err = os.Open(path)
if err != nil {
return nil, err
}
defer f.Close()
z, _, err := comp.PeekCompression(f)
if err != nil {
return nil, err
}
if z == compression.ZStd {
layerType = types.OCILayerZStd
}
return tarball.LayerFromFile(path, tarball.WithMediaType(layerType))
}
// If we're dealing with a named pipe, trying to open it multiple times will
// fail, so we need to do a streaming upload.
//
// returns nil, nil for non-streaming files
func streamFile(path string) (*os.File, error) {
if path == "-" {
return os.Stdin, nil
}
fi, err := os.Stat(path)
if err != nil {
return nil, err
}
if !fi.Mode().IsRegular() {
return os.Open(path)
}
return nil, nil
}
-35
View File
@@ -1,35 +0,0 @@
// Copyright 2019 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package crane
import (
"context"
"github.com/google/go-containerregistry/pkg/name"
"github.com/google/go-containerregistry/pkg/v1/remote"
)
// Catalog returns the repositories in a registry's catalog.
func Catalog(src string, opt ...Option) (res []string, err error) {
o := makeOptions(opt...)
reg, err := name.NewRegistry(src, o.Name...)
if err != nil {
return nil, err
}
// This context gets overridden by remote.WithContext, which is set by
// crane.WithContext.
return remote.Catalog(context.Background(), reg, o.Remote...)
}
-24
View File
@@ -1,24 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package crane
// Config returns the config file for the remote image ref.
func Config(ref string, opt ...Option) ([]byte, error) {
i, _, err := getImage(ref, opt...)
if err != nil {
return nil, err
}
return i.RawConfigFile()
}
-185
View File
@@ -1,185 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package crane
import (
"errors"
"fmt"
"net/http"
"github.com/google/go-containerregistry/pkg/logs"
"github.com/google/go-containerregistry/pkg/name"
"github.com/google/go-containerregistry/pkg/v1/remote"
"github.com/google/go-containerregistry/pkg/v1/remote/transport"
"golang.org/x/sync/errgroup"
)
// ErrRefusingToClobberExistingTag is returned when NoClobber is true and the
// tag already exists in the target registry/repo.
var ErrRefusingToClobberExistingTag = errors.New("refusing to clobber existing tag")
// Copy copies a remote image or index from src to dst.
func Copy(src, dst string, opt ...Option) error {
o := makeOptions(opt...)
srcRef, err := name.ParseReference(src, o.Name...)
if err != nil {
return fmt.Errorf("parsing reference %q: %w", src, err)
}
dstRef, err := name.ParseReference(dst, o.Name...)
if err != nil {
return fmt.Errorf("parsing reference for %q: %w", dst, err)
}
puller, err := remote.NewPuller(o.Remote...)
if err != nil {
return err
}
if tag, ok := dstRef.(name.Tag); ok {
if o.noclobber {
logs.Progress.Printf("Checking existing tag %v", tag)
head, err := puller.Head(o.ctx, tag)
var terr *transport.Error
if errors.As(err, &terr) {
if terr.StatusCode != http.StatusNotFound && terr.StatusCode != http.StatusForbidden {
return err
}
} else if err != nil {
return err
}
if head != nil {
return fmt.Errorf("%w %s@%s", ErrRefusingToClobberExistingTag, tag, head.Digest)
}
}
}
pusher, err := remote.NewPusher(o.Remote...)
if err != nil {
return err
}
logs.Progress.Printf("Copying from %v to %v", srcRef, dstRef)
desc, err := puller.Get(o.ctx, srcRef)
if err != nil {
return fmt.Errorf("fetching %q: %w", src, err)
}
if o.Platform == nil {
return pusher.Push(o.ctx, dstRef, desc)
}
// If platform is explicitly set, don't copy the whole index, just the appropriate image.
img, err := desc.Image()
if err != nil {
return err
}
return pusher.Push(o.ctx, dstRef, img)
}
// CopyRepository copies every tag from src to dst.
func CopyRepository(src, dst string, opt ...Option) error {
o := makeOptions(opt...)
srcRepo, err := name.NewRepository(src, o.Name...)
if err != nil {
return err
}
dstRepo, err := name.NewRepository(dst, o.Name...)
if err != nil {
return fmt.Errorf("parsing reference for %q: %w", dst, err)
}
puller, err := remote.NewPuller(o.Remote...)
if err != nil {
return err
}
ignoredTags := map[string]struct{}{}
if o.noclobber {
// TODO: It would be good to propagate noclobber down into remote so we can use Etags.
have, err := puller.List(o.ctx, dstRepo)
if err != nil {
var terr *transport.Error
if errors.As(err, &terr) {
// Some registries create repository on first push, so listing tags will fail.
// If we see 404 or 403, assume we failed because the repository hasn't been created yet.
if terr.StatusCode != http.StatusNotFound && terr.StatusCode != http.StatusForbidden {
return err
}
} else {
return err
}
}
for _, tag := range have {
ignoredTags[tag] = struct{}{}
}
}
pusher, err := remote.NewPusher(o.Remote...)
if err != nil {
return err
}
lister, err := puller.Lister(o.ctx, srcRepo)
if err != nil {
return err
}
g, ctx := errgroup.WithContext(o.ctx)
g.SetLimit(o.jobs)
for lister.HasNext() {
tags, err := lister.Next(ctx)
if err != nil {
return err
}
for _, tag := range tags.Tags {
tag := tag
if o.noclobber {
if _, ok := ignoredTags[tag]; ok {
logs.Progress.Printf("Skipping %s due to no-clobber", tag)
continue
}
}
g.Go(func() error {
srcTag, err := name.ParseReference(src+":"+tag, o.Name...)
if err != nil {
return fmt.Errorf("failed to parse tag: %w", err)
}
dstTag, err := name.ParseReference(dst+":"+tag, o.Name...)
if err != nil {
return fmt.Errorf("failed to parse tag: %w", err)
}
logs.Progress.Printf("Fetching %s", srcTag)
desc, err := puller.Get(ctx, srcTag)
if err != nil {
return err
}
logs.Progress.Printf("Pushing %s", dstTag)
return pusher.Push(ctx, dstTag, desc)
})
}
}
return g.Wait()
}
-33
View File
@@ -1,33 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package crane
import (
"fmt"
"github.com/google/go-containerregistry/pkg/name"
"github.com/google/go-containerregistry/pkg/v1/remote"
)
// Delete deletes the remote reference at src.
func Delete(src string, opt ...Option) error {
o := makeOptions(opt...)
ref, err := name.ParseReference(src, o.Name...)
if err != nil {
return fmt.Errorf("parsing reference %q: %w", src, err)
}
return remote.Delete(ref, o.Remote...)
}
-52
View File
@@ -1,52 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package crane
import "github.com/google/go-containerregistry/pkg/logs"
// Digest returns the sha256 hash of the remote image at ref.
func Digest(ref string, opt ...Option) (string, error) {
o := makeOptions(opt...)
if o.Platform != nil {
desc, err := getManifest(ref, opt...)
if err != nil {
return "", err
}
if !desc.MediaType.IsIndex() {
return desc.Digest.String(), nil
}
// TODO: does not work for indexes which contain schema v1 manifests
img, err := desc.Image()
if err != nil {
return "", err
}
digest, err := img.Digest()
if err != nil {
return "", err
}
return digest.String(), nil
}
desc, err := Head(ref, opt...)
if err != nil {
logs.Warn.Printf("HEAD request failed, falling back on GET: %v", err)
rdesc, err := getManifest(ref, opt...)
if err != nil {
return "", err
}
return rdesc.Digest.String(), nil
}
return desc.Digest.String(), nil
}
-16
View File
@@ -1,16 +0,0 @@
// Copyright 2019 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// Package crane holds libraries used to implement the crane CLI.
package crane
-54
View File
@@ -1,54 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package crane
import (
"io"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/mutate"
)
// Export writes the filesystem contents (as a tarball) of img to w.
// If img has a single layer, just write the (uncompressed) contents to w so
// that this "just works" for images that just wrap a single blob.
func Export(img v1.Image, w io.Writer) error {
layers, err := img.Layers()
if err != nil {
return err
}
if len(layers) == 1 {
// If it's a single layer...
l := layers[0]
mt, err := l.MediaType()
if err != nil {
return err
}
if !mt.IsLayer() {
// ...and isn't an OCI mediaType, we don't have to flatten it.
// This lets export work for single layer, non-tarball images.
rc, err := l.Uncompressed()
if err != nil {
return err
}
_, err = io.Copy(w, rc)
return err
}
}
fs := mutate.Extract(img)
_, err = io.Copy(w, fs)
return err
}
-72
View File
@@ -1,72 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package crane
import (
"archive/tar"
"bytes"
"io"
"sort"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/empty"
"github.com/google/go-containerregistry/pkg/v1/mutate"
"github.com/google/go-containerregistry/pkg/v1/tarball"
)
// Layer creates a layer from a single file map. These layers are reproducible and consistent.
// A filemap is a path -> file content map representing a file system.
func Layer(filemap map[string][]byte) (v1.Layer, error) {
b := &bytes.Buffer{}
w := tar.NewWriter(b)
fn := make([]string, 0, len(filemap))
for f := range filemap {
fn = append(fn, f)
}
sort.Strings(fn)
for _, f := range fn {
c := filemap[f]
if err := w.WriteHeader(&tar.Header{
Name: f,
Size: int64(len(c)),
}); err != nil {
return nil, err
}
if _, err := w.Write(c); err != nil {
return nil, err
}
}
if err := w.Close(); err != nil {
return nil, err
}
// Return a new copy of the buffer each time it's opened.
return tarball.LayerFromOpener(func() (io.ReadCloser, error) {
return io.NopCloser(bytes.NewBuffer(b.Bytes())), nil
})
}
// Image creates a image with the given filemaps as its contents. These images are reproducible and consistent.
// A filemap is a path -> file content map representing a file system.
func Image(filemap map[string][]byte) (v1.Image, error) {
y, err := Layer(filemap)
if err != nil {
return nil, err
}
return mutate.AppendLayers(empty.Image, y)
}
-61
View File
@@ -1,61 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package crane
import (
"fmt"
"github.com/google/go-containerregistry/pkg/name"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/remote"
)
func getImage(r string, opt ...Option) (v1.Image, name.Reference, error) {
o := makeOptions(opt...)
ref, err := name.ParseReference(r, o.Name...)
if err != nil {
return nil, nil, fmt.Errorf("parsing reference %q: %w", r, err)
}
img, err := remote.Image(ref, o.Remote...)
if err != nil {
return nil, nil, fmt.Errorf("reading image %q: %w", ref, err)
}
return img, ref, nil
}
func getManifest(r string, opt ...Option) (*remote.Descriptor, error) {
o := makeOptions(opt...)
ref, err := name.ParseReference(r, o.Name...)
if err != nil {
return nil, fmt.Errorf("parsing reference %q: %w", r, err)
}
return remote.Get(ref, o.Remote...)
}
// Get calls remote.Get and returns an uninterpreted response.
func Get(r string, opt ...Option) (*remote.Descriptor, error) {
return getManifest(r, opt...)
}
// Head performs a HEAD request for a manifest and returns a content descriptor
// based on the registry's response.
func Head(r string, opt ...Option) (*v1.Descriptor, error) {
o := makeOptions(opt...)
ref, err := name.ParseReference(r, o.Name...)
if err != nil {
return nil, err
}
return remote.Head(ref, o.Remote...)
}
-33
View File
@@ -1,33 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package crane
import (
"fmt"
"github.com/google/go-containerregistry/pkg/name"
"github.com/google/go-containerregistry/pkg/v1/remote"
)
// ListTags returns the tags in repository src.
func ListTags(src string, opt ...Option) ([]string, error) {
o := makeOptions(opt...)
repo, err := name.NewRepository(src, o.Name...)
if err != nil {
return nil, fmt.Errorf("parsing repo %q: %w", src, err)
}
return remote.List(repo, o.Remote...)
}
-32
View File
@@ -1,32 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package crane
// Manifest returns the manifest for the remote image or index ref.
func Manifest(ref string, opt ...Option) ([]byte, error) {
desc, err := getManifest(ref, opt...)
if err != nil {
return nil, err
}
o := makeOptions(opt...)
if o.Platform != nil {
img, err := desc.Image()
if err != nil {
return nil, err
}
return img.RawManifest()
}
return desc.Manifest, nil
}
-178
View File
@@ -1,178 +0,0 @@
// Copyright 2019 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package crane
import (
"context"
"crypto/tls"
"net/http"
"github.com/google/go-containerregistry/pkg/authn"
"github.com/google/go-containerregistry/pkg/name"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/remote"
)
// Options hold the options that crane uses when calling other packages.
type Options struct {
Name []name.Option
Remote []remote.Option
Platform *v1.Platform
Keychain authn.Keychain
Transport http.RoundTripper
auth authn.Authenticator
insecure bool
jobs int
noclobber bool
ctx context.Context
}
// GetOptions exposes the underlying []remote.Option, []name.Option, and
// platform, based on the passed Option. Generally, you shouldn't need to use
// this unless you've painted yourself into a dependency corner as we have
// with the crane and gcrane cli packages.
func GetOptions(opts ...Option) Options {
return makeOptions(opts...)
}
func makeOptions(opts ...Option) Options {
opt := Options{
Remote: []remote.Option{
remote.WithAuthFromKeychain(authn.DefaultKeychain),
},
Keychain: authn.DefaultKeychain,
jobs: 4,
ctx: context.Background(),
}
for _, o := range opts {
o(&opt)
}
// Allow for untrusted certificates if the user
// passed Insecure but no custom transport.
if opt.insecure && opt.Transport == nil {
transport := remote.DefaultTransport.(*http.Transport).Clone()
transport.TLSClientConfig = &tls.Config{
InsecureSkipVerify: true, //nolint: gosec
}
WithTransport(transport)(&opt)
} else if opt.Transport == nil {
opt.Transport = remote.DefaultTransport
}
return opt
}
// Option is a functional option for crane.
type Option func(*Options)
// WithTransport is a functional option for overriding the default transport
// for remote operations. Setting a transport will override the Insecure option's
// configuration allowing for image registries to use untrusted certificates.
func WithTransport(t http.RoundTripper) Option {
return func(o *Options) {
o.Remote = append(o.Remote, remote.WithTransport(t))
o.Transport = t
}
}
// Insecure is an Option that allows image references to be fetched without TLS.
// This will also allow for untrusted (e.g. self-signed) certificates in cases where
// the default transport is used (i.e. when WithTransport is not used).
func Insecure(o *Options) {
o.Name = append(o.Name, name.Insecure)
o.insecure = true
}
// WithPlatform is an Option to specify the platform.
func WithPlatform(platform *v1.Platform) Option {
return func(o *Options) {
if platform != nil {
o.Remote = append(o.Remote, remote.WithPlatform(*platform))
}
o.Platform = platform
}
}
// WithAuthFromKeychain is a functional option for overriding the default
// authenticator for remote operations, using an authn.Keychain to find
// credentials.
//
// By default, crane will use authn.DefaultKeychain.
func WithAuthFromKeychain(keys authn.Keychain) Option {
return func(o *Options) {
// Replace the default keychain at position 0.
o.Remote[0] = remote.WithAuthFromKeychain(keys)
o.Keychain = keys
}
}
// WithAuth is a functional option for overriding the default authenticator
// for remote operations.
//
// By default, crane will use authn.DefaultKeychain.
func WithAuth(auth authn.Authenticator) Option {
return func(o *Options) {
// Replace the default keychain at position 0.
o.Remote[0] = remote.WithAuth(auth)
o.auth = auth
}
}
// WithUserAgent adds the given string to the User-Agent header for any HTTP
// requests.
func WithUserAgent(ua string) Option {
return func(o *Options) {
o.Remote = append(o.Remote, remote.WithUserAgent(ua))
}
}
// WithNondistributable is an option that allows pushing non-distributable
// layers.
func WithNondistributable() Option {
return func(o *Options) {
o.Remote = append(o.Remote, remote.WithNondistributable)
}
}
// WithContext is a functional option for setting the context.
func WithContext(ctx context.Context) Option {
return func(o *Options) {
o.ctx = ctx
o.Remote = append(o.Remote, remote.WithContext(ctx))
}
}
// WithJobs sets the number of concurrent jobs to run.
//
// The default number of jobs is GOMAXPROCS.
func WithJobs(jobs int) Option {
return func(o *Options) {
if jobs > 0 {
o.jobs = jobs
}
o.Remote = append(o.Remote, remote.WithJobs(o.jobs))
}
}
// WithNoClobber modifies behavior to avoid overwriting existing tags, if possible.
func WithNoClobber(noclobber bool) Option {
return func(o *Options) {
o.noclobber = noclobber
}
}
-142
View File
@@ -1,142 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package crane
import (
"fmt"
"os"
legacy "github.com/google/go-containerregistry/pkg/legacy/tarball"
"github.com/google/go-containerregistry/pkg/name"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/empty"
"github.com/google/go-containerregistry/pkg/v1/layout"
"github.com/google/go-containerregistry/pkg/v1/remote"
"github.com/google/go-containerregistry/pkg/v1/tarball"
)
// Tag applied to images that were pulled by digest. This denotes that the
// image was (probably) never tagged with this, but lets us avoid applying the
// ":latest" tag which might be misleading.
const iWasADigestTag = "i-was-a-digest"
// Pull returns a v1.Image of the remote image src.
func Pull(src string, opt ...Option) (v1.Image, error) {
o := makeOptions(opt...)
ref, err := name.ParseReference(src, o.Name...)
if err != nil {
return nil, fmt.Errorf("parsing reference %q: %w", src, err)
}
return remote.Image(ref, o.Remote...)
}
// Save writes the v1.Image img as a tarball at path with tag src.
func Save(img v1.Image, src, path string) error {
imgMap := map[string]v1.Image{src: img}
return MultiSave(imgMap, path)
}
// MultiSave writes collection of v1.Image img with tag as a tarball.
func MultiSave(imgMap map[string]v1.Image, path string, opt ...Option) error {
o := makeOptions(opt...)
tagToImage := map[name.Tag]v1.Image{}
for src, img := range imgMap {
ref, err := name.ParseReference(src, o.Name...)
if err != nil {
return fmt.Errorf("parsing ref %q: %w", src, err)
}
// WriteToFile wants a tag to write to the tarball, but we might have
// been given a digest.
// If the original ref was a tag, use that. Otherwise, if it was a
// digest, tag the image with :i-was-a-digest instead.
tag, ok := ref.(name.Tag)
if !ok {
d, ok := ref.(name.Digest)
if !ok {
return fmt.Errorf("ref wasn't a tag or digest")
}
tag = d.Tag(iWasADigestTag)
}
tagToImage[tag] = img
}
// no progress channel (for now)
return tarball.MultiWriteToFile(path, tagToImage)
}
// PullLayer returns the given layer from a registry.
func PullLayer(ref string, opt ...Option) (v1.Layer, error) {
o := makeOptions(opt...)
digest, err := name.NewDigest(ref, o.Name...)
if err != nil {
return nil, err
}
return remote.Layer(digest, o.Remote...)
}
// SaveLegacy writes the v1.Image img as a legacy tarball at path with tag src.
func SaveLegacy(img v1.Image, src, path string) error {
imgMap := map[string]v1.Image{src: img}
return MultiSave(imgMap, path)
}
// MultiSaveLegacy writes collection of v1.Image img with tag as a legacy tarball.
func MultiSaveLegacy(imgMap map[string]v1.Image, path string) error {
refToImage := map[name.Reference]v1.Image{}
for src, img := range imgMap {
ref, err := name.ParseReference(src)
if err != nil {
return fmt.Errorf("parsing ref %q: %w", src, err)
}
refToImage[ref] = img
}
w, err := os.Create(path)
if err != nil {
return err
}
defer w.Close()
return legacy.MultiWrite(refToImage, w)
}
// SaveOCI writes the v1.Image img as an OCI Image Layout at path. If a layout
// already exists at that path, it will add the image to the index.
func SaveOCI(img v1.Image, path string) error {
imgMap := map[string]v1.Image{"": img}
return MultiSaveOCI(imgMap, path)
}
// MultiSaveOCI writes collection of v1.Image img as an OCI Image Layout at path. If a layout
// already exists at that path, it will add the image to the index.
func MultiSaveOCI(imgMap map[string]v1.Image, path string) error {
p, err := layout.FromPath(path)
if err != nil {
p, err = layout.Write(path, empty.Index)
if err != nil {
return err
}
}
for _, img := range imgMap {
if err = p.AppendImage(img); err != nil {
return err
}
}
return nil
}
-65
View File
@@ -1,65 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package crane
import (
"fmt"
"github.com/google/go-containerregistry/pkg/name"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/remote"
"github.com/google/go-containerregistry/pkg/v1/tarball"
)
// Load reads the tarball at path as a v1.Image.
func Load(path string, opt ...Option) (v1.Image, error) {
return LoadTag(path, "", opt...)
}
// LoadTag reads a tag from the tarball at path as a v1.Image.
// If tag is "", will attempt to read the tarball as a single image.
func LoadTag(path, tag string, opt ...Option) (v1.Image, error) {
if tag == "" {
return tarball.ImageFromPath(path, nil)
}
o := makeOptions(opt...)
t, err := name.NewTag(tag, o.Name...)
if err != nil {
return nil, fmt.Errorf("parsing tag %q: %w", tag, err)
}
return tarball.ImageFromPath(path, &t)
}
// Push pushes the v1.Image img to a registry as dst.
func Push(img v1.Image, dst string, opt ...Option) error {
o := makeOptions(opt...)
tag, err := name.ParseReference(dst, o.Name...)
if err != nil {
return fmt.Errorf("parsing reference %q: %w", dst, err)
}
return remote.Write(tag, img, o.Remote...)
}
// Upload pushes the v1.Layer to a given repo.
func Upload(layer v1.Layer, repo string, opt ...Option) error {
o := makeOptions(opt...)
ref, err := name.NewRepository(repo, o.Name...)
if err != nil {
return fmt.Errorf("parsing repo %q: %w", repo, err)
}
return remote.WriteLayer(ref, layer, o.Remote...)
}
-39
View File
@@ -1,39 +0,0 @@
// Copyright 2019 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package crane
import (
"fmt"
"github.com/google/go-containerregistry/pkg/name"
"github.com/google/go-containerregistry/pkg/v1/remote"
)
// Tag adds tag to the remote img.
func Tag(img, tag string, opt ...Option) error {
o := makeOptions(opt...)
ref, err := name.ParseReference(img, o.Name...)
if err != nil {
return fmt.Errorf("parsing reference %q: %w", img, err)
}
desc, err := remote.Get(ref, o.Remote...)
if err != nil {
return fmt.Errorf("fetching %q: %w", img, err)
}
dst := ref.Context().Tag(tag)
return remote.Tag(dst, desc, o.Remote...)
}
-33
View File
@@ -1,33 +0,0 @@
// Copyright 2019 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package legacy
import (
v1 "github.com/google/go-containerregistry/pkg/v1"
)
// LayerConfigFile is the configuration file that holds the metadata describing
// a v1 layer. See:
// https://github.com/moby/moby/blob/master/image/spec/v1.md
type LayerConfigFile struct {
v1.ConfigFile
ContainerConfig v1.Config `json:"container_config,omitempty"`
ID string `json:"id,omitempty"`
Parent string `json:"parent,omitempty"`
Throwaway bool `json:"throwaway,omitempty"`
Comment string `json:"comment,omitempty"`
}
-18
View File
@@ -1,18 +0,0 @@
// Copyright 2019 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// Package legacy provides functionality to work with docker images in the v1
// format.
// See: https://github.com/moby/moby/blob/master/image/spec/v1.md
package legacy
@@ -1,6 +0,0 @@
# `legacy/tarball`
[![GoDoc](https://godoc.org/github.com/google/go-containerregistry/pkg/legacy/tarball?status.svg)](https://godoc.org/github.com/google/go-containerregistry/pkg/legacy/tarball)
This package implements support for writing legacy tarballs, as described
[here](https://github.com/moby/moby/blob/749d90e10f989802638ae542daf54257f3bf71f2/image/spec/v1.2.md#combined-image-json--filesystem-changeset-format).
@@ -1,18 +0,0 @@
// Copyright 2019 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// Package tarball provides facilities for writing v1 docker images
// (https://github.com/moby/moby/blob/master/image/spec/v1.md) from/to a tarball
// on-disk.
package tarball
@@ -1,371 +0,0 @@
// Copyright 2019 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package tarball
import (
"archive/tar"
"bytes"
"encoding/json"
"fmt"
"io"
"sort"
"strings"
"github.com/google/go-containerregistry/pkg/legacy"
"github.com/google/go-containerregistry/pkg/name"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/partial"
"github.com/google/go-containerregistry/pkg/v1/tarball"
)
// repositoriesTarDescriptor represents the repositories file inside a `docker save` tarball.
type repositoriesTarDescriptor map[string]map[string]string
// v1Layer represents a layer with metadata needed by the v1 image spec https://github.com/moby/moby/blob/master/image/spec/v1.md.
type v1Layer struct {
// config is the layer metadata.
config *legacy.LayerConfigFile
// layer is the v1.Layer object this v1Layer represents.
layer v1.Layer
}
// json returns the raw bytes of the json metadata of the given v1Layer.
func (l *v1Layer) json() ([]byte, error) {
return json.Marshal(l.config)
}
// version returns the raw bytes of the "VERSION" file of the given v1Layer.
func (l *v1Layer) version() []byte {
return []byte("1.0")
}
// v1LayerID computes the v1 image format layer id for the given v1.Layer with the given v1 parent ID and raw image config.
func v1LayerID(layer v1.Layer, parentID string, rawConfig []byte) (string, error) {
d, err := layer.Digest()
if err != nil {
return "", fmt.Errorf("unable to get layer digest to generate v1 layer ID: %w", err)
}
s := fmt.Sprintf("%s %s", d.Hex, parentID)
if len(rawConfig) != 0 {
s = fmt.Sprintf("%s %s", s, string(rawConfig))
}
h, _, _ := v1.SHA256(strings.NewReader(s))
return h.Hex, nil
}
// newTopV1Layer creates a new v1Layer for a layer other than the top layer in a v1 image tarball.
func newV1Layer(layer v1.Layer, parent *v1Layer, history v1.History) (*v1Layer, error) {
parentID := ""
if parent != nil {
parentID = parent.config.ID
}
id, err := v1LayerID(layer, parentID, nil)
if err != nil {
return nil, fmt.Errorf("unable to generate v1 layer ID: %w", err)
}
result := &v1Layer{
layer: layer,
config: &legacy.LayerConfigFile{
ConfigFile: v1.ConfigFile{
Created: history.Created,
Author: history.Author,
},
ContainerConfig: v1.Config{
Cmd: []string{history.CreatedBy},
},
ID: id,
Parent: parentID,
Throwaway: history.EmptyLayer,
Comment: history.Comment,
},
}
return result, nil
}
// newTopV1Layer creates a new v1Layer for the top layer in a v1 image tarball.
func newTopV1Layer(layer v1.Layer, parent *v1Layer, history v1.History, imgConfig *v1.ConfigFile, rawConfig []byte) (*v1Layer, error) {
result, err := newV1Layer(layer, parent, history)
if err != nil {
return nil, err
}
id, err := v1LayerID(layer, result.config.Parent, rawConfig)
if err != nil {
return nil, fmt.Errorf("unable to generate v1 layer ID for top layer: %w", err)
}
result.config.ID = id
result.config.Architecture = imgConfig.Architecture
result.config.Container = imgConfig.Container
result.config.DockerVersion = imgConfig.DockerVersion //nolint:staticcheck // Field will be removed in next release
result.config.OS = imgConfig.OS
result.config.Config = imgConfig.Config
result.config.Created = imgConfig.Created
return result, nil
}
// splitTag splits the given tagged image name <registry>/<repository>:<tag>
// into <registry>/<repository> and <tag>.
func splitTag(name string) (string, string) {
// Split on ":"
parts := strings.Split(name, ":")
// Verify that we aren't confusing a tag for a hostname w/ port for the purposes of weak validation.
if len(parts) > 1 && !strings.Contains(parts[len(parts)-1], "/") {
base := strings.Join(parts[:len(parts)-1], ":")
tag := parts[len(parts)-1]
return base, tag
}
return name, ""
}
// addTags adds the given image tags to the given "repositories" file descriptor in a v1 image tarball.
func addTags(repos repositoriesTarDescriptor, tags []string, topLayerID string) {
for _, t := range tags {
base, tag := splitTag(t)
tagToID, ok := repos[base]
if !ok {
tagToID = make(map[string]string)
repos[base] = tagToID
}
tagToID[tag] = topLayerID
}
}
// updateLayerSources updates the given layer digest to descriptor map with the descriptor of the given layer in the given image if it's an undistributable layer.
func updateLayerSources(layerSources map[v1.Hash]v1.Descriptor, layer v1.Layer, img v1.Image) error {
d, err := layer.Digest()
if err != nil {
return err
}
// Add to LayerSources if it's a foreign layer.
desc, err := partial.BlobDescriptor(img, d)
if err != nil {
return err
}
if !desc.MediaType.IsDistributable() {
diffid, err := partial.BlobToDiffID(img, d)
if err != nil {
return err
}
layerSources[diffid] = *desc
}
return nil
}
// Write is a wrapper to write a single image in V1 format and tag to a tarball.
func Write(ref name.Reference, img v1.Image, w io.Writer) error {
return MultiWrite(map[name.Reference]v1.Image{ref: img}, w)
}
// filterEmpty filters out the history corresponding to empty layers from the
// given history.
func filterEmpty(h []v1.History) []v1.History {
result := []v1.History{}
for _, i := range h {
if i.EmptyLayer {
continue
}
result = append(result, i)
}
return result
}
// MultiWrite writes the contents of each image to the provided reader, in the V1 image tarball format.
// The contents are written in the following format:
// One manifest.json file at the top level containing information about several images.
// One repositories file mapping from the image <registry>/<repo name> to <tag> to the id of the top most layer.
// For every layer, a directory named with the layer ID is created with the following contents:
//
// layer.tar - The uncompressed layer tarball.
// <layer id>.json- Layer metadata json.
// VERSION- Schema version string. Always set to "1.0".
//
// One file for the config blob, named after its SHA.
func MultiWrite(refToImage map[name.Reference]v1.Image, w io.Writer) error {
tf := tar.NewWriter(w)
defer tf.Close()
sortedImages, imageToTags := dedupRefToImage(refToImage)
var m tarball.Manifest
repos := make(repositoriesTarDescriptor)
seenLayerIDs := make(map[string]struct{})
for _, img := range sortedImages {
tags := imageToTags[img]
// Write the config.
cfgName, err := img.ConfigName()
if err != nil {
return err
}
cfgFileName := fmt.Sprintf("%s.json", cfgName.Hex)
cfgBlob, err := img.RawConfigFile()
if err != nil {
return err
}
if err := writeTarEntry(tf, cfgFileName, bytes.NewReader(cfgBlob), int64(len(cfgBlob))); err != nil {
return err
}
cfg, err := img.ConfigFile()
if err != nil {
return err
}
// Store foreign layer info.
layerSources := make(map[v1.Hash]v1.Descriptor)
// Write the layers.
layers, err := img.Layers()
if err != nil {
return err
}
history := filterEmpty(cfg.History)
// Create a blank config history if the config didn't have a history.
if len(history) == 0 && len(layers) != 0 {
history = make([]v1.History, len(layers))
} else if len(layers) != len(history) {
return fmt.Errorf("image config had layer history which did not match the number of layers, got len(history)=%d, len(layers)=%d, want len(history)=len(layers)", len(history), len(layers))
}
layerFiles := make([]string, len(layers))
var prev *v1Layer
for i, l := range layers {
if err := updateLayerSources(layerSources, l, img); err != nil {
return fmt.Errorf("unable to update image metadata to include undistributable layer source information: %w", err)
}
var cur *v1Layer
if i < (len(layers) - 1) {
cur, err = newV1Layer(l, prev, history[i])
} else {
cur, err = newTopV1Layer(l, prev, history[i], cfg, cfgBlob)
}
if err != nil {
return err
}
layerFiles[i] = fmt.Sprintf("%s/layer.tar", cur.config.ID)
if _, ok := seenLayerIDs[cur.config.ID]; ok {
prev = cur
continue
}
seenLayerIDs[cur.config.ID] = struct{}{}
// If the v1.Layer implements UncompressedSize efficiently, use that
// for the tar header. Otherwise, this iterates over Uncompressed().
// NOTE: If using a streaming layer, this may consume the layer.
size, err := partial.UncompressedSize(l)
if err != nil {
return err
}
u, err := l.Uncompressed()
if err != nil {
return err
}
defer u.Close()
if err := writeTarEntry(tf, layerFiles[i], u, size); err != nil {
return err
}
j, err := cur.json()
if err != nil {
return err
}
if err := writeTarEntry(tf, fmt.Sprintf("%s/json", cur.config.ID), bytes.NewReader(j), int64(len(j))); err != nil {
return err
}
v := cur.version()
if err := writeTarEntry(tf, fmt.Sprintf("%s/VERSION", cur.config.ID), bytes.NewReader(v), int64(len(v))); err != nil {
return err
}
prev = cur
}
// Generate the tar descriptor and write it.
m = append(m, tarball.Descriptor{
Config: cfgFileName,
RepoTags: tags,
Layers: layerFiles,
LayerSources: layerSources,
})
// prev should be the top layer here. Use it to add the image tags
// to the tarball repositories file.
addTags(repos, tags, prev.config.ID)
}
mBytes, err := json.Marshal(m)
if err != nil {
return err
}
if err := writeTarEntry(tf, "manifest.json", bytes.NewReader(mBytes), int64(len(mBytes))); err != nil {
return err
}
reposBytes, err := json.Marshal(&repos)
if err != nil {
return err
}
return writeTarEntry(tf, "repositories", bytes.NewReader(reposBytes), int64(len(reposBytes)))
}
func dedupRefToImage(refToImage map[name.Reference]v1.Image) ([]v1.Image, map[v1.Image][]string) {
imageToTags := make(map[v1.Image][]string)
for ref, img := range refToImage {
if tag, ok := ref.(name.Tag); ok {
if tags, ok := imageToTags[img]; ok && tags != nil {
imageToTags[img] = append(tags, tag.String())
} else {
imageToTags[img] = []string{tag.String()}
}
} else {
if _, ok := imageToTags[img]; !ok {
imageToTags[img] = nil
}
}
}
// Force specific order on tags
imgs := make([]v1.Image, 0, len(imageToTags))
for img, tags := range imageToTags {
sort.Strings(tags)
imgs = append(imgs, img)
}
sort.Slice(imgs, func(i, j int) bool {
cfI, err := imgs[i].ConfigName()
if err != nil {
return false
}
cfJ, err := imgs[j].ConfigName()
if err != nil {
return false
}
return cfI.Hex < cfJ.Hex
})
return imgs, imageToTags
}
// Writes a file to the provided writer with a corresponding tar header
func writeTarEntry(tf *tar.Writer, path string, r io.Reader, size int64) error {
hdr := &tar.Header{
Mode: 0644,
Typeflag: tar.TypeReg,
Size: size,
Name: path,
}
if err := tf.WriteHeader(hdr); err != nil {
return err
}
_, err := io.Copy(tf, r)
return err
}
-39
View File
@@ -1,39 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// Package logs exposes the loggers used by this library.
package logs
import (
"io"
"log"
)
var (
// Warn is used to log non-fatal errors.
Warn = log.New(io.Discard, "", log.LstdFlags)
// Progress is used to log notable, successful events.
Progress = log.New(io.Discard, "", log.LstdFlags)
// Debug is used to log information that is useful for debugging.
Debug = log.New(io.Discard, "", log.LstdFlags)
)
// Enabled checks to see if the logger's writer is set to something other
// than io.Discard. This allows callers to avoid expensive operations
// that will end up in /dev/null anyway.
func Enabled(l *log.Logger) bool {
return l.Writer() != io.Discard
}
-3
View File
@@ -1,3 +0,0 @@
# `name`
[![GoDoc](https://godoc.org/github.com/google/go-containerregistry/pkg/name?status.svg)](https://godoc.org/github.com/google/go-containerregistry/pkg/name)
-43
View File
@@ -1,43 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package name
import (
"strings"
"unicode/utf8"
)
// stripRunesFn returns a function which returns -1 (i.e. a value which
// signals deletion in strings.Map) for runes in 'runes', and the rune otherwise.
func stripRunesFn(runes string) func(rune) rune {
return func(r rune) rune {
if strings.ContainsRune(runes, r) {
return -1
}
return r
}
}
// checkElement checks a given named element matches character and length restrictions.
// Returns true if the given element adheres to the given restrictions, false otherwise.
func checkElement(name, element, allowedRunes string, minRunes, maxRunes int) error {
numRunes := utf8.RuneCountInString(element)
if (numRunes < minRunes) || (maxRunes < numRunes) {
return newErrBadName("%s must be between %d and %d characters in length: %s", name, minRunes, maxRunes, element)
} else if len(strings.Map(stripRunesFn(allowedRunes), element)) != 0 {
return newErrBadName("%s can only contain the characters `%s`: %s", name, allowedRunes, element)
}
return nil
}
-133
View File
@@ -1,133 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package name
import (
// nolint: depguard
_ "crypto/sha256" // Recommended by go-digest.
"encoding"
"encoding/json"
"strings"
"github.com/opencontainers/go-digest"
)
const digestDelim = "@"
// Digest stores a digest name in a structured form.
type Digest struct {
Repository
digest string
original string
}
var _ Reference = (*Digest)(nil)
var _ encoding.TextMarshaler = (*Digest)(nil)
var _ encoding.TextUnmarshaler = (*Digest)(nil)
var _ json.Marshaler = (*Digest)(nil)
var _ json.Unmarshaler = (*Digest)(nil)
// Context implements Reference.
func (d Digest) Context() Repository {
return d.Repository
}
// Identifier implements Reference.
func (d Digest) Identifier() string {
return d.DigestStr()
}
// DigestStr returns the digest component of the Digest.
func (d Digest) DigestStr() string {
return d.digest
}
// Name returns the name from which the Digest was derived.
func (d Digest) Name() string {
return d.Repository.Name() + digestDelim + d.DigestStr()
}
// String returns the original input string.
func (d Digest) String() string {
return d.original
}
// MarshalJSON formats the digest into a string for JSON serialization.
func (d Digest) MarshalJSON() ([]byte, error) {
return json.Marshal(d.String())
}
// UnmarshalJSON parses a JSON string into a Digest.
func (d *Digest) UnmarshalJSON(data []byte) error {
var s string
if err := json.Unmarshal(data, &s); err != nil {
return err
}
n, err := NewDigest(s)
if err != nil {
return err
}
*d = n
return nil
}
// MarshalText formats the digest into a string for text serialization.
func (d Digest) MarshalText() ([]byte, error) {
return []byte(d.String()), nil
}
// UnmarshalText parses a text string into a Digest.
func (d *Digest) UnmarshalText(data []byte) error {
n, err := NewDigest(string(data))
if err != nil {
return err
}
*d = n
return nil
}
// NewDigest returns a new Digest representing the given name.
func NewDigest(name string, opts ...Option) (Digest, error) {
// Split on "@"
parts := strings.Split(name, digestDelim)
if len(parts) != 2 {
return Digest{}, newErrBadName("a digest must contain exactly one '@' separator (e.g. registry/repository@digest) saw: %s", name)
}
base := parts[0]
dig := parts[1]
prefix := digest.Canonical.String() + ":"
if !strings.HasPrefix(dig, prefix) {
return Digest{}, newErrBadName("unsupported digest algorithm: %s", dig)
}
hex := strings.TrimPrefix(dig, prefix)
if err := digest.Canonical.Validate(hex); err != nil {
return Digest{}, err
}
tag, err := NewTag(base, opts...)
if err == nil {
base = tag.Repository.Name()
}
repo, err := NewRepository(base, opts...)
if err != nil {
return Digest{}, err
}
return Digest{
Repository: repo,
digest: dig,
original: name,
}, nil
}
-42
View File
@@ -1,42 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// Package name defines structured types for representing image references.
//
// What's in a name? For image references, not nearly enough!
//
// Image references look a lot like URLs, but they differ in that they don't
// contain the scheme (http or https), they can end with a :tag or a @digest
// (the latter being validated), and they perform defaulting for missing
// components.
//
// Since image references don't contain the scheme, we do our best to infer
// if we use http or https from the given hostname. We allow http fallback for
// any host that looks like localhost (localhost, 127.0.0.1, ::1), ends in
// ".local", or is in the "private" address space per RFC 1918. For everything
// else, we assume https only. To override this heuristic, use the Insecure
// option.
//
// Image references with a digest signal to us that we should verify the content
// of the image matches the digest. E.g. when pulling a Digest reference, we'll
// calculate the sha256 of the manifest returned by the registry and error out
// if it doesn't match what we asked for.
//
// For defaulting, we interpret "ubuntu" as
// "index.docker.io/library/ubuntu:latest" because we add the missing repo
// "library", the missing registry "index.docker.io", and the missing tag
// "latest". To disable this defaulting, use the StrictValidation option. This
// is useful e.g. to only allow image references that explicitly set a tag or
// digest, so that you don't accidentally pull "latest".
package name
-48
View File
@@ -1,48 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package name
import (
"errors"
"fmt"
)
// ErrBadName is an error for when a bad docker name is supplied.
type ErrBadName struct {
info string
}
func (e *ErrBadName) Error() string {
return e.info
}
// Is reports whether target is an error of type ErrBadName
func (e *ErrBadName) Is(target error) bool {
var berr *ErrBadName
return errors.As(target, &berr)
}
// newErrBadName returns a ErrBadName which returns the given formatted string from Error().
func newErrBadName(fmtStr string, args ...any) *ErrBadName {
return &ErrBadName{fmt.Sprintf(fmtStr, args...)}
}
// IsErrBadName returns true if the given error is an ErrBadName.
//
// Deprecated: Use errors.Is.
func IsErrBadName(err error) bool {
var berr *ErrBadName
return errors.As(err, &berr)
}
-83
View File
@@ -1,83 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package name
const (
// DefaultRegistry is the registry name that will be used if no registry
// provided and the default is not overridden.
DefaultRegistry = "index.docker.io"
defaultRegistryAlias = "docker.io"
// DefaultTag is the tag name that will be used if no tag provided and the
// default is not overridden.
DefaultTag = "latest"
)
type options struct {
strict bool // weak by default
insecure bool // secure by default
defaultRegistry string
defaultTag string
}
func makeOptions(opts ...Option) options {
opt := options{
defaultRegistry: DefaultRegistry,
defaultTag: DefaultTag,
}
for _, o := range opts {
o(&opt)
}
return opt
}
// Option is a functional option for name parsing.
type Option func(*options)
// StrictValidation is an Option that requires image references to be fully
// specified; i.e. no defaulting for registry (dockerhub), repo (library),
// or tag (latest).
func StrictValidation(opts *options) {
opts.strict = true
}
// WeakValidation is an Option that sets defaults when parsing names, see
// StrictValidation.
func WeakValidation(opts *options) {
opts.strict = false
}
// Insecure is an Option that allows image references to be fetched without TLS.
func Insecure(opts *options) {
opts.insecure = true
}
// OptionFn is a function that returns an option.
type OptionFn func() Option
// WithDefaultRegistry sets the default registry that will be used if one is not
// provided.
func WithDefaultRegistry(r string) Option {
return func(opts *options) {
opts.defaultRegistry = r
}
}
// WithDefaultTag sets the default tag that will be used if one is not provided.
func WithDefaultTag(t string) Option {
return func(opts *options) {
opts.defaultTag = t
}
}
-75
View File
@@ -1,75 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package name
import (
"fmt"
)
// Reference defines the interface that consumers use when they can
// take either a tag or a digest.
type Reference interface {
fmt.Stringer
// Context accesses the Repository context of the reference.
Context() Repository
// Identifier accesses the type-specific portion of the reference.
Identifier() string
// Name is the fully-qualified reference name.
Name() string
// Scope is the scope needed to access this reference.
Scope(string) string
}
// ParseReference parses the string as a reference, either by tag or digest.
func ParseReference(s string, opts ...Option) (Reference, error) {
if t, err := NewTag(s, opts...); err == nil {
return t, nil
}
if d, err := NewDigest(s, opts...); err == nil {
return d, nil
}
return nil, newErrBadName("could not parse reference: %s", s)
}
type stringConst string
// MustParseReference behaves like ParseReference, but panics instead of
// returning an error. It's intended for use in tests, or when a value is
// expected to be valid at code authoring time.
//
// To discourage its use in scenarios where the value is not known at code
// authoring time, it must be passed a string constant:
//
// const str = "valid/string"
// MustParseReference(str)
// MustParseReference("another/valid/string")
// MustParseReference(str + "/and/more")
//
// These will not compile:
//
// var str = "valid/string"
// MustParseReference(str)
// MustParseReference(strings.Join([]string{"valid", "string"}, "/"))
func MustParseReference(s stringConst, opts ...Option) Reference {
ref, err := ParseReference(string(s), opts...)
if err != nil {
panic(err)
}
return ref
}
-179
View File
@@ -1,179 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package name
import (
"encoding"
"encoding/json"
"net"
"net/url"
"path"
"regexp"
"strings"
)
// Detect more complex forms of local references.
var reLocal = regexp.MustCompile(`.*\.local(?:host)?(?::\d{1,5})?$`)
// Detect the loopback IP (127.0.0.1)
var reLoopback = regexp.MustCompile(regexp.QuoteMeta("127.0.0.1"))
// Detect the loopback IPV6 (::1)
var reipv6Loopback = regexp.MustCompile(regexp.QuoteMeta("::1"))
// Registry stores a docker registry name in a structured form.
type Registry struct {
insecure bool
registry string
}
var _ encoding.TextMarshaler = (*Registry)(nil)
var _ encoding.TextUnmarshaler = (*Registry)(nil)
var _ json.Marshaler = (*Registry)(nil)
var _ json.Unmarshaler = (*Registry)(nil)
// RegistryStr returns the registry component of the Registry.
func (r Registry) RegistryStr() string {
return r.registry
}
// Name returns the name from which the Registry was derived.
func (r Registry) Name() string {
return r.RegistryStr()
}
func (r Registry) String() string {
return r.Name()
}
// Repo returns a Repository in the Registry with the given name.
func (r Registry) Repo(repo ...string) Repository {
return Repository{Registry: r, repository: path.Join(repo...)}
}
// Scope returns the scope required to access the registry.
func (r Registry) Scope(string) string {
// The only resource under 'registry' is 'catalog'. http://goo.gl/N9cN9Z
return "registry:catalog:*"
}
func (r Registry) isRFC1918() bool {
ipStr := strings.Split(r.Name(), ":")[0]
ip := net.ParseIP(ipStr)
if ip == nil {
return false
}
for _, cidr := range []string{"10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16"} {
_, block, _ := net.ParseCIDR(cidr)
if block.Contains(ip) {
return true
}
}
return false
}
// Scheme returns https scheme for all the endpoints except localhost or when explicitly defined.
func (r Registry) Scheme() string {
if r.insecure {
return "http"
}
if r.isRFC1918() {
return "http"
}
if strings.HasPrefix(r.Name(), "localhost:") {
return "http"
}
if reLocal.MatchString(r.Name()) {
return "http"
}
if reLoopback.MatchString(r.Name()) {
return "http"
}
if reipv6Loopback.MatchString(r.Name()) {
return "http"
}
return "https"
}
func checkRegistry(name string) error {
// Per RFC 3986, registries (authorities) are required to be prefixed with "//"
// url.Host == hostname[:port] == authority
if url, err := url.Parse("//" + name); err != nil || url.Host != name {
return newErrBadName("registries must be valid RFC 3986 URI authorities: %s", name)
}
return nil
}
// NewRegistry returns a Registry based on the given name.
// Strict validation requires explicit, valid RFC 3986 URI authorities to be given.
func NewRegistry(name string, opts ...Option) (Registry, error) {
opt := makeOptions(opts...)
if opt.strict && len(name) == 0 {
return Registry{}, newErrBadName("strict validation requires the registry to be explicitly defined")
}
if err := checkRegistry(name); err != nil {
return Registry{}, err
}
if name == "" {
name = opt.defaultRegistry
}
// Rewrite "docker.io" to "index.docker.io".
// See: https://github.com/google/go-containerregistry/issues/68
if name == defaultRegistryAlias {
name = DefaultRegistry
}
return Registry{registry: name, insecure: opt.insecure}, nil
}
// NewInsecureRegistry returns an Insecure Registry based on the given name.
//
// Deprecated: Use the Insecure Option with NewRegistry instead.
func NewInsecureRegistry(name string, opts ...Option) (Registry, error) {
opts = append(opts, Insecure)
return NewRegistry(name, opts...)
}
// MarshalJSON formats the Registry into a string for JSON serialization.
func (r Registry) MarshalJSON() ([]byte, error) { return json.Marshal(r.String()) }
// UnmarshalJSON parses a JSON string into a Registry.
func (r *Registry) UnmarshalJSON(data []byte) error {
var s string
if err := json.Unmarshal(data, &s); err != nil {
return err
}
n, err := NewRegistry(s)
if err != nil {
return err
}
*r = n
return nil
}
// MarshalText formats the registry into a string for text serialization.
func (r Registry) MarshalText() ([]byte, error) { return []byte(r.String()), nil }
// UnmarshalText parses a text string into a Registry.
func (r *Registry) UnmarshalText(data []byte) error {
n, err := NewRegistry(string(data))
if err != nil {
return err
}
*r = n
return nil
}
-158
View File
@@ -1,158 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package name
import (
"encoding"
"encoding/json"
"fmt"
"strings"
)
const (
defaultNamespace = "library"
repositoryChars = "abcdefghijklmnopqrstuvwxyz0123456789_-./"
regRepoDelimiter = "/"
)
// Repository stores a docker repository name in a structured form.
type Repository struct {
Registry
repository string
}
var _ encoding.TextMarshaler = (*Repository)(nil)
var _ encoding.TextUnmarshaler = (*Repository)(nil)
var _ json.Marshaler = (*Repository)(nil)
var _ json.Unmarshaler = (*Repository)(nil)
// See https://docs.docker.com/docker-hub/official_repos
func hasImplicitNamespace(repo string, reg Registry) bool {
return !strings.ContainsRune(repo, '/') && reg.RegistryStr() == DefaultRegistry
}
// RepositoryStr returns the repository component of the Repository.
func (r Repository) RepositoryStr() string {
if hasImplicitNamespace(r.repository, r.Registry) {
return fmt.Sprintf("%s/%s", defaultNamespace, r.repository)
}
return r.repository
}
// Name returns the name from which the Repository was derived.
func (r Repository) Name() string {
regName := r.Registry.Name()
if regName != "" {
return regName + regRepoDelimiter + r.RepositoryStr()
}
// TODO: As far as I can tell, this is unreachable.
return r.RepositoryStr()
}
func (r Repository) String() string {
return r.Name()
}
// Scope returns the scope required to perform the given action on the registry.
// TODO(jonjohnsonjr): consider moving scopes to a separate package.
func (r Repository) Scope(action string) string {
return fmt.Sprintf("repository:%s:%s", r.RepositoryStr(), action)
}
func checkRepository(repository string) error {
return checkElement("repository", repository, repositoryChars, 2, 255)
}
// NewRepository returns a new Repository representing the given name, according to the given strictness.
func NewRepository(name string, opts ...Option) (Repository, error) {
opt := makeOptions(opts...)
if len(name) == 0 {
return Repository{}, newErrBadName("a repository name must be specified")
}
var registry string
repo := name
parts := strings.SplitN(name, regRepoDelimiter, 2)
if len(parts) == 2 && (strings.ContainsRune(parts[0], '.') || strings.ContainsRune(parts[0], ':')) {
// The first part of the repository is treated as the registry domain
// iff it contains a '.' or ':' character, otherwise it is all repository
// and the domain defaults to Docker Hub.
registry = parts[0]
repo = parts[1]
}
if err := checkRepository(repo); err != nil {
return Repository{}, err
}
reg, err := NewRegistry(registry, opts...)
if err != nil {
return Repository{}, err
}
if hasImplicitNamespace(repo, reg) && opt.strict {
return Repository{}, newErrBadName("strict validation requires the full repository path (missing 'library')")
}
return Repository{reg, repo}, nil
}
// Tag returns a Tag in this Repository.
func (r Repository) Tag(identifier string) Tag {
t := Tag{
tag: identifier,
Repository: r,
}
t.original = t.Name()
return t
}
// Digest returns a Digest in this Repository.
func (r Repository) Digest(identifier string) Digest {
d := Digest{
digest: identifier,
Repository: r,
}
d.original = d.Name()
return d
}
// MarshalJSON formats the Repository into a string for JSON serialization.
func (r Repository) MarshalJSON() ([]byte, error) { return json.Marshal(r.String()) }
// UnmarshalJSON parses a JSON string into a Repository.
func (r *Repository) UnmarshalJSON(data []byte) error {
var s string
if err := json.Unmarshal(data, &s); err != nil {
return err
}
n, err := NewRepository(s)
if err != nil {
return err
}
*r = n
return nil
}
// MarshalText formats the repository name into a string for text serialization.
func (r Repository) MarshalText() ([]byte, error) { return []byte(r.String()), nil }
// UnmarshalText parses a text string into a Repository.
func (r *Repository) UnmarshalText(data []byte) error {
n, err := NewRepository(string(data))
if err != nil {
return err
}
*r = n
return nil
}
-146
View File
@@ -1,146 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package name
import (
"encoding"
"encoding/json"
"strings"
)
const (
// TODO(dekkagaijin): use the docker/distribution regexes for validation.
tagChars = "abcdefghijklmnopqrstuvwxyz0123456789_-.ABCDEFGHIJKLMNOPQRSTUVWXYZ"
tagDelim = ":"
)
// Tag stores a docker tag name in a structured form.
type Tag struct {
Repository
tag string
original string
}
var _ Reference = (*Tag)(nil)
var _ encoding.TextMarshaler = (*Tag)(nil)
var _ encoding.TextUnmarshaler = (*Tag)(nil)
var _ json.Marshaler = (*Tag)(nil)
var _ json.Unmarshaler = (*Tag)(nil)
// Context implements Reference.
func (t Tag) Context() Repository {
return t.Repository
}
// Identifier implements Reference.
func (t Tag) Identifier() string {
return t.TagStr()
}
// TagStr returns the tag component of the Tag.
func (t Tag) TagStr() string {
return t.tag
}
// Name returns the name from which the Tag was derived.
func (t Tag) Name() string {
return t.Repository.Name() + tagDelim + t.TagStr()
}
// String returns the original input string.
func (t Tag) String() string {
return t.original
}
// Scope returns the scope required to perform the given action on the tag.
func (t Tag) Scope(action string) string {
return t.Repository.Scope(action)
}
func checkTag(name string) error {
return checkElement("tag", name, tagChars, 1, 128)
}
// NewTag returns a new Tag representing the given name, according to the given strictness.
func NewTag(name string, opts ...Option) (Tag, error) {
opt := makeOptions(opts...)
base := name
tag := ""
// Split on ":"
parts := strings.Split(name, tagDelim)
// Verify that we aren't confusing a tag for a hostname w/ port for the purposes of weak validation.
if len(parts) > 1 && !strings.Contains(parts[len(parts)-1], regRepoDelimiter) {
base = strings.Join(parts[:len(parts)-1], tagDelim)
tag = parts[len(parts)-1]
if tag == "" {
return Tag{}, newErrBadName("%s must specify a tag name after the colon", name)
}
}
// We don't require a tag, but if we get one check it's valid,
// even when not being strict.
// If we are being strict, we want to validate the tag regardless in case
// it's empty.
if tag != "" || opt.strict {
if err := checkTag(tag); err != nil {
return Tag{}, err
}
}
if tag == "" {
tag = opt.defaultTag
}
repo, err := NewRepository(base, opts...)
if err != nil {
return Tag{}, err
}
return Tag{
Repository: repo,
tag: tag,
original: name,
}, nil
}
// MarshalJSON formats the Tag into a string for JSON serialization.
func (t Tag) MarshalJSON() ([]byte, error) { return json.Marshal(t.String()) }
// UnmarshalJSON parses a JSON string into a Tag.
func (t *Tag) UnmarshalJSON(data []byte) error {
var s string
if err := json.Unmarshal(data, &s); err != nil {
return err
}
n, err := NewTag(s)
if err != nil {
return err
}
*t = n
return nil
}
// MarshalText formats the tag into a string for text serialization.
func (t Tag) MarshalText() ([]byte, error) { return []byte(t.String()), nil }
// UnmarshalText parses a text string into a Tag.
func (t *Tag) UnmarshalText(data []byte) error {
n, err := NewTag(string(data))
if err != nil {
return err
}
*t = n
return nil
}
-152
View File
@@ -1,152 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package v1
import (
"encoding/json"
"io"
"time"
)
// ConfigFile is the configuration file that holds the metadata describing
// how to launch a container. See:
// https://github.com/opencontainers/image-spec/blob/master/config.md
//
// docker_version and os.version are not part of the spec but included
// for backwards compatibility.
type ConfigFile struct {
Architecture string `json:"architecture"`
Author string `json:"author,omitempty"`
Container string `json:"container,omitempty"`
Created Time `json:"created,omitempty"`
// Deprecated: This field is deprecated and will be removed in the next release.
DockerVersion string `json:"docker_version,omitempty"`
History []History `json:"history,omitempty"`
OS string `json:"os"`
RootFS RootFS `json:"rootfs"`
Config Config `json:"config"`
OSVersion string `json:"os.version,omitempty"`
Variant string `json:"variant,omitempty"`
OSFeatures []string `json:"os.features,omitempty"`
}
// Platform attempts to generates a Platform from the ConfigFile fields.
func (cf *ConfigFile) Platform() *Platform {
if cf.OS == "" && cf.Architecture == "" && cf.OSVersion == "" && cf.Variant == "" && len(cf.OSFeatures) == 0 {
return nil
}
return &Platform{
OS: cf.OS,
Architecture: cf.Architecture,
OSVersion: cf.OSVersion,
Variant: cf.Variant,
OSFeatures: cf.OSFeatures,
}
}
// History is one entry of a list recording how this container image was built.
type History struct {
Author string `json:"author,omitempty"`
Created Time `json:"created,omitempty"`
CreatedBy string `json:"created_by,omitempty"`
Comment string `json:"comment,omitempty"`
EmptyLayer bool `json:"empty_layer,omitempty"`
}
// Time is a wrapper around time.Time to help with deep copying
type Time struct {
time.Time
}
// DeepCopyInto creates a deep-copy of the Time value. The underlying time.Time
// type is effectively immutable in the time API, so it is safe to
// copy-by-assign, despite the presence of (unexported) Pointer fields.
func (t *Time) DeepCopyInto(out *Time) {
*out = *t
}
// RootFS holds the ordered list of file system deltas that comprise the
// container image's root filesystem.
type RootFS struct {
Type string `json:"type"`
DiffIDs []Hash `json:"diff_ids"`
}
// HealthConfig holds configuration settings for the HEALTHCHECK feature.
type HealthConfig struct {
// Test is the test to perform to check that the container is healthy.
// An empty slice means to inherit the default.
// The options are:
// {} : inherit healthcheck
// {"NONE"} : disable healthcheck
// {"CMD", args...} : exec arguments directly
// {"CMD-SHELL", command} : run command with system's default shell
Test []string `json:",omitempty"`
// Zero means to inherit. Durations are expressed as integer nanoseconds.
Interval time.Duration `json:",omitempty"` // Interval is the time to wait between checks.
Timeout time.Duration `json:",omitempty"` // Timeout is the time to wait before considering the check to have hung.
StartPeriod time.Duration `json:",omitempty"` // The start period for the container to initialize before the retries starts to count down.
// Retries is the number of consecutive failures needed to consider a container as unhealthy.
// Zero means inherit.
Retries int `json:",omitempty"`
}
// Config is a submessage of the config file described as:
//
// The execution parameters which SHOULD be used as a base when running
// a container using the image.
//
// The names of the fields in this message are chosen to reflect the JSON
// payload of the Config as defined here:
// https://git.io/vrAET
// and
// https://github.com/opencontainers/image-spec/blob/master/config.md
type Config struct {
AttachStderr bool `json:"AttachStderr,omitempty"`
AttachStdin bool `json:"AttachStdin,omitempty"`
AttachStdout bool `json:"AttachStdout,omitempty"`
Cmd []string `json:"Cmd,omitempty"`
Healthcheck *HealthConfig `json:"Healthcheck,omitempty"`
Domainname string `json:"Domainname,omitempty"`
Entrypoint []string `json:"Entrypoint,omitempty"`
Env []string `json:"Env,omitempty"`
Hostname string `json:"Hostname,omitempty"`
Image string `json:"Image,omitempty"`
Labels map[string]string `json:"Labels,omitempty"`
OnBuild []string `json:"OnBuild,omitempty"`
OpenStdin bool `json:"OpenStdin,omitempty"`
StdinOnce bool `json:"StdinOnce,omitempty"`
Tty bool `json:"Tty,omitempty"`
User string `json:"User,omitempty"`
Volumes map[string]struct{} `json:"Volumes,omitempty"`
WorkingDir string `json:"WorkingDir,omitempty"`
ExposedPorts map[string]struct{} `json:"ExposedPorts,omitempty"`
ArgsEscaped bool `json:"ArgsEscaped,omitempty"`
NetworkDisabled bool `json:"NetworkDisabled,omitempty"`
MacAddress string `json:"MacAddress,omitempty"`
StopSignal string `json:"StopSignal,omitempty"`
Shell []string `json:"Shell,omitempty"`
}
// ParseConfigFile parses the io.Reader's contents into a ConfigFile.
func ParseConfigFile(r io.Reader) (*ConfigFile, error) {
cf := ConfigFile{}
if err := json.NewDecoder(r).Decode(&cf); err != nil {
return nil, err
}
return &cf, nil
}
-18
View File
@@ -1,18 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// +k8s:deepcopy-gen=package
// Package v1 defines structured types for OCI v1 images
package v1
-8
View File
@@ -1,8 +0,0 @@
# `empty`
[![GoDoc](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/empty?status.svg)](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/empty)
The empty packages provides an empty base for constructing a `v1.Image` or `v1.ImageIndex`.
This is especially useful when paired with the [`mutate`](/pkg/v1/mutate) package,
see [`mutate.Append`](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/mutate#Append)
and [`mutate.AppendManifests`](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/mutate#AppendManifests).
-16
View File
@@ -1,16 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// Package empty provides an implementation of v1.Image equivalent to "FROM scratch".
package empty
-52
View File
@@ -1,52 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package empty
import (
"fmt"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/partial"
"github.com/google/go-containerregistry/pkg/v1/types"
)
// Image is a singleton empty image, think: FROM scratch.
var Image, _ = partial.UncompressedToImage(emptyImage{})
type emptyImage struct{}
// MediaType implements partial.UncompressedImageCore.
func (i emptyImage) MediaType() (types.MediaType, error) {
return types.DockerManifestSchema2, nil
}
// RawConfigFile implements partial.UncompressedImageCore.
func (i emptyImage) RawConfigFile() ([]byte, error) {
return partial.RawConfigFile(i)
}
// ConfigFile implements v1.Image.
func (i emptyImage) ConfigFile() (*v1.ConfigFile, error) {
return &v1.ConfigFile{
RootFS: v1.RootFS{
// Some clients check this.
Type: "layers",
},
}, nil
}
func (i emptyImage) LayerByDiffID(h v1.Hash) (partial.UncompressedLayer, error) {
return nil, fmt.Errorf("LayerByDiffID(%s): empty image", h)
}
-65
View File
@@ -1,65 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package empty
import (
"encoding/json"
"errors"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/partial"
"github.com/google/go-containerregistry/pkg/v1/types"
)
// Index is a singleton empty index, think: FROM scratch.
var Index = emptyIndex{}
type emptyIndex struct{}
func (i emptyIndex) MediaType() (types.MediaType, error) {
return types.OCIImageIndex, nil
}
func (i emptyIndex) Digest() (v1.Hash, error) {
return partial.Digest(i)
}
func (i emptyIndex) Size() (int64, error) {
return partial.Size(i)
}
func (i emptyIndex) IndexManifest() (*v1.IndexManifest, error) {
return base(), nil
}
func (i emptyIndex) RawManifest() ([]byte, error) {
return json.Marshal(base())
}
func (i emptyIndex) Image(v1.Hash) (v1.Image, error) {
return nil, errors.New("empty index")
}
func (i emptyIndex) ImageIndex(v1.Hash) (v1.ImageIndex, error) {
return nil, errors.New("empty index")
}
func base() *v1.IndexManifest {
return &v1.IndexManifest{
SchemaVersion: 2,
MediaType: types.OCIImageIndex,
Manifests: []v1.Descriptor{},
}
}
-130
View File
@@ -1,130 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package v1
import (
"crypto"
"encoding"
"encoding/hex"
"encoding/json"
"fmt"
"hash"
"io"
"strings"
)
// Hash is an unqualified digest of some content, e.g. sha256:deadbeef
type Hash struct {
// Algorithm holds the algorithm used to compute the hash.
Algorithm string
// Hex holds the hex portion of the content hash.
Hex string
}
var _ encoding.TextMarshaler = (*Hash)(nil)
var _ encoding.TextUnmarshaler = (*Hash)(nil)
var _ json.Marshaler = (*Hash)(nil)
var _ json.Unmarshaler = (*Hash)(nil)
// String reverses NewHash returning the string-form of the hash.
func (h Hash) String() string {
return fmt.Sprintf("%s:%s", h.Algorithm, h.Hex)
}
// NewHash validates the input string is a hash and returns a strongly type Hash object.
func NewHash(s string) (Hash, error) {
h := Hash{}
if err := h.parse(s); err != nil {
return Hash{}, err
}
return h, nil
}
// MarshalJSON implements json.Marshaler
func (h Hash) MarshalJSON() ([]byte, error) { return json.Marshal(h.String()) }
// UnmarshalJSON implements json.Unmarshaler
func (h *Hash) UnmarshalJSON(data []byte) error {
var s string
if err := json.Unmarshal(data, &s); err != nil {
return err
}
return h.parse(s)
}
// MarshalText implements encoding.TextMarshaler. This is required to use
// v1.Hash as a key in a map when marshalling JSON.
func (h Hash) MarshalText() ([]byte, error) { return []byte(h.String()), nil }
// UnmarshalText implements encoding.TextUnmarshaler. This is required to use
// v1.Hash as a key in a map when unmarshalling JSON.
func (h *Hash) UnmarshalText(text []byte) error { return h.parse(string(text)) }
// Hasher returns a hash.Hash for the named algorithm (e.g. "sha256")
func Hasher(name string) (hash.Hash, error) {
switch name {
case "sha256":
return crypto.SHA256.New(), nil
default:
return nil, fmt.Errorf("unsupported hash: %q", name)
}
}
func (h *Hash) parse(unquoted string) error {
algo, body, ok := strings.Cut(unquoted, ":")
if !ok || algo == "" || body == "" {
return fmt.Errorf("cannot parse hash: %q", unquoted)
}
rest := strings.TrimLeft(body, "0123456789abcdef")
if len(rest) != 0 {
return fmt.Errorf("found non-hex character in hash: %c", rest[0])
}
var wantBytes int
switch algo {
case "sha256":
wantBytes = crypto.SHA256.Size()
default:
hasher, err := Hasher(algo)
if err != nil {
return err
}
wantBytes = hasher.Size()
}
// Compare the hex to the expected size (2 hex characters per byte)
if len(body) != hex.EncodedLen(wantBytes) {
return fmt.Errorf("wrong number of hex digits for %s: %s", algo, body)
}
h.Algorithm = algo
h.Hex = body
return nil
}
// SHA256 computes the Hash of the provided io.Reader's content.
func SHA256(r io.Reader) (Hash, int64, error) {
hasher := crypto.SHA256.New()
n, err := io.Copy(hasher, r)
if err != nil {
return Hash{}, 0, err
}
return Hash{
Algorithm: "sha256",
Hex: hex.EncodeToString(hasher.Sum(make([]byte, 0, hasher.Size()))),
}, n, nil
}
-59
View File
@@ -1,59 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package v1
import (
"github.com/google/go-containerregistry/pkg/v1/types"
)
// Image defines the interface for interacting with an OCI v1 image.
type Image interface {
// Layers returns the ordered collection of filesystem layers that comprise this image.
// The order of the list is oldest/base layer first, and most-recent/top layer last.
Layers() ([]Layer, error)
// MediaType of this image's manifest.
MediaType() (types.MediaType, error)
// Size returns the size of the manifest.
Size() (int64, error)
// ConfigName returns the hash of the image's config file, also known as
// the Image ID.
ConfigName() (Hash, error)
// ConfigFile returns this image's config file.
ConfigFile() (*ConfigFile, error)
// RawConfigFile returns the serialized bytes of ConfigFile().
RawConfigFile() ([]byte, error)
// Digest returns the sha256 of this image's manifest.
Digest() (Hash, error)
// Manifest returns this image's Manifest object.
Manifest() (*Manifest, error)
// RawManifest returns the serialized bytes of Manifest()
RawManifest() ([]byte, error)
// LayerByDigest returns a Layer for interacting with a particular layer of
// the image, looking it up by "digest" (the compressed hash).
LayerByDigest(Hash) (Layer, error)
// LayerByDiffID is an analog to LayerByDigest, looking up by "diff id"
// (the uncompressed hash).
LayerByDiffID(Hash) (Layer, error)
}
-43
View File
@@ -1,43 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package v1
import (
"github.com/google/go-containerregistry/pkg/v1/types"
)
// ImageIndex defines the interface for interacting with an OCI image index.
type ImageIndex interface {
// MediaType of this image's manifest.
MediaType() (types.MediaType, error)
// Digest returns the sha256 of this index's manifest.
Digest() (Hash, error)
// Size returns the size of the manifest.
Size() (int64, error)
// IndexManifest returns this image index's manifest object.
IndexManifest() (*IndexManifest, error)
// RawManifest returns the serialized bytes of IndexManifest().
RawManifest() ([]byte, error)
// Image returns a v1.Image that this ImageIndex references.
Image(Hash) (Image, error)
// ImageIndex returns a v1.ImageIndex that this ImageIndex references.
ImageIndex(Hash) (ImageIndex, error)
}
-42
View File
@@ -1,42 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package v1
import (
"io"
"github.com/google/go-containerregistry/pkg/v1/types"
)
// Layer is an interface for accessing the properties of a particular layer of a v1.Image
type Layer interface {
// Digest returns the Hash of the compressed layer.
Digest() (Hash, error)
// DiffID returns the Hash of the uncompressed layer.
DiffID() (Hash, error)
// Compressed returns an io.ReadCloser for the compressed layer contents.
Compressed() (io.ReadCloser, error)
// Uncompressed returns an io.ReadCloser for the uncompressed layer contents.
Uncompressed() (io.ReadCloser, error)
// Size returns the compressed size of the Layer.
Size() (int64, error)
// MediaType returns the media type of the Layer.
MediaType() (types.MediaType, error)
}
@@ -1,5 +0,0 @@
# `layout`
[![GoDoc](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/layout?status.svg)](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/layout)
The `layout` package implements support for interacting with an [OCI Image Layout](https://github.com/opencontainers/image-spec/blob/master/image-layout.md).
-37
View File
@@ -1,37 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package layout
import (
"io"
"os"
v1 "github.com/google/go-containerregistry/pkg/v1"
)
// Blob returns a blob with the given hash from the Path.
func (l Path) Blob(h v1.Hash) (io.ReadCloser, error) {
return os.Open(l.blobPath(h))
}
// Bytes is a convenience function to return a blob from the Path as
// a byte slice.
func (l Path) Bytes(h v1.Hash) ([]byte, error) {
return os.ReadFile(l.blobPath(h))
}
func (l Path) blobPath(h v1.Hash) string {
return l.path("blobs", h.Algorithm, h.Hex)
}
-19
View File
@@ -1,19 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// Package layout provides facilities for reading/writing artifacts from/to
// an OCI image layout on disk, see:
//
// https://github.com/opencontainers/image-spec/blob/master/image-layout.md
package layout
-137
View File
@@ -1,137 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// This is an EXPERIMENTAL package, and may change in arbitrary ways without notice.
package layout
import (
"fmt"
"io/fs"
"path/filepath"
"strings"
v1 "github.com/google/go-containerregistry/pkg/v1"
)
// GarbageCollect removes unreferenced blobs from the oci-layout
//
// This is an experimental api, and not subject to any stability guarantees
// We may abandon it at any time, without prior notice.
// Deprecated: Use it at your own risk!
func (l Path) GarbageCollect() ([]v1.Hash, error) {
idx, err := l.ImageIndex()
if err != nil {
return nil, err
}
blobsToKeep := map[string]bool{}
if err := l.garbageCollectImageIndex(idx, blobsToKeep); err != nil {
return nil, err
}
blobsDir := l.path("blobs")
removedBlobs := []v1.Hash{}
err = filepath.WalkDir(blobsDir, func(path string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
if d.IsDir() {
return nil
}
rel, err := filepath.Rel(blobsDir, path)
if err != nil {
return err
}
hashString := strings.Replace(rel, "/", ":", 1)
if present := blobsToKeep[hashString]; !present {
h, err := v1.NewHash(hashString)
if err != nil {
return err
}
removedBlobs = append(removedBlobs, h)
}
return nil
})
if err != nil {
return nil, err
}
return removedBlobs, nil
}
func (l Path) garbageCollectImageIndex(index v1.ImageIndex, blobsToKeep map[string]bool) error {
idxm, err := index.IndexManifest()
if err != nil {
return err
}
h, err := index.Digest()
if err != nil {
return err
}
blobsToKeep[h.String()] = true
for _, descriptor := range idxm.Manifests {
if descriptor.MediaType.IsImage() {
img, err := index.Image(descriptor.Digest)
if err != nil {
return err
}
if err := l.garbageCollectImage(img, blobsToKeep); err != nil {
return err
}
} else if descriptor.MediaType.IsIndex() {
idx, err := index.ImageIndex(descriptor.Digest)
if err != nil {
return err
}
if err := l.garbageCollectImageIndex(idx, blobsToKeep); err != nil {
return err
}
} else {
return fmt.Errorf("gc: unknown media type: %s", descriptor.MediaType)
}
}
return nil
}
func (l Path) garbageCollectImage(image v1.Image, blobsToKeep map[string]bool) error {
h, err := image.Digest()
if err != nil {
return err
}
blobsToKeep[h.String()] = true
h, err = image.ConfigName()
if err != nil {
return err
}
blobsToKeep[h.String()] = true
ls, err := image.Layers()
if err != nil {
return err
}
for _, l := range ls {
h, err := l.Digest()
if err != nil {
return err
}
blobsToKeep[h.String()] = true
}
return nil
}
-139
View File
@@ -1,139 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package layout
import (
"fmt"
"io"
"os"
"sync"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/partial"
"github.com/google/go-containerregistry/pkg/v1/types"
)
type layoutImage struct {
path Path
desc v1.Descriptor
manifestLock sync.Mutex // Protects rawManifest
rawManifest []byte
}
var _ partial.CompressedImageCore = (*layoutImage)(nil)
// Image reads a v1.Image with digest h from the Path.
func (l Path) Image(h v1.Hash) (v1.Image, error) {
ii, err := l.ImageIndex()
if err != nil {
return nil, err
}
return ii.Image(h)
}
func (li *layoutImage) MediaType() (types.MediaType, error) {
return li.desc.MediaType, nil
}
// Implements WithManifest for partial.Blobset.
func (li *layoutImage) Manifest() (*v1.Manifest, error) {
return partial.Manifest(li)
}
func (li *layoutImage) RawManifest() ([]byte, error) {
li.manifestLock.Lock()
defer li.manifestLock.Unlock()
if li.rawManifest != nil {
return li.rawManifest, nil
}
b, err := li.path.Bytes(li.desc.Digest)
if err != nil {
return nil, err
}
li.rawManifest = b
return li.rawManifest, nil
}
func (li *layoutImage) RawConfigFile() ([]byte, error) {
manifest, err := li.Manifest()
if err != nil {
return nil, err
}
return li.path.Bytes(manifest.Config.Digest)
}
func (li *layoutImage) LayerByDigest(h v1.Hash) (partial.CompressedLayer, error) {
manifest, err := li.Manifest()
if err != nil {
return nil, err
}
if h == manifest.Config.Digest {
return &compressedBlob{
path: li.path,
desc: manifest.Config,
}, nil
}
for _, desc := range manifest.Layers {
if h == desc.Digest {
return &compressedBlob{
path: li.path,
desc: desc,
}, nil
}
}
return nil, fmt.Errorf("could not find layer in image: %s", h)
}
type compressedBlob struct {
path Path
desc v1.Descriptor
}
func (b *compressedBlob) Digest() (v1.Hash, error) {
return b.desc.Digest, nil
}
func (b *compressedBlob) Compressed() (io.ReadCloser, error) {
return b.path.Blob(b.desc.Digest)
}
func (b *compressedBlob) Size() (int64, error) {
return b.desc.Size, nil
}
func (b *compressedBlob) MediaType() (types.MediaType, error) {
return b.desc.MediaType, nil
}
// Descriptor implements partial.withDescriptor.
func (b *compressedBlob) Descriptor() (*v1.Descriptor, error) {
return &b.desc, nil
}
// See partial.Exists.
func (b *compressedBlob) Exists() (bool, error) {
_, err := os.Stat(b.path.blobPath(b.desc.Digest))
if os.IsNotExist(err) {
return false, nil
}
return err == nil, err
}
-161
View File
@@ -1,161 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package layout
import (
"encoding/json"
"errors"
"fmt"
"io"
"os"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/partial"
"github.com/google/go-containerregistry/pkg/v1/types"
)
var _ v1.ImageIndex = (*layoutIndex)(nil)
type layoutIndex struct {
mediaType types.MediaType
path Path
rawIndex []byte
}
// ImageIndexFromPath is a convenience function which constructs a Path and returns its v1.ImageIndex.
func ImageIndexFromPath(path string) (v1.ImageIndex, error) {
lp, err := FromPath(path)
if err != nil {
return nil, err
}
return lp.ImageIndex()
}
// ImageIndex returns a v1.ImageIndex for the Path.
func (l Path) ImageIndex() (v1.ImageIndex, error) {
rawIndex, err := os.ReadFile(l.path("index.json"))
if err != nil {
return nil, err
}
idx := &layoutIndex{
mediaType: types.OCIImageIndex,
path: l,
rawIndex: rawIndex,
}
return idx, nil
}
func (i *layoutIndex) MediaType() (types.MediaType, error) {
return i.mediaType, nil
}
func (i *layoutIndex) Digest() (v1.Hash, error) {
return partial.Digest(i)
}
func (i *layoutIndex) Size() (int64, error) {
return partial.Size(i)
}
func (i *layoutIndex) IndexManifest() (*v1.IndexManifest, error) {
var index v1.IndexManifest
err := json.Unmarshal(i.rawIndex, &index)
return &index, err
}
func (i *layoutIndex) RawManifest() ([]byte, error) {
return i.rawIndex, nil
}
func (i *layoutIndex) Image(h v1.Hash) (v1.Image, error) {
// Look up the digest in our manifest first to return a better error.
desc, err := i.findDescriptor(h)
if err != nil {
return nil, err
}
if !isExpectedMediaType(desc.MediaType, types.OCIManifestSchema1, types.DockerManifestSchema2) {
return nil, fmt.Errorf("unexpected media type for %v: %s", h, desc.MediaType)
}
img := &layoutImage{
path: i.path,
desc: *desc,
}
return partial.CompressedToImage(img)
}
func (i *layoutIndex) ImageIndex(h v1.Hash) (v1.ImageIndex, error) {
// Look up the digest in our manifest first to return a better error.
desc, err := i.findDescriptor(h)
if err != nil {
return nil, err
}
if !isExpectedMediaType(desc.MediaType, types.OCIImageIndex, types.DockerManifestList) {
return nil, fmt.Errorf("unexpected media type for %v: %s", h, desc.MediaType)
}
rawIndex, err := i.path.Bytes(h)
if err != nil {
return nil, err
}
return &layoutIndex{
mediaType: desc.MediaType,
path: i.path,
rawIndex: rawIndex,
}, nil
}
func (i *layoutIndex) Blob(h v1.Hash) (io.ReadCloser, error) {
return i.path.Blob(h)
}
func (i *layoutIndex) findDescriptor(h v1.Hash) (*v1.Descriptor, error) {
im, err := i.IndexManifest()
if err != nil {
return nil, err
}
if h == (v1.Hash{}) {
if len(im.Manifests) != 1 {
return nil, errors.New("oci layout must contain only a single image to be used with layout.Image")
}
return &(im.Manifests)[0], nil
}
for _, desc := range im.Manifests {
if desc.Digest == h {
return &desc, nil
}
}
return nil, fmt.Errorf("could not find descriptor in index: %s", h)
}
// TODO: Pull this out into methods on types.MediaType? e.g. instead, have:
// * mt.IsIndex()
// * mt.IsImage()
func isExpectedMediaType(mt types.MediaType, expected ...types.MediaType) bool {
for _, allowed := range expected {
if mt == allowed {
return true
}
}
return false
}
@@ -1,25 +0,0 @@
// Copyright 2019 The original author or authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package layout
import "path/filepath"
// Path represents an OCI image layout rooted in a file system path
type Path string
func (l Path) path(elem ...string) string {
complete := []string{string(l)}
return filepath.Join(append(complete, elem...)...)
}
-71
View File
@@ -1,71 +0,0 @@
// Copyright 2019 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package layout
import v1 "github.com/google/go-containerregistry/pkg/v1"
// Option is a functional option for Layout.
type Option func(*options)
type options struct {
descOpts []descriptorOption
}
func makeOptions(opts ...Option) *options {
o := &options{
descOpts: []descriptorOption{},
}
for _, apply := range opts {
apply(o)
}
return o
}
type descriptorOption func(*v1.Descriptor)
// WithAnnotations adds annotations to the artifact descriptor.
func WithAnnotations(annotations map[string]string) Option {
return func(o *options) {
o.descOpts = append(o.descOpts, func(desc *v1.Descriptor) {
if desc.Annotations == nil {
desc.Annotations = make(map[string]string)
}
for k, v := range annotations {
desc.Annotations[k] = v
}
})
}
}
// WithURLs adds urls to the artifact descriptor.
func WithURLs(urls []string) Option {
return func(o *options) {
o.descOpts = append(o.descOpts, func(desc *v1.Descriptor) {
if desc.URLs == nil {
desc.URLs = []string{}
}
desc.URLs = append(desc.URLs, urls...)
})
}
}
// WithPlatform sets the platform of the artifact descriptor.
func WithPlatform(platform v1.Platform) Option {
return func(o *options) {
o.descOpts = append(o.descOpts, func(desc *v1.Descriptor) {
desc.Platform = &platform
})
}
}
-32
View File
@@ -1,32 +0,0 @@
// Copyright 2019 The original author or authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package layout
import (
"os"
"path/filepath"
)
// FromPath reads an OCI image layout at path and constructs a layout.Path.
func FromPath(path string) (Path, error) {
// TODO: check oci-layout exists
_, err := os.Stat(filepath.Join(path, "index.json"))
if err != nil {
return "", err
}
return Path(path), nil
}
-492
View File
@@ -1,492 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package layout
import (
"bytes"
"encoding/json"
"errors"
"fmt"
"io"
"os"
"path/filepath"
"runtime"
"sync"
"github.com/google/go-containerregistry/pkg/logs"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/match"
"github.com/google/go-containerregistry/pkg/v1/mutate"
"github.com/google/go-containerregistry/pkg/v1/partial"
"github.com/google/go-containerregistry/pkg/v1/stream"
"github.com/google/go-containerregistry/pkg/v1/types"
"golang.org/x/sync/errgroup"
)
var layoutFile = `{
"imageLayoutVersion": "1.0.0"
}`
// renameMutex guards os.Rename calls in AppendImage on Windows only.
var renameMutex sync.Mutex
// AppendImage writes a v1.Image to the Path and updates
// the index.json to reference it.
func (l Path) AppendImage(img v1.Image, options ...Option) error {
if err := l.WriteImage(img); err != nil {
return err
}
desc, err := partial.Descriptor(img)
if err != nil {
return err
}
o := makeOptions(options...)
for _, opt := range o.descOpts {
opt(desc)
}
return l.AppendDescriptor(*desc)
}
// AppendIndex writes a v1.ImageIndex to the Path and updates
// the index.json to reference it.
func (l Path) AppendIndex(ii v1.ImageIndex, options ...Option) error {
if err := l.WriteIndex(ii); err != nil {
return err
}
desc, err := partial.Descriptor(ii)
if err != nil {
return err
}
o := makeOptions(options...)
for _, opt := range o.descOpts {
opt(desc)
}
return l.AppendDescriptor(*desc)
}
// AppendDescriptor adds a descriptor to the index.json of the Path.
func (l Path) AppendDescriptor(desc v1.Descriptor) error {
ii, err := l.ImageIndex()
if err != nil {
return err
}
index, err := ii.IndexManifest()
if err != nil {
return err
}
index.Manifests = append(index.Manifests, desc)
rawIndex, err := json.MarshalIndent(index, "", " ")
if err != nil {
return err
}
return l.WriteFile("index.json", rawIndex, os.ModePerm)
}
// ReplaceImage writes a v1.Image to the Path and updates
// the index.json to reference it, replacing any existing one that matches matcher, if found.
func (l Path) ReplaceImage(img v1.Image, matcher match.Matcher, options ...Option) error {
if err := l.WriteImage(img); err != nil {
return err
}
return l.replaceDescriptor(img, matcher, options...)
}
// ReplaceIndex writes a v1.ImageIndex to the Path and updates
// the index.json to reference it, replacing any existing one that matches matcher, if found.
func (l Path) ReplaceIndex(ii v1.ImageIndex, matcher match.Matcher, options ...Option) error {
if err := l.WriteIndex(ii); err != nil {
return err
}
return l.replaceDescriptor(ii, matcher, options...)
}
// replaceDescriptor adds a descriptor to the index.json of the Path, replacing
// any one matching matcher, if found.
func (l Path) replaceDescriptor(appendable mutate.Appendable, matcher match.Matcher, options ...Option) error {
ii, err := l.ImageIndex()
if err != nil {
return err
}
desc, err := partial.Descriptor(appendable)
if err != nil {
return err
}
o := makeOptions(options...)
for _, opt := range o.descOpts {
opt(desc)
}
add := mutate.IndexAddendum{
Add: appendable,
Descriptor: *desc,
}
ii = mutate.AppendManifests(mutate.RemoveManifests(ii, matcher), add)
index, err := ii.IndexManifest()
if err != nil {
return err
}
rawIndex, err := json.MarshalIndent(index, "", " ")
if err != nil {
return err
}
return l.WriteFile("index.json", rawIndex, os.ModePerm)
}
// RemoveDescriptors removes any descriptors that match the match.Matcher from the index.json of the Path.
func (l Path) RemoveDescriptors(matcher match.Matcher) error {
ii, err := l.ImageIndex()
if err != nil {
return err
}
ii = mutate.RemoveManifests(ii, matcher)
index, err := ii.IndexManifest()
if err != nil {
return err
}
rawIndex, err := json.MarshalIndent(index, "", " ")
if err != nil {
return err
}
return l.WriteFile("index.json", rawIndex, os.ModePerm)
}
// WriteFile write a file with arbitrary data at an arbitrary location in a v1
// layout. Used mostly internally to write files like "oci-layout" and
// "index.json", also can be used to write other arbitrary files. Do *not* use
// this to write blobs. Use only WriteBlob() for that.
func (l Path) WriteFile(name string, data []byte, perm os.FileMode) error {
if err := os.MkdirAll(l.path(), os.ModePerm); err != nil && !os.IsExist(err) {
return err
}
return os.WriteFile(l.path(name), data, perm)
}
// WriteBlob copies a file to the blobs/ directory in the Path from the given ReadCloser at
// blobs/{hash.Algorithm}/{hash.Hex}.
func (l Path) WriteBlob(hash v1.Hash, r io.ReadCloser) error {
return l.writeBlob(hash, -1, r, nil)
}
func (l Path) writeBlob(hash v1.Hash, size int64, rc io.ReadCloser, renamer func() (v1.Hash, error)) error {
defer rc.Close()
if hash.Hex == "" && renamer == nil {
panic("writeBlob called an invalid hash and no renamer")
}
dir := l.path("blobs", hash.Algorithm)
if err := os.MkdirAll(dir, os.ModePerm); err != nil && !os.IsExist(err) {
return err
}
// Check if blob already exists and is the correct size
file := filepath.Join(dir, hash.Hex)
if s, err := os.Stat(file); err == nil && !s.IsDir() && (s.Size() == size || size == -1) {
return nil
}
// If a renamer func was provided write to a temporary file
open := func() (*os.File, error) { return os.Create(file) }
if renamer != nil {
open = func() (*os.File, error) { return os.CreateTemp(dir, hash.Hex) }
}
w, err := open()
if err != nil {
return err
}
if renamer != nil {
// Delete temp file if an error is encountered before renaming
defer func() {
if err := os.Remove(w.Name()); err != nil && !errors.Is(err, os.ErrNotExist) {
logs.Warn.Printf("error removing temporary file after encountering an error while writing blob: %v", err)
}
}()
}
defer w.Close()
// Write to file and exit if not renaming
if n, err := io.Copy(w, rc); err != nil || renamer == nil {
return err
} else if size != -1 && n != size {
return fmt.Errorf("expected blob size %d, but only wrote %d", size, n)
}
// Always close reader before renaming, since Close computes the digest in
// the case of streaming layers. If Close is not called explicitly, it will
// occur in a goroutine that is not guaranteed to succeed before renamer is
// called. When renamer is the layer's Digest method, it can return
// ErrNotComputed.
if err := rc.Close(); err != nil {
return err
}
// Always close file before renaming
if err := w.Close(); err != nil {
return err
}
// Rename file based on the final hash
finalHash, err := renamer()
if err != nil {
return fmt.Errorf("error getting final digest of layer: %w", err)
}
renamePath := l.path("blobs", finalHash.Algorithm, finalHash.Hex)
if runtime.GOOS == "windows" {
renameMutex.Lock()
defer renameMutex.Unlock()
}
return os.Rename(w.Name(), renamePath)
}
// writeLayer writes the compressed layer to a blob. Unlike WriteBlob it will
// write to a temporary file (suffixed with .tmp) within the layout until the
// compressed reader is fully consumed and written to disk. Also unlike
// WriteBlob, it will not skip writing and exit without error when a blob file
// exists, but does not have the correct size. (The blob hash is not
// considered, because it may be expensive to compute.)
func (l Path) writeLayer(layer v1.Layer) error {
d, err := layer.Digest()
if errors.Is(err, stream.ErrNotComputed) {
// Allow digest errors, since streams may not have calculated the hash
// yet. Instead, use an empty value, which will be transformed into a
// random file name with `os.CreateTemp` and the final digest will be
// calculated after writing to a temp file and before renaming to the
// final path.
d = v1.Hash{Algorithm: "sha256", Hex: ""}
} else if err != nil {
return err
}
s, err := layer.Size()
if errors.Is(err, stream.ErrNotComputed) {
// Allow size errors, since streams may not have calculated the size
// yet. Instead, use zero as a sentinel value meaning that no size
// comparison can be done and any sized blob file should be considered
// valid and not overwritten.
//
// TODO: Provide an option to always overwrite blobs.
s = -1
} else if err != nil {
return err
}
r, err := layer.Compressed()
if err != nil {
return err
}
if err := l.writeBlob(d, s, r, layer.Digest); err != nil {
return fmt.Errorf("error writing layer: %w", err)
}
return nil
}
// RemoveBlob removes a file from the blobs directory in the Path
// at blobs/{hash.Algorithm}/{hash.Hex}
// It does *not* remove any reference to it from other manifests or indexes, or
// from the root index.json.
func (l Path) RemoveBlob(hash v1.Hash) error {
dir := l.path("blobs", hash.Algorithm)
err := os.Remove(filepath.Join(dir, hash.Hex))
if err != nil && !os.IsNotExist(err) {
return err
}
return nil
}
// WriteImage writes an image, including its manifest, config and all of its
// layers, to the blobs directory. If any blob already exists, as determined by
// the hash filename, does not write it.
// This function does *not* update the `index.json` file. If you want to write the
// image and also update the `index.json`, call AppendImage(), which wraps this
// and also updates the `index.json`.
func (l Path) WriteImage(img v1.Image) error {
layers, err := img.Layers()
if err != nil {
return err
}
// Write the layers concurrently.
var g errgroup.Group
for _, layer := range layers {
layer := layer
g.Go(func() error {
return l.writeLayer(layer)
})
}
if err := g.Wait(); err != nil {
return err
}
// Write the config.
cfgName, err := img.ConfigName()
if err != nil {
return err
}
cfgBlob, err := img.RawConfigFile()
if err != nil {
return err
}
if err := l.WriteBlob(cfgName, io.NopCloser(bytes.NewReader(cfgBlob))); err != nil {
return err
}
// Write the img manifest.
d, err := img.Digest()
if err != nil {
return err
}
manifest, err := img.RawManifest()
if err != nil {
return err
}
return l.WriteBlob(d, io.NopCloser(bytes.NewReader(manifest)))
}
type withLayer interface {
Layer(v1.Hash) (v1.Layer, error)
}
type withBlob interface {
Blob(v1.Hash) (io.ReadCloser, error)
}
func (l Path) writeIndexToFile(indexFile string, ii v1.ImageIndex) error {
index, err := ii.IndexManifest()
if err != nil {
return err
}
// Walk the descriptors and write any v1.Image or v1.ImageIndex that we find.
// If we come across something we don't expect, just write it as a blob.
for _, desc := range index.Manifests {
switch desc.MediaType {
case types.OCIImageIndex, types.DockerManifestList:
ii, err := ii.ImageIndex(desc.Digest)
if err != nil {
return err
}
if err := l.WriteIndex(ii); err != nil {
return err
}
case types.OCIManifestSchema1, types.DockerManifestSchema2:
img, err := ii.Image(desc.Digest)
if err != nil {
return err
}
if err := l.WriteImage(img); err != nil {
return err
}
default:
// TODO: The layout could reference arbitrary things, which we should
// probably just pass through.
var blob io.ReadCloser
// Workaround for #819.
if wl, ok := ii.(withLayer); ok {
layer, lerr := wl.Layer(desc.Digest)
if lerr != nil {
return lerr
}
blob, err = layer.Compressed()
} else if wb, ok := ii.(withBlob); ok {
blob, err = wb.Blob(desc.Digest)
}
if err != nil {
return err
}
if err := l.WriteBlob(desc.Digest, blob); err != nil {
return err
}
}
}
rawIndex, err := ii.RawManifest()
if err != nil {
return err
}
return l.WriteFile(indexFile, rawIndex, os.ModePerm)
}
// WriteIndex writes an index to the blobs directory. Walks down the children,
// including its children manifests and/or indexes, and down the tree until all of
// config and all layers, have been written. If any blob already exists, as determined by
// the hash filename, does not write it.
// This function does *not* update the `index.json` file. If you want to write the
// index and also update the `index.json`, call AppendIndex(), which wraps this
// and also updates the `index.json`.
func (l Path) WriteIndex(ii v1.ImageIndex) error {
// Always just write oci-layout file, since it's small.
if err := l.WriteFile("oci-layout", []byte(layoutFile), os.ModePerm); err != nil {
return err
}
h, err := ii.Digest()
if err != nil {
return err
}
indexFile := filepath.Join("blobs", h.Algorithm, h.Hex)
return l.writeIndexToFile(indexFile, ii)
}
// Write constructs a Path at path from an ImageIndex.
//
// The contents are written in the following format:
// At the top level, there is:
//
// One oci-layout file containing the version of this image-layout.
// One index.json file listing descriptors for the contained images.
//
// Under blobs/, there is, for each image:
//
// One file for each layer, named after the layer's SHA.
// One file for each config blob, named after its SHA.
// One file for each manifest blob, named after its SHA.
func Write(path string, ii v1.ImageIndex) (Path, error) {
lp := Path(path)
// Always just write oci-layout file, since it's small.
if err := lp.WriteFile("oci-layout", []byte(layoutFile), os.ModePerm); err != nil {
return "", err
}
// TODO create blobs/ in case there is a blobs file which would prevent the directory from being created
return lp, lp.writeIndexToFile("index.json", ii)
}
-71
View File
@@ -1,71 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package v1
import (
"encoding/json"
"io"
"github.com/google/go-containerregistry/pkg/v1/types"
)
// Manifest represents the OCI image manifest in a structured way.
type Manifest struct {
SchemaVersion int64 `json:"schemaVersion"`
MediaType types.MediaType `json:"mediaType,omitempty"`
Config Descriptor `json:"config"`
Layers []Descriptor `json:"layers"`
Annotations map[string]string `json:"annotations,omitempty"`
Subject *Descriptor `json:"subject,omitempty"`
}
// IndexManifest represents an OCI image index in a structured way.
type IndexManifest struct {
SchemaVersion int64 `json:"schemaVersion"`
MediaType types.MediaType `json:"mediaType,omitempty"`
Manifests []Descriptor `json:"manifests"`
Annotations map[string]string `json:"annotations,omitempty"`
Subject *Descriptor `json:"subject,omitempty"`
}
// Descriptor holds a reference from the manifest to one of its constituent elements.
type Descriptor struct {
MediaType types.MediaType `json:"mediaType"`
Size int64 `json:"size"`
Digest Hash `json:"digest"`
Data []byte `json:"data,omitempty"`
URLs []string `json:"urls,omitempty"`
Annotations map[string]string `json:"annotations,omitempty"`
Platform *Platform `json:"platform,omitempty"`
ArtifactType string `json:"artifactType,omitempty"`
}
// ParseManifest parses the io.Reader's contents into a Manifest.
func ParseManifest(r io.Reader) (*Manifest, error) {
m := Manifest{}
if err := json.NewDecoder(r).Decode(&m); err != nil {
return nil, err
}
return &m, nil
}
// ParseIndexManifest parses the io.Reader's contents into an IndexManifest.
func ParseIndexManifest(r io.Reader) (*IndexManifest, error) {
im := IndexManifest{}
if err := json.NewDecoder(r).Decode(&im); err != nil {
return nil, err
}
return &im, nil
}
-92
View File
@@ -1,92 +0,0 @@
// Copyright 2020 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// Package match provides functionality for conveniently matching a v1.Descriptor.
package match
import (
v1 "github.com/google/go-containerregistry/pkg/v1"
imagespec "github.com/opencontainers/image-spec/specs-go/v1"
)
// Matcher function that is given a v1.Descriptor, and returns whether or
// not it matches a given rule. Can match on anything it wants in the Descriptor.
type Matcher func(desc v1.Descriptor) bool
// Name returns a match.Matcher that matches based on the value of the
//
// "org.opencontainers.image.ref.name" annotation:
//
// github.com/opencontainers/image-spec/blob/v1.0.1/annotations.md#pre-defined-annotation-keys
func Name(name string) Matcher {
return Annotation(imagespec.AnnotationRefName, name)
}
// Annotation returns a match.Matcher that matches based on the provided annotation.
func Annotation(key, value string) Matcher {
return func(desc v1.Descriptor) bool {
if desc.Annotations == nil {
return false
}
if aValue, ok := desc.Annotations[key]; ok && aValue == value {
return true
}
return false
}
}
// Platforms returns a match.Matcher that matches on any one of the provided platforms.
// Ignores any descriptors that do not have a platform.
func Platforms(platforms ...v1.Platform) Matcher {
return func(desc v1.Descriptor) bool {
if desc.Platform == nil {
return false
}
for _, platform := range platforms {
if desc.Platform.Equals(platform) {
return true
}
}
return false
}
}
// MediaTypes returns a match.Matcher that matches at least one of the provided media types.
func MediaTypes(mediaTypes ...string) Matcher {
mts := map[string]bool{}
for _, media := range mediaTypes {
mts[media] = true
}
return func(desc v1.Descriptor) bool {
if desc.MediaType == "" {
return false
}
if _, ok := mts[string(desc.MediaType)]; ok {
return true
}
return false
}
}
// Digests returns a match.Matcher that matches at least one of the provided Digests
func Digests(digests ...v1.Hash) Matcher {
digs := map[v1.Hash]bool{}
for _, digest := range digests {
digs[digest] = true
}
return func(desc v1.Descriptor) bool {
_, ok := digs[desc.Digest]
return ok
}
}
-56
View File
@@ -1,56 +0,0 @@
# `mutate`
[![GoDoc](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/mutate?status.svg)](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/mutate)
The `v1.Image`, `v1.ImageIndex`, and `v1.Layer` interfaces provide only
accessor methods, so they are essentially immutable. If you want to change
something about them, you need to produce a new instance of that interface.
A common use case for this library is to read an image from somewhere (a source),
change something about it, and write the image somewhere else (a sink).
Graphically, this looks something like:
<p align="center">
<img src="/images/mutate.dot.svg" />
</p>
## Mutations
This is obviously not a comprehensive set of useful transformations (PRs welcome!),
but a rough summary of what the `mutate` package currently does:
### `Config` and `ConfigFile`
These allow you to change the [image configuration](https://github.com/opencontainers/image-spec/blob/master/config.md#properties),
e.g. to change the entrypoint, environment, author, etc.
### `Time`, `Canonical`, and `CreatedAt`
These are useful in the context of [reproducible builds](https://reproducible-builds.org/),
where you may want to strip timestamps and other non-reproducible information.
### `Append`, `AppendLayers`, and `AppendManifests`
These functions allow the extension of a `v1.Image` or `v1.ImageIndex` with
new layers or manifests.
For constructing an image `FROM scratch`, see the [`empty`](/pkg/v1/empty) package.
### `MediaType` and `IndexMediaType`
Sometimes, it is necessary to change the media type of an image or index,
e.g. to appease a registry with strict validation of images (_looking at you, GCR_).
### `Rebase`
Rebase has [its own README](/cmd/crane/rebase.md).
This is the underlying implementation of [`crane rebase`](https://github.com/google/go-containerregistry/blob/main/cmd/crane/doc/crane_rebase.md).
### `Extract`
Extract will flatten an image filesystem into a single tar stream,
respecting whiteout files.
This is the underlying implementation of [`crane export`](https://github.com/google/go-containerregistry/blob/main/cmd/crane/doc/crane_export.md).
-16
View File
@@ -1,16 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// Package mutate provides facilities for mutating v1.Images of any kind.
package mutate
-293
View File
@@ -1,293 +0,0 @@
// Copyright 2019 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package mutate
import (
"bytes"
"encoding/json"
"errors"
"sync"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/partial"
"github.com/google/go-containerregistry/pkg/v1/stream"
"github.com/google/go-containerregistry/pkg/v1/types"
)
type image struct {
base v1.Image
adds []Addendum
computed bool
configFile *v1.ConfigFile
manifest *v1.Manifest
annotations map[string]string
mediaType *types.MediaType
configMediaType *types.MediaType
diffIDMap map[v1.Hash]v1.Layer
digestMap map[v1.Hash]v1.Layer
subject *v1.Descriptor
sync.Mutex
}
var _ v1.Image = (*image)(nil)
func (i *image) MediaType() (types.MediaType, error) {
if i.mediaType != nil {
return *i.mediaType, nil
}
return i.base.MediaType()
}
func (i *image) compute() error {
i.Lock()
defer i.Unlock()
// Don't re-compute if already computed.
if i.computed {
return nil
}
var configFile *v1.ConfigFile
if i.configFile != nil {
configFile = i.configFile
} else {
cf, err := i.base.ConfigFile()
if err != nil {
return err
}
configFile = cf.DeepCopy()
}
diffIDs := configFile.RootFS.DiffIDs
history := configFile.History
diffIDMap := make(map[v1.Hash]v1.Layer)
digestMap := make(map[v1.Hash]v1.Layer)
for _, add := range i.adds {
history = append(history, add.History)
if add.Layer != nil {
diffID, err := add.Layer.DiffID()
if err != nil {
return err
}
diffIDs = append(diffIDs, diffID)
diffIDMap[diffID] = add.Layer
}
}
m, err := i.base.Manifest()
if err != nil {
return err
}
manifest := m.DeepCopy()
manifestLayers := manifest.Layers
for _, add := range i.adds {
if add.Layer == nil {
// Empty layers include only history in manifest.
continue
}
desc, err := partial.Descriptor(add.Layer)
if err != nil {
return err
}
// Fields in the addendum override the original descriptor.
if len(add.Annotations) != 0 {
desc.Annotations = add.Annotations
}
if len(add.URLs) != 0 {
desc.URLs = add.URLs
}
if add.MediaType != "" {
desc.MediaType = add.MediaType
}
manifestLayers = append(manifestLayers, *desc)
digestMap[desc.Digest] = add.Layer
}
configFile.RootFS.DiffIDs = diffIDs
configFile.History = history
manifest.Layers = manifestLayers
rcfg, err := json.Marshal(configFile)
if err != nil {
return err
}
d, sz, err := v1.SHA256(bytes.NewBuffer(rcfg))
if err != nil {
return err
}
manifest.Config.Digest = d
manifest.Config.Size = sz
// If Data was set in the base image, we need to update it in the mutated image.
if m.Config.Data != nil {
manifest.Config.Data = rcfg
}
// If the user wants to mutate the media type of the config
if i.configMediaType != nil {
manifest.Config.MediaType = *i.configMediaType
}
if i.mediaType != nil {
manifest.MediaType = *i.mediaType
}
if i.annotations != nil {
if manifest.Annotations == nil {
manifest.Annotations = map[string]string{}
}
for k, v := range i.annotations {
manifest.Annotations[k] = v
}
}
manifest.Subject = i.subject
i.configFile = configFile
i.manifest = manifest
i.diffIDMap = diffIDMap
i.digestMap = digestMap
i.computed = true
return nil
}
// Layers returns the ordered collection of filesystem layers that comprise this image.
// The order of the list is oldest/base layer first, and most-recent/top layer last.
func (i *image) Layers() ([]v1.Layer, error) {
if err := i.compute(); errors.Is(err, stream.ErrNotComputed) {
// Image contains a streamable layer which has not yet been
// consumed. Just return the layers we have in case the caller
// is going to consume the layers.
layers, err := i.base.Layers()
if err != nil {
return nil, err
}
for _, add := range i.adds {
layers = append(layers, add.Layer)
}
return layers, nil
} else if err != nil {
return nil, err
}
diffIDs, err := partial.DiffIDs(i)
if err != nil {
return nil, err
}
ls := make([]v1.Layer, 0, len(diffIDs))
for _, h := range diffIDs {
l, err := i.LayerByDiffID(h)
if err != nil {
return nil, err
}
ls = append(ls, l)
}
return ls, nil
}
// ConfigName returns the hash of the image's config file.
func (i *image) ConfigName() (v1.Hash, error) {
if err := i.compute(); err != nil {
return v1.Hash{}, err
}
return partial.ConfigName(i)
}
// ConfigFile returns this image's config file.
func (i *image) ConfigFile() (*v1.ConfigFile, error) {
if err := i.compute(); err != nil {
return nil, err
}
return i.configFile.DeepCopy(), nil
}
// RawConfigFile returns the serialized bytes of ConfigFile()
func (i *image) RawConfigFile() ([]byte, error) {
if err := i.compute(); err != nil {
return nil, err
}
return json.Marshal(i.configFile)
}
// Digest returns the sha256 of this image's manifest.
func (i *image) Digest() (v1.Hash, error) {
if err := i.compute(); err != nil {
return v1.Hash{}, err
}
return partial.Digest(i)
}
// Size implements v1.Image.
func (i *image) Size() (int64, error) {
if err := i.compute(); err != nil {
return -1, err
}
return partial.Size(i)
}
// Manifest returns this image's Manifest object.
func (i *image) Manifest() (*v1.Manifest, error) {
if err := i.compute(); err != nil {
return nil, err
}
return i.manifest.DeepCopy(), nil
}
// RawManifest returns the serialized bytes of Manifest()
func (i *image) RawManifest() ([]byte, error) {
if err := i.compute(); err != nil {
return nil, err
}
return json.Marshal(i.manifest)
}
// LayerByDigest returns a Layer for interacting with a particular layer of
// the image, looking it up by "digest" (the compressed hash).
func (i *image) LayerByDigest(h v1.Hash) (v1.Layer, error) {
if cn, err := i.ConfigName(); err != nil {
return nil, err
} else if h == cn {
return partial.ConfigLayer(i)
}
if layer, ok := i.digestMap[h]; ok {
return layer, nil
}
return i.base.LayerByDigest(h)
}
// LayerByDiffID is an analog to LayerByDigest, looking up by "diff id"
// (the uncompressed hash).
func (i *image) LayerByDiffID(h v1.Hash) (v1.Layer, error) {
if layer, ok := i.diffIDMap[h]; ok {
return layer, nil
}
return i.base.LayerByDiffID(h)
}
func validate(adds []Addendum) error {
for _, add := range adds {
if add.Layer == nil && !add.History.EmptyLayer {
return errors.New("unable to add a nil layer to the image")
}
}
return nil
}
-232
View File
@@ -1,232 +0,0 @@
// Copyright 2019 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package mutate
import (
"encoding/json"
"errors"
"fmt"
"sync"
"github.com/google/go-containerregistry/pkg/logs"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/match"
"github.com/google/go-containerregistry/pkg/v1/partial"
"github.com/google/go-containerregistry/pkg/v1/stream"
"github.com/google/go-containerregistry/pkg/v1/types"
)
func computeDescriptor(ia IndexAddendum) (*v1.Descriptor, error) {
desc, err := partial.Descriptor(ia.Add)
if err != nil {
return nil, err
}
// The IndexAddendum allows overriding Descriptor values.
if ia.Size != 0 {
desc.Size = ia.Size
}
if string(ia.MediaType) != "" {
desc.MediaType = ia.MediaType
}
if ia.Digest != (v1.Hash{}) {
desc.Digest = ia.Digest
}
if ia.Platform != nil {
desc.Platform = ia.Platform
}
if len(ia.URLs) != 0 {
desc.URLs = ia.URLs
}
if len(ia.Annotations) != 0 {
desc.Annotations = ia.Annotations
}
if ia.Data != nil {
desc.Data = ia.Data
}
return desc, nil
}
type index struct {
base v1.ImageIndex
adds []IndexAddendum
// remove is removed before adds
remove match.Matcher
computed bool
manifest *v1.IndexManifest
annotations map[string]string
mediaType *types.MediaType
imageMap map[v1.Hash]v1.Image
indexMap map[v1.Hash]v1.ImageIndex
layerMap map[v1.Hash]v1.Layer
subject *v1.Descriptor
sync.Mutex
}
var _ v1.ImageIndex = (*index)(nil)
func (i *index) MediaType() (types.MediaType, error) {
if i.mediaType != nil {
return *i.mediaType, nil
}
return i.base.MediaType()
}
func (i *index) Size() (int64, error) { return partial.Size(i) }
func (i *index) compute() error {
i.Lock()
defer i.Unlock()
// Don't re-compute if already computed.
if i.computed {
return nil
}
i.imageMap = make(map[v1.Hash]v1.Image)
i.indexMap = make(map[v1.Hash]v1.ImageIndex)
i.layerMap = make(map[v1.Hash]v1.Layer)
m, err := i.base.IndexManifest()
if err != nil {
return err
}
manifest := m.DeepCopy()
manifests := manifest.Manifests
if i.remove != nil {
var cleanedManifests []v1.Descriptor
for _, m := range manifests {
if !i.remove(m) {
cleanedManifests = append(cleanedManifests, m)
}
}
manifests = cleanedManifests
}
for _, add := range i.adds {
desc, err := computeDescriptor(add)
if err != nil {
return err
}
manifests = append(manifests, *desc)
if idx, ok := add.Add.(v1.ImageIndex); ok {
i.indexMap[desc.Digest] = idx
} else if img, ok := add.Add.(v1.Image); ok {
i.imageMap[desc.Digest] = img
} else if l, ok := add.Add.(v1.Layer); ok {
i.layerMap[desc.Digest] = l
} else {
logs.Warn.Printf("Unexpected index addendum: %T", add.Add)
}
}
manifest.Manifests = manifests
if i.mediaType != nil {
manifest.MediaType = *i.mediaType
}
if i.annotations != nil {
if manifest.Annotations == nil {
manifest.Annotations = map[string]string{}
}
for k, v := range i.annotations {
manifest.Annotations[k] = v
}
}
manifest.Subject = i.subject
i.manifest = manifest
i.computed = true
return nil
}
func (i *index) Image(h v1.Hash) (v1.Image, error) {
if img, ok := i.imageMap[h]; ok {
return img, nil
}
return i.base.Image(h)
}
func (i *index) ImageIndex(h v1.Hash) (v1.ImageIndex, error) {
if idx, ok := i.indexMap[h]; ok {
return idx, nil
}
return i.base.ImageIndex(h)
}
type withLayer interface {
Layer(v1.Hash) (v1.Layer, error)
}
// Workaround for #819.
func (i *index) Layer(h v1.Hash) (v1.Layer, error) {
if layer, ok := i.layerMap[h]; ok {
return layer, nil
}
if wl, ok := i.base.(withLayer); ok {
return wl.Layer(h)
}
return nil, fmt.Errorf("layer not found: %s", h)
}
// Digest returns the sha256 of this image's manifest.
func (i *index) Digest() (v1.Hash, error) {
if err := i.compute(); err != nil {
return v1.Hash{}, err
}
return partial.Digest(i)
}
// Manifest returns this image's Manifest object.
func (i *index) IndexManifest() (*v1.IndexManifest, error) {
if err := i.compute(); err != nil {
return nil, err
}
return i.manifest.DeepCopy(), nil
}
// RawManifest returns the serialized bytes of Manifest()
func (i *index) RawManifest() ([]byte, error) {
if err := i.compute(); err != nil {
return nil, err
}
return json.Marshal(i.manifest)
}
func (i *index) Manifests() ([]partial.Describable, error) {
if err := i.compute(); errors.Is(err, stream.ErrNotComputed) {
// Index contains a streamable layer which has not yet been
// consumed. Just return the manifests we have in case the caller
// is going to consume the streamable layers.
manifests, err := partial.Manifests(i.base)
if err != nil {
return nil, err
}
for _, add := range i.adds {
manifests = append(manifests, add.Add)
}
return manifests, nil
} else if err != nil {
return nil, err
}
return partial.ComputeManifests(i)
}
-546
View File
@@ -1,546 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package mutate
import (
"archive/tar"
"bytes"
"encoding/json"
"errors"
"fmt"
"io"
"maps"
"path/filepath"
"strings"
"time"
"github.com/google/go-containerregistry/internal/gzip"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/empty"
"github.com/google/go-containerregistry/pkg/v1/match"
"github.com/google/go-containerregistry/pkg/v1/partial"
"github.com/google/go-containerregistry/pkg/v1/tarball"
"github.com/google/go-containerregistry/pkg/v1/types"
)
const whiteoutPrefix = ".wh."
// Addendum contains layers and history to be appended
// to a base image
type Addendum struct {
Layer v1.Layer
History v1.History
URLs []string
Annotations map[string]string
MediaType types.MediaType
}
// AppendLayers applies layers to a base image.
func AppendLayers(base v1.Image, layers ...v1.Layer) (v1.Image, error) {
additions := make([]Addendum, 0, len(layers))
for _, layer := range layers {
additions = append(additions, Addendum{Layer: layer})
}
return Append(base, additions...)
}
// Append will apply the list of addendums to the base image
func Append(base v1.Image, adds ...Addendum) (v1.Image, error) {
if len(adds) == 0 {
return base, nil
}
if err := validate(adds); err != nil {
return nil, err
}
return &image{
base: base,
adds: adds,
}, nil
}
// Appendable is an interface that represents something that can be appended
// to an ImageIndex. We need to be able to construct a v1.Descriptor in order
// to append something, and this is the minimum required information for that.
type Appendable interface {
MediaType() (types.MediaType, error)
Digest() (v1.Hash, error)
Size() (int64, error)
}
// IndexAddendum represents an appendable thing and all the properties that
// we may want to override in the resulting v1.Descriptor.
type IndexAddendum struct {
Add Appendable
v1.Descriptor
}
// AppendManifests appends a manifest to the ImageIndex.
func AppendManifests(base v1.ImageIndex, adds ...IndexAddendum) v1.ImageIndex {
return &index{
base: base,
adds: adds,
}
}
// RemoveManifests removes any descriptors that match the match.Matcher.
func RemoveManifests(base v1.ImageIndex, matcher match.Matcher) v1.ImageIndex {
return &index{
base: base,
remove: matcher,
}
}
// Config mutates the provided v1.Image to have the provided v1.Config
func Config(base v1.Image, cfg v1.Config) (v1.Image, error) {
cf, err := base.ConfigFile()
if err != nil {
return nil, err
}
cf.Config = cfg
return ConfigFile(base, cf)
}
// Subject mutates the subject on an image or index manifest.
//
// The input is expected to be a v1.Image or v1.ImageIndex, and
// returns the same type. You can type-assert the result like so:
//
// img := Subject(empty.Image, subj).(v1.Image)
//
// Or for an index:
//
// idx := Subject(empty.Index, subj).(v1.ImageIndex)
//
// If the input is not an Image or ImageIndex, the result will
// attempt to lazily annotate the raw manifest.
func Subject(f partial.WithRawManifest, subject v1.Descriptor) partial.WithRawManifest {
if img, ok := f.(v1.Image); ok {
return &image{
base: img,
subject: &subject,
}
}
if idx, ok := f.(v1.ImageIndex); ok {
return &index{
base: idx,
subject: &subject,
}
}
return arbitraryRawManifest{a: f, subject: &subject}
}
// Annotations mutates the annotations on an annotatable image or index manifest.
//
// The annotatable input is expected to be a v1.Image or v1.ImageIndex, and
// returns the same type. You can type-assert the result like so:
//
// img := Annotations(empty.Image, map[string]string{
// "foo": "bar",
// }).(v1.Image)
//
// Or for an index:
//
// idx := Annotations(empty.Index, map[string]string{
// "foo": "bar",
// }).(v1.ImageIndex)
//
// If the input Annotatable is not an Image or ImageIndex, the result will
// attempt to lazily annotate the raw manifest.
func Annotations(f partial.WithRawManifest, anns map[string]string) partial.WithRawManifest {
if img, ok := f.(v1.Image); ok {
return &image{
base: img,
annotations: maps.Clone(anns),
}
}
if idx, ok := f.(v1.ImageIndex); ok {
return &index{
base: idx,
annotations: maps.Clone(anns),
}
}
return arbitraryRawManifest{a: f, anns: maps.Clone(anns)}
}
type arbitraryRawManifest struct {
a partial.WithRawManifest
anns map[string]string
subject *v1.Descriptor
}
func (a arbitraryRawManifest) RawManifest() ([]byte, error) {
b, err := a.a.RawManifest()
if err != nil {
return nil, err
}
var m map[string]any
if err := json.Unmarshal(b, &m); err != nil {
return nil, err
}
if ann, ok := m["annotations"]; ok {
if annm, ok := ann.(map[string]string); ok {
for k, v := range a.anns {
annm[k] = v
}
} else {
return nil, fmt.Errorf(".annotations is not a map: %T", ann)
}
} else {
m["annotations"] = a.anns
}
if a.subject != nil {
m["subject"] = a.subject
}
return json.Marshal(m)
}
// ConfigFile mutates the provided v1.Image to have the provided v1.ConfigFile
func ConfigFile(base v1.Image, cfg *v1.ConfigFile) (v1.Image, error) {
m, err := base.Manifest()
if err != nil {
return nil, err
}
image := &image{
base: base,
manifest: m.DeepCopy(),
configFile: cfg,
}
return image, nil
}
// CreatedAt mutates the provided v1.Image to have the provided v1.Time
func CreatedAt(base v1.Image, created v1.Time) (v1.Image, error) {
cf, err := base.ConfigFile()
if err != nil {
return nil, err
}
cfg := cf.DeepCopy()
cfg.Created = created
return ConfigFile(base, cfg)
}
// Extract takes an image and returns an io.ReadCloser containing the image's
// flattened filesystem.
//
// Callers can read the filesystem contents by passing the reader to
// tar.NewReader, or io.Copy it directly to some output.
//
// If a caller doesn't read the full contents, they should Close it to free up
// resources used during extraction.
func Extract(img v1.Image) io.ReadCloser {
pr, pw := io.Pipe()
go func() {
// Close the writer with any errors encountered during
// extraction. These errors will be returned by the reader end
// on subsequent reads. If err == nil, the reader will return
// EOF.
pw.CloseWithError(extract(img, pw))
}()
return pr
}
// Adapted from https://github.com/google/containerregistry/blob/da03b395ccdc4e149e34fbb540483efce962dc64/client/v2_2/docker_image_.py#L816
func extract(img v1.Image, w io.Writer) error {
tarWriter := tar.NewWriter(w)
defer tarWriter.Close()
fileMap := map[string]bool{}
layers, err := img.Layers()
if err != nil {
return fmt.Errorf("retrieving image layers: %w", err)
}
// we iterate through the layers in reverse order because it makes handling
// whiteout layers more efficient, since we can just keep track of the removed
// files as we see .wh. layers and ignore those in previous layers.
for i := len(layers) - 1; i >= 0; i-- {
layer := layers[i]
layerReader, err := layer.Uncompressed()
if err != nil {
return fmt.Errorf("reading layer contents: %w", err)
}
defer layerReader.Close()
tarReader := tar.NewReader(layerReader)
for {
header, err := tarReader.Next()
if errors.Is(err, io.EOF) {
break
}
if err != nil {
return fmt.Errorf("reading tar: %w", err)
}
// Some tools prepend everything with "./", so if we don't Clean the
// name, we may have duplicate entries, which angers tar-split.
header.Name = filepath.Clean(header.Name)
// force PAX format to remove Name/Linkname length limit of 100 characters
// required by USTAR and to not depend on internal tar package guess which
// prefers USTAR over PAX
header.Format = tar.FormatPAX
basename := filepath.Base(header.Name)
dirname := filepath.Dir(header.Name)
tombstone := strings.HasPrefix(basename, whiteoutPrefix)
if tombstone {
basename = basename[len(whiteoutPrefix):]
}
// check if we have seen value before
// if we're checking a directory, don't filepath.Join names
var name string
if header.Typeflag == tar.TypeDir {
name = header.Name
} else {
name = filepath.Join(dirname, basename)
}
if _, ok := fileMap[name]; ok && !tombstone {
continue
}
// check for a whited out parent directory
if inWhiteoutDir(fileMap, name) {
continue
}
// mark file as handled. non-directory implicitly tombstones
// any entries with a matching (or child) name
fileMap[name] = tombstone || (header.Typeflag != tar.TypeDir)
if !tombstone {
if err := tarWriter.WriteHeader(header); err != nil {
return err
}
if header.Size > 0 {
if _, err := io.CopyN(tarWriter, tarReader, header.Size); err != nil {
return err
}
}
}
}
}
return nil
}
func inWhiteoutDir(fileMap map[string]bool, file string) bool {
for file != "" {
dirname := filepath.Dir(file)
if file == dirname {
break
}
if val, ok := fileMap[dirname]; ok && val {
return true
}
file = dirname
}
return false
}
// Time sets all timestamps in an image to the given timestamp.
func Time(img v1.Image, t time.Time) (v1.Image, error) {
newImage := empty.Image
layers, err := img.Layers()
if err != nil {
return nil, fmt.Errorf("getting image layers: %w", err)
}
ocf, err := img.ConfigFile()
if err != nil {
return nil, fmt.Errorf("getting original config file: %w", err)
}
addendums := make([]Addendum, max(len(ocf.History), len(layers)))
var historyIdx, addendumIdx int
for layerIdx := 0; layerIdx < len(layers); addendumIdx, layerIdx = addendumIdx+1, layerIdx+1 {
newLayer, err := layerTime(layers[layerIdx], t)
if err != nil {
return nil, fmt.Errorf("setting layer times: %w", err)
}
// try to search for the history entry that corresponds to this layer
for ; historyIdx < len(ocf.History); historyIdx++ {
addendums[addendumIdx].History = ocf.History[historyIdx]
// if it's an EmptyLayer, do not set the Layer and have the Addendum with just the History
// and move on to the next History entry
if ocf.History[historyIdx].EmptyLayer {
addendumIdx++
continue
}
// otherwise, we can exit from the cycle
historyIdx++
break
}
if addendumIdx < len(addendums) {
addendums[addendumIdx].Layer = newLayer
}
}
// add all leftover History entries
for ; historyIdx < len(ocf.History); historyIdx, addendumIdx = historyIdx+1, addendumIdx+1 {
addendums[addendumIdx].History = ocf.History[historyIdx]
}
newImage, err = Append(newImage, addendums...)
if err != nil {
return nil, fmt.Errorf("appending layers: %w", err)
}
cf, err := newImage.ConfigFile()
if err != nil {
return nil, fmt.Errorf("setting config file: %w", err)
}
cfg := cf.DeepCopy()
// Copy basic config over
cfg.Architecture = ocf.Architecture
cfg.OS = ocf.OS
cfg.OSVersion = ocf.OSVersion
cfg.Config = ocf.Config
// Strip away timestamps from the config file
cfg.Created = v1.Time{Time: t}
for i, h := range cfg.History {
h.Created = v1.Time{Time: t}
h.CreatedBy = ocf.History[i].CreatedBy
h.Comment = ocf.History[i].Comment
h.EmptyLayer = ocf.History[i].EmptyLayer
// Explicitly ignore Author field; which hinders reproducibility
h.Author = ""
cfg.History[i] = h
}
return ConfigFile(newImage, cfg)
}
func layerTime(layer v1.Layer, t time.Time) (v1.Layer, error) {
layerReader, err := layer.Uncompressed()
if err != nil {
return nil, fmt.Errorf("getting layer: %w", err)
}
defer layerReader.Close()
w := new(bytes.Buffer)
tarWriter := tar.NewWriter(w)
defer tarWriter.Close()
tarReader := tar.NewReader(layerReader)
for {
header, err := tarReader.Next()
if errors.Is(err, io.EOF) {
break
}
if err != nil {
return nil, fmt.Errorf("reading layer: %w", err)
}
header.ModTime = t
//PAX and GNU Format support additional timestamps in the header
if header.Format == tar.FormatPAX || header.Format == tar.FormatGNU {
header.AccessTime = t
header.ChangeTime = t
}
if err := tarWriter.WriteHeader(header); err != nil {
return nil, fmt.Errorf("writing tar header: %w", err)
}
if header.Typeflag == tar.TypeReg {
// TODO(#1168): This should be lazy, and not buffer the entire layer contents.
if _, err = io.CopyN(tarWriter, tarReader, header.Size); err != nil {
return nil, fmt.Errorf("writing layer file: %w", err)
}
}
}
if err := tarWriter.Close(); err != nil {
return nil, err
}
b := w.Bytes()
// gzip the contents, then create the layer
opener := func() (io.ReadCloser, error) {
return gzip.ReadCloser(io.NopCloser(bytes.NewReader(b))), nil
}
layer, err = tarball.LayerFromOpener(opener)
if err != nil {
return nil, fmt.Errorf("creating layer: %w", err)
}
return layer, nil
}
// Canonical is a helper function to combine Time and configFile
// to remove any randomness during a docker build.
func Canonical(img v1.Image) (v1.Image, error) {
// Set all timestamps to 0
created := time.Time{}
img, err := Time(img, created)
if err != nil {
return nil, err
}
cf, err := img.ConfigFile()
if err != nil {
return nil, err
}
// Get rid of host-dependent random config
cfg := cf.DeepCopy()
cfg.Container = ""
cfg.Config.Hostname = ""
cfg.DockerVersion = "" //nolint:staticcheck // Field will be removed in next release
return ConfigFile(img, cfg)
}
// MediaType modifies the MediaType() of the given image.
func MediaType(img v1.Image, mt types.MediaType) v1.Image {
return &image{
base: img,
mediaType: &mt,
}
}
// ConfigMediaType modifies the MediaType() of the given image's Config.
//
// If !mt.IsConfig(), this will be the image's artifactType in any indexes it's a part of.
func ConfigMediaType(img v1.Image, mt types.MediaType) v1.Image {
return &image{
base: img,
configMediaType: &mt,
}
}
// IndexMediaType modifies the MediaType() of the given index.
func IndexMediaType(idx v1.ImageIndex, mt types.MediaType) v1.ImageIndex {
return &index{
base: idx,
mediaType: &mt,
}
}
-144
View File
@@ -1,144 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package mutate
import (
"fmt"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/empty"
)
// Rebase returns a new v1.Image where the oldBase in orig is replaced by newBase.
func Rebase(orig, oldBase, newBase v1.Image) (v1.Image, error) {
// Verify that oldBase's layers are present in orig, otherwise orig is
// not based on oldBase at all.
origLayers, err := orig.Layers()
if err != nil {
return nil, fmt.Errorf("failed to get layers for original: %w", err)
}
oldBaseLayers, err := oldBase.Layers()
if err != nil {
return nil, err
}
if len(oldBaseLayers) > len(origLayers) {
return nil, fmt.Errorf("image %q is not based on %q (too few layers)", orig, oldBase)
}
for i, l := range oldBaseLayers {
oldLayerDigest, err := l.Digest()
if err != nil {
return nil, fmt.Errorf("failed to get digest of layer %d of %q: %w", i, oldBase, err)
}
origLayerDigest, err := origLayers[i].Digest()
if err != nil {
return nil, fmt.Errorf("failed to get digest of layer %d of %q: %w", i, orig, err)
}
if oldLayerDigest != origLayerDigest {
return nil, fmt.Errorf("image %q is not based on %q (layer %d mismatch)", orig, oldBase, i)
}
}
oldConfig, err := oldBase.ConfigFile()
if err != nil {
return nil, fmt.Errorf("failed to get config for old base: %w", err)
}
origConfig, err := orig.ConfigFile()
if err != nil {
return nil, fmt.Errorf("failed to get config for original: %w", err)
}
newConfig, err := newBase.ConfigFile()
if err != nil {
return nil, fmt.Errorf("could not get config for new base: %w", err)
}
// Stitch together an image that contains:
// - original image's config
// - new base image's os/arch properties
// - new base image's layers + top of original image's layers
// - new base image's history + top of original image's history
rebasedImage, err := Config(empty.Image, *origConfig.Config.DeepCopy())
if err != nil {
return nil, fmt.Errorf("failed to create empty image with original config: %w", err)
}
// Add new config properties from existing images.
rebasedConfig, err := rebasedImage.ConfigFile()
if err != nil {
return nil, fmt.Errorf("could not get config for rebased image: %w", err)
}
// OS/Arch properties from new base
rebasedConfig.Architecture = newConfig.Architecture
rebasedConfig.OS = newConfig.OS
rebasedConfig.OSVersion = newConfig.OSVersion
// Apply config properties to rebased.
rebasedImage, err = ConfigFile(rebasedImage, rebasedConfig)
if err != nil {
return nil, fmt.Errorf("failed to replace config for rebased image: %w", err)
}
// Get new base layers and config for history.
newBaseLayers, err := newBase.Layers()
if err != nil {
return nil, fmt.Errorf("could not get new base layers for new base: %w", err)
}
// Add new base layers.
rebasedImage, err = Append(rebasedImage, createAddendums(0, 0, newConfig.History, newBaseLayers)...)
if err != nil {
return nil, fmt.Errorf("failed to append new base image: %w", err)
}
// Add original layers above the old base.
rebasedImage, err = Append(rebasedImage, createAddendums(len(oldConfig.History), len(oldBaseLayers)+1, origConfig.History, origLayers)...)
if err != nil {
return nil, fmt.Errorf("failed to append original image: %w", err)
}
return rebasedImage, nil
}
// createAddendums makes a list of addendums from a history and layers starting from a specific history and layer
// indexes.
func createAddendums(startHistory, startLayer int, history []v1.History, layers []v1.Layer) []Addendum {
var adds []Addendum
// History should be a superset of layers; empty layers (e.g. ENV statements) only exist in history.
// They cannot be iterated identically but must be walked independently, only advancing the iterator for layers
// when a history entry for a non-empty layer is seen.
layerIndex := 0
for historyIndex := range history {
var layer v1.Layer
emptyLayer := history[historyIndex].EmptyLayer
if !emptyLayer {
layer = layers[layerIndex]
layerIndex++
}
if historyIndex >= startHistory || layerIndex >= startLayer {
adds = append(adds, Addendum{
Layer: layer,
History: history[historyIndex],
})
}
}
// In the event history was malformed or non-existent, append the remaining layers.
for i := layerIndex; i < len(layers); i++ {
if i >= startLayer {
adds = append(adds, Addendum{Layer: layers[layerIndex]})
}
}
return adds
}
-82
View File
@@ -1,82 +0,0 @@
# `partial`
[![GoDoc](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/partial?status.svg)](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/partial)
## Partial Implementations
There are roughly two kinds of image representations: compressed and uncompressed.
The implementations for these kinds of images are almost identical, with the only
major difference being how blobs (config and layers) are fetched. This common
code lives in this package, where you provide a _partial_ implementation of a
compressed or uncompressed image, and you get back a full `v1.Image` implementation.
### Examples
In a registry, blobs are compressed, so it's easiest to implement a `v1.Image` in terms
of compressed layers. `remote.remoteImage` does this by implementing `CompressedImageCore`:
```go
type CompressedImageCore interface {
RawConfigFile() ([]byte, error)
MediaType() (types.MediaType, error)
RawManifest() ([]byte, error)
LayerByDigest(v1.Hash) (CompressedLayer, error)
}
```
In a tarball, blobs are (often) uncompressed, so it's easiest to implement a `v1.Image` in terms
of uncompressed layers. `tarball.uncompressedImage` does this by implementing `UncompressedImageCore`:
```go
type UncompressedImageCore interface {
RawConfigFile() ([]byte, error)
MediaType() (types.MediaType, error)
LayerByDiffID(v1.Hash) (UncompressedLayer, error)
}
```
## Optional Methods
Where possible, we access some information via optional methods as an optimization.
### [`partial.Descriptor`](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/partial#Descriptor)
There are some properties of a [`Descriptor`](https://github.com/opencontainers/image-spec/blob/master/descriptor.md#properties) that aren't derivable from just image data:
* `MediaType`
* `Platform`
* `URLs`
* `Annotations`
For example, in a `tarball.Image`, there is a `LayerSources` field that contains
an entire layer descriptor with `URLs` information for foreign layers. This
information can be passed through to callers by implementing this optional
`Descriptor` method.
See [`#654`](https://github.com/google/go-containerregistry/pull/654).
### [`partial.UncompressedSize`](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/partial#UncompressedSize)
Usually, you don't need to know the uncompressed size of a layer, since that
information isn't stored in a config file (just he sha256 is needed); however,
there are cases where it is very helpful to know the layer size, e.g. when
writing the uncompressed layer into a tarball.
See [`#655`](https://github.com/google/go-containerregistry/pull/655).
### [`partial.Exists`](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/partial#Exists)
We generally don't care about the existence of something as granular as a
layer, and would rather ensure all the invariants of an image are upheld via
the `validate` package. However, there are situations where we want to do a
quick smoke test to ensure that the underlying storage engine hasn't been
corrupted by something e.g. deleting files or blobs. Thus, we've exposed an
optional `Exists` method that does an existence check without actually reading
any bytes.
The `remote` package implements this via `HEAD` requests.
The `layout` package implements this via `os.Stat`.
See [`#838`](https://github.com/google/go-containerregistry/pull/838).
@@ -1,188 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package partial
import (
"io"
"github.com/google/go-containerregistry/internal/and"
"github.com/google/go-containerregistry/internal/compression"
"github.com/google/go-containerregistry/internal/gzip"
"github.com/google/go-containerregistry/internal/zstd"
comp "github.com/google/go-containerregistry/pkg/compression"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/types"
)
// CompressedLayer represents the bare minimum interface a natively
// compressed layer must implement for us to produce a v1.Layer
type CompressedLayer interface {
// Digest returns the Hash of the compressed layer.
Digest() (v1.Hash, error)
// Compressed returns an io.ReadCloser for the compressed layer contents.
Compressed() (io.ReadCloser, error)
// Size returns the compressed size of the Layer.
Size() (int64, error)
// Returns the mediaType for the compressed Layer
MediaType() (types.MediaType, error)
}
// compressedLayerExtender implements v1.Image using the compressed base properties.
type compressedLayerExtender struct {
CompressedLayer
}
// Uncompressed implements v1.Layer
func (cle *compressedLayerExtender) Uncompressed() (io.ReadCloser, error) {
rc, err := cle.Compressed()
if err != nil {
return nil, err
}
// Often, the "compressed" bytes are not actually-compressed.
// Peek at the first two bytes to determine whether it's correct to
// wrap this with gzip.UnzipReadCloser or zstd.UnzipReadCloser.
cp, pr, err := compression.PeekCompression(rc)
if err != nil {
return nil, err
}
prc := &and.ReadCloser{
Reader: pr,
CloseFunc: rc.Close,
}
switch cp {
case comp.GZip:
return gzip.UnzipReadCloser(prc)
case comp.ZStd:
return zstd.UnzipReadCloser(prc)
default:
return prc, nil
}
}
// DiffID implements v1.Layer
func (cle *compressedLayerExtender) DiffID() (v1.Hash, error) {
// If our nested CompressedLayer implements DiffID,
// then delegate to it instead.
if wdi, ok := cle.CompressedLayer.(WithDiffID); ok {
return wdi.DiffID()
}
r, err := cle.Uncompressed()
if err != nil {
return v1.Hash{}, err
}
defer r.Close()
h, _, err := v1.SHA256(r)
return h, err
}
// CompressedToLayer fills in the missing methods from a CompressedLayer so that it implements v1.Layer
func CompressedToLayer(ul CompressedLayer) (v1.Layer, error) {
return &compressedLayerExtender{ul}, nil
}
// CompressedImageCore represents the base minimum interface a natively
// compressed image must implement for us to produce a v1.Image.
type CompressedImageCore interface {
ImageCore
// RawManifest returns the serialized bytes of the manifest.
RawManifest() ([]byte, error)
// LayerByDigest is a variation on the v1.Image method, which returns
// a CompressedLayer instead.
LayerByDigest(v1.Hash) (CompressedLayer, error)
}
// compressedImageExtender implements v1.Image by extending CompressedImageCore with the
// appropriate methods computed from the minimal core.
type compressedImageExtender struct {
CompressedImageCore
}
// Assert that our extender type completes the v1.Image interface
var _ v1.Image = (*compressedImageExtender)(nil)
// Digest implements v1.Image
func (i *compressedImageExtender) Digest() (v1.Hash, error) {
return Digest(i)
}
// ConfigName implements v1.Image
func (i *compressedImageExtender) ConfigName() (v1.Hash, error) {
return ConfigName(i)
}
// Layers implements v1.Image
func (i *compressedImageExtender) Layers() ([]v1.Layer, error) {
hs, err := FSLayers(i)
if err != nil {
return nil, err
}
ls := make([]v1.Layer, 0, len(hs))
for _, h := range hs {
l, err := i.LayerByDigest(h)
if err != nil {
return nil, err
}
ls = append(ls, l)
}
return ls, nil
}
// LayerByDigest implements v1.Image
func (i *compressedImageExtender) LayerByDigest(h v1.Hash) (v1.Layer, error) {
cl, err := i.CompressedImageCore.LayerByDigest(h)
if err != nil {
return nil, err
}
return CompressedToLayer(cl)
}
// LayerByDiffID implements v1.Image
func (i *compressedImageExtender) LayerByDiffID(h v1.Hash) (v1.Layer, error) {
h, err := DiffIDToBlob(i, h)
if err != nil {
return nil, err
}
return i.LayerByDigest(h)
}
// ConfigFile implements v1.Image
func (i *compressedImageExtender) ConfigFile() (*v1.ConfigFile, error) {
return ConfigFile(i)
}
// Manifest implements v1.Image
func (i *compressedImageExtender) Manifest() (*v1.Manifest, error) {
return Manifest(i)
}
// Size implements v1.Image
func (i *compressedImageExtender) Size() (int64, error) {
return Size(i)
}
// CompressedToImage fills in the missing methods from a CompressedImageCore so that it implements v1.Image
func CompressedToImage(cic CompressedImageCore) (v1.Image, error) {
return &compressedImageExtender{
CompressedImageCore: cic,
}, nil
}
-17
View File
@@ -1,17 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// Package partial defines methods for building up a v1.Image from
// minimal subsets that are sufficient for defining a v1.Image.
package partial
-28
View File
@@ -1,28 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package partial
import (
"github.com/google/go-containerregistry/pkg/v1/types"
)
// ImageCore is the core set of properties without which we cannot build a v1.Image
type ImageCore interface {
// RawConfigFile returns the serialized bytes of this image's config file.
RawConfigFile() ([]byte, error)
// MediaType of this image's manifest.
MediaType() (types.MediaType, error)
}
-165
View File
@@ -1,165 +0,0 @@
// Copyright 2020 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package partial
import (
"fmt"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/match"
"github.com/google/go-containerregistry/pkg/v1/types"
)
// FindManifests given a v1.ImageIndex, find the manifests that fit the matcher.
func FindManifests(index v1.ImageIndex, matcher match.Matcher) ([]v1.Descriptor, error) {
// get the actual manifest list
indexManifest, err := index.IndexManifest()
if err != nil {
return nil, fmt.Errorf("unable to get raw index: %w", err)
}
manifests := []v1.Descriptor{}
// try to get the root of our image
for _, manifest := range indexManifest.Manifests {
if matcher(manifest) {
manifests = append(manifests, manifest)
}
}
return manifests, nil
}
// FindImages given a v1.ImageIndex, find the images that fit the matcher. If a Descriptor
// matches the provider Matcher, but the referenced item is not an Image, ignores it.
// Only returns those that match the Matcher and are images.
func FindImages(index v1.ImageIndex, matcher match.Matcher) ([]v1.Image, error) {
matches := []v1.Image{}
manifests, err := FindManifests(index, matcher)
if err != nil {
return nil, err
}
for _, desc := range manifests {
// if it is not an image, ignore it
if !desc.MediaType.IsImage() {
continue
}
img, err := index.Image(desc.Digest)
if err != nil {
return nil, err
}
matches = append(matches, img)
}
return matches, nil
}
// FindIndexes given a v1.ImageIndex, find the indexes that fit the matcher. If a Descriptor
// matches the provider Matcher, but the referenced item is not an Index, ignores it.
// Only returns those that match the Matcher and are indexes.
func FindIndexes(index v1.ImageIndex, matcher match.Matcher) ([]v1.ImageIndex, error) {
matches := []v1.ImageIndex{}
manifests, err := FindManifests(index, matcher)
if err != nil {
return nil, err
}
for _, desc := range manifests {
if !desc.MediaType.IsIndex() {
continue
}
// if it is not an index, ignore it
idx, err := index.ImageIndex(desc.Digest)
if err != nil {
return nil, err
}
matches = append(matches, idx)
}
return matches, nil
}
type withManifests interface {
Manifests() ([]Describable, error)
}
type withLayer interface {
Layer(v1.Hash) (v1.Layer, error)
}
type describable struct {
desc v1.Descriptor
}
func (d describable) Digest() (v1.Hash, error) {
return d.desc.Digest, nil
}
func (d describable) Size() (int64, error) {
return d.desc.Size, nil
}
func (d describable) MediaType() (types.MediaType, error) {
return d.desc.MediaType, nil
}
func (d describable) Descriptor() (*v1.Descriptor, error) {
return &d.desc, nil
}
// Manifests is analogous to v1.Image.Layers in that it allows values in the
// returned list to be lazily evaluated, which enables an index to contain
// an image that contains a streaming layer.
//
// This should have been part of the v1.ImageIndex interface, but wasn't.
// It is instead usable through this extension interface.
func Manifests(idx v1.ImageIndex) ([]Describable, error) {
if wm, ok := idx.(withManifests); ok {
return wm.Manifests()
}
return ComputeManifests(idx)
}
// ComputeManifests provides a fallback implementation for Manifests.
func ComputeManifests(idx v1.ImageIndex) ([]Describable, error) {
m, err := idx.IndexManifest()
if err != nil {
return nil, err
}
manifests := []Describable{}
for _, desc := range m.Manifests {
switch {
case desc.MediaType.IsImage():
img, err := idx.Image(desc.Digest)
if err != nil {
return nil, err
}
manifests = append(manifests, img)
case desc.MediaType.IsIndex():
idx, err := idx.ImageIndex(desc.Digest)
if err != nil {
return nil, err
}
manifests = append(manifests, idx)
default:
if wl, ok := idx.(withLayer); ok {
layer, err := wl.Layer(desc.Digest)
if err != nil {
return nil, err
}
manifests = append(manifests, layer)
} else {
manifests = append(manifests, describable{desc})
}
}
}
return manifests, nil
}
@@ -1,223 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package partial
import (
"bytes"
"io"
"sync"
"github.com/google/go-containerregistry/internal/gzip"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/types"
)
// UncompressedLayer represents the bare minimum interface a natively
// uncompressed layer must implement for us to produce a v1.Layer
type UncompressedLayer interface {
// DiffID returns the Hash of the uncompressed layer.
DiffID() (v1.Hash, error)
// Uncompressed returns an io.ReadCloser for the uncompressed layer contents.
Uncompressed() (io.ReadCloser, error)
// Returns the mediaType for the compressed Layer
MediaType() (types.MediaType, error)
}
// uncompressedLayerExtender implements v1.Image using the uncompressed base properties.
type uncompressedLayerExtender struct {
UncompressedLayer
// Memoize size/hash so that the methods aren't twice as
// expensive as doing this manually.
hash v1.Hash
size int64
hashSizeError error
once sync.Once
}
// Compressed implements v1.Layer
func (ule *uncompressedLayerExtender) Compressed() (io.ReadCloser, error) {
u, err := ule.Uncompressed()
if err != nil {
return nil, err
}
return gzip.ReadCloser(u), nil
}
// Digest implements v1.Layer
func (ule *uncompressedLayerExtender) Digest() (v1.Hash, error) {
ule.calcSizeHash()
return ule.hash, ule.hashSizeError
}
// Size implements v1.Layer
func (ule *uncompressedLayerExtender) Size() (int64, error) {
ule.calcSizeHash()
return ule.size, ule.hashSizeError
}
func (ule *uncompressedLayerExtender) calcSizeHash() {
ule.once.Do(func() {
var r io.ReadCloser
r, ule.hashSizeError = ule.Compressed()
if ule.hashSizeError != nil {
return
}
defer r.Close()
ule.hash, ule.size, ule.hashSizeError = v1.SHA256(r)
})
}
// UncompressedToLayer fills in the missing methods from an UncompressedLayer so that it implements v1.Layer
func UncompressedToLayer(ul UncompressedLayer) (v1.Layer, error) {
return &uncompressedLayerExtender{UncompressedLayer: ul}, nil
}
// UncompressedImageCore represents the bare minimum interface a natively
// uncompressed image must implement for us to produce a v1.Image
type UncompressedImageCore interface {
ImageCore
// LayerByDiffID is a variation on the v1.Image method, which returns
// an UncompressedLayer instead.
LayerByDiffID(v1.Hash) (UncompressedLayer, error)
}
// UncompressedToImage fills in the missing methods from an UncompressedImageCore so that it implements v1.Image.
func UncompressedToImage(uic UncompressedImageCore) (v1.Image, error) {
return &uncompressedImageExtender{
UncompressedImageCore: uic,
}, nil
}
// uncompressedImageExtender implements v1.Image by extending UncompressedImageCore with the
// appropriate methods computed from the minimal core.
type uncompressedImageExtender struct {
UncompressedImageCore
lock sync.Mutex
manifest *v1.Manifest
}
// Assert that our extender type completes the v1.Image interface
var _ v1.Image = (*uncompressedImageExtender)(nil)
// Digest implements v1.Image
func (i *uncompressedImageExtender) Digest() (v1.Hash, error) {
return Digest(i)
}
// Manifest implements v1.Image
func (i *uncompressedImageExtender) Manifest() (*v1.Manifest, error) {
i.lock.Lock()
defer i.lock.Unlock()
if i.manifest != nil {
return i.manifest, nil
}
b, err := i.RawConfigFile()
if err != nil {
return nil, err
}
cfgHash, cfgSize, err := v1.SHA256(bytes.NewReader(b))
if err != nil {
return nil, err
}
m := &v1.Manifest{
SchemaVersion: 2,
MediaType: types.DockerManifestSchema2,
Config: v1.Descriptor{
MediaType: types.DockerConfigJSON,
Size: cfgSize,
Digest: cfgHash,
},
}
ls, err := i.Layers()
if err != nil {
return nil, err
}
m.Layers = make([]v1.Descriptor, len(ls))
for i, l := range ls {
desc, err := Descriptor(l)
if err != nil {
return nil, err
}
m.Layers[i] = *desc
}
i.manifest = m
return i.manifest, nil
}
// RawManifest implements v1.Image
func (i *uncompressedImageExtender) RawManifest() ([]byte, error) {
return RawManifest(i)
}
// Size implements v1.Image
func (i *uncompressedImageExtender) Size() (int64, error) {
return Size(i)
}
// ConfigName implements v1.Image
func (i *uncompressedImageExtender) ConfigName() (v1.Hash, error) {
return ConfigName(i)
}
// ConfigFile implements v1.Image
func (i *uncompressedImageExtender) ConfigFile() (*v1.ConfigFile, error) {
return ConfigFile(i)
}
// Layers implements v1.Image
func (i *uncompressedImageExtender) Layers() ([]v1.Layer, error) {
diffIDs, err := DiffIDs(i)
if err != nil {
return nil, err
}
ls := make([]v1.Layer, 0, len(diffIDs))
for _, h := range diffIDs {
l, err := i.LayerByDiffID(h)
if err != nil {
return nil, err
}
ls = append(ls, l)
}
return ls, nil
}
// LayerByDiffID implements v1.Image
func (i *uncompressedImageExtender) LayerByDiffID(diffID v1.Hash) (v1.Layer, error) {
ul, err := i.UncompressedImageCore.LayerByDiffID(diffID)
if err != nil {
return nil, err
}
return UncompressedToLayer(ul)
}
// LayerByDigest implements v1.Image
func (i *uncompressedImageExtender) LayerByDigest(h v1.Hash) (v1.Layer, error) {
diffID, err := BlobToDiffID(i, h)
if err != nil {
return nil, err
}
return i.LayerByDiffID(diffID)
}
-436
View File
@@ -1,436 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package partial
import (
"bytes"
"encoding/json"
"fmt"
"io"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/types"
)
// WithRawConfigFile defines the subset of v1.Image used by these helper methods
type WithRawConfigFile interface {
// RawConfigFile returns the serialized bytes of this image's config file.
RawConfigFile() ([]byte, error)
}
// ConfigFile is a helper for implementing v1.Image
func ConfigFile(i WithRawConfigFile) (*v1.ConfigFile, error) {
b, err := i.RawConfigFile()
if err != nil {
return nil, err
}
return v1.ParseConfigFile(bytes.NewReader(b))
}
// ConfigName is a helper for implementing v1.Image
func ConfigName(i WithRawConfigFile) (v1.Hash, error) {
b, err := i.RawConfigFile()
if err != nil {
return v1.Hash{}, err
}
h, _, err := v1.SHA256(bytes.NewReader(b))
return h, err
}
type configLayer struct {
hash v1.Hash
content []byte
}
// Digest implements v1.Layer
func (cl *configLayer) Digest() (v1.Hash, error) {
return cl.hash, nil
}
// DiffID implements v1.Layer
func (cl *configLayer) DiffID() (v1.Hash, error) {
return cl.hash, nil
}
// Uncompressed implements v1.Layer
func (cl *configLayer) Uncompressed() (io.ReadCloser, error) {
return io.NopCloser(bytes.NewBuffer(cl.content)), nil
}
// Compressed implements v1.Layer
func (cl *configLayer) Compressed() (io.ReadCloser, error) {
return io.NopCloser(bytes.NewBuffer(cl.content)), nil
}
// Size implements v1.Layer
func (cl *configLayer) Size() (int64, error) {
return int64(len(cl.content)), nil
}
func (cl *configLayer) MediaType() (types.MediaType, error) {
// Defaulting this to OCIConfigJSON as it should remain
// backwards compatible with DockerConfigJSON
return types.OCIConfigJSON, nil
}
var _ v1.Layer = (*configLayer)(nil)
// withConfigLayer allows partial image implementations to provide a layer
// for their config file.
type withConfigLayer interface {
ConfigLayer() (v1.Layer, error)
}
// ConfigLayer implements v1.Layer from the raw config bytes.
// This is so that clients (e.g. remote) can access the config as a blob.
//
// Images that want to return a specific layer implementation can implement
// withConfigLayer.
func ConfigLayer(i WithRawConfigFile) (v1.Layer, error) {
if wcl, ok := unwrap(i).(withConfigLayer); ok {
return wcl.ConfigLayer()
}
h, err := ConfigName(i)
if err != nil {
return nil, err
}
rcfg, err := i.RawConfigFile()
if err != nil {
return nil, err
}
return &configLayer{
hash: h,
content: rcfg,
}, nil
}
// WithConfigFile defines the subset of v1.Image used by these helper methods
type WithConfigFile interface {
// ConfigFile returns this image's config file.
ConfigFile() (*v1.ConfigFile, error)
}
// DiffIDs is a helper for implementing v1.Image
func DiffIDs(i WithConfigFile) ([]v1.Hash, error) {
cfg, err := i.ConfigFile()
if err != nil {
return nil, err
}
return cfg.RootFS.DiffIDs, nil
}
// RawConfigFile is a helper for implementing v1.Image
func RawConfigFile(i WithConfigFile) ([]byte, error) {
cfg, err := i.ConfigFile()
if err != nil {
return nil, err
}
return json.Marshal(cfg)
}
// WithRawManifest defines the subset of v1.Image used by these helper methods
type WithRawManifest interface {
// RawManifest returns the serialized bytes of this image's config file.
RawManifest() ([]byte, error)
}
// Digest is a helper for implementing v1.Image
func Digest(i WithRawManifest) (v1.Hash, error) {
mb, err := i.RawManifest()
if err != nil {
return v1.Hash{}, err
}
digest, _, err := v1.SHA256(bytes.NewReader(mb))
return digest, err
}
// Manifest is a helper for implementing v1.Image
func Manifest(i WithRawManifest) (*v1.Manifest, error) {
b, err := i.RawManifest()
if err != nil {
return nil, err
}
return v1.ParseManifest(bytes.NewReader(b))
}
// WithManifest defines the subset of v1.Image used by these helper methods
type WithManifest interface {
// Manifest returns this image's Manifest object.
Manifest() (*v1.Manifest, error)
}
// RawManifest is a helper for implementing v1.Image
func RawManifest(i WithManifest) ([]byte, error) {
m, err := i.Manifest()
if err != nil {
return nil, err
}
return json.Marshal(m)
}
// Size is a helper for implementing v1.Image
func Size(i WithRawManifest) (int64, error) {
b, err := i.RawManifest()
if err != nil {
return -1, err
}
return int64(len(b)), nil
}
// FSLayers is a helper for implementing v1.Image
func FSLayers(i WithManifest) ([]v1.Hash, error) {
m, err := i.Manifest()
if err != nil {
return nil, err
}
fsl := make([]v1.Hash, len(m.Layers))
for i, l := range m.Layers {
fsl[i] = l.Digest
}
return fsl, nil
}
// BlobSize is a helper for implementing v1.Image
func BlobSize(i WithManifest, h v1.Hash) (int64, error) {
d, err := BlobDescriptor(i, h)
if err != nil {
return -1, err
}
return d.Size, nil
}
// BlobDescriptor is a helper for implementing v1.Image
func BlobDescriptor(i WithManifest, h v1.Hash) (*v1.Descriptor, error) {
m, err := i.Manifest()
if err != nil {
return nil, err
}
if m.Config.Digest == h {
return &m.Config, nil
}
for _, l := range m.Layers {
if l.Digest == h {
return &l, nil
}
}
return nil, fmt.Errorf("blob %v not found", h)
}
// WithManifestAndConfigFile defines the subset of v1.Image used by these helper methods
type WithManifestAndConfigFile interface {
WithConfigFile
// Manifest returns this image's Manifest object.
Manifest() (*v1.Manifest, error)
}
// BlobToDiffID is a helper for mapping between compressed
// and uncompressed blob hashes.
func BlobToDiffID(i WithManifestAndConfigFile, h v1.Hash) (v1.Hash, error) {
blobs, err := FSLayers(i)
if err != nil {
return v1.Hash{}, err
}
diffIDs, err := DiffIDs(i)
if err != nil {
return v1.Hash{}, err
}
if len(blobs) != len(diffIDs) {
return v1.Hash{}, fmt.Errorf("mismatched fs layers (%d) and diff ids (%d)", len(blobs), len(diffIDs))
}
for i, blob := range blobs {
if blob == h {
return diffIDs[i], nil
}
}
return v1.Hash{}, fmt.Errorf("unknown blob %v", h)
}
// DiffIDToBlob is a helper for mapping between uncompressed
// and compressed blob hashes.
func DiffIDToBlob(wm WithManifestAndConfigFile, h v1.Hash) (v1.Hash, error) {
blobs, err := FSLayers(wm)
if err != nil {
return v1.Hash{}, err
}
diffIDs, err := DiffIDs(wm)
if err != nil {
return v1.Hash{}, err
}
if len(blobs) != len(diffIDs) {
return v1.Hash{}, fmt.Errorf("mismatched fs layers (%d) and diff ids (%d)", len(blobs), len(diffIDs))
}
for i, diffID := range diffIDs {
if diffID == h {
return blobs[i], nil
}
}
return v1.Hash{}, fmt.Errorf("unknown diffID %v", h)
}
// WithDiffID defines the subset of v1.Layer for exposing the DiffID method.
type WithDiffID interface {
DiffID() (v1.Hash, error)
}
// withDescriptor allows partial layer implementations to provide a layer
// descriptor to the partial image manifest builder. This allows partial
// uncompressed layers to provide foreign layer metadata like URLs to the
// uncompressed image manifest.
type withDescriptor interface {
Descriptor() (*v1.Descriptor, error)
}
// Describable represents something for which we can produce a v1.Descriptor.
type Describable interface {
Digest() (v1.Hash, error)
MediaType() (types.MediaType, error)
Size() (int64, error)
}
// Descriptor returns a v1.Descriptor given a Describable. It also encodes
// some logic for unwrapping things that have been wrapped by
// CompressedToLayer, UncompressedToLayer, CompressedToImage, or
// UncompressedToImage.
func Descriptor(d Describable) (*v1.Descriptor, error) {
// If Describable implements Descriptor itself, return that.
if wd, ok := unwrap(d).(withDescriptor); ok {
return wd.Descriptor()
}
// If all else fails, compute the descriptor from the individual methods.
var (
desc v1.Descriptor
err error
)
if desc.Size, err = d.Size(); err != nil {
return nil, err
}
if desc.Digest, err = d.Digest(); err != nil {
return nil, err
}
if desc.MediaType, err = d.MediaType(); err != nil {
return nil, err
}
if wat, ok := d.(withArtifactType); ok {
if desc.ArtifactType, err = wat.ArtifactType(); err != nil {
return nil, err
}
} else {
if wrm, ok := d.(WithRawManifest); ok && desc.MediaType.IsImage() {
mf, _ := Manifest(wrm)
// Failing to parse as a manifest should just be ignored.
// The manifest might not be valid, and that's okay.
if mf != nil && !mf.Config.MediaType.IsConfig() {
desc.ArtifactType = string(mf.Config.MediaType)
}
}
}
return &desc, nil
}
type withArtifactType interface {
ArtifactType() (string, error)
}
type withUncompressedSize interface {
UncompressedSize() (int64, error)
}
// UncompressedSize returns the size of the Uncompressed layer. If the
// underlying implementation doesn't implement UncompressedSize directly,
// this will compute the uncompressedSize by reading everything returned
// by Compressed(). This is potentially expensive and may consume the contents
// for streaming layers.
func UncompressedSize(l v1.Layer) (int64, error) {
// If the layer implements UncompressedSize itself, return that.
if wus, ok := unwrap(l).(withUncompressedSize); ok {
return wus.UncompressedSize()
}
// The layer doesn't implement UncompressedSize, we need to compute it.
rc, err := l.Uncompressed()
if err != nil {
return -1, err
}
defer rc.Close()
return io.Copy(io.Discard, rc)
}
type withExists interface {
Exists() (bool, error)
}
// Exists checks to see if a layer exists. This is a hack to work around the
// mistakes of the partial package. Don't use this.
func Exists(l v1.Layer) (bool, error) {
// If the layer implements Exists itself, return that.
if we, ok := unwrap(l).(withExists); ok {
return we.Exists()
}
// The layer doesn't implement Exists, so we hope that calling Compressed()
// is enough to trigger an error if the layer does not exist.
rc, err := l.Compressed()
if err != nil {
return false, err
}
defer rc.Close()
// We may want to try actually reading a single byte, but if we need to do
// that, we should just fix this hack.
return true, nil
}
// Recursively unwrap our wrappers so that we can check for the original implementation.
// We might want to expose this?
func unwrap(i any) any {
if ule, ok := i.(*uncompressedLayerExtender); ok {
return unwrap(ule.UncompressedLayer)
}
if cle, ok := i.(*compressedLayerExtender); ok {
return unwrap(cle.CompressedLayer)
}
if uie, ok := i.(*uncompressedImageExtender); ok {
return unwrap(uie.UncompressedImageCore)
}
if cie, ok := i.(*compressedImageExtender); ok {
return unwrap(cie.CompressedImageCore)
}
return i
}
// ArtifactType returns the artifact type for the given manifest.
//
// If the manifest reports its own artifact type, that's returned, otherwise
// the manifest is parsed and, if successful, its config.mediaType is returned.
func ArtifactType(w WithManifest) (string, error) {
if wat, ok := w.(withArtifactType); ok {
return wat.ArtifactType()
}
mf, _ := w.Manifest()
// Failing to parse as a manifest should just be ignored.
// The manifest might not be valid, and that's okay.
if mf != nil && !mf.Config.MediaType.IsConfig() {
return string(mf.Config.MediaType), nil
}
return "", nil
}
-149
View File
@@ -1,149 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package v1
import (
"fmt"
"sort"
"strings"
)
// Platform represents the target os/arch for an image.
type Platform struct {
Architecture string `json:"architecture"`
OS string `json:"os"`
OSVersion string `json:"os.version,omitempty"`
OSFeatures []string `json:"os.features,omitempty"`
Variant string `json:"variant,omitempty"`
Features []string `json:"features,omitempty"`
}
func (p Platform) String() string {
if p.OS == "" {
return ""
}
var b strings.Builder
b.WriteString(p.OS)
if p.Architecture != "" {
b.WriteString("/")
b.WriteString(p.Architecture)
}
if p.Variant != "" {
b.WriteString("/")
b.WriteString(p.Variant)
}
if p.OSVersion != "" {
b.WriteString(":")
b.WriteString(p.OSVersion)
}
return b.String()
}
// ParsePlatform parses a string representing a Platform, if possible.
func ParsePlatform(s string) (*Platform, error) {
var p Platform
parts := strings.Split(strings.TrimSpace(s), ":")
if len(parts) == 2 {
p.OSVersion = parts[1]
}
parts = strings.Split(parts[0], "/")
if len(parts) > 0 {
p.OS = parts[0]
}
if len(parts) > 1 {
p.Architecture = parts[1]
}
if len(parts) > 2 {
p.Variant = parts[2]
}
if len(parts) > 3 {
return nil, fmt.Errorf("too many slashes in platform spec: %s", s)
}
return &p, nil
}
// Equals returns true if the given platform is semantically equivalent to this one.
// The order of Features and OSFeatures is not important.
func (p Platform) Equals(o Platform) bool {
return p.OS == o.OS &&
p.Architecture == o.Architecture &&
p.Variant == o.Variant &&
p.OSVersion == o.OSVersion &&
stringSliceEqualIgnoreOrder(p.OSFeatures, o.OSFeatures) &&
stringSliceEqualIgnoreOrder(p.Features, o.Features)
}
// Satisfies returns true if this Platform "satisfies" the given spec Platform.
//
// Note that this is different from Equals and that Satisfies is not reflexive.
//
// The given spec represents "requirements" such that any missing values in the
// spec are not compared.
//
// For OSFeatures and Features, Satisfies will return true if this Platform's
// fields contain a superset of the values in the spec's fields (order ignored).
func (p Platform) Satisfies(spec Platform) bool {
return satisfies(spec.OS, p.OS) &&
satisfies(spec.Architecture, p.Architecture) &&
satisfies(spec.Variant, p.Variant) &&
satisfies(spec.OSVersion, p.OSVersion) &&
satisfiesList(spec.OSFeatures, p.OSFeatures) &&
satisfiesList(spec.Features, p.Features)
}
func satisfies(want, have string) bool {
return want == "" || want == have
}
func satisfiesList(want, have []string) bool {
if len(want) == 0 {
return true
}
set := map[string]struct{}{}
for _, h := range have {
set[h] = struct{}{}
}
for _, w := range want {
if _, ok := set[w]; !ok {
return false
}
}
return true
}
// stringSliceEqual compares 2 string slices and returns if their contents are identical.
func stringSliceEqual(a, b []string) bool {
if len(a) != len(b) {
return false
}
for i, elm := range a {
if elm != b[i] {
return false
}
}
return true
}
// stringSliceEqualIgnoreOrder compares 2 string slices and returns if their contents are identical, ignoring order
func stringSliceEqualIgnoreOrder(a, b []string) bool {
if a != nil && b != nil {
sort.Strings(a)
sort.Strings(b)
}
return stringSliceEqual(a, b)
}
-25
View File
@@ -1,25 +0,0 @@
// Copyright 2020 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package v1
// Update representation of an update of transfer progress. Some functions
// in this module can take a channel to which updates will be sent while a
// transfer is in progress.
// +k8s:deepcopy-gen=false
type Update struct {
Total int64
Complete int64
Error error
}
-117
View File
@@ -1,117 +0,0 @@
# `remote`
[![GoDoc](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/remote?status.svg)](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/remote)
The `remote` package implements a client for accessing a registry,
per the [OCI distribution spec](https://github.com/opencontainers/distribution-spec/blob/master/spec.md).
It leans heavily on the lower level [`transport`](/pkg/v1/remote/transport) package, which handles the
authentication handshake and structured errors.
## Usage
```go
package main
import (
"github.com/google/go-containerregistry/pkg/authn"
"github.com/google/go-containerregistry/pkg/name"
"github.com/google/go-containerregistry/pkg/v1/remote"
)
func main() {
ref, err := name.ParseReference("gcr.io/google-containers/pause")
if err != nil {
panic(err)
}
img, err := remote.Image(ref, remote.WithAuthFromKeychain(authn.DefaultKeychain))
if err != nil {
panic(err)
}
// do stuff with img
}
```
## Structure
<p align="center">
<img src="/images/remote.dot.svg" />
</p>
## Background
There are a lot of confusingly similar terms that come up when talking about images in registries.
### Anatomy of an image
In general...
* A tag refers to an image manifest.
* An image manifest references a config file and an orderered list of _compressed_ layers by sha256 digest.
* A config file references an ordered list of _uncompressed_ layers by sha256 digest and contains runtime configuration.
* The sha256 digest of the config file is the [image id](https://github.com/opencontainers/image-spec/blob/master/config.md#imageid) for the image.
For example, an image with two layers would look something like this:
![image anatomy](/images/image-anatomy.dot.svg)
### Anatomy of an index
In the normal case, an [index](https://github.com/opencontainers/image-spec/blob/master/image-index.md) is used to represent a multi-platform image.
This was the original use case for a [manifest
list](https://docs.docker.com/registry/spec/manifest-v2-2/#manifest-list).
![image index anatomy](/images/index-anatomy.dot.svg)
It is possible for an index to reference another index, per the OCI
[image-spec](https://github.com/opencontainers/image-spec/blob/master/media-types.md#compatibility-matrix).
In theory, both an image and image index can reference arbitrary things via
[descriptors](https://github.com/opencontainers/image-spec/blob/master/descriptor.md),
e.g. see the [image layout
example](https://github.com/opencontainers/image-spec/blob/master/image-layout.md#index-example),
which references an application/xml file from an image index.
That could look something like this:
![strange image index anatomy](/images/index-anatomy-strange.dot.svg)
Using a recursive index like this might not be possible with all registries,
but this flexibility allows for some interesting applications, e.g. the
[OCI Artifacts](https://github.com/opencontainers/artifacts) effort.
### Anatomy of an image upload
The structure of an image requires a delicate ordering when uploading an image to a registry.
Below is a (slightly simplified) figure that describes how an image is prepared for upload
to a registry and how the data flows between various artifacts:
![upload](/images/upload.dot.svg)
Note that:
* A config file references the uncompressed layer contents by sha256.
* A manifest references the compressed layer contents by sha256 and the size of the layer.
* A manifest references the config file contents by sha256 and the size of the file.
It follows that during an upload, we need to upload layers before the config file,
and we need to upload the config file before the manifest.
Sometimes, we know all of this information ahead of time, (e.g. when copying from remote.Image),
so the ordering is less important.
In other cases, e.g. when using a [`stream.Layer`](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/stream#Layer),
we can't compute anything until we have already uploaded the layer, so we need to be careful about ordering.
## Caveats
### schema 1
This package does not support schema 1 images, see [`#377`](https://github.com/google/go-containerregistry/issues/377),
however, it's possible to do _something_ useful with them via [`remote.Get`](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/remote#Get),
which doesn't try to interpret what is returned by the registry.
[`crane.Copy`](https://godoc.org/github.com/google/go-containerregistry/pkg/crane#Copy) takes advantage of this to implement support for copying schema 1 images,
see [here](https://github.com/google/go-containerregistry/blob/main/pkg/internal/legacy/copy.go).
-159
View File
@@ -1,159 +0,0 @@
// Copyright 2019 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package remote
import (
"context"
"encoding/json"
"fmt"
"net/http"
"net/url"
"github.com/google/go-containerregistry/pkg/name"
"github.com/google/go-containerregistry/pkg/v1/remote/transport"
)
type Catalogs struct {
Repos []string `json:"repositories"`
Next string `json:"next,omitempty"`
}
// CatalogPage calls /_catalog, returning the list of repositories on the registry.
func CatalogPage(target name.Registry, last string, n int, options ...Option) ([]string, error) {
o, err := makeOptions(options...)
if err != nil {
return nil, err
}
f, err := newPuller(o).fetcher(o.context, target)
if err != nil {
return nil, err
}
uri := url.URL{
Scheme: target.Scheme(),
Host: target.RegistryStr(),
Path: "/v2/_catalog",
RawQuery: fmt.Sprintf("last=%s&n=%d", url.QueryEscape(last), n),
}
req, err := http.NewRequest(http.MethodGet, uri.String(), nil)
if err != nil {
return nil, err
}
resp, err := f.client.Do(req.WithContext(o.context))
if err != nil {
return nil, err
}
defer resp.Body.Close()
if err := transport.CheckError(resp, http.StatusOK); err != nil {
return nil, err
}
var parsed Catalogs
if err := json.NewDecoder(resp.Body).Decode(&parsed); err != nil {
return nil, err
}
return parsed.Repos, nil
}
// Catalog calls /_catalog, returning the list of repositories on the registry.
func Catalog(ctx context.Context, target name.Registry, options ...Option) ([]string, error) {
o, err := makeOptions(options...)
if err != nil {
return nil, err
}
// WithContext overrides the ctx passed directly.
if o.context != context.Background() {
ctx = o.context
}
return newPuller(o).catalog(ctx, target, o.pageSize)
}
func (f *fetcher) catalogPage(ctx context.Context, reg name.Registry, next string, pageSize int) (*Catalogs, error) {
if next == "" {
uri := &url.URL{
Scheme: reg.Scheme(),
Host: reg.RegistryStr(),
Path: "/v2/_catalog",
}
if pageSize > 0 {
uri.RawQuery = fmt.Sprintf("n=%d", pageSize)
}
next = uri.String()
}
req, err := http.NewRequestWithContext(ctx, "GET", next, nil)
if err != nil {
return nil, err
}
resp, err := f.client.Do(req)
if err != nil {
return nil, err
}
if err := transport.CheckError(resp, http.StatusOK); err != nil {
return nil, err
}
parsed := Catalogs{}
if err := json.NewDecoder(resp.Body).Decode(&parsed); err != nil {
return nil, err
}
if err := resp.Body.Close(); err != nil {
return nil, err
}
uri, err := getNextPageURL(resp)
if err != nil {
return nil, err
}
if uri != nil {
parsed.Next = uri.String()
}
return &parsed, nil
}
type Catalogger struct {
f *fetcher
reg name.Registry
pageSize int
page *Catalogs
err error
needMore bool
}
func (l *Catalogger) Next(ctx context.Context) (*Catalogs, error) {
if l.needMore {
l.page, l.err = l.f.catalogPage(ctx, l.reg, l.page.Next, l.pageSize)
} else {
l.needMore = true
}
return l.page, l.err
}
func (l *Catalogger) HasNext() bool {
return l.page != nil && (!l.needMore || l.page.Next != "")
}
-72
View File
@@ -1,72 +0,0 @@
// Copyright 2019 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package remote
import (
"context"
"fmt"
"net/http"
"github.com/google/go-containerregistry/pkg/authn"
"github.com/google/go-containerregistry/pkg/name"
"github.com/google/go-containerregistry/pkg/v1/remote/transport"
)
// CheckPushPermission returns an error if the given keychain cannot authorize
// a push operation to the given ref.
//
// This can be useful to check whether the caller has permission to push an
// image before doing work to construct the image.
//
// TODO(#412): Remove the need for this method.
func CheckPushPermission(ref name.Reference, kc authn.Keychain, t http.RoundTripper) error {
auth, err := kc.Resolve(ref.Context().Registry)
if err != nil {
return fmt.Errorf("resolving authorization for %v failed: %w", ref.Context().Registry, err)
}
scopes := []string{ref.Scope(transport.PushScope)}
tr, err := transport.NewWithContext(context.TODO(), ref.Context().Registry, auth, t, scopes)
if err != nil {
return fmt.Errorf("creating push check transport for %v failed: %w", ref.Context().Registry, err)
}
// TODO(jasonhall): Against GCR, just doing the token handshake is
// enough, but this doesn't extend to Dockerhub
// (https://github.com/docker/hub-feedback/issues/1771), so we actually
// need to initiate an upload to tell whether the credentials can
// authorize a push. Figure out how to return early here when we can,
// to avoid a roundtrip for spec-compliant registries.
w := writer{
repo: ref.Context(),
client: &http.Client{Transport: tr},
}
loc, _, err := w.initiateUpload(context.Background(), "", "", "")
if loc != "" {
// Since we're only initiating the upload to check whether we
// can, we should attempt to cancel it, in case initiating
// reserves some resources on the server. We shouldn't wait for
// cancelling to complete, and we don't care if it fails.
go w.cancelUpload(loc)
}
return err
}
func (w *writer) cancelUpload(loc string) {
req, err := http.NewRequest(http.MethodDelete, loc, nil)
if err != nil {
return
}
_, _ = w.client.Do(req)
}
-28
View File
@@ -1,28 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package remote
import (
"github.com/google/go-containerregistry/pkg/name"
)
// Delete removes the specified image reference from the remote registry.
func Delete(ref name.Reference, options ...Option) error {
o, err := makeOptions(options...)
if err != nil {
return err
}
return newPusher(o).Delete(o.context, ref)
}
@@ -1,198 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package remote
import (
"context"
"errors"
"fmt"
"github.com/google/go-containerregistry/pkg/logs"
"github.com/google/go-containerregistry/pkg/name"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/partial"
"github.com/google/go-containerregistry/pkg/v1/types"
)
var allManifestMediaTypes = append(append([]types.MediaType{
types.DockerManifestSchema1,
types.DockerManifestSchema1Signed,
}, acceptableImageMediaTypes...), acceptableIndexMediaTypes...)
// ErrSchema1 indicates that we received a schema1 manifest from the registry.
// This library doesn't have plans to support this legacy image format:
// https://github.com/google/go-containerregistry/issues/377
var ErrSchema1 = errors.New("see https://github.com/google/go-containerregistry/issues/377")
// newErrSchema1 returns an ErrSchema1 with the unexpected MediaType.
func newErrSchema1(schema types.MediaType) error {
return fmt.Errorf("unsupported MediaType: %q, %w", schema, ErrSchema1)
}
// Descriptor provides access to metadata about remote artifact and accessors
// for efficiently converting it into a v1.Image or v1.ImageIndex.
type Descriptor struct {
fetcher fetcher
v1.Descriptor
ref name.Reference
Manifest []byte
ctx context.Context
// So we can share this implementation with Image.
platform v1.Platform
}
func (d *Descriptor) toDesc() v1.Descriptor {
return d.Descriptor
}
// RawManifest exists to satisfy the Taggable interface.
func (d *Descriptor) RawManifest() ([]byte, error) {
return d.Manifest, nil
}
// Get returns a remote.Descriptor for the given reference. The response from
// the registry is left un-interpreted, for the most part. This is useful for
// querying what kind of artifact a reference represents.
//
// See Head if you don't need the response body.
func Get(ref name.Reference, options ...Option) (*Descriptor, error) {
return get(ref, allManifestMediaTypes, options...)
}
// Head returns a v1.Descriptor for the given reference by issuing a HEAD
// request.
//
// Note that the server response will not have a body, so any errors encountered
// should be retried with Get to get more details.
func Head(ref name.Reference, options ...Option) (*v1.Descriptor, error) {
o, err := makeOptions(options...)
if err != nil {
return nil, err
}
return newPuller(o).Head(o.context, ref)
}
// Handle options and fetch the manifest with the acceptable MediaTypes in the
// Accept header.
func get(ref name.Reference, acceptable []types.MediaType, options ...Option) (*Descriptor, error) {
o, err := makeOptions(options...)
if err != nil {
return nil, err
}
return newPuller(o).get(o.context, ref, acceptable, o.platform)
}
// Image converts the Descriptor into a v1.Image.
//
// If the fetched artifact is already an image, it will just return it.
//
// If the fetched artifact is an index, it will attempt to resolve the index to
// a child image with the appropriate platform.
//
// See WithPlatform to set the desired platform.
func (d *Descriptor) Image() (v1.Image, error) {
switch d.MediaType {
case types.DockerManifestSchema1, types.DockerManifestSchema1Signed:
// We don't care to support schema 1 images:
// https://github.com/google/go-containerregistry/issues/377
return nil, newErrSchema1(d.MediaType)
case types.OCIImageIndex, types.DockerManifestList:
// We want an image but the registry has an index, resolve it to an image.
return d.remoteIndex().imageByPlatform(d.platform)
case types.OCIManifestSchema1, types.DockerManifestSchema2:
// These are expected. Enumerated here to allow a default case.
default:
// We could just return an error here, but some registries (e.g. static
// registries) don't set the Content-Type headers correctly, so instead...
logs.Warn.Printf("Unexpected media type for Image(): %s", d.MediaType)
}
// Wrap the v1.Layers returned by this v1.Image in a hint for downstream
// remote.Write calls to facilitate cross-repo "mounting".
imgCore, err := partial.CompressedToImage(d.remoteImage())
if err != nil {
return nil, err
}
return &mountableImage{
Image: imgCore,
Reference: d.ref,
}, nil
}
// Schema1 converts the Descriptor into a v1.Image for v2 schema 1 media types.
//
// The v1.Image returned by this method does not implement the entire interface because it would be inefficient.
// This exists mostly to make it easier to copy schema 1 images around or look at their filesystems.
// This is separate from Image() to avoid a backward incompatible change for callers expecting ErrSchema1.
func (d *Descriptor) Schema1() (v1.Image, error) {
i := &schema1{
ref: d.ref,
fetcher: d.fetcher,
ctx: d.ctx,
manifest: d.Manifest,
mediaType: d.MediaType,
descriptor: &d.Descriptor,
}
return &mountableImage{
Image: i,
Reference: d.ref,
}, nil
}
// ImageIndex converts the Descriptor into a v1.ImageIndex.
func (d *Descriptor) ImageIndex() (v1.ImageIndex, error) {
switch d.MediaType {
case types.DockerManifestSchema1, types.DockerManifestSchema1Signed:
// We don't care to support schema 1 images:
// https://github.com/google/go-containerregistry/issues/377
return nil, newErrSchema1(d.MediaType)
case types.OCIManifestSchema1, types.DockerManifestSchema2:
// We want an index but the registry has an image, nothing we can do.
return nil, fmt.Errorf("unexpected media type for ImageIndex(): %s; call Image() instead", d.MediaType)
case types.OCIImageIndex, types.DockerManifestList:
// These are expected.
default:
// We could just return an error here, but some registries (e.g. static
// registries) don't set the Content-Type headers correctly, so instead...
logs.Warn.Printf("Unexpected media type for ImageIndex(): %s", d.MediaType)
}
return d.remoteIndex(), nil
}
func (d *Descriptor) remoteImage() *remoteImage {
return &remoteImage{
ref: d.ref,
ctx: d.ctx,
fetcher: d.fetcher,
manifest: d.Manifest,
mediaType: d.MediaType,
descriptor: &d.Descriptor,
}
}
func (d *Descriptor) remoteIndex() *remoteIndex {
return &remoteIndex{
ref: d.ref,
ctx: d.ctx,
fetcher: d.fetcher,
manifest: d.Manifest,
mediaType: d.MediaType,
descriptor: &d.Descriptor,
}
}
-17
View File
@@ -1,17 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// Package remote provides facilities for reading/writing v1.Images from/to
// a remote image registry.
package remote
-317
View File
@@ -1,317 +0,0 @@
// Copyright 2023 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package remote
import (
"bytes"
"context"
"fmt"
"io"
"net/http"
"net/url"
"strings"
"github.com/google/go-containerregistry/internal/redact"
"github.com/google/go-containerregistry/internal/verify"
"github.com/google/go-containerregistry/pkg/authn"
"github.com/google/go-containerregistry/pkg/name"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/remote/transport"
"github.com/google/go-containerregistry/pkg/v1/types"
)
const (
kib = 1024
mib = 1024 * kib
manifestLimit = 100 * mib
)
// fetcher implements methods for reading from a registry.
type fetcher struct {
target resource
client *http.Client
}
func makeFetcher(ctx context.Context, target resource, o *options) (*fetcher, error) {
auth := o.auth
if o.keychain != nil {
kauth, err := authn.Resolve(ctx, o.keychain, target)
if err != nil {
return nil, err
}
auth = kauth
}
reg, ok := target.(name.Registry)
if !ok {
repo, ok := target.(name.Repository)
if !ok {
return nil, fmt.Errorf("unexpected resource: %T", target)
}
reg = repo.Registry
}
tr, err := transport.NewWithContext(ctx, reg, auth, o.transport, []string{target.Scope(transport.PullScope)})
if err != nil {
return nil, err
}
return &fetcher{
target: target,
client: &http.Client{Transport: tr},
}, nil
}
func (f *fetcher) Do(req *http.Request) (*http.Response, error) {
return f.client.Do(req)
}
type resource interface {
Scheme() string
RegistryStr() string
Scope(string) string
authn.Resource
}
// url returns a url.Url for the specified path in the context of this remote image reference.
func (f *fetcher) url(resource, identifier string) url.URL {
u := url.URL{
Scheme: f.target.Scheme(),
Host: f.target.RegistryStr(),
// Default path if this is not a repository.
Path: "/v2/_catalog",
}
if repo, ok := f.target.(name.Repository); ok {
u.Path = fmt.Sprintf("/v2/%s/%s/%s", repo.RepositoryStr(), resource, identifier)
}
return u
}
func (f *fetcher) get(ctx context.Context, ref name.Reference, acceptable []types.MediaType, platform v1.Platform) (*Descriptor, error) {
b, desc, err := f.fetchManifest(ctx, ref, acceptable)
if err != nil {
return nil, err
}
return &Descriptor{
ref: ref,
ctx: ctx,
fetcher: *f,
Manifest: b,
Descriptor: *desc,
platform: platform,
}, nil
}
func (f *fetcher) fetchManifest(ctx context.Context, ref name.Reference, acceptable []types.MediaType) ([]byte, *v1.Descriptor, error) {
u := f.url("manifests", ref.Identifier())
req, err := http.NewRequest(http.MethodGet, u.String(), nil)
if err != nil {
return nil, nil, err
}
accept := []string{}
for _, mt := range acceptable {
accept = append(accept, string(mt))
}
req.Header.Set("Accept", strings.Join(accept, ","))
resp, err := f.client.Do(req.WithContext(ctx))
if err != nil {
return nil, nil, err
}
defer resp.Body.Close()
if err := transport.CheckError(resp, http.StatusOK); err != nil {
return nil, nil, err
}
manifest, err := io.ReadAll(io.LimitReader(resp.Body, manifestLimit))
if err != nil {
return nil, nil, err
}
digest, size, err := v1.SHA256(bytes.NewReader(manifest))
if err != nil {
return nil, nil, err
}
mediaType := types.MediaType(resp.Header.Get("Content-Type"))
contentDigest, err := v1.NewHash(resp.Header.Get("Docker-Content-Digest"))
if err == nil && mediaType == types.DockerManifestSchema1Signed {
// If we can parse the digest from the header, and it's a signed schema 1
// manifest, let's use that for the digest to appease older registries.
digest = contentDigest
}
// Validate the digest matches what we asked for, if pulling by digest.
if dgst, ok := ref.(name.Digest); ok {
if digest.String() != dgst.DigestStr() {
return nil, nil, fmt.Errorf("manifest digest: %q does not match requested digest: %q for %q", digest, dgst.DigestStr(), ref)
}
}
var artifactType string
mf, _ := v1.ParseManifest(bytes.NewReader(manifest))
// Failing to parse as a manifest should just be ignored.
// The manifest might not be valid, and that's okay.
if mf != nil && !mf.Config.MediaType.IsConfig() {
artifactType = string(mf.Config.MediaType)
}
// Do nothing for tags; I give up.
//
// We'd like to validate that the "Docker-Content-Digest" header matches what is returned by the registry,
// but so many registries implement this incorrectly that it's not worth checking.
//
// For reference:
// https://github.com/GoogleContainerTools/kaniko/issues/298
// Return all this info since we have to calculate it anyway.
desc := v1.Descriptor{
Digest: digest,
Size: size,
MediaType: mediaType,
ArtifactType: artifactType,
}
return manifest, &desc, nil
}
func (f *fetcher) headManifest(ctx context.Context, ref name.Reference, acceptable []types.MediaType) (*v1.Descriptor, error) {
u := f.url("manifests", ref.Identifier())
req, err := http.NewRequest(http.MethodHead, u.String(), nil)
if err != nil {
return nil, err
}
accept := []string{}
for _, mt := range acceptable {
accept = append(accept, string(mt))
}
req.Header.Set("Accept", strings.Join(accept, ","))
resp, err := f.client.Do(req.WithContext(ctx))
if err != nil {
return nil, err
}
defer resp.Body.Close()
if err := transport.CheckError(resp, http.StatusOK); err != nil {
return nil, err
}
mth := resp.Header.Get("Content-Type")
if mth == "" {
return nil, fmt.Errorf("HEAD %s: response did not include Content-Type header", u.String())
}
mediaType := types.MediaType(mth)
size := resp.ContentLength
if size == -1 {
return nil, fmt.Errorf("GET %s: response did not include Content-Length header", u.String())
}
dh := resp.Header.Get("Docker-Content-Digest")
if dh == "" {
return nil, fmt.Errorf("HEAD %s: response did not include Docker-Content-Digest header", u.String())
}
digest, err := v1.NewHash(dh)
if err != nil {
return nil, err
}
// Validate the digest matches what we asked for, if pulling by digest.
if dgst, ok := ref.(name.Digest); ok {
if digest.String() != dgst.DigestStr() {
return nil, fmt.Errorf("manifest digest: %q does not match requested digest: %q for %q", digest, dgst.DigestStr(), ref)
}
}
// Return all this info since we have to calculate it anyway.
return &v1.Descriptor{
Digest: digest,
Size: size,
MediaType: mediaType,
}, nil
}
func (f *fetcher) fetchBlob(ctx context.Context, size int64, h v1.Hash) (io.ReadCloser, error) {
u := f.url("blobs", h.String())
req, err := http.NewRequest(http.MethodGet, u.String(), nil)
if err != nil {
return nil, err
}
resp, err := f.client.Do(req.WithContext(ctx))
if err != nil {
return nil, redact.Error(err)
}
if err := transport.CheckError(resp, http.StatusOK); err != nil {
resp.Body.Close()
return nil, err
}
// Do whatever we can.
// If we have an expected size and Content-Length doesn't match, return an error.
// If we don't have an expected size and we do have a Content-Length, use Content-Length.
if hsize := resp.ContentLength; hsize != -1 {
if size == verify.SizeUnknown {
size = hsize
} else if hsize != size {
return nil, fmt.Errorf("GET %s: Content-Length header %d does not match expected size %d", u.String(), hsize, size)
}
}
return verify.ReadCloser(resp.Body, size, h)
}
func (f *fetcher) headBlob(ctx context.Context, h v1.Hash) (*http.Response, error) {
u := f.url("blobs", h.String())
req, err := http.NewRequest(http.MethodHead, u.String(), nil)
if err != nil {
return nil, err
}
resp, err := f.client.Do(req.WithContext(ctx))
if err != nil {
return nil, redact.Error(err)
}
if err := transport.CheckError(resp, http.StatusOK); err != nil {
resp.Body.Close()
return nil, err
}
return resp, nil
}
func (f *fetcher) blobExists(ctx context.Context, h v1.Hash) (bool, error) {
u := f.url("blobs", h.String())
req, err := http.NewRequest(http.MethodHead, u.String(), nil)
if err != nil {
return false, err
}
resp, err := f.client.Do(req.WithContext(ctx))
if err != nil {
return false, redact.Error(err)
}
defer resp.Body.Close()
if err := transport.CheckError(resp, http.StatusOK, http.StatusNotFound); err != nil {
return false, err
}
return resp.StatusCode == http.StatusOK, nil
}
-277
View File
@@ -1,277 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package remote
import (
"bytes"
"context"
"io"
"net/http"
"net/url"
"sync"
"github.com/google/go-containerregistry/internal/redact"
"github.com/google/go-containerregistry/internal/verify"
"github.com/google/go-containerregistry/pkg/name"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/partial"
"github.com/google/go-containerregistry/pkg/v1/remote/transport"
"github.com/google/go-containerregistry/pkg/v1/types"
)
var acceptableImageMediaTypes = []types.MediaType{
types.DockerManifestSchema2,
types.OCIManifestSchema1,
}
// remoteImage accesses an image from a remote registry
type remoteImage struct {
fetcher fetcher
ref name.Reference
ctx context.Context
manifestLock sync.Mutex // Protects manifest
manifest []byte
configLock sync.Mutex // Protects config
config []byte
mediaType types.MediaType
descriptor *v1.Descriptor
}
func (r *remoteImage) ArtifactType() (string, error) {
// kind of a hack, but RawManifest does appropriate locking/memoization
// and makes sure r.descriptor is populated.
if _, err := r.RawManifest(); err != nil {
return "", err
}
return r.descriptor.ArtifactType, nil
}
var _ partial.CompressedImageCore = (*remoteImage)(nil)
// Image provides access to a remote image reference.
func Image(ref name.Reference, options ...Option) (v1.Image, error) {
desc, err := Get(ref, options...)
if err != nil {
return nil, err
}
return desc.Image()
}
func (r *remoteImage) MediaType() (types.MediaType, error) {
if string(r.mediaType) != "" {
return r.mediaType, nil
}
return types.DockerManifestSchema2, nil
}
func (r *remoteImage) RawManifest() ([]byte, error) {
r.manifestLock.Lock()
defer r.manifestLock.Unlock()
if r.manifest != nil {
return r.manifest, nil
}
// NOTE(jonjohnsonjr): We should never get here because the public entrypoints
// do type-checking via remote.Descriptor. I've left this here for tests that
// directly instantiate a remoteImage.
manifest, desc, err := r.fetcher.fetchManifest(r.ctx, r.ref, acceptableImageMediaTypes)
if err != nil {
return nil, err
}
if r.descriptor == nil {
r.descriptor = desc
}
r.mediaType = desc.MediaType
r.manifest = manifest
return r.manifest, nil
}
func (r *remoteImage) RawConfigFile() ([]byte, error) {
r.configLock.Lock()
defer r.configLock.Unlock()
if r.config != nil {
return r.config, nil
}
m, err := partial.Manifest(r)
if err != nil {
return nil, err
}
if m.Config.Data != nil {
if err := verify.Descriptor(m.Config); err != nil {
return nil, err
}
r.config = m.Config.Data
return r.config, nil
}
body, err := r.fetcher.fetchBlob(r.ctx, m.Config.Size, m.Config.Digest)
if err != nil {
return nil, err
}
defer body.Close()
r.config, err = io.ReadAll(body)
if err != nil {
return nil, err
}
return r.config, nil
}
// Descriptor retains the original descriptor from an index manifest.
// See partial.Descriptor.
func (r *remoteImage) Descriptor() (*v1.Descriptor, error) {
// kind of a hack, but RawManifest does appropriate locking/memoization
// and makes sure r.descriptor is populated.
_, err := r.RawManifest()
return r.descriptor, err
}
func (r *remoteImage) ConfigLayer() (v1.Layer, error) {
if _, err := r.RawManifest(); err != nil {
return nil, err
}
m, err := partial.Manifest(r)
if err != nil {
return nil, err
}
return partial.CompressedToLayer(&remoteImageLayer{
ri: r,
ctx: r.ctx,
digest: m.Config.Digest,
})
}
// remoteImageLayer implements partial.CompressedLayer
type remoteImageLayer struct {
ri *remoteImage
ctx context.Context
digest v1.Hash
}
// Digest implements partial.CompressedLayer
func (rl *remoteImageLayer) Digest() (v1.Hash, error) {
return rl.digest, nil
}
// Compressed implements partial.CompressedLayer
func (rl *remoteImageLayer) Compressed() (io.ReadCloser, error) {
urls := []url.URL{rl.ri.fetcher.url("blobs", rl.digest.String())}
// Add alternative layer sources from URLs (usually none).
d, err := partial.BlobDescriptor(rl, rl.digest)
if err != nil {
return nil, err
}
if d.Data != nil {
return verify.ReadCloser(io.NopCloser(bytes.NewReader(d.Data)), d.Size, d.Digest)
}
// We don't want to log binary layers -- this can break terminals.
ctx := redact.NewContext(rl.ctx, "omitting binary blobs from logs")
for _, s := range d.URLs {
u, err := url.Parse(s)
if err != nil {
return nil, err
}
urls = append(urls, *u)
}
// The lastErr for most pulls will be the same (the first error), but for
// foreign layers we'll want to surface the last one, since we try to pull
// from the registry first, which would often fail.
// TODO: Maybe we don't want to try pulling from the registry first?
var lastErr error
for _, u := range urls {
req, err := http.NewRequest(http.MethodGet, u.String(), nil)
if err != nil {
return nil, err
}
resp, err := rl.ri.fetcher.Do(req.WithContext(ctx))
if err != nil {
lastErr = err
continue
}
if err := transport.CheckError(resp, http.StatusOK); err != nil {
resp.Body.Close()
lastErr = err
continue
}
return verify.ReadCloser(resp.Body, d.Size, rl.digest)
}
return nil, lastErr
}
// Manifest implements partial.WithManifest so that we can use partial.BlobSize below.
func (rl *remoteImageLayer) Manifest() (*v1.Manifest, error) {
return partial.Manifest(rl.ri)
}
// MediaType implements v1.Layer
func (rl *remoteImageLayer) MediaType() (types.MediaType, error) {
bd, err := partial.BlobDescriptor(rl, rl.digest)
if err != nil {
return "", err
}
return bd.MediaType, nil
}
// Size implements partial.CompressedLayer
func (rl *remoteImageLayer) Size() (int64, error) {
// Look up the size of this digest in the manifest to avoid a request.
return partial.BlobSize(rl, rl.digest)
}
// ConfigFile implements partial.WithManifestAndConfigFile so that we can use partial.BlobToDiffID below.
func (rl *remoteImageLayer) ConfigFile() (*v1.ConfigFile, error) {
return partial.ConfigFile(rl.ri)
}
// DiffID implements partial.WithDiffID so that we don't recompute a DiffID that we already have
// available in our ConfigFile.
func (rl *remoteImageLayer) DiffID() (v1.Hash, error) {
return partial.BlobToDiffID(rl, rl.digest)
}
// Descriptor retains the original descriptor from an image manifest.
// See partial.Descriptor.
func (rl *remoteImageLayer) Descriptor() (*v1.Descriptor, error) {
return partial.BlobDescriptor(rl, rl.digest)
}
// See partial.Exists.
func (rl *remoteImageLayer) Exists() (bool, error) {
return rl.ri.fetcher.blobExists(rl.ri.ctx, rl.digest)
}
// LayerByDigest implements partial.CompressedLayer
func (r *remoteImage) LayerByDigest(h v1.Hash) (partial.CompressedLayer, error) {
return &remoteImageLayer{
ri: r,
ctx: r.ctx,
digest: h,
}, nil
}
-287
View File
@@ -1,287 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package remote
import (
"bytes"
"context"
"fmt"
"sync"
"github.com/google/go-containerregistry/internal/verify"
"github.com/google/go-containerregistry/pkg/name"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/partial"
"github.com/google/go-containerregistry/pkg/v1/types"
)
var acceptableIndexMediaTypes = []types.MediaType{
types.DockerManifestList,
types.OCIImageIndex,
}
// remoteIndex accesses an index from a remote registry
type remoteIndex struct {
fetcher fetcher
ref name.Reference
ctx context.Context
manifestLock sync.Mutex // Protects manifest
manifest []byte
mediaType types.MediaType
descriptor *v1.Descriptor
}
// Index provides access to a remote index reference.
func Index(ref name.Reference, options ...Option) (v1.ImageIndex, error) {
desc, err := get(ref, acceptableIndexMediaTypes, options...)
if err != nil {
return nil, err
}
return desc.ImageIndex()
}
func (r *remoteIndex) MediaType() (types.MediaType, error) {
if string(r.mediaType) != "" {
return r.mediaType, nil
}
return types.DockerManifestList, nil
}
func (r *remoteIndex) Digest() (v1.Hash, error) {
return partial.Digest(r)
}
func (r *remoteIndex) Size() (int64, error) {
return partial.Size(r)
}
func (r *remoteIndex) RawManifest() ([]byte, error) {
r.manifestLock.Lock()
defer r.manifestLock.Unlock()
if r.manifest != nil {
return r.manifest, nil
}
// NOTE(jonjohnsonjr): We should never get here because the public entrypoints
// do type-checking via remote.Descriptor. I've left this here for tests that
// directly instantiate a remoteIndex.
manifest, desc, err := r.fetcher.fetchManifest(r.ctx, r.ref, acceptableIndexMediaTypes)
if err != nil {
return nil, err
}
if r.descriptor == nil {
r.descriptor = desc
}
r.mediaType = desc.MediaType
r.manifest = manifest
return r.manifest, nil
}
func (r *remoteIndex) IndexManifest() (*v1.IndexManifest, error) {
b, err := r.RawManifest()
if err != nil {
return nil, err
}
return v1.ParseIndexManifest(bytes.NewReader(b))
}
func (r *remoteIndex) Image(h v1.Hash) (v1.Image, error) {
desc, err := r.childByHash(h)
if err != nil {
return nil, err
}
// Descriptor.Image will handle coercing nested indexes into an Image.
return desc.Image()
}
// Descriptor retains the original descriptor from an index manifest.
// See partial.Descriptor.
func (r *remoteIndex) Descriptor() (*v1.Descriptor, error) {
// kind of a hack, but RawManifest does appropriate locking/memoization
// and makes sure r.descriptor is populated.
_, err := r.RawManifest()
return r.descriptor, err
}
func (r *remoteIndex) ImageIndex(h v1.Hash) (v1.ImageIndex, error) {
desc, err := r.childByHash(h)
if err != nil {
return nil, err
}
return desc.ImageIndex()
}
// Workaround for #819.
func (r *remoteIndex) Layer(h v1.Hash) (v1.Layer, error) {
index, err := r.IndexManifest()
if err != nil {
return nil, err
}
for _, childDesc := range index.Manifests {
if h == childDesc.Digest {
l, err := partial.CompressedToLayer(&remoteLayer{
fetcher: r.fetcher,
ctx: r.ctx,
digest: h,
})
if err != nil {
return nil, err
}
return &MountableLayer{
Layer: l,
Reference: r.ref.Context().Digest(h.String()),
}, nil
}
}
return nil, fmt.Errorf("layer not found: %s", h)
}
func (r *remoteIndex) imageByPlatform(platform v1.Platform) (v1.Image, error) {
desc, err := r.childByPlatform(platform)
if err != nil {
return nil, err
}
// Descriptor.Image will handle coercing nested indexes into an Image.
return desc.Image()
}
// This naively matches the first manifest with matching platform attributes.
//
// We should probably use this instead:
//
// github.com/containerd/containerd/platforms
//
// But first we'd need to migrate to:
//
// github.com/opencontainers/image-spec/specs-go/v1
func (r *remoteIndex) childByPlatform(platform v1.Platform) (*Descriptor, error) {
index, err := r.IndexManifest()
if err != nil {
return nil, err
}
for _, childDesc := range index.Manifests {
// If platform is missing from child descriptor, assume it's amd64/linux.
p := defaultPlatform
if childDesc.Platform != nil {
p = *childDesc.Platform
}
if matchesPlatform(p, platform) {
return r.childDescriptor(childDesc, platform)
}
}
return nil, fmt.Errorf("no child with platform %+v in index %s", platform, r.ref)
}
func (r *remoteIndex) childByHash(h v1.Hash) (*Descriptor, error) {
index, err := r.IndexManifest()
if err != nil {
return nil, err
}
for _, childDesc := range index.Manifests {
if h == childDesc.Digest {
return r.childDescriptor(childDesc, defaultPlatform)
}
}
return nil, fmt.Errorf("no child with digest %s in index %s", h, r.ref)
}
// Convert one of this index's child's v1.Descriptor into a remote.Descriptor, with the given platform option.
func (r *remoteIndex) childDescriptor(child v1.Descriptor, platform v1.Platform) (*Descriptor, error) {
ref := r.ref.Context().Digest(child.Digest.String())
var (
manifest []byte
err error
)
if child.Data != nil {
if err := verify.Descriptor(child); err != nil {
return nil, err
}
manifest = child.Data
} else {
manifest, _, err = r.fetcher.fetchManifest(r.ctx, ref, []types.MediaType{child.MediaType})
if err != nil {
return nil, err
}
}
if child.MediaType.IsImage() {
mf, _ := v1.ParseManifest(bytes.NewReader(manifest))
// Failing to parse as a manifest should just be ignored.
// The manifest might not be valid, and that's okay.
if mf != nil && !mf.Config.MediaType.IsConfig() {
child.ArtifactType = string(mf.Config.MediaType)
}
}
return &Descriptor{
ref: ref,
ctx: r.ctx,
fetcher: r.fetcher,
Manifest: manifest,
Descriptor: child,
platform: platform,
}, nil
}
// matchesPlatform checks if the given platform matches the required platforms.
// The given platform matches the required platform if
// - architecture and OS are identical.
// - OS version and variant are identical if provided.
// - features and OS features of the required platform are subsets of those of the given platform.
func matchesPlatform(given, required v1.Platform) bool {
// Required fields that must be identical.
if given.Architecture != required.Architecture || given.OS != required.OS {
return false
}
// Optional fields that may be empty, but must be identical if provided.
if required.OSVersion != "" && given.OSVersion != required.OSVersion {
return false
}
if required.Variant != "" && given.Variant != required.Variant {
return false
}
// Verify required platform's features are a subset of given platform's features.
if !isSubset(given.OSFeatures, required.OSFeatures) {
return false
}
if !isSubset(given.Features, required.Features) {
return false
}
return true
}
// isSubset checks if the required array of strings is a subset of the given lst.
func isSubset(lst, required []string) bool {
set := make(map[string]bool)
for _, value := range lst {
set[value] = true
}
for _, value := range required {
if _, ok := set[value]; !ok {
return false
}
}
return true
}
-77
View File
@@ -1,77 +0,0 @@
// Copyright 2019 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package remote
import (
"context"
"io"
"github.com/google/go-containerregistry/internal/redact"
"github.com/google/go-containerregistry/internal/verify"
"github.com/google/go-containerregistry/pkg/name"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/types"
)
// remoteImagelayer implements partial.CompressedLayer
type remoteLayer struct {
ctx context.Context
fetcher fetcher
digest v1.Hash
}
// Compressed implements partial.CompressedLayer
func (rl *remoteLayer) Compressed() (io.ReadCloser, error) {
// We don't want to log binary layers -- this can break terminals.
ctx := redact.NewContext(rl.ctx, "omitting binary blobs from logs")
return rl.fetcher.fetchBlob(ctx, verify.SizeUnknown, rl.digest)
}
// Compressed implements partial.CompressedLayer
func (rl *remoteLayer) Size() (int64, error) {
resp, err := rl.fetcher.headBlob(rl.ctx, rl.digest)
if err != nil {
return -1, err
}
defer resp.Body.Close()
return resp.ContentLength, nil
}
// Digest implements partial.CompressedLayer
func (rl *remoteLayer) Digest() (v1.Hash, error) {
return rl.digest, nil
}
// MediaType implements v1.Layer
func (rl *remoteLayer) MediaType() (types.MediaType, error) {
return types.DockerLayer, nil
}
// See partial.Exists.
func (rl *remoteLayer) Exists() (bool, error) {
return rl.fetcher.blobExists(rl.ctx, rl.digest)
}
// Layer reads the given blob reference from a registry as a Layer. A blob
// reference here is just a punned name.Digest where the digest portion is the
// digest of the blob to be read and the repository portion is the repo where
// that blob lives.
func Layer(ref name.Digest, options ...Option) (v1.Layer, error) {
o, err := makeOptions(options...)
if err != nil {
return nil, err
}
return newPuller(o).Layer(o.context, ref)
}
-152
View File
@@ -1,152 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package remote
import (
"context"
"encoding/json"
"fmt"
"net/http"
"net/url"
"strings"
"github.com/google/go-containerregistry/pkg/name"
"github.com/google/go-containerregistry/pkg/v1/remote/transport"
)
// ListWithContext calls List with the given context.
//
// Deprecated: Use List and WithContext. This will be removed in a future release.
func ListWithContext(ctx context.Context, repo name.Repository, options ...Option) ([]string, error) {
return List(repo, append(options, WithContext(ctx))...)
}
// List calls /tags/list for the given repository, returning the list of tags
// in the "tags" property.
func List(repo name.Repository, options ...Option) ([]string, error) {
o, err := makeOptions(options...)
if err != nil {
return nil, err
}
return newPuller(o).List(o.context, repo)
}
type Tags struct {
Name string `json:"name"`
Tags []string `json:"tags"`
Next string `json:"next,omitempty"`
}
func (f *fetcher) listPage(ctx context.Context, repo name.Repository, next string, pageSize int) (*Tags, error) {
if next == "" {
uri := &url.URL{
Scheme: repo.Scheme(),
Host: repo.RegistryStr(),
Path: fmt.Sprintf("/v2/%s/tags/list", repo.RepositoryStr()),
}
if pageSize > 0 {
uri.RawQuery = fmt.Sprintf("n=%d", pageSize)
}
next = uri.String()
}
req, err := http.NewRequestWithContext(ctx, "GET", next, nil)
if err != nil {
return nil, err
}
resp, err := f.client.Do(req)
if err != nil {
return nil, err
}
if err := transport.CheckError(resp, http.StatusOK); err != nil {
return nil, err
}
parsed := Tags{}
if err := json.NewDecoder(resp.Body).Decode(&parsed); err != nil {
return nil, err
}
if err := resp.Body.Close(); err != nil {
return nil, err
}
uri, err := getNextPageURL(resp)
if err != nil {
return nil, err
}
if uri != nil {
parsed.Next = uri.String()
}
return &parsed, nil
}
// getNextPageURL checks if there is a Link header in a http.Response which
// contains a link to the next page. If yes it returns the url.URL of the next
// page otherwise it returns nil.
func getNextPageURL(resp *http.Response) (*url.URL, error) {
link := resp.Header.Get("Link")
if link == "" {
return nil, nil
}
if link[0] != '<' {
return nil, fmt.Errorf("failed to parse link header: missing '<' in: %s", link)
}
end := strings.Index(link, ">")
if end == -1 {
return nil, fmt.Errorf("failed to parse link header: missing '>' in: %s", link)
}
link = link[1:end]
linkURL, err := url.Parse(link)
if err != nil {
return nil, err
}
if resp.Request == nil || resp.Request.URL == nil {
return nil, nil
}
linkURL = resp.Request.URL.ResolveReference(linkURL)
return linkURL, nil
}
type Lister struct {
f *fetcher
repo name.Repository
pageSize int
page *Tags
err error
needMore bool
}
func (l *Lister) Next(ctx context.Context) (*Tags, error) {
if l.needMore {
l.page, l.err = l.f.listPage(ctx, l.repo, l.page.Next, l.pageSize)
} else {
l.needMore = true
}
return l.page, l.err
}
func (l *Lister) HasNext() bool {
return l.page != nil && (!l.needMore || l.page.Next != "")
}
-108
View File
@@ -1,108 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package remote
import (
"github.com/google/go-containerregistry/pkg/name"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/partial"
)
// MountableLayer wraps a v1.Layer in a shim that enables the layer to be
// "mounted" when published to another registry.
type MountableLayer struct {
v1.Layer
Reference name.Reference
}
// Descriptor retains the original descriptor from an image manifest.
// See partial.Descriptor.
func (ml *MountableLayer) Descriptor() (*v1.Descriptor, error) {
return partial.Descriptor(ml.Layer)
}
// Exists is a hack. See partial.Exists.
func (ml *MountableLayer) Exists() (bool, error) {
return partial.Exists(ml.Layer)
}
// mountableImage wraps the v1.Layer references returned by the embedded v1.Image
// in MountableLayer's so that remote.Write might attempt to mount them from their
// source repository.
type mountableImage struct {
v1.Image
Reference name.Reference
}
// Layers implements v1.Image
func (mi *mountableImage) Layers() ([]v1.Layer, error) {
ls, err := mi.Image.Layers()
if err != nil {
return nil, err
}
mls := make([]v1.Layer, 0, len(ls))
for _, l := range ls {
mls = append(mls, &MountableLayer{
Layer: l,
Reference: mi.Reference,
})
}
return mls, nil
}
// LayerByDigest implements v1.Image
func (mi *mountableImage) LayerByDigest(d v1.Hash) (v1.Layer, error) {
l, err := mi.Image.LayerByDigest(d)
if err != nil {
return nil, err
}
return &MountableLayer{
Layer: l,
Reference: mi.Reference,
}, nil
}
// LayerByDiffID implements v1.Image
func (mi *mountableImage) LayerByDiffID(d v1.Hash) (v1.Layer, error) {
l, err := mi.Image.LayerByDiffID(d)
if err != nil {
return nil, err
}
return &MountableLayer{
Layer: l,
Reference: mi.Reference,
}, nil
}
// Descriptor retains the original descriptor from an index manifest.
// See partial.Descriptor.
func (mi *mountableImage) Descriptor() (*v1.Descriptor, error) {
return partial.Descriptor(mi.Image)
}
// ConfigLayer retains the original reference so that it can be mounted.
// See partial.ConfigLayer.
func (mi *mountableImage) ConfigLayer() (v1.Layer, error) {
l, err := partial.ConfigLayer(mi.Image)
if err != nil {
return nil, err
}
return &MountableLayer{
Layer: l,
Reference: mi.Reference,
}, nil
}
@@ -1,46 +0,0 @@
// Copyright 2020 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package remote
import (
"github.com/google/go-containerregistry/pkg/name"
"golang.org/x/sync/errgroup"
)
// MultiWrite writes the given Images or ImageIndexes to the given refs, as
// efficiently as possible, by deduping shared layer blobs while uploading them
// in parallel.
func MultiWrite(todo map[name.Reference]Taggable, options ...Option) (rerr error) {
o, err := makeOptions(options...)
if err != nil {
return err
}
if o.progress != nil {
defer func() { o.progress.Close(rerr) }()
}
p := newPusher(o)
g, ctx := errgroup.WithContext(o.context)
g.SetLimit(o.jobs)
for ref, t := range todo {
ref, t := ref, t
g.Go(func() error {
return p.Push(ctx, ref, t)
})
}
return g.Wait()
}
-354
View File
@@ -1,354 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package remote
import (
"context"
"errors"
"io"
"net"
"net/http"
"syscall"
"time"
"github.com/google/go-containerregistry/internal/retry"
"github.com/google/go-containerregistry/pkg/authn"
"github.com/google/go-containerregistry/pkg/logs"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/remote/transport"
)
// Option is a functional option for remote operations.
type Option func(*options) error
type options struct {
auth authn.Authenticator
keychain authn.Keychain
transport http.RoundTripper
context context.Context
jobs int
userAgent string
allowNondistributableArtifacts bool
progress *progress
retryBackoff Backoff
retryPredicate retry.Predicate
retryStatusCodes []int
// Only these options can overwrite Reuse()d options.
platform v1.Platform
pageSize int
filter map[string]string
// Set by Reuse, we currently store one or the other.
puller *Puller
pusher *Pusher
}
var defaultPlatform = v1.Platform{
Architecture: "amd64",
OS: "linux",
}
// Backoff is an alias of retry.Backoff to expose this configuration option to consumers of this lib
type Backoff = retry.Backoff
var defaultRetryPredicate retry.Predicate = func(err error) bool {
// Various failure modes here, as we're often reading from and writing to
// the network.
if retry.IsTemporary(err) || errors.Is(err, io.ErrUnexpectedEOF) || errors.Is(err, io.EOF) || errors.Is(err, syscall.EPIPE) || errors.Is(err, syscall.ECONNRESET) || errors.Is(err, net.ErrClosed) {
logs.Warn.Printf("retrying %v", err)
return true
}
return false
}
// Try this three times, waiting 1s after first failure, 3s after second.
var defaultRetryBackoff = Backoff{
Duration: 1.0 * time.Second,
Factor: 3.0,
Jitter: 0.1,
Steps: 3,
}
// Useful for tests
var fastBackoff = Backoff{
Duration: 1.0 * time.Millisecond,
Factor: 3.0,
Jitter: 0.1,
Steps: 3,
}
var defaultRetryStatusCodes = []int{
http.StatusRequestTimeout,
http.StatusInternalServerError,
http.StatusBadGateway,
http.StatusServiceUnavailable,
http.StatusGatewayTimeout,
499, // nginx-specific, client closed request
522, // Cloudflare-specific, connection timeout
}
const (
defaultJobs = 4
// ECR returns an error if n > 1000:
// https://github.com/google/go-containerregistry/issues/1091
defaultPageSize = 1000
)
// DefaultTransport is based on http.DefaultTransport with modifications
// documented inline below.
var DefaultTransport http.RoundTripper = &http.Transport{
Proxy: http.ProxyFromEnvironment,
DialContext: (&net.Dialer{
Timeout: 30 * time.Second,
KeepAlive: 30 * time.Second,
}).DialContext,
ForceAttemptHTTP2: true,
MaxIdleConns: 100,
IdleConnTimeout: 90 * time.Second,
TLSHandshakeTimeout: 10 * time.Second,
ExpectContinueTimeout: 1 * time.Second,
// We usually are dealing with 2 hosts (at most), split MaxIdleConns between them.
MaxIdleConnsPerHost: 50,
}
func makeOptions(opts ...Option) (*options, error) {
o := &options{
transport: DefaultTransport,
platform: defaultPlatform,
context: context.Background(),
jobs: defaultJobs,
pageSize: defaultPageSize,
retryPredicate: defaultRetryPredicate,
retryBackoff: defaultRetryBackoff,
retryStatusCodes: defaultRetryStatusCodes,
}
for _, option := range opts {
if err := option(o); err != nil {
return nil, err
}
}
switch {
case o.auth != nil && o.keychain != nil:
// It is a better experience to explicitly tell a caller their auth is misconfigured
// than potentially fail silently when the correct auth is overridden by option misuse.
return nil, errors.New("provide an option for either authn.Authenticator or authn.Keychain, not both")
case o.auth == nil:
o.auth = authn.Anonymous
}
// transport.Wrapper is a signal that consumers are opt-ing into providing their own transport without any additional wrapping.
// This is to allow consumers full control over the transports logic, such as providing retry logic.
if _, ok := o.transport.(*transport.Wrapper); !ok {
// Wrap the transport in something that logs requests and responses.
// It's expensive to generate the dumps, so skip it if we're writing
// to nothing.
if logs.Enabled(logs.Debug) {
o.transport = transport.NewLogger(o.transport)
}
// Using customized retry predicate if provided, and fallback to default if not.
predicate := o.retryPredicate
if predicate == nil {
predicate = defaultRetryPredicate
}
// Wrap the transport in something that can retry network flakes.
o.transport = transport.NewRetry(o.transport, transport.WithRetryBackoff(o.retryBackoff), transport.WithRetryPredicate(predicate), transport.WithRetryStatusCodes(o.retryStatusCodes...))
// Wrap this last to prevent transport.New from double-wrapping.
if o.userAgent != "" {
o.transport = transport.NewUserAgent(o.transport, o.userAgent)
}
}
return o, nil
}
// WithTransport is a functional option for overriding the default transport
// for remote operations.
// If transport.Wrapper is provided, this signals that the consumer does *not* want any further wrapping to occur.
// i.e. logging, retry and useragent
//
// The default transport is DefaultTransport.
func WithTransport(t http.RoundTripper) Option {
return func(o *options) error {
o.transport = t
return nil
}
}
// WithAuth is a functional option for overriding the default authenticator
// for remote operations.
// It is an error to use both WithAuth and WithAuthFromKeychain in the same Option set.
//
// The default authenticator is authn.Anonymous.
func WithAuth(auth authn.Authenticator) Option {
return func(o *options) error {
o.auth = auth
return nil
}
}
// WithAuthFromKeychain is a functional option for overriding the default
// authenticator for remote operations, using an authn.Keychain to find
// credentials.
// It is an error to use both WithAuth and WithAuthFromKeychain in the same Option set.
//
// The default authenticator is authn.Anonymous.
func WithAuthFromKeychain(keys authn.Keychain) Option {
return func(o *options) error {
o.keychain = keys
return nil
}
}
// WithPlatform is a functional option for overriding the default platform
// that Image and Descriptor.Image use for resolving an index to an image.
//
// The default platform is amd64/linux.
func WithPlatform(p v1.Platform) Option {
return func(o *options) error {
o.platform = p
return nil
}
}
// WithContext is a functional option for setting the context in http requests
// performed by a given function. Note that this context is used for _all_
// http requests, not just the initial volley. E.g., for remote.Image, the
// context will be set on http requests generated by subsequent calls to
// RawConfigFile() and even methods on layers returned by Layers().
//
// The default context is context.Background().
func WithContext(ctx context.Context) Option {
return func(o *options) error {
o.context = ctx
return nil
}
}
// WithJobs is a functional option for setting the parallelism of remote
// operations performed by a given function. Note that not all remote
// operations support parallelism.
//
// The default value is 4.
func WithJobs(jobs int) Option {
return func(o *options) error {
if jobs <= 0 {
return errors.New("jobs must be greater than zero")
}
o.jobs = jobs
return nil
}
}
// WithUserAgent adds the given string to the User-Agent header for any HTTP
// requests. This header will also include "go-containerregistry/${version}".
//
// If you want to completely overwrite the User-Agent header, use WithTransport.
func WithUserAgent(ua string) Option {
return func(o *options) error {
o.userAgent = ua
return nil
}
}
// WithNondistributable includes non-distributable (foreign) layers
// when writing images, see:
// https://github.com/opencontainers/image-spec/blob/master/layer.md#non-distributable-layers
//
// The default behaviour is to skip these layers
func WithNondistributable(o *options) error {
o.allowNondistributableArtifacts = true
return nil
}
// WithProgress takes a channel that will receive progress updates as bytes are written.
//
// Sending updates to an unbuffered channel will block writes, so callers
// should provide a buffered channel to avoid potential deadlocks.
func WithProgress(updates chan<- v1.Update) Option {
return func(o *options) error {
o.progress = &progress{updates: updates}
o.progress.lastUpdate = &v1.Update{}
return nil
}
}
// WithPageSize sets the given size as the value of parameter 'n' in the request.
//
// To omit the `n` parameter entirely, use WithPageSize(0).
// The default value is 1000.
func WithPageSize(size int) Option {
return func(o *options) error {
o.pageSize = size
return nil
}
}
// WithRetryBackoff sets the httpBackoff for retry HTTP operations.
func WithRetryBackoff(backoff Backoff) Option {
return func(o *options) error {
o.retryBackoff = backoff
return nil
}
}
// WithRetryPredicate sets the predicate for retry HTTP operations.
func WithRetryPredicate(predicate retry.Predicate) Option {
return func(o *options) error {
o.retryPredicate = predicate
return nil
}
}
// WithRetryStatusCodes sets which http response codes will be retried.
func WithRetryStatusCodes(codes ...int) Option {
return func(o *options) error {
o.retryStatusCodes = codes
return nil
}
}
// WithFilter sets the filter querystring for HTTP operations.
func WithFilter(key string, value string) Option {
return func(o *options) error {
if o.filter == nil {
o.filter = map[string]string{}
}
o.filter[key] = value
return nil
}
}
// Reuse takes a Puller or Pusher and reuses it for remote interactions
// rather than starting from a clean slate. For example, it will reuse token exchanges
// when possible and avoid sending redundant HEAD requests.
//
// Reuse will take precedence over other options passed to most remote functions because
// most options deal with setting up auth and transports, which Reuse intetionally skips.
func Reuse[I *Puller | *Pusher](i I) Option {
return func(o *options) error {
if puller, ok := any(i).(*Puller); ok {
o.puller = puller
} else if pusher, ok := any(i).(*Pusher); ok {
o.pusher = pusher
}
return nil
}
}
@@ -1,76 +0,0 @@
// Copyright 2022 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package remote
import (
"io"
"sync"
"sync/atomic"
v1 "github.com/google/go-containerregistry/pkg/v1"
)
type progress struct {
sync.Mutex
updates chan<- v1.Update
lastUpdate *v1.Update
}
func (p *progress) total(delta int64) {
p.Lock()
defer p.Unlock()
atomic.AddInt64(&p.lastUpdate.Total, delta)
}
func (p *progress) complete(delta int64) {
p.Lock()
defer p.Unlock()
p.updates <- v1.Update{
Total: p.lastUpdate.Total,
Complete: atomic.AddInt64(&p.lastUpdate.Complete, delta),
}
}
func (p *progress) err(err error) error {
if err != nil && p.updates != nil {
p.updates <- v1.Update{Error: err}
}
return err
}
func (p *progress) Close(err error) {
_ = p.err(err)
close(p.updates)
}
type progressReader struct {
rc io.ReadCloser
count *int64 // number of bytes this reader has read, to support resetting on retry.
progress *progress
}
func (r *progressReader) Read(b []byte) (int, error) {
n, err := r.rc.Read(b)
if err != nil {
return n, err
}
atomic.AddInt64(r.count, int64(n))
// TODO: warn/debug log if sending takes too long, or if sending is blocked while context is canceled.
r.progress.complete(int64(n))
return n, nil
}
func (r *progressReader) Close() error { return r.rc.Close() }
-222
View File
@@ -1,222 +0,0 @@
// Copyright 2023 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package remote
import (
"context"
"sync"
"github.com/google/go-containerregistry/pkg/name"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/partial"
"github.com/google/go-containerregistry/pkg/v1/types"
)
type Puller struct {
o *options
// map[resource]*reader
readers sync.Map
}
func NewPuller(options ...Option) (*Puller, error) {
o, err := makeOptions(options...)
if err != nil {
return nil, err
}
return newPuller(o), nil
}
func newPuller(o *options) *Puller {
if o.puller != nil {
return o.puller
}
return &Puller{
o: o,
}
}
type reader struct {
// in
target resource
o *options
// f()
once sync.Once
// out
f *fetcher
err error
}
// this will run once per reader instance
func (r *reader) init(ctx context.Context) error {
r.once.Do(func() {
r.f, r.err = makeFetcher(ctx, r.target, r.o)
})
return r.err
}
func (p *Puller) fetcher(ctx context.Context, target resource) (*fetcher, error) {
v, _ := p.readers.LoadOrStore(target, &reader{
target: target,
o: p.o,
})
rr := v.(*reader)
return rr.f, rr.init(ctx)
}
// Head is like remote.Head, but avoids re-authenticating when possible.
func (p *Puller) Head(ctx context.Context, ref name.Reference) (*v1.Descriptor, error) {
f, err := p.fetcher(ctx, ref.Context())
if err != nil {
return nil, err
}
return f.headManifest(ctx, ref, allManifestMediaTypes)
}
// Get is like remote.Get, but avoids re-authenticating when possible.
func (p *Puller) Get(ctx context.Context, ref name.Reference) (*Descriptor, error) {
return p.get(ctx, ref, allManifestMediaTypes, p.o.platform)
}
func (p *Puller) get(ctx context.Context, ref name.Reference, acceptable []types.MediaType, platform v1.Platform) (*Descriptor, error) {
f, err := p.fetcher(ctx, ref.Context())
if err != nil {
return nil, err
}
return f.get(ctx, ref, acceptable, platform)
}
// Layer is like remote.Layer, but avoids re-authenticating when possible.
func (p *Puller) Layer(ctx context.Context, ref name.Digest) (v1.Layer, error) {
f, err := p.fetcher(ctx, ref.Context())
if err != nil {
return nil, err
}
h, err := v1.NewHash(ref.Identifier())
if err != nil {
return nil, err
}
l, err := partial.CompressedToLayer(&remoteLayer{
fetcher: *f,
ctx: ctx,
digest: h,
})
if err != nil {
return nil, err
}
return &MountableLayer{
Layer: l,
Reference: ref,
}, nil
}
// List lists tags in a repo and handles pagination, returning the full list of tags.
func (p *Puller) List(ctx context.Context, repo name.Repository) ([]string, error) {
lister, err := p.Lister(ctx, repo)
if err != nil {
return nil, err
}
tagList := []string{}
for lister.HasNext() {
tags, err := lister.Next(ctx)
if err != nil {
return nil, err
}
tagList = append(tagList, tags.Tags...)
}
return tagList, nil
}
// Lister lists tags in a repo and returns a Lister for paginating through the results.
func (p *Puller) Lister(ctx context.Context, repo name.Repository) (*Lister, error) {
return p.lister(ctx, repo, p.o.pageSize)
}
func (p *Puller) lister(ctx context.Context, repo name.Repository, pageSize int) (*Lister, error) {
f, err := p.fetcher(ctx, repo)
if err != nil {
return nil, err
}
page, err := f.listPage(ctx, repo, "", pageSize)
if err != nil {
return nil, err
}
return &Lister{
f: f,
repo: repo,
pageSize: pageSize,
page: page,
err: err,
}, nil
}
// Catalog lists repos in a registry and handles pagination, returning the full list of repos.
func (p *Puller) Catalog(ctx context.Context, reg name.Registry) ([]string, error) {
return p.catalog(ctx, reg, p.o.pageSize)
}
func (p *Puller) catalog(ctx context.Context, reg name.Registry, pageSize int) ([]string, error) {
catalogger, err := p.catalogger(ctx, reg, pageSize)
if err != nil {
return nil, err
}
repoList := []string{}
for catalogger.HasNext() {
repos, err := catalogger.Next(ctx)
if err != nil {
return nil, err
}
repoList = append(repoList, repos.Repos...)
}
return repoList, nil
}
// Catalogger lists repos in a registry and returns a Catalogger for paginating through the results.
func (p *Puller) Catalogger(ctx context.Context, reg name.Registry) (*Catalogger, error) {
return p.catalogger(ctx, reg, p.o.pageSize)
}
func (p *Puller) catalogger(ctx context.Context, reg name.Registry, pageSize int) (*Catalogger, error) {
f, err := p.fetcher(ctx, reg)
if err != nil {
return nil, err
}
page, err := f.catalogPage(ctx, reg, "", pageSize)
if err != nil {
return nil, err
}
return &Catalogger{
f: f,
reg: reg,
pageSize: pageSize,
page: page,
err: err,
}, nil
}
func (p *Puller) referrers(ctx context.Context, d name.Digest, filter map[string]string) (v1.ImageIndex, error) {
f, err := p.fetcher(ctx, d.Context())
if err != nil {
return nil, err
}
return f.fetchReferrers(ctx, filter, d)
}
-573
View File
@@ -1,573 +0,0 @@
// Copyright 2023 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package remote
import (
"bytes"
"context"
"errors"
"fmt"
"net/http"
"net/url"
"sync"
"github.com/google/go-containerregistry/pkg/logs"
"github.com/google/go-containerregistry/pkg/name"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/partial"
"github.com/google/go-containerregistry/pkg/v1/remote/transport"
"github.com/google/go-containerregistry/pkg/v1/stream"
"github.com/google/go-containerregistry/pkg/v1/types"
"golang.org/x/sync/errgroup"
)
type manifest interface {
Taggable
partial.Describable
}
// key is either v1.Hash or v1.Layer (for stream.Layer)
type workers struct {
// map[v1.Hash|v1.Layer]*sync.Once
onces sync.Map
// map[v1.Hash|v1.Layer]error
errors sync.Map
}
func nop() error {
return nil
}
func (w *workers) err(digest v1.Hash) error {
v, ok := w.errors.Load(digest)
if !ok || v == nil {
return nil
}
return v.(error)
}
func (w *workers) Do(digest v1.Hash, f func() error) error {
// We don't care if it was loaded or not because the sync.Once will do it for us.
once, _ := w.onces.LoadOrStore(digest, &sync.Once{})
once.(*sync.Once).Do(func() {
w.errors.Store(digest, f())
})
err := w.err(digest)
if err != nil {
// Allow this to be retried by another caller.
w.onces.Delete(digest)
}
return err
}
func (w *workers) Stream(layer v1.Layer, f func() error) error {
// We don't care if it was loaded or not because the sync.Once will do it for us.
once, _ := w.onces.LoadOrStore(layer, &sync.Once{})
once.(*sync.Once).Do(func() {
w.errors.Store(layer, f())
})
v, ok := w.errors.Load(layer)
if !ok || v == nil {
return nil
}
return v.(error)
}
type Pusher struct {
o *options
// map[name.Repository]*repoWriter
writers sync.Map
}
func NewPusher(options ...Option) (*Pusher, error) {
o, err := makeOptions(options...)
if err != nil {
return nil, err
}
return newPusher(o), nil
}
func newPusher(o *options) *Pusher {
if o.pusher != nil {
return o.pusher
}
return &Pusher{
o: o,
}
}
func (p *Pusher) writer(ctx context.Context, repo name.Repository, o *options) (*repoWriter, error) {
v, _ := p.writers.LoadOrStore(repo, &repoWriter{
repo: repo,
o: o,
})
rw := v.(*repoWriter)
return rw, rw.init(ctx)
}
func (p *Pusher) Put(ctx context.Context, ref name.Reference, t Taggable) error {
w, err := p.writer(ctx, ref.Context(), p.o)
if err != nil {
return err
}
m, err := taggableToManifest(t)
if err != nil {
return err
}
return w.commitManifest(ctx, ref, m)
}
func (p *Pusher) Push(ctx context.Context, ref name.Reference, t Taggable) error {
w, err := p.writer(ctx, ref.Context(), p.o)
if err != nil {
return err
}
return w.writeManifest(ctx, ref, t)
}
func (p *Pusher) Upload(ctx context.Context, repo name.Repository, l v1.Layer) error {
w, err := p.writer(ctx, repo, p.o)
if err != nil {
return err
}
return w.writeLayer(ctx, l)
}
func (p *Pusher) Delete(ctx context.Context, ref name.Reference) error {
w, err := p.writer(ctx, ref.Context(), p.o)
if err != nil {
return err
}
u := url.URL{
Scheme: ref.Context().Scheme(),
Host: ref.Context().RegistryStr(),
Path: fmt.Sprintf("/v2/%s/manifests/%s", ref.Context().RepositoryStr(), ref.Identifier()),
}
req, err := http.NewRequest(http.MethodDelete, u.String(), nil)
if err != nil {
return err
}
resp, err := w.w.client.Do(req.WithContext(ctx))
if err != nil {
return err
}
defer resp.Body.Close()
return transport.CheckError(resp, http.StatusOK, http.StatusAccepted)
// TODO(jason): If the manifest had a `subject`, and if the registry
// doesn't support Referrers, update the index pointed to by the
// subject's fallback tag to remove the descriptor for this manifest.
}
type repoWriter struct {
repo name.Repository
o *options
once sync.Once
w *writer
err error
work *workers
}
// this will run once per repoWriter instance
func (rw *repoWriter) init(ctx context.Context) error {
rw.once.Do(func() {
rw.work = &workers{}
rw.w, rw.err = makeWriter(ctx, rw.repo, nil, rw.o)
})
return rw.err
}
func (rw *repoWriter) writeDeps(ctx context.Context, m manifest) error {
if img, ok := m.(v1.Image); ok {
return rw.writeLayers(ctx, img)
}
if idx, ok := m.(v1.ImageIndex); ok {
return rw.writeChildren(ctx, idx)
}
// This has no deps, not an error (e.g. something you want to just PUT).
return nil
}
type describable struct {
desc v1.Descriptor
}
func (d describable) Digest() (v1.Hash, error) {
return d.desc.Digest, nil
}
func (d describable) Size() (int64, error) {
return d.desc.Size, nil
}
func (d describable) MediaType() (types.MediaType, error) {
return d.desc.MediaType, nil
}
type tagManifest struct {
Taggable
partial.Describable
}
func taggableToManifest(t Taggable) (manifest, error) {
if m, ok := t.(manifest); ok {
return m, nil
}
if d, ok := t.(*Descriptor); ok {
if d.MediaType.IsIndex() {
return d.ImageIndex()
}
if d.MediaType.IsImage() {
return d.Image()
}
if d.MediaType.IsSchema1() {
return d.Schema1()
}
return tagManifest{t, describable{d.toDesc()}}, nil
}
desc := v1.Descriptor{
// A reasonable default if Taggable doesn't implement MediaType.
MediaType: types.DockerManifestSchema2,
}
b, err := t.RawManifest()
if err != nil {
return nil, err
}
if wmt, ok := t.(withMediaType); ok {
desc.MediaType, err = wmt.MediaType()
if err != nil {
return nil, err
}
}
desc.Digest, desc.Size, err = v1.SHA256(bytes.NewReader(b))
if err != nil {
return nil, err
}
return tagManifest{t, describable{desc}}, nil
}
func (rw *repoWriter) writeManifest(ctx context.Context, ref name.Reference, t Taggable) error {
m, err := taggableToManifest(t)
if err != nil {
return err
}
needDeps := true
digest, err := m.Digest()
if errors.Is(err, stream.ErrNotComputed) {
if err := rw.writeDeps(ctx, m); err != nil {
return err
}
needDeps = false
digest, err = m.Digest()
if err != nil {
return err
}
} else if err != nil {
return err
}
// This may be a lazy child where we have no ref until digest is computed.
if ref == nil {
ref = rw.repo.Digest(digest.String())
}
// For tags, we want to do this check outside of our Work.Do closure because
// we don't want to dedupe based on the manifest digest.
_, byTag := ref.(name.Tag)
if byTag {
if exists, err := rw.manifestExists(ctx, ref, t); err != nil {
return err
} else if exists {
return nil
}
}
// The following work.Do will get deduped by digest, so it won't happen unless
// this tag happens to be the first commitManifest to run for that digest.
needPut := byTag
if err := rw.work.Do(digest, func() error {
if !byTag {
if exists, err := rw.manifestExists(ctx, ref, t); err != nil {
return err
} else if exists {
return nil
}
}
if needDeps {
if err := rw.writeDeps(ctx, m); err != nil {
return err
}
}
needPut = false
return rw.commitManifest(ctx, ref, m)
}); err != nil {
return err
}
if !needPut {
return nil
}
// Only runs for tags that got deduped by digest.
return rw.commitManifest(ctx, ref, m)
}
func (rw *repoWriter) writeChildren(ctx context.Context, idx v1.ImageIndex) error {
children, err := partial.Manifests(idx)
if err != nil {
return err
}
g, ctx := errgroup.WithContext(ctx)
g.SetLimit(rw.o.jobs)
for _, child := range children {
child := child
if err := rw.writeChild(ctx, child, g); err != nil {
return err
}
}
return g.Wait()
}
func (rw *repoWriter) writeChild(ctx context.Context, child partial.Describable, g *errgroup.Group) error {
switch child := child.(type) {
case v1.ImageIndex:
// For recursive index, we want to do a depth-first launching of goroutines
// to avoid deadlocking.
//
// Note that this is rare, so the impact of this should be really small.
return rw.writeManifest(ctx, nil, child)
case v1.Image:
g.Go(func() error {
return rw.writeManifest(ctx, nil, child)
})
case v1.Layer:
g.Go(func() error {
return rw.writeLayer(ctx, child)
})
default:
// This can't happen.
return fmt.Errorf("encountered unknown child: %T", child)
}
return nil
}
// TODO: Consider caching some representation of the tags/digests in the destination
// repository as a hint to avoid this optimistic check in cases where we will most
// likely have to do a PUT anyway, e.g. if we are overwriting a tag we just wrote.
func (rw *repoWriter) manifestExists(ctx context.Context, ref name.Reference, t Taggable) (bool, error) {
f := &fetcher{
target: ref.Context(),
client: rw.w.client,
}
m, err := taggableToManifest(t)
if err != nil {
return false, err
}
digest, err := m.Digest()
if err != nil {
// Possibly due to streaming layers.
return false, nil
}
got, err := f.headManifest(ctx, ref, allManifestMediaTypes)
if err != nil {
var terr *transport.Error
if errors.As(err, &terr) {
if terr.StatusCode == http.StatusNotFound {
return false, nil
}
// We treat a 403 here as non-fatal because this existence check is an optimization and
// some registries will return a 403 instead of a 404 in certain situations.
// E.g. https://jfrog.atlassian.net/browse/RTFACT-13797
if terr.StatusCode == http.StatusForbidden {
logs.Debug.Printf("manifestExists unexpected 403: %v", err)
return false, nil
}
}
return false, err
}
if digest != got.Digest {
// Mark that we saw this digest in the registry so we don't have to check it again.
rw.work.Do(got.Digest, nop)
return false, nil
}
if tag, ok := ref.(name.Tag); ok {
logs.Progress.Printf("existing manifest: %s@%s", tag.Identifier(), got.Digest)
} else {
logs.Progress.Print("existing manifest: ", got.Digest)
}
return true, nil
}
func (rw *repoWriter) commitManifest(ctx context.Context, ref name.Reference, m manifest) error {
if rw.o.progress != nil {
size, err := m.Size()
if err != nil {
return err
}
rw.o.progress.total(size)
}
return rw.w.commitManifest(ctx, m, ref)
}
func (rw *repoWriter) writeLayers(pctx context.Context, img v1.Image) error {
ls, err := img.Layers()
if err != nil {
return err
}
g, ctx := errgroup.WithContext(pctx)
g.SetLimit(rw.o.jobs)
for _, l := range ls {
l := l
g.Go(func() error {
return rw.writeLayer(ctx, l)
})
}
mt, err := img.MediaType()
if err != nil {
return err
}
if mt.IsSchema1() {
return g.Wait()
}
cl, err := partial.ConfigLayer(img)
if errors.Is(err, stream.ErrNotComputed) {
if err := g.Wait(); err != nil {
return err
}
cl, err := partial.ConfigLayer(img)
if err != nil {
return err
}
return rw.writeLayer(pctx, cl)
} else if err != nil {
return err
}
g.Go(func() error {
return rw.writeLayer(ctx, cl)
})
return g.Wait()
}
func (rw *repoWriter) writeLayer(ctx context.Context, l v1.Layer) error {
// Skip any non-distributable things.
mt, err := l.MediaType()
if err != nil {
return err
}
if !mt.IsDistributable() && !rw.o.allowNondistributableArtifacts {
return nil
}
digest, err := l.Digest()
if err != nil {
if errors.Is(err, stream.ErrNotComputed) {
return rw.lazyWriteLayer(ctx, l)
}
return err
}
return rw.work.Do(digest, func() error {
if rw.o.progress != nil {
size, err := l.Size()
if err != nil {
return err
}
rw.o.progress.total(size)
}
return rw.w.uploadOne(ctx, l)
})
}
func (rw *repoWriter) lazyWriteLayer(ctx context.Context, l v1.Layer) error {
return rw.work.Stream(l, func() error {
if err := rw.w.uploadOne(ctx, l); err != nil {
return err
}
// Mark this upload completed.
digest, err := l.Digest()
if err != nil {
return err
}
rw.work.Do(digest, nop)
if rw.o.progress != nil {
size, err := l.Size()
if err != nil {
return err
}
rw.o.progress.total(size)
}
return nil
})
}
@@ -1,117 +0,0 @@
// Copyright 2023 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package remote
import (
"bytes"
"context"
"errors"
"io"
"net/http"
"strings"
"github.com/google/go-containerregistry/pkg/name"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/empty"
"github.com/google/go-containerregistry/pkg/v1/mutate"
"github.com/google/go-containerregistry/pkg/v1/remote/transport"
"github.com/google/go-containerregistry/pkg/v1/types"
)
// Referrers returns a list of descriptors that refer to the given manifest digest.
//
// The subject manifest doesn't have to exist in the registry for there to be descriptors that refer to it.
func Referrers(d name.Digest, options ...Option) (v1.ImageIndex, error) {
o, err := makeOptions(options...)
if err != nil {
return nil, err
}
return newPuller(o).referrers(o.context, d, o.filter)
}
// https://github.com/opencontainers/distribution-spec/blob/main/spec.md#referrers-tag-schema
func fallbackTag(d name.Digest) name.Tag {
return d.Context().Tag(strings.Replace(d.DigestStr(), ":", "-", 1))
}
func (f *fetcher) fetchReferrers(ctx context.Context, filter map[string]string, d name.Digest) (v1.ImageIndex, error) {
// Check the Referrers API endpoint first.
u := f.url("referrers", d.DigestStr())
req, err := http.NewRequestWithContext(ctx, http.MethodGet, u.String(), nil)
if err != nil {
return nil, err
}
req.Header.Set("Accept", string(types.OCIImageIndex))
resp, err := f.client.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
if err := transport.CheckError(resp, http.StatusOK, http.StatusNotFound, http.StatusBadRequest, http.StatusNotAcceptable); err != nil {
return nil, err
}
var b []byte
if resp.StatusCode == http.StatusOK && resp.Header.Get("Content-Type") == string(types.OCIImageIndex) {
b, err = io.ReadAll(resp.Body)
if err != nil {
return nil, err
}
} else {
// The registry doesn't support the Referrers API endpoint, so we'll use the fallback tag scheme.
b, _, err = f.fetchManifest(ctx, fallbackTag(d), []types.MediaType{types.OCIImageIndex})
var terr *transport.Error
if errors.As(err, &terr) && terr.StatusCode == http.StatusNotFound {
// Not found just means there are no attachments yet. Start with an empty manifest.
return empty.Index, nil
} else if err != nil {
return nil, err
}
}
h, sz, err := v1.SHA256(bytes.NewReader(b))
if err != nil {
return nil, err
}
idx := &remoteIndex{
fetcher: *f,
ctx: ctx,
manifest: b,
mediaType: types.OCIImageIndex,
descriptor: &v1.Descriptor{
Digest: h,
MediaType: types.OCIImageIndex,
Size: sz,
},
}
return filterReferrersResponse(filter, idx), nil
}
// If filter applied, filter out by artifactType.
// See https://github.com/opencontainers/distribution-spec/blob/main/spec.md#listing-referrers
func filterReferrersResponse(filter map[string]string, in v1.ImageIndex) v1.ImageIndex {
if filter == nil {
return in
}
v, ok := filter["artifactType"]
if !ok {
return in
}
return mutate.RemoveManifests(in, func(desc v1.Descriptor) bool {
return desc.ArtifactType != v
})
}
-118
View File
@@ -1,118 +0,0 @@
// Copyright 2023 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package remote
import (
"bytes"
"context"
"encoding/json"
"github.com/google/go-containerregistry/pkg/name"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/partial"
"github.com/google/go-containerregistry/pkg/v1/types"
)
type schema1 struct {
ref name.Reference
ctx context.Context
fetcher fetcher
manifest []byte
mediaType types.MediaType
descriptor *v1.Descriptor
}
func (s *schema1) Layers() ([]v1.Layer, error) {
m := schema1Manifest{}
if err := json.NewDecoder(bytes.NewReader(s.manifest)).Decode(&m); err != nil {
return nil, err
}
layers := []v1.Layer{}
for i := len(m.FSLayers) - 1; i >= 0; i-- {
fsl := m.FSLayers[i]
h, err := v1.NewHash(fsl.BlobSum)
if err != nil {
return nil, err
}
l, err := s.LayerByDigest(h)
if err != nil {
return nil, err
}
layers = append(layers, l)
}
return layers, nil
}
func (s *schema1) MediaType() (types.MediaType, error) {
return s.mediaType, nil
}
func (s *schema1) Size() (int64, error) {
return s.descriptor.Size, nil
}
func (s *schema1) ConfigName() (v1.Hash, error) {
return partial.ConfigName(s)
}
func (s *schema1) ConfigFile() (*v1.ConfigFile, error) {
return nil, newErrSchema1(s.mediaType)
}
func (s *schema1) RawConfigFile() ([]byte, error) {
return []byte("{}"), nil
}
func (s *schema1) Digest() (v1.Hash, error) {
return s.descriptor.Digest, nil
}
func (s *schema1) Manifest() (*v1.Manifest, error) {
return nil, newErrSchema1(s.mediaType)
}
func (s *schema1) RawManifest() ([]byte, error) {
return s.manifest, nil
}
func (s *schema1) LayerByDigest(h v1.Hash) (v1.Layer, error) {
l, err := partial.CompressedToLayer(&remoteLayer{
fetcher: s.fetcher,
ctx: s.ctx,
digest: h,
})
if err != nil {
return nil, err
}
return &MountableLayer{
Layer: l,
Reference: s.ref.Context().Digest(h.String()),
}, nil
}
func (s *schema1) LayerByDiffID(v1.Hash) (v1.Layer, error) {
return nil, newErrSchema1(s.mediaType)
}
type fslayer struct {
BlobSum string `json:"blobSum"`
}
type schema1Manifest struct {
FSLayers []fslayer `json:"fsLayers"`
}
@@ -1,129 +0,0 @@
# `transport`
[![GoDoc](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/transport?status.svg)](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/transport)
The [distribution protocol](https://github.com/opencontainers/distribution-spec) is fairly simple, but correctly [implementing authentication](../../../authn/README.md) is **hard**.
This package [implements](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/remote/transport#New) an [`http.RoundTripper`](https://godoc.org/net/http#RoundTripper)
that transparently performs:
* [Token
Authentication](https://docs.docker.com/registry/spec/auth/token/) and
* [OAuth2
Authentication](https://docs.docker.com/registry/spec/auth/oauth/)
for registry clients.
## Raison d'être
> Why not just use the [`docker/distribution`](https://godoc.org/github.com/docker/distribution/registry/client/auth) client?
Great question! Mostly, because I don't want to depend on [`prometheus/client_golang`](https://github.com/prometheus/client_golang).
As a performance optimization, that client uses [a cache](https://github.com/docker/distribution/blob/a8371794149d1d95f1e846744b05c87f2f825e5a/registry/client/repository.go#L173) to keep track of a mapping between blob digests and their [descriptors](https://github.com/docker/distribution/blob/a8371794149d1d95f1e846744b05c87f2f825e5a/blobs.go#L57-L86). Unfortunately, the cache [uses prometheus](https://github.com/docker/distribution/blob/a8371794149d1d95f1e846744b05c87f2f825e5a/registry/storage/cache/cachedblobdescriptorstore.go#L44) to track hits and misses, so if you want to use that client you have to pull in all of prometheus, which is pretty large.
![docker/distribution](../../../../images/docker.dot.svg)
> Why does it matter if you depend on prometheus? Who cares?
It's generally polite to your downstream to reduce the number of dependencies your package requires:
* Downloading your package is faster, which helps our Australian friends and people on airplanes.
* There is less code to compile, which speeds up builds and saves the planet from global warming.
* You reduce the likelihood of inflicting dependency hell upon your consumers.
* [Tim Hockin](https://twitter.com/thockin/status/958606077456654336) prefers it based on his experience working on Kubernetes, and he's a pretty smart guy.
> Okay, what about [`containerd/containerd`](https://godoc.org/github.com/containerd/containerd/remotes/docker)?
Similar reasons! That ends up pulling in grpc, protobuf, and logrus.
![containerd/containerd](../../../../images/containerd.dot.svg)
> Well... what about [`containers/image`](https://godoc.org/github.com/containers/image/docker)?
That just uses the the `docker/distribution` client... and more!
![containers/image](../../../../images/containers.dot.svg)
> Wow, what about this package?
Of course, this package isn't perfect either. `transport` depends on `authn`,
which in turn depends on docker's config file parsing and handling package,
which you don't strictly need but almost certainly want if you're going to be
interacting with a registry.
![google/go-containerregistry](../../../../images/ggcr.dot.svg)
*These graphs were generated by
[`kisielk/godepgraph`](https://github.com/kisielk/godepgraph).*
## Usage
This is heavily used by the
[`remote`](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/remote)
package, which implements higher level image-centric functionality, but this
package is useful if you want to interact directly with the registry to do
something that `remote` doesn't support, e.g. [to handle with schema 1
images](https://github.com/google/go-containerregistry/pull/509).
This package also includes some [error
handling](https://github.com/opencontainers/distribution-spec/blob/60be706c34ee7805bdd1d3d11affec53b0dfb8fb/spec.md#errors)
facilities in the form of
[`CheckError`](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/remote/transport#CheckError),
which will parse the response body into a structured error for unexpected http
status codes.
Here's a "simple" program that writes the result of
[listing tags](https://github.com/opencontainers/distribution-spec/blob/60be706c34ee7805bdd1d3d11affec53b0dfb8fb/spec.md#tags)
for [`gcr.io/google-containers/pause`](https://gcr.io/google-containers/pause)
to stdout.
```go
package main
import (
"io"
"net/http"
"os"
"github.com/google/go-containerregistry/pkg/authn"
"github.com/google/go-containerregistry/pkg/name"
"github.com/google/go-containerregistry/pkg/v1/remote/transport"
)
func main() {
repo, err := name.NewRepository("gcr.io/google-containers/pause")
if err != nil {
panic(err)
}
// Fetch credentials based on your docker config file, which is $HOME/.docker/config.json or $DOCKER_CONFIG.
auth, err := authn.DefaultKeychain.Resolve(repo.Registry)
if err != nil {
panic(err)
}
// Construct an http.Client that is authorized to pull from gcr.io/google-containers/pause.
scopes := []string{repo.Scope(transport.PullScope)}
t, err := transport.New(repo.Registry, auth, http.DefaultTransport, scopes)
if err != nil {
panic(err)
}
client := &http.Client{Transport: t}
// Make the actual request.
resp, err := client.Get("https://gcr.io/v2/google-containers/pause/tags/list")
if err != nil {
panic(err)
}
// Assert that we get a 200, otherwise attempt to parse body as a structured error.
if err := transport.CheckError(resp, http.StatusOK); err != nil {
panic(err)
}
// Write the response to stdout.
if _, err := io.Copy(os.Stdout, resp.Body); err != nil {
panic(err)
}
}
```
@@ -1,62 +0,0 @@
// Copyright 2018 Google LLC All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package transport
import (
"encoding/base64"
"fmt"
"net/http"
"github.com/google/go-containerregistry/pkg/authn"
)
type basicTransport struct {
inner http.RoundTripper
auth authn.Authenticator
target string
}
var _ http.RoundTripper = (*basicTransport)(nil)
// RoundTrip implements http.RoundTripper
func (bt *basicTransport) RoundTrip(in *http.Request) (*http.Response, error) {
if bt.auth != authn.Anonymous {
auth, err := authn.Authorization(in.Context(), bt.auth)
if err != nil {
return nil, err
}
// http.Client handles redirects at a layer above the http.RoundTripper
// abstraction, so to avoid forwarding Authorization headers to places
// we are redirected, only set it when the authorization header matches
// the host with which we are interacting.
// In case of redirect http.Client can use an empty Host, check URL too.
if in.Host == bt.target || in.URL.Host == bt.target {
if bearer := auth.RegistryToken; bearer != "" {
hdr := fmt.Sprintf("Bearer %s", bearer)
in.Header.Set("Authorization", hdr)
} else if user, pass := auth.Username, auth.Password; user != "" && pass != "" {
delimited := fmt.Sprintf("%s:%s", user, pass)
encoded := base64.StdEncoding.EncodeToString([]byte(delimited))
hdr := fmt.Sprintf("Basic %s", encoded)
in.Header.Set("Authorization", hdr)
} else if token := auth.Auth; token != "" {
hdr := fmt.Sprintf("Basic %s", token)
in.Header.Set("Authorization", hdr)
}
}
}
return bt.inner.RoundTrip(in)
}

Some files were not shown because too many files have changed in this diff Show More