working commit
This commit is contained in:
+117
@@ -0,0 +1,117 @@
|
||||
# `remote`
|
||||
|
||||
[](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/remote)
|
||||
|
||||
The `remote` package implements a client for accessing a registry,
|
||||
per the [OCI distribution spec](https://github.com/opencontainers/distribution-spec/blob/master/spec.md).
|
||||
|
||||
It leans heavily on the lower level [`transport`](/pkg/v1/remote/transport) package, which handles the
|
||||
authentication handshake and structured errors.
|
||||
|
||||
## Usage
|
||||
|
||||
```go
|
||||
package main
|
||||
|
||||
import (
|
||||
"github.com/google/go-containerregistry/pkg/authn"
|
||||
"github.com/google/go-containerregistry/pkg/name"
|
||||
"github.com/google/go-containerregistry/pkg/v1/remote"
|
||||
)
|
||||
|
||||
func main() {
|
||||
ref, err := name.ParseReference("gcr.io/google-containers/pause")
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
|
||||
img, err := remote.Image(ref, remote.WithAuthFromKeychain(authn.DefaultKeychain))
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
|
||||
// do stuff with img
|
||||
}
|
||||
```
|
||||
|
||||
## Structure
|
||||
|
||||
<p align="center">
|
||||
<img src="/images/remote.dot.svg" />
|
||||
</p>
|
||||
|
||||
|
||||
## Background
|
||||
|
||||
There are a lot of confusingly similar terms that come up when talking about images in registries.
|
||||
|
||||
### Anatomy of an image
|
||||
|
||||
In general...
|
||||
|
||||
* A tag refers to an image manifest.
|
||||
* An image manifest references a config file and an orderered list of _compressed_ layers by sha256 digest.
|
||||
* A config file references an ordered list of _uncompressed_ layers by sha256 digest and contains runtime configuration.
|
||||
* The sha256 digest of the config file is the [image id](https://github.com/opencontainers/image-spec/blob/master/config.md#imageid) for the image.
|
||||
|
||||
For example, an image with two layers would look something like this:
|
||||
|
||||

|
||||
|
||||
### Anatomy of an index
|
||||
|
||||
In the normal case, an [index](https://github.com/opencontainers/image-spec/blob/master/image-index.md) is used to represent a multi-platform image.
|
||||
This was the original use case for a [manifest
|
||||
list](https://docs.docker.com/registry/spec/manifest-v2-2/#manifest-list).
|
||||
|
||||

|
||||
|
||||
It is possible for an index to reference another index, per the OCI
|
||||
[image-spec](https://github.com/opencontainers/image-spec/blob/master/media-types.md#compatibility-matrix).
|
||||
In theory, both an image and image index can reference arbitrary things via
|
||||
[descriptors](https://github.com/opencontainers/image-spec/blob/master/descriptor.md),
|
||||
e.g. see the [image layout
|
||||
example](https://github.com/opencontainers/image-spec/blob/master/image-layout.md#index-example),
|
||||
which references an application/xml file from an image index.
|
||||
|
||||
That could look something like this:
|
||||
|
||||

|
||||
|
||||
Using a recursive index like this might not be possible with all registries,
|
||||
but this flexibility allows for some interesting applications, e.g. the
|
||||
[OCI Artifacts](https://github.com/opencontainers/artifacts) effort.
|
||||
|
||||
### Anatomy of an image upload
|
||||
|
||||
The structure of an image requires a delicate ordering when uploading an image to a registry.
|
||||
Below is a (slightly simplified) figure that describes how an image is prepared for upload
|
||||
to a registry and how the data flows between various artifacts:
|
||||
|
||||

|
||||
|
||||
Note that:
|
||||
|
||||
* A config file references the uncompressed layer contents by sha256.
|
||||
* A manifest references the compressed layer contents by sha256 and the size of the layer.
|
||||
* A manifest references the config file contents by sha256 and the size of the file.
|
||||
|
||||
It follows that during an upload, we need to upload layers before the config file,
|
||||
and we need to upload the config file before the manifest.
|
||||
|
||||
Sometimes, we know all of this information ahead of time, (e.g. when copying from remote.Image),
|
||||
so the ordering is less important.
|
||||
|
||||
In other cases, e.g. when using a [`stream.Layer`](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/stream#Layer),
|
||||
we can't compute anything until we have already uploaded the layer, so we need to be careful about ordering.
|
||||
|
||||
## Caveats
|
||||
|
||||
### schema 1
|
||||
|
||||
This package does not support schema 1 images, see [`#377`](https://github.com/google/go-containerregistry/issues/377),
|
||||
however, it's possible to do _something_ useful with them via [`remote.Get`](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/remote#Get),
|
||||
which doesn't try to interpret what is returned by the registry.
|
||||
|
||||
[`crane.Copy`](https://godoc.org/github.com/google/go-containerregistry/pkg/crane#Copy) takes advantage of this to implement support for copying schema 1 images,
|
||||
see [here](https://github.com/google/go-containerregistry/blob/main/pkg/internal/legacy/copy.go).
|
||||
+159
@@ -0,0 +1,159 @@
|
||||
// Copyright 2019 Google LLC All Rights Reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package remote
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/url"
|
||||
|
||||
"github.com/google/go-containerregistry/pkg/name"
|
||||
"github.com/google/go-containerregistry/pkg/v1/remote/transport"
|
||||
)
|
||||
|
||||
type Catalogs struct {
|
||||
Repos []string `json:"repositories"`
|
||||
Next string `json:"next,omitempty"`
|
||||
}
|
||||
|
||||
// CatalogPage calls /_catalog, returning the list of repositories on the registry.
|
||||
func CatalogPage(target name.Registry, last string, n int, options ...Option) ([]string, error) {
|
||||
o, err := makeOptions(options...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
f, err := newPuller(o).fetcher(o.context, target)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
uri := url.URL{
|
||||
Scheme: target.Scheme(),
|
||||
Host: target.RegistryStr(),
|
||||
Path: "/v2/_catalog",
|
||||
RawQuery: fmt.Sprintf("last=%s&n=%d", url.QueryEscape(last), n),
|
||||
}
|
||||
|
||||
req, err := http.NewRequest(http.MethodGet, uri.String(), nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
resp, err := f.client.Do(req.WithContext(o.context))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if err := transport.CheckError(resp, http.StatusOK); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
var parsed Catalogs
|
||||
if err := json.NewDecoder(resp.Body).Decode(&parsed); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return parsed.Repos, nil
|
||||
}
|
||||
|
||||
// Catalog calls /_catalog, returning the list of repositories on the registry.
|
||||
func Catalog(ctx context.Context, target name.Registry, options ...Option) ([]string, error) {
|
||||
o, err := makeOptions(options...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// WithContext overrides the ctx passed directly.
|
||||
if o.context != context.Background() {
|
||||
ctx = o.context
|
||||
}
|
||||
|
||||
return newPuller(o).catalog(ctx, target, o.pageSize)
|
||||
}
|
||||
|
||||
func (f *fetcher) catalogPage(ctx context.Context, reg name.Registry, next string, pageSize int) (*Catalogs, error) {
|
||||
if next == "" {
|
||||
uri := &url.URL{
|
||||
Scheme: reg.Scheme(),
|
||||
Host: reg.RegistryStr(),
|
||||
Path: "/v2/_catalog",
|
||||
}
|
||||
if pageSize > 0 {
|
||||
uri.RawQuery = fmt.Sprintf("n=%d", pageSize)
|
||||
}
|
||||
next = uri.String()
|
||||
}
|
||||
|
||||
req, err := http.NewRequestWithContext(ctx, "GET", next, nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
resp, err := f.client.Do(req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if err := transport.CheckError(resp, http.StatusOK); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
parsed := Catalogs{}
|
||||
if err := json.NewDecoder(resp.Body).Decode(&parsed); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if err := resp.Body.Close(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
uri, err := getNextPageURL(resp)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if uri != nil {
|
||||
parsed.Next = uri.String()
|
||||
}
|
||||
|
||||
return &parsed, nil
|
||||
}
|
||||
|
||||
type Catalogger struct {
|
||||
f *fetcher
|
||||
reg name.Registry
|
||||
pageSize int
|
||||
|
||||
page *Catalogs
|
||||
err error
|
||||
|
||||
needMore bool
|
||||
}
|
||||
|
||||
func (l *Catalogger) Next(ctx context.Context) (*Catalogs, error) {
|
||||
if l.needMore {
|
||||
l.page, l.err = l.f.catalogPage(ctx, l.reg, l.page.Next, l.pageSize)
|
||||
} else {
|
||||
l.needMore = true
|
||||
}
|
||||
return l.page, l.err
|
||||
}
|
||||
|
||||
func (l *Catalogger) HasNext() bool {
|
||||
return l.page != nil && (!l.needMore || l.page.Next != "")
|
||||
}
|
||||
+72
@@ -0,0 +1,72 @@
|
||||
// Copyright 2019 Google LLC All Rights Reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package remote
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"net/http"
|
||||
|
||||
"github.com/google/go-containerregistry/pkg/authn"
|
||||
"github.com/google/go-containerregistry/pkg/name"
|
||||
"github.com/google/go-containerregistry/pkg/v1/remote/transport"
|
||||
)
|
||||
|
||||
// CheckPushPermission returns an error if the given keychain cannot authorize
|
||||
// a push operation to the given ref.
|
||||
//
|
||||
// This can be useful to check whether the caller has permission to push an
|
||||
// image before doing work to construct the image.
|
||||
//
|
||||
// TODO(#412): Remove the need for this method.
|
||||
func CheckPushPermission(ref name.Reference, kc authn.Keychain, t http.RoundTripper) error {
|
||||
auth, err := kc.Resolve(ref.Context().Registry)
|
||||
if err != nil {
|
||||
return fmt.Errorf("resolving authorization for %v failed: %w", ref.Context().Registry, err)
|
||||
}
|
||||
|
||||
scopes := []string{ref.Scope(transport.PushScope)}
|
||||
tr, err := transport.NewWithContext(context.TODO(), ref.Context().Registry, auth, t, scopes)
|
||||
if err != nil {
|
||||
return fmt.Errorf("creating push check transport for %v failed: %w", ref.Context().Registry, err)
|
||||
}
|
||||
// TODO(jasonhall): Against GCR, just doing the token handshake is
|
||||
// enough, but this doesn't extend to Dockerhub
|
||||
// (https://github.com/docker/hub-feedback/issues/1771), so we actually
|
||||
// need to initiate an upload to tell whether the credentials can
|
||||
// authorize a push. Figure out how to return early here when we can,
|
||||
// to avoid a roundtrip for spec-compliant registries.
|
||||
w := writer{
|
||||
repo: ref.Context(),
|
||||
client: &http.Client{Transport: tr},
|
||||
}
|
||||
loc, _, err := w.initiateUpload(context.Background(), "", "", "")
|
||||
if loc != "" {
|
||||
// Since we're only initiating the upload to check whether we
|
||||
// can, we should attempt to cancel it, in case initiating
|
||||
// reserves some resources on the server. We shouldn't wait for
|
||||
// cancelling to complete, and we don't care if it fails.
|
||||
go w.cancelUpload(loc)
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func (w *writer) cancelUpload(loc string) {
|
||||
req, err := http.NewRequest(http.MethodDelete, loc, nil)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
_, _ = w.client.Do(req)
|
||||
}
|
||||
+28
@@ -0,0 +1,28 @@
|
||||
// Copyright 2018 Google LLC All Rights Reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package remote
|
||||
|
||||
import (
|
||||
"github.com/google/go-containerregistry/pkg/name"
|
||||
)
|
||||
|
||||
// Delete removes the specified image reference from the remote registry.
|
||||
func Delete(ref name.Reference, options ...Option) error {
|
||||
o, err := makeOptions(options...)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return newPusher(o).Delete(o.context, ref)
|
||||
}
|
||||
+198
@@ -0,0 +1,198 @@
|
||||
// Copyright 2018 Google LLC All Rights Reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package remote
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
|
||||
"github.com/google/go-containerregistry/pkg/logs"
|
||||
"github.com/google/go-containerregistry/pkg/name"
|
||||
v1 "github.com/google/go-containerregistry/pkg/v1"
|
||||
"github.com/google/go-containerregistry/pkg/v1/partial"
|
||||
"github.com/google/go-containerregistry/pkg/v1/types"
|
||||
)
|
||||
|
||||
var allManifestMediaTypes = append(append([]types.MediaType{
|
||||
types.DockerManifestSchema1,
|
||||
types.DockerManifestSchema1Signed,
|
||||
}, acceptableImageMediaTypes...), acceptableIndexMediaTypes...)
|
||||
|
||||
// ErrSchema1 indicates that we received a schema1 manifest from the registry.
|
||||
// This library doesn't have plans to support this legacy image format:
|
||||
// https://github.com/google/go-containerregistry/issues/377
|
||||
var ErrSchema1 = errors.New("see https://github.com/google/go-containerregistry/issues/377")
|
||||
|
||||
// newErrSchema1 returns an ErrSchema1 with the unexpected MediaType.
|
||||
func newErrSchema1(schema types.MediaType) error {
|
||||
return fmt.Errorf("unsupported MediaType: %q, %w", schema, ErrSchema1)
|
||||
}
|
||||
|
||||
// Descriptor provides access to metadata about remote artifact and accessors
|
||||
// for efficiently converting it into a v1.Image or v1.ImageIndex.
|
||||
type Descriptor struct {
|
||||
fetcher fetcher
|
||||
v1.Descriptor
|
||||
|
||||
ref name.Reference
|
||||
Manifest []byte
|
||||
ctx context.Context
|
||||
|
||||
// So we can share this implementation with Image.
|
||||
platform v1.Platform
|
||||
}
|
||||
|
||||
func (d *Descriptor) toDesc() v1.Descriptor {
|
||||
return d.Descriptor
|
||||
}
|
||||
|
||||
// RawManifest exists to satisfy the Taggable interface.
|
||||
func (d *Descriptor) RawManifest() ([]byte, error) {
|
||||
return d.Manifest, nil
|
||||
}
|
||||
|
||||
// Get returns a remote.Descriptor for the given reference. The response from
|
||||
// the registry is left un-interpreted, for the most part. This is useful for
|
||||
// querying what kind of artifact a reference represents.
|
||||
//
|
||||
// See Head if you don't need the response body.
|
||||
func Get(ref name.Reference, options ...Option) (*Descriptor, error) {
|
||||
return get(ref, allManifestMediaTypes, options...)
|
||||
}
|
||||
|
||||
// Head returns a v1.Descriptor for the given reference by issuing a HEAD
|
||||
// request.
|
||||
//
|
||||
// Note that the server response will not have a body, so any errors encountered
|
||||
// should be retried with Get to get more details.
|
||||
func Head(ref name.Reference, options ...Option) (*v1.Descriptor, error) {
|
||||
o, err := makeOptions(options...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return newPuller(o).Head(o.context, ref)
|
||||
}
|
||||
|
||||
// Handle options and fetch the manifest with the acceptable MediaTypes in the
|
||||
// Accept header.
|
||||
func get(ref name.Reference, acceptable []types.MediaType, options ...Option) (*Descriptor, error) {
|
||||
o, err := makeOptions(options...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return newPuller(o).get(o.context, ref, acceptable, o.platform)
|
||||
}
|
||||
|
||||
// Image converts the Descriptor into a v1.Image.
|
||||
//
|
||||
// If the fetched artifact is already an image, it will just return it.
|
||||
//
|
||||
// If the fetched artifact is an index, it will attempt to resolve the index to
|
||||
// a child image with the appropriate platform.
|
||||
//
|
||||
// See WithPlatform to set the desired platform.
|
||||
func (d *Descriptor) Image() (v1.Image, error) {
|
||||
switch d.MediaType {
|
||||
case types.DockerManifestSchema1, types.DockerManifestSchema1Signed:
|
||||
// We don't care to support schema 1 images:
|
||||
// https://github.com/google/go-containerregistry/issues/377
|
||||
return nil, newErrSchema1(d.MediaType)
|
||||
case types.OCIImageIndex, types.DockerManifestList:
|
||||
// We want an image but the registry has an index, resolve it to an image.
|
||||
return d.remoteIndex().imageByPlatform(d.platform)
|
||||
case types.OCIManifestSchema1, types.DockerManifestSchema2:
|
||||
// These are expected. Enumerated here to allow a default case.
|
||||
default:
|
||||
// We could just return an error here, but some registries (e.g. static
|
||||
// registries) don't set the Content-Type headers correctly, so instead...
|
||||
logs.Warn.Printf("Unexpected media type for Image(): %s", d.MediaType)
|
||||
}
|
||||
|
||||
// Wrap the v1.Layers returned by this v1.Image in a hint for downstream
|
||||
// remote.Write calls to facilitate cross-repo "mounting".
|
||||
imgCore, err := partial.CompressedToImage(d.remoteImage())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &mountableImage{
|
||||
Image: imgCore,
|
||||
Reference: d.ref,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// Schema1 converts the Descriptor into a v1.Image for v2 schema 1 media types.
|
||||
//
|
||||
// The v1.Image returned by this method does not implement the entire interface because it would be inefficient.
|
||||
// This exists mostly to make it easier to copy schema 1 images around or look at their filesystems.
|
||||
// This is separate from Image() to avoid a backward incompatible change for callers expecting ErrSchema1.
|
||||
func (d *Descriptor) Schema1() (v1.Image, error) {
|
||||
i := &schema1{
|
||||
ref: d.ref,
|
||||
fetcher: d.fetcher,
|
||||
ctx: d.ctx,
|
||||
manifest: d.Manifest,
|
||||
mediaType: d.MediaType,
|
||||
descriptor: &d.Descriptor,
|
||||
}
|
||||
|
||||
return &mountableImage{
|
||||
Image: i,
|
||||
Reference: d.ref,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// ImageIndex converts the Descriptor into a v1.ImageIndex.
|
||||
func (d *Descriptor) ImageIndex() (v1.ImageIndex, error) {
|
||||
switch d.MediaType {
|
||||
case types.DockerManifestSchema1, types.DockerManifestSchema1Signed:
|
||||
// We don't care to support schema 1 images:
|
||||
// https://github.com/google/go-containerregistry/issues/377
|
||||
return nil, newErrSchema1(d.MediaType)
|
||||
case types.OCIManifestSchema1, types.DockerManifestSchema2:
|
||||
// We want an index but the registry has an image, nothing we can do.
|
||||
return nil, fmt.Errorf("unexpected media type for ImageIndex(): %s; call Image() instead", d.MediaType)
|
||||
case types.OCIImageIndex, types.DockerManifestList:
|
||||
// These are expected.
|
||||
default:
|
||||
// We could just return an error here, but some registries (e.g. static
|
||||
// registries) don't set the Content-Type headers correctly, so instead...
|
||||
logs.Warn.Printf("Unexpected media type for ImageIndex(): %s", d.MediaType)
|
||||
}
|
||||
return d.remoteIndex(), nil
|
||||
}
|
||||
|
||||
func (d *Descriptor) remoteImage() *remoteImage {
|
||||
return &remoteImage{
|
||||
ref: d.ref,
|
||||
ctx: d.ctx,
|
||||
fetcher: d.fetcher,
|
||||
manifest: d.Manifest,
|
||||
mediaType: d.MediaType,
|
||||
descriptor: &d.Descriptor,
|
||||
}
|
||||
}
|
||||
|
||||
func (d *Descriptor) remoteIndex() *remoteIndex {
|
||||
return &remoteIndex{
|
||||
ref: d.ref,
|
||||
ctx: d.ctx,
|
||||
fetcher: d.fetcher,
|
||||
manifest: d.Manifest,
|
||||
mediaType: d.MediaType,
|
||||
descriptor: &d.Descriptor,
|
||||
}
|
||||
}
|
||||
+17
@@ -0,0 +1,17 @@
|
||||
// Copyright 2018 Google LLC All Rights Reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
// Package remote provides facilities for reading/writing v1.Images from/to
|
||||
// a remote image registry.
|
||||
package remote
|
||||
+317
@@ -0,0 +1,317 @@
|
||||
// Copyright 2023 Google LLC All Rights Reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package remote
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strings"
|
||||
|
||||
"github.com/google/go-containerregistry/internal/redact"
|
||||
"github.com/google/go-containerregistry/internal/verify"
|
||||
"github.com/google/go-containerregistry/pkg/authn"
|
||||
"github.com/google/go-containerregistry/pkg/name"
|
||||
v1 "github.com/google/go-containerregistry/pkg/v1"
|
||||
"github.com/google/go-containerregistry/pkg/v1/remote/transport"
|
||||
"github.com/google/go-containerregistry/pkg/v1/types"
|
||||
)
|
||||
|
||||
const (
|
||||
kib = 1024
|
||||
mib = 1024 * kib
|
||||
manifestLimit = 100 * mib
|
||||
)
|
||||
|
||||
// fetcher implements methods for reading from a registry.
|
||||
type fetcher struct {
|
||||
target resource
|
||||
client *http.Client
|
||||
}
|
||||
|
||||
func makeFetcher(ctx context.Context, target resource, o *options) (*fetcher, error) {
|
||||
auth := o.auth
|
||||
if o.keychain != nil {
|
||||
kauth, err := authn.Resolve(ctx, o.keychain, target)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
auth = kauth
|
||||
}
|
||||
|
||||
reg, ok := target.(name.Registry)
|
||||
if !ok {
|
||||
repo, ok := target.(name.Repository)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("unexpected resource: %T", target)
|
||||
}
|
||||
reg = repo.Registry
|
||||
}
|
||||
|
||||
tr, err := transport.NewWithContext(ctx, reg, auth, o.transport, []string{target.Scope(transport.PullScope)})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &fetcher{
|
||||
target: target,
|
||||
client: &http.Client{Transport: tr},
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (f *fetcher) Do(req *http.Request) (*http.Response, error) {
|
||||
return f.client.Do(req)
|
||||
}
|
||||
|
||||
type resource interface {
|
||||
Scheme() string
|
||||
RegistryStr() string
|
||||
Scope(string) string
|
||||
|
||||
authn.Resource
|
||||
}
|
||||
|
||||
// url returns a url.Url for the specified path in the context of this remote image reference.
|
||||
func (f *fetcher) url(resource, identifier string) url.URL {
|
||||
u := url.URL{
|
||||
Scheme: f.target.Scheme(),
|
||||
Host: f.target.RegistryStr(),
|
||||
// Default path if this is not a repository.
|
||||
Path: "/v2/_catalog",
|
||||
}
|
||||
if repo, ok := f.target.(name.Repository); ok {
|
||||
u.Path = fmt.Sprintf("/v2/%s/%s/%s", repo.RepositoryStr(), resource, identifier)
|
||||
}
|
||||
return u
|
||||
}
|
||||
|
||||
func (f *fetcher) get(ctx context.Context, ref name.Reference, acceptable []types.MediaType, platform v1.Platform) (*Descriptor, error) {
|
||||
b, desc, err := f.fetchManifest(ctx, ref, acceptable)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &Descriptor{
|
||||
ref: ref,
|
||||
ctx: ctx,
|
||||
fetcher: *f,
|
||||
Manifest: b,
|
||||
Descriptor: *desc,
|
||||
platform: platform,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (f *fetcher) fetchManifest(ctx context.Context, ref name.Reference, acceptable []types.MediaType) ([]byte, *v1.Descriptor, error) {
|
||||
u := f.url("manifests", ref.Identifier())
|
||||
req, err := http.NewRequest(http.MethodGet, u.String(), nil)
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
accept := []string{}
|
||||
for _, mt := range acceptable {
|
||||
accept = append(accept, string(mt))
|
||||
}
|
||||
req.Header.Set("Accept", strings.Join(accept, ","))
|
||||
|
||||
resp, err := f.client.Do(req.WithContext(ctx))
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if err := transport.CheckError(resp, http.StatusOK); err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
|
||||
manifest, err := io.ReadAll(io.LimitReader(resp.Body, manifestLimit))
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
|
||||
digest, size, err := v1.SHA256(bytes.NewReader(manifest))
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
|
||||
mediaType := types.MediaType(resp.Header.Get("Content-Type"))
|
||||
contentDigest, err := v1.NewHash(resp.Header.Get("Docker-Content-Digest"))
|
||||
if err == nil && mediaType == types.DockerManifestSchema1Signed {
|
||||
// If we can parse the digest from the header, and it's a signed schema 1
|
||||
// manifest, let's use that for the digest to appease older registries.
|
||||
digest = contentDigest
|
||||
}
|
||||
|
||||
// Validate the digest matches what we asked for, if pulling by digest.
|
||||
if dgst, ok := ref.(name.Digest); ok {
|
||||
if digest.String() != dgst.DigestStr() {
|
||||
return nil, nil, fmt.Errorf("manifest digest: %q does not match requested digest: %q for %q", digest, dgst.DigestStr(), ref)
|
||||
}
|
||||
}
|
||||
|
||||
var artifactType string
|
||||
mf, _ := v1.ParseManifest(bytes.NewReader(manifest))
|
||||
// Failing to parse as a manifest should just be ignored.
|
||||
// The manifest might not be valid, and that's okay.
|
||||
if mf != nil && !mf.Config.MediaType.IsConfig() {
|
||||
artifactType = string(mf.Config.MediaType)
|
||||
}
|
||||
|
||||
// Do nothing for tags; I give up.
|
||||
//
|
||||
// We'd like to validate that the "Docker-Content-Digest" header matches what is returned by the registry,
|
||||
// but so many registries implement this incorrectly that it's not worth checking.
|
||||
//
|
||||
// For reference:
|
||||
// https://github.com/GoogleContainerTools/kaniko/issues/298
|
||||
|
||||
// Return all this info since we have to calculate it anyway.
|
||||
desc := v1.Descriptor{
|
||||
Digest: digest,
|
||||
Size: size,
|
||||
MediaType: mediaType,
|
||||
ArtifactType: artifactType,
|
||||
}
|
||||
|
||||
return manifest, &desc, nil
|
||||
}
|
||||
|
||||
func (f *fetcher) headManifest(ctx context.Context, ref name.Reference, acceptable []types.MediaType) (*v1.Descriptor, error) {
|
||||
u := f.url("manifests", ref.Identifier())
|
||||
req, err := http.NewRequest(http.MethodHead, u.String(), nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
accept := []string{}
|
||||
for _, mt := range acceptable {
|
||||
accept = append(accept, string(mt))
|
||||
}
|
||||
req.Header.Set("Accept", strings.Join(accept, ","))
|
||||
|
||||
resp, err := f.client.Do(req.WithContext(ctx))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if err := transport.CheckError(resp, http.StatusOK); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
mth := resp.Header.Get("Content-Type")
|
||||
if mth == "" {
|
||||
return nil, fmt.Errorf("HEAD %s: response did not include Content-Type header", u.String())
|
||||
}
|
||||
mediaType := types.MediaType(mth)
|
||||
|
||||
size := resp.ContentLength
|
||||
if size == -1 {
|
||||
return nil, fmt.Errorf("GET %s: response did not include Content-Length header", u.String())
|
||||
}
|
||||
|
||||
dh := resp.Header.Get("Docker-Content-Digest")
|
||||
if dh == "" {
|
||||
return nil, fmt.Errorf("HEAD %s: response did not include Docker-Content-Digest header", u.String())
|
||||
}
|
||||
digest, err := v1.NewHash(dh)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Validate the digest matches what we asked for, if pulling by digest.
|
||||
if dgst, ok := ref.(name.Digest); ok {
|
||||
if digest.String() != dgst.DigestStr() {
|
||||
return nil, fmt.Errorf("manifest digest: %q does not match requested digest: %q for %q", digest, dgst.DigestStr(), ref)
|
||||
}
|
||||
}
|
||||
|
||||
// Return all this info since we have to calculate it anyway.
|
||||
return &v1.Descriptor{
|
||||
Digest: digest,
|
||||
Size: size,
|
||||
MediaType: mediaType,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (f *fetcher) fetchBlob(ctx context.Context, size int64, h v1.Hash) (io.ReadCloser, error) {
|
||||
u := f.url("blobs", h.String())
|
||||
req, err := http.NewRequest(http.MethodGet, u.String(), nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
resp, err := f.client.Do(req.WithContext(ctx))
|
||||
if err != nil {
|
||||
return nil, redact.Error(err)
|
||||
}
|
||||
|
||||
if err := transport.CheckError(resp, http.StatusOK); err != nil {
|
||||
resp.Body.Close()
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Do whatever we can.
|
||||
// If we have an expected size and Content-Length doesn't match, return an error.
|
||||
// If we don't have an expected size and we do have a Content-Length, use Content-Length.
|
||||
if hsize := resp.ContentLength; hsize != -1 {
|
||||
if size == verify.SizeUnknown {
|
||||
size = hsize
|
||||
} else if hsize != size {
|
||||
return nil, fmt.Errorf("GET %s: Content-Length header %d does not match expected size %d", u.String(), hsize, size)
|
||||
}
|
||||
}
|
||||
|
||||
return verify.ReadCloser(resp.Body, size, h)
|
||||
}
|
||||
|
||||
func (f *fetcher) headBlob(ctx context.Context, h v1.Hash) (*http.Response, error) {
|
||||
u := f.url("blobs", h.String())
|
||||
req, err := http.NewRequest(http.MethodHead, u.String(), nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
resp, err := f.client.Do(req.WithContext(ctx))
|
||||
if err != nil {
|
||||
return nil, redact.Error(err)
|
||||
}
|
||||
|
||||
if err := transport.CheckError(resp, http.StatusOK); err != nil {
|
||||
resp.Body.Close()
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return resp, nil
|
||||
}
|
||||
|
||||
func (f *fetcher) blobExists(ctx context.Context, h v1.Hash) (bool, error) {
|
||||
u := f.url("blobs", h.String())
|
||||
req, err := http.NewRequest(http.MethodHead, u.String(), nil)
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
|
||||
resp, err := f.client.Do(req.WithContext(ctx))
|
||||
if err != nil {
|
||||
return false, redact.Error(err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if err := transport.CheckError(resp, http.StatusOK, http.StatusNotFound); err != nil {
|
||||
return false, err
|
||||
}
|
||||
|
||||
return resp.StatusCode == http.StatusOK, nil
|
||||
}
|
||||
+277
@@ -0,0 +1,277 @@
|
||||
// Copyright 2018 Google LLC All Rights Reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package remote
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"sync"
|
||||
|
||||
"github.com/google/go-containerregistry/internal/redact"
|
||||
"github.com/google/go-containerregistry/internal/verify"
|
||||
"github.com/google/go-containerregistry/pkg/name"
|
||||
v1 "github.com/google/go-containerregistry/pkg/v1"
|
||||
"github.com/google/go-containerregistry/pkg/v1/partial"
|
||||
"github.com/google/go-containerregistry/pkg/v1/remote/transport"
|
||||
"github.com/google/go-containerregistry/pkg/v1/types"
|
||||
)
|
||||
|
||||
var acceptableImageMediaTypes = []types.MediaType{
|
||||
types.DockerManifestSchema2,
|
||||
types.OCIManifestSchema1,
|
||||
}
|
||||
|
||||
// remoteImage accesses an image from a remote registry
|
||||
type remoteImage struct {
|
||||
fetcher fetcher
|
||||
ref name.Reference
|
||||
ctx context.Context
|
||||
manifestLock sync.Mutex // Protects manifest
|
||||
manifest []byte
|
||||
configLock sync.Mutex // Protects config
|
||||
config []byte
|
||||
mediaType types.MediaType
|
||||
descriptor *v1.Descriptor
|
||||
}
|
||||
|
||||
func (r *remoteImage) ArtifactType() (string, error) {
|
||||
// kind of a hack, but RawManifest does appropriate locking/memoization
|
||||
// and makes sure r.descriptor is populated.
|
||||
if _, err := r.RawManifest(); err != nil {
|
||||
return "", err
|
||||
}
|
||||
return r.descriptor.ArtifactType, nil
|
||||
}
|
||||
|
||||
var _ partial.CompressedImageCore = (*remoteImage)(nil)
|
||||
|
||||
// Image provides access to a remote image reference.
|
||||
func Image(ref name.Reference, options ...Option) (v1.Image, error) {
|
||||
desc, err := Get(ref, options...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return desc.Image()
|
||||
}
|
||||
|
||||
func (r *remoteImage) MediaType() (types.MediaType, error) {
|
||||
if string(r.mediaType) != "" {
|
||||
return r.mediaType, nil
|
||||
}
|
||||
return types.DockerManifestSchema2, nil
|
||||
}
|
||||
|
||||
func (r *remoteImage) RawManifest() ([]byte, error) {
|
||||
r.manifestLock.Lock()
|
||||
defer r.manifestLock.Unlock()
|
||||
if r.manifest != nil {
|
||||
return r.manifest, nil
|
||||
}
|
||||
|
||||
// NOTE(jonjohnsonjr): We should never get here because the public entrypoints
|
||||
// do type-checking via remote.Descriptor. I've left this here for tests that
|
||||
// directly instantiate a remoteImage.
|
||||
manifest, desc, err := r.fetcher.fetchManifest(r.ctx, r.ref, acceptableImageMediaTypes)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if r.descriptor == nil {
|
||||
r.descriptor = desc
|
||||
}
|
||||
r.mediaType = desc.MediaType
|
||||
r.manifest = manifest
|
||||
return r.manifest, nil
|
||||
}
|
||||
|
||||
func (r *remoteImage) RawConfigFile() ([]byte, error) {
|
||||
r.configLock.Lock()
|
||||
defer r.configLock.Unlock()
|
||||
if r.config != nil {
|
||||
return r.config, nil
|
||||
}
|
||||
|
||||
m, err := partial.Manifest(r)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if m.Config.Data != nil {
|
||||
if err := verify.Descriptor(m.Config); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
r.config = m.Config.Data
|
||||
return r.config, nil
|
||||
}
|
||||
|
||||
body, err := r.fetcher.fetchBlob(r.ctx, m.Config.Size, m.Config.Digest)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer body.Close()
|
||||
|
||||
r.config, err = io.ReadAll(body)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return r.config, nil
|
||||
}
|
||||
|
||||
// Descriptor retains the original descriptor from an index manifest.
|
||||
// See partial.Descriptor.
|
||||
func (r *remoteImage) Descriptor() (*v1.Descriptor, error) {
|
||||
// kind of a hack, but RawManifest does appropriate locking/memoization
|
||||
// and makes sure r.descriptor is populated.
|
||||
_, err := r.RawManifest()
|
||||
return r.descriptor, err
|
||||
}
|
||||
|
||||
func (r *remoteImage) ConfigLayer() (v1.Layer, error) {
|
||||
if _, err := r.RawManifest(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
m, err := partial.Manifest(r)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return partial.CompressedToLayer(&remoteImageLayer{
|
||||
ri: r,
|
||||
ctx: r.ctx,
|
||||
digest: m.Config.Digest,
|
||||
})
|
||||
}
|
||||
|
||||
// remoteImageLayer implements partial.CompressedLayer
|
||||
type remoteImageLayer struct {
|
||||
ri *remoteImage
|
||||
ctx context.Context
|
||||
digest v1.Hash
|
||||
}
|
||||
|
||||
// Digest implements partial.CompressedLayer
|
||||
func (rl *remoteImageLayer) Digest() (v1.Hash, error) {
|
||||
return rl.digest, nil
|
||||
}
|
||||
|
||||
// Compressed implements partial.CompressedLayer
|
||||
func (rl *remoteImageLayer) Compressed() (io.ReadCloser, error) {
|
||||
urls := []url.URL{rl.ri.fetcher.url("blobs", rl.digest.String())}
|
||||
|
||||
// Add alternative layer sources from URLs (usually none).
|
||||
d, err := partial.BlobDescriptor(rl, rl.digest)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if d.Data != nil {
|
||||
return verify.ReadCloser(io.NopCloser(bytes.NewReader(d.Data)), d.Size, d.Digest)
|
||||
}
|
||||
|
||||
// We don't want to log binary layers -- this can break terminals.
|
||||
ctx := redact.NewContext(rl.ctx, "omitting binary blobs from logs")
|
||||
|
||||
for _, s := range d.URLs {
|
||||
u, err := url.Parse(s)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
urls = append(urls, *u)
|
||||
}
|
||||
|
||||
// The lastErr for most pulls will be the same (the first error), but for
|
||||
// foreign layers we'll want to surface the last one, since we try to pull
|
||||
// from the registry first, which would often fail.
|
||||
// TODO: Maybe we don't want to try pulling from the registry first?
|
||||
var lastErr error
|
||||
for _, u := range urls {
|
||||
req, err := http.NewRequest(http.MethodGet, u.String(), nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
resp, err := rl.ri.fetcher.Do(req.WithContext(ctx))
|
||||
if err != nil {
|
||||
lastErr = err
|
||||
continue
|
||||
}
|
||||
|
||||
if err := transport.CheckError(resp, http.StatusOK); err != nil {
|
||||
resp.Body.Close()
|
||||
lastErr = err
|
||||
continue
|
||||
}
|
||||
|
||||
return verify.ReadCloser(resp.Body, d.Size, rl.digest)
|
||||
}
|
||||
|
||||
return nil, lastErr
|
||||
}
|
||||
|
||||
// Manifest implements partial.WithManifest so that we can use partial.BlobSize below.
|
||||
func (rl *remoteImageLayer) Manifest() (*v1.Manifest, error) {
|
||||
return partial.Manifest(rl.ri)
|
||||
}
|
||||
|
||||
// MediaType implements v1.Layer
|
||||
func (rl *remoteImageLayer) MediaType() (types.MediaType, error) {
|
||||
bd, err := partial.BlobDescriptor(rl, rl.digest)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
return bd.MediaType, nil
|
||||
}
|
||||
|
||||
// Size implements partial.CompressedLayer
|
||||
func (rl *remoteImageLayer) Size() (int64, error) {
|
||||
// Look up the size of this digest in the manifest to avoid a request.
|
||||
return partial.BlobSize(rl, rl.digest)
|
||||
}
|
||||
|
||||
// ConfigFile implements partial.WithManifestAndConfigFile so that we can use partial.BlobToDiffID below.
|
||||
func (rl *remoteImageLayer) ConfigFile() (*v1.ConfigFile, error) {
|
||||
return partial.ConfigFile(rl.ri)
|
||||
}
|
||||
|
||||
// DiffID implements partial.WithDiffID so that we don't recompute a DiffID that we already have
|
||||
// available in our ConfigFile.
|
||||
func (rl *remoteImageLayer) DiffID() (v1.Hash, error) {
|
||||
return partial.BlobToDiffID(rl, rl.digest)
|
||||
}
|
||||
|
||||
// Descriptor retains the original descriptor from an image manifest.
|
||||
// See partial.Descriptor.
|
||||
func (rl *remoteImageLayer) Descriptor() (*v1.Descriptor, error) {
|
||||
return partial.BlobDescriptor(rl, rl.digest)
|
||||
}
|
||||
|
||||
// See partial.Exists.
|
||||
func (rl *remoteImageLayer) Exists() (bool, error) {
|
||||
return rl.ri.fetcher.blobExists(rl.ri.ctx, rl.digest)
|
||||
}
|
||||
|
||||
// LayerByDigest implements partial.CompressedLayer
|
||||
func (r *remoteImage) LayerByDigest(h v1.Hash) (partial.CompressedLayer, error) {
|
||||
return &remoteImageLayer{
|
||||
ri: r,
|
||||
ctx: r.ctx,
|
||||
digest: h,
|
||||
}, nil
|
||||
}
|
||||
+287
@@ -0,0 +1,287 @@
|
||||
// Copyright 2018 Google LLC All Rights Reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package remote
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"fmt"
|
||||
"sync"
|
||||
|
||||
"github.com/google/go-containerregistry/internal/verify"
|
||||
"github.com/google/go-containerregistry/pkg/name"
|
||||
v1 "github.com/google/go-containerregistry/pkg/v1"
|
||||
"github.com/google/go-containerregistry/pkg/v1/partial"
|
||||
"github.com/google/go-containerregistry/pkg/v1/types"
|
||||
)
|
||||
|
||||
var acceptableIndexMediaTypes = []types.MediaType{
|
||||
types.DockerManifestList,
|
||||
types.OCIImageIndex,
|
||||
}
|
||||
|
||||
// remoteIndex accesses an index from a remote registry
|
||||
type remoteIndex struct {
|
||||
fetcher fetcher
|
||||
ref name.Reference
|
||||
ctx context.Context
|
||||
manifestLock sync.Mutex // Protects manifest
|
||||
manifest []byte
|
||||
mediaType types.MediaType
|
||||
descriptor *v1.Descriptor
|
||||
}
|
||||
|
||||
// Index provides access to a remote index reference.
|
||||
func Index(ref name.Reference, options ...Option) (v1.ImageIndex, error) {
|
||||
desc, err := get(ref, acceptableIndexMediaTypes, options...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return desc.ImageIndex()
|
||||
}
|
||||
|
||||
func (r *remoteIndex) MediaType() (types.MediaType, error) {
|
||||
if string(r.mediaType) != "" {
|
||||
return r.mediaType, nil
|
||||
}
|
||||
return types.DockerManifestList, nil
|
||||
}
|
||||
|
||||
func (r *remoteIndex) Digest() (v1.Hash, error) {
|
||||
return partial.Digest(r)
|
||||
}
|
||||
|
||||
func (r *remoteIndex) Size() (int64, error) {
|
||||
return partial.Size(r)
|
||||
}
|
||||
|
||||
func (r *remoteIndex) RawManifest() ([]byte, error) {
|
||||
r.manifestLock.Lock()
|
||||
defer r.manifestLock.Unlock()
|
||||
if r.manifest != nil {
|
||||
return r.manifest, nil
|
||||
}
|
||||
|
||||
// NOTE(jonjohnsonjr): We should never get here because the public entrypoints
|
||||
// do type-checking via remote.Descriptor. I've left this here for tests that
|
||||
// directly instantiate a remoteIndex.
|
||||
manifest, desc, err := r.fetcher.fetchManifest(r.ctx, r.ref, acceptableIndexMediaTypes)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if r.descriptor == nil {
|
||||
r.descriptor = desc
|
||||
}
|
||||
r.mediaType = desc.MediaType
|
||||
r.manifest = manifest
|
||||
return r.manifest, nil
|
||||
}
|
||||
|
||||
func (r *remoteIndex) IndexManifest() (*v1.IndexManifest, error) {
|
||||
b, err := r.RawManifest()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return v1.ParseIndexManifest(bytes.NewReader(b))
|
||||
}
|
||||
|
||||
func (r *remoteIndex) Image(h v1.Hash) (v1.Image, error) {
|
||||
desc, err := r.childByHash(h)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Descriptor.Image will handle coercing nested indexes into an Image.
|
||||
return desc.Image()
|
||||
}
|
||||
|
||||
// Descriptor retains the original descriptor from an index manifest.
|
||||
// See partial.Descriptor.
|
||||
func (r *remoteIndex) Descriptor() (*v1.Descriptor, error) {
|
||||
// kind of a hack, but RawManifest does appropriate locking/memoization
|
||||
// and makes sure r.descriptor is populated.
|
||||
_, err := r.RawManifest()
|
||||
return r.descriptor, err
|
||||
}
|
||||
|
||||
func (r *remoteIndex) ImageIndex(h v1.Hash) (v1.ImageIndex, error) {
|
||||
desc, err := r.childByHash(h)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return desc.ImageIndex()
|
||||
}
|
||||
|
||||
// Workaround for #819.
|
||||
func (r *remoteIndex) Layer(h v1.Hash) (v1.Layer, error) {
|
||||
index, err := r.IndexManifest()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for _, childDesc := range index.Manifests {
|
||||
if h == childDesc.Digest {
|
||||
l, err := partial.CompressedToLayer(&remoteLayer{
|
||||
fetcher: r.fetcher,
|
||||
ctx: r.ctx,
|
||||
digest: h,
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &MountableLayer{
|
||||
Layer: l,
|
||||
Reference: r.ref.Context().Digest(h.String()),
|
||||
}, nil
|
||||
}
|
||||
}
|
||||
return nil, fmt.Errorf("layer not found: %s", h)
|
||||
}
|
||||
|
||||
func (r *remoteIndex) imageByPlatform(platform v1.Platform) (v1.Image, error) {
|
||||
desc, err := r.childByPlatform(platform)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Descriptor.Image will handle coercing nested indexes into an Image.
|
||||
return desc.Image()
|
||||
}
|
||||
|
||||
// This naively matches the first manifest with matching platform attributes.
|
||||
//
|
||||
// We should probably use this instead:
|
||||
//
|
||||
// github.com/containerd/containerd/platforms
|
||||
//
|
||||
// But first we'd need to migrate to:
|
||||
//
|
||||
// github.com/opencontainers/image-spec/specs-go/v1
|
||||
func (r *remoteIndex) childByPlatform(platform v1.Platform) (*Descriptor, error) {
|
||||
index, err := r.IndexManifest()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for _, childDesc := range index.Manifests {
|
||||
// If platform is missing from child descriptor, assume it's amd64/linux.
|
||||
p := defaultPlatform
|
||||
if childDesc.Platform != nil {
|
||||
p = *childDesc.Platform
|
||||
}
|
||||
|
||||
if matchesPlatform(p, platform) {
|
||||
return r.childDescriptor(childDesc, platform)
|
||||
}
|
||||
}
|
||||
return nil, fmt.Errorf("no child with platform %+v in index %s", platform, r.ref)
|
||||
}
|
||||
|
||||
func (r *remoteIndex) childByHash(h v1.Hash) (*Descriptor, error) {
|
||||
index, err := r.IndexManifest()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for _, childDesc := range index.Manifests {
|
||||
if h == childDesc.Digest {
|
||||
return r.childDescriptor(childDesc, defaultPlatform)
|
||||
}
|
||||
}
|
||||
return nil, fmt.Errorf("no child with digest %s in index %s", h, r.ref)
|
||||
}
|
||||
|
||||
// Convert one of this index's child's v1.Descriptor into a remote.Descriptor, with the given platform option.
|
||||
func (r *remoteIndex) childDescriptor(child v1.Descriptor, platform v1.Platform) (*Descriptor, error) {
|
||||
ref := r.ref.Context().Digest(child.Digest.String())
|
||||
var (
|
||||
manifest []byte
|
||||
err error
|
||||
)
|
||||
if child.Data != nil {
|
||||
if err := verify.Descriptor(child); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
manifest = child.Data
|
||||
} else {
|
||||
manifest, _, err = r.fetcher.fetchManifest(r.ctx, ref, []types.MediaType{child.MediaType})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
|
||||
if child.MediaType.IsImage() {
|
||||
mf, _ := v1.ParseManifest(bytes.NewReader(manifest))
|
||||
// Failing to parse as a manifest should just be ignored.
|
||||
// The manifest might not be valid, and that's okay.
|
||||
if mf != nil && !mf.Config.MediaType.IsConfig() {
|
||||
child.ArtifactType = string(mf.Config.MediaType)
|
||||
}
|
||||
}
|
||||
|
||||
return &Descriptor{
|
||||
ref: ref,
|
||||
ctx: r.ctx,
|
||||
fetcher: r.fetcher,
|
||||
Manifest: manifest,
|
||||
Descriptor: child,
|
||||
platform: platform,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// matchesPlatform checks if the given platform matches the required platforms.
|
||||
// The given platform matches the required platform if
|
||||
// - architecture and OS are identical.
|
||||
// - OS version and variant are identical if provided.
|
||||
// - features and OS features of the required platform are subsets of those of the given platform.
|
||||
func matchesPlatform(given, required v1.Platform) bool {
|
||||
// Required fields that must be identical.
|
||||
if given.Architecture != required.Architecture || given.OS != required.OS {
|
||||
return false
|
||||
}
|
||||
|
||||
// Optional fields that may be empty, but must be identical if provided.
|
||||
if required.OSVersion != "" && given.OSVersion != required.OSVersion {
|
||||
return false
|
||||
}
|
||||
if required.Variant != "" && given.Variant != required.Variant {
|
||||
return false
|
||||
}
|
||||
|
||||
// Verify required platform's features are a subset of given platform's features.
|
||||
if !isSubset(given.OSFeatures, required.OSFeatures) {
|
||||
return false
|
||||
}
|
||||
if !isSubset(given.Features, required.Features) {
|
||||
return false
|
||||
}
|
||||
|
||||
return true
|
||||
}
|
||||
|
||||
// isSubset checks if the required array of strings is a subset of the given lst.
|
||||
func isSubset(lst, required []string) bool {
|
||||
set := make(map[string]bool)
|
||||
for _, value := range lst {
|
||||
set[value] = true
|
||||
}
|
||||
|
||||
for _, value := range required {
|
||||
if _, ok := set[value]; !ok {
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
return true
|
||||
}
|
||||
+77
@@ -0,0 +1,77 @@
|
||||
// Copyright 2019 Google LLC All Rights Reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package remote
|
||||
|
||||
import (
|
||||
"context"
|
||||
"io"
|
||||
|
||||
"github.com/google/go-containerregistry/internal/redact"
|
||||
"github.com/google/go-containerregistry/internal/verify"
|
||||
"github.com/google/go-containerregistry/pkg/name"
|
||||
v1 "github.com/google/go-containerregistry/pkg/v1"
|
||||
"github.com/google/go-containerregistry/pkg/v1/types"
|
||||
)
|
||||
|
||||
// remoteImagelayer implements partial.CompressedLayer
|
||||
type remoteLayer struct {
|
||||
ctx context.Context
|
||||
fetcher fetcher
|
||||
digest v1.Hash
|
||||
}
|
||||
|
||||
// Compressed implements partial.CompressedLayer
|
||||
func (rl *remoteLayer) Compressed() (io.ReadCloser, error) {
|
||||
// We don't want to log binary layers -- this can break terminals.
|
||||
ctx := redact.NewContext(rl.ctx, "omitting binary blobs from logs")
|
||||
return rl.fetcher.fetchBlob(ctx, verify.SizeUnknown, rl.digest)
|
||||
}
|
||||
|
||||
// Compressed implements partial.CompressedLayer
|
||||
func (rl *remoteLayer) Size() (int64, error) {
|
||||
resp, err := rl.fetcher.headBlob(rl.ctx, rl.digest)
|
||||
if err != nil {
|
||||
return -1, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
return resp.ContentLength, nil
|
||||
}
|
||||
|
||||
// Digest implements partial.CompressedLayer
|
||||
func (rl *remoteLayer) Digest() (v1.Hash, error) {
|
||||
return rl.digest, nil
|
||||
}
|
||||
|
||||
// MediaType implements v1.Layer
|
||||
func (rl *remoteLayer) MediaType() (types.MediaType, error) {
|
||||
return types.DockerLayer, nil
|
||||
}
|
||||
|
||||
// See partial.Exists.
|
||||
func (rl *remoteLayer) Exists() (bool, error) {
|
||||
return rl.fetcher.blobExists(rl.ctx, rl.digest)
|
||||
}
|
||||
|
||||
// Layer reads the given blob reference from a registry as a Layer. A blob
|
||||
// reference here is just a punned name.Digest where the digest portion is the
|
||||
// digest of the blob to be read and the repository portion is the repo where
|
||||
// that blob lives.
|
||||
func Layer(ref name.Digest, options ...Option) (v1.Layer, error) {
|
||||
o, err := makeOptions(options...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return newPuller(o).Layer(o.context, ref)
|
||||
}
|
||||
+152
@@ -0,0 +1,152 @@
|
||||
// Copyright 2018 Google LLC All Rights Reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package remote
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strings"
|
||||
|
||||
"github.com/google/go-containerregistry/pkg/name"
|
||||
"github.com/google/go-containerregistry/pkg/v1/remote/transport"
|
||||
)
|
||||
|
||||
// ListWithContext calls List with the given context.
|
||||
//
|
||||
// Deprecated: Use List and WithContext. This will be removed in a future release.
|
||||
func ListWithContext(ctx context.Context, repo name.Repository, options ...Option) ([]string, error) {
|
||||
return List(repo, append(options, WithContext(ctx))...)
|
||||
}
|
||||
|
||||
// List calls /tags/list for the given repository, returning the list of tags
|
||||
// in the "tags" property.
|
||||
func List(repo name.Repository, options ...Option) ([]string, error) {
|
||||
o, err := makeOptions(options...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return newPuller(o).List(o.context, repo)
|
||||
}
|
||||
|
||||
type Tags struct {
|
||||
Name string `json:"name"`
|
||||
Tags []string `json:"tags"`
|
||||
Next string `json:"next,omitempty"`
|
||||
}
|
||||
|
||||
func (f *fetcher) listPage(ctx context.Context, repo name.Repository, next string, pageSize int) (*Tags, error) {
|
||||
if next == "" {
|
||||
uri := &url.URL{
|
||||
Scheme: repo.Scheme(),
|
||||
Host: repo.RegistryStr(),
|
||||
Path: fmt.Sprintf("/v2/%s/tags/list", repo.RepositoryStr()),
|
||||
}
|
||||
if pageSize > 0 {
|
||||
uri.RawQuery = fmt.Sprintf("n=%d", pageSize)
|
||||
}
|
||||
next = uri.String()
|
||||
}
|
||||
|
||||
req, err := http.NewRequestWithContext(ctx, "GET", next, nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
resp, err := f.client.Do(req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if err := transport.CheckError(resp, http.StatusOK); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
parsed := Tags{}
|
||||
if err := json.NewDecoder(resp.Body).Decode(&parsed); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if err := resp.Body.Close(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
uri, err := getNextPageURL(resp)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if uri != nil {
|
||||
parsed.Next = uri.String()
|
||||
}
|
||||
|
||||
return &parsed, nil
|
||||
}
|
||||
|
||||
// getNextPageURL checks if there is a Link header in a http.Response which
|
||||
// contains a link to the next page. If yes it returns the url.URL of the next
|
||||
// page otherwise it returns nil.
|
||||
func getNextPageURL(resp *http.Response) (*url.URL, error) {
|
||||
link := resp.Header.Get("Link")
|
||||
if link == "" {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
if link[0] != '<' {
|
||||
return nil, fmt.Errorf("failed to parse link header: missing '<' in: %s", link)
|
||||
}
|
||||
|
||||
end := strings.Index(link, ">")
|
||||
if end == -1 {
|
||||
return nil, fmt.Errorf("failed to parse link header: missing '>' in: %s", link)
|
||||
}
|
||||
link = link[1:end]
|
||||
|
||||
linkURL, err := url.Parse(link)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if resp.Request == nil || resp.Request.URL == nil {
|
||||
return nil, nil
|
||||
}
|
||||
linkURL = resp.Request.URL.ResolveReference(linkURL)
|
||||
return linkURL, nil
|
||||
}
|
||||
|
||||
type Lister struct {
|
||||
f *fetcher
|
||||
repo name.Repository
|
||||
pageSize int
|
||||
|
||||
page *Tags
|
||||
err error
|
||||
|
||||
needMore bool
|
||||
}
|
||||
|
||||
func (l *Lister) Next(ctx context.Context) (*Tags, error) {
|
||||
if l.needMore {
|
||||
l.page, l.err = l.f.listPage(ctx, l.repo, l.page.Next, l.pageSize)
|
||||
} else {
|
||||
l.needMore = true
|
||||
}
|
||||
return l.page, l.err
|
||||
}
|
||||
|
||||
func (l *Lister) HasNext() bool {
|
||||
return l.page != nil && (!l.needMore || l.page.Next != "")
|
||||
}
|
||||
+108
@@ -0,0 +1,108 @@
|
||||
// Copyright 2018 Google LLC All Rights Reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package remote
|
||||
|
||||
import (
|
||||
"github.com/google/go-containerregistry/pkg/name"
|
||||
v1 "github.com/google/go-containerregistry/pkg/v1"
|
||||
"github.com/google/go-containerregistry/pkg/v1/partial"
|
||||
)
|
||||
|
||||
// MountableLayer wraps a v1.Layer in a shim that enables the layer to be
|
||||
// "mounted" when published to another registry.
|
||||
type MountableLayer struct {
|
||||
v1.Layer
|
||||
|
||||
Reference name.Reference
|
||||
}
|
||||
|
||||
// Descriptor retains the original descriptor from an image manifest.
|
||||
// See partial.Descriptor.
|
||||
func (ml *MountableLayer) Descriptor() (*v1.Descriptor, error) {
|
||||
return partial.Descriptor(ml.Layer)
|
||||
}
|
||||
|
||||
// Exists is a hack. See partial.Exists.
|
||||
func (ml *MountableLayer) Exists() (bool, error) {
|
||||
return partial.Exists(ml.Layer)
|
||||
}
|
||||
|
||||
// mountableImage wraps the v1.Layer references returned by the embedded v1.Image
|
||||
// in MountableLayer's so that remote.Write might attempt to mount them from their
|
||||
// source repository.
|
||||
type mountableImage struct {
|
||||
v1.Image
|
||||
|
||||
Reference name.Reference
|
||||
}
|
||||
|
||||
// Layers implements v1.Image
|
||||
func (mi *mountableImage) Layers() ([]v1.Layer, error) {
|
||||
ls, err := mi.Image.Layers()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
mls := make([]v1.Layer, 0, len(ls))
|
||||
for _, l := range ls {
|
||||
mls = append(mls, &MountableLayer{
|
||||
Layer: l,
|
||||
Reference: mi.Reference,
|
||||
})
|
||||
}
|
||||
return mls, nil
|
||||
}
|
||||
|
||||
// LayerByDigest implements v1.Image
|
||||
func (mi *mountableImage) LayerByDigest(d v1.Hash) (v1.Layer, error) {
|
||||
l, err := mi.Image.LayerByDigest(d)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &MountableLayer{
|
||||
Layer: l,
|
||||
Reference: mi.Reference,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// LayerByDiffID implements v1.Image
|
||||
func (mi *mountableImage) LayerByDiffID(d v1.Hash) (v1.Layer, error) {
|
||||
l, err := mi.Image.LayerByDiffID(d)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &MountableLayer{
|
||||
Layer: l,
|
||||
Reference: mi.Reference,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// Descriptor retains the original descriptor from an index manifest.
|
||||
// See partial.Descriptor.
|
||||
func (mi *mountableImage) Descriptor() (*v1.Descriptor, error) {
|
||||
return partial.Descriptor(mi.Image)
|
||||
}
|
||||
|
||||
// ConfigLayer retains the original reference so that it can be mounted.
|
||||
// See partial.ConfigLayer.
|
||||
func (mi *mountableImage) ConfigLayer() (v1.Layer, error) {
|
||||
l, err := partial.ConfigLayer(mi.Image)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &MountableLayer{
|
||||
Layer: l,
|
||||
Reference: mi.Reference,
|
||||
}, nil
|
||||
}
|
||||
+46
@@ -0,0 +1,46 @@
|
||||
// Copyright 2020 Google LLC All Rights Reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package remote
|
||||
|
||||
import (
|
||||
"github.com/google/go-containerregistry/pkg/name"
|
||||
"golang.org/x/sync/errgroup"
|
||||
)
|
||||
|
||||
// MultiWrite writes the given Images or ImageIndexes to the given refs, as
|
||||
// efficiently as possible, by deduping shared layer blobs while uploading them
|
||||
// in parallel.
|
||||
func MultiWrite(todo map[name.Reference]Taggable, options ...Option) (rerr error) {
|
||||
o, err := makeOptions(options...)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if o.progress != nil {
|
||||
defer func() { o.progress.Close(rerr) }()
|
||||
}
|
||||
p := newPusher(o)
|
||||
|
||||
g, ctx := errgroup.WithContext(o.context)
|
||||
g.SetLimit(o.jobs)
|
||||
|
||||
for ref, t := range todo {
|
||||
ref, t := ref, t
|
||||
g.Go(func() error {
|
||||
return p.Push(ctx, ref, t)
|
||||
})
|
||||
}
|
||||
|
||||
return g.Wait()
|
||||
}
|
||||
+354
@@ -0,0 +1,354 @@
|
||||
// Copyright 2018 Google LLC All Rights Reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package remote
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"io"
|
||||
"net"
|
||||
"net/http"
|
||||
"syscall"
|
||||
"time"
|
||||
|
||||
"github.com/google/go-containerregistry/internal/retry"
|
||||
"github.com/google/go-containerregistry/pkg/authn"
|
||||
"github.com/google/go-containerregistry/pkg/logs"
|
||||
v1 "github.com/google/go-containerregistry/pkg/v1"
|
||||
"github.com/google/go-containerregistry/pkg/v1/remote/transport"
|
||||
)
|
||||
|
||||
// Option is a functional option for remote operations.
|
||||
type Option func(*options) error
|
||||
|
||||
type options struct {
|
||||
auth authn.Authenticator
|
||||
keychain authn.Keychain
|
||||
transport http.RoundTripper
|
||||
context context.Context
|
||||
jobs int
|
||||
userAgent string
|
||||
allowNondistributableArtifacts bool
|
||||
progress *progress
|
||||
retryBackoff Backoff
|
||||
retryPredicate retry.Predicate
|
||||
retryStatusCodes []int
|
||||
|
||||
// Only these options can overwrite Reuse()d options.
|
||||
platform v1.Platform
|
||||
pageSize int
|
||||
filter map[string]string
|
||||
|
||||
// Set by Reuse, we currently store one or the other.
|
||||
puller *Puller
|
||||
pusher *Pusher
|
||||
}
|
||||
|
||||
var defaultPlatform = v1.Platform{
|
||||
Architecture: "amd64",
|
||||
OS: "linux",
|
||||
}
|
||||
|
||||
// Backoff is an alias of retry.Backoff to expose this configuration option to consumers of this lib
|
||||
type Backoff = retry.Backoff
|
||||
|
||||
var defaultRetryPredicate retry.Predicate = func(err error) bool {
|
||||
// Various failure modes here, as we're often reading from and writing to
|
||||
// the network.
|
||||
if retry.IsTemporary(err) || errors.Is(err, io.ErrUnexpectedEOF) || errors.Is(err, io.EOF) || errors.Is(err, syscall.EPIPE) || errors.Is(err, syscall.ECONNRESET) || errors.Is(err, net.ErrClosed) {
|
||||
logs.Warn.Printf("retrying %v", err)
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// Try this three times, waiting 1s after first failure, 3s after second.
|
||||
var defaultRetryBackoff = Backoff{
|
||||
Duration: 1.0 * time.Second,
|
||||
Factor: 3.0,
|
||||
Jitter: 0.1,
|
||||
Steps: 3,
|
||||
}
|
||||
|
||||
// Useful for tests
|
||||
var fastBackoff = Backoff{
|
||||
Duration: 1.0 * time.Millisecond,
|
||||
Factor: 3.0,
|
||||
Jitter: 0.1,
|
||||
Steps: 3,
|
||||
}
|
||||
|
||||
var defaultRetryStatusCodes = []int{
|
||||
http.StatusRequestTimeout,
|
||||
http.StatusInternalServerError,
|
||||
http.StatusBadGateway,
|
||||
http.StatusServiceUnavailable,
|
||||
http.StatusGatewayTimeout,
|
||||
499, // nginx-specific, client closed request
|
||||
522, // Cloudflare-specific, connection timeout
|
||||
}
|
||||
|
||||
const (
|
||||
defaultJobs = 4
|
||||
|
||||
// ECR returns an error if n > 1000:
|
||||
// https://github.com/google/go-containerregistry/issues/1091
|
||||
defaultPageSize = 1000
|
||||
)
|
||||
|
||||
// DefaultTransport is based on http.DefaultTransport with modifications
|
||||
// documented inline below.
|
||||
var DefaultTransport http.RoundTripper = &http.Transport{
|
||||
Proxy: http.ProxyFromEnvironment,
|
||||
DialContext: (&net.Dialer{
|
||||
Timeout: 30 * time.Second,
|
||||
KeepAlive: 30 * time.Second,
|
||||
}).DialContext,
|
||||
ForceAttemptHTTP2: true,
|
||||
MaxIdleConns: 100,
|
||||
IdleConnTimeout: 90 * time.Second,
|
||||
TLSHandshakeTimeout: 10 * time.Second,
|
||||
ExpectContinueTimeout: 1 * time.Second,
|
||||
// We usually are dealing with 2 hosts (at most), split MaxIdleConns between them.
|
||||
MaxIdleConnsPerHost: 50,
|
||||
}
|
||||
|
||||
func makeOptions(opts ...Option) (*options, error) {
|
||||
o := &options{
|
||||
transport: DefaultTransport,
|
||||
platform: defaultPlatform,
|
||||
context: context.Background(),
|
||||
jobs: defaultJobs,
|
||||
pageSize: defaultPageSize,
|
||||
retryPredicate: defaultRetryPredicate,
|
||||
retryBackoff: defaultRetryBackoff,
|
||||
retryStatusCodes: defaultRetryStatusCodes,
|
||||
}
|
||||
|
||||
for _, option := range opts {
|
||||
if err := option(o); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
|
||||
switch {
|
||||
case o.auth != nil && o.keychain != nil:
|
||||
// It is a better experience to explicitly tell a caller their auth is misconfigured
|
||||
// than potentially fail silently when the correct auth is overridden by option misuse.
|
||||
return nil, errors.New("provide an option for either authn.Authenticator or authn.Keychain, not both")
|
||||
case o.auth == nil:
|
||||
o.auth = authn.Anonymous
|
||||
}
|
||||
|
||||
// transport.Wrapper is a signal that consumers are opt-ing into providing their own transport without any additional wrapping.
|
||||
// This is to allow consumers full control over the transports logic, such as providing retry logic.
|
||||
if _, ok := o.transport.(*transport.Wrapper); !ok {
|
||||
// Wrap the transport in something that logs requests and responses.
|
||||
// It's expensive to generate the dumps, so skip it if we're writing
|
||||
// to nothing.
|
||||
if logs.Enabled(logs.Debug) {
|
||||
o.transport = transport.NewLogger(o.transport)
|
||||
}
|
||||
|
||||
// Using customized retry predicate if provided, and fallback to default if not.
|
||||
predicate := o.retryPredicate
|
||||
if predicate == nil {
|
||||
predicate = defaultRetryPredicate
|
||||
}
|
||||
|
||||
// Wrap the transport in something that can retry network flakes.
|
||||
o.transport = transport.NewRetry(o.transport, transport.WithRetryBackoff(o.retryBackoff), transport.WithRetryPredicate(predicate), transport.WithRetryStatusCodes(o.retryStatusCodes...))
|
||||
// Wrap this last to prevent transport.New from double-wrapping.
|
||||
if o.userAgent != "" {
|
||||
o.transport = transport.NewUserAgent(o.transport, o.userAgent)
|
||||
}
|
||||
}
|
||||
|
||||
return o, nil
|
||||
}
|
||||
|
||||
// WithTransport is a functional option for overriding the default transport
|
||||
// for remote operations.
|
||||
// If transport.Wrapper is provided, this signals that the consumer does *not* want any further wrapping to occur.
|
||||
// i.e. logging, retry and useragent
|
||||
//
|
||||
// The default transport is DefaultTransport.
|
||||
func WithTransport(t http.RoundTripper) Option {
|
||||
return func(o *options) error {
|
||||
o.transport = t
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// WithAuth is a functional option for overriding the default authenticator
|
||||
// for remote operations.
|
||||
// It is an error to use both WithAuth and WithAuthFromKeychain in the same Option set.
|
||||
//
|
||||
// The default authenticator is authn.Anonymous.
|
||||
func WithAuth(auth authn.Authenticator) Option {
|
||||
return func(o *options) error {
|
||||
o.auth = auth
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// WithAuthFromKeychain is a functional option for overriding the default
|
||||
// authenticator for remote operations, using an authn.Keychain to find
|
||||
// credentials.
|
||||
// It is an error to use both WithAuth and WithAuthFromKeychain in the same Option set.
|
||||
//
|
||||
// The default authenticator is authn.Anonymous.
|
||||
func WithAuthFromKeychain(keys authn.Keychain) Option {
|
||||
return func(o *options) error {
|
||||
o.keychain = keys
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// WithPlatform is a functional option for overriding the default platform
|
||||
// that Image and Descriptor.Image use for resolving an index to an image.
|
||||
//
|
||||
// The default platform is amd64/linux.
|
||||
func WithPlatform(p v1.Platform) Option {
|
||||
return func(o *options) error {
|
||||
o.platform = p
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// WithContext is a functional option for setting the context in http requests
|
||||
// performed by a given function. Note that this context is used for _all_
|
||||
// http requests, not just the initial volley. E.g., for remote.Image, the
|
||||
// context will be set on http requests generated by subsequent calls to
|
||||
// RawConfigFile() and even methods on layers returned by Layers().
|
||||
//
|
||||
// The default context is context.Background().
|
||||
func WithContext(ctx context.Context) Option {
|
||||
return func(o *options) error {
|
||||
o.context = ctx
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// WithJobs is a functional option for setting the parallelism of remote
|
||||
// operations performed by a given function. Note that not all remote
|
||||
// operations support parallelism.
|
||||
//
|
||||
// The default value is 4.
|
||||
func WithJobs(jobs int) Option {
|
||||
return func(o *options) error {
|
||||
if jobs <= 0 {
|
||||
return errors.New("jobs must be greater than zero")
|
||||
}
|
||||
o.jobs = jobs
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// WithUserAgent adds the given string to the User-Agent header for any HTTP
|
||||
// requests. This header will also include "go-containerregistry/${version}".
|
||||
//
|
||||
// If you want to completely overwrite the User-Agent header, use WithTransport.
|
||||
func WithUserAgent(ua string) Option {
|
||||
return func(o *options) error {
|
||||
o.userAgent = ua
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// WithNondistributable includes non-distributable (foreign) layers
|
||||
// when writing images, see:
|
||||
// https://github.com/opencontainers/image-spec/blob/master/layer.md#non-distributable-layers
|
||||
//
|
||||
// The default behaviour is to skip these layers
|
||||
func WithNondistributable(o *options) error {
|
||||
o.allowNondistributableArtifacts = true
|
||||
return nil
|
||||
}
|
||||
|
||||
// WithProgress takes a channel that will receive progress updates as bytes are written.
|
||||
//
|
||||
// Sending updates to an unbuffered channel will block writes, so callers
|
||||
// should provide a buffered channel to avoid potential deadlocks.
|
||||
func WithProgress(updates chan<- v1.Update) Option {
|
||||
return func(o *options) error {
|
||||
o.progress = &progress{updates: updates}
|
||||
o.progress.lastUpdate = &v1.Update{}
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// WithPageSize sets the given size as the value of parameter 'n' in the request.
|
||||
//
|
||||
// To omit the `n` parameter entirely, use WithPageSize(0).
|
||||
// The default value is 1000.
|
||||
func WithPageSize(size int) Option {
|
||||
return func(o *options) error {
|
||||
o.pageSize = size
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// WithRetryBackoff sets the httpBackoff for retry HTTP operations.
|
||||
func WithRetryBackoff(backoff Backoff) Option {
|
||||
return func(o *options) error {
|
||||
o.retryBackoff = backoff
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// WithRetryPredicate sets the predicate for retry HTTP operations.
|
||||
func WithRetryPredicate(predicate retry.Predicate) Option {
|
||||
return func(o *options) error {
|
||||
o.retryPredicate = predicate
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// WithRetryStatusCodes sets which http response codes will be retried.
|
||||
func WithRetryStatusCodes(codes ...int) Option {
|
||||
return func(o *options) error {
|
||||
o.retryStatusCodes = codes
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// WithFilter sets the filter querystring for HTTP operations.
|
||||
func WithFilter(key string, value string) Option {
|
||||
return func(o *options) error {
|
||||
if o.filter == nil {
|
||||
o.filter = map[string]string{}
|
||||
}
|
||||
o.filter[key] = value
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// Reuse takes a Puller or Pusher and reuses it for remote interactions
|
||||
// rather than starting from a clean slate. For example, it will reuse token exchanges
|
||||
// when possible and avoid sending redundant HEAD requests.
|
||||
//
|
||||
// Reuse will take precedence over other options passed to most remote functions because
|
||||
// most options deal with setting up auth and transports, which Reuse intetionally skips.
|
||||
func Reuse[I *Puller | *Pusher](i I) Option {
|
||||
return func(o *options) error {
|
||||
if puller, ok := any(i).(*Puller); ok {
|
||||
o.puller = puller
|
||||
} else if pusher, ok := any(i).(*Pusher); ok {
|
||||
o.pusher = pusher
|
||||
}
|
||||
return nil
|
||||
}
|
||||
}
|
||||
+76
@@ -0,0 +1,76 @@
|
||||
// Copyright 2022 Google LLC All Rights Reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package remote
|
||||
|
||||
import (
|
||||
"io"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
|
||||
v1 "github.com/google/go-containerregistry/pkg/v1"
|
||||
)
|
||||
|
||||
type progress struct {
|
||||
sync.Mutex
|
||||
updates chan<- v1.Update
|
||||
lastUpdate *v1.Update
|
||||
}
|
||||
|
||||
func (p *progress) total(delta int64) {
|
||||
p.Lock()
|
||||
defer p.Unlock()
|
||||
atomic.AddInt64(&p.lastUpdate.Total, delta)
|
||||
}
|
||||
|
||||
func (p *progress) complete(delta int64) {
|
||||
p.Lock()
|
||||
defer p.Unlock()
|
||||
p.updates <- v1.Update{
|
||||
Total: p.lastUpdate.Total,
|
||||
Complete: atomic.AddInt64(&p.lastUpdate.Complete, delta),
|
||||
}
|
||||
}
|
||||
|
||||
func (p *progress) err(err error) error {
|
||||
if err != nil && p.updates != nil {
|
||||
p.updates <- v1.Update{Error: err}
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func (p *progress) Close(err error) {
|
||||
_ = p.err(err)
|
||||
close(p.updates)
|
||||
}
|
||||
|
||||
type progressReader struct {
|
||||
rc io.ReadCloser
|
||||
|
||||
count *int64 // number of bytes this reader has read, to support resetting on retry.
|
||||
progress *progress
|
||||
}
|
||||
|
||||
func (r *progressReader) Read(b []byte) (int, error) {
|
||||
n, err := r.rc.Read(b)
|
||||
if err != nil {
|
||||
return n, err
|
||||
}
|
||||
atomic.AddInt64(r.count, int64(n))
|
||||
// TODO: warn/debug log if sending takes too long, or if sending is blocked while context is canceled.
|
||||
r.progress.complete(int64(n))
|
||||
return n, nil
|
||||
}
|
||||
|
||||
func (r *progressReader) Close() error { return r.rc.Close() }
|
||||
+222
@@ -0,0 +1,222 @@
|
||||
// Copyright 2023 Google LLC All Rights Reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package remote
|
||||
|
||||
import (
|
||||
"context"
|
||||
"sync"
|
||||
|
||||
"github.com/google/go-containerregistry/pkg/name"
|
||||
v1 "github.com/google/go-containerregistry/pkg/v1"
|
||||
"github.com/google/go-containerregistry/pkg/v1/partial"
|
||||
"github.com/google/go-containerregistry/pkg/v1/types"
|
||||
)
|
||||
|
||||
type Puller struct {
|
||||
o *options
|
||||
|
||||
// map[resource]*reader
|
||||
readers sync.Map
|
||||
}
|
||||
|
||||
func NewPuller(options ...Option) (*Puller, error) {
|
||||
o, err := makeOptions(options...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return newPuller(o), nil
|
||||
}
|
||||
|
||||
func newPuller(o *options) *Puller {
|
||||
if o.puller != nil {
|
||||
return o.puller
|
||||
}
|
||||
return &Puller{
|
||||
o: o,
|
||||
}
|
||||
}
|
||||
|
||||
type reader struct {
|
||||
// in
|
||||
target resource
|
||||
o *options
|
||||
|
||||
// f()
|
||||
once sync.Once
|
||||
|
||||
// out
|
||||
f *fetcher
|
||||
err error
|
||||
}
|
||||
|
||||
// this will run once per reader instance
|
||||
func (r *reader) init(ctx context.Context) error {
|
||||
r.once.Do(func() {
|
||||
r.f, r.err = makeFetcher(ctx, r.target, r.o)
|
||||
})
|
||||
return r.err
|
||||
}
|
||||
|
||||
func (p *Puller) fetcher(ctx context.Context, target resource) (*fetcher, error) {
|
||||
v, _ := p.readers.LoadOrStore(target, &reader{
|
||||
target: target,
|
||||
o: p.o,
|
||||
})
|
||||
rr := v.(*reader)
|
||||
return rr.f, rr.init(ctx)
|
||||
}
|
||||
|
||||
// Head is like remote.Head, but avoids re-authenticating when possible.
|
||||
func (p *Puller) Head(ctx context.Context, ref name.Reference) (*v1.Descriptor, error) {
|
||||
f, err := p.fetcher(ctx, ref.Context())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return f.headManifest(ctx, ref, allManifestMediaTypes)
|
||||
}
|
||||
|
||||
// Get is like remote.Get, but avoids re-authenticating when possible.
|
||||
func (p *Puller) Get(ctx context.Context, ref name.Reference) (*Descriptor, error) {
|
||||
return p.get(ctx, ref, allManifestMediaTypes, p.o.platform)
|
||||
}
|
||||
|
||||
func (p *Puller) get(ctx context.Context, ref name.Reference, acceptable []types.MediaType, platform v1.Platform) (*Descriptor, error) {
|
||||
f, err := p.fetcher(ctx, ref.Context())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return f.get(ctx, ref, acceptable, platform)
|
||||
}
|
||||
|
||||
// Layer is like remote.Layer, but avoids re-authenticating when possible.
|
||||
func (p *Puller) Layer(ctx context.Context, ref name.Digest) (v1.Layer, error) {
|
||||
f, err := p.fetcher(ctx, ref.Context())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
h, err := v1.NewHash(ref.Identifier())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
l, err := partial.CompressedToLayer(&remoteLayer{
|
||||
fetcher: *f,
|
||||
ctx: ctx,
|
||||
digest: h,
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &MountableLayer{
|
||||
Layer: l,
|
||||
Reference: ref,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// List lists tags in a repo and handles pagination, returning the full list of tags.
|
||||
func (p *Puller) List(ctx context.Context, repo name.Repository) ([]string, error) {
|
||||
lister, err := p.Lister(ctx, repo)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
tagList := []string{}
|
||||
for lister.HasNext() {
|
||||
tags, err := lister.Next(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
tagList = append(tagList, tags.Tags...)
|
||||
}
|
||||
|
||||
return tagList, nil
|
||||
}
|
||||
|
||||
// Lister lists tags in a repo and returns a Lister for paginating through the results.
|
||||
func (p *Puller) Lister(ctx context.Context, repo name.Repository) (*Lister, error) {
|
||||
return p.lister(ctx, repo, p.o.pageSize)
|
||||
}
|
||||
|
||||
func (p *Puller) lister(ctx context.Context, repo name.Repository, pageSize int) (*Lister, error) {
|
||||
f, err := p.fetcher(ctx, repo)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
page, err := f.listPage(ctx, repo, "", pageSize)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &Lister{
|
||||
f: f,
|
||||
repo: repo,
|
||||
pageSize: pageSize,
|
||||
page: page,
|
||||
err: err,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// Catalog lists repos in a registry and handles pagination, returning the full list of repos.
|
||||
func (p *Puller) Catalog(ctx context.Context, reg name.Registry) ([]string, error) {
|
||||
return p.catalog(ctx, reg, p.o.pageSize)
|
||||
}
|
||||
|
||||
func (p *Puller) catalog(ctx context.Context, reg name.Registry, pageSize int) ([]string, error) {
|
||||
catalogger, err := p.catalogger(ctx, reg, pageSize)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
repoList := []string{}
|
||||
for catalogger.HasNext() {
|
||||
repos, err := catalogger.Next(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
repoList = append(repoList, repos.Repos...)
|
||||
}
|
||||
return repoList, nil
|
||||
}
|
||||
|
||||
// Catalogger lists repos in a registry and returns a Catalogger for paginating through the results.
|
||||
func (p *Puller) Catalogger(ctx context.Context, reg name.Registry) (*Catalogger, error) {
|
||||
return p.catalogger(ctx, reg, p.o.pageSize)
|
||||
}
|
||||
|
||||
func (p *Puller) catalogger(ctx context.Context, reg name.Registry, pageSize int) (*Catalogger, error) {
|
||||
f, err := p.fetcher(ctx, reg)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
page, err := f.catalogPage(ctx, reg, "", pageSize)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &Catalogger{
|
||||
f: f,
|
||||
reg: reg,
|
||||
pageSize: pageSize,
|
||||
page: page,
|
||||
err: err,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (p *Puller) referrers(ctx context.Context, d name.Digest, filter map[string]string) (v1.ImageIndex, error) {
|
||||
f, err := p.fetcher(ctx, d.Context())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return f.fetchReferrers(ctx, filter, d)
|
||||
}
|
||||
+573
@@ -0,0 +1,573 @@
|
||||
// Copyright 2023 Google LLC All Rights Reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package remote
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"sync"
|
||||
|
||||
"github.com/google/go-containerregistry/pkg/logs"
|
||||
"github.com/google/go-containerregistry/pkg/name"
|
||||
v1 "github.com/google/go-containerregistry/pkg/v1"
|
||||
"github.com/google/go-containerregistry/pkg/v1/partial"
|
||||
"github.com/google/go-containerregistry/pkg/v1/remote/transport"
|
||||
"github.com/google/go-containerregistry/pkg/v1/stream"
|
||||
"github.com/google/go-containerregistry/pkg/v1/types"
|
||||
"golang.org/x/sync/errgroup"
|
||||
)
|
||||
|
||||
type manifest interface {
|
||||
Taggable
|
||||
partial.Describable
|
||||
}
|
||||
|
||||
// key is either v1.Hash or v1.Layer (for stream.Layer)
|
||||
type workers struct {
|
||||
// map[v1.Hash|v1.Layer]*sync.Once
|
||||
onces sync.Map
|
||||
|
||||
// map[v1.Hash|v1.Layer]error
|
||||
errors sync.Map
|
||||
}
|
||||
|
||||
func nop() error {
|
||||
return nil
|
||||
}
|
||||
|
||||
func (w *workers) err(digest v1.Hash) error {
|
||||
v, ok := w.errors.Load(digest)
|
||||
if !ok || v == nil {
|
||||
return nil
|
||||
}
|
||||
return v.(error)
|
||||
}
|
||||
|
||||
func (w *workers) Do(digest v1.Hash, f func() error) error {
|
||||
// We don't care if it was loaded or not because the sync.Once will do it for us.
|
||||
once, _ := w.onces.LoadOrStore(digest, &sync.Once{})
|
||||
|
||||
once.(*sync.Once).Do(func() {
|
||||
w.errors.Store(digest, f())
|
||||
})
|
||||
|
||||
err := w.err(digest)
|
||||
if err != nil {
|
||||
// Allow this to be retried by another caller.
|
||||
w.onces.Delete(digest)
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func (w *workers) Stream(layer v1.Layer, f func() error) error {
|
||||
// We don't care if it was loaded or not because the sync.Once will do it for us.
|
||||
once, _ := w.onces.LoadOrStore(layer, &sync.Once{})
|
||||
|
||||
once.(*sync.Once).Do(func() {
|
||||
w.errors.Store(layer, f())
|
||||
})
|
||||
|
||||
v, ok := w.errors.Load(layer)
|
||||
if !ok || v == nil {
|
||||
return nil
|
||||
}
|
||||
|
||||
return v.(error)
|
||||
}
|
||||
|
||||
type Pusher struct {
|
||||
o *options
|
||||
|
||||
// map[name.Repository]*repoWriter
|
||||
writers sync.Map
|
||||
}
|
||||
|
||||
func NewPusher(options ...Option) (*Pusher, error) {
|
||||
o, err := makeOptions(options...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return newPusher(o), nil
|
||||
}
|
||||
|
||||
func newPusher(o *options) *Pusher {
|
||||
if o.pusher != nil {
|
||||
return o.pusher
|
||||
}
|
||||
return &Pusher{
|
||||
o: o,
|
||||
}
|
||||
}
|
||||
|
||||
func (p *Pusher) writer(ctx context.Context, repo name.Repository, o *options) (*repoWriter, error) {
|
||||
v, _ := p.writers.LoadOrStore(repo, &repoWriter{
|
||||
repo: repo,
|
||||
o: o,
|
||||
})
|
||||
rw := v.(*repoWriter)
|
||||
return rw, rw.init(ctx)
|
||||
}
|
||||
|
||||
func (p *Pusher) Put(ctx context.Context, ref name.Reference, t Taggable) error {
|
||||
w, err := p.writer(ctx, ref.Context(), p.o)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
m, err := taggableToManifest(t)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
return w.commitManifest(ctx, ref, m)
|
||||
}
|
||||
|
||||
func (p *Pusher) Push(ctx context.Context, ref name.Reference, t Taggable) error {
|
||||
w, err := p.writer(ctx, ref.Context(), p.o)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return w.writeManifest(ctx, ref, t)
|
||||
}
|
||||
|
||||
func (p *Pusher) Upload(ctx context.Context, repo name.Repository, l v1.Layer) error {
|
||||
w, err := p.writer(ctx, repo, p.o)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return w.writeLayer(ctx, l)
|
||||
}
|
||||
|
||||
func (p *Pusher) Delete(ctx context.Context, ref name.Reference) error {
|
||||
w, err := p.writer(ctx, ref.Context(), p.o)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
u := url.URL{
|
||||
Scheme: ref.Context().Scheme(),
|
||||
Host: ref.Context().RegistryStr(),
|
||||
Path: fmt.Sprintf("/v2/%s/manifests/%s", ref.Context().RepositoryStr(), ref.Identifier()),
|
||||
}
|
||||
|
||||
req, err := http.NewRequest(http.MethodDelete, u.String(), nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
resp, err := w.w.client.Do(req.WithContext(ctx))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
return transport.CheckError(resp, http.StatusOK, http.StatusAccepted)
|
||||
|
||||
// TODO(jason): If the manifest had a `subject`, and if the registry
|
||||
// doesn't support Referrers, update the index pointed to by the
|
||||
// subject's fallback tag to remove the descriptor for this manifest.
|
||||
}
|
||||
|
||||
type repoWriter struct {
|
||||
repo name.Repository
|
||||
o *options
|
||||
once sync.Once
|
||||
|
||||
w *writer
|
||||
err error
|
||||
|
||||
work *workers
|
||||
}
|
||||
|
||||
// this will run once per repoWriter instance
|
||||
func (rw *repoWriter) init(ctx context.Context) error {
|
||||
rw.once.Do(func() {
|
||||
rw.work = &workers{}
|
||||
rw.w, rw.err = makeWriter(ctx, rw.repo, nil, rw.o)
|
||||
})
|
||||
return rw.err
|
||||
}
|
||||
|
||||
func (rw *repoWriter) writeDeps(ctx context.Context, m manifest) error {
|
||||
if img, ok := m.(v1.Image); ok {
|
||||
return rw.writeLayers(ctx, img)
|
||||
}
|
||||
|
||||
if idx, ok := m.(v1.ImageIndex); ok {
|
||||
return rw.writeChildren(ctx, idx)
|
||||
}
|
||||
|
||||
// This has no deps, not an error (e.g. something you want to just PUT).
|
||||
return nil
|
||||
}
|
||||
|
||||
type describable struct {
|
||||
desc v1.Descriptor
|
||||
}
|
||||
|
||||
func (d describable) Digest() (v1.Hash, error) {
|
||||
return d.desc.Digest, nil
|
||||
}
|
||||
|
||||
func (d describable) Size() (int64, error) {
|
||||
return d.desc.Size, nil
|
||||
}
|
||||
|
||||
func (d describable) MediaType() (types.MediaType, error) {
|
||||
return d.desc.MediaType, nil
|
||||
}
|
||||
|
||||
type tagManifest struct {
|
||||
Taggable
|
||||
partial.Describable
|
||||
}
|
||||
|
||||
func taggableToManifest(t Taggable) (manifest, error) {
|
||||
if m, ok := t.(manifest); ok {
|
||||
return m, nil
|
||||
}
|
||||
|
||||
if d, ok := t.(*Descriptor); ok {
|
||||
if d.MediaType.IsIndex() {
|
||||
return d.ImageIndex()
|
||||
}
|
||||
|
||||
if d.MediaType.IsImage() {
|
||||
return d.Image()
|
||||
}
|
||||
|
||||
if d.MediaType.IsSchema1() {
|
||||
return d.Schema1()
|
||||
}
|
||||
|
||||
return tagManifest{t, describable{d.toDesc()}}, nil
|
||||
}
|
||||
|
||||
desc := v1.Descriptor{
|
||||
// A reasonable default if Taggable doesn't implement MediaType.
|
||||
MediaType: types.DockerManifestSchema2,
|
||||
}
|
||||
|
||||
b, err := t.RawManifest()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if wmt, ok := t.(withMediaType); ok {
|
||||
desc.MediaType, err = wmt.MediaType()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
|
||||
desc.Digest, desc.Size, err = v1.SHA256(bytes.NewReader(b))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return tagManifest{t, describable{desc}}, nil
|
||||
}
|
||||
|
||||
func (rw *repoWriter) writeManifest(ctx context.Context, ref name.Reference, t Taggable) error {
|
||||
m, err := taggableToManifest(t)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
needDeps := true
|
||||
|
||||
digest, err := m.Digest()
|
||||
if errors.Is(err, stream.ErrNotComputed) {
|
||||
if err := rw.writeDeps(ctx, m); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
needDeps = false
|
||||
|
||||
digest, err = m.Digest()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
} else if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// This may be a lazy child where we have no ref until digest is computed.
|
||||
if ref == nil {
|
||||
ref = rw.repo.Digest(digest.String())
|
||||
}
|
||||
|
||||
// For tags, we want to do this check outside of our Work.Do closure because
|
||||
// we don't want to dedupe based on the manifest digest.
|
||||
_, byTag := ref.(name.Tag)
|
||||
if byTag {
|
||||
if exists, err := rw.manifestExists(ctx, ref, t); err != nil {
|
||||
return err
|
||||
} else if exists {
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// The following work.Do will get deduped by digest, so it won't happen unless
|
||||
// this tag happens to be the first commitManifest to run for that digest.
|
||||
needPut := byTag
|
||||
|
||||
if err := rw.work.Do(digest, func() error {
|
||||
if !byTag {
|
||||
if exists, err := rw.manifestExists(ctx, ref, t); err != nil {
|
||||
return err
|
||||
} else if exists {
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
if needDeps {
|
||||
if err := rw.writeDeps(ctx, m); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
needPut = false
|
||||
return rw.commitManifest(ctx, ref, m)
|
||||
}); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if !needPut {
|
||||
return nil
|
||||
}
|
||||
|
||||
// Only runs for tags that got deduped by digest.
|
||||
return rw.commitManifest(ctx, ref, m)
|
||||
}
|
||||
|
||||
func (rw *repoWriter) writeChildren(ctx context.Context, idx v1.ImageIndex) error {
|
||||
children, err := partial.Manifests(idx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
g, ctx := errgroup.WithContext(ctx)
|
||||
g.SetLimit(rw.o.jobs)
|
||||
|
||||
for _, child := range children {
|
||||
child := child
|
||||
if err := rw.writeChild(ctx, child, g); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
return g.Wait()
|
||||
}
|
||||
|
||||
func (rw *repoWriter) writeChild(ctx context.Context, child partial.Describable, g *errgroup.Group) error {
|
||||
switch child := child.(type) {
|
||||
case v1.ImageIndex:
|
||||
// For recursive index, we want to do a depth-first launching of goroutines
|
||||
// to avoid deadlocking.
|
||||
//
|
||||
// Note that this is rare, so the impact of this should be really small.
|
||||
return rw.writeManifest(ctx, nil, child)
|
||||
case v1.Image:
|
||||
g.Go(func() error {
|
||||
return rw.writeManifest(ctx, nil, child)
|
||||
})
|
||||
case v1.Layer:
|
||||
g.Go(func() error {
|
||||
return rw.writeLayer(ctx, child)
|
||||
})
|
||||
default:
|
||||
// This can't happen.
|
||||
return fmt.Errorf("encountered unknown child: %T", child)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// TODO: Consider caching some representation of the tags/digests in the destination
|
||||
// repository as a hint to avoid this optimistic check in cases where we will most
|
||||
// likely have to do a PUT anyway, e.g. if we are overwriting a tag we just wrote.
|
||||
func (rw *repoWriter) manifestExists(ctx context.Context, ref name.Reference, t Taggable) (bool, error) {
|
||||
f := &fetcher{
|
||||
target: ref.Context(),
|
||||
client: rw.w.client,
|
||||
}
|
||||
|
||||
m, err := taggableToManifest(t)
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
|
||||
digest, err := m.Digest()
|
||||
if err != nil {
|
||||
// Possibly due to streaming layers.
|
||||
return false, nil
|
||||
}
|
||||
got, err := f.headManifest(ctx, ref, allManifestMediaTypes)
|
||||
if err != nil {
|
||||
var terr *transport.Error
|
||||
if errors.As(err, &terr) {
|
||||
if terr.StatusCode == http.StatusNotFound {
|
||||
return false, nil
|
||||
}
|
||||
|
||||
// We treat a 403 here as non-fatal because this existence check is an optimization and
|
||||
// some registries will return a 403 instead of a 404 in certain situations.
|
||||
// E.g. https://jfrog.atlassian.net/browse/RTFACT-13797
|
||||
if terr.StatusCode == http.StatusForbidden {
|
||||
logs.Debug.Printf("manifestExists unexpected 403: %v", err)
|
||||
return false, nil
|
||||
}
|
||||
}
|
||||
|
||||
return false, err
|
||||
}
|
||||
|
||||
if digest != got.Digest {
|
||||
// Mark that we saw this digest in the registry so we don't have to check it again.
|
||||
rw.work.Do(got.Digest, nop)
|
||||
|
||||
return false, nil
|
||||
}
|
||||
|
||||
if tag, ok := ref.(name.Tag); ok {
|
||||
logs.Progress.Printf("existing manifest: %s@%s", tag.Identifier(), got.Digest)
|
||||
} else {
|
||||
logs.Progress.Print("existing manifest: ", got.Digest)
|
||||
}
|
||||
|
||||
return true, nil
|
||||
}
|
||||
|
||||
func (rw *repoWriter) commitManifest(ctx context.Context, ref name.Reference, m manifest) error {
|
||||
if rw.o.progress != nil {
|
||||
size, err := m.Size()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
rw.o.progress.total(size)
|
||||
}
|
||||
|
||||
return rw.w.commitManifest(ctx, m, ref)
|
||||
}
|
||||
|
||||
func (rw *repoWriter) writeLayers(pctx context.Context, img v1.Image) error {
|
||||
ls, err := img.Layers()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
g, ctx := errgroup.WithContext(pctx)
|
||||
g.SetLimit(rw.o.jobs)
|
||||
|
||||
for _, l := range ls {
|
||||
l := l
|
||||
|
||||
g.Go(func() error {
|
||||
return rw.writeLayer(ctx, l)
|
||||
})
|
||||
}
|
||||
|
||||
mt, err := img.MediaType()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if mt.IsSchema1() {
|
||||
return g.Wait()
|
||||
}
|
||||
|
||||
cl, err := partial.ConfigLayer(img)
|
||||
if errors.Is(err, stream.ErrNotComputed) {
|
||||
if err := g.Wait(); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
cl, err := partial.ConfigLayer(img)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
return rw.writeLayer(pctx, cl)
|
||||
} else if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
g.Go(func() error {
|
||||
return rw.writeLayer(ctx, cl)
|
||||
})
|
||||
|
||||
return g.Wait()
|
||||
}
|
||||
|
||||
func (rw *repoWriter) writeLayer(ctx context.Context, l v1.Layer) error {
|
||||
// Skip any non-distributable things.
|
||||
mt, err := l.MediaType()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !mt.IsDistributable() && !rw.o.allowNondistributableArtifacts {
|
||||
return nil
|
||||
}
|
||||
|
||||
digest, err := l.Digest()
|
||||
if err != nil {
|
||||
if errors.Is(err, stream.ErrNotComputed) {
|
||||
return rw.lazyWriteLayer(ctx, l)
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
return rw.work.Do(digest, func() error {
|
||||
if rw.o.progress != nil {
|
||||
size, err := l.Size()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
rw.o.progress.total(size)
|
||||
}
|
||||
return rw.w.uploadOne(ctx, l)
|
||||
})
|
||||
}
|
||||
|
||||
func (rw *repoWriter) lazyWriteLayer(ctx context.Context, l v1.Layer) error {
|
||||
return rw.work.Stream(l, func() error {
|
||||
if err := rw.w.uploadOne(ctx, l); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// Mark this upload completed.
|
||||
digest, err := l.Digest()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
rw.work.Do(digest, nop)
|
||||
|
||||
if rw.o.progress != nil {
|
||||
size, err := l.Size()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
rw.o.progress.total(size)
|
||||
}
|
||||
|
||||
return nil
|
||||
})
|
||||
}
|
||||
+117
@@ -0,0 +1,117 @@
|
||||
// Copyright 2023 Google LLC All Rights Reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package remote
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"errors"
|
||||
"io"
|
||||
"net/http"
|
||||
"strings"
|
||||
|
||||
"github.com/google/go-containerregistry/pkg/name"
|
||||
v1 "github.com/google/go-containerregistry/pkg/v1"
|
||||
"github.com/google/go-containerregistry/pkg/v1/empty"
|
||||
"github.com/google/go-containerregistry/pkg/v1/mutate"
|
||||
"github.com/google/go-containerregistry/pkg/v1/remote/transport"
|
||||
"github.com/google/go-containerregistry/pkg/v1/types"
|
||||
)
|
||||
|
||||
// Referrers returns a list of descriptors that refer to the given manifest digest.
|
||||
//
|
||||
// The subject manifest doesn't have to exist in the registry for there to be descriptors that refer to it.
|
||||
func Referrers(d name.Digest, options ...Option) (v1.ImageIndex, error) {
|
||||
o, err := makeOptions(options...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return newPuller(o).referrers(o.context, d, o.filter)
|
||||
}
|
||||
|
||||
// https://github.com/opencontainers/distribution-spec/blob/main/spec.md#referrers-tag-schema
|
||||
func fallbackTag(d name.Digest) name.Tag {
|
||||
return d.Context().Tag(strings.Replace(d.DigestStr(), ":", "-", 1))
|
||||
}
|
||||
|
||||
func (f *fetcher) fetchReferrers(ctx context.Context, filter map[string]string, d name.Digest) (v1.ImageIndex, error) {
|
||||
// Check the Referrers API endpoint first.
|
||||
u := f.url("referrers", d.DigestStr())
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, u.String(), nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
req.Header.Set("Accept", string(types.OCIImageIndex))
|
||||
|
||||
resp, err := f.client.Do(req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if err := transport.CheckError(resp, http.StatusOK, http.StatusNotFound, http.StatusBadRequest, http.StatusNotAcceptable); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
var b []byte
|
||||
if resp.StatusCode == http.StatusOK && resp.Header.Get("Content-Type") == string(types.OCIImageIndex) {
|
||||
b, err = io.ReadAll(resp.Body)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
} else {
|
||||
// The registry doesn't support the Referrers API endpoint, so we'll use the fallback tag scheme.
|
||||
b, _, err = f.fetchManifest(ctx, fallbackTag(d), []types.MediaType{types.OCIImageIndex})
|
||||
var terr *transport.Error
|
||||
if errors.As(err, &terr) && terr.StatusCode == http.StatusNotFound {
|
||||
// Not found just means there are no attachments yet. Start with an empty manifest.
|
||||
return empty.Index, nil
|
||||
} else if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
|
||||
h, sz, err := v1.SHA256(bytes.NewReader(b))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
idx := &remoteIndex{
|
||||
fetcher: *f,
|
||||
ctx: ctx,
|
||||
manifest: b,
|
||||
mediaType: types.OCIImageIndex,
|
||||
descriptor: &v1.Descriptor{
|
||||
Digest: h,
|
||||
MediaType: types.OCIImageIndex,
|
||||
Size: sz,
|
||||
},
|
||||
}
|
||||
return filterReferrersResponse(filter, idx), nil
|
||||
}
|
||||
|
||||
// If filter applied, filter out by artifactType.
|
||||
// See https://github.com/opencontainers/distribution-spec/blob/main/spec.md#listing-referrers
|
||||
func filterReferrersResponse(filter map[string]string, in v1.ImageIndex) v1.ImageIndex {
|
||||
if filter == nil {
|
||||
return in
|
||||
}
|
||||
v, ok := filter["artifactType"]
|
||||
if !ok {
|
||||
return in
|
||||
}
|
||||
return mutate.RemoveManifests(in, func(desc v1.Descriptor) bool {
|
||||
return desc.ArtifactType != v
|
||||
})
|
||||
}
|
||||
+118
@@ -0,0 +1,118 @@
|
||||
// Copyright 2023 Google LLC All Rights Reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package remote
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
|
||||
"github.com/google/go-containerregistry/pkg/name"
|
||||
v1 "github.com/google/go-containerregistry/pkg/v1"
|
||||
"github.com/google/go-containerregistry/pkg/v1/partial"
|
||||
"github.com/google/go-containerregistry/pkg/v1/types"
|
||||
)
|
||||
|
||||
type schema1 struct {
|
||||
ref name.Reference
|
||||
ctx context.Context
|
||||
fetcher fetcher
|
||||
manifest []byte
|
||||
mediaType types.MediaType
|
||||
descriptor *v1.Descriptor
|
||||
}
|
||||
|
||||
func (s *schema1) Layers() ([]v1.Layer, error) {
|
||||
m := schema1Manifest{}
|
||||
if err := json.NewDecoder(bytes.NewReader(s.manifest)).Decode(&m); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
layers := []v1.Layer{}
|
||||
for i := len(m.FSLayers) - 1; i >= 0; i-- {
|
||||
fsl := m.FSLayers[i]
|
||||
|
||||
h, err := v1.NewHash(fsl.BlobSum)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
l, err := s.LayerByDigest(h)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
layers = append(layers, l)
|
||||
}
|
||||
|
||||
return layers, nil
|
||||
}
|
||||
|
||||
func (s *schema1) MediaType() (types.MediaType, error) {
|
||||
return s.mediaType, nil
|
||||
}
|
||||
|
||||
func (s *schema1) Size() (int64, error) {
|
||||
return s.descriptor.Size, nil
|
||||
}
|
||||
|
||||
func (s *schema1) ConfigName() (v1.Hash, error) {
|
||||
return partial.ConfigName(s)
|
||||
}
|
||||
|
||||
func (s *schema1) ConfigFile() (*v1.ConfigFile, error) {
|
||||
return nil, newErrSchema1(s.mediaType)
|
||||
}
|
||||
|
||||
func (s *schema1) RawConfigFile() ([]byte, error) {
|
||||
return []byte("{}"), nil
|
||||
}
|
||||
|
||||
func (s *schema1) Digest() (v1.Hash, error) {
|
||||
return s.descriptor.Digest, nil
|
||||
}
|
||||
|
||||
func (s *schema1) Manifest() (*v1.Manifest, error) {
|
||||
return nil, newErrSchema1(s.mediaType)
|
||||
}
|
||||
|
||||
func (s *schema1) RawManifest() ([]byte, error) {
|
||||
return s.manifest, nil
|
||||
}
|
||||
|
||||
func (s *schema1) LayerByDigest(h v1.Hash) (v1.Layer, error) {
|
||||
l, err := partial.CompressedToLayer(&remoteLayer{
|
||||
fetcher: s.fetcher,
|
||||
ctx: s.ctx,
|
||||
digest: h,
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &MountableLayer{
|
||||
Layer: l,
|
||||
Reference: s.ref.Context().Digest(h.String()),
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (s *schema1) LayerByDiffID(v1.Hash) (v1.Layer, error) {
|
||||
return nil, newErrSchema1(s.mediaType)
|
||||
}
|
||||
|
||||
type fslayer struct {
|
||||
BlobSum string `json:"blobSum"`
|
||||
}
|
||||
|
||||
type schema1Manifest struct {
|
||||
FSLayers []fslayer `json:"fsLayers"`
|
||||
}
|
||||
+129
@@ -0,0 +1,129 @@
|
||||
# `transport`
|
||||
|
||||
[](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/transport)
|
||||
|
||||
The [distribution protocol](https://github.com/opencontainers/distribution-spec) is fairly simple, but correctly [implementing authentication](../../../authn/README.md) is **hard**.
|
||||
|
||||
This package [implements](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/remote/transport#New) an [`http.RoundTripper`](https://godoc.org/net/http#RoundTripper)
|
||||
that transparently performs:
|
||||
* [Token
|
||||
Authentication](https://docs.docker.com/registry/spec/auth/token/) and
|
||||
* [OAuth2
|
||||
Authentication](https://docs.docker.com/registry/spec/auth/oauth/)
|
||||
|
||||
for registry clients.
|
||||
|
||||
## Raison d'être
|
||||
|
||||
> Why not just use the [`docker/distribution`](https://godoc.org/github.com/docker/distribution/registry/client/auth) client?
|
||||
|
||||
Great question! Mostly, because I don't want to depend on [`prometheus/client_golang`](https://github.com/prometheus/client_golang).
|
||||
|
||||
As a performance optimization, that client uses [a cache](https://github.com/docker/distribution/blob/a8371794149d1d95f1e846744b05c87f2f825e5a/registry/client/repository.go#L173) to keep track of a mapping between blob digests and their [descriptors](https://github.com/docker/distribution/blob/a8371794149d1d95f1e846744b05c87f2f825e5a/blobs.go#L57-L86). Unfortunately, the cache [uses prometheus](https://github.com/docker/distribution/blob/a8371794149d1d95f1e846744b05c87f2f825e5a/registry/storage/cache/cachedblobdescriptorstore.go#L44) to track hits and misses, so if you want to use that client you have to pull in all of prometheus, which is pretty large.
|
||||
|
||||

|
||||
|
||||
> Why does it matter if you depend on prometheus? Who cares?
|
||||
|
||||
It's generally polite to your downstream to reduce the number of dependencies your package requires:
|
||||
|
||||
* Downloading your package is faster, which helps our Australian friends and people on airplanes.
|
||||
* There is less code to compile, which speeds up builds and saves the planet from global warming.
|
||||
* You reduce the likelihood of inflicting dependency hell upon your consumers.
|
||||
* [Tim Hockin](https://twitter.com/thockin/status/958606077456654336) prefers it based on his experience working on Kubernetes, and he's a pretty smart guy.
|
||||
|
||||
> Okay, what about [`containerd/containerd`](https://godoc.org/github.com/containerd/containerd/remotes/docker)?
|
||||
|
||||
Similar reasons! That ends up pulling in grpc, protobuf, and logrus.
|
||||
|
||||

|
||||
|
||||
> Well... what about [`containers/image`](https://godoc.org/github.com/containers/image/docker)?
|
||||
|
||||
That just uses the the `docker/distribution` client... and more!
|
||||
|
||||

|
||||
|
||||
> Wow, what about this package?
|
||||
|
||||
Of course, this package isn't perfect either. `transport` depends on `authn`,
|
||||
which in turn depends on docker's config file parsing and handling package,
|
||||
which you don't strictly need but almost certainly want if you're going to be
|
||||
interacting with a registry.
|
||||
|
||||

|
||||
|
||||
*These graphs were generated by
|
||||
[`kisielk/godepgraph`](https://github.com/kisielk/godepgraph).*
|
||||
|
||||
## Usage
|
||||
|
||||
This is heavily used by the
|
||||
[`remote`](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/remote)
|
||||
package, which implements higher level image-centric functionality, but this
|
||||
package is useful if you want to interact directly with the registry to do
|
||||
something that `remote` doesn't support, e.g. [to handle with schema 1
|
||||
images](https://github.com/google/go-containerregistry/pull/509).
|
||||
|
||||
This package also includes some [error
|
||||
handling](https://github.com/opencontainers/distribution-spec/blob/60be706c34ee7805bdd1d3d11affec53b0dfb8fb/spec.md#errors)
|
||||
facilities in the form of
|
||||
[`CheckError`](https://godoc.org/github.com/google/go-containerregistry/pkg/v1/remote/transport#CheckError),
|
||||
which will parse the response body into a structured error for unexpected http
|
||||
status codes.
|
||||
|
||||
Here's a "simple" program that writes the result of
|
||||
[listing tags](https://github.com/opencontainers/distribution-spec/blob/60be706c34ee7805bdd1d3d11affec53b0dfb8fb/spec.md#tags)
|
||||
for [`gcr.io/google-containers/pause`](https://gcr.io/google-containers/pause)
|
||||
to stdout.
|
||||
|
||||
```go
|
||||
package main
|
||||
|
||||
import (
|
||||
"io"
|
||||
"net/http"
|
||||
"os"
|
||||
|
||||
"github.com/google/go-containerregistry/pkg/authn"
|
||||
"github.com/google/go-containerregistry/pkg/name"
|
||||
"github.com/google/go-containerregistry/pkg/v1/remote/transport"
|
||||
)
|
||||
|
||||
func main() {
|
||||
repo, err := name.NewRepository("gcr.io/google-containers/pause")
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
|
||||
// Fetch credentials based on your docker config file, which is $HOME/.docker/config.json or $DOCKER_CONFIG.
|
||||
auth, err := authn.DefaultKeychain.Resolve(repo.Registry)
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
|
||||
// Construct an http.Client that is authorized to pull from gcr.io/google-containers/pause.
|
||||
scopes := []string{repo.Scope(transport.PullScope)}
|
||||
t, err := transport.New(repo.Registry, auth, http.DefaultTransport, scopes)
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
client := &http.Client{Transport: t}
|
||||
|
||||
// Make the actual request.
|
||||
resp, err := client.Get("https://gcr.io/v2/google-containers/pause/tags/list")
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
|
||||
// Assert that we get a 200, otherwise attempt to parse body as a structured error.
|
||||
if err := transport.CheckError(resp, http.StatusOK); err != nil {
|
||||
panic(err)
|
||||
}
|
||||
|
||||
// Write the response to stdout.
|
||||
if _, err := io.Copy(os.Stdout, resp.Body); err != nil {
|
||||
panic(err)
|
||||
}
|
||||
}
|
||||
```
|
||||
+62
@@ -0,0 +1,62 @@
|
||||
// Copyright 2018 Google LLC All Rights Reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package transport
|
||||
|
||||
import (
|
||||
"encoding/base64"
|
||||
"fmt"
|
||||
"net/http"
|
||||
|
||||
"github.com/google/go-containerregistry/pkg/authn"
|
||||
)
|
||||
|
||||
type basicTransport struct {
|
||||
inner http.RoundTripper
|
||||
auth authn.Authenticator
|
||||
target string
|
||||
}
|
||||
|
||||
var _ http.RoundTripper = (*basicTransport)(nil)
|
||||
|
||||
// RoundTrip implements http.RoundTripper
|
||||
func (bt *basicTransport) RoundTrip(in *http.Request) (*http.Response, error) {
|
||||
if bt.auth != authn.Anonymous {
|
||||
auth, err := authn.Authorization(in.Context(), bt.auth)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// http.Client handles redirects at a layer above the http.RoundTripper
|
||||
// abstraction, so to avoid forwarding Authorization headers to places
|
||||
// we are redirected, only set it when the authorization header matches
|
||||
// the host with which we are interacting.
|
||||
// In case of redirect http.Client can use an empty Host, check URL too.
|
||||
if in.Host == bt.target || in.URL.Host == bt.target {
|
||||
if bearer := auth.RegistryToken; bearer != "" {
|
||||
hdr := fmt.Sprintf("Bearer %s", bearer)
|
||||
in.Header.Set("Authorization", hdr)
|
||||
} else if user, pass := auth.Username, auth.Password; user != "" && pass != "" {
|
||||
delimited := fmt.Sprintf("%s:%s", user, pass)
|
||||
encoded := base64.StdEncoding.EncodeToString([]byte(delimited))
|
||||
hdr := fmt.Sprintf("Basic %s", encoded)
|
||||
in.Header.Set("Authorization", hdr)
|
||||
} else if token := auth.Auth; token != "" {
|
||||
hdr := fmt.Sprintf("Basic %s", token)
|
||||
in.Header.Set("Authorization", hdr)
|
||||
}
|
||||
}
|
||||
}
|
||||
return bt.inner.RoundTrip(in)
|
||||
}
|
||||
+407
@@ -0,0 +1,407 @@
|
||||
// Copyright 2018 Google LLC All Rights Reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package transport
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"net"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strings"
|
||||
"sync"
|
||||
|
||||
authchallenge "github.com/docker/distribution/registry/client/auth/challenge"
|
||||
|
||||
"github.com/google/go-containerregistry/internal/redact"
|
||||
"github.com/google/go-containerregistry/pkg/authn"
|
||||
"github.com/google/go-containerregistry/pkg/logs"
|
||||
"github.com/google/go-containerregistry/pkg/name"
|
||||
)
|
||||
|
||||
type Token struct {
|
||||
Token string `json:"token"`
|
||||
AccessToken string `json:"access_token,omitempty"`
|
||||
RefreshToken string `json:"refresh_token"`
|
||||
ExpiresIn int `json:"expires_in"`
|
||||
}
|
||||
|
||||
// Exchange requests a registry Token with the given scopes.
|
||||
func Exchange(ctx context.Context, reg name.Registry, auth authn.Authenticator, t http.RoundTripper, scopes []string, pr *Challenge) (*Token, error) {
|
||||
if strings.ToLower(pr.Scheme) != "bearer" {
|
||||
// TODO: Pretend token for basic?
|
||||
return nil, fmt.Errorf("challenge scheme %q is not bearer", pr.Scheme)
|
||||
}
|
||||
bt, err := fromChallenge(reg, auth, t, pr, scopes...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
authcfg, err := authn.Authorization(ctx, auth)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
tok, err := bt.Refresh(ctx, authcfg)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return tok, nil
|
||||
}
|
||||
|
||||
// FromToken returns a transport given a Challenge + Token.
|
||||
func FromToken(reg name.Registry, auth authn.Authenticator, t http.RoundTripper, pr *Challenge, tok *Token) (http.RoundTripper, error) {
|
||||
if strings.ToLower(pr.Scheme) != "bearer" {
|
||||
return &Wrapper{&basicTransport{inner: t, auth: auth, target: reg.RegistryStr()}}, nil
|
||||
}
|
||||
bt, err := fromChallenge(reg, auth, t, pr)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if tok.Token != "" {
|
||||
bt.bearer.RegistryToken = tok.Token
|
||||
}
|
||||
return &Wrapper{bt}, nil
|
||||
}
|
||||
|
||||
func fromChallenge(reg name.Registry, auth authn.Authenticator, t http.RoundTripper, pr *Challenge, scopes ...string) (*bearerTransport, error) {
|
||||
// We require the realm, which tells us where to send our Basic auth to turn it into Bearer auth.
|
||||
realm, ok := pr.Parameters["realm"]
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("malformed www-authenticate, missing realm: %v", pr.Parameters)
|
||||
}
|
||||
service := pr.Parameters["service"]
|
||||
scheme := "https"
|
||||
if pr.Insecure {
|
||||
scheme = "http"
|
||||
}
|
||||
return &bearerTransport{
|
||||
inner: t,
|
||||
basic: auth,
|
||||
realm: realm,
|
||||
registry: reg,
|
||||
service: service,
|
||||
scopes: scopes,
|
||||
scheme: scheme,
|
||||
}, nil
|
||||
}
|
||||
|
||||
type bearerTransport struct {
|
||||
mx sync.RWMutex
|
||||
// Wrapped by bearerTransport.
|
||||
inner http.RoundTripper
|
||||
// Basic credentials that we exchange for bearer tokens.
|
||||
basic authn.Authenticator
|
||||
// Holds the bearer response from the token service.
|
||||
bearer authn.AuthConfig
|
||||
// Registry to which we send bearer tokens.
|
||||
registry name.Registry
|
||||
// See https://tools.ietf.org/html/rfc6750#section-3
|
||||
realm string
|
||||
// See https://docs.docker.com/registry/spec/auth/token/
|
||||
service string
|
||||
scopes []string
|
||||
// Scheme we should use, determined by ping response.
|
||||
scheme string
|
||||
}
|
||||
|
||||
var _ http.RoundTripper = (*bearerTransport)(nil)
|
||||
|
||||
var portMap = map[string]string{
|
||||
"http": "80",
|
||||
"https": "443",
|
||||
}
|
||||
|
||||
func stringSet(ss []string) map[string]struct{} {
|
||||
set := make(map[string]struct{})
|
||||
for _, s := range ss {
|
||||
set[s] = struct{}{}
|
||||
}
|
||||
return set
|
||||
}
|
||||
|
||||
// RoundTrip implements http.RoundTripper
|
||||
func (bt *bearerTransport) RoundTrip(in *http.Request) (*http.Response, error) {
|
||||
sendRequest := func() (*http.Response, error) {
|
||||
// http.Client handles redirects at a layer above the http.RoundTripper
|
||||
// abstraction, so to avoid forwarding Authorization headers to places
|
||||
// we are redirected, only set it when the authorization header matches
|
||||
// the registry with which we are interacting.
|
||||
// In case of redirect http.Client can use an empty Host, check URL too.
|
||||
if matchesHost(bt.registry.RegistryStr(), in, bt.scheme) {
|
||||
bt.mx.RLock()
|
||||
localToken := bt.bearer.RegistryToken
|
||||
bt.mx.RUnlock()
|
||||
hdr := fmt.Sprintf("Bearer %s", localToken)
|
||||
in.Header.Set("Authorization", hdr)
|
||||
}
|
||||
return bt.inner.RoundTrip(in)
|
||||
}
|
||||
|
||||
res, err := sendRequest()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// If we hit a WWW-Authenticate challenge, it might be due to expired tokens or insufficient scope.
|
||||
if challenges := authchallenge.ResponseChallenges(res); len(challenges) != 0 {
|
||||
// close out old response, since we will not return it.
|
||||
res.Body.Close()
|
||||
|
||||
newScopes := []string{}
|
||||
bt.mx.Lock()
|
||||
got := stringSet(bt.scopes)
|
||||
for _, wac := range challenges {
|
||||
// TODO(jonjohnsonjr): Should we also update "realm" or "service"?
|
||||
if want, ok := wac.Parameters["scope"]; ok {
|
||||
// Add any scopes that we don't already request.
|
||||
if _, ok := got[want]; !ok {
|
||||
newScopes = append(newScopes, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Some registries seem to only look at the first scope parameter during a token exchange.
|
||||
// If a request fails because it's missing a scope, we should put those at the beginning,
|
||||
// otherwise the registry might just ignore it :/
|
||||
newScopes = append(newScopes, bt.scopes...)
|
||||
bt.scopes = newScopes
|
||||
bt.mx.Unlock()
|
||||
|
||||
// TODO(jonjohnsonjr): Teach transport.Error about "error" and "error_description" from challenge.
|
||||
|
||||
// Retry the request to attempt to get a valid token.
|
||||
if err = bt.refresh(in.Context()); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return sendRequest()
|
||||
}
|
||||
|
||||
return res, err
|
||||
}
|
||||
|
||||
// It's unclear which authentication flow to use based purely on the protocol,
|
||||
// so we rely on heuristics and fallbacks to support as many registries as possible.
|
||||
// The basic token exchange is attempted first, falling back to the oauth flow.
|
||||
// If the IdentityToken is set, this indicates that we should start with the oauth flow.
|
||||
func (bt *bearerTransport) refresh(ctx context.Context) error {
|
||||
auth, err := authn.Authorization(ctx, bt.basic)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if auth.RegistryToken != "" {
|
||||
bt.mx.Lock()
|
||||
bt.bearer.RegistryToken = auth.RegistryToken
|
||||
bt.mx.Unlock()
|
||||
return nil
|
||||
}
|
||||
|
||||
response, err := bt.Refresh(ctx, auth)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// Some registries set access_token instead of token. See #54.
|
||||
if response.AccessToken != "" {
|
||||
response.Token = response.AccessToken
|
||||
}
|
||||
|
||||
// Find a token to turn into a Bearer authenticator
|
||||
if response.Token != "" {
|
||||
bt.mx.Lock()
|
||||
bt.bearer.RegistryToken = response.Token
|
||||
bt.mx.Unlock()
|
||||
}
|
||||
|
||||
// If we obtained a refresh token from the oauth flow, use that for refresh() now.
|
||||
if response.RefreshToken != "" {
|
||||
bt.basic = authn.FromConfig(authn.AuthConfig{
|
||||
IdentityToken: response.RefreshToken,
|
||||
})
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func (bt *bearerTransport) Refresh(ctx context.Context, auth *authn.AuthConfig) (*Token, error) {
|
||||
var (
|
||||
content []byte
|
||||
err error
|
||||
)
|
||||
if auth.IdentityToken != "" {
|
||||
// If the secret being stored is an identity token,
|
||||
// the Username should be set to <token>, which indicates
|
||||
// we are using an oauth flow.
|
||||
content, err = bt.refreshOauth(ctx)
|
||||
var terr *Error
|
||||
if errors.As(err, &terr) && terr.StatusCode == http.StatusNotFound {
|
||||
// Note: Not all token servers implement oauth2.
|
||||
// If the request to the endpoint returns 404 using the HTTP POST method,
|
||||
// refer to Token Documentation for using the HTTP GET method supported by all token servers.
|
||||
content, err = bt.refreshBasic(ctx)
|
||||
}
|
||||
} else {
|
||||
content, err = bt.refreshBasic(ctx)
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
var response Token
|
||||
if err := json.Unmarshal(content, &response); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if response.Token == "" && response.AccessToken == "" {
|
||||
return &response, fmt.Errorf("no token in bearer response:\n%s", content)
|
||||
}
|
||||
|
||||
return &response, nil
|
||||
}
|
||||
|
||||
func matchesHost(host string, in *http.Request, scheme string) bool {
|
||||
canonicalHeaderHost := canonicalAddress(in.Host, scheme)
|
||||
canonicalURLHost := canonicalAddress(in.URL.Host, scheme)
|
||||
canonicalRegistryHost := canonicalAddress(host, scheme)
|
||||
return canonicalHeaderHost == canonicalRegistryHost || canonicalURLHost == canonicalRegistryHost
|
||||
}
|
||||
|
||||
func canonicalAddress(host, scheme string) (address string) {
|
||||
// The host may be any one of:
|
||||
// - hostname
|
||||
// - hostname:port
|
||||
// - ipv4
|
||||
// - ipv4:port
|
||||
// - ipv6
|
||||
// - [ipv6]:port
|
||||
// As net.SplitHostPort returns an error if the host does not contain a port, we should only attempt
|
||||
// to call it when we know that the address contains a port
|
||||
if strings.Count(host, ":") == 1 || (strings.Count(host, ":") >= 2 && strings.Contains(host, "]:")) {
|
||||
hostname, port, err := net.SplitHostPort(host)
|
||||
if err != nil {
|
||||
return host
|
||||
}
|
||||
if port == "" {
|
||||
port = portMap[scheme]
|
||||
}
|
||||
|
||||
return net.JoinHostPort(hostname, port)
|
||||
}
|
||||
|
||||
return net.JoinHostPort(host, portMap[scheme])
|
||||
}
|
||||
|
||||
// https://docs.docker.com/registry/spec/auth/oauth/
|
||||
func (bt *bearerTransport) refreshOauth(ctx context.Context) ([]byte, error) {
|
||||
auth, err := authn.Authorization(ctx, bt.basic)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
u, err := url.Parse(bt.realm)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
v := url.Values{}
|
||||
bt.mx.RLock()
|
||||
v.Set("scope", strings.Join(bt.scopes, " "))
|
||||
bt.mx.RUnlock()
|
||||
if bt.service != "" {
|
||||
v.Set("service", bt.service)
|
||||
}
|
||||
v.Set("client_id", defaultUserAgent)
|
||||
if auth.IdentityToken != "" {
|
||||
v.Set("grant_type", "refresh_token")
|
||||
v.Set("refresh_token", auth.IdentityToken)
|
||||
} else if auth.Username != "" && auth.Password != "" {
|
||||
// TODO(#629): This is unreachable.
|
||||
v.Set("grant_type", "password")
|
||||
v.Set("username", auth.Username)
|
||||
v.Set("password", auth.Password)
|
||||
v.Set("access_type", "offline")
|
||||
}
|
||||
|
||||
client := http.Client{Transport: bt.inner}
|
||||
req, err := http.NewRequest(http.MethodPost, u.String(), strings.NewReader(v.Encode()))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
|
||||
|
||||
// We don't want to log credentials.
|
||||
ctx = redact.NewContext(ctx, "oauth token response contains credentials")
|
||||
|
||||
resp, err := client.Do(req.WithContext(ctx))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if err := CheckError(resp, http.StatusOK); err != nil {
|
||||
if bt.basic == authn.Anonymous {
|
||||
logs.Warn.Printf("No matching credentials were found for %q", bt.registry)
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return io.ReadAll(resp.Body)
|
||||
}
|
||||
|
||||
// https://docs.docker.com/registry/spec/auth/token/
|
||||
func (bt *bearerTransport) refreshBasic(ctx context.Context) ([]byte, error) {
|
||||
u, err := url.Parse(bt.realm)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
b := &basicTransport{
|
||||
inner: bt.inner,
|
||||
auth: bt.basic,
|
||||
target: u.Host,
|
||||
}
|
||||
client := http.Client{Transport: b}
|
||||
|
||||
v := u.Query()
|
||||
bt.mx.RLock()
|
||||
v["scope"] = bt.scopes
|
||||
bt.mx.RUnlock()
|
||||
v.Set("service", bt.service)
|
||||
u.RawQuery = v.Encode()
|
||||
|
||||
req, err := http.NewRequest(http.MethodGet, u.String(), nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// We don't want to log credentials.
|
||||
ctx = redact.NewContext(ctx, "basic token response contains credentials")
|
||||
|
||||
resp, err := client.Do(req.WithContext(ctx))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if err := CheckError(resp, http.StatusOK); err != nil {
|
||||
if bt.basic == authn.Anonymous {
|
||||
logs.Warn.Printf("No matching credentials were found for %q", bt.registry)
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return io.ReadAll(resp.Body)
|
||||
}
|
||||
+18
@@ -0,0 +1,18 @@
|
||||
// Copyright 2018 Google LLC All Rights Reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
// Package transport provides facilities for setting up an authenticated
|
||||
// http.RoundTripper given an Authenticator and base RoundTripper. See
|
||||
// transport.New for more information.
|
||||
package transport
|
||||
+196
@@ -0,0 +1,196 @@
|
||||
// Copyright 2018 Google LLC All Rights Reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package transport
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"strings"
|
||||
|
||||
"github.com/google/go-containerregistry/internal/redact"
|
||||
)
|
||||
|
||||
// Error implements error to support the following error specification:
|
||||
// https://github.com/distribution/distribution/blob/aac2f6c8b7c5a6c60190848bab5cbeed2b5ba0a9/docs/spec/api.md#errors
|
||||
type Error struct {
|
||||
Errors []Diagnostic `json:"errors,omitempty"`
|
||||
// The http status code returned.
|
||||
StatusCode int
|
||||
// The request that failed.
|
||||
Request *http.Request
|
||||
// The raw body if we couldn't understand it.
|
||||
rawBody string
|
||||
|
||||
// Bit of a hack to make it easier to force a retry.
|
||||
temporary bool
|
||||
}
|
||||
|
||||
// Check that Error implements error
|
||||
var _ error = (*Error)(nil)
|
||||
|
||||
// Error implements error
|
||||
func (e *Error) Error() string {
|
||||
prefix := ""
|
||||
if e.Request != nil {
|
||||
prefix = fmt.Sprintf("%s %s: ", e.Request.Method, redact.URL(e.Request.URL))
|
||||
}
|
||||
return prefix + e.responseErr()
|
||||
}
|
||||
|
||||
func (e *Error) responseErr() string {
|
||||
switch len(e.Errors) {
|
||||
case 0:
|
||||
if len(e.rawBody) == 0 {
|
||||
if e.Request != nil && e.Request.Method == http.MethodHead {
|
||||
return fmt.Sprintf("unexpected status code %d %s (HEAD responses have no body, use GET for details)", e.StatusCode, http.StatusText(e.StatusCode))
|
||||
}
|
||||
return fmt.Sprintf("unexpected status code %d %s", e.StatusCode, http.StatusText(e.StatusCode))
|
||||
}
|
||||
return fmt.Sprintf("unexpected status code %d %s: %s", e.StatusCode, http.StatusText(e.StatusCode), e.rawBody)
|
||||
case 1:
|
||||
return e.Errors[0].String()
|
||||
default:
|
||||
var errors []string
|
||||
for _, d := range e.Errors {
|
||||
errors = append(errors, d.String())
|
||||
}
|
||||
return fmt.Sprintf("multiple errors returned: %s",
|
||||
strings.Join(errors, "; "))
|
||||
}
|
||||
}
|
||||
|
||||
// Temporary returns whether the request that preceded the error is temporary.
|
||||
func (e *Error) Temporary() bool {
|
||||
if e.temporary {
|
||||
return true
|
||||
}
|
||||
|
||||
if len(e.Errors) == 0 {
|
||||
_, ok := temporaryStatusCodes[e.StatusCode]
|
||||
return ok
|
||||
}
|
||||
for _, d := range e.Errors {
|
||||
if _, ok := temporaryErrorCodes[d.Code]; !ok {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// Diagnostic represents a single error returned by a Docker registry interaction.
|
||||
type Diagnostic struct {
|
||||
Code ErrorCode `json:"code"`
|
||||
Message string `json:"message,omitempty"`
|
||||
Detail any `json:"detail,omitempty"`
|
||||
}
|
||||
|
||||
// String stringifies the Diagnostic in the form: $Code: $Message[; $Detail]
|
||||
func (d Diagnostic) String() string {
|
||||
msg := fmt.Sprintf("%s: %s", d.Code, d.Message)
|
||||
if d.Detail != nil {
|
||||
msg = fmt.Sprintf("%s; %v", msg, d.Detail)
|
||||
}
|
||||
return msg
|
||||
}
|
||||
|
||||
// ErrorCode is an enumeration of supported error codes.
|
||||
type ErrorCode string
|
||||
|
||||
// The set of error conditions a registry may return:
|
||||
// https://github.com/distribution/distribution/blob/aac2f6c8b7c5a6c60190848bab5cbeed2b5ba0a9/docs/spec/api.md#errors-2
|
||||
const (
|
||||
BlobUnknownErrorCode ErrorCode = "BLOB_UNKNOWN"
|
||||
BlobUploadInvalidErrorCode ErrorCode = "BLOB_UPLOAD_INVALID"
|
||||
BlobUploadUnknownErrorCode ErrorCode = "BLOB_UPLOAD_UNKNOWN"
|
||||
DigestInvalidErrorCode ErrorCode = "DIGEST_INVALID"
|
||||
ManifestBlobUnknownErrorCode ErrorCode = "MANIFEST_BLOB_UNKNOWN"
|
||||
ManifestInvalidErrorCode ErrorCode = "MANIFEST_INVALID"
|
||||
ManifestUnknownErrorCode ErrorCode = "MANIFEST_UNKNOWN"
|
||||
ManifestUnverifiedErrorCode ErrorCode = "MANIFEST_UNVERIFIED"
|
||||
NameInvalidErrorCode ErrorCode = "NAME_INVALID"
|
||||
NameUnknownErrorCode ErrorCode = "NAME_UNKNOWN"
|
||||
SizeInvalidErrorCode ErrorCode = "SIZE_INVALID"
|
||||
TagInvalidErrorCode ErrorCode = "TAG_INVALID"
|
||||
UnauthorizedErrorCode ErrorCode = "UNAUTHORIZED"
|
||||
DeniedErrorCode ErrorCode = "DENIED"
|
||||
UnsupportedErrorCode ErrorCode = "UNSUPPORTED"
|
||||
TooManyRequestsErrorCode ErrorCode = "TOOMANYREQUESTS"
|
||||
UnknownErrorCode ErrorCode = "UNKNOWN"
|
||||
|
||||
// This isn't defined by either docker or OCI spec, but is defined by docker/distribution:
|
||||
// https://github.com/distribution/distribution/blob/6a977a5a754baa213041443f841705888107362a/registry/api/errcode/register.go#L60
|
||||
UnavailableErrorCode ErrorCode = "UNAVAILABLE"
|
||||
)
|
||||
|
||||
// TODO: Include other error types.
|
||||
var temporaryErrorCodes = map[ErrorCode]struct{}{
|
||||
BlobUploadInvalidErrorCode: {},
|
||||
TooManyRequestsErrorCode: {},
|
||||
UnknownErrorCode: {},
|
||||
UnavailableErrorCode: {},
|
||||
}
|
||||
|
||||
var temporaryStatusCodes = map[int]struct{}{
|
||||
http.StatusRequestTimeout: {},
|
||||
http.StatusInternalServerError: {},
|
||||
http.StatusBadGateway: {},
|
||||
http.StatusServiceUnavailable: {},
|
||||
http.StatusGatewayTimeout: {},
|
||||
}
|
||||
|
||||
// CheckError returns a structured error if the response status is not in codes.
|
||||
func CheckError(resp *http.Response, codes ...int) error {
|
||||
for _, code := range codes {
|
||||
if resp.StatusCode == code {
|
||||
// This is one of the supported status codes.
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
b, err := io.ReadAll(resp.Body)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
return makeError(resp, b)
|
||||
}
|
||||
|
||||
func makeError(resp *http.Response, body []byte) *Error {
|
||||
// https://github.com/distribution/distribution/blob/aac2f6c8b7c5a6c60190848bab5cbeed2b5ba0a9/docs/spec/api.md#errors
|
||||
structuredError := &Error{}
|
||||
|
||||
// This can fail if e.g. the response body is not valid JSON. That's fine,
|
||||
// we'll construct an appropriate error string from the body and status code.
|
||||
_ = json.Unmarshal(body, structuredError)
|
||||
|
||||
structuredError.rawBody = string(body)
|
||||
structuredError.StatusCode = resp.StatusCode
|
||||
structuredError.Request = resp.Request
|
||||
|
||||
return structuredError
|
||||
}
|
||||
|
||||
func retryError(resp *http.Response) error {
|
||||
b, err := io.ReadAll(resp.Body)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
rerr := makeError(resp, b)
|
||||
rerr.temporary = true
|
||||
return rerr
|
||||
}
|
||||
+91
@@ -0,0 +1,91 @@
|
||||
// Copyright 2020 Google LLC All Rights Reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package transport
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/http/httputil"
|
||||
"time"
|
||||
|
||||
"github.com/google/go-containerregistry/internal/redact"
|
||||
"github.com/google/go-containerregistry/pkg/logs"
|
||||
)
|
||||
|
||||
type logTransport struct {
|
||||
inner http.RoundTripper
|
||||
}
|
||||
|
||||
// NewLogger returns a transport that logs requests and responses to
|
||||
// github.com/google/go-containerregistry/pkg/logs.Debug.
|
||||
func NewLogger(inner http.RoundTripper) http.RoundTripper {
|
||||
return &logTransport{inner}
|
||||
}
|
||||
|
||||
func (t *logTransport) RoundTrip(in *http.Request) (out *http.Response, err error) {
|
||||
// Inspired by: github.com/motemen/go-loghttp
|
||||
|
||||
// We redact token responses and binary blobs in response/request.
|
||||
omitBody, reason := redact.FromContext(in.Context())
|
||||
if omitBody {
|
||||
logs.Debug.Printf("--> %s %s [body redacted: %s]", in.Method, in.URL, reason)
|
||||
} else {
|
||||
logs.Debug.Printf("--> %s %s", in.Method, in.URL)
|
||||
}
|
||||
|
||||
// Save these headers so we can redact Authorization.
|
||||
savedHeaders := in.Header.Clone()
|
||||
if in.Header != nil && in.Header.Get("authorization") != "" {
|
||||
in.Header.Set("authorization", "<redacted>")
|
||||
}
|
||||
|
||||
b, err := httputil.DumpRequestOut(in, !omitBody)
|
||||
if err == nil {
|
||||
logs.Debug.Println(string(b))
|
||||
} else {
|
||||
logs.Debug.Printf("Failed to dump request %s %s: %v", in.Method, in.URL, err)
|
||||
}
|
||||
|
||||
// Restore the non-redacted headers.
|
||||
in.Header = savedHeaders
|
||||
|
||||
start := time.Now()
|
||||
out, err = t.inner.RoundTrip(in)
|
||||
duration := time.Since(start)
|
||||
if err != nil {
|
||||
logs.Debug.Printf("<-- %v %s %s (%s)", err, in.Method, in.URL, duration)
|
||||
}
|
||||
if out != nil {
|
||||
msg := fmt.Sprintf("<-- %d", out.StatusCode)
|
||||
if out.Request != nil {
|
||||
msg = fmt.Sprintf("%s %s", msg, out.Request.URL)
|
||||
}
|
||||
msg = fmt.Sprintf("%s (%s)", msg, duration)
|
||||
|
||||
if omitBody {
|
||||
msg = fmt.Sprintf("%s [body redacted: %s]", msg, reason)
|
||||
}
|
||||
|
||||
logs.Debug.Print(msg)
|
||||
|
||||
b, err := httputil.DumpResponse(out, !omitBody)
|
||||
if err == nil {
|
||||
logs.Debug.Println(string(b))
|
||||
} else {
|
||||
logs.Debug.Printf("Failed to dump response %s %s: %v", in.Method, in.URL, err)
|
||||
}
|
||||
}
|
||||
return
|
||||
}
|
||||
+217
@@ -0,0 +1,217 @@
|
||||
// Copyright 2018 Google LLC All Rights Reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package transport
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
authchallenge "github.com/docker/distribution/registry/client/auth/challenge"
|
||||
"github.com/google/go-containerregistry/pkg/logs"
|
||||
"github.com/google/go-containerregistry/pkg/name"
|
||||
)
|
||||
|
||||
// 300ms is the default fallback period for go's DNS dialer but we could make this configurable.
|
||||
var fallbackDelay = 300 * time.Millisecond
|
||||
|
||||
type Challenge struct {
|
||||
Scheme string
|
||||
|
||||
// Following the challenge there are often key/value pairs
|
||||
// e.g. Bearer service="gcr.io",realm="https://auth.gcr.io/v36/tokenz"
|
||||
Parameters map[string]string
|
||||
|
||||
// Whether we had to use http to complete the Ping.
|
||||
Insecure bool
|
||||
}
|
||||
|
||||
// Ping does a GET /v2/ against the registry and returns the response.
|
||||
func Ping(ctx context.Context, reg name.Registry, t http.RoundTripper) (*Challenge, error) {
|
||||
// This first attempts to use "https" for every request, falling back to http
|
||||
// if the registry matches our localhost heuristic or if it is intentionally
|
||||
// set to insecure via name.NewInsecureRegistry.
|
||||
schemes := []string{"https"}
|
||||
if reg.Scheme() == "http" {
|
||||
schemes = append(schemes, "http")
|
||||
}
|
||||
if len(schemes) == 1 {
|
||||
return pingSingle(ctx, reg, t, schemes[0])
|
||||
}
|
||||
return pingParallel(ctx, reg, t, schemes)
|
||||
}
|
||||
|
||||
func pingSingle(ctx context.Context, reg name.Registry, t http.RoundTripper, scheme string) (*Challenge, error) {
|
||||
client := http.Client{Transport: t}
|
||||
url := fmt.Sprintf("%s://%s/v2/", scheme, reg.RegistryStr())
|
||||
req, err := http.NewRequest(http.MethodGet, url, nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
resp, err := client.Do(req.WithContext(ctx))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer func() {
|
||||
// By draining the body, make sure to reuse the connection made by
|
||||
// the ping for the following access to the registry
|
||||
io.Copy(io.Discard, resp.Body)
|
||||
resp.Body.Close()
|
||||
}()
|
||||
|
||||
insecure := scheme == "http"
|
||||
|
||||
switch resp.StatusCode {
|
||||
case http.StatusOK:
|
||||
// If we get a 200, then no authentication is needed.
|
||||
return &Challenge{
|
||||
Insecure: insecure,
|
||||
}, nil
|
||||
case http.StatusUnauthorized:
|
||||
if challenges := authchallenge.ResponseChallenges(resp); len(challenges) != 0 {
|
||||
// If we hit more than one, let's try to find one that we know how to handle.
|
||||
wac := pickFromMultipleChallenges(challenges)
|
||||
return &Challenge{
|
||||
Scheme: wac.Scheme,
|
||||
Parameters: wac.Parameters,
|
||||
Insecure: insecure,
|
||||
}, nil
|
||||
}
|
||||
// Otherwise, just return the challenge without parameters.
|
||||
return &Challenge{
|
||||
Scheme: resp.Header.Get("WWW-Authenticate"),
|
||||
Insecure: insecure,
|
||||
}, nil
|
||||
default:
|
||||
return nil, CheckError(resp, http.StatusOK, http.StatusUnauthorized)
|
||||
}
|
||||
}
|
||||
|
||||
// Based on the golang happy eyeballs dialParallel impl in net/dial.go.
|
||||
func pingParallel(ctx context.Context, reg name.Registry, t http.RoundTripper, schemes []string) (*Challenge, error) {
|
||||
returned := make(chan struct{})
|
||||
defer close(returned)
|
||||
|
||||
type pingResult struct {
|
||||
*Challenge
|
||||
error
|
||||
primary bool
|
||||
done bool
|
||||
}
|
||||
|
||||
results := make(chan pingResult)
|
||||
|
||||
startRacer := func(ctx context.Context, scheme string) {
|
||||
pr, err := pingSingle(ctx, reg, t, scheme)
|
||||
select {
|
||||
case results <- pingResult{Challenge: pr, error: err, primary: scheme == "https", done: true}:
|
||||
case <-returned:
|
||||
if pr != nil {
|
||||
logs.Debug.Printf("%s lost race", scheme)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
var primary, fallback pingResult
|
||||
|
||||
primaryCtx, primaryCancel := context.WithCancel(ctx)
|
||||
defer primaryCancel()
|
||||
go startRacer(primaryCtx, schemes[0])
|
||||
|
||||
fallbackTimer := time.NewTimer(fallbackDelay)
|
||||
defer fallbackTimer.Stop()
|
||||
|
||||
for {
|
||||
select {
|
||||
case <-fallbackTimer.C:
|
||||
fallbackCtx, fallbackCancel := context.WithCancel(ctx)
|
||||
defer fallbackCancel()
|
||||
go startRacer(fallbackCtx, schemes[1])
|
||||
|
||||
case res := <-results:
|
||||
if res.error == nil {
|
||||
return res.Challenge, nil
|
||||
}
|
||||
if res.primary {
|
||||
primary = res
|
||||
} else {
|
||||
fallback = res
|
||||
}
|
||||
if primary.done && fallback.done {
|
||||
return nil, multierrs{primary.error, fallback.error}
|
||||
}
|
||||
if res.primary && fallbackTimer.Stop() {
|
||||
// Primary failed and we haven't started the fallback,
|
||||
// reset time to start fallback immediately.
|
||||
fallbackTimer.Reset(0)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func pickFromMultipleChallenges(challenges []authchallenge.Challenge) authchallenge.Challenge {
|
||||
// It might happen there are multiple www-authenticate headers, e.g. `Negotiate` and `Basic`.
|
||||
// Picking simply the first one could result eventually in `unrecognized challenge` error,
|
||||
// that's why we're looping through the challenges in search for one that can be handled.
|
||||
allowedSchemes := []string{"basic", "bearer"}
|
||||
|
||||
for _, wac := range challenges {
|
||||
currentScheme := strings.ToLower(wac.Scheme)
|
||||
for _, allowed := range allowedSchemes {
|
||||
if allowed == currentScheme {
|
||||
return wac
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return challenges[0]
|
||||
}
|
||||
|
||||
type multierrs []error
|
||||
|
||||
func (m multierrs) Error() string {
|
||||
var b strings.Builder
|
||||
hasWritten := false
|
||||
for _, err := range m {
|
||||
if hasWritten {
|
||||
b.WriteString("; ")
|
||||
}
|
||||
hasWritten = true
|
||||
b.WriteString(err.Error())
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
|
||||
func (m multierrs) As(target any) bool {
|
||||
for _, err := range m {
|
||||
if errors.As(err, target) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func (m multierrs) Is(target error) bool {
|
||||
for _, err := range m {
|
||||
if errors.Is(err, target) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
+111
@@ -0,0 +1,111 @@
|
||||
// Copyright 2018 Google LLC All Rights Reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package transport
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"time"
|
||||
|
||||
"github.com/google/go-containerregistry/internal/retry"
|
||||
)
|
||||
|
||||
// Sleep for 0.1 then 0.3 seconds. This should cover networking blips.
|
||||
var defaultBackoff = retry.Backoff{
|
||||
Duration: 100 * time.Millisecond,
|
||||
Factor: 3.0,
|
||||
Jitter: 0.1,
|
||||
Steps: 3,
|
||||
}
|
||||
|
||||
var _ http.RoundTripper = (*retryTransport)(nil)
|
||||
|
||||
// retryTransport wraps a RoundTripper and retries temporary network errors.
|
||||
type retryTransport struct {
|
||||
inner http.RoundTripper
|
||||
backoff retry.Backoff
|
||||
predicate retry.Predicate
|
||||
codes []int
|
||||
}
|
||||
|
||||
// Option is a functional option for retryTransport.
|
||||
type Option func(*options)
|
||||
|
||||
type options struct {
|
||||
backoff retry.Backoff
|
||||
predicate retry.Predicate
|
||||
codes []int
|
||||
}
|
||||
|
||||
// Backoff is an alias of retry.Backoff to expose this configuration option to consumers of this lib
|
||||
type Backoff = retry.Backoff
|
||||
|
||||
// WithRetryBackoff sets the backoff for retry operations.
|
||||
func WithRetryBackoff(backoff Backoff) Option {
|
||||
return func(o *options) {
|
||||
o.backoff = backoff
|
||||
}
|
||||
}
|
||||
|
||||
// WithRetryPredicate sets the predicate for retry operations.
|
||||
func WithRetryPredicate(predicate func(error) bool) Option {
|
||||
return func(o *options) {
|
||||
o.predicate = predicate
|
||||
}
|
||||
}
|
||||
|
||||
// WithRetryStatusCodes sets which http response codes will be retried.
|
||||
func WithRetryStatusCodes(codes ...int) Option {
|
||||
return func(o *options) {
|
||||
o.codes = codes
|
||||
}
|
||||
}
|
||||
|
||||
// NewRetry returns a transport that retries errors.
|
||||
func NewRetry(inner http.RoundTripper, opts ...Option) http.RoundTripper {
|
||||
o := &options{
|
||||
backoff: defaultBackoff,
|
||||
predicate: retry.IsTemporary,
|
||||
}
|
||||
|
||||
for _, opt := range opts {
|
||||
opt(o)
|
||||
}
|
||||
|
||||
return &retryTransport{
|
||||
inner: inner,
|
||||
backoff: o.backoff,
|
||||
predicate: o.predicate,
|
||||
codes: o.codes,
|
||||
}
|
||||
}
|
||||
|
||||
func (t *retryTransport) RoundTrip(in *http.Request) (out *http.Response, err error) {
|
||||
roundtrip := func() error {
|
||||
out, err = t.inner.RoundTrip(in)
|
||||
if !retry.Ever(in.Context()) {
|
||||
return nil
|
||||
}
|
||||
if out != nil {
|
||||
for _, code := range t.codes {
|
||||
if out.StatusCode == code {
|
||||
return retryError(out)
|
||||
}
|
||||
}
|
||||
}
|
||||
return err
|
||||
}
|
||||
retry.Retry(roundtrip, t.predicate, t.backoff)
|
||||
return
|
||||
}
|
||||
+44
@@ -0,0 +1,44 @@
|
||||
// Copyright 2019 Google LLC All Rights Reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package transport
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
|
||||
"github.com/google/go-containerregistry/pkg/name"
|
||||
)
|
||||
|
||||
type schemeTransport struct {
|
||||
// Scheme we should use, determined by ping response.
|
||||
scheme string
|
||||
|
||||
// Registry we're talking to.
|
||||
registry name.Registry
|
||||
|
||||
// Wrapped by schemeTransport.
|
||||
inner http.RoundTripper
|
||||
}
|
||||
|
||||
// RoundTrip implements http.RoundTripper
|
||||
func (st *schemeTransport) RoundTrip(in *http.Request) (*http.Response, error) {
|
||||
// When we ping() the registry, we determine whether to use http or https
|
||||
// based on which scheme was successful. That is only valid for the
|
||||
// registry server and not e.g. a separate token server or blob storage,
|
||||
// so we should only override the scheme if the host is the registry.
|
||||
if matchesHost(st.registry.String(), in, st.scheme) {
|
||||
in.URL.Scheme = st.scheme
|
||||
}
|
||||
return st.inner.RoundTrip(in)
|
||||
}
|
||||
+24
@@ -0,0 +1,24 @@
|
||||
// Copyright 2018 Google LLC All Rights Reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package transport
|
||||
|
||||
// Scopes suitable to qualify each Repository
|
||||
const (
|
||||
PullScope string = "pull"
|
||||
PushScope string = "push,pull"
|
||||
// For now DELETE is PUSH, which is the read/write ACL.
|
||||
DeleteScope string = PushScope
|
||||
CatalogScope string = "catalog"
|
||||
)
|
||||
Generated
Vendored
+109
@@ -0,0 +1,109 @@
|
||||
// Copyright 2018 Google LLC All Rights Reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package transport
|
||||
|
||||
import (
|
||||
"context"
|
||||
"net/http"
|
||||
"strings"
|
||||
|
||||
"github.com/google/go-containerregistry/pkg/authn"
|
||||
"github.com/google/go-containerregistry/pkg/name"
|
||||
)
|
||||
|
||||
// New returns a new RoundTripper based on the provided RoundTripper that has been
|
||||
// setup to authenticate with the remote registry "reg", in the capacity
|
||||
// laid out by the specified scopes.
|
||||
//
|
||||
// Deprecated: Use NewWithContext.
|
||||
func New(reg name.Registry, auth authn.Authenticator, t http.RoundTripper, scopes []string) (http.RoundTripper, error) {
|
||||
return NewWithContext(context.Background(), reg, auth, t, scopes)
|
||||
}
|
||||
|
||||
// NewWithContext returns a new RoundTripper based on the provided RoundTripper that has been
|
||||
// set up to authenticate with the remote registry "reg", in the capacity
|
||||
// laid out by the specified scopes.
|
||||
// In case the RoundTripper is already of the type Wrapper it assumes
|
||||
// authentication was already done prior to this call, so it just returns
|
||||
// the provided RoundTripper without further action
|
||||
func NewWithContext(ctx context.Context, reg name.Registry, auth authn.Authenticator, t http.RoundTripper, scopes []string) (http.RoundTripper, error) {
|
||||
// When the transport provided is of the type Wrapper this function assumes that the caller already
|
||||
// executed the necessary login and check.
|
||||
switch t.(type) {
|
||||
case *Wrapper:
|
||||
return t, nil
|
||||
}
|
||||
// The handshake:
|
||||
// 1. Use "t" to ping() the registry for the authentication challenge.
|
||||
//
|
||||
// 2a. If we get back a 200, then simply use "t".
|
||||
//
|
||||
// 2b. If we get back a 401 with a Basic challenge, then use a transport
|
||||
// that just attachs auth each roundtrip.
|
||||
//
|
||||
// 2c. If we get back a 401 with a Bearer challenge, then use a transport
|
||||
// that attaches a bearer token to each request, and refreshes is on 401s.
|
||||
// Perform an initial refresh to seed the bearer token.
|
||||
|
||||
// First we ping the registry to determine the parameters of the authentication handshake
|
||||
// (if one is even necessary).
|
||||
pr, err := Ping(ctx, reg, t)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Wrap t with a useragent transport unless we already have one.
|
||||
if _, ok := t.(*userAgentTransport); !ok {
|
||||
t = NewUserAgent(t, "")
|
||||
}
|
||||
|
||||
scheme := "https"
|
||||
if pr.Insecure {
|
||||
scheme = "http"
|
||||
}
|
||||
|
||||
// Wrap t in a transport that selects the appropriate scheme based on the ping response.
|
||||
t = &schemeTransport{
|
||||
scheme: scheme,
|
||||
registry: reg,
|
||||
inner: t,
|
||||
}
|
||||
|
||||
if strings.ToLower(pr.Scheme) != "bearer" {
|
||||
return &Wrapper{&basicTransport{inner: t, auth: auth, target: reg.RegistryStr()}}, nil
|
||||
}
|
||||
|
||||
bt, err := fromChallenge(reg, auth, t, pr)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
bt.scopes = scopes
|
||||
|
||||
if err := bt.refresh(ctx); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &Wrapper{bt}, nil
|
||||
}
|
||||
|
||||
// Wrapper results in *not* wrapping supplied transport with additional logic such as retries, useragent and debug logging
|
||||
// Consumers are opt-ing into providing their own transport without any additional wrapping.
|
||||
type Wrapper struct {
|
||||
inner http.RoundTripper
|
||||
}
|
||||
|
||||
// RoundTrip delegates to the inner RoundTripper
|
||||
func (w *Wrapper) RoundTrip(in *http.Request) (*http.Response, error) {
|
||||
return w.inner.RoundTrip(in)
|
||||
}
|
||||
Generated
Vendored
+94
@@ -0,0 +1,94 @@
|
||||
// Copyright 2019 Google LLC All Rights Reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package transport
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"net/http"
|
||||
"runtime/debug"
|
||||
)
|
||||
|
||||
var (
|
||||
// Version can be set via:
|
||||
// -ldflags="-X 'github.com/google/go-containerregistry/pkg/v1/remote/transport.Version=$TAG'"
|
||||
Version string
|
||||
|
||||
ggcrVersion = defaultUserAgent
|
||||
)
|
||||
|
||||
const (
|
||||
defaultUserAgent = "go-containerregistry"
|
||||
moduleName = "github.com/google/go-containerregistry"
|
||||
)
|
||||
|
||||
type userAgentTransport struct {
|
||||
inner http.RoundTripper
|
||||
ua string
|
||||
}
|
||||
|
||||
func init() {
|
||||
if v := version(); v != "" {
|
||||
ggcrVersion = fmt.Sprintf("%s/%s", defaultUserAgent, v)
|
||||
}
|
||||
}
|
||||
|
||||
func version() string {
|
||||
if Version != "" {
|
||||
// Version was set via ldflags, just return it.
|
||||
return Version
|
||||
}
|
||||
|
||||
info, ok := debug.ReadBuildInfo()
|
||||
if !ok {
|
||||
return ""
|
||||
}
|
||||
|
||||
// Happens for crane and gcrane.
|
||||
if info.Main.Path == moduleName {
|
||||
return info.Main.Version
|
||||
}
|
||||
|
||||
// Anything else.
|
||||
for _, dep := range info.Deps {
|
||||
if dep.Path == moduleName {
|
||||
return dep.Version
|
||||
}
|
||||
}
|
||||
|
||||
return ""
|
||||
}
|
||||
|
||||
// NewUserAgent returns an http.Roundtripper that sets the user agent to
|
||||
// The provided string plus additional go-containerregistry information,
|
||||
// e.g. if provided "crane/v0.1.4" and this modules was built at v0.1.4:
|
||||
//
|
||||
// User-Agent: crane/v0.1.4 go-containerregistry/v0.1.4
|
||||
func NewUserAgent(inner http.RoundTripper, ua string) http.RoundTripper {
|
||||
if ua == "" {
|
||||
ua = ggcrVersion
|
||||
} else {
|
||||
ua = fmt.Sprintf("%s %s", ua, ggcrVersion)
|
||||
}
|
||||
return &userAgentTransport{
|
||||
inner: inner,
|
||||
ua: ua,
|
||||
}
|
||||
}
|
||||
|
||||
// RoundTrip implements http.RoundTripper
|
||||
func (ut *userAgentTransport) RoundTrip(in *http.Request) (*http.Response, error) {
|
||||
in.Header.Set("User-Agent", ut.ua)
|
||||
return ut.inner.RoundTrip(in)
|
||||
}
|
||||
+711
@@ -0,0 +1,711 @@
|
||||
// Copyright 2018 Google LLC All Rights Reserved.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
|
||||
package remote
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"sort"
|
||||
"strings"
|
||||
"sync"
|
||||
|
||||
"github.com/google/go-containerregistry/internal/redact"
|
||||
"github.com/google/go-containerregistry/internal/retry"
|
||||
"github.com/google/go-containerregistry/pkg/authn"
|
||||
"github.com/google/go-containerregistry/pkg/logs"
|
||||
"github.com/google/go-containerregistry/pkg/name"
|
||||
v1 "github.com/google/go-containerregistry/pkg/v1"
|
||||
"github.com/google/go-containerregistry/pkg/v1/remote/transport"
|
||||
"github.com/google/go-containerregistry/pkg/v1/stream"
|
||||
"github.com/google/go-containerregistry/pkg/v1/types"
|
||||
)
|
||||
|
||||
// Taggable is an interface that enables a manifest PUT (e.g. for tagging).
|
||||
type Taggable interface {
|
||||
RawManifest() ([]byte, error)
|
||||
}
|
||||
|
||||
// Write pushes the provided img to the specified image reference.
|
||||
func Write(ref name.Reference, img v1.Image, options ...Option) (rerr error) {
|
||||
return Push(ref, img, options...)
|
||||
}
|
||||
|
||||
// writer writes the elements of an image to a remote image reference.
|
||||
type writer struct {
|
||||
repo name.Repository
|
||||
auth authn.Authenticator
|
||||
transport http.RoundTripper
|
||||
|
||||
client *http.Client
|
||||
|
||||
progress *progress
|
||||
backoff Backoff
|
||||
predicate retry.Predicate
|
||||
|
||||
scopeLock sync.Mutex
|
||||
// Keep track of scopes that we have already requested.
|
||||
scopeSet map[string]struct{}
|
||||
scopes []string
|
||||
}
|
||||
|
||||
func makeWriter(ctx context.Context, repo name.Repository, ls []v1.Layer, o *options) (*writer, error) {
|
||||
auth := o.auth
|
||||
if o.keychain != nil {
|
||||
kauth, err := authn.Resolve(ctx, o.keychain, repo)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
auth = kauth
|
||||
}
|
||||
scopes := scopesForUploadingImage(repo, ls)
|
||||
tr, err := transport.NewWithContext(ctx, repo.Registry, auth, o.transport, scopes)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
scopeSet := map[string]struct{}{}
|
||||
for _, scope := range scopes {
|
||||
scopeSet[scope] = struct{}{}
|
||||
}
|
||||
return &writer{
|
||||
repo: repo,
|
||||
client: &http.Client{Transport: tr},
|
||||
auth: auth,
|
||||
transport: o.transport,
|
||||
progress: o.progress,
|
||||
backoff: o.retryBackoff,
|
||||
predicate: o.retryPredicate,
|
||||
scopes: scopes,
|
||||
scopeSet: scopeSet,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// url returns a url.Url for the specified path in the context of this remote image reference.
|
||||
func (w *writer) url(path string) url.URL {
|
||||
return url.URL{
|
||||
Scheme: w.repo.Scheme(),
|
||||
Host: w.repo.RegistryStr(),
|
||||
Path: path,
|
||||
}
|
||||
}
|
||||
|
||||
func (w *writer) maybeUpdateScopes(ctx context.Context, ml *MountableLayer) error {
|
||||
if ml.Reference.Context().String() == w.repo.String() {
|
||||
return nil
|
||||
}
|
||||
if ml.Reference.Context().Registry.String() != w.repo.Registry.String() {
|
||||
return nil
|
||||
}
|
||||
|
||||
scope := ml.Reference.Scope(transport.PullScope)
|
||||
|
||||
w.scopeLock.Lock()
|
||||
defer w.scopeLock.Unlock()
|
||||
|
||||
if _, ok := w.scopeSet[scope]; !ok {
|
||||
w.scopeSet[scope] = struct{}{}
|
||||
w.scopes = append(w.scopes, scope)
|
||||
|
||||
logs.Debug.Printf("Refreshing token to add scope %q", scope)
|
||||
wt, err := transport.NewWithContext(ctx, w.repo.Registry, w.auth, w.transport, w.scopes)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
w.client = &http.Client{Transport: wt}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// nextLocation extracts the fully-qualified URL to which we should send the next request in an upload sequence.
|
||||
func (w *writer) nextLocation(resp *http.Response) (string, error) {
|
||||
loc := resp.Header.Get("Location")
|
||||
if len(loc) == 0 {
|
||||
return "", errors.New("missing Location header")
|
||||
}
|
||||
u, err := url.Parse(loc)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
// If the location header returned is just a url path, then fully qualify it.
|
||||
// We cannot simply call w.url, since there might be an embedded query string.
|
||||
return resp.Request.URL.ResolveReference(u).String(), nil
|
||||
}
|
||||
|
||||
// checkExistingBlob checks if a blob exists already in the repository by making a
|
||||
// HEAD request to the blob store API. GCR performs an existence check on the
|
||||
// initiation if "mount" is specified, even if no "from" sources are specified.
|
||||
// However, this is not broadly applicable to all registries, e.g. ECR.
|
||||
func (w *writer) checkExistingBlob(ctx context.Context, h v1.Hash) (bool, error) {
|
||||
u := w.url(fmt.Sprintf("/v2/%s/blobs/%s", w.repo.RepositoryStr(), h.String()))
|
||||
|
||||
req, err := http.NewRequest(http.MethodHead, u.String(), nil)
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
|
||||
resp, err := w.client.Do(req.WithContext(ctx))
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if err := transport.CheckError(resp, http.StatusOK, http.StatusNotFound); err != nil {
|
||||
return false, err
|
||||
}
|
||||
|
||||
return resp.StatusCode == http.StatusOK, nil
|
||||
}
|
||||
|
||||
// initiateUpload initiates the blob upload, which starts with a POST that can
|
||||
// optionally include the hash of the layer and a list of repositories from
|
||||
// which that layer might be read. On failure, an error is returned.
|
||||
// On success, the layer was either mounted (nothing more to do) or a blob
|
||||
// upload was initiated and the body of that blob should be sent to the returned
|
||||
// location.
|
||||
func (w *writer) initiateUpload(ctx context.Context, from, mount, origin string) (location string, mounted bool, err error) {
|
||||
u := w.url(fmt.Sprintf("/v2/%s/blobs/uploads/", w.repo.RepositoryStr()))
|
||||
uv := url.Values{}
|
||||
if mount != "" && from != "" {
|
||||
// Quay will fail if we specify a "mount" without a "from".
|
||||
uv.Set("mount", mount)
|
||||
uv.Set("from", from)
|
||||
if origin != "" {
|
||||
uv.Set("origin", origin)
|
||||
}
|
||||
}
|
||||
u.RawQuery = uv.Encode()
|
||||
|
||||
// Make the request to initiate the blob upload.
|
||||
req, err := http.NewRequest(http.MethodPost, u.String(), nil)
|
||||
if err != nil {
|
||||
return "", false, err
|
||||
}
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
resp, err := w.client.Do(req.WithContext(ctx))
|
||||
if err != nil {
|
||||
if from != "" {
|
||||
// https://github.com/google/go-containerregistry/issues/1679
|
||||
logs.Warn.Printf("retrying without mount: %v", err)
|
||||
return w.initiateUpload(ctx, "", "", "")
|
||||
}
|
||||
return "", false, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if err := transport.CheckError(resp, http.StatusCreated, http.StatusAccepted); err != nil {
|
||||
if from != "" {
|
||||
// https://github.com/google/go-containerregistry/issues/1404
|
||||
logs.Warn.Printf("retrying without mount: %v", err)
|
||||
return w.initiateUpload(ctx, "", "", "")
|
||||
}
|
||||
return "", false, err
|
||||
}
|
||||
|
||||
// Check the response code to determine the result.
|
||||
switch resp.StatusCode {
|
||||
case http.StatusCreated:
|
||||
// We're done, we were able to fast-path.
|
||||
return "", true, nil
|
||||
case http.StatusAccepted:
|
||||
// Proceed to PATCH, upload has begun.
|
||||
loc, err := w.nextLocation(resp)
|
||||
return loc, false, err
|
||||
default:
|
||||
panic("Unreachable: initiateUpload")
|
||||
}
|
||||
}
|
||||
|
||||
// streamBlob streams the contents of the blob to the specified location.
|
||||
// On failure, this will return an error. On success, this will return the location
|
||||
// header indicating how to commit the streamed blob.
|
||||
func (w *writer) streamBlob(ctx context.Context, layer v1.Layer, streamLocation string) (commitLocation string, rerr error) {
|
||||
reset := func() {}
|
||||
defer func() {
|
||||
if rerr != nil {
|
||||
reset()
|
||||
}
|
||||
}()
|
||||
blob, err := layer.Compressed()
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
getBody := layer.Compressed
|
||||
if w.progress != nil {
|
||||
var count int64
|
||||
blob = &progressReader{rc: blob, progress: w.progress, count: &count}
|
||||
getBody = func() (io.ReadCloser, error) {
|
||||
blob, err := layer.Compressed()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &progressReader{rc: blob, progress: w.progress, count: &count}, nil
|
||||
}
|
||||
reset = func() {
|
||||
w.progress.complete(-count)
|
||||
}
|
||||
}
|
||||
|
||||
req, err := http.NewRequest(http.MethodPatch, streamLocation, blob)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if _, ok := layer.(*stream.Layer); !ok {
|
||||
// We can't retry streaming layers.
|
||||
req.GetBody = getBody
|
||||
|
||||
// If we know the size, set it.
|
||||
if size, err := layer.Size(); err == nil {
|
||||
req.ContentLength = size
|
||||
}
|
||||
}
|
||||
req.Header.Set("Content-Type", "application/octet-stream")
|
||||
|
||||
resp, err := w.client.Do(req.WithContext(ctx))
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if err := transport.CheckError(resp, http.StatusNoContent, http.StatusAccepted, http.StatusCreated); err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
// The blob has been uploaded, return the location header indicating
|
||||
// how to commit this layer.
|
||||
return w.nextLocation(resp)
|
||||
}
|
||||
|
||||
// commitBlob commits this blob by sending a PUT to the location returned from
|
||||
// streaming the blob.
|
||||
func (w *writer) commitBlob(ctx context.Context, location, digest string) error {
|
||||
u, err := url.Parse(location)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
v := u.Query()
|
||||
v.Set("digest", digest)
|
||||
u.RawQuery = v.Encode()
|
||||
|
||||
req, err := http.NewRequest(http.MethodPut, u.String(), nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
req.Header.Set("Content-Type", "application/octet-stream")
|
||||
|
||||
resp, err := w.client.Do(req.WithContext(ctx))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
return transport.CheckError(resp, http.StatusCreated)
|
||||
}
|
||||
|
||||
// incrProgress increments and sends a progress update, if WithProgress is used.
|
||||
func (w *writer) incrProgress(written int64) {
|
||||
if w.progress == nil {
|
||||
return
|
||||
}
|
||||
w.progress.complete(written)
|
||||
}
|
||||
|
||||
// uploadOne performs a complete upload of a single layer.
|
||||
func (w *writer) uploadOne(ctx context.Context, l v1.Layer) error {
|
||||
tryUpload := func() error {
|
||||
ctx := retry.Never(ctx)
|
||||
var from, mount, origin string
|
||||
if h, err := l.Digest(); err == nil {
|
||||
// If we know the digest, this isn't a streaming layer. Do an existence
|
||||
// check so we can skip uploading the layer if possible.
|
||||
existing, err := w.checkExistingBlob(ctx, h)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if existing {
|
||||
size, err := l.Size()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
w.incrProgress(size)
|
||||
logs.Progress.Printf("existing blob: %v", h)
|
||||
return nil
|
||||
}
|
||||
|
||||
mount = h.String()
|
||||
}
|
||||
if ml, ok := l.(*MountableLayer); ok {
|
||||
if err := w.maybeUpdateScopes(ctx, ml); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
from = ml.Reference.Context().RepositoryStr()
|
||||
origin = ml.Reference.Context().RegistryStr()
|
||||
|
||||
// This keeps breaking with DockerHub.
|
||||
// https://github.com/google/go-containerregistry/issues/1741
|
||||
if w.repo.RegistryStr() == name.DefaultRegistry && origin != w.repo.RegistryStr() {
|
||||
from = ""
|
||||
origin = ""
|
||||
}
|
||||
}
|
||||
|
||||
location, mounted, err := w.initiateUpload(ctx, from, mount, origin)
|
||||
if err != nil {
|
||||
return err
|
||||
} else if mounted {
|
||||
size, err := l.Size()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
w.incrProgress(size)
|
||||
h, err := l.Digest()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
logs.Progress.Printf("mounted blob: %s", h.String())
|
||||
return nil
|
||||
}
|
||||
|
||||
// Only log layers with +json or +yaml. We can let through other stuff if it becomes popular.
|
||||
// TODO(opencontainers/image-spec#791): Would be great to have an actual parser.
|
||||
mt, err := l.MediaType()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
smt := string(mt)
|
||||
if !strings.HasSuffix(smt, "+json") && !strings.HasSuffix(smt, "+yaml") {
|
||||
ctx = redact.NewContext(ctx, "omitting binary blobs from logs")
|
||||
}
|
||||
|
||||
location, err = w.streamBlob(ctx, l, location)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
h, err := l.Digest()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
digest := h.String()
|
||||
|
||||
if err := w.commitBlob(ctx, location, digest); err != nil {
|
||||
return err
|
||||
}
|
||||
logs.Progress.Printf("pushed blob: %s", digest)
|
||||
return nil
|
||||
}
|
||||
|
||||
return retry.Retry(tryUpload, w.predicate, w.backoff)
|
||||
}
|
||||
|
||||
type withMediaType interface {
|
||||
MediaType() (types.MediaType, error)
|
||||
}
|
||||
|
||||
// This is really silly, but go interfaces don't let me satisfy remote.Taggable
|
||||
// with remote.Descriptor because of name collisions between method names and
|
||||
// struct fields.
|
||||
//
|
||||
// Use reflection to either pull the v1.Descriptor out of remote.Descriptor or
|
||||
// create a descriptor based on the RawManifest and (optionally) MediaType.
|
||||
func unpackTaggable(t Taggable) ([]byte, *v1.Descriptor, error) {
|
||||
if d, ok := t.(*Descriptor); ok {
|
||||
return d.Manifest, &d.Descriptor, nil
|
||||
}
|
||||
b, err := t.RawManifest()
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
|
||||
// A reasonable default if Taggable doesn't implement MediaType.
|
||||
mt := types.DockerManifestSchema2
|
||||
|
||||
if wmt, ok := t.(withMediaType); ok {
|
||||
m, err := wmt.MediaType()
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
mt = m
|
||||
}
|
||||
|
||||
h, sz, err := v1.SHA256(bytes.NewReader(b))
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
|
||||
return b, &v1.Descriptor{
|
||||
MediaType: mt,
|
||||
Size: sz,
|
||||
Digest: h,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// commitSubjectReferrers is responsible for updating the fallback tag manifest to track descriptors referring to a subject for registries that don't yet support the Referrers API.
|
||||
// TODO: use conditional requests to avoid race conditions
|
||||
func (w *writer) commitSubjectReferrers(ctx context.Context, sub name.Digest, add v1.Descriptor) error {
|
||||
// Check if the registry supports Referrers API.
|
||||
// TODO: This should be done once per registry, not once per subject.
|
||||
u := w.url(fmt.Sprintf("/v2/%s/referrers/%s", w.repo.RepositoryStr(), sub.DigestStr()))
|
||||
req, err := http.NewRequest(http.MethodGet, u.String(), nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
req.Header.Set("Accept", string(types.OCIImageIndex))
|
||||
resp, err := w.client.Do(req.WithContext(ctx))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if err := transport.CheckError(resp, http.StatusOK, http.StatusNotFound, http.StatusBadRequest); err != nil {
|
||||
return err
|
||||
}
|
||||
if resp.StatusCode == http.StatusOK {
|
||||
// The registry supports Referrers API. The registry is responsible for updating the referrers list.
|
||||
return nil
|
||||
}
|
||||
|
||||
// The registry doesn't support Referrers API, we need to update the manifest tagged with the fallback tag.
|
||||
// Make the request to GET the current manifest.
|
||||
t := fallbackTag(sub)
|
||||
u = w.url(fmt.Sprintf("/v2/%s/manifests/%s", w.repo.RepositoryStr(), t.Identifier()))
|
||||
req, err = http.NewRequest(http.MethodGet, u.String(), nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
req.Header.Set("Accept", string(types.OCIImageIndex))
|
||||
resp, err = w.client.Do(req.WithContext(ctx))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
var im v1.IndexManifest
|
||||
if err := transport.CheckError(resp, http.StatusOK, http.StatusNotFound); err != nil {
|
||||
return err
|
||||
} else if resp.StatusCode == http.StatusNotFound {
|
||||
// Not found just means there are no attachments. Start with an empty index.
|
||||
im = v1.IndexManifest{
|
||||
SchemaVersion: 2,
|
||||
MediaType: types.OCIImageIndex,
|
||||
Manifests: []v1.Descriptor{add},
|
||||
}
|
||||
} else {
|
||||
if err := json.NewDecoder(resp.Body).Decode(&im); err != nil {
|
||||
return err
|
||||
}
|
||||
if im.SchemaVersion != 2 {
|
||||
return fmt.Errorf("fallback tag manifest is not a schema version 2: %d", im.SchemaVersion)
|
||||
}
|
||||
if im.MediaType != types.OCIImageIndex {
|
||||
return fmt.Errorf("fallback tag manifest is not an OCI image index: %s", im.MediaType)
|
||||
}
|
||||
for _, desc := range im.Manifests {
|
||||
if desc.Digest == add.Digest {
|
||||
// The digest is already attached, nothing to do.
|
||||
logs.Progress.Printf("fallback tag %s already had referrer", t.Identifier())
|
||||
return nil
|
||||
}
|
||||
}
|
||||
// Append the new descriptor to the index.
|
||||
im.Manifests = append(im.Manifests, add)
|
||||
}
|
||||
|
||||
// Sort the manifests for reproducibility.
|
||||
sort.Slice(im.Manifests, func(i, j int) bool {
|
||||
return im.Manifests[i].Digest.String() < im.Manifests[j].Digest.String()
|
||||
})
|
||||
logs.Progress.Printf("updating fallback tag %s with new referrer", t.Identifier())
|
||||
return w.commitManifest(ctx, fallbackTaggable{im}, t)
|
||||
}
|
||||
|
||||
type fallbackTaggable struct {
|
||||
im v1.IndexManifest
|
||||
}
|
||||
|
||||
func (f fallbackTaggable) RawManifest() ([]byte, error) { return json.Marshal(f.im) }
|
||||
func (f fallbackTaggable) MediaType() (types.MediaType, error) { return types.OCIImageIndex, nil }
|
||||
|
||||
// commitManifest does a PUT of the image's manifest.
|
||||
func (w *writer) commitManifest(ctx context.Context, t Taggable, ref name.Reference) error {
|
||||
// If the manifest refers to a subject, we need to check whether we need to update the fallback tag manifest.
|
||||
raw, err := t.RawManifest()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var mf struct {
|
||||
MediaType types.MediaType `json:"mediaType"`
|
||||
Subject *v1.Descriptor `json:"subject,omitempty"`
|
||||
Config struct {
|
||||
MediaType types.MediaType `json:"mediaType"`
|
||||
} `json:"config"`
|
||||
}
|
||||
if err := json.Unmarshal(raw, &mf); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
tryUpload := func() error {
|
||||
ctx := retry.Never(ctx)
|
||||
raw, desc, err := unpackTaggable(t)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
u := w.url(fmt.Sprintf("/v2/%s/manifests/%s", w.repo.RepositoryStr(), ref.Identifier()))
|
||||
|
||||
// Make the request to PUT the serialized manifest
|
||||
req, err := http.NewRequest(http.MethodPut, u.String(), bytes.NewBuffer(raw))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
req.Header.Set("Content-Type", string(desc.MediaType))
|
||||
|
||||
resp, err := w.client.Do(req.WithContext(ctx))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if err := transport.CheckError(resp, http.StatusOK, http.StatusCreated, http.StatusAccepted); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// If the manifest referred to a subject, we may need to update the fallback tag manifest.
|
||||
// TODO: If this fails, we'll retry the whole upload. We should retry just this part.
|
||||
if mf.Subject != nil {
|
||||
h, size, err := v1.SHA256(bytes.NewReader(raw))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
desc := v1.Descriptor{
|
||||
ArtifactType: string(mf.Config.MediaType),
|
||||
MediaType: mf.MediaType,
|
||||
Digest: h,
|
||||
Size: size,
|
||||
}
|
||||
if err := w.commitSubjectReferrers(ctx,
|
||||
ref.Context().Digest(mf.Subject.Digest.String()),
|
||||
desc); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
// The image was successfully pushed!
|
||||
logs.Progress.Printf("%v: digest: %v size: %d", ref, desc.Digest, desc.Size)
|
||||
w.incrProgress(int64(len(raw)))
|
||||
return nil
|
||||
}
|
||||
|
||||
return retry.Retry(tryUpload, w.predicate, w.backoff)
|
||||
}
|
||||
|
||||
func scopesForUploadingImage(repo name.Repository, layers []v1.Layer) []string {
|
||||
// use a map as set to remove duplicates scope strings
|
||||
scopeSet := map[string]struct{}{}
|
||||
|
||||
for _, l := range layers {
|
||||
if ml, ok := l.(*MountableLayer); ok {
|
||||
// we will add push scope for ref.Context() after the loop.
|
||||
// for now we ask pull scope for references of the same registry
|
||||
if ml.Reference.Context().String() != repo.String() && ml.Reference.Context().Registry.String() == repo.Registry.String() {
|
||||
scopeSet[ml.Reference.Scope(transport.PullScope)] = struct{}{}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
scopes := make([]string, 0, len(scopeSet)+1)
|
||||
// Push scope should be the first element because a few registries just look at the first scope to determine access.
|
||||
scopes = append(scopes, repo.Scope(transport.PushScope))
|
||||
|
||||
for scope := range scopeSet {
|
||||
scopes = append(scopes, scope)
|
||||
}
|
||||
|
||||
return scopes
|
||||
}
|
||||
|
||||
// WriteIndex pushes the provided ImageIndex to the specified image reference.
|
||||
// WriteIndex will attempt to push all of the referenced manifests before
|
||||
// attempting to push the ImageIndex, to retain referential integrity.
|
||||
func WriteIndex(ref name.Reference, ii v1.ImageIndex, options ...Option) (rerr error) {
|
||||
return Push(ref, ii, options...)
|
||||
}
|
||||
|
||||
// WriteLayer uploads the provided Layer to the specified repo.
|
||||
func WriteLayer(repo name.Repository, layer v1.Layer, options ...Option) (rerr error) {
|
||||
o, err := makeOptions(options...)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if o.progress != nil {
|
||||
defer func() { o.progress.Close(rerr) }()
|
||||
}
|
||||
return newPusher(o).Upload(o.context, repo, layer)
|
||||
}
|
||||
|
||||
// Tag adds a tag to the given Taggable via PUT /v2/.../manifests/<tag>
|
||||
//
|
||||
// Notable implementations of Taggable are v1.Image, v1.ImageIndex, and
|
||||
// remote.Descriptor.
|
||||
//
|
||||
// If t implements MediaType, we will use that for the Content-Type, otherwise
|
||||
// we will default to types.DockerManifestSchema2.
|
||||
//
|
||||
// Tag does not attempt to write anything other than the manifest, so callers
|
||||
// should ensure that all blobs or manifests that are referenced by t exist
|
||||
// in the target registry.
|
||||
func Tag(tag name.Tag, t Taggable, options ...Option) error {
|
||||
return Put(tag, t, options...)
|
||||
}
|
||||
|
||||
// Put adds a manifest from the given Taggable via PUT /v1/.../manifest/<ref>
|
||||
//
|
||||
// Notable implementations of Taggable are v1.Image, v1.ImageIndex, and
|
||||
// remote.Descriptor.
|
||||
//
|
||||
// If t implements MediaType, we will use that for the Content-Type, otherwise
|
||||
// we will default to types.DockerManifestSchema2.
|
||||
//
|
||||
// Put does not attempt to write anything other than the manifest, so callers
|
||||
// should ensure that all blobs or manifests that are referenced by t exist
|
||||
// in the target registry.
|
||||
func Put(ref name.Reference, t Taggable, options ...Option) error {
|
||||
o, err := makeOptions(options...)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return newPusher(o).Put(o.context, ref, t)
|
||||
}
|
||||
|
||||
// Push uploads the given Taggable to the specified reference.
|
||||
func Push(ref name.Reference, t Taggable, options ...Option) (rerr error) {
|
||||
o, err := makeOptions(options...)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if o.progress != nil {
|
||||
defer func() { o.progress.Close(rerr) }()
|
||||
}
|
||||
return newPusher(o).Push(o.context, ref, t)
|
||||
}
|
||||
Reference in New Issue
Block a user