working commit

This commit is contained in:
2026-02-21 13:16:56 +02:00
parent d650d58a6d
commit 7be3cf8de7
1136 changed files with 722443 additions and 0 deletions
+50
View File
@@ -0,0 +1,50 @@
This project is covered by two different licenses: MIT and Apache.
#### MIT License ####
The following files were ported to Go from C files of libyaml, and thus
are still covered by their original MIT license, with the additional
copyright staring in 2011 when the project was ported over:
apic.go emitterc.go parserc.go readerc.go scannerc.go
writerc.go yamlh.go yamlprivateh.go
Copyright (c) 2006-2010 Kirill Simonov
Copyright (c) 2006-2011 Kirill Simonov
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
of the Software, and to permit persons to whom the Software is furnished to do
so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
### Apache License ###
All the remaining project files are covered by the Apache license:
Copyright (c) 2011-2019 Canonical Ltd
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
+13
View File
@@ -0,0 +1,13 @@
Copyright 2011-2016 Canonical Ltd.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
+171
View File
@@ -0,0 +1,171 @@
go.yaml.in/yaml
===============
YAML Support for the Go Language
## Introduction
The `yaml` package enables [Go](https://go.dev/) programs to comfortably encode
and decode [YAML](https://yaml.org/) values.
It was originally developed within [Canonical](https://www.canonical.com) as
part of the [juju](https://juju.ubuntu.com) project, and is based on a pure Go
port of the well-known [libyaml](http://pyyaml.org/wiki/LibYAML) C library to
parse and generate YAML data quickly and reliably.
## Project Status
This project started as a fork of the extremely popular [go-yaml](
https://github.com/go-yaml/yaml/)
project, and is being maintained by the official [YAML organization](
https://github.com/yaml/).
The YAML team took over ongoing maintenance and development of the project after
discussion with go-yaml's author, @niemeyer, following his decision to
[label the project repository as "unmaintained"](
https://github.com/go-yaml/yaml/blob/944c86a7d2/README.md) in April 2025.
We have put together a team of dedicated maintainers including representatives
of go-yaml's most important downstream projects.
We will strive to earn the trust of the various go-yaml forks to switch back to
this repository as their upstream.
Please [contact us](https://cloud-native.slack.com/archives/C08PPAT8PS7) if you
would like to contribute or be involved.
## Compatibility
The `yaml` package supports most of YAML 1.2, but preserves some behavior from
1.1 for backwards compatibility.
Specifically, v3 of the `yaml` package:
* Supports YAML 1.1 bools (`yes`/`no`, `on`/`off`) as long as they are being
decoded into a typed bool value.
Otherwise they behave as a string.
Booleans in YAML 1.2 are `true`/`false` only.
* Supports octals encoded and decoded as `0777` per YAML 1.1, rather than
`0o777` as specified in YAML 1.2, because most parsers still use the old
format.
Octals in the `0o777` format are supported though, so new files work.
* Does not support base-60 floats.
These are gone from YAML 1.2, and were actually never supported by this
package as it's clearly a poor choice.
## Installation and Usage
The import path for the package is *go.yaml.in/yaml/v3*.
To install it, run:
```bash
go get go.yaml.in/yaml/v3
```
## API Documentation
See: <https://pkg.go.dev/go.yaml.in/yaml/v3>
## API Stability
The package API for yaml v3 will remain stable as described in [gopkg.in](
https://gopkg.in).
## Example
```go
package main
import (
"fmt"
"log"
"go.yaml.in/yaml/v3"
)
var data = `
a: Easy!
b:
c: 2
d: [3, 4]
`
// Note: struct fields must be public in order for unmarshal to
// correctly populate the data.
type T struct {
A string
B struct {
RenamedC int `yaml:"c"`
D []int `yaml:",flow"`
}
}
func main() {
t := T{}
err := yaml.Unmarshal([]byte(data), &t)
if err != nil {
log.Fatalf("error: %v", err)
}
fmt.Printf("--- t:\n%v\n\n", t)
d, err := yaml.Marshal(&t)
if err != nil {
log.Fatalf("error: %v", err)
}
fmt.Printf("--- t dump:\n%s\n\n", string(d))
m := make(map[interface{}]interface{})
err = yaml.Unmarshal([]byte(data), &m)
if err != nil {
log.Fatalf("error: %v", err)
}
fmt.Printf("--- m:\n%v\n\n", m)
d, err = yaml.Marshal(&m)
if err != nil {
log.Fatalf("error: %v", err)
}
fmt.Printf("--- m dump:\n%s\n\n", string(d))
}
```
This example will generate the following output:
```
--- t:
{Easy! {2 [3 4]}}
--- t dump:
a: Easy!
b:
c: 2
d: [3, 4]
--- m:
map[a:Easy! b:map[c:2 d:[3 4]]]
--- m dump:
a: Easy!
b:
c: 2
d:
- 3
- 4
```
## License
The yaml package is licensed under the MIT and Apache License 2.0 licenses.
Please see the LICENSE file for details.
+747
View File
@@ -0,0 +1,747 @@
//
// Copyright (c) 2011-2019 Canonical Ltd
// Copyright (c) 2006-2010 Kirill Simonov
//
// Permission is hereby granted, free of charge, to any person obtaining a copy of
// this software and associated documentation files (the "Software"), to deal in
// the Software without restriction, including without limitation the rights to
// use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
// of the Software, and to permit persons to whom the Software is furnished to do
// so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in all
// copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
// SOFTWARE.
package yaml
import (
"io"
)
func yaml_insert_token(parser *yaml_parser_t, pos int, token *yaml_token_t) {
//fmt.Println("yaml_insert_token", "pos:", pos, "typ:", token.typ, "head:", parser.tokens_head, "len:", len(parser.tokens))
// Check if we can move the queue at the beginning of the buffer.
if parser.tokens_head > 0 && len(parser.tokens) == cap(parser.tokens) {
if parser.tokens_head != len(parser.tokens) {
copy(parser.tokens, parser.tokens[parser.tokens_head:])
}
parser.tokens = parser.tokens[:len(parser.tokens)-parser.tokens_head]
parser.tokens_head = 0
}
parser.tokens = append(parser.tokens, *token)
if pos < 0 {
return
}
copy(parser.tokens[parser.tokens_head+pos+1:], parser.tokens[parser.tokens_head+pos:])
parser.tokens[parser.tokens_head+pos] = *token
}
// Create a new parser object.
func yaml_parser_initialize(parser *yaml_parser_t) bool {
*parser = yaml_parser_t{
raw_buffer: make([]byte, 0, input_raw_buffer_size),
buffer: make([]byte, 0, input_buffer_size),
}
return true
}
// Destroy a parser object.
func yaml_parser_delete(parser *yaml_parser_t) {
*parser = yaml_parser_t{}
}
// String read handler.
func yaml_string_read_handler(parser *yaml_parser_t, buffer []byte) (n int, err error) {
if parser.input_pos == len(parser.input) {
return 0, io.EOF
}
n = copy(buffer, parser.input[parser.input_pos:])
parser.input_pos += n
return n, nil
}
// Reader read handler.
func yaml_reader_read_handler(parser *yaml_parser_t, buffer []byte) (n int, err error) {
return parser.input_reader.Read(buffer)
}
// Set a string input.
func yaml_parser_set_input_string(parser *yaml_parser_t, input []byte) {
if parser.read_handler != nil {
panic("must set the input source only once")
}
parser.read_handler = yaml_string_read_handler
parser.input = input
parser.input_pos = 0
}
// Set a file input.
func yaml_parser_set_input_reader(parser *yaml_parser_t, r io.Reader) {
if parser.read_handler != nil {
panic("must set the input source only once")
}
parser.read_handler = yaml_reader_read_handler
parser.input_reader = r
}
// Set the source encoding.
func yaml_parser_set_encoding(parser *yaml_parser_t, encoding yaml_encoding_t) {
if parser.encoding != yaml_ANY_ENCODING {
panic("must set the encoding only once")
}
parser.encoding = encoding
}
// Create a new emitter object.
func yaml_emitter_initialize(emitter *yaml_emitter_t) {
*emitter = yaml_emitter_t{
buffer: make([]byte, output_buffer_size),
raw_buffer: make([]byte, 0, output_raw_buffer_size),
states: make([]yaml_emitter_state_t, 0, initial_stack_size),
events: make([]yaml_event_t, 0, initial_queue_size),
best_width: -1,
}
}
// Destroy an emitter object.
func yaml_emitter_delete(emitter *yaml_emitter_t) {
*emitter = yaml_emitter_t{}
}
// String write handler.
func yaml_string_write_handler(emitter *yaml_emitter_t, buffer []byte) error {
*emitter.output_buffer = append(*emitter.output_buffer, buffer...)
return nil
}
// yaml_writer_write_handler uses emitter.output_writer to write the
// emitted text.
func yaml_writer_write_handler(emitter *yaml_emitter_t, buffer []byte) error {
_, err := emitter.output_writer.Write(buffer)
return err
}
// Set a string output.
func yaml_emitter_set_output_string(emitter *yaml_emitter_t, output_buffer *[]byte) {
if emitter.write_handler != nil {
panic("must set the output target only once")
}
emitter.write_handler = yaml_string_write_handler
emitter.output_buffer = output_buffer
}
// Set a file output.
func yaml_emitter_set_output_writer(emitter *yaml_emitter_t, w io.Writer) {
if emitter.write_handler != nil {
panic("must set the output target only once")
}
emitter.write_handler = yaml_writer_write_handler
emitter.output_writer = w
}
// Set the output encoding.
func yaml_emitter_set_encoding(emitter *yaml_emitter_t, encoding yaml_encoding_t) {
if emitter.encoding != yaml_ANY_ENCODING {
panic("must set the output encoding only once")
}
emitter.encoding = encoding
}
// Set the canonical output style.
func yaml_emitter_set_canonical(emitter *yaml_emitter_t, canonical bool) {
emitter.canonical = canonical
}
// Set the indentation increment.
func yaml_emitter_set_indent(emitter *yaml_emitter_t, indent int) {
if indent < 2 || indent > 9 {
indent = 2
}
emitter.best_indent = indent
}
// Set the preferred line width.
func yaml_emitter_set_width(emitter *yaml_emitter_t, width int) {
if width < 0 {
width = -1
}
emitter.best_width = width
}
// Set if unescaped non-ASCII characters are allowed.
func yaml_emitter_set_unicode(emitter *yaml_emitter_t, unicode bool) {
emitter.unicode = unicode
}
// Set the preferred line break character.
func yaml_emitter_set_break(emitter *yaml_emitter_t, line_break yaml_break_t) {
emitter.line_break = line_break
}
///*
// * Destroy a token object.
// */
//
//YAML_DECLARE(void)
//yaml_token_delete(yaml_token_t *token)
//{
// assert(token); // Non-NULL token object expected.
//
// switch (token.type)
// {
// case YAML_TAG_DIRECTIVE_TOKEN:
// yaml_free(token.data.tag_directive.handle);
// yaml_free(token.data.tag_directive.prefix);
// break;
//
// case YAML_ALIAS_TOKEN:
// yaml_free(token.data.alias.value);
// break;
//
// case YAML_ANCHOR_TOKEN:
// yaml_free(token.data.anchor.value);
// break;
//
// case YAML_TAG_TOKEN:
// yaml_free(token.data.tag.handle);
// yaml_free(token.data.tag.suffix);
// break;
//
// case YAML_SCALAR_TOKEN:
// yaml_free(token.data.scalar.value);
// break;
//
// default:
// break;
// }
//
// memset(token, 0, sizeof(yaml_token_t));
//}
//
///*
// * Check if a string is a valid UTF-8 sequence.
// *
// * Check 'reader.c' for more details on UTF-8 encoding.
// */
//
//static int
//yaml_check_utf8(yaml_char_t *start, size_t length)
//{
// yaml_char_t *end = start+length;
// yaml_char_t *pointer = start;
//
// while (pointer < end) {
// unsigned char octet;
// unsigned int width;
// unsigned int value;
// size_t k;
//
// octet = pointer[0];
// width = (octet & 0x80) == 0x00 ? 1 :
// (octet & 0xE0) == 0xC0 ? 2 :
// (octet & 0xF0) == 0xE0 ? 3 :
// (octet & 0xF8) == 0xF0 ? 4 : 0;
// value = (octet & 0x80) == 0x00 ? octet & 0x7F :
// (octet & 0xE0) == 0xC0 ? octet & 0x1F :
// (octet & 0xF0) == 0xE0 ? octet & 0x0F :
// (octet & 0xF8) == 0xF0 ? octet & 0x07 : 0;
// if (!width) return 0;
// if (pointer+width > end) return 0;
// for (k = 1; k < width; k ++) {
// octet = pointer[k];
// if ((octet & 0xC0) != 0x80) return 0;
// value = (value << 6) + (octet & 0x3F);
// }
// if (!((width == 1) ||
// (width == 2 && value >= 0x80) ||
// (width == 3 && value >= 0x800) ||
// (width == 4 && value >= 0x10000))) return 0;
//
// pointer += width;
// }
//
// return 1;
//}
//
// Create STREAM-START.
func yaml_stream_start_event_initialize(event *yaml_event_t, encoding yaml_encoding_t) {
*event = yaml_event_t{
typ: yaml_STREAM_START_EVENT,
encoding: encoding,
}
}
// Create STREAM-END.
func yaml_stream_end_event_initialize(event *yaml_event_t) {
*event = yaml_event_t{
typ: yaml_STREAM_END_EVENT,
}
}
// Create DOCUMENT-START.
func yaml_document_start_event_initialize(
event *yaml_event_t,
version_directive *yaml_version_directive_t,
tag_directives []yaml_tag_directive_t,
implicit bool,
) {
*event = yaml_event_t{
typ: yaml_DOCUMENT_START_EVENT,
version_directive: version_directive,
tag_directives: tag_directives,
implicit: implicit,
}
}
// Create DOCUMENT-END.
func yaml_document_end_event_initialize(event *yaml_event_t, implicit bool) {
*event = yaml_event_t{
typ: yaml_DOCUMENT_END_EVENT,
implicit: implicit,
}
}
// Create ALIAS.
func yaml_alias_event_initialize(event *yaml_event_t, anchor []byte) bool {
*event = yaml_event_t{
typ: yaml_ALIAS_EVENT,
anchor: anchor,
}
return true
}
// Create SCALAR.
func yaml_scalar_event_initialize(event *yaml_event_t, anchor, tag, value []byte, plain_implicit, quoted_implicit bool, style yaml_scalar_style_t) bool {
*event = yaml_event_t{
typ: yaml_SCALAR_EVENT,
anchor: anchor,
tag: tag,
value: value,
implicit: plain_implicit,
quoted_implicit: quoted_implicit,
style: yaml_style_t(style),
}
return true
}
// Create SEQUENCE-START.
func yaml_sequence_start_event_initialize(event *yaml_event_t, anchor, tag []byte, implicit bool, style yaml_sequence_style_t) bool {
*event = yaml_event_t{
typ: yaml_SEQUENCE_START_EVENT,
anchor: anchor,
tag: tag,
implicit: implicit,
style: yaml_style_t(style),
}
return true
}
// Create SEQUENCE-END.
func yaml_sequence_end_event_initialize(event *yaml_event_t) bool {
*event = yaml_event_t{
typ: yaml_SEQUENCE_END_EVENT,
}
return true
}
// Create MAPPING-START.
func yaml_mapping_start_event_initialize(event *yaml_event_t, anchor, tag []byte, implicit bool, style yaml_mapping_style_t) {
*event = yaml_event_t{
typ: yaml_MAPPING_START_EVENT,
anchor: anchor,
tag: tag,
implicit: implicit,
style: yaml_style_t(style),
}
}
// Create MAPPING-END.
func yaml_mapping_end_event_initialize(event *yaml_event_t) {
*event = yaml_event_t{
typ: yaml_MAPPING_END_EVENT,
}
}
// Destroy an event object.
func yaml_event_delete(event *yaml_event_t) {
*event = yaml_event_t{}
}
///*
// * Create a document object.
// */
//
//YAML_DECLARE(int)
//yaml_document_initialize(document *yaml_document_t,
// version_directive *yaml_version_directive_t,
// tag_directives_start *yaml_tag_directive_t,
// tag_directives_end *yaml_tag_directive_t,
// start_implicit int, end_implicit int)
//{
// struct {
// error yaml_error_type_t
// } context
// struct {
// start *yaml_node_t
// end *yaml_node_t
// top *yaml_node_t
// } nodes = { NULL, NULL, NULL }
// version_directive_copy *yaml_version_directive_t = NULL
// struct {
// start *yaml_tag_directive_t
// end *yaml_tag_directive_t
// top *yaml_tag_directive_t
// } tag_directives_copy = { NULL, NULL, NULL }
// value yaml_tag_directive_t = { NULL, NULL }
// mark yaml_mark_t = { 0, 0, 0 }
//
// assert(document) // Non-NULL document object is expected.
// assert((tag_directives_start && tag_directives_end) ||
// (tag_directives_start == tag_directives_end))
// // Valid tag directives are expected.
//
// if (!STACK_INIT(&context, nodes, INITIAL_STACK_SIZE)) goto error
//
// if (version_directive) {
// version_directive_copy = yaml_malloc(sizeof(yaml_version_directive_t))
// if (!version_directive_copy) goto error
// version_directive_copy.major = version_directive.major
// version_directive_copy.minor = version_directive.minor
// }
//
// if (tag_directives_start != tag_directives_end) {
// tag_directive *yaml_tag_directive_t
// if (!STACK_INIT(&context, tag_directives_copy, INITIAL_STACK_SIZE))
// goto error
// for (tag_directive = tag_directives_start
// tag_directive != tag_directives_end; tag_directive ++) {
// assert(tag_directive.handle)
// assert(tag_directive.prefix)
// if (!yaml_check_utf8(tag_directive.handle,
// strlen((char *)tag_directive.handle)))
// goto error
// if (!yaml_check_utf8(tag_directive.prefix,
// strlen((char *)tag_directive.prefix)))
// goto error
// value.handle = yaml_strdup(tag_directive.handle)
// value.prefix = yaml_strdup(tag_directive.prefix)
// if (!value.handle || !value.prefix) goto error
// if (!PUSH(&context, tag_directives_copy, value))
// goto error
// value.handle = NULL
// value.prefix = NULL
// }
// }
//
// DOCUMENT_INIT(*document, nodes.start, nodes.end, version_directive_copy,
// tag_directives_copy.start, tag_directives_copy.top,
// start_implicit, end_implicit, mark, mark)
//
// return 1
//
//error:
// STACK_DEL(&context, nodes)
// yaml_free(version_directive_copy)
// while (!STACK_EMPTY(&context, tag_directives_copy)) {
// value yaml_tag_directive_t = POP(&context, tag_directives_copy)
// yaml_free(value.handle)
// yaml_free(value.prefix)
// }
// STACK_DEL(&context, tag_directives_copy)
// yaml_free(value.handle)
// yaml_free(value.prefix)
//
// return 0
//}
//
///*
// * Destroy a document object.
// */
//
//YAML_DECLARE(void)
//yaml_document_delete(document *yaml_document_t)
//{
// struct {
// error yaml_error_type_t
// } context
// tag_directive *yaml_tag_directive_t
//
// context.error = YAML_NO_ERROR // Eliminate a compiler warning.
//
// assert(document) // Non-NULL document object is expected.
//
// while (!STACK_EMPTY(&context, document.nodes)) {
// node yaml_node_t = POP(&context, document.nodes)
// yaml_free(node.tag)
// switch (node.type) {
// case YAML_SCALAR_NODE:
// yaml_free(node.data.scalar.value)
// break
// case YAML_SEQUENCE_NODE:
// STACK_DEL(&context, node.data.sequence.items)
// break
// case YAML_MAPPING_NODE:
// STACK_DEL(&context, node.data.mapping.pairs)
// break
// default:
// assert(0) // Should not happen.
// }
// }
// STACK_DEL(&context, document.nodes)
//
// yaml_free(document.version_directive)
// for (tag_directive = document.tag_directives.start
// tag_directive != document.tag_directives.end
// tag_directive++) {
// yaml_free(tag_directive.handle)
// yaml_free(tag_directive.prefix)
// }
// yaml_free(document.tag_directives.start)
//
// memset(document, 0, sizeof(yaml_document_t))
//}
//
///**
// * Get a document node.
// */
//
//YAML_DECLARE(yaml_node_t *)
//yaml_document_get_node(document *yaml_document_t, index int)
//{
// assert(document) // Non-NULL document object is expected.
//
// if (index > 0 && document.nodes.start + index <= document.nodes.top) {
// return document.nodes.start + index - 1
// }
// return NULL
//}
//
///**
// * Get the root object.
// */
//
//YAML_DECLARE(yaml_node_t *)
//yaml_document_get_root_node(document *yaml_document_t)
//{
// assert(document) // Non-NULL document object is expected.
//
// if (document.nodes.top != document.nodes.start) {
// return document.nodes.start
// }
// return NULL
//}
//
///*
// * Add a scalar node to a document.
// */
//
//YAML_DECLARE(int)
//yaml_document_add_scalar(document *yaml_document_t,
// tag *yaml_char_t, value *yaml_char_t, length int,
// style yaml_scalar_style_t)
//{
// struct {
// error yaml_error_type_t
// } context
// mark yaml_mark_t = { 0, 0, 0 }
// tag_copy *yaml_char_t = NULL
// value_copy *yaml_char_t = NULL
// node yaml_node_t
//
// assert(document) // Non-NULL document object is expected.
// assert(value) // Non-NULL value is expected.
//
// if (!tag) {
// tag = (yaml_char_t *)YAML_DEFAULT_SCALAR_TAG
// }
//
// if (!yaml_check_utf8(tag, strlen((char *)tag))) goto error
// tag_copy = yaml_strdup(tag)
// if (!tag_copy) goto error
//
// if (length < 0) {
// length = strlen((char *)value)
// }
//
// if (!yaml_check_utf8(value, length)) goto error
// value_copy = yaml_malloc(length+1)
// if (!value_copy) goto error
// memcpy(value_copy, value, length)
// value_copy[length] = '\0'
//
// SCALAR_NODE_INIT(node, tag_copy, value_copy, length, style, mark, mark)
// if (!PUSH(&context, document.nodes, node)) goto error
//
// return document.nodes.top - document.nodes.start
//
//error:
// yaml_free(tag_copy)
// yaml_free(value_copy)
//
// return 0
//}
//
///*
// * Add a sequence node to a document.
// */
//
//YAML_DECLARE(int)
//yaml_document_add_sequence(document *yaml_document_t,
// tag *yaml_char_t, style yaml_sequence_style_t)
//{
// struct {
// error yaml_error_type_t
// } context
// mark yaml_mark_t = { 0, 0, 0 }
// tag_copy *yaml_char_t = NULL
// struct {
// start *yaml_node_item_t
// end *yaml_node_item_t
// top *yaml_node_item_t
// } items = { NULL, NULL, NULL }
// node yaml_node_t
//
// assert(document) // Non-NULL document object is expected.
//
// if (!tag) {
// tag = (yaml_char_t *)YAML_DEFAULT_SEQUENCE_TAG
// }
//
// if (!yaml_check_utf8(tag, strlen((char *)tag))) goto error
// tag_copy = yaml_strdup(tag)
// if (!tag_copy) goto error
//
// if (!STACK_INIT(&context, items, INITIAL_STACK_SIZE)) goto error
//
// SEQUENCE_NODE_INIT(node, tag_copy, items.start, items.end,
// style, mark, mark)
// if (!PUSH(&context, document.nodes, node)) goto error
//
// return document.nodes.top - document.nodes.start
//
//error:
// STACK_DEL(&context, items)
// yaml_free(tag_copy)
//
// return 0
//}
//
///*
// * Add a mapping node to a document.
// */
//
//YAML_DECLARE(int)
//yaml_document_add_mapping(document *yaml_document_t,
// tag *yaml_char_t, style yaml_mapping_style_t)
//{
// struct {
// error yaml_error_type_t
// } context
// mark yaml_mark_t = { 0, 0, 0 }
// tag_copy *yaml_char_t = NULL
// struct {
// start *yaml_node_pair_t
// end *yaml_node_pair_t
// top *yaml_node_pair_t
// } pairs = { NULL, NULL, NULL }
// node yaml_node_t
//
// assert(document) // Non-NULL document object is expected.
//
// if (!tag) {
// tag = (yaml_char_t *)YAML_DEFAULT_MAPPING_TAG
// }
//
// if (!yaml_check_utf8(tag, strlen((char *)tag))) goto error
// tag_copy = yaml_strdup(tag)
// if (!tag_copy) goto error
//
// if (!STACK_INIT(&context, pairs, INITIAL_STACK_SIZE)) goto error
//
// MAPPING_NODE_INIT(node, tag_copy, pairs.start, pairs.end,
// style, mark, mark)
// if (!PUSH(&context, document.nodes, node)) goto error
//
// return document.nodes.top - document.nodes.start
//
//error:
// STACK_DEL(&context, pairs)
// yaml_free(tag_copy)
//
// return 0
//}
//
///*
// * Append an item to a sequence node.
// */
//
//YAML_DECLARE(int)
//yaml_document_append_sequence_item(document *yaml_document_t,
// sequence int, item int)
//{
// struct {
// error yaml_error_type_t
// } context
//
// assert(document) // Non-NULL document is required.
// assert(sequence > 0
// && document.nodes.start + sequence <= document.nodes.top)
// // Valid sequence id is required.
// assert(document.nodes.start[sequence-1].type == YAML_SEQUENCE_NODE)
// // A sequence node is required.
// assert(item > 0 && document.nodes.start + item <= document.nodes.top)
// // Valid item id is required.
//
// if (!PUSH(&context,
// document.nodes.start[sequence-1].data.sequence.items, item))
// return 0
//
// return 1
//}
//
///*
// * Append a pair of a key and a value to a mapping node.
// */
//
//YAML_DECLARE(int)
//yaml_document_append_mapping_pair(document *yaml_document_t,
// mapping int, key int, value int)
//{
// struct {
// error yaml_error_type_t
// } context
//
// pair yaml_node_pair_t
//
// assert(document) // Non-NULL document is required.
// assert(mapping > 0
// && document.nodes.start + mapping <= document.nodes.top)
// // Valid mapping id is required.
// assert(document.nodes.start[mapping-1].type == YAML_MAPPING_NODE)
// // A mapping node is required.
// assert(key > 0 && document.nodes.start + key <= document.nodes.top)
// // Valid key id is required.
// assert(value > 0 && document.nodes.start + value <= document.nodes.top)
// // Valid value id is required.
//
// pair.key = key
// pair.value = value
//
// if (!PUSH(&context,
// document.nodes.start[mapping-1].data.mapping.pairs, pair))
// return 0
//
// return 1
//}
//
//
+1018
View File
File diff suppressed because it is too large Load Diff
+2054
View File
File diff suppressed because it is too large Load Diff
+577
View File
@@ -0,0 +1,577 @@
//
// Copyright (c) 2011-2019 Canonical Ltd
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package yaml
import (
"encoding"
"fmt"
"io"
"reflect"
"regexp"
"sort"
"strconv"
"strings"
"time"
"unicode/utf8"
)
type encoder struct {
emitter yaml_emitter_t
event yaml_event_t
out []byte
flow bool
indent int
doneInit bool
}
func newEncoder() *encoder {
e := &encoder{}
yaml_emitter_initialize(&e.emitter)
yaml_emitter_set_output_string(&e.emitter, &e.out)
yaml_emitter_set_unicode(&e.emitter, true)
return e
}
func newEncoderWithWriter(w io.Writer) *encoder {
e := &encoder{}
yaml_emitter_initialize(&e.emitter)
yaml_emitter_set_output_writer(&e.emitter, w)
yaml_emitter_set_unicode(&e.emitter, true)
return e
}
func (e *encoder) init() {
if e.doneInit {
return
}
if e.indent == 0 {
e.indent = 4
}
e.emitter.best_indent = e.indent
yaml_stream_start_event_initialize(&e.event, yaml_UTF8_ENCODING)
e.emit()
e.doneInit = true
}
func (e *encoder) finish() {
e.emitter.open_ended = false
yaml_stream_end_event_initialize(&e.event)
e.emit()
}
func (e *encoder) destroy() {
yaml_emitter_delete(&e.emitter)
}
func (e *encoder) emit() {
// This will internally delete the e.event value.
e.must(yaml_emitter_emit(&e.emitter, &e.event))
}
func (e *encoder) must(ok bool) {
if !ok {
msg := e.emitter.problem
if msg == "" {
msg = "unknown problem generating YAML content"
}
failf("%s", msg)
}
}
func (e *encoder) marshalDoc(tag string, in reflect.Value) {
e.init()
var node *Node
if in.IsValid() {
node, _ = in.Interface().(*Node)
}
if node != nil && node.Kind == DocumentNode {
e.nodev(in)
} else {
yaml_document_start_event_initialize(&e.event, nil, nil, true)
e.emit()
e.marshal(tag, in)
yaml_document_end_event_initialize(&e.event, true)
e.emit()
}
}
func (e *encoder) marshal(tag string, in reflect.Value) {
tag = shortTag(tag)
if !in.IsValid() || in.Kind() == reflect.Ptr && in.IsNil() {
e.nilv()
return
}
iface := in.Interface()
switch value := iface.(type) {
case *Node:
e.nodev(in)
return
case Node:
if !in.CanAddr() {
var n = reflect.New(in.Type()).Elem()
n.Set(in)
in = n
}
e.nodev(in.Addr())
return
case time.Time:
e.timev(tag, in)
return
case *time.Time:
e.timev(tag, in.Elem())
return
case time.Duration:
e.stringv(tag, reflect.ValueOf(value.String()))
return
case Marshaler:
v, err := value.MarshalYAML()
if err != nil {
fail(err)
}
if v == nil {
e.nilv()
return
}
e.marshal(tag, reflect.ValueOf(v))
return
case encoding.TextMarshaler:
text, err := value.MarshalText()
if err != nil {
fail(err)
}
in = reflect.ValueOf(string(text))
case nil:
e.nilv()
return
}
switch in.Kind() {
case reflect.Interface:
e.marshal(tag, in.Elem())
case reflect.Map:
e.mapv(tag, in)
case reflect.Ptr:
e.marshal(tag, in.Elem())
case reflect.Struct:
e.structv(tag, in)
case reflect.Slice, reflect.Array:
e.slicev(tag, in)
case reflect.String:
e.stringv(tag, in)
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
e.intv(tag, in)
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr:
e.uintv(tag, in)
case reflect.Float32, reflect.Float64:
e.floatv(tag, in)
case reflect.Bool:
e.boolv(tag, in)
default:
panic("cannot marshal type: " + in.Type().String())
}
}
func (e *encoder) mapv(tag string, in reflect.Value) {
e.mappingv(tag, func() {
keys := keyList(in.MapKeys())
sort.Sort(keys)
for _, k := range keys {
e.marshal("", k)
e.marshal("", in.MapIndex(k))
}
})
}
func (e *encoder) fieldByIndex(v reflect.Value, index []int) (field reflect.Value) {
for _, num := range index {
for {
if v.Kind() == reflect.Ptr {
if v.IsNil() {
return reflect.Value{}
}
v = v.Elem()
continue
}
break
}
v = v.Field(num)
}
return v
}
func (e *encoder) structv(tag string, in reflect.Value) {
sinfo, err := getStructInfo(in.Type())
if err != nil {
panic(err)
}
e.mappingv(tag, func() {
for _, info := range sinfo.FieldsList {
var value reflect.Value
if info.Inline == nil {
value = in.Field(info.Num)
} else {
value = e.fieldByIndex(in, info.Inline)
if !value.IsValid() {
continue
}
}
if info.OmitEmpty && isZero(value) {
continue
}
e.marshal("", reflect.ValueOf(info.Key))
e.flow = info.Flow
e.marshal("", value)
}
if sinfo.InlineMap >= 0 {
m := in.Field(sinfo.InlineMap)
if m.Len() > 0 {
e.flow = false
keys := keyList(m.MapKeys())
sort.Sort(keys)
for _, k := range keys {
if _, found := sinfo.FieldsMap[k.String()]; found {
panic(fmt.Sprintf("cannot have key %q in inlined map: conflicts with struct field", k.String()))
}
e.marshal("", k)
e.flow = false
e.marshal("", m.MapIndex(k))
}
}
}
})
}
func (e *encoder) mappingv(tag string, f func()) {
implicit := tag == ""
style := yaml_BLOCK_MAPPING_STYLE
if e.flow {
e.flow = false
style = yaml_FLOW_MAPPING_STYLE
}
yaml_mapping_start_event_initialize(&e.event, nil, []byte(tag), implicit, style)
e.emit()
f()
yaml_mapping_end_event_initialize(&e.event)
e.emit()
}
func (e *encoder) slicev(tag string, in reflect.Value) {
implicit := tag == ""
style := yaml_BLOCK_SEQUENCE_STYLE
if e.flow {
e.flow = false
style = yaml_FLOW_SEQUENCE_STYLE
}
e.must(yaml_sequence_start_event_initialize(&e.event, nil, []byte(tag), implicit, style))
e.emit()
n := in.Len()
for i := 0; i < n; i++ {
e.marshal("", in.Index(i))
}
e.must(yaml_sequence_end_event_initialize(&e.event))
e.emit()
}
// isBase60 returns whether s is in base 60 notation as defined in YAML 1.1.
//
// The base 60 float notation in YAML 1.1 is a terrible idea and is unsupported
// in YAML 1.2 and by this package, but these should be marshalled quoted for
// the time being for compatibility with other parsers.
func isBase60Float(s string) (result bool) {
// Fast path.
if s == "" {
return false
}
c := s[0]
if !(c == '+' || c == '-' || c >= '0' && c <= '9') || strings.IndexByte(s, ':') < 0 {
return false
}
// Do the full match.
return base60float.MatchString(s)
}
// From http://yaml.org/type/float.html, except the regular expression there
// is bogus. In practice parsers do not enforce the "\.[0-9_]*" suffix.
var base60float = regexp.MustCompile(`^[-+]?[0-9][0-9_]*(?::[0-5]?[0-9])+(?:\.[0-9_]*)?$`)
// isOldBool returns whether s is bool notation as defined in YAML 1.1.
//
// We continue to force strings that YAML 1.1 would interpret as booleans to be
// rendered as quotes strings so that the marshalled output valid for YAML 1.1
// parsing.
func isOldBool(s string) (result bool) {
switch s {
case "y", "Y", "yes", "Yes", "YES", "on", "On", "ON",
"n", "N", "no", "No", "NO", "off", "Off", "OFF":
return true
default:
return false
}
}
func (e *encoder) stringv(tag string, in reflect.Value) {
var style yaml_scalar_style_t
s := in.String()
canUsePlain := true
switch {
case !utf8.ValidString(s):
if tag == binaryTag {
failf("explicitly tagged !!binary data must be base64-encoded")
}
if tag != "" {
failf("cannot marshal invalid UTF-8 data as %s", shortTag(tag))
}
// It can't be encoded directly as YAML so use a binary tag
// and encode it as base64.
tag = binaryTag
s = encodeBase64(s)
case tag == "":
// Check to see if it would resolve to a specific
// tag when encoded unquoted. If it doesn't,
// there's no need to quote it.
rtag, _ := resolve("", s)
canUsePlain = rtag == strTag && !(isBase60Float(s) || isOldBool(s))
}
// Note: it's possible for user code to emit invalid YAML
// if they explicitly specify a tag and a string containing
// text that's incompatible with that tag.
switch {
case strings.Contains(s, "\n"):
if e.flow {
style = yaml_DOUBLE_QUOTED_SCALAR_STYLE
} else {
style = yaml_LITERAL_SCALAR_STYLE
}
case canUsePlain:
style = yaml_PLAIN_SCALAR_STYLE
default:
style = yaml_DOUBLE_QUOTED_SCALAR_STYLE
}
e.emitScalar(s, "", tag, style, nil, nil, nil, nil)
}
func (e *encoder) boolv(tag string, in reflect.Value) {
var s string
if in.Bool() {
s = "true"
} else {
s = "false"
}
e.emitScalar(s, "", tag, yaml_PLAIN_SCALAR_STYLE, nil, nil, nil, nil)
}
func (e *encoder) intv(tag string, in reflect.Value) {
s := strconv.FormatInt(in.Int(), 10)
e.emitScalar(s, "", tag, yaml_PLAIN_SCALAR_STYLE, nil, nil, nil, nil)
}
func (e *encoder) uintv(tag string, in reflect.Value) {
s := strconv.FormatUint(in.Uint(), 10)
e.emitScalar(s, "", tag, yaml_PLAIN_SCALAR_STYLE, nil, nil, nil, nil)
}
func (e *encoder) timev(tag string, in reflect.Value) {
t := in.Interface().(time.Time)
s := t.Format(time.RFC3339Nano)
e.emitScalar(s, "", tag, yaml_PLAIN_SCALAR_STYLE, nil, nil, nil, nil)
}
func (e *encoder) floatv(tag string, in reflect.Value) {
// Issue #352: When formatting, use the precision of the underlying value
precision := 64
if in.Kind() == reflect.Float32 {
precision = 32
}
s := strconv.FormatFloat(in.Float(), 'g', -1, precision)
switch s {
case "+Inf":
s = ".inf"
case "-Inf":
s = "-.inf"
case "NaN":
s = ".nan"
}
e.emitScalar(s, "", tag, yaml_PLAIN_SCALAR_STYLE, nil, nil, nil, nil)
}
func (e *encoder) nilv() {
e.emitScalar("null", "", "", yaml_PLAIN_SCALAR_STYLE, nil, nil, nil, nil)
}
func (e *encoder) emitScalar(value, anchor, tag string, style yaml_scalar_style_t, head, line, foot, tail []byte) {
// TODO Kill this function. Replace all initialize calls by their underlining Go literals.
implicit := tag == ""
if !implicit {
tag = longTag(tag)
}
e.must(yaml_scalar_event_initialize(&e.event, []byte(anchor), []byte(tag), []byte(value), implicit, implicit, style))
e.event.head_comment = head
e.event.line_comment = line
e.event.foot_comment = foot
e.event.tail_comment = tail
e.emit()
}
func (e *encoder) nodev(in reflect.Value) {
e.node(in.Interface().(*Node), "")
}
func (e *encoder) node(node *Node, tail string) {
// Zero nodes behave as nil.
if node.Kind == 0 && node.IsZero() {
e.nilv()
return
}
// If the tag was not explicitly requested, and dropping it won't change the
// implicit tag of the value, don't include it in the presentation.
var tag = node.Tag
var stag = shortTag(tag)
var forceQuoting bool
if tag != "" && node.Style&TaggedStyle == 0 {
if node.Kind == ScalarNode {
if stag == strTag && node.Style&(SingleQuotedStyle|DoubleQuotedStyle|LiteralStyle|FoldedStyle) != 0 {
tag = ""
} else {
rtag, _ := resolve("", node.Value)
if rtag == stag {
tag = ""
} else if stag == strTag {
tag = ""
forceQuoting = true
}
}
} else {
var rtag string
switch node.Kind {
case MappingNode:
rtag = mapTag
case SequenceNode:
rtag = seqTag
}
if rtag == stag {
tag = ""
}
}
}
switch node.Kind {
case DocumentNode:
yaml_document_start_event_initialize(&e.event, nil, nil, true)
e.event.head_comment = []byte(node.HeadComment)
e.emit()
for _, node := range node.Content {
e.node(node, "")
}
yaml_document_end_event_initialize(&e.event, true)
e.event.foot_comment = []byte(node.FootComment)
e.emit()
case SequenceNode:
style := yaml_BLOCK_SEQUENCE_STYLE
if node.Style&FlowStyle != 0 {
style = yaml_FLOW_SEQUENCE_STYLE
}
e.must(yaml_sequence_start_event_initialize(&e.event, []byte(node.Anchor), []byte(longTag(tag)), tag == "", style))
e.event.head_comment = []byte(node.HeadComment)
e.emit()
for _, node := range node.Content {
e.node(node, "")
}
e.must(yaml_sequence_end_event_initialize(&e.event))
e.event.line_comment = []byte(node.LineComment)
e.event.foot_comment = []byte(node.FootComment)
e.emit()
case MappingNode:
style := yaml_BLOCK_MAPPING_STYLE
if node.Style&FlowStyle != 0 {
style = yaml_FLOW_MAPPING_STYLE
}
yaml_mapping_start_event_initialize(&e.event, []byte(node.Anchor), []byte(longTag(tag)), tag == "", style)
e.event.tail_comment = []byte(tail)
e.event.head_comment = []byte(node.HeadComment)
e.emit()
// The tail logic below moves the foot comment of prior keys to the following key,
// since the value for each key may be a nested structure and the foot needs to be
// processed only the entirety of the value is streamed. The last tail is processed
// with the mapping end event.
var tail string
for i := 0; i+1 < len(node.Content); i += 2 {
k := node.Content[i]
foot := k.FootComment
if foot != "" {
kopy := *k
kopy.FootComment = ""
k = &kopy
}
e.node(k, tail)
tail = foot
v := node.Content[i+1]
e.node(v, "")
}
yaml_mapping_end_event_initialize(&e.event)
e.event.tail_comment = []byte(tail)
e.event.line_comment = []byte(node.LineComment)
e.event.foot_comment = []byte(node.FootComment)
e.emit()
case AliasNode:
yaml_alias_event_initialize(&e.event, []byte(node.Value))
e.event.head_comment = []byte(node.HeadComment)
e.event.line_comment = []byte(node.LineComment)
e.event.foot_comment = []byte(node.FootComment)
e.emit()
case ScalarNode:
value := node.Value
if !utf8.ValidString(value) {
if stag == binaryTag {
failf("explicitly tagged !!binary data must be base64-encoded")
}
if stag != "" {
failf("cannot marshal invalid UTF-8 data as %s", stag)
}
// It can't be encoded directly as YAML so use a binary tag
// and encode it as base64.
tag = binaryTag
value = encodeBase64(value)
}
style := yaml_PLAIN_SCALAR_STYLE
switch {
case node.Style&DoubleQuotedStyle != 0:
style = yaml_DOUBLE_QUOTED_SCALAR_STYLE
case node.Style&SingleQuotedStyle != 0:
style = yaml_SINGLE_QUOTED_SCALAR_STYLE
case node.Style&LiteralStyle != 0:
style = yaml_LITERAL_SCALAR_STYLE
case node.Style&FoldedStyle != 0:
style = yaml_FOLDED_SCALAR_STYLE
case strings.Contains(value, "\n"):
style = yaml_LITERAL_SCALAR_STYLE
case forceQuoting:
style = yaml_DOUBLE_QUOTED_SCALAR_STYLE
}
e.emitScalar(value, node.Anchor, tag, style, []byte(node.HeadComment), []byte(node.LineComment), []byte(node.FootComment), []byte(tail))
default:
failf("cannot encode node with unknown kind %d", node.Kind)
}
}
+1274
View File
File diff suppressed because it is too large Load Diff
+434
View File
@@ -0,0 +1,434 @@
//
// Copyright (c) 2011-2019 Canonical Ltd
// Copyright (c) 2006-2010 Kirill Simonov
//
// Permission is hereby granted, free of charge, to any person obtaining a copy of
// this software and associated documentation files (the "Software"), to deal in
// the Software without restriction, including without limitation the rights to
// use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
// of the Software, and to permit persons to whom the Software is furnished to do
// so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in all
// copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
// SOFTWARE.
package yaml
import (
"io"
)
// Set the reader error and return 0.
func yaml_parser_set_reader_error(parser *yaml_parser_t, problem string, offset int, value int) bool {
parser.error = yaml_READER_ERROR
parser.problem = problem
parser.problem_offset = offset
parser.problem_value = value
return false
}
// Byte order marks.
const (
bom_UTF8 = "\xef\xbb\xbf"
bom_UTF16LE = "\xff\xfe"
bom_UTF16BE = "\xfe\xff"
)
// Determine the input stream encoding by checking the BOM symbol. If no BOM is
// found, the UTF-8 encoding is assumed. Return 1 on success, 0 on failure.
func yaml_parser_determine_encoding(parser *yaml_parser_t) bool {
// Ensure that we had enough bytes in the raw buffer.
for !parser.eof && len(parser.raw_buffer)-parser.raw_buffer_pos < 3 {
if !yaml_parser_update_raw_buffer(parser) {
return false
}
}
// Determine the encoding.
buf := parser.raw_buffer
pos := parser.raw_buffer_pos
avail := len(buf) - pos
if avail >= 2 && buf[pos] == bom_UTF16LE[0] && buf[pos+1] == bom_UTF16LE[1] {
parser.encoding = yaml_UTF16LE_ENCODING
parser.raw_buffer_pos += 2
parser.offset += 2
} else if avail >= 2 && buf[pos] == bom_UTF16BE[0] && buf[pos+1] == bom_UTF16BE[1] {
parser.encoding = yaml_UTF16BE_ENCODING
parser.raw_buffer_pos += 2
parser.offset += 2
} else if avail >= 3 && buf[pos] == bom_UTF8[0] && buf[pos+1] == bom_UTF8[1] && buf[pos+2] == bom_UTF8[2] {
parser.encoding = yaml_UTF8_ENCODING
parser.raw_buffer_pos += 3
parser.offset += 3
} else {
parser.encoding = yaml_UTF8_ENCODING
}
return true
}
// Update the raw buffer.
func yaml_parser_update_raw_buffer(parser *yaml_parser_t) bool {
size_read := 0
// Return if the raw buffer is full.
if parser.raw_buffer_pos == 0 && len(parser.raw_buffer) == cap(parser.raw_buffer) {
return true
}
// Return on EOF.
if parser.eof {
return true
}
// Move the remaining bytes in the raw buffer to the beginning.
if parser.raw_buffer_pos > 0 && parser.raw_buffer_pos < len(parser.raw_buffer) {
copy(parser.raw_buffer, parser.raw_buffer[parser.raw_buffer_pos:])
}
parser.raw_buffer = parser.raw_buffer[:len(parser.raw_buffer)-parser.raw_buffer_pos]
parser.raw_buffer_pos = 0
// Call the read handler to fill the buffer.
size_read, err := parser.read_handler(parser, parser.raw_buffer[len(parser.raw_buffer):cap(parser.raw_buffer)])
parser.raw_buffer = parser.raw_buffer[:len(parser.raw_buffer)+size_read]
if err == io.EOF {
parser.eof = true
} else if err != nil {
return yaml_parser_set_reader_error(parser, "input error: "+err.Error(), parser.offset, -1)
}
return true
}
// Ensure that the buffer contains at least `length` characters.
// Return true on success, false on failure.
//
// The length is supposed to be significantly less that the buffer size.
func yaml_parser_update_buffer(parser *yaml_parser_t, length int) bool {
if parser.read_handler == nil {
panic("read handler must be set")
}
// [Go] This function was changed to guarantee the requested length size at EOF.
// The fact we need to do this is pretty awful, but the description above implies
// for that to be the case, and there are tests
// If the EOF flag is set and the raw buffer is empty, do nothing.
if parser.eof && parser.raw_buffer_pos == len(parser.raw_buffer) {
// [Go] ACTUALLY! Read the documentation of this function above.
// This is just broken. To return true, we need to have the
// given length in the buffer. Not doing that means every single
// check that calls this function to make sure the buffer has a
// given length is Go) panicking; or C) accessing invalid memory.
//return true
}
// Return if the buffer contains enough characters.
if parser.unread >= length {
return true
}
// Determine the input encoding if it is not known yet.
if parser.encoding == yaml_ANY_ENCODING {
if !yaml_parser_determine_encoding(parser) {
return false
}
}
// Move the unread characters to the beginning of the buffer.
buffer_len := len(parser.buffer)
if parser.buffer_pos > 0 && parser.buffer_pos < buffer_len {
copy(parser.buffer, parser.buffer[parser.buffer_pos:])
buffer_len -= parser.buffer_pos
parser.buffer_pos = 0
} else if parser.buffer_pos == buffer_len {
buffer_len = 0
parser.buffer_pos = 0
}
// Open the whole buffer for writing, and cut it before returning.
parser.buffer = parser.buffer[:cap(parser.buffer)]
// Fill the buffer until it has enough characters.
first := true
for parser.unread < length {
// Fill the raw buffer if necessary.
if !first || parser.raw_buffer_pos == len(parser.raw_buffer) {
if !yaml_parser_update_raw_buffer(parser) {
parser.buffer = parser.buffer[:buffer_len]
return false
}
}
first = false
// Decode the raw buffer.
inner:
for parser.raw_buffer_pos != len(parser.raw_buffer) {
var value rune
var width int
raw_unread := len(parser.raw_buffer) - parser.raw_buffer_pos
// Decode the next character.
switch parser.encoding {
case yaml_UTF8_ENCODING:
// Decode a UTF-8 character. Check RFC 3629
// (http://www.ietf.org/rfc/rfc3629.txt) for more details.
//
// The following table (taken from the RFC) is used for
// decoding.
//
// Char. number range | UTF-8 octet sequence
// (hexadecimal) | (binary)
// --------------------+------------------------------------
// 0000 0000-0000 007F | 0xxxxxxx
// 0000 0080-0000 07FF | 110xxxxx 10xxxxxx
// 0000 0800-0000 FFFF | 1110xxxx 10xxxxxx 10xxxxxx
// 0001 0000-0010 FFFF | 11110xxx 10xxxxxx 10xxxxxx 10xxxxxx
//
// Additionally, the characters in the range 0xD800-0xDFFF
// are prohibited as they are reserved for use with UTF-16
// surrogate pairs.
// Determine the length of the UTF-8 sequence.
octet := parser.raw_buffer[parser.raw_buffer_pos]
switch {
case octet&0x80 == 0x00:
width = 1
case octet&0xE0 == 0xC0:
width = 2
case octet&0xF0 == 0xE0:
width = 3
case octet&0xF8 == 0xF0:
width = 4
default:
// The leading octet is invalid.
return yaml_parser_set_reader_error(parser,
"invalid leading UTF-8 octet",
parser.offset, int(octet))
}
// Check if the raw buffer contains an incomplete character.
if width > raw_unread {
if parser.eof {
return yaml_parser_set_reader_error(parser,
"incomplete UTF-8 octet sequence",
parser.offset, -1)
}
break inner
}
// Decode the leading octet.
switch {
case octet&0x80 == 0x00:
value = rune(octet & 0x7F)
case octet&0xE0 == 0xC0:
value = rune(octet & 0x1F)
case octet&0xF0 == 0xE0:
value = rune(octet & 0x0F)
case octet&0xF8 == 0xF0:
value = rune(octet & 0x07)
default:
value = 0
}
// Check and decode the trailing octets.
for k := 1; k < width; k++ {
octet = parser.raw_buffer[parser.raw_buffer_pos+k]
// Check if the octet is valid.
if (octet & 0xC0) != 0x80 {
return yaml_parser_set_reader_error(parser,
"invalid trailing UTF-8 octet",
parser.offset+k, int(octet))
}
// Decode the octet.
value = (value << 6) + rune(octet&0x3F)
}
// Check the length of the sequence against the value.
switch {
case width == 1:
case width == 2 && value >= 0x80:
case width == 3 && value >= 0x800:
case width == 4 && value >= 0x10000:
default:
return yaml_parser_set_reader_error(parser,
"invalid length of a UTF-8 sequence",
parser.offset, -1)
}
// Check the range of the value.
if value >= 0xD800 && value <= 0xDFFF || value > 0x10FFFF {
return yaml_parser_set_reader_error(parser,
"invalid Unicode character",
parser.offset, int(value))
}
case yaml_UTF16LE_ENCODING, yaml_UTF16BE_ENCODING:
var low, high int
if parser.encoding == yaml_UTF16LE_ENCODING {
low, high = 0, 1
} else {
low, high = 1, 0
}
// The UTF-16 encoding is not as simple as one might
// naively think. Check RFC 2781
// (http://www.ietf.org/rfc/rfc2781.txt).
//
// Normally, two subsequent bytes describe a Unicode
// character. However a special technique (called a
// surrogate pair) is used for specifying character
// values larger than 0xFFFF.
//
// A surrogate pair consists of two pseudo-characters:
// high surrogate area (0xD800-0xDBFF)
// low surrogate area (0xDC00-0xDFFF)
//
// The following formulas are used for decoding
// and encoding characters using surrogate pairs:
//
// U = U' + 0x10000 (0x01 00 00 <= U <= 0x10 FF FF)
// U' = yyyyyyyyyyxxxxxxxxxx (0 <= U' <= 0x0F FF FF)
// W1 = 110110yyyyyyyyyy
// W2 = 110111xxxxxxxxxx
//
// where U is the character value, W1 is the high surrogate
// area, W2 is the low surrogate area.
// Check for incomplete UTF-16 character.
if raw_unread < 2 {
if parser.eof {
return yaml_parser_set_reader_error(parser,
"incomplete UTF-16 character",
parser.offset, -1)
}
break inner
}
// Get the character.
value = rune(parser.raw_buffer[parser.raw_buffer_pos+low]) +
(rune(parser.raw_buffer[parser.raw_buffer_pos+high]) << 8)
// Check for unexpected low surrogate area.
if value&0xFC00 == 0xDC00 {
return yaml_parser_set_reader_error(parser,
"unexpected low surrogate area",
parser.offset, int(value))
}
// Check for a high surrogate area.
if value&0xFC00 == 0xD800 {
width = 4
// Check for incomplete surrogate pair.
if raw_unread < 4 {
if parser.eof {
return yaml_parser_set_reader_error(parser,
"incomplete UTF-16 surrogate pair",
parser.offset, -1)
}
break inner
}
// Get the next character.
value2 := rune(parser.raw_buffer[parser.raw_buffer_pos+low+2]) +
(rune(parser.raw_buffer[parser.raw_buffer_pos+high+2]) << 8)
// Check for a low surrogate area.
if value2&0xFC00 != 0xDC00 {
return yaml_parser_set_reader_error(parser,
"expected low surrogate area",
parser.offset+2, int(value2))
}
// Generate the value of the surrogate pair.
value = 0x10000 + ((value & 0x3FF) << 10) + (value2 & 0x3FF)
} else {
width = 2
}
default:
panic("impossible")
}
// Check if the character is in the allowed range:
// #x9 | #xA | #xD | [#x20-#x7E] (8 bit)
// | #x85 | [#xA0-#xD7FF] | [#xE000-#xFFFD] (16 bit)
// | [#x10000-#x10FFFF] (32 bit)
switch {
case value == 0x09:
case value == 0x0A:
case value == 0x0D:
case value >= 0x20 && value <= 0x7E:
case value == 0x85:
case value >= 0xA0 && value <= 0xD7FF:
case value >= 0xE000 && value <= 0xFFFD:
case value >= 0x10000 && value <= 0x10FFFF:
default:
return yaml_parser_set_reader_error(parser,
"control characters are not allowed",
parser.offset, int(value))
}
// Move the raw pointers.
parser.raw_buffer_pos += width
parser.offset += width
// Finally put the character into the buffer.
if value <= 0x7F {
// 0000 0000-0000 007F . 0xxxxxxx
parser.buffer[buffer_len+0] = byte(value)
buffer_len += 1
} else if value <= 0x7FF {
// 0000 0080-0000 07FF . 110xxxxx 10xxxxxx
parser.buffer[buffer_len+0] = byte(0xC0 + (value >> 6))
parser.buffer[buffer_len+1] = byte(0x80 + (value & 0x3F))
buffer_len += 2
} else if value <= 0xFFFF {
// 0000 0800-0000 FFFF . 1110xxxx 10xxxxxx 10xxxxxx
parser.buffer[buffer_len+0] = byte(0xE0 + (value >> 12))
parser.buffer[buffer_len+1] = byte(0x80 + ((value >> 6) & 0x3F))
parser.buffer[buffer_len+2] = byte(0x80 + (value & 0x3F))
buffer_len += 3
} else {
// 0001 0000-0010 FFFF . 11110xxx 10xxxxxx 10xxxxxx 10xxxxxx
parser.buffer[buffer_len+0] = byte(0xF0 + (value >> 18))
parser.buffer[buffer_len+1] = byte(0x80 + ((value >> 12) & 0x3F))
parser.buffer[buffer_len+2] = byte(0x80 + ((value >> 6) & 0x3F))
parser.buffer[buffer_len+3] = byte(0x80 + (value & 0x3F))
buffer_len += 4
}
parser.unread++
}
// On EOF, put NUL into the buffer and return.
if parser.eof {
parser.buffer[buffer_len] = 0
buffer_len++
parser.unread++
break
}
}
// [Go] Read the documentation of this function above. To return true,
// we need to have the given length in the buffer. Not doing that means
// every single check that calls this function to make sure the buffer
// has a given length is Go) panicking; or C) accessing invalid memory.
// This happens here due to the EOF above breaking early.
for buffer_len < length {
parser.buffer[buffer_len] = 0
buffer_len++
}
parser.buffer = parser.buffer[:buffer_len]
return true
}
+326
View File
@@ -0,0 +1,326 @@
//
// Copyright (c) 2011-2019 Canonical Ltd
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package yaml
import (
"encoding/base64"
"math"
"regexp"
"strconv"
"strings"
"time"
)
type resolveMapItem struct {
value interface{}
tag string
}
var resolveTable = make([]byte, 256)
var resolveMap = make(map[string]resolveMapItem)
func init() {
t := resolveTable
t[int('+')] = 'S' // Sign
t[int('-')] = 'S'
for _, c := range "0123456789" {
t[int(c)] = 'D' // Digit
}
for _, c := range "yYnNtTfFoO~" {
t[int(c)] = 'M' // In map
}
t[int('.')] = '.' // Float (potentially in map)
var resolveMapList = []struct {
v interface{}
tag string
l []string
}{
{true, boolTag, []string{"true", "True", "TRUE"}},
{false, boolTag, []string{"false", "False", "FALSE"}},
{nil, nullTag, []string{"", "~", "null", "Null", "NULL"}},
{math.NaN(), floatTag, []string{".nan", ".NaN", ".NAN"}},
{math.Inf(+1), floatTag, []string{".inf", ".Inf", ".INF"}},
{math.Inf(+1), floatTag, []string{"+.inf", "+.Inf", "+.INF"}},
{math.Inf(-1), floatTag, []string{"-.inf", "-.Inf", "-.INF"}},
{"<<", mergeTag, []string{"<<"}},
}
m := resolveMap
for _, item := range resolveMapList {
for _, s := range item.l {
m[s] = resolveMapItem{item.v, item.tag}
}
}
}
const (
nullTag = "!!null"
boolTag = "!!bool"
strTag = "!!str"
intTag = "!!int"
floatTag = "!!float"
timestampTag = "!!timestamp"
seqTag = "!!seq"
mapTag = "!!map"
binaryTag = "!!binary"
mergeTag = "!!merge"
)
var longTags = make(map[string]string)
var shortTags = make(map[string]string)
func init() {
for _, stag := range []string{nullTag, boolTag, strTag, intTag, floatTag, timestampTag, seqTag, mapTag, binaryTag, mergeTag} {
ltag := longTag(stag)
longTags[stag] = ltag
shortTags[ltag] = stag
}
}
const longTagPrefix = "tag:yaml.org,2002:"
func shortTag(tag string) string {
if strings.HasPrefix(tag, longTagPrefix) {
if stag, ok := shortTags[tag]; ok {
return stag
}
return "!!" + tag[len(longTagPrefix):]
}
return tag
}
func longTag(tag string) string {
if strings.HasPrefix(tag, "!!") {
if ltag, ok := longTags[tag]; ok {
return ltag
}
return longTagPrefix + tag[2:]
}
return tag
}
func resolvableTag(tag string) bool {
switch tag {
case "", strTag, boolTag, intTag, floatTag, nullTag, timestampTag:
return true
}
return false
}
var yamlStyleFloat = regexp.MustCompile(`^[-+]?(\.[0-9]+|[0-9]+(\.[0-9]*)?)([eE][-+]?[0-9]+)?$`)
func resolve(tag string, in string) (rtag string, out interface{}) {
tag = shortTag(tag)
if !resolvableTag(tag) {
return tag, in
}
defer func() {
switch tag {
case "", rtag, strTag, binaryTag:
return
case floatTag:
if rtag == intTag {
switch v := out.(type) {
case int64:
rtag = floatTag
out = float64(v)
return
case int:
rtag = floatTag
out = float64(v)
return
}
}
}
failf("cannot decode %s `%s` as a %s", shortTag(rtag), in, shortTag(tag))
}()
// Any data is accepted as a !!str or !!binary.
// Otherwise, the prefix is enough of a hint about what it might be.
hint := byte('N')
if in != "" {
hint = resolveTable[in[0]]
}
if hint != 0 && tag != strTag && tag != binaryTag {
// Handle things we can lookup in a map.
if item, ok := resolveMap[in]; ok {
return item.tag, item.value
}
// Base 60 floats are a bad idea, were dropped in YAML 1.2, and
// are purposefully unsupported here. They're still quoted on
// the way out for compatibility with other parser, though.
switch hint {
case 'M':
// We've already checked the map above.
case '.':
// Not in the map, so maybe a normal float.
floatv, err := strconv.ParseFloat(in, 64)
if err == nil {
return floatTag, floatv
}
case 'D', 'S':
// Int, float, or timestamp.
// Only try values as a timestamp if the value is unquoted or there's an explicit
// !!timestamp tag.
if tag == "" || tag == timestampTag {
t, ok := parseTimestamp(in)
if ok {
return timestampTag, t
}
}
plain := strings.Replace(in, "_", "", -1)
intv, err := strconv.ParseInt(plain, 0, 64)
if err == nil {
if intv == int64(int(intv)) {
return intTag, int(intv)
} else {
return intTag, intv
}
}
uintv, err := strconv.ParseUint(plain, 0, 64)
if err == nil {
return intTag, uintv
}
if yamlStyleFloat.MatchString(plain) {
floatv, err := strconv.ParseFloat(plain, 64)
if err == nil {
return floatTag, floatv
}
}
if strings.HasPrefix(plain, "0b") {
intv, err := strconv.ParseInt(plain[2:], 2, 64)
if err == nil {
if intv == int64(int(intv)) {
return intTag, int(intv)
} else {
return intTag, intv
}
}
uintv, err := strconv.ParseUint(plain[2:], 2, 64)
if err == nil {
return intTag, uintv
}
} else if strings.HasPrefix(plain, "-0b") {
intv, err := strconv.ParseInt("-"+plain[3:], 2, 64)
if err == nil {
if true || intv == int64(int(intv)) {
return intTag, int(intv)
} else {
return intTag, intv
}
}
}
// Octals as introduced in version 1.2 of the spec.
// Octals from the 1.1 spec, spelled as 0777, are still
// decoded by default in v3 as well for compatibility.
// May be dropped in v4 depending on how usage evolves.
if strings.HasPrefix(plain, "0o") {
intv, err := strconv.ParseInt(plain[2:], 8, 64)
if err == nil {
if intv == int64(int(intv)) {
return intTag, int(intv)
} else {
return intTag, intv
}
}
uintv, err := strconv.ParseUint(plain[2:], 8, 64)
if err == nil {
return intTag, uintv
}
} else if strings.HasPrefix(plain, "-0o") {
intv, err := strconv.ParseInt("-"+plain[3:], 8, 64)
if err == nil {
if true || intv == int64(int(intv)) {
return intTag, int(intv)
} else {
return intTag, intv
}
}
}
default:
panic("internal error: missing handler for resolver table: " + string(rune(hint)) + " (with " + in + ")")
}
}
return strTag, in
}
// encodeBase64 encodes s as base64 that is broken up into multiple lines
// as appropriate for the resulting length.
func encodeBase64(s string) string {
const lineLen = 70
encLen := base64.StdEncoding.EncodedLen(len(s))
lines := encLen/lineLen + 1
buf := make([]byte, encLen*2+lines)
in := buf[0:encLen]
out := buf[encLen:]
base64.StdEncoding.Encode(in, []byte(s))
k := 0
for i := 0; i < len(in); i += lineLen {
j := i + lineLen
if j > len(in) {
j = len(in)
}
k += copy(out[k:], in[i:j])
if lines > 1 {
out[k] = '\n'
k++
}
}
return string(out[:k])
}
// This is a subset of the formats allowed by the regular expression
// defined at http://yaml.org/type/timestamp.html.
var allowedTimestampFormats = []string{
"2006-1-2T15:4:5.999999999Z07:00", // RCF3339Nano with short date fields.
"2006-1-2t15:4:5.999999999Z07:00", // RFC3339Nano with short date fields and lower-case "t".
"2006-1-2 15:4:5.999999999", // space separated with no time zone
"2006-1-2", // date only
// Notable exception: time.Parse cannot handle: "2001-12-14 21:59:43.10 -5"
// from the set of examples.
}
// parseTimestamp parses s as a timestamp string and
// returns the timestamp and reports whether it succeeded.
// Timestamp formats are defined at http://yaml.org/type/timestamp.html
func parseTimestamp(s string) (time.Time, bool) {
// TODO write code to check all the formats supported by
// http://yaml.org/type/timestamp.html instead of using time.Parse.
// Quick check: all date formats start with YYYY-.
i := 0
for ; i < len(s); i++ {
if c := s[i]; c < '0' || c > '9' {
break
}
}
if i != 4 || i == len(s) || s[i] != '-' {
return time.Time{}, false
}
for _, format := range allowedTimestampFormats {
if t, err := time.Parse(format, s); err == nil {
return t, true
}
}
return time.Time{}, false
}
+3040
View File
File diff suppressed because it is too large Load Diff
+134
View File
@@ -0,0 +1,134 @@
//
// Copyright (c) 2011-2019 Canonical Ltd
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package yaml
import (
"reflect"
"unicode"
)
type keyList []reflect.Value
func (l keyList) Len() int { return len(l) }
func (l keyList) Swap(i, j int) { l[i], l[j] = l[j], l[i] }
func (l keyList) Less(i, j int) bool {
a := l[i]
b := l[j]
ak := a.Kind()
bk := b.Kind()
for (ak == reflect.Interface || ak == reflect.Ptr) && !a.IsNil() {
a = a.Elem()
ak = a.Kind()
}
for (bk == reflect.Interface || bk == reflect.Ptr) && !b.IsNil() {
b = b.Elem()
bk = b.Kind()
}
af, aok := keyFloat(a)
bf, bok := keyFloat(b)
if aok && bok {
if af != bf {
return af < bf
}
if ak != bk {
return ak < bk
}
return numLess(a, b)
}
if ak != reflect.String || bk != reflect.String {
return ak < bk
}
ar, br := []rune(a.String()), []rune(b.String())
digits := false
for i := 0; i < len(ar) && i < len(br); i++ {
if ar[i] == br[i] {
digits = unicode.IsDigit(ar[i])
continue
}
al := unicode.IsLetter(ar[i])
bl := unicode.IsLetter(br[i])
if al && bl {
return ar[i] < br[i]
}
if al || bl {
if digits {
return al
} else {
return bl
}
}
var ai, bi int
var an, bn int64
if ar[i] == '0' || br[i] == '0' {
for j := i - 1; j >= 0 && unicode.IsDigit(ar[j]); j-- {
if ar[j] != '0' {
an = 1
bn = 1
break
}
}
}
for ai = i; ai < len(ar) && unicode.IsDigit(ar[ai]); ai++ {
an = an*10 + int64(ar[ai]-'0')
}
for bi = i; bi < len(br) && unicode.IsDigit(br[bi]); bi++ {
bn = bn*10 + int64(br[bi]-'0')
}
if an != bn {
return an < bn
}
if ai != bi {
return ai < bi
}
return ar[i] < br[i]
}
return len(ar) < len(br)
}
// keyFloat returns a float value for v if it is a number/bool
// and whether it is a number/bool or not.
func keyFloat(v reflect.Value) (f float64, ok bool) {
switch v.Kind() {
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
return float64(v.Int()), true
case reflect.Float32, reflect.Float64:
return v.Float(), true
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr:
return float64(v.Uint()), true
case reflect.Bool:
if v.Bool() {
return 1, true
}
return 0, true
}
return 0, false
}
// numLess returns whether a < b.
// a and b must necessarily have the same kind.
func numLess(a, b reflect.Value) bool {
switch a.Kind() {
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
return a.Int() < b.Int()
case reflect.Float32, reflect.Float64:
return a.Float() < b.Float()
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr:
return a.Uint() < b.Uint()
case reflect.Bool:
return !a.Bool() && b.Bool()
}
panic("not a number")
}
+48
View File
@@ -0,0 +1,48 @@
//
// Copyright (c) 2011-2019 Canonical Ltd
// Copyright (c) 2006-2010 Kirill Simonov
//
// Permission is hereby granted, free of charge, to any person obtaining a copy of
// this software and associated documentation files (the "Software"), to deal in
// the Software without restriction, including without limitation the rights to
// use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
// of the Software, and to permit persons to whom the Software is furnished to do
// so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in all
// copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
// SOFTWARE.
package yaml
// Set the writer error and return false.
func yaml_emitter_set_writer_error(emitter *yaml_emitter_t, problem string) bool {
emitter.error = yaml_WRITER_ERROR
emitter.problem = problem
return false
}
// Flush the output buffer.
func yaml_emitter_flush(emitter *yaml_emitter_t) bool {
if emitter.write_handler == nil {
panic("write handler not set")
}
// Check if the buffer is empty.
if emitter.buffer_pos == 0 {
return true
}
if err := emitter.write_handler(emitter, emitter.buffer[:emitter.buffer_pos]); err != nil {
return yaml_emitter_set_writer_error(emitter, "write error: "+err.Error())
}
emitter.buffer_pos = 0
return true
}
+703
View File
@@ -0,0 +1,703 @@
//
// Copyright (c) 2011-2019 Canonical Ltd
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// Package yaml implements YAML support for the Go language.
//
// Source code and other details for the project are available at GitHub:
//
// https://github.com/yaml/go-yaml
package yaml
import (
"errors"
"fmt"
"io"
"reflect"
"strings"
"sync"
"unicode/utf8"
)
// The Unmarshaler interface may be implemented by types to customize their
// behavior when being unmarshaled from a YAML document.
type Unmarshaler interface {
UnmarshalYAML(value *Node) error
}
type obsoleteUnmarshaler interface {
UnmarshalYAML(unmarshal func(interface{}) error) error
}
// The Marshaler interface may be implemented by types to customize their
// behavior when being marshaled into a YAML document. The returned value
// is marshaled in place of the original value implementing Marshaler.
//
// If an error is returned by MarshalYAML, the marshaling procedure stops
// and returns with the provided error.
type Marshaler interface {
MarshalYAML() (interface{}, error)
}
// Unmarshal decodes the first document found within the in byte slice
// and assigns decoded values into the out value.
//
// Maps and pointers (to a struct, string, int, etc) are accepted as out
// values. If an internal pointer within a struct is not initialized,
// the yaml package will initialize it if necessary for unmarshalling
// the provided data. The out parameter must not be nil.
//
// The type of the decoded values should be compatible with the respective
// values in out. If one or more values cannot be decoded due to a type
// mismatches, decoding continues partially until the end of the YAML
// content, and a *yaml.TypeError is returned with details for all
// missed values.
//
// Struct fields are only unmarshalled if they are exported (have an
// upper case first letter), and are unmarshalled using the field name
// lowercased as the default key. Custom keys may be defined via the
// "yaml" name in the field tag: the content preceding the first comma
// is used as the key, and the following comma-separated options are
// used to tweak the marshalling process (see Marshal).
// Conflicting names result in a runtime error.
//
// For example:
//
// type T struct {
// F int `yaml:"a,omitempty"`
// B int
// }
// var t T
// yaml.Unmarshal([]byte("a: 1\nb: 2"), &t)
//
// See the documentation of Marshal for the format of tags and a list of
// supported tag options.
func Unmarshal(in []byte, out interface{}) (err error) {
return unmarshal(in, out, false)
}
// A Decoder reads and decodes YAML values from an input stream.
type Decoder struct {
parser *parser
knownFields bool
}
// NewDecoder returns a new decoder that reads from r.
//
// The decoder introduces its own buffering and may read
// data from r beyond the YAML values requested.
func NewDecoder(r io.Reader) *Decoder {
return &Decoder{
parser: newParserFromReader(r),
}
}
// KnownFields ensures that the keys in decoded mappings to
// exist as fields in the struct being decoded into.
func (dec *Decoder) KnownFields(enable bool) {
dec.knownFields = enable
}
// Decode reads the next YAML-encoded value from its input
// and stores it in the value pointed to by v.
//
// See the documentation for Unmarshal for details about the
// conversion of YAML into a Go value.
func (dec *Decoder) Decode(v interface{}) (err error) {
d := newDecoder()
d.knownFields = dec.knownFields
defer handleErr(&err)
node := dec.parser.parse()
if node == nil {
return io.EOF
}
out := reflect.ValueOf(v)
if out.Kind() == reflect.Ptr && !out.IsNil() {
out = out.Elem()
}
d.unmarshal(node, out)
if len(d.terrors) > 0 {
return &TypeError{d.terrors}
}
return nil
}
// Decode decodes the node and stores its data into the value pointed to by v.
//
// See the documentation for Unmarshal for details about the
// conversion of YAML into a Go value.
func (n *Node) Decode(v interface{}) (err error) {
d := newDecoder()
defer handleErr(&err)
out := reflect.ValueOf(v)
if out.Kind() == reflect.Ptr && !out.IsNil() {
out = out.Elem()
}
d.unmarshal(n, out)
if len(d.terrors) > 0 {
return &TypeError{d.terrors}
}
return nil
}
func unmarshal(in []byte, out interface{}, strict bool) (err error) {
defer handleErr(&err)
d := newDecoder()
p := newParser(in)
defer p.destroy()
node := p.parse()
if node != nil {
v := reflect.ValueOf(out)
if v.Kind() == reflect.Ptr && !v.IsNil() {
v = v.Elem()
}
d.unmarshal(node, v)
}
if len(d.terrors) > 0 {
return &TypeError{d.terrors}
}
return nil
}
// Marshal serializes the value provided into a YAML document. The structure
// of the generated document will reflect the structure of the value itself.
// Maps and pointers (to struct, string, int, etc) are accepted as the in value.
//
// Struct fields are only marshalled if they are exported (have an upper case
// first letter), and are marshalled using the field name lowercased as the
// default key. Custom keys may be defined via the "yaml" name in the field
// tag: the content preceding the first comma is used as the key, and the
// following comma-separated options are used to tweak the marshalling process.
// Conflicting names result in a runtime error.
//
// The field tag format accepted is:
//
// `(...) yaml:"[<key>][,<flag1>[,<flag2>]]" (...)`
//
// The following flags are currently supported:
//
// omitempty Only include the field if it's not set to the zero
// value for the type or to empty slices or maps.
// Zero valued structs will be omitted if all their public
// fields are zero, unless they implement an IsZero
// method (see the IsZeroer interface type), in which
// case the field will be excluded if IsZero returns true.
//
// flow Marshal using a flow style (useful for structs,
// sequences and maps).
//
// inline Inline the field, which must be a struct or a map,
// causing all of its fields or keys to be processed as if
// they were part of the outer struct. For maps, keys must
// not conflict with the yaml keys of other struct fields.
//
// In addition, if the key is "-", the field is ignored.
//
// For example:
//
// type T struct {
// F int `yaml:"a,omitempty"`
// B int
// }
// yaml.Marshal(&T{B: 2}) // Returns "b: 2\n"
// yaml.Marshal(&T{F: 1}} // Returns "a: 1\nb: 0\n"
func Marshal(in interface{}) (out []byte, err error) {
defer handleErr(&err)
e := newEncoder()
defer e.destroy()
e.marshalDoc("", reflect.ValueOf(in))
e.finish()
out = e.out
return
}
// An Encoder writes YAML values to an output stream.
type Encoder struct {
encoder *encoder
}
// NewEncoder returns a new encoder that writes to w.
// The Encoder should be closed after use to flush all data
// to w.
func NewEncoder(w io.Writer) *Encoder {
return &Encoder{
encoder: newEncoderWithWriter(w),
}
}
// Encode writes the YAML encoding of v to the stream.
// If multiple items are encoded to the stream, the
// second and subsequent document will be preceded
// with a "---" document separator, but the first will not.
//
// See the documentation for Marshal for details about the conversion of Go
// values to YAML.
func (e *Encoder) Encode(v interface{}) (err error) {
defer handleErr(&err)
e.encoder.marshalDoc("", reflect.ValueOf(v))
return nil
}
// Encode encodes value v and stores its representation in n.
//
// See the documentation for Marshal for details about the
// conversion of Go values into YAML.
func (n *Node) Encode(v interface{}) (err error) {
defer handleErr(&err)
e := newEncoder()
defer e.destroy()
e.marshalDoc("", reflect.ValueOf(v))
e.finish()
p := newParser(e.out)
p.textless = true
defer p.destroy()
doc := p.parse()
*n = *doc.Content[0]
return nil
}
// SetIndent changes the used indentation used when encoding.
func (e *Encoder) SetIndent(spaces int) {
if spaces < 0 {
panic("yaml: cannot indent to a negative number of spaces")
}
e.encoder.indent = spaces
}
// CompactSeqIndent makes it so that '- ' is considered part of the indentation.
func (e *Encoder) CompactSeqIndent() {
e.encoder.emitter.compact_sequence_indent = true
}
// DefaultSeqIndent makes it so that '- ' is not considered part of the indentation.
func (e *Encoder) DefaultSeqIndent() {
e.encoder.emitter.compact_sequence_indent = false
}
// Close closes the encoder by writing any remaining data.
// It does not write a stream terminating string "...".
func (e *Encoder) Close() (err error) {
defer handleErr(&err)
e.encoder.finish()
return nil
}
func handleErr(err *error) {
if v := recover(); v != nil {
if e, ok := v.(yamlError); ok {
*err = e.err
} else {
panic(v)
}
}
}
type yamlError struct {
err error
}
func fail(err error) {
panic(yamlError{err})
}
func failf(format string, args ...interface{}) {
panic(yamlError{fmt.Errorf("yaml: "+format, args...)})
}
// A TypeError is returned by Unmarshal when one or more fields in
// the YAML document cannot be properly decoded into the requested
// types. When this error is returned, the value is still
// unmarshaled partially.
type TypeError struct {
Errors []string
}
func (e *TypeError) Error() string {
return fmt.Sprintf("yaml: unmarshal errors:\n %s", strings.Join(e.Errors, "\n "))
}
type Kind uint32
const (
DocumentNode Kind = 1 << iota
SequenceNode
MappingNode
ScalarNode
AliasNode
)
type Style uint32
const (
TaggedStyle Style = 1 << iota
DoubleQuotedStyle
SingleQuotedStyle
LiteralStyle
FoldedStyle
FlowStyle
)
// Node represents an element in the YAML document hierarchy. While documents
// are typically encoded and decoded into higher level types, such as structs
// and maps, Node is an intermediate representation that allows detailed
// control over the content being decoded or encoded.
//
// It's worth noting that although Node offers access into details such as
// line numbers, colums, and comments, the content when re-encoded will not
// have its original textual representation preserved. An effort is made to
// render the data plesantly, and to preserve comments near the data they
// describe, though.
//
// Values that make use of the Node type interact with the yaml package in the
// same way any other type would do, by encoding and decoding yaml data
// directly or indirectly into them.
//
// For example:
//
// var person struct {
// Name string
// Address yaml.Node
// }
// err := yaml.Unmarshal(data, &person)
//
// Or by itself:
//
// var person Node
// err := yaml.Unmarshal(data, &person)
type Node struct {
// Kind defines whether the node is a document, a mapping, a sequence,
// a scalar value, or an alias to another node. The specific data type of
// scalar nodes may be obtained via the ShortTag and LongTag methods.
Kind Kind
// Style allows customizing the apperance of the node in the tree.
Style Style
// Tag holds the YAML tag defining the data type for the value.
// When decoding, this field will always be set to the resolved tag,
// even when it wasn't explicitly provided in the YAML content.
// When encoding, if this field is unset the value type will be
// implied from the node properties, and if it is set, it will only
// be serialized into the representation if TaggedStyle is used or
// the implicit tag diverges from the provided one.
Tag string
// Value holds the unescaped and unquoted represenation of the value.
Value string
// Anchor holds the anchor name for this node, which allows aliases to point to it.
Anchor string
// Alias holds the node that this alias points to. Only valid when Kind is AliasNode.
Alias *Node
// Content holds contained nodes for documents, mappings, and sequences.
Content []*Node
// HeadComment holds any comments in the lines preceding the node and
// not separated by an empty line.
HeadComment string
// LineComment holds any comments at the end of the line where the node is in.
LineComment string
// FootComment holds any comments following the node and before empty lines.
FootComment string
// Line and Column hold the node position in the decoded YAML text.
// These fields are not respected when encoding the node.
Line int
Column int
}
// IsZero returns whether the node has all of its fields unset.
func (n *Node) IsZero() bool {
return n.Kind == 0 && n.Style == 0 && n.Tag == "" && n.Value == "" && n.Anchor == "" && n.Alias == nil && n.Content == nil &&
n.HeadComment == "" && n.LineComment == "" && n.FootComment == "" && n.Line == 0 && n.Column == 0
}
// LongTag returns the long form of the tag that indicates the data type for
// the node. If the Tag field isn't explicitly defined, one will be computed
// based on the node properties.
func (n *Node) LongTag() string {
return longTag(n.ShortTag())
}
// ShortTag returns the short form of the YAML tag that indicates data type for
// the node. If the Tag field isn't explicitly defined, one will be computed
// based on the node properties.
func (n *Node) ShortTag() string {
if n.indicatedString() {
return strTag
}
if n.Tag == "" || n.Tag == "!" {
switch n.Kind {
case MappingNode:
return mapTag
case SequenceNode:
return seqTag
case AliasNode:
if n.Alias != nil {
return n.Alias.ShortTag()
}
case ScalarNode:
tag, _ := resolve("", n.Value)
return tag
case 0:
// Special case to make the zero value convenient.
if n.IsZero() {
return nullTag
}
}
return ""
}
return shortTag(n.Tag)
}
func (n *Node) indicatedString() bool {
return n.Kind == ScalarNode &&
(shortTag(n.Tag) == strTag ||
(n.Tag == "" || n.Tag == "!") && n.Style&(SingleQuotedStyle|DoubleQuotedStyle|LiteralStyle|FoldedStyle) != 0)
}
// SetString is a convenience function that sets the node to a string value
// and defines its style in a pleasant way depending on its content.
func (n *Node) SetString(s string) {
n.Kind = ScalarNode
if utf8.ValidString(s) {
n.Value = s
n.Tag = strTag
} else {
n.Value = encodeBase64(s)
n.Tag = binaryTag
}
if strings.Contains(n.Value, "\n") {
n.Style = LiteralStyle
}
}
// --------------------------------------------------------------------------
// Maintain a mapping of keys to structure field indexes
// The code in this section was copied from mgo/bson.
// structInfo holds details for the serialization of fields of
// a given struct.
type structInfo struct {
FieldsMap map[string]fieldInfo
FieldsList []fieldInfo
// InlineMap is the number of the field in the struct that
// contains an ,inline map, or -1 if there's none.
InlineMap int
// InlineUnmarshalers holds indexes to inlined fields that
// contain unmarshaler values.
InlineUnmarshalers [][]int
}
type fieldInfo struct {
Key string
Num int
OmitEmpty bool
Flow bool
// Id holds the unique field identifier, so we can cheaply
// check for field duplicates without maintaining an extra map.
Id int
// Inline holds the field index if the field is part of an inlined struct.
Inline []int
}
var structMap = make(map[reflect.Type]*structInfo)
var fieldMapMutex sync.RWMutex
var unmarshalerType reflect.Type
func init() {
var v Unmarshaler
unmarshalerType = reflect.ValueOf(&v).Elem().Type()
}
func getStructInfo(st reflect.Type) (*structInfo, error) {
fieldMapMutex.RLock()
sinfo, found := structMap[st]
fieldMapMutex.RUnlock()
if found {
return sinfo, nil
}
n := st.NumField()
fieldsMap := make(map[string]fieldInfo)
fieldsList := make([]fieldInfo, 0, n)
inlineMap := -1
inlineUnmarshalers := [][]int(nil)
for i := 0; i != n; i++ {
field := st.Field(i)
if field.PkgPath != "" && !field.Anonymous {
continue // Private field
}
info := fieldInfo{Num: i}
tag := field.Tag.Get("yaml")
if tag == "" && strings.Index(string(field.Tag), ":") < 0 {
tag = string(field.Tag)
}
if tag == "-" {
continue
}
inline := false
fields := strings.Split(tag, ",")
if len(fields) > 1 {
for _, flag := range fields[1:] {
switch flag {
case "omitempty":
info.OmitEmpty = true
case "flow":
info.Flow = true
case "inline":
inline = true
default:
return nil, errors.New(fmt.Sprintf("unsupported flag %q in tag %q of type %s", flag, tag, st))
}
}
tag = fields[0]
}
if inline {
switch field.Type.Kind() {
case reflect.Map:
if inlineMap >= 0 {
return nil, errors.New("multiple ,inline maps in struct " + st.String())
}
if field.Type.Key() != reflect.TypeOf("") {
return nil, errors.New("option ,inline needs a map with string keys in struct " + st.String())
}
inlineMap = info.Num
case reflect.Struct, reflect.Ptr:
ftype := field.Type
for ftype.Kind() == reflect.Ptr {
ftype = ftype.Elem()
}
if ftype.Kind() != reflect.Struct {
return nil, errors.New("option ,inline may only be used on a struct or map field")
}
if reflect.PtrTo(ftype).Implements(unmarshalerType) {
inlineUnmarshalers = append(inlineUnmarshalers, []int{i})
} else {
sinfo, err := getStructInfo(ftype)
if err != nil {
return nil, err
}
for _, index := range sinfo.InlineUnmarshalers {
inlineUnmarshalers = append(inlineUnmarshalers, append([]int{i}, index...))
}
for _, finfo := range sinfo.FieldsList {
if _, found := fieldsMap[finfo.Key]; found {
msg := "duplicated key '" + finfo.Key + "' in struct " + st.String()
return nil, errors.New(msg)
}
if finfo.Inline == nil {
finfo.Inline = []int{i, finfo.Num}
} else {
finfo.Inline = append([]int{i}, finfo.Inline...)
}
finfo.Id = len(fieldsList)
fieldsMap[finfo.Key] = finfo
fieldsList = append(fieldsList, finfo)
}
}
default:
return nil, errors.New("option ,inline may only be used on a struct or map field")
}
continue
}
if tag != "" {
info.Key = tag
} else {
info.Key = strings.ToLower(field.Name)
}
if _, found = fieldsMap[info.Key]; found {
msg := "duplicated key '" + info.Key + "' in struct " + st.String()
return nil, errors.New(msg)
}
info.Id = len(fieldsList)
fieldsList = append(fieldsList, info)
fieldsMap[info.Key] = info
}
sinfo = &structInfo{
FieldsMap: fieldsMap,
FieldsList: fieldsList,
InlineMap: inlineMap,
InlineUnmarshalers: inlineUnmarshalers,
}
fieldMapMutex.Lock()
structMap[st] = sinfo
fieldMapMutex.Unlock()
return sinfo, nil
}
// IsZeroer is used to check whether an object is zero to
// determine whether it should be omitted when marshaling
// with the omitempty flag. One notable implementation
// is time.Time.
type IsZeroer interface {
IsZero() bool
}
func isZero(v reflect.Value) bool {
kind := v.Kind()
if z, ok := v.Interface().(IsZeroer); ok {
if (kind == reflect.Ptr || kind == reflect.Interface) && v.IsNil() {
return true
}
return z.IsZero()
}
switch kind {
case reflect.String:
return len(v.String()) == 0
case reflect.Interface, reflect.Ptr:
return v.IsNil()
case reflect.Slice:
return v.Len() == 0
case reflect.Map:
return v.Len() == 0
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
return v.Int() == 0
case reflect.Float32, reflect.Float64:
return v.Float() == 0
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr:
return v.Uint() == 0
case reflect.Bool:
return !v.Bool()
case reflect.Struct:
vt := v.Type()
for i := v.NumField() - 1; i >= 0; i-- {
if vt.Field(i).PkgPath != "" {
continue // Private field
}
if !isZero(v.Field(i)) {
return false
}
}
return true
}
return false
}
+811
View File
@@ -0,0 +1,811 @@
//
// Copyright (c) 2011-2019 Canonical Ltd
// Copyright (c) 2006-2010 Kirill Simonov
//
// Permission is hereby granted, free of charge, to any person obtaining a copy of
// this software and associated documentation files (the "Software"), to deal in
// the Software without restriction, including without limitation the rights to
// use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
// of the Software, and to permit persons to whom the Software is furnished to do
// so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in all
// copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
// SOFTWARE.
package yaml
import (
"fmt"
"io"
)
// The version directive data.
type yaml_version_directive_t struct {
major int8 // The major version number.
minor int8 // The minor version number.
}
// The tag directive data.
type yaml_tag_directive_t struct {
handle []byte // The tag handle.
prefix []byte // The tag prefix.
}
type yaml_encoding_t int
// The stream encoding.
const (
// Let the parser choose the encoding.
yaml_ANY_ENCODING yaml_encoding_t = iota
yaml_UTF8_ENCODING // The default UTF-8 encoding.
yaml_UTF16LE_ENCODING // The UTF-16-LE encoding with BOM.
yaml_UTF16BE_ENCODING // The UTF-16-BE encoding with BOM.
)
type yaml_break_t int
// Line break types.
const (
// Let the parser choose the break type.
yaml_ANY_BREAK yaml_break_t = iota
yaml_CR_BREAK // Use CR for line breaks (Mac style).
yaml_LN_BREAK // Use LN for line breaks (Unix style).
yaml_CRLN_BREAK // Use CR LN for line breaks (DOS style).
)
type yaml_error_type_t int
// Many bad things could happen with the parser and emitter.
const (
// No error is produced.
yaml_NO_ERROR yaml_error_type_t = iota
yaml_MEMORY_ERROR // Cannot allocate or reallocate a block of memory.
yaml_READER_ERROR // Cannot read or decode the input stream.
yaml_SCANNER_ERROR // Cannot scan the input stream.
yaml_PARSER_ERROR // Cannot parse the input stream.
yaml_COMPOSER_ERROR // Cannot compose a YAML document.
yaml_WRITER_ERROR // Cannot write to the output stream.
yaml_EMITTER_ERROR // Cannot emit a YAML stream.
)
// The pointer position.
type yaml_mark_t struct {
index int // The position index.
line int // The position line.
column int // The position column.
}
// Node Styles
type yaml_style_t int8
type yaml_scalar_style_t yaml_style_t
// Scalar styles.
const (
// Let the emitter choose the style.
yaml_ANY_SCALAR_STYLE yaml_scalar_style_t = 0
yaml_PLAIN_SCALAR_STYLE yaml_scalar_style_t = 1 << iota // The plain scalar style.
yaml_SINGLE_QUOTED_SCALAR_STYLE // The single-quoted scalar style.
yaml_DOUBLE_QUOTED_SCALAR_STYLE // The double-quoted scalar style.
yaml_LITERAL_SCALAR_STYLE // The literal scalar style.
yaml_FOLDED_SCALAR_STYLE // The folded scalar style.
)
type yaml_sequence_style_t yaml_style_t
// Sequence styles.
const (
// Let the emitter choose the style.
yaml_ANY_SEQUENCE_STYLE yaml_sequence_style_t = iota
yaml_BLOCK_SEQUENCE_STYLE // The block sequence style.
yaml_FLOW_SEQUENCE_STYLE // The flow sequence style.
)
type yaml_mapping_style_t yaml_style_t
// Mapping styles.
const (
// Let the emitter choose the style.
yaml_ANY_MAPPING_STYLE yaml_mapping_style_t = iota
yaml_BLOCK_MAPPING_STYLE // The block mapping style.
yaml_FLOW_MAPPING_STYLE // The flow mapping style.
)
// Tokens
type yaml_token_type_t int
// Token types.
const (
// An empty token.
yaml_NO_TOKEN yaml_token_type_t = iota
yaml_STREAM_START_TOKEN // A STREAM-START token.
yaml_STREAM_END_TOKEN // A STREAM-END token.
yaml_VERSION_DIRECTIVE_TOKEN // A VERSION-DIRECTIVE token.
yaml_TAG_DIRECTIVE_TOKEN // A TAG-DIRECTIVE token.
yaml_DOCUMENT_START_TOKEN // A DOCUMENT-START token.
yaml_DOCUMENT_END_TOKEN // A DOCUMENT-END token.
yaml_BLOCK_SEQUENCE_START_TOKEN // A BLOCK-SEQUENCE-START token.
yaml_BLOCK_MAPPING_START_TOKEN // A BLOCK-SEQUENCE-END token.
yaml_BLOCK_END_TOKEN // A BLOCK-END token.
yaml_FLOW_SEQUENCE_START_TOKEN // A FLOW-SEQUENCE-START token.
yaml_FLOW_SEQUENCE_END_TOKEN // A FLOW-SEQUENCE-END token.
yaml_FLOW_MAPPING_START_TOKEN // A FLOW-MAPPING-START token.
yaml_FLOW_MAPPING_END_TOKEN // A FLOW-MAPPING-END token.
yaml_BLOCK_ENTRY_TOKEN // A BLOCK-ENTRY token.
yaml_FLOW_ENTRY_TOKEN // A FLOW-ENTRY token.
yaml_KEY_TOKEN // A KEY token.
yaml_VALUE_TOKEN // A VALUE token.
yaml_ALIAS_TOKEN // An ALIAS token.
yaml_ANCHOR_TOKEN // An ANCHOR token.
yaml_TAG_TOKEN // A TAG token.
yaml_SCALAR_TOKEN // A SCALAR token.
)
func (tt yaml_token_type_t) String() string {
switch tt {
case yaml_NO_TOKEN:
return "yaml_NO_TOKEN"
case yaml_STREAM_START_TOKEN:
return "yaml_STREAM_START_TOKEN"
case yaml_STREAM_END_TOKEN:
return "yaml_STREAM_END_TOKEN"
case yaml_VERSION_DIRECTIVE_TOKEN:
return "yaml_VERSION_DIRECTIVE_TOKEN"
case yaml_TAG_DIRECTIVE_TOKEN:
return "yaml_TAG_DIRECTIVE_TOKEN"
case yaml_DOCUMENT_START_TOKEN:
return "yaml_DOCUMENT_START_TOKEN"
case yaml_DOCUMENT_END_TOKEN:
return "yaml_DOCUMENT_END_TOKEN"
case yaml_BLOCK_SEQUENCE_START_TOKEN:
return "yaml_BLOCK_SEQUENCE_START_TOKEN"
case yaml_BLOCK_MAPPING_START_TOKEN:
return "yaml_BLOCK_MAPPING_START_TOKEN"
case yaml_BLOCK_END_TOKEN:
return "yaml_BLOCK_END_TOKEN"
case yaml_FLOW_SEQUENCE_START_TOKEN:
return "yaml_FLOW_SEQUENCE_START_TOKEN"
case yaml_FLOW_SEQUENCE_END_TOKEN:
return "yaml_FLOW_SEQUENCE_END_TOKEN"
case yaml_FLOW_MAPPING_START_TOKEN:
return "yaml_FLOW_MAPPING_START_TOKEN"
case yaml_FLOW_MAPPING_END_TOKEN:
return "yaml_FLOW_MAPPING_END_TOKEN"
case yaml_BLOCK_ENTRY_TOKEN:
return "yaml_BLOCK_ENTRY_TOKEN"
case yaml_FLOW_ENTRY_TOKEN:
return "yaml_FLOW_ENTRY_TOKEN"
case yaml_KEY_TOKEN:
return "yaml_KEY_TOKEN"
case yaml_VALUE_TOKEN:
return "yaml_VALUE_TOKEN"
case yaml_ALIAS_TOKEN:
return "yaml_ALIAS_TOKEN"
case yaml_ANCHOR_TOKEN:
return "yaml_ANCHOR_TOKEN"
case yaml_TAG_TOKEN:
return "yaml_TAG_TOKEN"
case yaml_SCALAR_TOKEN:
return "yaml_SCALAR_TOKEN"
}
return "<unknown token>"
}
// The token structure.
type yaml_token_t struct {
// The token type.
typ yaml_token_type_t
// The start/end of the token.
start_mark, end_mark yaml_mark_t
// The stream encoding (for yaml_STREAM_START_TOKEN).
encoding yaml_encoding_t
// The alias/anchor/scalar value or tag/tag directive handle
// (for yaml_ALIAS_TOKEN, yaml_ANCHOR_TOKEN, yaml_SCALAR_TOKEN, yaml_TAG_TOKEN, yaml_TAG_DIRECTIVE_TOKEN).
value []byte
// The tag suffix (for yaml_TAG_TOKEN).
suffix []byte
// The tag directive prefix (for yaml_TAG_DIRECTIVE_TOKEN).
prefix []byte
// The scalar style (for yaml_SCALAR_TOKEN).
style yaml_scalar_style_t
// The version directive major/minor (for yaml_VERSION_DIRECTIVE_TOKEN).
major, minor int8
}
// Events
type yaml_event_type_t int8
// Event types.
const (
// An empty event.
yaml_NO_EVENT yaml_event_type_t = iota
yaml_STREAM_START_EVENT // A STREAM-START event.
yaml_STREAM_END_EVENT // A STREAM-END event.
yaml_DOCUMENT_START_EVENT // A DOCUMENT-START event.
yaml_DOCUMENT_END_EVENT // A DOCUMENT-END event.
yaml_ALIAS_EVENT // An ALIAS event.
yaml_SCALAR_EVENT // A SCALAR event.
yaml_SEQUENCE_START_EVENT // A SEQUENCE-START event.
yaml_SEQUENCE_END_EVENT // A SEQUENCE-END event.
yaml_MAPPING_START_EVENT // A MAPPING-START event.
yaml_MAPPING_END_EVENT // A MAPPING-END event.
yaml_TAIL_COMMENT_EVENT
)
var eventStrings = []string{
yaml_NO_EVENT: "none",
yaml_STREAM_START_EVENT: "stream start",
yaml_STREAM_END_EVENT: "stream end",
yaml_DOCUMENT_START_EVENT: "document start",
yaml_DOCUMENT_END_EVENT: "document end",
yaml_ALIAS_EVENT: "alias",
yaml_SCALAR_EVENT: "scalar",
yaml_SEQUENCE_START_EVENT: "sequence start",
yaml_SEQUENCE_END_EVENT: "sequence end",
yaml_MAPPING_START_EVENT: "mapping start",
yaml_MAPPING_END_EVENT: "mapping end",
yaml_TAIL_COMMENT_EVENT: "tail comment",
}
func (e yaml_event_type_t) String() string {
if e < 0 || int(e) >= len(eventStrings) {
return fmt.Sprintf("unknown event %d", e)
}
return eventStrings[e]
}
// The event structure.
type yaml_event_t struct {
// The event type.
typ yaml_event_type_t
// The start and end of the event.
start_mark, end_mark yaml_mark_t
// The document encoding (for yaml_STREAM_START_EVENT).
encoding yaml_encoding_t
// The version directive (for yaml_DOCUMENT_START_EVENT).
version_directive *yaml_version_directive_t
// The list of tag directives (for yaml_DOCUMENT_START_EVENT).
tag_directives []yaml_tag_directive_t
// The comments
head_comment []byte
line_comment []byte
foot_comment []byte
tail_comment []byte
// The anchor (for yaml_SCALAR_EVENT, yaml_SEQUENCE_START_EVENT, yaml_MAPPING_START_EVENT, yaml_ALIAS_EVENT).
anchor []byte
// The tag (for yaml_SCALAR_EVENT, yaml_SEQUENCE_START_EVENT, yaml_MAPPING_START_EVENT).
tag []byte
// The scalar value (for yaml_SCALAR_EVENT).
value []byte
// Is the document start/end indicator implicit, or the tag optional?
// (for yaml_DOCUMENT_START_EVENT, yaml_DOCUMENT_END_EVENT, yaml_SEQUENCE_START_EVENT, yaml_MAPPING_START_EVENT, yaml_SCALAR_EVENT).
implicit bool
// Is the tag optional for any non-plain style? (for yaml_SCALAR_EVENT).
quoted_implicit bool
// The style (for yaml_SCALAR_EVENT, yaml_SEQUENCE_START_EVENT, yaml_MAPPING_START_EVENT).
style yaml_style_t
}
func (e *yaml_event_t) scalar_style() yaml_scalar_style_t { return yaml_scalar_style_t(e.style) }
func (e *yaml_event_t) sequence_style() yaml_sequence_style_t { return yaml_sequence_style_t(e.style) }
func (e *yaml_event_t) mapping_style() yaml_mapping_style_t { return yaml_mapping_style_t(e.style) }
// Nodes
const (
yaml_NULL_TAG = "tag:yaml.org,2002:null" // The tag !!null with the only possible value: null.
yaml_BOOL_TAG = "tag:yaml.org,2002:bool" // The tag !!bool with the values: true and false.
yaml_STR_TAG = "tag:yaml.org,2002:str" // The tag !!str for string values.
yaml_INT_TAG = "tag:yaml.org,2002:int" // The tag !!int for integer values.
yaml_FLOAT_TAG = "tag:yaml.org,2002:float" // The tag !!float for float values.
yaml_TIMESTAMP_TAG = "tag:yaml.org,2002:timestamp" // The tag !!timestamp for date and time values.
yaml_SEQ_TAG = "tag:yaml.org,2002:seq" // The tag !!seq is used to denote sequences.
yaml_MAP_TAG = "tag:yaml.org,2002:map" // The tag !!map is used to denote mapping.
// Not in original libyaml.
yaml_BINARY_TAG = "tag:yaml.org,2002:binary"
yaml_MERGE_TAG = "tag:yaml.org,2002:merge"
yaml_DEFAULT_SCALAR_TAG = yaml_STR_TAG // The default scalar tag is !!str.
yaml_DEFAULT_SEQUENCE_TAG = yaml_SEQ_TAG // The default sequence tag is !!seq.
yaml_DEFAULT_MAPPING_TAG = yaml_MAP_TAG // The default mapping tag is !!map.
)
type yaml_node_type_t int
// Node types.
const (
// An empty node.
yaml_NO_NODE yaml_node_type_t = iota
yaml_SCALAR_NODE // A scalar node.
yaml_SEQUENCE_NODE // A sequence node.
yaml_MAPPING_NODE // A mapping node.
)
// An element of a sequence node.
type yaml_node_item_t int
// An element of a mapping node.
type yaml_node_pair_t struct {
key int // The key of the element.
value int // The value of the element.
}
// The node structure.
type yaml_node_t struct {
typ yaml_node_type_t // The node type.
tag []byte // The node tag.
// The node data.
// The scalar parameters (for yaml_SCALAR_NODE).
scalar struct {
value []byte // The scalar value.
length int // The length of the scalar value.
style yaml_scalar_style_t // The scalar style.
}
// The sequence parameters (for YAML_SEQUENCE_NODE).
sequence struct {
items_data []yaml_node_item_t // The stack of sequence items.
style yaml_sequence_style_t // The sequence style.
}
// The mapping parameters (for yaml_MAPPING_NODE).
mapping struct {
pairs_data []yaml_node_pair_t // The stack of mapping pairs (key, value).
pairs_start *yaml_node_pair_t // The beginning of the stack.
pairs_end *yaml_node_pair_t // The end of the stack.
pairs_top *yaml_node_pair_t // The top of the stack.
style yaml_mapping_style_t // The mapping style.
}
start_mark yaml_mark_t // The beginning of the node.
end_mark yaml_mark_t // The end of the node.
}
// The document structure.
type yaml_document_t struct {
// The document nodes.
nodes []yaml_node_t
// The version directive.
version_directive *yaml_version_directive_t
// The list of tag directives.
tag_directives_data []yaml_tag_directive_t
tag_directives_start int // The beginning of the tag directives list.
tag_directives_end int // The end of the tag directives list.
start_implicit int // Is the document start indicator implicit?
end_implicit int // Is the document end indicator implicit?
// The start/end of the document.
start_mark, end_mark yaml_mark_t
}
// The prototype of a read handler.
//
// The read handler is called when the parser needs to read more bytes from the
// source. The handler should write not more than size bytes to the buffer.
// The number of written bytes should be set to the size_read variable.
//
// [in,out] data A pointer to an application data specified by
//
// yaml_parser_set_input().
//
// [out] buffer The buffer to write the data from the source.
// [in] size The size of the buffer.
// [out] size_read The actual number of bytes read from the source.
//
// On success, the handler should return 1. If the handler failed,
// the returned value should be 0. On EOF, the handler should set the
// size_read to 0 and return 1.
type yaml_read_handler_t func(parser *yaml_parser_t, buffer []byte) (n int, err error)
// This structure holds information about a potential simple key.
type yaml_simple_key_t struct {
possible bool // Is a simple key possible?
required bool // Is a simple key required?
token_number int // The number of the token.
mark yaml_mark_t // The position mark.
}
// The states of the parser.
type yaml_parser_state_t int
const (
yaml_PARSE_STREAM_START_STATE yaml_parser_state_t = iota
yaml_PARSE_IMPLICIT_DOCUMENT_START_STATE // Expect the beginning of an implicit document.
yaml_PARSE_DOCUMENT_START_STATE // Expect DOCUMENT-START.
yaml_PARSE_DOCUMENT_CONTENT_STATE // Expect the content of a document.
yaml_PARSE_DOCUMENT_END_STATE // Expect DOCUMENT-END.
yaml_PARSE_BLOCK_NODE_STATE // Expect a block node.
yaml_PARSE_BLOCK_NODE_OR_INDENTLESS_SEQUENCE_STATE // Expect a block node or indentless sequence.
yaml_PARSE_FLOW_NODE_STATE // Expect a flow node.
yaml_PARSE_BLOCK_SEQUENCE_FIRST_ENTRY_STATE // Expect the first entry of a block sequence.
yaml_PARSE_BLOCK_SEQUENCE_ENTRY_STATE // Expect an entry of a block sequence.
yaml_PARSE_INDENTLESS_SEQUENCE_ENTRY_STATE // Expect an entry of an indentless sequence.
yaml_PARSE_BLOCK_MAPPING_FIRST_KEY_STATE // Expect the first key of a block mapping.
yaml_PARSE_BLOCK_MAPPING_KEY_STATE // Expect a block mapping key.
yaml_PARSE_BLOCK_MAPPING_VALUE_STATE // Expect a block mapping value.
yaml_PARSE_FLOW_SEQUENCE_FIRST_ENTRY_STATE // Expect the first entry of a flow sequence.
yaml_PARSE_FLOW_SEQUENCE_ENTRY_STATE // Expect an entry of a flow sequence.
yaml_PARSE_FLOW_SEQUENCE_ENTRY_MAPPING_KEY_STATE // Expect a key of an ordered mapping.
yaml_PARSE_FLOW_SEQUENCE_ENTRY_MAPPING_VALUE_STATE // Expect a value of an ordered mapping.
yaml_PARSE_FLOW_SEQUENCE_ENTRY_MAPPING_END_STATE // Expect the and of an ordered mapping entry.
yaml_PARSE_FLOW_MAPPING_FIRST_KEY_STATE // Expect the first key of a flow mapping.
yaml_PARSE_FLOW_MAPPING_KEY_STATE // Expect a key of a flow mapping.
yaml_PARSE_FLOW_MAPPING_VALUE_STATE // Expect a value of a flow mapping.
yaml_PARSE_FLOW_MAPPING_EMPTY_VALUE_STATE // Expect an empty value of a flow mapping.
yaml_PARSE_END_STATE // Expect nothing.
)
func (ps yaml_parser_state_t) String() string {
switch ps {
case yaml_PARSE_STREAM_START_STATE:
return "yaml_PARSE_STREAM_START_STATE"
case yaml_PARSE_IMPLICIT_DOCUMENT_START_STATE:
return "yaml_PARSE_IMPLICIT_DOCUMENT_START_STATE"
case yaml_PARSE_DOCUMENT_START_STATE:
return "yaml_PARSE_DOCUMENT_START_STATE"
case yaml_PARSE_DOCUMENT_CONTENT_STATE:
return "yaml_PARSE_DOCUMENT_CONTENT_STATE"
case yaml_PARSE_DOCUMENT_END_STATE:
return "yaml_PARSE_DOCUMENT_END_STATE"
case yaml_PARSE_BLOCK_NODE_STATE:
return "yaml_PARSE_BLOCK_NODE_STATE"
case yaml_PARSE_BLOCK_NODE_OR_INDENTLESS_SEQUENCE_STATE:
return "yaml_PARSE_BLOCK_NODE_OR_INDENTLESS_SEQUENCE_STATE"
case yaml_PARSE_FLOW_NODE_STATE:
return "yaml_PARSE_FLOW_NODE_STATE"
case yaml_PARSE_BLOCK_SEQUENCE_FIRST_ENTRY_STATE:
return "yaml_PARSE_BLOCK_SEQUENCE_FIRST_ENTRY_STATE"
case yaml_PARSE_BLOCK_SEQUENCE_ENTRY_STATE:
return "yaml_PARSE_BLOCK_SEQUENCE_ENTRY_STATE"
case yaml_PARSE_INDENTLESS_SEQUENCE_ENTRY_STATE:
return "yaml_PARSE_INDENTLESS_SEQUENCE_ENTRY_STATE"
case yaml_PARSE_BLOCK_MAPPING_FIRST_KEY_STATE:
return "yaml_PARSE_BLOCK_MAPPING_FIRST_KEY_STATE"
case yaml_PARSE_BLOCK_MAPPING_KEY_STATE:
return "yaml_PARSE_BLOCK_MAPPING_KEY_STATE"
case yaml_PARSE_BLOCK_MAPPING_VALUE_STATE:
return "yaml_PARSE_BLOCK_MAPPING_VALUE_STATE"
case yaml_PARSE_FLOW_SEQUENCE_FIRST_ENTRY_STATE:
return "yaml_PARSE_FLOW_SEQUENCE_FIRST_ENTRY_STATE"
case yaml_PARSE_FLOW_SEQUENCE_ENTRY_STATE:
return "yaml_PARSE_FLOW_SEQUENCE_ENTRY_STATE"
case yaml_PARSE_FLOW_SEQUENCE_ENTRY_MAPPING_KEY_STATE:
return "yaml_PARSE_FLOW_SEQUENCE_ENTRY_MAPPING_KEY_STATE"
case yaml_PARSE_FLOW_SEQUENCE_ENTRY_MAPPING_VALUE_STATE:
return "yaml_PARSE_FLOW_SEQUENCE_ENTRY_MAPPING_VALUE_STATE"
case yaml_PARSE_FLOW_SEQUENCE_ENTRY_MAPPING_END_STATE:
return "yaml_PARSE_FLOW_SEQUENCE_ENTRY_MAPPING_END_STATE"
case yaml_PARSE_FLOW_MAPPING_FIRST_KEY_STATE:
return "yaml_PARSE_FLOW_MAPPING_FIRST_KEY_STATE"
case yaml_PARSE_FLOW_MAPPING_KEY_STATE:
return "yaml_PARSE_FLOW_MAPPING_KEY_STATE"
case yaml_PARSE_FLOW_MAPPING_VALUE_STATE:
return "yaml_PARSE_FLOW_MAPPING_VALUE_STATE"
case yaml_PARSE_FLOW_MAPPING_EMPTY_VALUE_STATE:
return "yaml_PARSE_FLOW_MAPPING_EMPTY_VALUE_STATE"
case yaml_PARSE_END_STATE:
return "yaml_PARSE_END_STATE"
}
return "<unknown parser state>"
}
// This structure holds aliases data.
type yaml_alias_data_t struct {
anchor []byte // The anchor.
index int // The node id.
mark yaml_mark_t // The anchor mark.
}
// The parser structure.
//
// All members are internal. Manage the structure using the
// yaml_parser_ family of functions.
type yaml_parser_t struct {
// Error handling
error yaml_error_type_t // Error type.
problem string // Error description.
// The byte about which the problem occurred.
problem_offset int
problem_value int
problem_mark yaml_mark_t
// The error context.
context string
context_mark yaml_mark_t
// Reader stuff
read_handler yaml_read_handler_t // Read handler.
input_reader io.Reader // File input data.
input []byte // String input data.
input_pos int
eof bool // EOF flag
buffer []byte // The working buffer.
buffer_pos int // The current position of the buffer.
unread int // The number of unread characters in the buffer.
newlines int // The number of line breaks since last non-break/non-blank character
raw_buffer []byte // The raw buffer.
raw_buffer_pos int // The current position of the buffer.
encoding yaml_encoding_t // The input encoding.
offset int // The offset of the current position (in bytes).
mark yaml_mark_t // The mark of the current position.
// Comments
head_comment []byte // The current head comments
line_comment []byte // The current line comments
foot_comment []byte // The current foot comments
tail_comment []byte // Foot comment that happens at the end of a block.
stem_comment []byte // Comment in item preceding a nested structure (list inside list item, etc)
comments []yaml_comment_t // The folded comments for all parsed tokens
comments_head int
// Scanner stuff
stream_start_produced bool // Have we started to scan the input stream?
stream_end_produced bool // Have we reached the end of the input stream?
flow_level int // The number of unclosed '[' and '{' indicators.
tokens []yaml_token_t // The tokens queue.
tokens_head int // The head of the tokens queue.
tokens_parsed int // The number of tokens fetched from the queue.
token_available bool // Does the tokens queue contain a token ready for dequeueing.
indent int // The current indentation level.
indents []int // The indentation levels stack.
simple_key_allowed bool // May a simple key occur at the current position?
simple_keys []yaml_simple_key_t // The stack of simple keys.
simple_keys_by_tok map[int]int // possible simple_key indexes indexed by token_number
// Parser stuff
state yaml_parser_state_t // The current parser state.
states []yaml_parser_state_t // The parser states stack.
marks []yaml_mark_t // The stack of marks.
tag_directives []yaml_tag_directive_t // The list of TAG directives.
// Dumper stuff
aliases []yaml_alias_data_t // The alias data.
document *yaml_document_t // The currently parsed document.
}
type yaml_comment_t struct {
scan_mark yaml_mark_t // Position where scanning for comments started
token_mark yaml_mark_t // Position after which tokens will be associated with this comment
start_mark yaml_mark_t // Position of '#' comment mark
end_mark yaml_mark_t // Position where comment terminated
head []byte
line []byte
foot []byte
}
// Emitter Definitions
// The prototype of a write handler.
//
// The write handler is called when the emitter needs to flush the accumulated
// characters to the output. The handler should write @a size bytes of the
// @a buffer to the output.
//
// @param[in,out] data A pointer to an application data specified by
//
// yaml_emitter_set_output().
//
// @param[in] buffer The buffer with bytes to be written.
// @param[in] size The size of the buffer.
//
// @returns On success, the handler should return @c 1. If the handler failed,
// the returned value should be @c 0.
type yaml_write_handler_t func(emitter *yaml_emitter_t, buffer []byte) error
type yaml_emitter_state_t int
// The emitter states.
const (
// Expect STREAM-START.
yaml_EMIT_STREAM_START_STATE yaml_emitter_state_t = iota
yaml_EMIT_FIRST_DOCUMENT_START_STATE // Expect the first DOCUMENT-START or STREAM-END.
yaml_EMIT_DOCUMENT_START_STATE // Expect DOCUMENT-START or STREAM-END.
yaml_EMIT_DOCUMENT_CONTENT_STATE // Expect the content of a document.
yaml_EMIT_DOCUMENT_END_STATE // Expect DOCUMENT-END.
yaml_EMIT_FLOW_SEQUENCE_FIRST_ITEM_STATE // Expect the first item of a flow sequence.
yaml_EMIT_FLOW_SEQUENCE_TRAIL_ITEM_STATE // Expect the next item of a flow sequence, with the comma already written out
yaml_EMIT_FLOW_SEQUENCE_ITEM_STATE // Expect an item of a flow sequence.
yaml_EMIT_FLOW_MAPPING_FIRST_KEY_STATE // Expect the first key of a flow mapping.
yaml_EMIT_FLOW_MAPPING_TRAIL_KEY_STATE // Expect the next key of a flow mapping, with the comma already written out
yaml_EMIT_FLOW_MAPPING_KEY_STATE // Expect a key of a flow mapping.
yaml_EMIT_FLOW_MAPPING_SIMPLE_VALUE_STATE // Expect a value for a simple key of a flow mapping.
yaml_EMIT_FLOW_MAPPING_VALUE_STATE // Expect a value of a flow mapping.
yaml_EMIT_BLOCK_SEQUENCE_FIRST_ITEM_STATE // Expect the first item of a block sequence.
yaml_EMIT_BLOCK_SEQUENCE_ITEM_STATE // Expect an item of a block sequence.
yaml_EMIT_BLOCK_MAPPING_FIRST_KEY_STATE // Expect the first key of a block mapping.
yaml_EMIT_BLOCK_MAPPING_KEY_STATE // Expect the key of a block mapping.
yaml_EMIT_BLOCK_MAPPING_SIMPLE_VALUE_STATE // Expect a value for a simple key of a block mapping.
yaml_EMIT_BLOCK_MAPPING_VALUE_STATE // Expect a value of a block mapping.
yaml_EMIT_END_STATE // Expect nothing.
)
// The emitter structure.
//
// All members are internal. Manage the structure using the @c yaml_emitter_
// family of functions.
type yaml_emitter_t struct {
// Error handling
error yaml_error_type_t // Error type.
problem string // Error description.
// Writer stuff
write_handler yaml_write_handler_t // Write handler.
output_buffer *[]byte // String output data.
output_writer io.Writer // File output data.
buffer []byte // The working buffer.
buffer_pos int // The current position of the buffer.
raw_buffer []byte // The raw buffer.
raw_buffer_pos int // The current position of the buffer.
encoding yaml_encoding_t // The stream encoding.
// Emitter stuff
canonical bool // If the output is in the canonical style?
best_indent int // The number of indentation spaces.
best_width int // The preferred width of the output lines.
unicode bool // Allow unescaped non-ASCII characters?
line_break yaml_break_t // The preferred line break.
state yaml_emitter_state_t // The current emitter state.
states []yaml_emitter_state_t // The stack of states.
events []yaml_event_t // The event queue.
events_head int // The head of the event queue.
indents []int // The stack of indentation levels.
tag_directives []yaml_tag_directive_t // The list of tag directives.
indent int // The current indentation level.
compact_sequence_indent bool // Is '- ' is considered part of the indentation for sequence elements?
flow_level int // The current flow level.
root_context bool // Is it the document root context?
sequence_context bool // Is it a sequence context?
mapping_context bool // Is it a mapping context?
simple_key_context bool // Is it a simple mapping key context?
line int // The current line.
column int // The current column.
whitespace bool // If the last character was a whitespace?
indention bool // If the last character was an indentation character (' ', '-', '?', ':')?
open_ended bool // If an explicit document end is required?
space_above bool // Is there's an empty line above?
foot_indent int // The indent used to write the foot comment above, or -1 if none.
// Anchor analysis.
anchor_data struct {
anchor []byte // The anchor value.
alias bool // Is it an alias?
}
// Tag analysis.
tag_data struct {
handle []byte // The tag handle.
suffix []byte // The tag suffix.
}
// Scalar analysis.
scalar_data struct {
value []byte // The scalar value.
multiline bool // Does the scalar contain line breaks?
flow_plain_allowed bool // Can the scalar be expessed in the flow plain style?
block_plain_allowed bool // Can the scalar be expressed in the block plain style?
single_quoted_allowed bool // Can the scalar be expressed in the single quoted style?
block_allowed bool // Can the scalar be expressed in the literal or folded styles?
style yaml_scalar_style_t // The output style.
}
// Comments
head_comment []byte
line_comment []byte
foot_comment []byte
tail_comment []byte
key_line_comment []byte
// Dumper stuff
opened bool // If the stream was already opened?
closed bool // If the stream was already closed?
// The information associated with the document nodes.
anchors *struct {
references int // The number of references.
anchor int // The anchor id.
serialized bool // If the node has been emitted?
}
last_anchor_id int // The last assigned anchor id.
document *yaml_document_t // The currently emitted document.
}
+198
View File
@@ -0,0 +1,198 @@
//
// Copyright (c) 2011-2019 Canonical Ltd
// Copyright (c) 2006-2010 Kirill Simonov
//
// Permission is hereby granted, free of charge, to any person obtaining a copy of
// this software and associated documentation files (the "Software"), to deal in
// the Software without restriction, including without limitation the rights to
// use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
// of the Software, and to permit persons to whom the Software is furnished to do
// so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in all
// copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
// SOFTWARE.
package yaml
const (
// The size of the input raw buffer.
input_raw_buffer_size = 512
// The size of the input buffer.
// It should be possible to decode the whole raw buffer.
input_buffer_size = input_raw_buffer_size * 3
// The size of the output buffer.
output_buffer_size = 128
// The size of the output raw buffer.
// It should be possible to encode the whole output buffer.
output_raw_buffer_size = (output_buffer_size*2 + 2)
// The size of other stacks and queues.
initial_stack_size = 16
initial_queue_size = 16
initial_string_size = 16
)
// Check if the character at the specified position is an alphabetical
// character, a digit, '_', or '-'.
func is_alpha(b []byte, i int) bool {
return b[i] >= '0' && b[i] <= '9' || b[i] >= 'A' && b[i] <= 'Z' || b[i] >= 'a' && b[i] <= 'z' || b[i] == '_' || b[i] == '-'
}
// Check if the character at the specified position is a digit.
func is_digit(b []byte, i int) bool {
return b[i] >= '0' && b[i] <= '9'
}
// Get the value of a digit.
func as_digit(b []byte, i int) int {
return int(b[i]) - '0'
}
// Check if the character at the specified position is a hex-digit.
func is_hex(b []byte, i int) bool {
return b[i] >= '0' && b[i] <= '9' || b[i] >= 'A' && b[i] <= 'F' || b[i] >= 'a' && b[i] <= 'f'
}
// Get the value of a hex-digit.
func as_hex(b []byte, i int) int {
bi := b[i]
if bi >= 'A' && bi <= 'F' {
return int(bi) - 'A' + 10
}
if bi >= 'a' && bi <= 'f' {
return int(bi) - 'a' + 10
}
return int(bi) - '0'
}
// Check if the character is ASCII.
func is_ascii(b []byte, i int) bool {
return b[i] <= 0x7F
}
// Check if the character at the start of the buffer can be printed unescaped.
func is_printable(b []byte, i int) bool {
return ((b[i] == 0x0A) || // . == #x0A
(b[i] >= 0x20 && b[i] <= 0x7E) || // #x20 <= . <= #x7E
(b[i] == 0xC2 && b[i+1] >= 0xA0) || // #0xA0 <= . <= #xD7FF
(b[i] > 0xC2 && b[i] < 0xED) ||
(b[i] == 0xED && b[i+1] < 0xA0) ||
(b[i] == 0xEE) ||
(b[i] == 0xEF && // #xE000 <= . <= #xFFFD
!(b[i+1] == 0xBB && b[i+2] == 0xBF) && // && . != #xFEFF
!(b[i+1] == 0xBF && (b[i+2] == 0xBE || b[i+2] == 0xBF))))
}
// Check if the character at the specified position is NUL.
func is_z(b []byte, i int) bool {
return b[i] == 0x00
}
// Check if the beginning of the buffer is a BOM.
func is_bom(b []byte, i int) bool {
return b[0] == 0xEF && b[1] == 0xBB && b[2] == 0xBF
}
// Check if the character at the specified position is space.
func is_space(b []byte, i int) bool {
return b[i] == ' '
}
// Check if the character at the specified position is tab.
func is_tab(b []byte, i int) bool {
return b[i] == '\t'
}
// Check if the character at the specified position is blank (space or tab).
func is_blank(b []byte, i int) bool {
//return is_space(b, i) || is_tab(b, i)
return b[i] == ' ' || b[i] == '\t'
}
// Check if the character at the specified position is a line break.
func is_break(b []byte, i int) bool {
return (b[i] == '\r' || // CR (#xD)
b[i] == '\n' || // LF (#xA)
b[i] == 0xC2 && b[i+1] == 0x85 || // NEL (#x85)
b[i] == 0xE2 && b[i+1] == 0x80 && b[i+2] == 0xA8 || // LS (#x2028)
b[i] == 0xE2 && b[i+1] == 0x80 && b[i+2] == 0xA9) // PS (#x2029)
}
func is_crlf(b []byte, i int) bool {
return b[i] == '\r' && b[i+1] == '\n'
}
// Check if the character is a line break or NUL.
func is_breakz(b []byte, i int) bool {
//return is_break(b, i) || is_z(b, i)
return (
// is_break:
b[i] == '\r' || // CR (#xD)
b[i] == '\n' || // LF (#xA)
b[i] == 0xC2 && b[i+1] == 0x85 || // NEL (#x85)
b[i] == 0xE2 && b[i+1] == 0x80 && b[i+2] == 0xA8 || // LS (#x2028)
b[i] == 0xE2 && b[i+1] == 0x80 && b[i+2] == 0xA9 || // PS (#x2029)
// is_z:
b[i] == 0)
}
// Check if the character is a line break, space, or NUL.
func is_spacez(b []byte, i int) bool {
//return is_space(b, i) || is_breakz(b, i)
return (
// is_space:
b[i] == ' ' ||
// is_breakz:
b[i] == '\r' || // CR (#xD)
b[i] == '\n' || // LF (#xA)
b[i] == 0xC2 && b[i+1] == 0x85 || // NEL (#x85)
b[i] == 0xE2 && b[i+1] == 0x80 && b[i+2] == 0xA8 || // LS (#x2028)
b[i] == 0xE2 && b[i+1] == 0x80 && b[i+2] == 0xA9 || // PS (#x2029)
b[i] == 0)
}
// Check if the character is a line break, space, tab, or NUL.
func is_blankz(b []byte, i int) bool {
//return is_blank(b, i) || is_breakz(b, i)
return (
// is_blank:
b[i] == ' ' || b[i] == '\t' ||
// is_breakz:
b[i] == '\r' || // CR (#xD)
b[i] == '\n' || // LF (#xA)
b[i] == 0xC2 && b[i+1] == 0x85 || // NEL (#x85)
b[i] == 0xE2 && b[i+1] == 0x80 && b[i+2] == 0xA8 || // LS (#x2028)
b[i] == 0xE2 && b[i+1] == 0x80 && b[i+2] == 0xA9 || // PS (#x2029)
b[i] == 0)
}
// Determine the width of the character.
func width(b byte) int {
// Don't replace these by a switch without first
// confirming that it is being inlined.
if b&0x80 == 0x00 {
return 1
}
if b&0xE0 == 0xC0 {
return 2
}
if b&0xF0 == 0xE0 {
return 3
}
if b&0xF8 == 0xF0 {
return 4
}
return 0
}
+10
View File
@@ -0,0 +1,10 @@
# Copyright 2025 The go-yaml Project Contributors
# SPDX-License-Identifier: Apache-2.0
/.cache/
/.claude/
/CLAUDE.md
/yts/testdata/
/go-yaml
/note/
/*.yaml
+33
View File
@@ -0,0 +1,33 @@
# Copyright 2025 The go-yaml Project Contributors
# SPDX-License-Identifier: Apache-2.0
# This is the configuration file of typos (spell checker)
# https://github.com/crate-ci/typos
[files]
# excluded file
extend-exclude = [
"yts/testdata", # third-party test data
]
# setting for Go files configuration
[type.go]
extend-ignore-re = [
'ba-dum-tss\W+', # this one can be found in test files
'"yYnNtTfFoO', # this one can be found in test files
'ba\?r', # this one can be found in test files
]
[type.go.extend-words]
# Here is a list of words we want to ignore in Go files
typ = "typ" # commonly used abbreviation for "type" in Go as "type" is a reserved identifier
# setting for YAML files configuration
[type.yaml]
extend-ignore-re = [
'ba\?r', # this one can be found in test files
]
[default.extend-words]
caf = "caf" # part of "café" shown as "caf\u00e9" in Unicode escape examples
deprecat = "deprecat" # Used as part of a command in a docs/ file
+210
View File
@@ -0,0 +1,210 @@
Contributing to go-yaml
=======================
Thank you for your interest in contributing to go-yaml!
This document provides guidelines and instructions for contributing to this
project.
## Code of Conduct
By participating in this project, you agree to follow our Code of Conduct.
We expect all contributors to:
- Be respectful and inclusive
- Use welcoming and inclusive language
- Be collaborative and constructive
- Focus on what is best for both the Go and YAML communities
## How to Contribute
### Reporting Issues
Before submitting an issue, please:
- Check if the issue already exists in our issue tracker
- Use a clear and descriptive title
- Provide detailed steps to reproduce the issue
- Include relevant code samples and error messages
- Specify your Go version and operating system
- Use the `go-yaml` CLI tool described below
### Using the `go-yaml` CLI Tool
This tool can be used to inspect both the internal stages and final results of
YAML processing with the go-yaml library.
It should be used when reporting most bugs.
The `go-yaml` CLI tool uses the `go.yaml.in/yaml/v4` library to decode and
encode YAML.
Decoding YAML is a multi-stage process that involves tokens, events, and nodes.
The `go-yaml` CLI tool lets you see all of these intermediate stages of the
decoding process.
This is crucial for understanding what go-yaml is doing internally.
The `go-yaml` CLI tool can be built with the `make go-yaml` command or installed
with the `go install go.yaml.in/yaml/v4/cmd/go-yaml@latest` command.
You can learn about all of its options with the `go-yaml -h` command.
Here is an example of using it on a small piece of YAML:
```bash
./go-yaml -t <<< '
foo: &a1 bar
*a1: baz
```
### Coding Conventions
- Follow standard Go coding conventions
- Use `make fmt` to format your code
- Write descriptive comments for non-obvious code
- Add tests for your work
- Keep line length to 80 characters
- Use meaningful variable and function names
- Start doc and comment sentences on a new line
- Test your changes with the `go-yaml` CLI tool when working on parsing logic
### Commit Conventions
- No merge commits
- Commit subject line should:
- Start with a capital letter
- Not end with a period
- Be no more than 50 characters
### Pull Requests
1. Fork the repository
1. Create a new branch for your changes
1. Make your changes following our coding conventions
- If you are not sure about the coding conventions, please ask
- Look at the existing code for examples
1. Write clear commit messages
1. Update tests and documentation
1. Submit a pull request
### Testing
- Ensure all tests pass with `make test`
- Add new tests for new functionality
- Update existing tests when modifying functionality
## Development Process
- This project makes use of a GNU makefile (`GNUmakefile`) for many dev tasks
- The makefile doesn't use your locally installed Go commands; it auto-installs
them, so that all results are deterministic
- Fork and clone the repository
- Make your changes
- Run tests, linters and formatters
- `make fmt`
- `make tidy`
- `make lint`
- `make test`
- You can use `make check` to run all of the above
- Submit a [Pull Request](https://github.com/yaml/go-yaml/pulls)
### Using Your Own Go with the Makefile
We ask that you always test with the makefile installed Go before committing,
since it is deterministic and uses the exact same flow as the go-yaml CI.
We also realize that many Go devs need to run their locally installed Go
commands for their development environment, and might want to use them with
the go-yaml makefile.
If you need to use your own Go utils with the makefile, set `GO_YAML_PATH` to
the directory(s) containing them (either by exporting it or passing it to
`make`).
Something like this:
```bash
export GO_YAML_PATH=$(dirname "$(command -v go)")
make test
# or
make test GO_YAML_PATH=$(dirname "$(command -v go)")
```
**Note:** `GO-VERSION` and `GO_YAML_PATH` are mutually exclusive.
When `GO_YAML_PATH` is set, the makefile uses your own Go environment and
ignores any `GO-VERSION` setting.
### Using the Makefile Environment as a Shell
Sometimes you might want to run your own shell commands using the same binaries
that the makefile installs.
To get a subshell with this environment, run one of:
```bash
make shell
make bash
make zsh
make shell GO-VERSION=1.23.4
```
## Makefile Targets
The repository's makefile (`GNUmakefile`) provides a number of useful targets:
- `make test` runs all tests including yaml-test-suite tests
- `make test-unit` runs just the unit tests
- `make test-internal` runs just the internal tests
- `make test-yts` runs just the yaml-test-suite tests
- `make test v=1 count=3` runs the tests with options
- `make test GO-VERSION=1.23.4` runs the tests with a specific Go version
- `make test GO_YAML_PATH=/path/to/go/bin` uses your own Go installation
- `make shell` opens a shell with the project's dependencies set up
- `make shell GO-VERSION=1.23.4` opens a shell with a specific Go version
- `make fmt` runs `golangci-lint fmt ./...`
- `make lint` runs `golangci-lint run`
- `make tidy` runs `go mod tidy`
- `make distclean` cleans the project completely
## Getting Help
If you need help, you can:
- Open an [issue](https://github.com/yaml/go-yaml/issues) with your question
- Start a [discussion](https://github.com/yaml/go-yaml/discussions)
- Read through our [documentation](https://pkg.go.dev/go.yaml.in/yaml/v4)
- Check the [migration guide](docs/v3-to-v4-migration.md) if upgrading from v3
- Join our [Slack channel](https://cloud-native.slack.com/archives/C08PPAT8PS7)
## We are a Work in Progress
This project is very much a team effort.
We are just getting things rolling and trying to get the foundations in place.
There are lots of opinions and ideas about how to do things, even within the
core team.
Once our process is more mature, we will likely change the rules here.
We'll make the new rules as a team.
For now, please stick to the rules as they are.
This project is focused on serving the needs of both the Go and YAML
communities.
Sometimes those needs can be in conflict, but we'll try to find common ground.
## Thank You
Thank you for contributing to go-yaml!
+191
View File
@@ -0,0 +1,191 @@
# Copyright 2025 The go-yaml Project Contributors
# SPDX-License-Identifier: Apache-2.0
# Auto-install https://github.com/makeplus/makes at specific commit:
MAKES := .cache/makes
MAKES-LOCAL := .cache/local
MAKES-COMMIT ?= 4e48a743c3652b88adc4a257398d895a801e6d11
$(shell [ -d $(MAKES) ] || ( \
git clone -q https://github.com/makeplus/makes $(MAKES) && \
git -C $(MAKES) reset -q --hard $(MAKES-COMMIT)))
ifneq ($(shell git -C $(MAKES) rev-parse HEAD), \
$(shell git -C $(MAKES) rev-parse $(MAKES-COMMIT)))
$(error $(MAKES) is not at the correct commit: $(MAKES-COMMIT). \
Remove $(MAKES) and try again.)
endif
include $(MAKES)/init.mk
include $(MAKES)/shellcheck.mk
# Auto-install go unless GO_YAML_PATH is set:
ifdef GO_YAML_PATH
override export PATH := $(GO_YAML_PATH):$(PATH)
else
GO-VERSION ?= 1.25.5
endif
GO-VERSION-NEEDED := $(GO-VERSION)
# yaml-test-suite info:
YTS-URL ?= https://github.com/yaml/yaml-test-suite
YTS-TAG ?= data-2022-01-17
YTS-DIR := yts/testdata/$(YTS-TAG)
CLI-BINARY := go-yaml
# Pager for viewing documentation:
PAGER ?= less -FRX
# Setup and include go.mk and shell.mk:
# We need to limit `find` to avoid dirs like `.cache/` and any git worktrees,
# as this makes `make` operations very slow:
REPO-DIRS := $(shell find * -maxdepth 0 -type d \
! -exec test -f {}/.git \; -print)
GO-FILES := $(shell find $(REPO-DIRS) -name '*.go')
ifndef GO-VERSION-NEEDED
GO-NO-DEP-GO := true
endif
include $(MAKES)/go.mk
# Set this from the `make` command to override:
GOLANGCI-LINT-VERSION ?= v2.8.0
GOLANGCI-LINT-INSTALLER := \
https://github.com/golangci/golangci-lint/raw/main/install.sh
GOLANGCI-LINT := $(LOCAL-BIN)/golangci-lint
GOLANGCI-LINT-VERSIONED := $(GOLANGCI-LINT)-$(GOLANGCI-LINT-VERSION)
SHELL-DEPS += $(GOLANGCI-LINT-VERSIONED)
ifdef GO-VERSION-NEEDED
GO-DEPS += $(GO)
else
SHELL-DEPS := $(filter-out $(GO),$(SHELL-DEPS))
endif
SHELL-NAME := makes go-yaml
include $(MAKES)/clean.mk
include $(MAKES)/shell.mk
MAKES-CLEAN := $(CLI-BINARY) $(GOLANGCI-LINT)
MAKES-REALCLEAN := $(dir $(YTS-DIR))
SHELL-SCRIPTS = \
util/common.bash \
$(shell grep -rl '^.!/usr/bin/env bash' util | \
grep -v '\.sw')
COVER-TESTS := \
. \
./cmd/... \
./internal/... \
# v=1 for verbose
MAKE := $(MAKE) --no-print-directory
v ?=
cover ?=
fuzz ?=
time ?= 60s
opts ?=
TEST-OPTS := \
$(if $v, -v)\
$(if $(cover), --cover)\
$(if $(fuzz), --fuzz=FuzzEncodeFromJSON --fuzztime=$(time))\
$(if $(opts), $(opts))\
# Test rules:
test: test-main test-internal test-cmd test-yts-all test-shell
@echo 'ALL TESTS PASS'
check:
$(MAKE) fmt
$(MAKE) tidy
$(MAKE) lint
$(MAKE) test
test-main: $(GO-DEPS)
go test .$(TEST-OPTS)
@echo 'ALL MAIN FILES PASS'
test-cmd: $(GO-DEPS)
go test ./cmd/...$(TEST-OPTS)
@echo 'ALL CMD FILES PASS'
test-internal: $(GO-DEPS)
go test ./internal/...$(TEST-OPTS)
@echo 'ALL INTERNAL FILES PASS'
test-cover: $(GO-DEPS)
go test . $(COVER-TESTS) -vet=off --cover$(TEST-OPTS)
test-yts: $(GO-DEPS) $(YTS-DIR)
go test ./yts$(TEST-OPTS)
test-yts-all: $(GO-DEPS) $(YTS-DIR)
@echo 'Testing yaml-test-suite'
util/yaml-test-suite all
test-yts-fail: $(GO-DEPS) $(YTS-DIR)
@echo 'Testing yaml-test-suite failures'
util/yaml-test-suite fail
test-shell: $(SHELLCHECK)
shellcheck $(SHELL-SCRIPTS)
@echo 'ALL SHELL FILES PASS'
test-count: $(GO-DEPS)
util/test-count
yts-dir: $(YTS-DIR)
get-test-data: $(YTS-DIR)
# Install golangci-lint for GitHub Actions:
golangci-lint-install: $(GOLANGCI-LINT)
fmt: $(GOLANGCI-LINT-VERSIONED)
$< fmt ./...
lint: $(GOLANGCI-LINT-VERSIONED)
$< run ./...
tidy: $(GO-DEPS)
go mod tidy
cli: $(CLI-BINARY)
$(CLI-BINARY): $(GO)
go build -o $@ ./cmd/$@
run-examples: $(GO)
@for dir in example/*/; do \
(set -x; go run "$${dir}main.go") || \
{ echo "$$dir failed"; break; }; \
done
# CLI documentation (go doc) - view in terminal:
doc: $(GO-DEPS)
@go doc -all . | $(PAGER)
# HTTP documentation server - opens browser:
doc-http: $(GO-DEPS)
go doc -http -all
# Setup rules:
$(YTS-DIR):
git clone -q $(YTS-URL) $@
git -C $@ checkout -q $(YTS-TAG)
# Downloads golangci-lint binary and moves to versioned path
# (.cache/local/bin/golangci-lint-<version>).
$(GOLANGCI-LINT-VERSIONED): $(GO-DEPS)
curl -sSfL $(GOLANGCI-LINT-INSTALLER) | \
bash -s -- -b $(LOCAL-BIN) $(GOLANGCI-LINT-VERSION)
mv $(GOLANGCI-LINT) $@
# Moves golangci-lint-<version> to golangci-lint for CI requirement
$(GOLANGCI-LINT): $(GOLANGCI-LINT-VERSIONED)
cp $< $@
+202
View File
@@ -0,0 +1,202 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright 2025 - The go-yaml Project Contributors
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
+21
View File
@@ -0,0 +1,21 @@
The following files were ported to Go from C files of libyaml, and thus are
still covered by their original MIT license, with the additional copyright
starting in 2011 when the project was ported over:
- internal/libyaml/api.go
- internal/libyaml/emitter.go
- internal/libyaml/parser.go
- internal/libyaml/reader.go
- internal/libyaml/scanner.go
- internal/libyaml/writer.go
- internal/libyaml/yaml.go
- internal/libyaml/yamlprivate.go
Copyright 2006-2010 Kirill Simonov
https://opensource.org/license/mit
All the remaining project files are covered by the Apache license:
Copyright 2011-2019 Canonical Ltd
Copyright 2025 The go-yaml Project Contributors
http://www.apache.org/licenses/LICENSE-2.0
+266
View File
@@ -0,0 +1,266 @@
go.yaml.in/yaml
===============
YAML Support for the Go Language
## Introduction
The `yaml` package enables [Go](https://go.dev/) programs to comfortably encode
and decode [YAML](https://yaml.org/) values.
It was originally developed within [Canonical](https://www.canonical.com) as
part of the [juju](https://juju.ubuntu.com) project, and is based on a pure Go
port of the well-known [libyaml](http://pyyaml.org/wiki/LibYAML) C library to
parse and generate YAML data quickly and reliably.
## Project Status
This project started as a fork of the extremely popular [go-yaml](
https://github.com/go-yaml/yaml/)
project, and is being maintained by the official [YAML organization](
https://github.com/yaml/).
The YAML team took over ongoing maintenance and development of the project after
discussion with go-yaml's author, @niemeyer, following his decision to
[label the project repository as "unmaintained"](
https://github.com/go-yaml/yaml/blob/944c86a7d2/README.md) in April 2025.
We have put together a team of dedicated maintainers including representatives
of go-yaml's most important downstream projects.
We will strive to earn the trust of the various go-yaml forks to switch back to
this repository as their upstream.
Please [contact us](https://cloud-native.slack.com/archives/C08PPAT8PS7) if you
would like to contribute or be involved.
### Version Intentions
Versions `v1`, `v2`, and `v3` will remain as **frozen legacy**.
They will receive **security-fixes only** so that existing consumers keep
working without breaking changes.
All ongoing work, including new features and routine bug-fixes, will happen in
**`v4`**.
If youre starting a new project or upgrading an existing one, please use the
`go.yaml.in/yaml/v4` import path.
## Compatibility
The `yaml` package supports most of YAML 1.2, but preserves some behavior from
1.1 for backwards compatibility.
Specifically, v3 of the `yaml` package:
* Supports YAML 1.1 bools (`yes`/`no`, `on`/`off`) as long as they are being
decoded into a typed bool value.
Otherwise they behave as a string.
Booleans in YAML 1.2 are `true`/`false` only.
* Supports octals encoded and decoded as `0777` per YAML 1.1, rather than
`0o777` as specified in YAML 1.2, because most parsers still use the old
format.
Octals in the `0o777` format are supported though, so new files work.
* Does not support base-60 floats.
These are gone from YAML 1.2, and were actually never supported by this
package as it's clearly a poor choice.
## Installation and Usage
The import path for the package is *go.yaml.in/yaml/v4*.
To install it, run:
```bash
go get go.yaml.in/yaml/v4
```
## API Documentation
See: <https://pkg.go.dev/go.yaml.in/yaml/v4>
## API Stability
The package API for yaml v3 will remain stable as described in [gopkg.in](
https://gopkg.in).
## Example
```go
package main
import (
"fmt"
"log"
"go.yaml.in/yaml/v4"
)
var data = `
a: Easy!
b:
c: 2
d: [3, 4]
`
// Note: struct fields must be public in order for unmarshal to
// correctly populate the data.
type T struct {
A string
B struct {
RenamedC int `yaml:"c"`
D []int `yaml:",flow"`
}
}
func main() {
t := T{}
err := yaml.Unmarshal([]byte(data), &t)
if err != nil {
log.Fatalf("error: %v", err)
}
fmt.Printf("--- t:\n%v\n\n", t)
d, err := yaml.Marshal(&t)
if err != nil {
log.Fatalf("error: %v", err)
}
fmt.Printf("--- t dump:\n%s\n\n", string(d))
m := make(map[any]any)
err = yaml.Unmarshal([]byte(data), &m)
if err != nil {
log.Fatalf("error: %v", err)
}
fmt.Printf("--- m:\n%v\n\n", m)
d, err = yaml.Marshal(&m)
if err != nil {
log.Fatalf("error: %v", err)
}
fmt.Printf("--- m dump:\n%s\n\n", string(d))
}
```
This example will generate the following output:
```
--- t:
{Easy! {2 [3 4]}}
--- t dump:
a: Easy!
b:
c: 2
d: [3, 4]
--- m:
map[a:Easy! b:map[c:2 d:[3 4]]]
--- m dump:
a: Easy!
b:
c: 2
d:
- 3
- 4
```
## Development and Testing with `make`
This project's makefile (`GNUmakefile`) is set up to support all of the
project's testing, automation and development tasks in a completely
deterministic way.
Some `make` commands are:
* `make test`
* `make lint tidy`
* `make test-shell`
* `make test v=1`
* `make test o='-foo --bar=baz'` # Add extra CLI options
* `make test GO-VERSION=1.2.34`
* `make test GO_YAML_PATH=/usr/local/go/bin`
* `make shell` # Start a shell with the local `go` environment
* `make shell GO-VERSION=1.2.34`
* `make distclean` # Remove all generated files including `.cache/`
### Dependency Auto-install
By default, this makefile will not use your system's Go installation, or any
other system tools that it needs.
The only things from your system that it relies on are:
* Linux or macOS
* GNU `make` (3.81+)
* `git`
* `bash`
* `curl`
Everything else, including Go and Go utils, are installed and cached as they
are needed by the makefile (under `.cache/`).
> **Note**: Use `make shell` to get a subshell with the same environment that
> the makefile set up for its commands.
### Using your own Go
If you want to use your own Go installation and utils, export `GO_YAML_PATH` to
the directory containing the `go` binary.
Use something like this:
```
export GO_YAML_PATH=$(dirname "$(command -v go)")
make <rule>
# or:
make <rule> GO_YAML_PATH=$(dirname "$(command -v go)")
```
> **Note:** `GO-VERSION` and `GO_YAML_PATH` are mutually exclusive.
> When `GO_YAML_PATH` is set, the Makefile uses your own Go installation and
> ignores any `GO-VERSION` setting.
## The `go-yaml` CLI Tool
This repository includes a `go-yaml` CLI tool which can be used to understand
the internal stages and final results of YAML processing with the go-yaml
library.
We strongly encourage you to show pertinent output from this command when
reporting and discussing issues.
```bash
make go-yaml
./go-yaml --help
./go-yaml <<< '
foo: &a1 bar
*a1: baz
' -n # Show value on decoded Node structs (formatted in YAML)
```
You can also install it with:
```bash
go install go.yaml.in/yaml/v4/cmd/go-yaml@latest
```
## License
The yaml package is licensed under the MIT and Apache License 2.0 licenses.
Please see the LICENSE file for details.
+115
View File
@@ -0,0 +1,115 @@
//
// Copyright (c) 2025 The go-yaml Project Contributors
// SPDX-License-Identifier: Apache-2.0
//
// Package yaml implements YAML 1.1/1.2 encoding and decoding for Go programs.
//
// # Quick Start
//
// For simple encoding and decoding, use [Unmarshal] and [Marshal]:
//
// type Config struct {
// Name string `yaml:"name"`
// Version string `yaml:"version"`
// }
//
// // Decode YAML to Go struct
// var config Config
// err := yaml.Unmarshal(yamlData, &config)
//
// // Encode Go struct to YAML
// data, err := yaml.Marshal(&config)
//
// For encoding/decoding with options, use [Load] and [Dump]:
//
// // Decode with strict field checking
// err := yaml.Load(data, &config, yaml.WithKnownFields())
//
// // Encode with custom indent
// data, err := yaml.Dump(&config, yaml.WithIndent(2))
//
// // Decode all documents from multi-document stream
// var docs []Config
// err := yaml.Load(multiDocYAML, &docs, yaml.WithAllDocuments())
//
// // Encode multiple documents as multi-document stream
// docs := []Config{config1, config2}
// data, err := yaml.Dump(docs, yaml.WithAllDocuments())
//
// # Streaming with Loader and Dumper
//
// For multi-document streams or when you need custom options, use [Loader] and [Dumper]:
//
// // Load multiple documents from a stream
// loader, err := yaml.NewLoader(reader)
// if err != nil {
// log.Fatal(err)
// }
// for {
// var doc any
// if err := loader.Load(&doc); err == io.EOF {
// break
// } else if err != nil {
// log.Fatal(err)
// }
// // Process document...
// }
//
// // Dump multiple documents to a stream
// dumper, err := yaml.NewDumper(writer, yaml.WithIndent(2))
// if err != nil {
// log.Fatal(err)
// }
// dumper.Dump(&doc1)
// dumper.Dump(&doc2)
// dumper.Close()
//
// # Options System
//
// Configure YAML processing behavior with functional options:
//
// yaml.NewDumper(w,
// yaml.WithIndent(2), // Indentation spacing
// yaml.WithCompactSeqIndent(), // Compact sequences (defaults to true)
// yaml.WithLineWidth(80), // Line wrapping width
// yaml.WithUnicode(false), // Escape non-ASCII (override default true)
// yaml.WithKnownFields(), // Strict field checking (defaults to true)
// yaml.WithUniqueKeys(), // Prevent duplicate keys (defaults to true)
// yaml.WithSingleDocument(), // Single document mode
// )
//
// Or use version-specific option presets for consistent formatting:
//
// yaml.NewDumper(w, yaml.V3)
//
// Options can be combined and later options override earlier ones:
//
// // Start with v3 defaults, then override indent
// yaml.NewDumper(w,
// yaml.V3,
// yaml.WithIndent(4),
// )
//
// Load options from YAML configuration files:
//
// opts, err := yaml.OptsYAML(configYAML)
// dumper, err := yaml.NewDumper(w, opts)
//
// # YAML Compatibility
//
// This package supports most of YAML 1.2, but preserves some YAML 1.1
// behavior for backward compatibility:
//
// - YAML 1.1 booleans (yes/no, on/off) are supported when decoding into
// typed bool values, otherwise treated as strings
// - Octals can use 0777 format (YAML 1.1) or 0o777 format (YAML 1.2)
// - Base-60 floats are not supported (removed in YAML 1.2)
//
// # Version Defaults
//
// [NewLoader] and [NewDumper] use v4 defaults (2-space indentation, compact
// sequences). The older [Marshal] and [Unmarshal] functions use v3 defaults
// for backward compatibility. Use the options system to select different
// version defaults if needed.
package yaml
+118
View File
@@ -0,0 +1,118 @@
// Copyright 2025 The go-yaml Project Contributors
// SPDX-License-Identifier: Apache-2.0
// This file contains the Dumper API for writing YAML documents.
//
// Primary functions:
// - Dump: Encode value(s) to YAML (use WithAll for multi-doc)
// - NewDumper: Create a streaming dumper to io.Writer
package yaml
import (
"bytes"
"errors"
"io"
"reflect"
"go.yaml.in/yaml/v4/internal/libyaml"
)
// Dump encodes a value to YAML with the given options.
//
// By default, Dump encodes a single value as a single YAML document.
//
// Use WithAllDocuments() to encode multiple values as a multi-document stream:
//
// docs := []Config{config1, config2, config3}
// yaml.Dump(docs, yaml.WithAllDocuments())
//
// When WithAllDocuments is used, in must be a slice.
// Each element is encoded as a separate YAML document with "---" separators.
//
// See [Marshal] for details about the conversion of Go values to YAML.
func Dump(in any, opts ...Option) (out []byte, err error) {
defer handleErr(&err)
o, err := libyaml.ApplyOptions(opts...)
if err != nil {
return nil, err
}
var buf bytes.Buffer
d, err := NewDumper(&buf, func(opts *libyaml.Options) error {
*opts = *o // Copy options
return nil
})
if err != nil {
return nil, err
}
if o.AllDocuments {
// Multi-document mode: in must be a slice
inVal := reflect.ValueOf(in)
if inVal.Kind() != reflect.Slice {
return nil, &LoadErrors{Errors: []*libyaml.ConstructError{{
Err: errors.New("yaml: WithAllDocuments requires a slice input"),
}}}
}
// Dump each element as a separate document
for i := 0; i < inVal.Len(); i++ {
if err := d.Dump(inVal.Index(i).Interface()); err != nil {
return nil, err
}
}
} else {
// Single-document mode
if err := d.Dump(in); err != nil {
return nil, err
}
}
if err := d.Close(); err != nil {
return nil, err
}
return buf.Bytes(), nil
}
// A Dumper writes YAML values to an output stream with configurable options.
type Dumper struct {
encoder *libyaml.Representer
opts *libyaml.Options
}
// NewDumper returns a new Dumper that writes to w with the given options.
//
// The Dumper should be closed after use to flush all data to w.
func NewDumper(w io.Writer, opts ...Option) (*Dumper, error) {
o, err := libyaml.ApplyOptions(opts...)
if err != nil {
return nil, err
}
return &Dumper{
encoder: libyaml.NewRepresenter(w, o),
opts: o,
}, nil
}
// Dump writes the YAML encoding of v to the stream.
//
// If multiple values are dumped to the stream, the second and subsequent
// documents will be preceded with a "---" document separator.
//
// See the documentation for [Marshal] for details about the conversion of Go
// values to YAML.
func (d *Dumper) Dump(v any) (err error) {
defer handleErr(&err)
d.encoder.MarshalDoc("", reflect.ValueOf(v))
return nil
}
// Close closes the Dumper by writing any remaining data.
// It does not write a stream terminating string "...".
func (d *Dumper) Close() (err error) {
defer handleErr(&err)
d.encoder.Finish()
return nil
}
+533
View File
@@ -0,0 +1,533 @@
# internal/libyaml
This package provides low-level YAML processing functionality through a 3-stage
pipeline: Scanner → Parser → Emitter.
It implements the libyaml C library functionality in Go.
## Directory Overview
The `internal/libyaml` package implements the core YAML processing stages:
1. **Scanner** - Tokenizes YAML text into tokens
2. **Parser** - Converts tokens into events following YAML grammar rules
3. **Emitter** - Serializes events back into YAML text
## File Organization
### Main Source Files
- **scanner.go** - YAML scanner/tokenizer implementation
- **parser.go** - YAML parser (tokens → events)
- **emitter.go** - YAML emitter (events → YAML output)
- **api.go** - Public API for Parser and Emitter types
- **yaml.go** - Core types and constants (Event, Token, enums)
- **reader.go** - Input handling and encoding detection
- **writer.go** - Output handling
- **yamlprivate.go** - Internal types and helper functions
### Test Files
- **scanner_test.go** - Scanner tests
- **parser_test.go** - Parser tests
- **emitter_test.go** - Emitter tests
- **api_test.go** - API tests
- **yaml_test.go** - Utility function tests
- **reader_test.go** - Reader tests
- **writer_test.go** - Writer tests
- **yamlprivate_test.go** - Character classification tests
- **loader_test.go** - Data loader scalar resolution tests
- **yamldatatest_test.go** - YAML test data loading framework
- **yamldatatest_loader.go** - YAML test data loader with scalar type resolution (exported for reuse)
### Test Data Files (in `testdata/`)
- **scanner.yaml** - Scanner test cases
- **parser.yaml** - Parser test cases
- **emitter.yaml** - Emitter test cases
- **api.yaml** - API test cases
- **yaml.yaml** - Utility function test cases
- **reader.yaml** - Reader test cases
- **writer.yaml** - Writer test cases
- **yamlprivate.yaml** - Character classification test cases
- **loader.yaml** - Data loader scalar resolution test cases
## Processing Pipeline
### 1. Scanner (scanner.go)
The scanner converts YAML text into tokens.
**Input**: Raw YAML text (string or []byte)
**Output**: Stream of tokens
**Token types include**:
- `SCALAR_TOKEN` - Plain, quoted, or block scalar values
- `KEY_TOKEN`, `VALUE_TOKEN` - Mapping key/value indicators
- `BLOCK_MAPPING_START_TOKEN`, `FLOW_MAPPING_START_TOKEN` - Mapping delimiters
- `BLOCK_SEQUENCE_START_TOKEN`, `FLOW_SEQUENCE_START_TOKEN` - Sequence delimiters
- `ANCHOR_TOKEN`, `ALIAS_TOKEN` - Anchor definitions and references
- `TAG_TOKEN` - Type tags
- `DOCUMENT_START_TOKEN`, `DOCUMENT_END_TOKEN` - Document boundaries
**Responsibilities**:
- Character encoding detection (UTF-8, UTF-16LE, UTF-16BE)
- Line break normalization
- Indentation tracking
- Quote and escape sequence handling
### 2. Parser (parser.go)
The parser converts tokens into events following YAML grammar rules.
**Input**: Stream of tokens from Scanner
**Output**: Stream of events
**Event types include**:
- `STREAM_START_EVENT`, `STREAM_END_EVENT` - Stream boundaries
- `DOCUMENT_START_EVENT`, `DOCUMENT_END_EVENT` - Document boundaries
- `SCALAR_EVENT` - Scalar values
- `MAPPING_START_EVENT`, `MAPPING_END_EVENT` - Mapping boundaries
- `SEQUENCE_START_EVENT`, `SEQUENCE_END_EVENT` - Sequence boundaries
- `ALIAS_EVENT` - Anchor references
**Responsibilities**:
- Implementing YAML grammar and validation
- Managing document directives (%YAML, %TAG)
- Resolving anchors and aliases
- Tracking implicit vs explicit markers
- Style preservation (plain, single-quoted, double-quoted, literal, folded)
### 3. Emitter (emitter.go)
The emitter converts events back into YAML text.
**Input**: Stream of events
**Output**: YAML text
**Responsibilities**:
- Style selection (plain/quoted scalars, block/flow collections)
- Formatting control (canonical mode, indentation, line width)
- Character encoding
- Anchor and tag serialization
- Document marker generation (---, ...)
**Configuration options**:
- `Canonical` - Emit in canonical YAML form
- `Indent` - Indentation width (2-9 spaces)
- `Width` - Line width (-1 for unlimited)
- `Unicode` - Enable Unicode character output
- `LineBreak` - Line break style (LN, CR, CRLN)
## Testing Framework
### Test Architecture
The testing framework uses a data-driven approach:
1. **Test data** is stored in YAML files in the `testdata/` directory
2. **Test logic** is implemented in Go files (`*_test.go`)
3. **One-to-one pairing**: Each `testdata/foo.yaml` has a corresponding `foo_test.go`
**Benefits**:
- Easy to add new test cases without writing Go code
- Test data is human-readable and self-documenting
- Test logic is reusable across many test cases
- Test data is separated from test code for clarity
- Tests can become a common suite for multiple YAML frameworks
### Test Data Files
Each YAML file contains test cases for a specific component:
- **scanner.yaml** - Scanner/tokenization tests
- Token sequence verification
- Token property validation (value, style)
- Error detection
- **parser.yaml** - Parser/event generation tests
- Event sequence verification
- Event property validation (anchor, tag, value, directives)
- Error detection
- **emitter.yaml** - Emitter/serialization tests
- Event-to-YAML conversion
- Configuration options testing
- Roundtrip testing (parse → emit)
- Writer integration
- **api.yaml** - API constructor and method tests
- Constructor validation
- Method behavior and state changes
- Panic conditions
- Cleanup verification
- **yaml.yaml** - Utility function tests
- Enum String() methods
- Style accessor methods
- **reader.yaml** - Reader/input handling tests
- Encoding detection (UTF-8, UTF-16LE, UTF-16BE)
- Buffer management
- Error handling
- **writer.yaml** - Writer/output handling tests
- Buffer flushing
- Output handlers (string, io.Writer)
- Error conditions
- **yamlprivate.yaml** - Character classification tests
- Character type predicates (isAlpha, isDigit, isHex, etc.)
- Character conversion functions (asDigit, asHex, width)
- Unicode handling
- **loader.yaml** - Data loader scalar resolution tests
- Numeric type resolution (integers, floats)
- Boolean and null value handling
- String vs numeric type disambiguation
- Mixed-type collections
### Test Framework Implementation
The test framework is implemented in `yamldatatest_loader.go` and `yamldatatest_test.go`:
**Core functions**:
- `LoadYAML(data []byte) (interface{}, error)` - Parses YAML using libyaml parser with scalar type resolution (exported)
- `UnmarshalStruct(target interface{}, data map[string]interface{}) error` - Populates structs (exported)
- `LoadTestCases(filename string) ([]TestCase, error)` - Loads and parses test YAML files
- `coerceScalar(value string) interface{}` - Resolves scalar strings to appropriate Go types (int, float64, bool, nil, string)
**Core types**:
- `TestCase` struct - Umbrella structure containing fields for all test types
- Uses `interface{}` for flexible field types
- Post-processing converts generic fields to specific types
**Post-processing**:
After loading, the framework processes test data:
- Converts `Want` (interface{}) to `WantEvents`, `WantTokens`, or `WantSpecs` based on test type
- Converts `Want` (interface{}) to `WantContains` (handles both scalar and sequence)
- Converts `Checks` to field validation specifications
### Test Types
#### Scanner Tests
**scan-tokens** - Verify token sequence
```yaml
- scan-tokens:
name: Simple scalar
yaml: |-
hello
want:
- STREAM_START_TOKEN
- SCALAR_TOKEN
- STREAM_END_TOKEN
```
**scan-tokens-detailed** - Verify token properties
```yaml
- scan-tokens-detailed:
name: Single quoted scalar
yaml: |-
'hello world'
want:
- STREAM_START_TOKEN
- SCALAR_TOKEN:
style: SINGLE_QUOTED_SCALAR_STYLE
value: hello world
- STREAM_END_TOKEN
```
**scan-error** - Verify error detection
```yaml
- scan-error:
name: Invalid character
yaml: "\x01"
```
#### Parser Tests
**parse-events** - Verify event sequence
```yaml
- parse-events:
name: Simple mapping
yaml: |
key: value
want:
- STREAM_START_EVENT
- DOCUMENT_START_EVENT
- MAPPING_START_EVENT
- SCALAR_EVENT
- SCALAR_EVENT
- MAPPING_END_EVENT
- DOCUMENT_END_EVENT
- STREAM_END_EVENT
```
**parse-events-detailed** - Verify event properties
```yaml
- parse-events-detailed:
name: Anchor and alias
yaml: |
- &anchor value
- *anchor
want:
- STREAM_START_EVENT
- DOCUMENT_START_EVENT
- SEQUENCE_START_EVENT
- SCALAR_EVENT:
anchor: anchor
value: value
- ALIAS_EVENT:
anchor: anchor
- SEQUENCE_END_EVENT
- DOCUMENT_END_EVENT
- STREAM_END_EVENT
```
**parse-error** - Verify error detection
```yaml
- parse-error:
name: Error state
yaml: |
key: : invalid
```
#### Emitter Tests
**emit** - Emit events and verify output contains expected strings
```yaml
- emit:
name: Simple scalar
data:
- STREAM_START_EVENT:
encoding: UTF8_ENCODING
- DOCUMENT_START_EVENT:
implicit: true
- SCALAR_EVENT:
value: hello
implicit: true
style: PLAIN_SCALAR_STYLE
- DOCUMENT_END_EVENT:
implicit: true
- STREAM_END_EVENT
want: hello
```
**emit-config** - Emit with configuration
```yaml
- emit-config:
name: Custom indent
conf:
indent: 4
data:
- STREAM_START_EVENT:
encoding: UTF8_ENCODING
- DOCUMENT_START_EVENT:
implicit: true
- MAPPING_START_EVENT:
implicit: true
style: BLOCK_MAPPING_STYLE
# ... more events
want: key
```
**roundtrip** - Parse → emit, verify output
```yaml
- roundtrip:
name: Roundtrip
yaml: |
key: value
list:
- item1
- item2
want:
- key
- value
- item1
```
**emit-writer** - Emit to io.Writer
```yaml
- emit-writer:
name: Writer
data:
- STREAM_START_EVENT:
encoding: UTF8_ENCODING
# ... more events
want: test
```
#### API Tests
**api-new** - Test constructors
```yaml
- api-new:
name: New parser
with: NewParser
test:
- nil: [raw-buffer, false]
- cap: [raw-buffer, 512]
- nil: [buffer, false]
- cap: [buffer, 1536]
```
**api-method** - Test methods and field state
```yaml
- api-method:
name: Parser set input string
with: NewParser
byte: true
call: [SetInputString, 'key: value']
test:
- eq: [input, 'key: value']
- eq: [input-pos, 0]
- nil: [read-handler, false]
```
**api-panic** - Test methods that should panic
```yaml
- api-panic:
name: Parser set input string twice
with: NewParser
byte: true
init: [SetInputString, first]
call: [SetInputString, second]
want: must set the input source only once
```
**api-delete** - Test cleanup
```yaml
- api-delete:
name: Parser delete
with: NewParser
byte: true
init: [SetInputString, test]
test:
- len: [input, 0]
- len: [buffer, 0]
```
**api-new-event** - Test event constructors
```yaml
- api-new-event:
name: New stream start event
call: [NewStreamStartEvent, UTF8_ENCODING]
test:
- eq: [Type, STREAM_START_EVENT]
- eq: [encoding, UTF8_ENCODING]
```
#### Utility Tests
**enum-string** - Test String() methods of enums
```yaml
- enum-string:
name: Scalar style plain
enum: [ScalarStyle, PLAIN_SCALAR_STYLE]
want: Plain
```
**style-accessor** - Test style accessor methods
```yaml
- style-accessor:
name: Event scalar style
test: [ScalarStyle, DOUBLE_QUOTED_SCALAR_STYLE]
```
#### Loader Tests
**scalar-resolution** - Test scalar type resolution
```yaml
- scalar-resolution:
name: Positive integer
yaml: "42"
want: 42
- scalar-resolution:
name: Negative float
yaml: "-2.5"
want: -2.5
```
**Resolution order**:
1. Boolean (true, false)
2. Null (null keyword only)
3. Hexadecimal integer (0x prefix)
4. Float (contains .)
5. Decimal integer
6. String (fallback)
### Common Keys in Test YAML Files
Test cases use a **type-as-key** format where the test type is the map key:
```yaml
- test-type:
name: Test case name
# ... other fields
```
**Common fields**:
- **name** - Test case name (title case convention)
- **yaml** - Input YAML string to test
- **want** - Expected result (format varies by test type)
- For api-panic: string containing expected panic message substring
- For scan-error/parse-error: boolean (defaults to true if omitted; set to false if no error expected)
- For enum-string: string representing expected String() output
- For other types: varies (may be sequence or scalar)
- **data** - For emitter tests: list of event specifications to emit
- **conf** - For emitter config tests: emitter configuration options
- **with** - For API tests: constructor name (NewParser, NewEmitter)
- **call** - For API tests: method call [MethodName, arg1, arg2, ...]
- **init** - For API panic tests: setup method call before main method
- **byte** - For API tests: boolean flag to convert string args to []byte
- **test** - For API tests: list of field validation checks in format `operator: [field, value]` where operator is one of: nil, cap, len, eq, gte, len-gt.
- **test** - For style-accessor tests: array of [Method, STYLE] where Method is the accessor method (e.g., ScalarStyle) and STYLE is the style constant (e.g., DOUBLE_QUOTED_SCALAR_STYLE).
- **enum** - For enum tests: array of [Type, Value] where Type is the enum type (e.g., ScalarStyle) and Value is the constant (e.g., PLAIN_SCALAR_STYLE)
**Note on scalar type resolution**: Unquoted scalar values in test data are automatically resolved to appropriate Go types (int, float64, bool, nil) by the `LoadYAML` function. Quoted scalars remain as strings.
### Running Tests
```bash
# Run all tests in the package
go test ./internal/libyaml
# Run specific test file
go test ./internal/libyaml -run TestScanner
go test ./internal/libyaml -run TestParser
go test ./internal/libyaml -run TestEmitter
go test ./internal/libyaml -run TestAPI
go test ./internal/libyaml -run TestYAML
go test ./internal/libyaml -run TestLoader
# Run specific test case (using subtest name)
go test ./internal/libyaml -run TestScanner/Block_sequence
go test ./internal/libyaml -run TestParser/Anchor_and_alias
go test ./internal/libyaml -run TestEmitter/Flow_mapping
go test ./internal/libyaml -run TestLoader/Scientific_notation_lowercase_e
# Run with verbose output
go test -v ./internal/libyaml
# Run with coverage
go test -cover ./internal/libyaml
```
+733
View File
@@ -0,0 +1,733 @@
// Copyright 2006-2010 Kirill Simonov
// Copyright 2011-2019 Canonical Ltd
// Copyright 2025 The go-yaml Project Contributors
// SPDX-License-Identifier: Apache-2.0 AND MIT
// High-level API helpers for parser and emitter initialization and
// configuration.
// Provides convenience functions for token insertion and stream management.
package libyaml
import (
"io"
)
func (parser *Parser) insertToken(pos int, token *Token) {
// fmt.Println("yaml_insert_token", "pos:", pos, "typ:", token.typ, "head:", parser.tokens_head, "len:", len(parser.tokens))
// Check if we can move the queue at the beginning of the buffer.
if parser.tokens_head > 0 && len(parser.tokens) == cap(parser.tokens) {
if parser.tokens_head != len(parser.tokens) {
copy(parser.tokens, parser.tokens[parser.tokens_head:])
}
parser.tokens = parser.tokens[:len(parser.tokens)-parser.tokens_head]
parser.tokens_head = 0
}
parser.tokens = append(parser.tokens, *token)
if pos < 0 {
return
}
copy(parser.tokens[parser.tokens_head+pos+1:], parser.tokens[parser.tokens_head+pos:])
parser.tokens[parser.tokens_head+pos] = *token
}
// NewParser creates a new parser object.
func NewParser() Parser {
return Parser{
raw_buffer: make([]byte, 0, input_raw_buffer_size),
buffer: make([]byte, 0, input_buffer_size),
}
}
// Delete a parser object.
func (parser *Parser) Delete() {
*parser = Parser{}
}
// String read handler.
func yamlStringReadHandler(parser *Parser, buffer []byte) (n int, err error) {
if parser.input_pos == len(parser.input) {
return 0, io.EOF
}
n = copy(buffer, parser.input[parser.input_pos:])
parser.input_pos += n
return n, nil
}
// Reader read handler.
func yamlReaderReadHandler(parser *Parser, buffer []byte) (n int, err error) {
return parser.input_reader.Read(buffer)
}
// SetInputString sets a string input.
func (parser *Parser) SetInputString(input []byte) {
if parser.read_handler != nil {
panic("must set the input source only once")
}
parser.read_handler = yamlStringReadHandler
parser.input = input
parser.input_pos = 0
}
// SetInputReader sets a file input.
func (parser *Parser) SetInputReader(r io.Reader) {
if parser.read_handler != nil {
panic("must set the input source only once")
}
parser.read_handler = yamlReaderReadHandler
parser.input_reader = r
}
// SetEncoding sets the source encoding.
func (parser *Parser) SetEncoding(encoding Encoding) {
if parser.encoding != ANY_ENCODING {
panic("must set the encoding only once")
}
parser.encoding = encoding
}
// GetPendingComments returns the parser's comment queue for CLI access.
func (parser *Parser) GetPendingComments() []Comment {
return parser.comments
}
// GetCommentsHead returns the current position in the comment queue.
func (parser *Parser) GetCommentsHead() int {
return parser.comments_head
}
// NewEmitter creates a new emitter object.
func NewEmitter() Emitter {
return Emitter{
buffer: make([]byte, output_buffer_size),
states: make([]EmitterState, 0, initial_stack_size),
events: make([]Event, 0, initial_queue_size),
best_width: -1,
}
}
// Delete an emitter object.
func (emitter *Emitter) Delete() {
*emitter = Emitter{}
}
// String write handler.
func yamlStringWriteHandler(emitter *Emitter, buffer []byte) error {
*emitter.output_buffer = append(*emitter.output_buffer, buffer...)
return nil
}
// yamlWriterWriteHandler uses emitter.output_writer to write the
// emitted text.
func yamlWriterWriteHandler(emitter *Emitter, buffer []byte) error {
_, err := emitter.output_writer.Write(buffer)
return err
}
// SetOutputString sets a string output.
func (emitter *Emitter) SetOutputString(output_buffer *[]byte) {
if emitter.write_handler != nil {
panic("must set the output target only once")
}
emitter.write_handler = yamlStringWriteHandler
emitter.output_buffer = output_buffer
}
// SetOutputWriter sets a file output.
func (emitter *Emitter) SetOutputWriter(w io.Writer) {
if emitter.write_handler != nil {
panic("must set the output target only once")
}
emitter.write_handler = yamlWriterWriteHandler
emitter.output_writer = w
}
// SetEncoding sets the output encoding.
func (emitter *Emitter) SetEncoding(encoding Encoding) {
if emitter.encoding != ANY_ENCODING {
panic("must set the output encoding only once")
}
emitter.encoding = encoding
}
// SetCanonical sets the canonical output style.
func (emitter *Emitter) SetCanonical(canonical bool) {
emitter.canonical = canonical
}
// SetIndent sets the indentation increment.
func (emitter *Emitter) SetIndent(indent int) {
if indent < 2 || indent > 9 {
indent = 2
}
emitter.BestIndent = indent
}
// SetWidth sets the preferred line width.
func (emitter *Emitter) SetWidth(width int) {
if width < 0 {
width = -1
}
emitter.best_width = width
}
// SetUnicode sets if unescaped non-ASCII characters are allowed.
func (emitter *Emitter) SetUnicode(unicode bool) {
emitter.unicode = unicode
}
// SetLineBreak sets the preferred line break character.
func (emitter *Emitter) SetLineBreak(line_break LineBreak) {
emitter.line_break = line_break
}
///*
// * Destroy a token object.
// */
//
//YAML_DECLARE(void)
//yaml_token_delete(yaml_token_t *token)
//{
// assert(token); // Non-NULL token object expected.
//
// switch (token.type)
// {
// case YAML_TAG_DIRECTIVE_TOKEN:
// yaml_free(token.data.tag_directive.handle);
// yaml_free(token.data.tag_directive.prefix);
// break;
//
// case YAML_ALIAS_TOKEN:
// yaml_free(token.data.alias.value);
// break;
//
// case YAML_ANCHOR_TOKEN:
// yaml_free(token.data.anchor.value);
// break;
//
// case YAML_TAG_TOKEN:
// yaml_free(token.data.tag.handle);
// yaml_free(token.data.tag.suffix);
// break;
//
// case YAML_SCALAR_TOKEN:
// yaml_free(token.data.scalar.value);
// break;
//
// default:
// break;
// }
//
// memset(token, 0, sizeof(yaml_token_t));
//}
//
///*
// * Check if a string is a valid UTF-8 sequence.
// *
// * Check 'reader.c' for more details on UTF-8 encoding.
// */
//
//static int
//yaml_check_utf8(yaml_char_t *start, size_t length)
//{
// yaml_char_t *end = start+length;
// yaml_char_t *pointer = start;
//
// while (pointer < end) {
// unsigned char octet;
// unsigned int width;
// unsigned int value;
// size_t k;
//
// octet = pointer[0];
// width = (octet & 0x80) == 0x00 ? 1 :
// (octet & 0xE0) == 0xC0 ? 2 :
// (octet & 0xF0) == 0xE0 ? 3 :
// (octet & 0xF8) == 0xF0 ? 4 : 0;
// value = (octet & 0x80) == 0x00 ? octet & 0x7F :
// (octet & 0xE0) == 0xC0 ? octet & 0x1F :
// (octet & 0xF0) == 0xE0 ? octet & 0x0F :
// (octet & 0xF8) == 0xF0 ? octet & 0x07 : 0;
// if (!width) return 0;
// if (pointer+width > end) return 0;
// for (k = 1; k < width; k ++) {
// octet = pointer[k];
// if ((octet & 0xC0) != 0x80) return 0;
// value = (value << 6) + (octet & 0x3F);
// }
// if (!((width == 1) ||
// (width == 2 && value >= 0x80) ||
// (width == 3 && value >= 0x800) ||
// (width == 4 && value >= 0x10000))) return 0;
//
// pointer += width;
// }
//
// return 1;
//}
//
// NewStreamStartEvent creates a new STREAM-START event.
func NewStreamStartEvent(encoding Encoding) Event {
return Event{
Type: STREAM_START_EVENT,
encoding: encoding,
}
}
// NewStreamEndEvent creates a new STREAM-END event.
func NewStreamEndEvent() Event {
return Event{
Type: STREAM_END_EVENT,
}
}
// NewDocumentStartEvent creates a new DOCUMENT-START event.
func NewDocumentStartEvent(version_directive *VersionDirective, tag_directives []TagDirective, implicit bool) Event {
return Event{
Type: DOCUMENT_START_EVENT,
versionDirective: version_directive,
tagDirectives: tag_directives,
Implicit: implicit,
}
}
// NewDocumentEndEvent creates a new DOCUMENT-END event.
func NewDocumentEndEvent(implicit bool) Event {
return Event{
Type: DOCUMENT_END_EVENT,
Implicit: implicit,
}
}
// NewAliasEvent creates a new ALIAS event.
func NewAliasEvent(anchor []byte) Event {
return Event{
Type: ALIAS_EVENT,
Anchor: anchor,
}
}
// NewScalarEvent creates a new SCALAR event.
func NewScalarEvent(anchor, tag, value []byte, plain_implicit, quoted_implicit bool, style ScalarStyle) Event {
return Event{
Type: SCALAR_EVENT,
Anchor: anchor,
Tag: tag,
Value: value,
Implicit: plain_implicit,
quoted_implicit: quoted_implicit,
Style: Style(style),
}
}
// NewSequenceStartEvent creates a new SEQUENCE-START event.
func NewSequenceStartEvent(anchor, tag []byte, implicit bool, style SequenceStyle) Event {
return Event{
Type: SEQUENCE_START_EVENT,
Anchor: anchor,
Tag: tag,
Implicit: implicit,
Style: Style(style),
}
}
// NewSequenceEndEvent creates a new SEQUENCE-END event.
func NewSequenceEndEvent() Event {
return Event{
Type: SEQUENCE_END_EVENT,
}
}
// NewMappingStartEvent creates a new MAPPING-START event.
func NewMappingStartEvent(anchor, tag []byte, implicit bool, style MappingStyle) Event {
return Event{
Type: MAPPING_START_EVENT,
Anchor: anchor,
Tag: tag,
Implicit: implicit,
Style: Style(style),
}
}
// NewMappingEndEvent creates a new MAPPING-END event.
func NewMappingEndEvent() Event {
return Event{
Type: MAPPING_END_EVENT,
}
}
// Delete an event object.
func (e *Event) Delete() {
*e = Event{}
}
///*
// * Create a document object.
// */
//
//YAML_DECLARE(int)
//yaml_document_initialize(document *yaml_document_t,
// version_directive *yaml_version_directive_t,
// tag_directives_start *yaml_tag_directive_t,
// tag_directives_end *yaml_tag_directive_t,
// start_implicit int, end_implicit int)
//{
// struct {
// error yaml_error_type_t
// } context
// struct {
// start *yaml_node_t
// end *yaml_node_t
// top *yaml_node_t
// } nodes = { NULL, NULL, NULL }
// version_directive_copy *yaml_version_directive_t = NULL
// struct {
// start *yaml_tag_directive_t
// end *yaml_tag_directive_t
// top *yaml_tag_directive_t
// } tag_directives_copy = { NULL, NULL, NULL }
// value yaml_tag_directive_t = { NULL, NULL }
// mark yaml_mark_t = { 0, 0, 0 }
//
// assert(document) // Non-NULL document object is expected.
// assert((tag_directives_start && tag_directives_end) ||
// (tag_directives_start == tag_directives_end))
// // Valid tag directives are expected.
//
// if (!STACK_INIT(&context, nodes, INITIAL_STACK_SIZE)) goto error
//
// if (version_directive) {
// version_directive_copy = yaml_malloc(sizeof(yaml_version_directive_t))
// if (!version_directive_copy) goto error
// version_directive_copy.major = version_directive.major
// version_directive_copy.minor = version_directive.minor
// }
//
// if (tag_directives_start != tag_directives_end) {
// tag_directive *yaml_tag_directive_t
// if (!STACK_INIT(&context, tag_directives_copy, INITIAL_STACK_SIZE))
// goto error
// for (tag_directive = tag_directives_start
// tag_directive != tag_directives_end; tag_directive ++) {
// assert(tag_directive.handle)
// assert(tag_directive.prefix)
// if (!yaml_check_utf8(tag_directive.handle,
// strlen((char *)tag_directive.handle)))
// goto error
// if (!yaml_check_utf8(tag_directive.prefix,
// strlen((char *)tag_directive.prefix)))
// goto error
// value.handle = yaml_strdup(tag_directive.handle)
// value.prefix = yaml_strdup(tag_directive.prefix)
// if (!value.handle || !value.prefix) goto error
// if (!PUSH(&context, tag_directives_copy, value))
// goto error
// value.handle = NULL
// value.prefix = NULL
// }
// }
//
// DOCUMENT_INIT(*document, nodes.start, nodes.end, version_directive_copy,
// tag_directives_copy.start, tag_directives_copy.top,
// start_implicit, end_implicit, mark, mark)
//
// return 1
//
//error:
// STACK_DEL(&context, nodes)
// yaml_free(version_directive_copy)
// while (!STACK_EMPTY(&context, tag_directives_copy)) {
// value yaml_tag_directive_t = POP(&context, tag_directives_copy)
// yaml_free(value.handle)
// yaml_free(value.prefix)
// }
// STACK_DEL(&context, tag_directives_copy)
// yaml_free(value.handle)
// yaml_free(value.prefix)
//
// return 0
//}
//
///*
// * Destroy a document object.
// */
//
//YAML_DECLARE(void)
//yaml_document_delete(document *yaml_document_t)
//{
// struct {
// error yaml_error_type_t
// } context
// tag_directive *yaml_tag_directive_t
//
// context.error = YAML_NO_ERROR // Eliminate a compiler warning.
//
// assert(document) // Non-NULL document object is expected.
//
// while (!STACK_EMPTY(&context, document.nodes)) {
// node yaml_node_t = POP(&context, document.nodes)
// yaml_free(node.tag)
// switch (node.type) {
// case YAML_SCALAR_NODE:
// yaml_free(node.data.scalar.value)
// break
// case YAML_SEQUENCE_NODE:
// STACK_DEL(&context, node.data.sequence.items)
// break
// case YAML_MAPPING_NODE:
// STACK_DEL(&context, node.data.mapping.pairs)
// break
// default:
// assert(0) // Should not happen.
// }
// }
// STACK_DEL(&context, document.nodes)
//
// yaml_free(document.version_directive)
// for (tag_directive = document.tag_directives.start
// tag_directive != document.tag_directives.end
// tag_directive++) {
// yaml_free(tag_directive.handle)
// yaml_free(tag_directive.prefix)
// }
// yaml_free(document.tag_directives.start)
//
// memset(document, 0, sizeof(yaml_document_t))
//}
//
///**
// * Get a document node.
// */
//
//YAML_DECLARE(yaml_node_t *)
//yaml_document_get_node(document *yaml_document_t, index int)
//{
// assert(document) // Non-NULL document object is expected.
//
// if (index > 0 && document.nodes.start + index <= document.nodes.top) {
// return document.nodes.start + index - 1
// }
// return NULL
//}
//
///**
// * Get the root object.
// */
//
//YAML_DECLARE(yaml_node_t *)
//yaml_document_get_root_node(document *yaml_document_t)
//{
// assert(document) // Non-NULL document object is expected.
//
// if (document.nodes.top != document.nodes.start) {
// return document.nodes.start
// }
// return NULL
//}
//
///*
// * Add a scalar node to a document.
// */
//
//YAML_DECLARE(int)
//yaml_document_add_scalar(document *yaml_document_t,
// tag *yaml_char_t, value *yaml_char_t, length int,
// style yaml_scalar_style_t)
//{
// struct {
// error yaml_error_type_t
// } context
// mark yaml_mark_t = { 0, 0, 0 }
// tag_copy *yaml_char_t = NULL
// value_copy *yaml_char_t = NULL
// node yaml_node_t
//
// assert(document) // Non-NULL document object is expected.
// assert(value) // Non-NULL value is expected.
//
// if (!tag) {
// tag = (yaml_char_t *)YAML_DEFAULT_SCALAR_TAG
// }
//
// if (!yaml_check_utf8(tag, strlen((char *)tag))) goto error
// tag_copy = yaml_strdup(tag)
// if (!tag_copy) goto error
//
// if (length < 0) {
// length = strlen((char *)value)
// }
//
// if (!yaml_check_utf8(value, length)) goto error
// value_copy = yaml_malloc(length+1)
// if (!value_copy) goto error
// memcpy(value_copy, value, length)
// value_copy[length] = '\0'
//
// SCALAR_NODE_INIT(node, tag_copy, value_copy, length, style, mark, mark)
// if (!PUSH(&context, document.nodes, node)) goto error
//
// return document.nodes.top - document.nodes.start
//
//error:
// yaml_free(tag_copy)
// yaml_free(value_copy)
//
// return 0
//}
//
///*
// * Add a sequence node to a document.
// */
//
//YAML_DECLARE(int)
//yaml_document_add_sequence(document *yaml_document_t,
// tag *yaml_char_t, style yaml_sequence_style_t)
//{
// struct {
// error yaml_error_type_t
// } context
// mark yaml_mark_t = { 0, 0, 0 }
// tag_copy *yaml_char_t = NULL
// struct {
// start *yaml_node_item_t
// end *yaml_node_item_t
// top *yaml_node_item_t
// } items = { NULL, NULL, NULL }
// node yaml_node_t
//
// assert(document) // Non-NULL document object is expected.
//
// if (!tag) {
// tag = (yaml_char_t *)YAML_DEFAULT_SEQUENCE_TAG
// }
//
// if (!yaml_check_utf8(tag, strlen((char *)tag))) goto error
// tag_copy = yaml_strdup(tag)
// if (!tag_copy) goto error
//
// if (!STACK_INIT(&context, items, INITIAL_STACK_SIZE)) goto error
//
// SEQUENCE_NODE_INIT(node, tag_copy, items.start, items.end,
// style, mark, mark)
// if (!PUSH(&context, document.nodes, node)) goto error
//
// return document.nodes.top - document.nodes.start
//
//error:
// STACK_DEL(&context, items)
// yaml_free(tag_copy)
//
// return 0
//}
//
///*
// * Add a mapping node to a document.
// */
//
//YAML_DECLARE(int)
//yaml_document_add_mapping(document *yaml_document_t,
// tag *yaml_char_t, style yaml_mapping_style_t)
//{
// struct {
// error yaml_error_type_t
// } context
// mark yaml_mark_t = { 0, 0, 0 }
// tag_copy *yaml_char_t = NULL
// struct {
// start *yaml_node_pair_t
// end *yaml_node_pair_t
// top *yaml_node_pair_t
// } pairs = { NULL, NULL, NULL }
// node yaml_node_t
//
// assert(document) // Non-NULL document object is expected.
//
// if (!tag) {
// tag = (yaml_char_t *)YAML_DEFAULT_MAPPING_TAG
// }
//
// if (!yaml_check_utf8(tag, strlen((char *)tag))) goto error
// tag_copy = yaml_strdup(tag)
// if (!tag_copy) goto error
//
// if (!STACK_INIT(&context, pairs, INITIAL_STACK_SIZE)) goto error
//
// MAPPING_NODE_INIT(node, tag_copy, pairs.start, pairs.end,
// style, mark, mark)
// if (!PUSH(&context, document.nodes, node)) goto error
//
// return document.nodes.top - document.nodes.start
//
//error:
// STACK_DEL(&context, pairs)
// yaml_free(tag_copy)
//
// return 0
//}
//
///*
// * Append an item to a sequence node.
// */
//
//YAML_DECLARE(int)
//yaml_document_append_sequence_item(document *yaml_document_t,
// sequence int, item int)
//{
// struct {
// error yaml_error_type_t
// } context
//
// assert(document) // Non-NULL document is required.
// assert(sequence > 0
// && document.nodes.start + sequence <= document.nodes.top)
// // Valid sequence id is required.
// assert(document.nodes.start[sequence-1].type == YAML_SEQUENCE_NODE)
// // A sequence node is required.
// assert(item > 0 && document.nodes.start + item <= document.nodes.top)
// // Valid item id is required.
//
// if (!PUSH(&context,
// document.nodes.start[sequence-1].data.sequence.items, item))
// return 0
//
// return 1
//}
//
///*
// * Append a pair of a key and a value to a mapping node.
// */
//
//YAML_DECLARE(int)
//yaml_document_append_mapping_pair(document *yaml_document_t,
// mapping int, key int, value int)
//{
// struct {
// error yaml_error_type_t
// } context
//
// pair yaml_node_pair_t
//
// assert(document) // Non-NULL document is required.
// assert(mapping > 0
// && document.nodes.start + mapping <= document.nodes.top)
// // Valid mapping id is required.
// assert(document.nodes.start[mapping-1].type == YAML_MAPPING_NODE)
// // A mapping node is required.
// assert(key > 0 && document.nodes.start + key <= document.nodes.top)
// // Valid key id is required.
// assert(value > 0 && document.nodes.start + value <= document.nodes.top)
// // Valid value id is required.
//
// pair.key = key
// pair.value = value
//
// if (!PUSH(&context,
// document.nodes.start[mapping-1].data.mapping.pairs, pair))
// return 0
//
// return 1
//}
//
//
+362
View File
@@ -0,0 +1,362 @@
// Copyright 2011-2019 Canonical Ltd
// Copyright 2025 The go-yaml Project Contributors
// SPDX-License-Identifier: Apache-2.0
// Composer stage: Builds a node tree from a libyaml event stream.
// Handles document structure, anchors, and comment attachment.
package libyaml
import (
"fmt"
"io"
)
// Composer produces a node tree out of a libyaml event stream.
type Composer struct {
Parser Parser
event Event
doc *Node
anchors map[string]*Node
doneInit bool
Textless bool
streamNodes bool // enable stream node emission
returnStream bool // flag to return stream node next
atStreamEnd bool // at stream end
encoding Encoding // stream encoding from STREAM_START
}
// NewComposer creates a new composer from a byte slice.
func NewComposer(b []byte) *Composer {
p := Composer{
Parser: NewParser(),
}
if len(b) == 0 {
b = []byte{'\n'}
}
p.Parser.SetInputString(b)
return &p
}
// NewComposerFromReader creates a new composer from an io.Reader.
func NewComposerFromReader(r io.Reader) *Composer {
p := Composer{
Parser: NewParser(),
}
p.Parser.SetInputReader(r)
return &p
}
func (c *Composer) init() {
if c.doneInit {
return
}
c.anchors = make(map[string]*Node)
// Peek to get the encoding from STREAM_START_EVENT
if c.peek() == STREAM_START_EVENT {
c.encoding = c.event.GetEncoding()
}
c.expect(STREAM_START_EVENT)
c.doneInit = true
// If stream nodes are enabled, prepare to return the first stream node
if c.streamNodes {
c.returnStream = true
}
}
func (c *Composer) Destroy() {
if c.event.Type != NO_EVENT {
c.event.Delete()
}
c.Parser.Delete()
}
// SetStreamNodes enables or disables stream node emission.
func (c *Composer) SetStreamNodes(enable bool) {
c.streamNodes = enable
}
// expect consumes an event from the event stream and
// checks that it's of the expected type.
func (c *Composer) expect(e EventType) {
if c.event.Type == NO_EVENT {
if err := c.Parser.Parse(&c.event); err != nil {
c.fail(err)
}
}
if c.event.Type == STREAM_END_EVENT {
failf("attempted to go past the end of stream; corrupted value?")
}
if c.event.Type != e {
c.fail(fmt.Errorf("expected %s event but got %s", e, c.event.Type))
}
c.event.Delete()
c.event.Type = NO_EVENT
}
// peek peeks at the next event in the event stream,
// puts the results into c.event and returns the event type.
func (c *Composer) peek() EventType {
if c.event.Type != NO_EVENT {
return c.event.Type
}
// It's curious choice from the underlying API to generally return a
// positive result on success, but on this case return true in an error
// scenario. This was the source of bugs in the past (issue #666).
if err := c.Parser.Parse(&c.event); err != nil {
c.fail(err)
}
return c.event.Type
}
func (c *Composer) fail(err error) {
Fail(err)
}
func (c *Composer) anchor(n *Node, anchor []byte) {
if anchor != nil {
n.Anchor = string(anchor)
c.anchors[n.Anchor] = n
}
}
// Parse parses the next YAML node from the event stream.
func (c *Composer) Parse() *Node {
c.init()
// Handle stream nodes if enabled
if c.streamNodes {
// Check for stream end first
if c.peek() == STREAM_END_EVENT {
// If we haven't returned the final stream node yet, return it now
if !c.atStreamEnd {
c.atStreamEnd = true
return c.createStreamNode()
}
// Already returned final stream node
return nil
}
// Check if we should return a stream node before the next document
if c.returnStream {
c.returnStream = false
n := c.createStreamNode()
// Capture directives from upcoming document
c.captureDirectives(n)
return n
}
}
switch c.peek() {
case SCALAR_EVENT:
return c.scalar()
case ALIAS_EVENT:
return c.alias()
case MAPPING_START_EVENT:
return c.mapping()
case SEQUENCE_START_EVENT:
return c.sequence()
case DOCUMENT_START_EVENT:
return c.document()
case STREAM_END_EVENT:
// Happens when attempting to decode an empty buffer (when not using stream nodes).
return nil
case TAIL_COMMENT_EVENT:
panic("internal error: unexpected tail comment event (please report)")
default:
panic("internal error: attempted to parse unknown event (please report): " + c.event.Type.String())
}
}
func (c *Composer) node(kind Kind, defaultTag, tag, value string) *Node {
var style Style
if tag != "" && tag != "!" {
// Normalize tag to short form (e.g., tag:yaml.org,2002:str -> !!str)
tag = shortTag(tag)
style = TaggedStyle
} else if defaultTag != "" {
tag = defaultTag
} else if kind == ScalarNode {
// Delegate to resolver to determine tag from value
tag, _ = resolve("", value)
}
n := &Node{
Kind: kind,
Tag: tag,
Value: value,
Style: style,
}
if !c.Textless {
n.Line = c.event.StartMark.Line + 1
n.Column = c.event.StartMark.Column + 1
n.HeadComment = string(c.event.HeadComment)
n.LineComment = string(c.event.LineComment)
n.FootComment = string(c.event.FootComment)
}
return n
}
func (c *Composer) parseChild(parent *Node) *Node {
child := c.Parse()
parent.Content = append(parent.Content, child)
return child
}
func (c *Composer) document() *Node {
n := c.node(DocumentNode, "", "", "")
c.doc = n
c.expect(DOCUMENT_START_EVENT)
c.parseChild(n)
if c.peek() == DOCUMENT_END_EVENT {
n.FootComment = string(c.event.FootComment)
}
c.expect(DOCUMENT_END_EVENT)
// If stream nodes enabled, prepare to return a stream node next
if c.streamNodes {
c.returnStream = true
}
return n
}
func (c *Composer) createStreamNode() *Node {
n := &Node{
Kind: StreamNode,
Encoding: c.encoding,
}
if !c.Textless && c.event.Type != NO_EVENT {
n.Line = c.event.StartMark.Line + 1
n.Column = c.event.StartMark.Column + 1
}
return n
}
// captureDirectives captures version and tag directives from upcoming DOCUMENT_START.
func (c *Composer) captureDirectives(n *Node) {
if c.peek() == DOCUMENT_START_EVENT {
if vd := c.event.GetVersionDirective(); vd != nil {
n.Version = &StreamVersionDirective{
Major: vd.Major(),
Minor: vd.Minor(),
}
}
if tds := c.event.GetTagDirectives(); len(tds) > 0 {
n.TagDirectives = make([]StreamTagDirective, len(tds))
for i, td := range tds {
n.TagDirectives[i] = StreamTagDirective{
Handle: td.GetHandle(),
Prefix: td.GetPrefix(),
}
}
}
}
}
func (c *Composer) alias() *Node {
n := c.node(AliasNode, "", "", string(c.event.Anchor))
n.Alias = c.anchors[n.Value]
if n.Alias == nil {
msg := fmt.Sprintf("unknown anchor '%s' referenced", n.Value)
Fail(&ParserError{
Message: msg,
Mark: Mark{
Line: n.Line,
Column: n.Column,
},
})
}
c.expect(ALIAS_EVENT)
return n
}
func (c *Composer) scalar() *Node {
parsedStyle := c.event.ScalarStyle()
var nodeStyle Style
switch {
case parsedStyle&DOUBLE_QUOTED_SCALAR_STYLE != 0:
nodeStyle = DoubleQuotedStyle
case parsedStyle&SINGLE_QUOTED_SCALAR_STYLE != 0:
nodeStyle = SingleQuotedStyle
case parsedStyle&LITERAL_SCALAR_STYLE != 0:
nodeStyle = LiteralStyle
case parsedStyle&FOLDED_SCALAR_STYLE != 0:
nodeStyle = FoldedStyle
}
nodeValue := string(c.event.Value)
nodeTag := string(c.event.Tag)
var defaultTag string
if nodeStyle != 0 {
defaultTag = strTag
}
n := c.node(ScalarNode, defaultTag, nodeTag, nodeValue)
n.Style |= nodeStyle
c.anchor(n, c.event.Anchor)
c.expect(SCALAR_EVENT)
return n
}
func (c *Composer) sequence() *Node {
n := c.node(SequenceNode, seqTag, string(c.event.Tag), "")
if c.event.SequenceStyle()&FLOW_SEQUENCE_STYLE != 0 {
n.Style |= FlowStyle
}
c.anchor(n, c.event.Anchor)
c.expect(SEQUENCE_START_EVENT)
for c.peek() != SEQUENCE_END_EVENT {
c.parseChild(n)
}
n.LineComment = string(c.event.LineComment)
n.FootComment = string(c.event.FootComment)
c.expect(SEQUENCE_END_EVENT)
return n
}
func (c *Composer) mapping() *Node {
n := c.node(MappingNode, mapTag, string(c.event.Tag), "")
block := true
if c.event.MappingStyle()&FLOW_MAPPING_STYLE != 0 {
block = false
n.Style |= FlowStyle
}
c.anchor(n, c.event.Anchor)
c.expect(MAPPING_START_EVENT)
for c.peek() != MAPPING_END_EVENT {
k := c.parseChild(n)
if block && k.FootComment != "" {
// Must be a foot comment for the prior value when being dedented.
if len(n.Content) > 2 {
n.Content[len(n.Content)-3].FootComment = k.FootComment
k.FootComment = ""
}
}
v := c.parseChild(n)
if k.FootComment == "" && v.FootComment != "" {
k.FootComment = v.FootComment
v.FootComment = ""
}
if c.peek() == TAIL_COMMENT_EVENT {
if k.FootComment == "" {
k.FootComment = string(c.event.FootComment)
}
c.expect(TAIL_COMMENT_EVENT)
}
}
n.LineComment = string(c.event.LineComment)
n.FootComment = string(c.event.FootComment)
if n.Style&FlowStyle == 0 && n.FootComment != "" && len(n.Content) > 1 {
n.Content[len(n.Content)-2].FootComment = n.FootComment
n.FootComment = ""
}
c.expect(MAPPING_END_EVENT)
return n
}
func Fail(err error) {
panic(&YAMLError{err})
}
func failf(format string, args ...any) {
panic(&YAMLError{fmt.Errorf("yaml: "+format, args...)})
}
File diff suppressed because it is too large Load Diff
+8
View File
@@ -0,0 +1,8 @@
// Copyright 2025 The go-yaml Project Contributors
// SPDX-License-Identifier: Apache-2.0
// Package libyaml contains internal helpers for working with YAML
//
// It's a reworked version of the original libyaml package from go-yaml v2/v3,
// adapted to work with Go specifications
package libyaml
File diff suppressed because it is too large Load Diff
+171
View File
@@ -0,0 +1,171 @@
// Copyright 2025 The go-yaml Project Contributors
// SPDX-License-Identifier: Apache-2.0
// Error types for YAML parsing and emitting.
// Provides structured error reporting with line/column information.
package libyaml
import (
"errors"
"fmt"
"strings"
)
type MarkedYAMLError struct {
// optional context
ContextMark Mark
ContextMessage string
Mark Mark
Message string
}
func (e MarkedYAMLError) Error() string {
var builder strings.Builder
builder.WriteString("yaml: ")
if len(e.ContextMessage) > 0 {
fmt.Fprintf(&builder, "%s at %s: ", e.ContextMessage, e.ContextMark)
}
if len(e.ContextMessage) == 0 || e.ContextMark != e.Mark {
fmt.Fprintf(&builder, "%s: ", e.Mark)
}
builder.WriteString(e.Message)
return builder.String()
}
type ParserError MarkedYAMLError
func (e ParserError) Error() string {
return MarkedYAMLError(e).Error()
}
type ScannerError MarkedYAMLError
func (e ScannerError) Error() string {
return MarkedYAMLError(e).Error()
}
type ReaderError struct {
Offset int
Value int
Err error
}
func (e ReaderError) Error() string {
return fmt.Sprintf("yaml: offset %d: %s", e.Offset, e.Err)
}
func (e ReaderError) Unwrap() error {
return e.Err
}
type EmitterError struct {
Message string
}
func (e EmitterError) Error() string {
return fmt.Sprintf("yaml: %s", e.Message)
}
type WriterError struct {
Err error
}
func (e WriterError) Error() string {
return fmt.Sprintf("yaml: %s", e.Err)
}
func (e WriterError) Unwrap() error {
return e.Err
}
// ConstructError represents a single, non-fatal error that occurred during
// the constructing of a YAML document into a Go value.
type ConstructError struct {
Err error
Line int
Column int
}
func (e *ConstructError) Error() string {
return fmt.Sprintf("line %d: %s", e.Line, e.Err.Error())
}
func (e *ConstructError) Unwrap() error {
return e.Err
}
// LoadErrors is returned when one or more fields cannot be properly decoded.
type LoadErrors struct {
Errors []*ConstructError
}
func (e *LoadErrors) Error() string {
var b strings.Builder
b.WriteString("yaml: construct errors:")
for _, err := range e.Errors {
b.WriteString("\n ")
b.WriteString(err.Error())
}
return b.String()
}
// As implements errors.As for Go versions prior to 1.20 that don't support
// the Unwrap() []error interface. It allows [LoadErrors] to match against
// *ConstructError targets by returning the first error in the list.
func (e *LoadErrors) As(target any) bool {
switch t := target.(type) {
case **ConstructError:
if len(e.Errors) == 0 {
return false
}
*t = e.Errors[0]
return true
case **TypeError:
var msgs []string
for _, err := range e.Errors {
msgs = append(msgs, err.Error())
}
*t = &TypeError{Errors: msgs}
return true
}
return false
}
// Is implements errors.Is for Go versions prior to 1.20 that don't support
// the Unwrap() []error interface. It checks if any wrapped error matches
// the target error.
func (e *LoadErrors) Is(target error) bool {
for _, err := range e.Errors {
if errors.Is(err, target) {
return true
}
}
return false
}
// TypeError is an obsolete error type retained for compatibility.
//
// A TypeError is returned by Unmarshal when one or more fields in
// the YAML document cannot be properly decoded into the requested
// types. When this error is returned, the value is still
// unmarshaled partially.
//
// Deprecated: Use [LoadErrors] instead.
type TypeError struct {
Errors []string
}
func (e *TypeError) Error() string {
return fmt.Sprintf("yaml: unmarshal errors:\n %s", strings.Join(e.Errors, "\n "))
}
// YAMLError is an internal error wrapper type.
type YAMLError struct {
Err error
}
func (e *YAMLError) Error() string {
return e.Err.Error()
}
+363
View File
@@ -0,0 +1,363 @@
// Copyright 2011-2019 Canonical Ltd
// Copyright 2025 The go-yaml Project Contributors
// SPDX-License-Identifier: Apache-2.0
// Node types and constants for YAML tree representation.
// Defines Kind, Style, and Node structure for intermediate YAML representation.
package libyaml
import (
"reflect"
"strings"
"unicode"
"unicode/utf8"
)
// Tag constants for YAML types
const (
nullTag = "!!null"
boolTag = "!!bool"
strTag = "!!str"
intTag = "!!int"
floatTag = "!!float"
timestampTag = "!!timestamp"
seqTag = "!!seq"
mapTag = "!!map"
binaryTag = "!!binary"
mergeTag = "!!merge"
)
const longTagPrefix = "tag:yaml.org,2002:"
var (
longTags = make(map[string]string)
shortTags = make(map[string]string)
)
func init() {
for _, stag := range []string{nullTag, boolTag, strTag, intTag, floatTag, timestampTag, seqTag, mapTag, binaryTag, mergeTag} {
ltag := longTag(stag)
longTags[stag] = ltag
shortTags[ltag] = stag
}
}
func shortTag(tag string) string {
if strings.HasPrefix(tag, longTagPrefix) {
if stag, ok := shortTags[tag]; ok {
return stag
}
return "!!" + tag[len(longTagPrefix):]
}
return tag
}
func longTag(tag string) string {
if strings.HasPrefix(tag, "!!") {
if ltag, ok := longTags[tag]; ok {
return ltag
}
return longTagPrefix + tag[2:]
}
return tag
}
// Kind represents the type of YAML node
type Kind uint32
const (
DocumentNode Kind = 1 << iota
SequenceNode
MappingNode
ScalarNode
AliasNode
StreamNode
)
// Style represents the formatting style of a YAML node
type Style uint32
const (
TaggedStyle Style = 1 << iota
DoubleQuotedStyle
SingleQuotedStyle
LiteralStyle
FoldedStyle
FlowStyle
)
// StreamVersionDirective represents a YAML %YAML version directive for stream nodes.
type StreamVersionDirective struct {
Major int
Minor int
}
// StreamTagDirective represents a YAML %TAG directive for stream nodes.
type StreamTagDirective struct {
Handle string
Prefix string
}
// Node represents an element in the YAML document hierarchy. While documents
// are typically encoded and decoded into higher level types, such as structs
// and maps, Node is an intermediate representation that allows detailed
// control over the content being decoded or encoded.
//
// It's worth noting that although Node offers access into details such as
// line numbers, columns, and comments, the content when re-encoded will not
// have its original textual representation preserved. An effort is made to
// render the data pleasantly, and to preserve comments near the data they
// describe, though.
//
// Values that make use of the Node type interact with the yaml package in the
// same way any other type would do, by encoding and decoding yaml data
// directly or indirectly into them.
//
// For example:
//
// var person struct {
// Name string
// Address yaml.Node
// }
// err := yaml.Unmarshal(data, &person)
//
// Or by itself:
//
// var person Node
// err := yaml.Unmarshal(data, &person)
type Node struct {
// Kind defines whether the node is a document, a mapping, a sequence,
// a scalar value, or an alias to another node. The specific data type of
// scalar nodes may be obtained via the ShortTag and LongTag methods.
Kind Kind
// Style allows customizing the appearance of the node in the tree.
Style Style
// Tag holds the YAML tag defining the data type for the value.
// When decoding, this field will always be set to the resolved tag,
// even when it wasn't explicitly provided in the YAML content.
// When encoding, if this field is unset the value type will be
// implied from the node properties, and if it is set, it will only
// be serialized into the representation if TaggedStyle is used or
// the implicit tag diverges from the provided one.
Tag string
// Value holds the unescaped and unquoted representation of the value.
Value string
// Anchor holds the anchor name for this node, which allows aliases to point to it.
Anchor string
// Alias holds the node that this alias points to. Only valid when Kind is AliasNode.
Alias *Node
// Content holds contained nodes for documents, mappings, and sequences.
Content []*Node
// HeadComment holds any comments in the lines preceding the node and
// not separated by an empty line.
HeadComment string
// LineComment holds any comments at the end of the line where the node is in.
LineComment string
// FootComment holds any comments following the node and before empty lines.
FootComment string
// Line and Column hold the node position in the decoded YAML text.
// These fields are not respected when encoding the node.
Line int
Column int
// StreamNode-specific fields (only valid when Kind == StreamNode)
// Encoding holds the stream encoding (UTF-8, UTF-16LE, UTF-16BE).
// Only valid for StreamNode.
Encoding Encoding
// Version holds the YAML version directive (%YAML).
// Only valid for StreamNode.
Version *StreamVersionDirective
// TagDirectives holds the %TAG directives.
// Only valid for StreamNode.
TagDirectives []StreamTagDirective
}
// IsZero returns whether the node has all of its fields unset.
func (n *Node) IsZero() bool {
return n.Kind == 0 && n.Style == 0 && n.Tag == "" && n.Value == "" && n.Anchor == "" && n.Alias == nil && n.Content == nil &&
n.HeadComment == "" && n.LineComment == "" && n.FootComment == "" && n.Line == 0 && n.Column == 0 &&
n.Encoding == 0 && n.Version == nil && n.TagDirectives == nil
}
// LongTag returns the long form of the tag that indicates the data type for
// the node. If the Tag field isn't explicitly defined, one will be computed
// based on the node properties.
func (n *Node) LongTag() string {
return longTag(n.ShortTag())
}
// ShortTag returns the short form of the YAML tag that indicates data type for
// the node. If the Tag field isn't explicitly defined, one will be computed
// based on the node properties.
func (n *Node) ShortTag() string {
if n.indicatedString() {
return strTag
}
if n.Tag == "" || n.Tag == "!" {
switch n.Kind {
case MappingNode:
return mapTag
case SequenceNode:
return seqTag
case AliasNode:
if n.Alias != nil {
return n.Alias.ShortTag()
}
case ScalarNode:
return strTag
case 0:
// Special case to make the zero value convenient.
if n.IsZero() {
return nullTag
}
}
return ""
}
return shortTag(n.Tag)
}
func (n *Node) indicatedString() bool {
return n.Kind == ScalarNode &&
(shortTag(n.Tag) == strTag ||
(n.Tag == "" || n.Tag == "!") && n.Style&(SingleQuotedStyle|DoubleQuotedStyle|LiteralStyle|FoldedStyle) != 0)
}
// shouldUseLiteralStyle determines if a string should use literal style.
// It returns true if the string contains newlines AND meets additional criteria:
// - is at least 2 characters long
// - contains at least one non-whitespace character
func shouldUseLiteralStyle(s string) bool {
if !strings.Contains(s, "\n") || len(s) < 2 {
return false
}
// Must contain at least one non-whitespace character
for _, r := range s {
if !unicode.IsSpace(r) {
return true
}
}
return false
}
// SetString is a convenience function that sets the node to a string value
// and defines its style in a pleasant way depending on its content.
func (n *Node) SetString(s string) {
n.Kind = ScalarNode
if utf8.ValidString(s) {
n.Value = s
n.Tag = strTag
} else {
n.Value = encodeBase64(s)
n.Tag = binaryTag
}
if shouldUseLiteralStyle(n.Value) {
n.Style = LiteralStyle
}
}
// Decode decodes the node and stores its data into the value pointed to by v.
//
// See the documentation for Unmarshal for details about the
// conversion of YAML into a Go value.
func (n *Node) Decode(v any) (err error) {
d := NewConstructor(DefaultOptions)
defer handleErr(&err)
out := reflect.ValueOf(v)
if out.Kind() == reflect.Pointer && !out.IsNil() {
out = out.Elem()
}
d.Construct(n, out)
if len(d.TypeErrors) > 0 {
return &LoadErrors{Errors: d.TypeErrors}
}
return nil
}
// Load decodes the node and stores its data into the value pointed to by v,
// applying the given options.
//
// This method is useful when you need to preserve options like WithKnownFields()
// inside custom UnmarshalYAML implementations.
//
// Maps and pointers (to a struct, string, int, etc) are accepted as v
// values. If an internal pointer within a struct is not initialized,
// the yaml package will initialize it if necessary. The v parameter
// must not be nil.
//
// See the documentation of the package-level Load function for details
// about YAML to Go conversion and tag options.
func (n *Node) Load(v any, opts ...Option) (err error) {
defer handleErr(&err)
o, err := ApplyOptions(opts...)
if err != nil {
return err
}
d := NewConstructor(o)
out := reflect.ValueOf(v)
if out.Kind() == reflect.Pointer && !out.IsNil() {
out = out.Elem()
}
d.Construct(n, out)
if len(d.TypeErrors) > 0 {
return &LoadErrors{Errors: d.TypeErrors}
}
return nil
}
// Encode encodes value v and stores its representation in n.
//
// See the documentation for Marshal for details about the
// conversion of Go values into YAML.
func (n *Node) Encode(v any) (err error) {
defer handleErr(&err)
e := NewRepresenter(noWriter, DefaultOptions)
defer e.Destroy()
e.MarshalDoc("", reflect.ValueOf(v))
e.Finish()
p := NewComposer(e.Out)
p.Textless = true
defer p.Destroy()
doc := p.Parse()
*n = *doc.Content[0]
return nil
}
// Dump encodes value v and stores its representation in n,
// applying the given options.
//
// This method is useful when you need to apply specific encoding options
// while building Node trees programmatically.
//
// See the documentation for Marshal for details about the
// conversion of Go values into YAML.
func (n *Node) Dump(v any, opts ...Option) (err error) {
defer handleErr(&err)
o, err := ApplyOptions(opts...)
if err != nil {
return err
}
e := NewRepresenter(noWriter, o)
defer e.Destroy()
e.MarshalDoc("", reflect.ValueOf(v))
e.Finish()
p := NewComposer(e.Out)
p.Textless = true
defer p.Destroy()
doc := p.Parse()
*n = *doc.Content[0]
return nil
}
+390
View File
@@ -0,0 +1,390 @@
//
// Copyright (c) 2025 The go-yaml Project Contributors
// SPDX-License-Identifier: Apache-2.0
//
// Options configuration for loading and dumping YAML.
// Provides centralized control for indentation, line width, strictness, and
// more.
package libyaml
import (
"errors"
"fmt"
)
// Options holds configuration for both loading and dumping YAML.
type Options struct {
// Loading options
KnownFields bool // Enforce known fields in structs
SingleDocument bool // Only load first document
UniqueKeys bool // Enforce unique keys in mappings
StreamNodes bool // Enable stream node emission
AllDocuments bool // Load/Dump all documents in multi-document streams
// Dumping options
Indent int // Indentation spaces (2-9)
CompactSeqIndent bool // Whether '- ' counts as indentation
LineWidth int // Preferred line width (-1 for unlimited)
Unicode bool // Allow non-ASCII characters
Canonical bool // Canonical YAML output
LineBreak LineBreak // Line ending style
ExplicitStart bool // Always emit ---
ExplicitEnd bool // Always emit ...
FlowSimpleCollections bool // Use flow style for simple collections
QuotePreference QuoteStyle // Preferred quote style when quoting is required
}
// Option allows configuring YAML loading and dumping operations.
type Option func(*Options) error
// WithIndent sets the number of spaces to use for indentation when
// dumping YAML content.
//
// Valid values are 2-9. Common choices: 2 (compact), 4 (readable).
func WithIndent(indent int) Option {
return func(o *Options) error {
if indent < 2 || indent > 9 {
return errors.New("yaml: indent must be between 2 and 9 spaces")
}
o.Indent = indent
return nil
}
}
// WithCompactSeqIndent configures whether the sequence indicator '- ' is
// considered part of the indentation when dumping YAML content.
//
// If compact is true, '- ' is treated as part of the indentation.
// If compact is false, '- ' is not treated as part of the indentation.
// When called without arguments, defaults to true.
func WithCompactSeqIndent(compact ...bool) Option {
if len(compact) > 1 {
return func(o *Options) error {
return errors.New("yaml: WithCompactSeqIndent accepts at most one argument")
}
}
val := len(compact) == 0 || compact[0]
return func(o *Options) error {
o.CompactSeqIndent = val
return nil
}
}
// WithKnownFields enables or disables strict field checking during YAML loading.
//
// When enabled, loading will return an error if the YAML input contains fields
// that do not correspond to any fields in the target struct.
// When called without arguments, defaults to true.
func WithKnownFields(knownFields ...bool) Option {
if len(knownFields) > 1 {
return func(o *Options) error {
return errors.New("yaml: WithKnownFields accepts at most one argument")
}
}
val := len(knownFields) == 0 || knownFields[0]
return func(o *Options) error {
o.KnownFields = val
return nil
}
}
// WithSingleDocument configures the Loader to only process the first document
// in a YAML stream. After the first document is loaded, subsequent calls to
// Load will return io.EOF.
//
// When called without arguments, defaults to true.
//
// This is useful when you expect exactly one document and want behavior
// similar to [Unmarshal].
func WithSingleDocument(singleDocument ...bool) Option {
if len(singleDocument) > 1 {
return func(o *Options) error {
return errors.New("yaml: WithSingleDocument accepts at most one argument")
}
}
val := len(singleDocument) == 0 || singleDocument[0]
return func(o *Options) error {
o.SingleDocument = val
return nil
}
}
// WithStreamNodes enables returning stream boundary nodes when loading YAML.
//
// When enabled, Loader.Load returns an interleaved sequence of StreamNode and
// DocumentNode values:
//
// [StreamNode, DocNode, StreamNode, DocNode, ..., StreamNode]
//
// StreamNodes contain metadata about the stream including:
// - Encoding (UTF-8, UTF-16LE, UTF-16BE)
// - YAML version directive (%YAML)
// - Tag directives (%TAG)
// - Position information (Line, Column)
//
// An empty YAML stream returns a single StreamNode.
// When called without arguments, defaults to true.
//
// The default is false.
func WithStreamNodes(enable ...bool) Option {
if len(enable) > 1 {
return func(o *Options) error {
return errors.New("yaml: WithStreamNodes accepts at most one argument")
}
}
val := len(enable) == 0 || enable[0]
return func(o *Options) error {
o.StreamNodes = val
return nil
}
}
// WithAllDocuments enables multi-document mode for Load and Dump operations.
//
// When used with Load, the target must be a pointer to a slice.
// All documents in the YAML stream will be decoded into the slice.
// Zero documents results in an empty slice (no error).
//
// When used with Dump, the input must be a slice.
// Each element will be encoded as a separate YAML document
// with "---" separators.
//
// When called without arguments, defaults to true.
//
// The default is false (single-document mode).
func WithAllDocuments(all ...bool) Option {
if len(all) > 1 {
return func(o *Options) error {
return errors.New("yaml: WithAllDocuments accepts at most one argument")
}
}
val := len(all) == 0 || all[0]
return func(o *Options) error {
o.AllDocuments = val
return nil
}
}
// WithLineWidth sets the preferred line width for YAML output.
//
// When encoding long strings, the encoder will attempt to wrap them at this
// width using literal block style (|). Set to -1 or 0 for unlimited width.
//
// The default is 80 characters.
func WithLineWidth(width int) Option {
return func(o *Options) error {
if width < 0 {
width = -1
}
o.LineWidth = width
return nil
}
}
// WithUnicode controls whether non-ASCII characters are allowed in YAML output.
//
// When true, non-ASCII characters appear as-is (e.g., "café").
// When false, non-ASCII characters are escaped (e.g., "caf\u00e9").
// When called without arguments, defaults to true.
//
// The default is true.
func WithUnicode(unicode ...bool) Option {
if len(unicode) > 1 {
return func(o *Options) error {
return errors.New("yaml: WithUnicode accepts at most one argument")
}
}
val := len(unicode) == 0 || unicode[0]
return func(o *Options) error {
o.Unicode = val
return nil
}
}
// WithUniqueKeys enables or disables duplicate key detection during YAML loading.
//
// When enabled, loading will return an error if the YAML input contains
// duplicate keys in any mapping. This is a security feature that prevents
// key override attacks.
// When called without arguments, defaults to true.
//
// The default is true.
func WithUniqueKeys(uniqueKeys ...bool) Option {
if len(uniqueKeys) > 1 {
return func(o *Options) error {
return errors.New("yaml: WithUniqueKeys accepts at most one argument")
}
}
val := len(uniqueKeys) == 0 || uniqueKeys[0]
return func(o *Options) error {
o.UniqueKeys = val
return nil
}
}
// WithCanonical forces canonical YAML output format.
//
// When enabled, the encoder outputs strictly canonical YAML with explicit
// tags for all values. This produces verbose output primarily useful for
// debugging and YAML spec compliance testing.
// When called without arguments, defaults to true.
//
// The default is false.
func WithCanonical(canonical ...bool) Option {
if len(canonical) > 1 {
return func(o *Options) error {
return errors.New("yaml: WithCanonical accepts at most one argument")
}
}
val := len(canonical) == 0 || canonical[0]
return func(o *Options) error {
o.Canonical = val
return nil
}
}
// WithLineBreak sets the line ending style for YAML output.
//
// Available options:
// - LineBreakLN: Unix-style \n (default)
// - LineBreakCR: Old Mac-style \r
// - LineBreakCRLN: Windows-style \r\n
//
// The default is LineBreakLN.
func WithLineBreak(lineBreak LineBreak) Option {
return func(o *Options) error {
o.LineBreak = lineBreak
return nil
}
}
// WithExplicitStart controls whether document start markers (---) are always emitted.
//
// When true, every document begins with an explicit "---" marker.
// When false (default), the marker is omitted for the first document.
// When called without arguments, defaults to true.
func WithExplicitStart(explicit ...bool) Option {
if len(explicit) > 1 {
return func(o *Options) error {
return errors.New("yaml: WithExplicitStart accepts at most one argument")
}
}
val := len(explicit) == 0 || explicit[0]
return func(o *Options) error {
o.ExplicitStart = val
return nil
}
}
// WithExplicitEnd controls whether document end markers (...) are always emitted.
//
// When true, every document ends with an explicit "..." marker.
// When false (default), the marker is omitted.
// When called without arguments, defaults to true.
func WithExplicitEnd(explicit ...bool) Option {
if len(explicit) > 1 {
return func(o *Options) error {
return errors.New("yaml: WithExplicitEnd accepts at most one argument")
}
}
val := len(explicit) == 0 || explicit[0]
return func(o *Options) error {
o.ExplicitEnd = val
return nil
}
}
// WithFlowSimpleCollections controls whether simple collections use flow style.
//
// When true, sequences and mappings containing only scalar values (no nested
// collections) are rendered in flow style if they fit within the line width.
// Example: {name: test, count: 42} or [a, b, c]
// When called without arguments, defaults to true.
//
// When false (default), all collections use block style.
func WithFlowSimpleCollections(flow ...bool) Option {
if len(flow) > 1 {
return func(o *Options) error {
return errors.New("yaml: WithFlowSimpleCollections accepts at most one argument")
}
}
val := len(flow) == 0 || flow[0]
return func(o *Options) error {
o.FlowSimpleCollections = val
return nil
}
}
// WithQuotePreference sets the preferred quote style for strings that require
// quoting.
//
// This option only affects strings that require quoting per the YAML spec.
// Plain strings that don't need quoting remain unquoted regardless of this
// setting. Quoting is required for:
// - Strings that look like other YAML types (true, false, null, 123, etc.)
// - Strings with leading/trailing whitespace
// - Strings containing special YAML syntax characters
// - Empty strings in certain contexts
//
// Quote styles:
// - QuoteSingle: Use single quotes (v4 default)
// - QuoteDouble: Use double quotes
// - QuoteLegacy: Legacy v2/v3 behavior (mixed quoting)
func WithQuotePreference(style QuoteStyle) Option {
return func(o *Options) error {
switch style {
case QuoteSingle, QuoteDouble, QuoteLegacy:
o.QuotePreference = style
return nil
default:
return fmt.Errorf("invalid QuoteStyle value: %d", style)
}
}
}
// CombineOptions combines multiple options into a single Option.
// This is useful for creating option presets or combining version defaults
// with custom options.
func CombineOptions(opts ...Option) Option {
return func(o *Options) error {
for _, opt := range opts {
if err := opt(o); err != nil {
return err
}
}
return nil
}
}
// ApplyOptions applies the given options to a new options struct.
// Starts with v4 defaults.
func ApplyOptions(opts ...Option) (*Options, error) {
o := &Options{
Canonical: false,
LineBreak: LN_BREAK,
// v4 defaults
Indent: 2,
CompactSeqIndent: true,
LineWidth: 80,
Unicode: true,
UniqueKeys: true,
}
for _, opt := range opts {
if err := opt(o); err != nil {
return nil, err
}
}
return o, nil
}
// DefaultOptions holds the default options for APIs that don't accept options.
var DefaultOptions = &Options{
Indent: 4,
LineWidth: -1,
Unicode: true,
UniqueKeys: true,
QuotePreference: QuoteLegacy,
}
File diff suppressed because it is too large Load Diff
+441
View File
@@ -0,0 +1,441 @@
// Copyright 2006-2010 Kirill Simonov
// Copyright 2011-2019 Canonical Ltd
// Copyright 2025 The go-yaml Project Contributors
// SPDX-License-Identifier: Apache-2.0 AND MIT
// Input reader with encoding detection and buffering.
// Handles BOM detection, UTF-8/UTF-16 conversion, and provides buffered input
// for the scanner.
package libyaml
import (
"errors"
"fmt"
"io"
)
func formatReaderError(problem string, offset int, value int) error {
return ReaderError{
Offset: offset,
Value: value,
Err: errors.New(problem),
}
}
// Byte order marks.
const (
bom_UTF8 = "\xef\xbb\xbf"
bom_UTF16LE = "\xff\xfe"
bom_UTF16BE = "\xfe\xff"
)
// Determine the input stream encoding by checking the BOM symbol. If no BOM is
// found, the UTF-8 encoding is assumed. Return 1 on success, 0 on failure.
func (parser *Parser) determineEncoding() error {
// Ensure that we had enough bytes in the raw buffer.
for !parser.eof && len(parser.raw_buffer)-parser.raw_buffer_pos < 3 {
if err := parser.updateRawBuffer(); err != nil {
return err
}
}
// Determine the encoding.
buf := parser.raw_buffer
pos := parser.raw_buffer_pos
avail := len(buf) - pos
if avail >= 2 && buf[pos] == bom_UTF16LE[0] && buf[pos+1] == bom_UTF16LE[1] {
parser.encoding = UTF16LE_ENCODING
parser.raw_buffer_pos += 2
parser.offset += 2
} else if avail >= 2 && buf[pos] == bom_UTF16BE[0] && buf[pos+1] == bom_UTF16BE[1] {
parser.encoding = UTF16BE_ENCODING
parser.raw_buffer_pos += 2
parser.offset += 2
} else if avail >= 3 && buf[pos] == bom_UTF8[0] && buf[pos+1] == bom_UTF8[1] && buf[pos+2] == bom_UTF8[2] {
parser.encoding = UTF8_ENCODING
parser.raw_buffer_pos += 3
parser.offset += 3
} else {
parser.encoding = UTF8_ENCODING
}
return nil
}
// Update the raw buffer.
func (parser *Parser) updateRawBuffer() error {
size_read := 0
// Return if the raw buffer is full.
if parser.raw_buffer_pos == 0 && len(parser.raw_buffer) == cap(parser.raw_buffer) {
return nil
}
// Return on EOF.
if parser.eof {
return nil
}
// Move the remaining bytes in the raw buffer to the beginning.
if parser.raw_buffer_pos > 0 && parser.raw_buffer_pos < len(parser.raw_buffer) {
copy(parser.raw_buffer, parser.raw_buffer[parser.raw_buffer_pos:])
}
parser.raw_buffer = parser.raw_buffer[:len(parser.raw_buffer)-parser.raw_buffer_pos]
parser.raw_buffer_pos = 0
// Call the read handler to fill the buffer.
size_read, err := parser.read_handler(parser, parser.raw_buffer[len(parser.raw_buffer):cap(parser.raw_buffer)])
parser.raw_buffer = parser.raw_buffer[:len(parser.raw_buffer)+size_read]
if err == io.EOF {
parser.eof = true
} else if err != nil {
return ReaderError{
Offset: parser.offset,
Value: -1,
Err: fmt.Errorf("input error: %w", err),
}
}
return nil
}
// Ensure that the buffer contains at least `length` characters.
// Return true on success, false on failure.
//
// The length is supposed to be significantly less that the buffer size.
func (parser *Parser) updateBuffer(length int) error {
if parser.read_handler == nil {
panic("read handler must be set")
}
// [Go] This function was changed to guarantee the requested length size at EOF.
// The fact we need to do this is pretty awful, but the description above implies
// for that to be the case, and there are tests
// If the EOF flag is set and the raw buffer is empty, do nothing.
//
//nolint:staticcheck // there is no problem with this empty branch as it's documentation.
if parser.eof && parser.raw_buffer_pos == len(parser.raw_buffer) {
// [Go] ACTUALLY! Read the documentation of this function above.
// This is just broken. To return true, we need to have the
// given length in the buffer. Not doing that means every single
// check that calls this function to make sure the buffer has a
// given length is Go) panicking; or C) accessing invalid memory.
// return true
}
// Return if the buffer contains enough characters.
if parser.unread >= length {
return nil
}
// Determine the input encoding if it is not known yet.
if parser.encoding == ANY_ENCODING {
if err := parser.determineEncoding(); err != nil {
return err
}
}
// Move the unread characters to the beginning of the buffer.
buffer_len := len(parser.buffer)
if parser.buffer_pos > 0 && parser.buffer_pos < buffer_len {
copy(parser.buffer, parser.buffer[parser.buffer_pos:])
buffer_len -= parser.buffer_pos
parser.buffer_pos = 0
} else if parser.buffer_pos == buffer_len {
buffer_len = 0
parser.buffer_pos = 0
}
// Open the whole buffer for writing, and cut it before returning.
parser.buffer = parser.buffer[:cap(parser.buffer)]
// Fill the buffer until it has enough characters.
first := true
for parser.unread < length {
// Fill the raw buffer if necessary.
if !first || parser.raw_buffer_pos == len(parser.raw_buffer) {
if err := parser.updateRawBuffer(); err != nil {
parser.buffer = parser.buffer[:buffer_len]
return err
}
}
first = false
// Decode the raw buffer.
inner:
for parser.raw_buffer_pos != len(parser.raw_buffer) {
var value rune
var width int
raw_unread := len(parser.raw_buffer) - parser.raw_buffer_pos
// Decode the next character.
switch parser.encoding {
case UTF8_ENCODING:
// Decode a UTF-8 character. Check RFC 3629
// (http://www.ietf.org/rfc/rfc3629.txt) for more details.
//
// The following table (taken from the RFC) is used for
// decoding.
//
// Char. number range | UTF-8 octet sequence
// (hexadecimal) | (binary)
// --------------------+------------------------------------
// 0000 0000-0000 007F | 0xxxxxxx
// 0000 0080-0000 07FF | 110xxxxx 10xxxxxx
// 0000 0800-0000 FFFF | 1110xxxx 10xxxxxx 10xxxxxx
// 0001 0000-0010 FFFF | 11110xxx 10xxxxxx 10xxxxxx 10xxxxxx
//
// Additionally, the characters in the range 0xD800-0xDFFF
// are prohibited as they are reserved for use with UTF-16
// surrogate pairs.
// Determine the length of the UTF-8 sequence.
octet := parser.raw_buffer[parser.raw_buffer_pos]
switch {
case octet&0x80 == 0x00:
width = 1
case octet&0xE0 == 0xC0:
width = 2
case octet&0xF0 == 0xE0:
width = 3
case octet&0xF8 == 0xF0:
width = 4
default:
// The leading octet is invalid.
return formatReaderError(
"invalid leading UTF-8 octet",
parser.offset, int(octet))
}
// Check if the raw buffer contains an incomplete character.
if width > raw_unread {
if parser.eof {
return formatReaderError(
"incomplete UTF-8 octet sequence",
parser.offset, -1)
}
break inner
}
// Decode the leading octet.
switch {
case octet&0x80 == 0x00:
value = rune(octet & 0x7F)
case octet&0xE0 == 0xC0:
value = rune(octet & 0x1F)
case octet&0xF0 == 0xE0:
value = rune(octet & 0x0F)
case octet&0xF8 == 0xF0:
value = rune(octet & 0x07)
default:
value = 0
}
// Check and decode the trailing octets.
for k := 1; k < width; k++ {
octet = parser.raw_buffer[parser.raw_buffer_pos+k]
// Check if the octet is valid.
if (octet & 0xC0) != 0x80 {
return formatReaderError(
"invalid trailing UTF-8 octet",
parser.offset+k, int(octet))
}
// Decode the octet.
value = (value << 6) + rune(octet&0x3F)
}
// Check the length of the sequence against the value.
switch {
case width == 1:
case width == 2 && value >= 0x80:
case width == 3 && value >= 0x800:
case width == 4 && value >= 0x10000:
default:
return formatReaderError(
"invalid length of a UTF-8 sequence",
parser.offset, -1)
}
// Check the range of the value.
if value >= 0xD800 && value <= 0xDFFF || value > 0x10FFFF {
return formatReaderError(
"invalid Unicode character",
parser.offset, int(value))
}
case UTF16LE_ENCODING, UTF16BE_ENCODING:
var low, high int
if parser.encoding == UTF16LE_ENCODING {
low, high = 0, 1
} else {
low, high = 1, 0
}
// The UTF-16 encoding is not as simple as one might
// naively think. Check RFC 2781
// (http://www.ietf.org/rfc/rfc2781.txt).
//
// Normally, two subsequent bytes describe a Unicode
// character. However a special technique (called a
// surrogate pair) is used for specifying character
// values larger than 0xFFFF.
//
// A surrogate pair consists of two pseudo-characters:
// high surrogate area (0xD800-0xDBFF)
// low surrogate area (0xDC00-0xDFFF)
//
// The following formulas are used for decoding
// and encoding characters using surrogate pairs:
//
// U = U' + 0x10000 (0x01 00 00 <= U <= 0x10 FF FF)
// U' = yyyyyyyyyyxxxxxxxxxx (0 <= U' <= 0x0F FF FF)
// W1 = 110110yyyyyyyyyy
// W2 = 110111xxxxxxxxxx
//
// where U is the character value, W1 is the high surrogate
// area, W2 is the low surrogate area.
// Check for incomplete UTF-16 character.
if raw_unread < 2 {
if parser.eof {
return formatReaderError(
"incomplete UTF-16 character",
parser.offset, -1)
}
break inner
}
// Get the character.
value = rune(parser.raw_buffer[parser.raw_buffer_pos+low]) +
(rune(parser.raw_buffer[parser.raw_buffer_pos+high]) << 8)
// Check for unexpected low surrogate area.
if value&0xFC00 == 0xDC00 {
return formatReaderError(
"unexpected low surrogate area",
parser.offset, int(value))
}
// Check for a high surrogate area.
if value&0xFC00 == 0xD800 {
width = 4
// Check for incomplete surrogate pair.
if raw_unread < 4 {
if parser.eof {
return formatReaderError(
"incomplete UTF-16 surrogate pair",
parser.offset, -1)
}
break inner
}
// Get the next character.
value2 := rune(parser.raw_buffer[parser.raw_buffer_pos+low+2]) +
(rune(parser.raw_buffer[parser.raw_buffer_pos+high+2]) << 8)
// Check for a low surrogate area.
if value2&0xFC00 != 0xDC00 {
return formatReaderError(
"expected low surrogate area",
parser.offset+2, int(value2))
}
// Generate the value of the surrogate pair.
value = 0x10000 + ((value & 0x3FF) << 10) + (value2 & 0x3FF)
} else {
width = 2
}
default:
panic("impossible")
}
// YAML 1.2 compatible character sets
// Check if the character is in the allowed range:
// For JSON compatibility in quoted scalars, we must allow all
// non-C0 characters. This includes ASCII DEL (0x7F) and the
// C1 control block [#x80-#x9F].
// ref: https://yaml.org/spec/1.2.2/#51-character-set
switch {
// 8 bit set
// Tab (\t)
case value == 0x09:
// Line feed (LF \n)
case value == 0x0A:
// Carriage Return (CR \r)
case value == 0x0D:
// 16 bit set
// Printable ASCII
case value >= 0x20 && value <= 0x7E:
// DEL, C1 control
// incompatible with YAML versions <= 1.1
case value >= 0x7F && value <= 0x9F:
// and Basic Multilingual Plane (BMP),
case value >= 0xA0 && value <= 0xD7FF:
// Additional Unicode Areas
case value >= 0xE000 && value <= 0xFFFD:
// 32 bit set
case value >= 0x10000 && value <= 0x10FFFF:
default:
return formatReaderError(
"control characters are not allowed",
parser.offset, int(value))
}
// Move the raw pointers.
parser.raw_buffer_pos += width
parser.offset += width
// Finally put the character into the buffer.
if value <= 0x7F {
// 0000 0000-0000 007F . 0xxxxxxx
parser.buffer[buffer_len+0] = byte(value)
buffer_len += 1
} else if value <= 0x7FF {
// 0000 0080-0000 07FF . 110xxxxx 10xxxxxx
parser.buffer[buffer_len+0] = byte(0xC0 + (value >> 6))
parser.buffer[buffer_len+1] = byte(0x80 + (value & 0x3F))
buffer_len += 2
} else if value <= 0xFFFF {
// 0000 0800-0000 FFFF . 1110xxxx 10xxxxxx 10xxxxxx
parser.buffer[buffer_len+0] = byte(0xE0 + (value >> 12))
parser.buffer[buffer_len+1] = byte(0x80 + ((value >> 6) & 0x3F))
parser.buffer[buffer_len+2] = byte(0x80 + (value & 0x3F))
buffer_len += 3
} else {
// 0001 0000-0010 FFFF . 11110xxx 10xxxxxx 10xxxxxx 10xxxxxx
parser.buffer[buffer_len+0] = byte(0xF0 + (value >> 18))
parser.buffer[buffer_len+1] = byte(0x80 + ((value >> 12) & 0x3F))
parser.buffer[buffer_len+2] = byte(0x80 + ((value >> 6) & 0x3F))
parser.buffer[buffer_len+3] = byte(0x80 + (value & 0x3F))
buffer_len += 4
}
parser.unread++
}
// On EOF, put NUL into the buffer and return.
if parser.eof {
parser.buffer[buffer_len] = 0
buffer_len++
parser.unread++
break
}
}
// [Go] Read the documentation of this function above. To return true,
// we need to have the given length in the buffer. Not doing that means
// every single check that calls this function to make sure the buffer
// has a given length is Go) panicking; or C) accessing invalid memory.
// This happens here due to the EOF above breaking early.
for buffer_len < length {
parser.buffer[buffer_len] = 0
buffer_len++
}
parser.buffer = parser.buffer[:buffer_len]
return nil
}
+571
View File
@@ -0,0 +1,571 @@
// Copyright 2011-2019 Canonical Ltd
// Copyright 2025 The go-yaml Project Contributors
// SPDX-License-Identifier: Apache-2.0
// Representer stage: Converts Go values to YAML nodes.
// Handles marshaling from Go types to the intermediate node representation.
package libyaml
import (
"encoding"
"fmt"
"io"
"reflect"
"regexp"
"sort"
"strconv"
"strings"
"time"
"unicode"
"unicode/utf8"
)
type keyList []reflect.Value
func (l keyList) Len() int { return len(l) }
func (l keyList) Swap(i, j int) { l[i], l[j] = l[j], l[i] }
func (l keyList) Less(i, j int) bool {
a := l[i]
b := l[j]
ak := a.Kind()
bk := b.Kind()
for (ak == reflect.Interface || ak == reflect.Pointer) && !a.IsNil() {
a = a.Elem()
ak = a.Kind()
}
for (bk == reflect.Interface || bk == reflect.Pointer) && !b.IsNil() {
b = b.Elem()
bk = b.Kind()
}
af, aok := keyFloat(a)
bf, bok := keyFloat(b)
if aok && bok {
if af != bf {
return af < bf
}
if ak != bk {
return ak < bk
}
return numLess(a, b)
}
if ak != reflect.String || bk != reflect.String {
return ak < bk
}
ar, br := []rune(a.String()), []rune(b.String())
digits := false
for i := 0; i < len(ar) && i < len(br); i++ {
if ar[i] == br[i] {
digits = unicode.IsDigit(ar[i])
continue
}
al := unicode.IsLetter(ar[i])
bl := unicode.IsLetter(br[i])
if al && bl {
return ar[i] < br[i]
}
if al || bl {
if digits {
return al
} else {
return bl
}
}
var ai, bi int
var an, bn int64
if ar[i] == '0' || br[i] == '0' {
for j := i - 1; j >= 0 && unicode.IsDigit(ar[j]); j-- {
if ar[j] != '0' {
an = 1
bn = 1
break
}
}
}
for ai = i; ai < len(ar) && unicode.IsDigit(ar[ai]); ai++ {
an = an*10 + int64(ar[ai]-'0')
}
for bi = i; bi < len(br) && unicode.IsDigit(br[bi]); bi++ {
bn = bn*10 + int64(br[bi]-'0')
}
if an != bn {
return an < bn
}
if ai != bi {
return ai < bi
}
return ar[i] < br[i]
}
return len(ar) < len(br)
}
// keyFloat returns a float value for v if it is a number/bool
// and whether it is a number/bool or not.
func keyFloat(v reflect.Value) (f float64, ok bool) {
switch v.Kind() {
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
return float64(v.Int()), true
case reflect.Float32, reflect.Float64:
return v.Float(), true
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr:
return float64(v.Uint()), true
case reflect.Bool:
if v.Bool() {
return 1, true
}
return 0, true
}
return 0, false
}
// numLess returns whether a < b.
// a and b must necessarily have the same kind.
func numLess(a, b reflect.Value) bool {
switch a.Kind() {
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
return a.Int() < b.Int()
case reflect.Float32, reflect.Float64:
return a.Float() < b.Float()
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr:
return a.Uint() < b.Uint()
case reflect.Bool:
return !a.Bool() && b.Bool()
}
panic("not a number")
}
// Sentinel values for newRepresenter parameters.
// These provide clarity at call sites, similar to http.NoBody.
var (
noWriter io.Writer = nil
noVersionDirective *VersionDirective = nil
noTagDirective []TagDirective = nil
)
type Representer struct {
Emitter Emitter
Out []byte
flow bool
Indent int
lineWidth int
doneInit bool
explicitStart bool
explicitEnd bool
flowSimpleCollections bool
quotePreference QuoteStyle
}
// NewRepresenter creates a new YAML representr with the given options.
//
// The writer parameter specifies the output destination for the representr.
// If writer is nil, the representr will write to an internal buffer.
func NewRepresenter(writer io.Writer, opts *Options) *Representer {
emitter := NewEmitter()
emitter.CompactSequenceIndent = opts.CompactSeqIndent
emitter.quotePreference = opts.QuotePreference
emitter.SetWidth(opts.LineWidth)
emitter.SetUnicode(opts.Unicode)
emitter.SetCanonical(opts.Canonical)
emitter.SetLineBreak(opts.LineBreak)
r := &Representer{
Emitter: emitter,
Indent: opts.Indent,
lineWidth: opts.LineWidth,
explicitStart: opts.ExplicitStart,
explicitEnd: opts.ExplicitEnd,
flowSimpleCollections: opts.FlowSimpleCollections,
quotePreference: opts.QuotePreference,
}
if writer != nil {
r.Emitter.SetOutputWriter(writer)
} else {
r.Emitter.SetOutputString(&r.Out)
}
return r
}
func (r *Representer) init() {
if r.doneInit {
return
}
if r.Indent == 0 {
r.Indent = 4
}
r.Emitter.BestIndent = r.Indent
r.emit(NewStreamStartEvent(UTF8_ENCODING))
r.doneInit = true
}
func (r *Representer) Finish() {
r.Emitter.OpenEnded = false
r.emit(NewStreamEndEvent())
}
func (r *Representer) Destroy() {
r.Emitter.Delete()
}
func (r *Representer) emit(event Event) {
// This will internally delete the event value.
r.must(r.Emitter.Emit(&event))
}
func (r *Representer) must(err error) {
if err != nil {
msg := err.Error()
if msg == "" {
msg = "unknown problem generating YAML content"
}
failf("%s", msg)
}
}
func (r *Representer) MarshalDoc(tag string, in reflect.Value) {
r.init()
var node *Node
if in.IsValid() {
node, _ = in.Interface().(*Node)
}
if node != nil && node.Kind == DocumentNode {
r.nodev(in)
} else {
// Use !explicitStart for implicit flag (true = implicit/no marker)
r.emit(NewDocumentStartEvent(noVersionDirective, noTagDirective, !r.explicitStart))
r.marshal(tag, in)
// Use !explicitEnd for implicit flag
r.emit(NewDocumentEndEvent(!r.explicitEnd))
}
}
func (r *Representer) marshal(tag string, in reflect.Value) {
tag = shortTag(tag)
if !in.IsValid() || in.Kind() == reflect.Pointer && in.IsNil() {
r.nilv()
return
}
iface := in.Interface()
switch value := iface.(type) {
case *Node:
r.nodev(in)
return
case Node:
if !in.CanAddr() {
n := reflect.New(in.Type()).Elem()
n.Set(in)
in = n
}
r.nodev(in.Addr())
return
case time.Time:
r.timev(tag, in)
return
case *time.Time:
r.timev(tag, in.Elem())
return
case time.Duration:
r.stringv(tag, reflect.ValueOf(value.String()))
return
case Marshaler:
v, err := value.MarshalYAML()
if err != nil {
Fail(err)
}
if v == nil {
r.nilv()
return
}
r.marshal(tag, reflect.ValueOf(v))
return
case encoding.TextMarshaler:
text, err := value.MarshalText()
if err != nil {
Fail(err)
}
in = reflect.ValueOf(string(text))
case nil:
r.nilv()
return
}
switch in.Kind() {
case reflect.Interface:
r.marshal(tag, in.Elem())
case reflect.Map:
r.mapv(tag, in)
case reflect.Pointer:
r.marshal(tag, in.Elem())
case reflect.Struct:
r.structv(tag, in)
case reflect.Slice, reflect.Array:
r.slicev(tag, in)
case reflect.String:
r.stringv(tag, in)
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
r.intv(tag, in)
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr:
r.uintv(tag, in)
case reflect.Float32, reflect.Float64:
r.floatv(tag, in)
case reflect.Bool:
r.boolv(tag, in)
default:
panic("cannot marshal type: " + in.Type().String())
}
}
func (r *Representer) mapv(tag string, in reflect.Value) {
r.mappingv(tag, func() {
keys := keyList(in.MapKeys())
sort.Sort(keys)
for _, k := range keys {
r.marshal("", k)
r.marshal("", in.MapIndex(k))
}
})
}
func (r *Representer) fieldByIndex(v reflect.Value, index []int) (field reflect.Value) {
for _, num := range index {
for {
if v.Kind() == reflect.Pointer {
if v.IsNil() {
return reflect.Value{}
}
v = v.Elem()
continue
}
break
}
v = v.Field(num)
}
return v
}
func (r *Representer) structv(tag string, in reflect.Value) {
sinfo, err := getStructInfo(in.Type())
if err != nil {
panic(err)
}
r.mappingv(tag, func() {
for _, info := range sinfo.FieldsList {
var value reflect.Value
if info.Inline == nil {
value = in.Field(info.Num)
} else {
value = r.fieldByIndex(in, info.Inline)
if !value.IsValid() {
continue
}
}
if info.OmitEmpty && isZero(value) {
continue
}
r.marshal("", reflect.ValueOf(info.Key))
r.flow = info.Flow
r.marshal("", value)
}
if sinfo.InlineMap >= 0 {
m := in.Field(sinfo.InlineMap)
if m.Len() > 0 {
r.flow = false
keys := keyList(m.MapKeys())
sort.Sort(keys)
for _, k := range keys {
if _, found := sinfo.FieldsMap[k.String()]; found {
panic(fmt.Sprintf("cannot have key %q in inlined map: conflicts with struct field", k.String()))
}
r.marshal("", k)
r.flow = false
r.marshal("", m.MapIndex(k))
}
}
}
})
}
func (r *Representer) mappingv(tag string, f func()) {
implicit := tag == ""
style := BLOCK_MAPPING_STYLE
if r.flow {
r.flow = false
style = FLOW_MAPPING_STYLE
}
r.emit(NewMappingStartEvent(nil, []byte(tag), implicit, style))
f()
r.emit(NewMappingEndEvent())
}
func (r *Representer) slicev(tag string, in reflect.Value) {
implicit := tag == ""
style := BLOCK_SEQUENCE_STYLE
if r.flow {
r.flow = false
style = FLOW_SEQUENCE_STYLE
}
r.emit(NewSequenceStartEvent(nil, []byte(tag), implicit, style))
n := in.Len()
for i := 0; i < n; i++ {
r.marshal("", in.Index(i))
}
r.emit(NewSequenceEndEvent())
}
// isBase60 returns whether s is in base 60 notation as defined in YAML 1.1.
//
// The base 60 float notation in YAML 1.1 is a terrible idea and is unsupported
// in YAML 1.2 and by this package, but these should be marshaled quoted for
// the time being for compatibility with other parsers.
func isBase60Float(s string) (result bool) {
// Fast path.
if s == "" {
return false
}
c := s[0]
if !(c == '+' || c == '-' || c >= '0' && c <= '9') || strings.IndexByte(s, ':') < 0 {
return false
}
// Do the full match.
return base60float.MatchString(s)
}
// From http://yaml.org/type/float.html, except the regular expression there
// is bogus. In practice parsers do not enforce the "\.[0-9_]*" suffix.
var base60float = regexp.MustCompile(`^[-+]?[0-9][0-9_]*(?::[0-5]?[0-9])+(?:\.[0-9_]*)?$`)
// isOldBool returns whether s is bool notation as defined in YAML 1.1.
//
// We continue to force strings that YAML 1.1 would interpret as booleans to be
// rendered as quotes strings so that the marshaled output valid for YAML 1.1
// parsing.
func isOldBool(s string) (result bool) {
switch s {
case "y", "Y", "yes", "Yes", "YES", "on", "On", "ON",
"n", "N", "no", "No", "NO", "off", "Off", "OFF":
return true
default:
return false
}
}
// looksLikeMerge returns true if the given string is the merge indicator "<<".
//
// When encoding a scalar with this exact value, it must be quoted to prevent it
// from being interpreted as a merge indicator during decoding.
func looksLikeMerge(s string) (result bool) {
return s == "<<"
}
func (r *Representer) stringv(tag string, in reflect.Value) {
var style ScalarStyle
s := in.String()
canUsePlain := true
switch {
case !utf8.ValidString(s):
if tag == binaryTag {
failf("explicitly tagged !!binary data must be base64-encoded")
}
if tag != "" {
failf("cannot marshal invalid UTF-8 data as %s", shortTag(tag))
}
// It can't be represented directly as YAML so use a binary tag
// and represent it as base64.
tag = binaryTag
s = encodeBase64(s)
case tag == "":
// Check to see if it would resolve to a specific
// tag when represented unquoted. If it doesn't,
// there's no need to quote it.
rtag, _ := resolve("", s)
canUsePlain = rtag == strTag &&
!(isBase60Float(s) ||
isOldBool(s) ||
looksLikeMerge(s))
}
// Note: it's possible for user code to emit invalid YAML
// if they explicitly specify a tag and a string containing
// text that's incompatible with that tag.
switch {
case strings.Contains(s, "\n"):
if r.flow || !shouldUseLiteralStyle(s) {
style = DOUBLE_QUOTED_SCALAR_STYLE
} else {
style = LITERAL_SCALAR_STYLE
}
case canUsePlain:
style = PLAIN_SCALAR_STYLE
default:
style = r.quotePreference.ScalarStyle()
}
r.emitScalar(s, "", tag, style, nil, nil, nil, nil)
}
func (r *Representer) boolv(tag string, in reflect.Value) {
var s string
if in.Bool() {
s = "true"
} else {
s = "false"
}
r.emitScalar(s, "", tag, PLAIN_SCALAR_STYLE, nil, nil, nil, nil)
}
func (r *Representer) intv(tag string, in reflect.Value) {
s := strconv.FormatInt(in.Int(), 10)
r.emitScalar(s, "", tag, PLAIN_SCALAR_STYLE, nil, nil, nil, nil)
}
func (r *Representer) uintv(tag string, in reflect.Value) {
s := strconv.FormatUint(in.Uint(), 10)
r.emitScalar(s, "", tag, PLAIN_SCALAR_STYLE, nil, nil, nil, nil)
}
func (r *Representer) timev(tag string, in reflect.Value) {
t := in.Interface().(time.Time)
s := t.Format(time.RFC3339Nano)
r.emitScalar(s, "", tag, PLAIN_SCALAR_STYLE, nil, nil, nil, nil)
}
func (r *Representer) floatv(tag string, in reflect.Value) {
// Issue #352: When formatting, use the precision of the underlying value
precision := 64
if in.Kind() == reflect.Float32 {
precision = 32
}
s := strconv.FormatFloat(in.Float(), 'g', -1, precision)
switch s {
case "+Inf":
s = ".inf"
case "-Inf":
s = "-.inf"
case "NaN":
s = ".nan"
}
r.emitScalar(s, "", tag, PLAIN_SCALAR_STYLE, nil, nil, nil, nil)
}
func (r *Representer) nilv() {
r.emitScalar("null", "", "", PLAIN_SCALAR_STYLE, nil, nil, nil, nil)
}
func (r *Representer) emitScalar(
value, anchor, tag string, style ScalarStyle, head, line, foot, tail []byte,
) {
// TODO Kill this function. Replace all initialize calls by their underlining Go literals.
implicit := tag == ""
if !implicit {
tag = longTag(tag)
}
event := NewScalarEvent([]byte(anchor), []byte(tag), []byte(value), implicit, implicit, style)
event.HeadComment = head
event.LineComment = line
event.FootComment = foot
event.TailComment = tail
r.emit(event)
}
func (r *Representer) nodev(in reflect.Value) {
r.node(in.Interface().(*Node), "")
}
+231
View File
@@ -0,0 +1,231 @@
// Copyright 2011-2019 Canonical Ltd
// Copyright 2025 The go-yaml Project Contributors
// SPDX-License-Identifier: Apache-2.0
// Tag resolution for YAML scalars.
// Determines implicit types (int, float, bool, null, timestamp) from untagged
// scalar values.
package libyaml
import (
"encoding/base64"
"math"
"regexp"
"strconv"
"strings"
"time"
)
type resolveMapItem struct {
value any
tag string
}
var (
resolveTable = make([]byte, 256)
resolveMap = make(map[string]resolveMapItem)
)
// negativeZero represents -0.0 for YAML encoding/decoding
// this is needed because Go constants cannot express -0.0
// https://staticcheck.dev/docs/checks/#SA4026
var negativeZero = math.Copysign(0.0, -1.0)
func init() {
t := resolveTable
t[int('+')] = 'S' // Sign
t[int('-')] = 'S'
for _, c := range "0123456789" {
t[int(c)] = 'D' // Digit
}
for _, c := range "yYnNtTfFoO~<" { // < for merge key <<
t[int(c)] = 'M' // In map
}
t[int('.')] = '.' // Float (potentially in map)
resolveMapList := []struct {
v any
tag string
l []string
}{
{true, boolTag, []string{"true", "True", "TRUE"}},
{false, boolTag, []string{"false", "False", "FALSE"}},
{nil, nullTag, []string{"", "~", "null", "Null", "NULL"}},
{math.NaN(), floatTag, []string{".nan", ".NaN", ".NAN"}},
{math.Inf(+1), floatTag, []string{".inf", ".Inf", ".INF"}},
{math.Inf(+1), floatTag, []string{"+.inf", "+.Inf", "+.INF"}},
{math.Inf(-1), floatTag, []string{"-.inf", "-.Inf", "-.INF"}},
{negativeZero, floatTag, []string{"-0", "-0.0"}},
{"<<", mergeTag, []string{"<<"}},
}
m := resolveMap
for _, item := range resolveMapList {
for _, s := range item.l {
m[s] = resolveMapItem{item.v, item.tag}
}
}
}
func resolvableTag(tag string) bool {
switch tag {
case "", strTag, boolTag, intTag, floatTag, nullTag, timestampTag:
return true
}
return false
}
var yamlStyleFloat = regexp.MustCompile(`^[-+]?(\.[0-9]+|[0-9]+(\.[0-9]*)?)([eE][-+]?[0-9]+)?$`)
func resolve(tag string, in string) (rtag string, out any) {
tag = shortTag(tag)
if !resolvableTag(tag) {
return tag, in
}
defer func() {
switch tag {
case "", rtag, strTag, binaryTag:
return
case floatTag:
if rtag == intTag {
switch v := out.(type) {
case int64:
rtag = floatTag
out = float64(v)
return
case int:
rtag = floatTag
out = float64(v)
return
}
}
}
failf("cannot construct %s `%s` as a %s", shortTag(rtag), in, shortTag(tag))
}()
// Any data is accepted as a !!str or !!binary.
// Otherwise, the prefix is enough of a hint about what it might be.
hint := byte('N')
if in != "" {
hint = resolveTable[in[0]]
}
if hint != 0 && tag != strTag && tag != binaryTag {
// Handle things we can lookup in a map.
if item, ok := resolveMap[in]; ok {
return item.tag, item.value
}
// Base 60 floats are a bad idea, were dropped in YAML 1.2, and
// are purposefully unsupported here. They're still quoted on
// the way out for compatibility with other parser, though.
switch hint {
case 'M':
// We've already checked the map above.
case '.':
// Not in the map, so maybe a normal float.
floatv, err := strconv.ParseFloat(in, 64)
if err == nil {
return floatTag, floatv
}
case 'D', 'S':
// Int, float, or timestamp.
// Only try values as a timestamp if the value is unquoted or there's an explicit
// !!timestamp tag.
if tag == "" || tag == timestampTag {
t, ok := parseTimestamp(in)
if ok {
return timestampTag, t
}
}
plain := strings.ReplaceAll(in, "_", "")
intv, err := strconv.ParseInt(plain, 0, 64)
if err == nil {
if intv == int64(int(intv)) {
return intTag, int(intv)
} else {
return intTag, intv
}
}
uintv, err := strconv.ParseUint(plain, 0, 64)
if err == nil {
return intTag, uintv
}
if yamlStyleFloat.MatchString(plain) {
floatv, err := strconv.ParseFloat(plain, 64)
if err == nil {
return floatTag, floatv
}
}
default:
panic("internal error: missing handler for resolver table: " + string(rune(hint)) + " (with " + in + ")")
}
}
return strTag, in
}
// encodeBase64 encodes s as base64 that is broken up into multiple lines
// as appropriate for the resulting length.
func encodeBase64(s string) string {
const lineLen = 70
encLen := base64.StdEncoding.EncodedLen(len(s))
lines := encLen/lineLen + 1
buf := make([]byte, encLen*2+lines)
in := buf[0:encLen]
out := buf[encLen:]
base64.StdEncoding.Encode(in, []byte(s))
k := 0
for i := 0; i < len(in); i += lineLen {
j := i + lineLen
if j > len(in) {
j = len(in)
}
k += copy(out[k:], in[i:j])
if lines > 1 {
out[k] = '\n'
k++
}
}
return string(out[:k])
}
// This is a subset of the formats allowed by the regular expression
// defined at http://yaml.org/type/timestamp.html.
var allowedTimestampFormats = []string{
"2006-1-2T15:4:5.999999999Z07:00", // RCF3339Nano with short date fields.
"2006-1-2t15:4:5.999999999Z07:00", // RFC3339Nano with short date fields and lower-case "t".
"2006-1-2 15:4:5.999999999", // space separated with no time zone
"2006-1-2", // date only
// Notable exception: time.Parse cannot handle: "2001-12-14 21:59:43.10 -5"
// from the set of examples.
}
// parseTimestamp parses s as a timestamp string and
// returns the timestamp and reports whether it succeeded.
// Timestamp formats are defined at http://yaml.org/type/timestamp.html
func parseTimestamp(s string) (time.Time, bool) {
// TODO write code to check all the formats supported by
// http://yaml.org/type/timestamp.html instead of using time.Parse.
// Quick check: all date formats start with YYYY-.
i := 0
for ; i < len(s); i++ {
if c := s[i]; c < '0' || c > '9' {
break
}
}
if i != 4 || i == len(s) || s[i] != '-' {
return time.Time{}, false
}
for _, format := range allowedTimestampFormats {
if t, err := time.Parse(format, s); err == nil {
return t, true
}
}
return time.Time{}, false
}
File diff suppressed because it is too large Load Diff
+219
View File
@@ -0,0 +1,219 @@
// Copyright 2011-2019 Canonical Ltd
// Copyright 2025 The go-yaml Project Contributors
// SPDX-License-Identifier: Apache-2.0
// Serializer stage: Converts representation tree (Nodes) to event stream.
// Walks the node tree and produces events for the emitter.
package libyaml
import (
"strings"
"unicode/utf8"
)
// node serializes a Node tree into YAML events.
// This is the core of the serializer stage - it walks the tree and produces events.
func (r *Representer) node(node *Node, tail string) {
// Zero nodes behave as nil.
if node.Kind == 0 && node.IsZero() {
r.nilv()
return
}
// If the tag was not explicitly requested, and dropping it won't change the
// implicit tag of the value, don't include it in the presentation.
tag := node.Tag
stag := shortTag(tag)
var forceQuoting bool
if tag != "" && node.Style&TaggedStyle == 0 {
if node.Kind == ScalarNode {
if stag == strTag && node.Style&(SingleQuotedStyle|DoubleQuotedStyle|LiteralStyle|FoldedStyle) != 0 {
tag = ""
} else {
rtag, _ := resolve("", node.Value)
if rtag == stag && stag != mergeTag {
tag = ""
} else if stag == strTag {
tag = ""
forceQuoting = true
}
}
} else {
var rtag string
switch node.Kind {
case MappingNode:
rtag = mapTag
case SequenceNode:
rtag = seqTag
}
if rtag == stag {
tag = ""
}
}
}
switch node.Kind {
case DocumentNode:
event := NewDocumentStartEvent(noVersionDirective, noTagDirective, !r.explicitStart)
event.HeadComment = []byte(node.HeadComment)
r.emit(event)
for _, node := range node.Content {
r.node(node, "")
}
event = NewDocumentEndEvent(!r.explicitEnd)
event.FootComment = []byte(node.FootComment)
r.emit(event)
case SequenceNode:
style := BLOCK_SEQUENCE_STYLE
// Use flow style if explicitly requested or if it's a simple
// collection (scalar-only contents that fit within line width,
// enabled via WithFlowSimpleCollections)
if node.Style&FlowStyle != 0 || r.isSimpleCollection(node) {
style = FLOW_SEQUENCE_STYLE
}
event := NewSequenceStartEvent([]byte(node.Anchor), []byte(longTag(tag)), tag == "", style)
event.HeadComment = []byte(node.HeadComment)
r.emit(event)
for _, node := range node.Content {
r.node(node, "")
}
event = NewSequenceEndEvent()
event.LineComment = []byte(node.LineComment)
event.FootComment = []byte(node.FootComment)
r.emit(event)
case MappingNode:
style := BLOCK_MAPPING_STYLE
// Use flow style if explicitly requested or if it's a simple
// collection (scalar-only contents that fit within line width,
// enabled via WithFlowSimpleCollections)
if node.Style&FlowStyle != 0 || r.isSimpleCollection(node) {
style = FLOW_MAPPING_STYLE
}
event := NewMappingStartEvent([]byte(node.Anchor), []byte(longTag(tag)), tag == "", style)
event.TailComment = []byte(tail)
event.HeadComment = []byte(node.HeadComment)
r.emit(event)
// The tail logic below moves the foot comment of prior keys to the following key,
// since the value for each key may be a nested structure and the foot needs to be
// processed only the entirety of the value is streamed. The last tail is processed
// with the mapping end event.
var tail string
for i := 0; i+1 < len(node.Content); i += 2 {
k := node.Content[i]
foot := k.FootComment
if foot != "" {
kopy := *k
kopy.FootComment = ""
k = &kopy
}
r.node(k, tail)
tail = foot
v := node.Content[i+1]
r.node(v, "")
}
event = NewMappingEndEvent()
event.TailComment = []byte(tail)
event.LineComment = []byte(node.LineComment)
event.FootComment = []byte(node.FootComment)
r.emit(event)
case AliasNode:
event := NewAliasEvent([]byte(node.Value))
event.HeadComment = []byte(node.HeadComment)
event.LineComment = []byte(node.LineComment)
event.FootComment = []byte(node.FootComment)
r.emit(event)
case ScalarNode:
value := node.Value
if !utf8.ValidString(value) {
if stag == binaryTag {
failf("explicitly tagged !!binary data must be base64-encoded")
}
if stag != "" {
failf("cannot marshal invalid UTF-8 data as %s", stag)
}
// It can't be represented directly as YAML so use a binary tag
// and represent it as base64.
tag = binaryTag
value = encodeBase64(value)
}
style := PLAIN_SCALAR_STYLE
switch {
case node.Style&DoubleQuotedStyle != 0:
style = DOUBLE_QUOTED_SCALAR_STYLE
case node.Style&SingleQuotedStyle != 0:
style = SINGLE_QUOTED_SCALAR_STYLE
case node.Style&LiteralStyle != 0:
style = LITERAL_SCALAR_STYLE
case node.Style&FoldedStyle != 0:
style = FOLDED_SCALAR_STYLE
case strings.Contains(value, "\n"):
style = LITERAL_SCALAR_STYLE
case forceQuoting:
style = r.quotePreference.ScalarStyle()
}
r.emitScalar(value, node.Anchor, tag, style, []byte(node.HeadComment), []byte(node.LineComment), []byte(node.FootComment), []byte(tail))
default:
failf("cannot represent node with unknown kind %d", node.Kind)
}
}
// isSimpleCollection checks if a node contains only scalar values and would
// fit within the line width when rendered in flow style.
func (r *Representer) isSimpleCollection(node *Node) bool {
if !r.flowSimpleCollections {
return false
}
if node.Kind != SequenceNode && node.Kind != MappingNode {
return false
}
// Check all children are scalars
for _, child := range node.Content {
if child.Kind != ScalarNode {
return false
}
}
// Estimate flow style length
estimatedLen := r.estimateFlowLength(node)
width := r.lineWidth
if width <= 0 {
width = 80 // Default width if not set
}
return estimatedLen > 0 && estimatedLen <= width
}
// estimateFlowLength estimates the character length of a node in flow style.
func (r *Representer) estimateFlowLength(node *Node) int {
if node.Kind == SequenceNode {
// [item1, item2, ...] = 2 + sum(len(items)) + 2*(len-1)
length := 2 // []
for i, child := range node.Content {
if i > 0 {
length += 2 // ", "
}
length += len(child.Value)
}
return length
}
if node.Kind == MappingNode {
// {key1: val1, key2: val2} = 2 + sum(key: val) + 2*(pairs-1)
length := 2 // {}
for i := 0; i < len(node.Content); i += 2 {
if i > 0 {
length += 2 // ", "
}
length += len(node.Content[i].Value) + 2 + len(node.Content[i+1].Value) // "key: val"
}
return length
}
return 0
}
+31
View File
@@ -0,0 +1,31 @@
// Copyright 2006-2010 Kirill Simonov
// Copyright 2011-2019 Canonical Ltd
// Copyright 2025 The go-yaml Project Contributors
// SPDX-License-Identifier: Apache-2.0 AND MIT
// Output writer with buffering.
// Provides write buffering for the emitter stage.
package libyaml
import "fmt"
// Flush the output buffer.
func (emitter *Emitter) flush() error {
if emitter.write_handler == nil {
panic("write handler not set")
}
// Check if the buffer is empty.
if emitter.buffer_pos == 0 {
return nil
}
if err := emitter.write_handler(emitter, emitter.buffer[:emitter.buffer_pos]); err != nil {
return WriterError{
Err: fmt.Errorf("write error: %w", err),
}
}
emitter.buffer_pos = 0
return nil
}
+834
View File
@@ -0,0 +1,834 @@
// Copyright 2006-2010 Kirill Simonov
// Copyright 2011-2019 Canonical Ltd
// Copyright 2025 The go-yaml Project Contributors
// SPDX-License-Identifier: Apache-2.0 AND MIT
// Core libyaml types and structures.
// Defines Parser, Emitter, Event, Token, and related constants for YAML
// processing.
package libyaml
import (
"fmt"
"io"
"strings"
)
// VersionDirective holds the YAML version directive data.
type VersionDirective struct {
major int8 // The major version number.
minor int8 // The minor version number.
}
// Major returns the major version number.
func (v *VersionDirective) Major() int { return int(v.major) }
// Minor returns the minor version number.
func (v *VersionDirective) Minor() int { return int(v.minor) }
// TagDirective holds the YAML tag directive data.
type TagDirective struct {
handle []byte // The tag handle.
prefix []byte // The tag prefix.
}
// GetHandle returns the tag handle.
func (t *TagDirective) GetHandle() string { return string(t.handle) }
// GetPrefix returns the tag prefix.
func (t *TagDirective) GetPrefix() string { return string(t.prefix) }
type Encoding int
// The stream encoding.
const (
// Let the parser choose the encoding.
ANY_ENCODING Encoding = iota
UTF8_ENCODING // The default UTF-8 encoding.
UTF16LE_ENCODING // The UTF-16-LE encoding with BOM.
UTF16BE_ENCODING // The UTF-16-BE encoding with BOM.
)
type LineBreak int
// Line break types.
const (
// Let the parser choose the break type.
ANY_BREAK LineBreak = iota
CR_BREAK // Use CR for line breaks (Mac style).
LN_BREAK // Use LN for line breaks (Unix style).
CRLN_BREAK // Use CR LN for line breaks (DOS style).
)
type QuoteStyle int
// Quote style types for required quoting.
const (
QuoteSingle QuoteStyle = iota // Prefer single quotes when quoting is required.
QuoteDouble // Prefer double quotes when quoting is required.
QuoteLegacy // Legacy behavior: double in representer, single in emitter.
)
// ScalarStyle returns the scalar style for this quote preference in the
// representer/serializer context.
// In this context, both QuoteDouble and QuoteLegacy use double quotes.
func (q QuoteStyle) ScalarStyle() ScalarStyle {
if q == QuoteDouble || q == QuoteLegacy {
return DOUBLE_QUOTED_SCALAR_STYLE
}
return SINGLE_QUOTED_SCALAR_STYLE
}
type ErrorType int
// Many bad things could happen with the parser and emitter.
const (
// No error is produced.
NO_ERROR ErrorType = iota
MEMORY_ERROR // Cannot allocate or reallocate a block of memory.
READER_ERROR // Cannot read or decode the input stream.
SCANNER_ERROR // Cannot scan the input stream.
PARSER_ERROR // Cannot parse the input stream.
COMPOSER_ERROR // Cannot compose a YAML document.
WRITER_ERROR // Cannot write to the output stream.
EMITTER_ERROR // Cannot emit a YAML stream.
)
// Mark holds the pointer position.
type Mark struct {
Index int // The position index.
Line int // The position line (1-indexed).
Column int // The position column (0-indexed internally, displayed as 1-indexed).
}
func (m Mark) String() string {
var builder strings.Builder
if m.Line == 0 {
return "<unknown position>"
}
fmt.Fprintf(&builder, "line %d", m.Line)
if m.Column != 0 {
fmt.Fprintf(&builder, ", column %d", m.Column+1)
}
return builder.String()
}
// Node Styles
type styleInt int8
type ScalarStyle styleInt
// Scalar styles.
const (
// Let the emitter choose the style.
ANY_SCALAR_STYLE ScalarStyle = 0
PLAIN_SCALAR_STYLE ScalarStyle = 1 << iota // The plain scalar style.
SINGLE_QUOTED_SCALAR_STYLE // The single-quoted scalar style.
DOUBLE_QUOTED_SCALAR_STYLE // The double-quoted scalar style.
LITERAL_SCALAR_STYLE // The literal scalar style.
FOLDED_SCALAR_STYLE // The folded scalar style.
)
// String returns a string representation of a [ScalarStyle].
func (style ScalarStyle) String() string {
switch style {
case PLAIN_SCALAR_STYLE:
return "Plain"
case SINGLE_QUOTED_SCALAR_STYLE:
return "Single"
case DOUBLE_QUOTED_SCALAR_STYLE:
return "Double"
case LITERAL_SCALAR_STYLE:
return "Literal"
case FOLDED_SCALAR_STYLE:
return "Folded"
default:
return ""
}
}
type SequenceStyle styleInt
// Sequence styles.
const (
// Let the emitter choose the style.
ANY_SEQUENCE_STYLE SequenceStyle = iota
BLOCK_SEQUENCE_STYLE // The block sequence style.
FLOW_SEQUENCE_STYLE // The flow sequence style.
)
type MappingStyle styleInt
// Mapping styles.
const (
// Let the emitter choose the style.
ANY_MAPPING_STYLE MappingStyle = iota
BLOCK_MAPPING_STYLE // The block mapping style.
FLOW_MAPPING_STYLE // The flow mapping style.
)
// Tokens
type TokenType int
// Token types.
const (
// An empty token.
NO_TOKEN TokenType = iota
STREAM_START_TOKEN // A STREAM-START token.
STREAM_END_TOKEN // A STREAM-END token.
VERSION_DIRECTIVE_TOKEN // A VERSION-DIRECTIVE token.
TAG_DIRECTIVE_TOKEN // A TAG-DIRECTIVE token.
DOCUMENT_START_TOKEN // A DOCUMENT-START token.
DOCUMENT_END_TOKEN // A DOCUMENT-END token.
BLOCK_SEQUENCE_START_TOKEN // A BLOCK-SEQUENCE-START token.
BLOCK_MAPPING_START_TOKEN // A BLOCK-SEQUENCE-END token.
BLOCK_END_TOKEN // A BLOCK-END token.
FLOW_SEQUENCE_START_TOKEN // A FLOW-SEQUENCE-START token.
FLOW_SEQUENCE_END_TOKEN // A FLOW-SEQUENCE-END token.
FLOW_MAPPING_START_TOKEN // A FLOW-MAPPING-START token.
FLOW_MAPPING_END_TOKEN // A FLOW-MAPPING-END token.
BLOCK_ENTRY_TOKEN // A BLOCK-ENTRY token.
FLOW_ENTRY_TOKEN // A FLOW-ENTRY token.
KEY_TOKEN // A KEY token.
VALUE_TOKEN // A VALUE token.
ALIAS_TOKEN // An ALIAS token.
ANCHOR_TOKEN // An ANCHOR token.
TAG_TOKEN // A TAG token.
SCALAR_TOKEN // A SCALAR token.
COMMENT_TOKEN // A COMMENT token.
)
func (tt TokenType) String() string {
switch tt {
case NO_TOKEN:
return "NO_TOKEN"
case STREAM_START_TOKEN:
return "STREAM_START_TOKEN"
case STREAM_END_TOKEN:
return "STREAM_END_TOKEN"
case VERSION_DIRECTIVE_TOKEN:
return "VERSION_DIRECTIVE_TOKEN"
case TAG_DIRECTIVE_TOKEN:
return "TAG_DIRECTIVE_TOKEN"
case DOCUMENT_START_TOKEN:
return "DOCUMENT_START_TOKEN"
case DOCUMENT_END_TOKEN:
return "DOCUMENT_END_TOKEN"
case BLOCK_SEQUENCE_START_TOKEN:
return "BLOCK_SEQUENCE_START_TOKEN"
case BLOCK_MAPPING_START_TOKEN:
return "BLOCK_MAPPING_START_TOKEN"
case BLOCK_END_TOKEN:
return "BLOCK_END_TOKEN"
case FLOW_SEQUENCE_START_TOKEN:
return "FLOW_SEQUENCE_START_TOKEN"
case FLOW_SEQUENCE_END_TOKEN:
return "FLOW_SEQUENCE_END_TOKEN"
case FLOW_MAPPING_START_TOKEN:
return "FLOW_MAPPING_START_TOKEN"
case FLOW_MAPPING_END_TOKEN:
return "FLOW_MAPPING_END_TOKEN"
case BLOCK_ENTRY_TOKEN:
return "BLOCK_ENTRY_TOKEN"
case FLOW_ENTRY_TOKEN:
return "FLOW_ENTRY_TOKEN"
case KEY_TOKEN:
return "KEY_TOKEN"
case VALUE_TOKEN:
return "VALUE_TOKEN"
case ALIAS_TOKEN:
return "ALIAS_TOKEN"
case ANCHOR_TOKEN:
return "ANCHOR_TOKEN"
case TAG_TOKEN:
return "TAG_TOKEN"
case SCALAR_TOKEN:
return "SCALAR_TOKEN"
case COMMENT_TOKEN:
return "COMMENT_TOKEN"
}
return "<unknown token>"
}
// Token holds information about a scanning token.
type Token struct {
// The token type.
Type TokenType
// The start/end of the token.
StartMark, EndMark Mark
// The stream encoding (for STREAM_START_TOKEN).
encoding Encoding
// The alias/anchor/scalar Value or tag/tag directive handle
// (for ALIAS_TOKEN, ANCHOR_TOKEN, SCALAR_TOKEN, TAG_TOKEN, TAG_DIRECTIVE_TOKEN).
Value []byte
// The tag suffix (for TAG_TOKEN).
suffix []byte
// The tag directive prefix (for TAG_DIRECTIVE_TOKEN).
prefix []byte
// The scalar Style (for SCALAR_TOKEN).
Style ScalarStyle
// The version directive major/minor (for VERSION_DIRECTIVE_TOKEN).
major, minor int8
}
// Events
type EventType int8
// Event types.
const (
// An empty event.
NO_EVENT EventType = iota
STREAM_START_EVENT // A STREAM-START event.
STREAM_END_EVENT // A STREAM-END event.
DOCUMENT_START_EVENT // A DOCUMENT-START event.
DOCUMENT_END_EVENT // A DOCUMENT-END event.
ALIAS_EVENT // An ALIAS event.
SCALAR_EVENT // A SCALAR event.
SEQUENCE_START_EVENT // A SEQUENCE-START event.
SEQUENCE_END_EVENT // A SEQUENCE-END event.
MAPPING_START_EVENT // A MAPPING-START event.
MAPPING_END_EVENT // A MAPPING-END event.
TAIL_COMMENT_EVENT
)
var eventStrings = []string{
NO_EVENT: "none",
STREAM_START_EVENT: "stream start",
STREAM_END_EVENT: "stream end",
DOCUMENT_START_EVENT: "document start",
DOCUMENT_END_EVENT: "document end",
ALIAS_EVENT: "alias",
SCALAR_EVENT: "scalar",
SEQUENCE_START_EVENT: "sequence start",
SEQUENCE_END_EVENT: "sequence end",
MAPPING_START_EVENT: "mapping start",
MAPPING_END_EVENT: "mapping end",
TAIL_COMMENT_EVENT: "tail comment",
}
func (e EventType) String() string {
if e < 0 || int(e) >= len(eventStrings) {
return fmt.Sprintf("unknown event %d", e)
}
return eventStrings[e]
}
// Event holds information about a parsing or emitting event.
type Event struct {
// The event type.
Type EventType
// The start and end of the event.
StartMark, EndMark Mark
// The document encoding (for STREAM_START_EVENT).
encoding Encoding
// The version directive (for DOCUMENT_START_EVENT).
versionDirective *VersionDirective
// The list of tag directives (for DOCUMENT_START_EVENT).
tagDirectives []TagDirective
// The comments
HeadComment []byte
LineComment []byte
FootComment []byte
TailComment []byte
// The Anchor (for SCALAR_EVENT, SEQUENCE_START_EVENT, MAPPING_START_EVENT, ALIAS_EVENT).
Anchor []byte
// The Tag (for SCALAR_EVENT, SEQUENCE_START_EVENT, MAPPING_START_EVENT).
Tag []byte
// The scalar Value (for SCALAR_EVENT).
Value []byte
// Is the document start/end indicator Implicit, or the tag optional?
// (for DOCUMENT_START_EVENT, DOCUMENT_END_EVENT, SEQUENCE_START_EVENT, MAPPING_START_EVENT, SCALAR_EVENT).
Implicit bool
// Is the tag optional for any non-plain style? (for SCALAR_EVENT).
quoted_implicit bool
// The Style (for SCALAR_EVENT, SEQUENCE_START_EVENT, MAPPING_START_EVENT).
Style Style
}
func (e *Event) ScalarStyle() ScalarStyle { return ScalarStyle(e.Style) }
func (e *Event) SequenceStyle() SequenceStyle { return SequenceStyle(e.Style) }
func (e *Event) MappingStyle() MappingStyle { return MappingStyle(e.Style) }
// GetEncoding returns the stream encoding (for STREAM_START_EVENT).
func (e *Event) GetEncoding() Encoding { return e.encoding }
// GetVersionDirective returns the version directive (for DOCUMENT_START_EVENT).
func (e *Event) GetVersionDirective() *VersionDirective { return e.versionDirective }
// GetTagDirectives returns the tag directives (for DOCUMENT_START_EVENT).
func (e *Event) GetTagDirectives() []TagDirective { return e.tagDirectives }
// Nodes
const (
NULL_TAG = "tag:yaml.org,2002:null" // The tag !!null with the only possible value: null.
BOOL_TAG = "tag:yaml.org,2002:bool" // The tag !!bool with the values: true and false.
STR_TAG = "tag:yaml.org,2002:str" // The tag !!str for string values.
INT_TAG = "tag:yaml.org,2002:int" // The tag !!int for integer values.
FLOAT_TAG = "tag:yaml.org,2002:float" // The tag !!float for float values.
TIMESTAMP_TAG = "tag:yaml.org,2002:timestamp" // The tag !!timestamp for date and time values.
SEQ_TAG = "tag:yaml.org,2002:seq" // The tag !!seq is used to denote sequences.
MAP_TAG = "tag:yaml.org,2002:map" // The tag !!map is used to denote mapping.
// Not in original libyaml.
BINARY_TAG = "tag:yaml.org,2002:binary"
MERGE_TAG = "tag:yaml.org,2002:merge"
DEFAULT_SCALAR_TAG = STR_TAG // The default scalar tag is !!str.
DEFAULT_SEQUENCE_TAG = SEQ_TAG // The default sequence tag is !!seq.
DEFAULT_MAPPING_TAG = MAP_TAG // The default mapping tag is !!map.
)
type NodeType int
// Node types.
const (
// An empty node.
NO_NODE NodeType = iota
SCALAR_NODE // A scalar node.
SEQUENCE_NODE // A sequence node.
MAPPING_NODE // A mapping node.
)
// NodeItem represents an element of a sequence node.
type NodeItem int
// NodePair represents an element of a mapping node.
type NodePair struct {
key int // The key of the element.
value int // The value of the element.
}
// parserNode represents a single node in the YAML document tree.
type parserNode struct {
typ NodeType // The node type.
tag []byte // The node tag.
// The node data.
// The scalar parameters (for SCALAR_NODE).
scalar struct {
value []byte // The scalar value.
length int // The length of the scalar value.
style ScalarStyle // The scalar style.
}
// The sequence parameters (for YAML_SEQUENCE_NODE).
sequence struct {
items_data []NodeItem // The stack of sequence items.
style SequenceStyle // The sequence style.
}
// The mapping parameters (for MAPPING_NODE).
mapping struct {
pairs_data []NodePair // The stack of mapping pairs (key, value).
pairs_start *NodePair // The beginning of the stack.
pairs_end *NodePair // The end of the stack.
pairs_top *NodePair // The top of the stack.
style MappingStyle // The mapping style.
}
start_mark Mark // The beginning of the node.
end_mark Mark // The end of the node.
}
// Document structure.
type Document struct {
// The document nodes.
nodes []parserNode
// The version directive.
version_directive *VersionDirective
// The list of tag directives.
tag_directives_data []TagDirective
tag_directives_start int // The beginning of the tag directives list.
tag_directives_end int // The end of the tag directives list.
start_implicit int // Is the document start indicator implicit?
end_implicit int // Is the document end indicator implicit?
// The start/end of the document.
start_mark, end_mark Mark
}
// ReadHandler is called when the [Parser] needs to read more bytes from the
// source. The handler should write not more than size bytes to the buffer.
// The number of written bytes should be set to the size_read variable.
//
// [in,out] data A pointer to an application data specified by
//
// yamlParser.setInput().
//
// [out] buffer The buffer to write the data from the source.
// [in] size The size of the buffer.
// [out] size_read The actual number of bytes read from the source.
//
// On success, the handler should return 1. If the handler failed,
// the returned value should be 0. On EOF, the handler should set the
// size_read to 0 and return 1.
type ReadHandler func(parser *Parser, buffer []byte) (n int, err error)
// SimpleKey holds information about a potential simple key.
type SimpleKey struct {
flow_level int // What flow level is the key at?
required bool // Is a simple key required?
token_number int // The number of the token.
mark Mark // The position mark.
}
// ParserState represents the state of the parser.
type ParserState int
const (
PARSE_STREAM_START_STATE ParserState = iota
PARSE_IMPLICIT_DOCUMENT_START_STATE // Expect the beginning of an implicit document.
PARSE_DOCUMENT_START_STATE // Expect DOCUMENT-START.
PARSE_DOCUMENT_CONTENT_STATE // Expect the content of a document.
PARSE_DOCUMENT_END_STATE // Expect DOCUMENT-END.
PARSE_BLOCK_NODE_STATE // Expect a block node.
PARSE_BLOCK_SEQUENCE_FIRST_ENTRY_STATE // Expect the first entry of a block sequence.
PARSE_BLOCK_SEQUENCE_ENTRY_STATE // Expect an entry of a block sequence.
PARSE_INDENTLESS_SEQUENCE_ENTRY_STATE // Expect an entry of an indentless sequence.
PARSE_BLOCK_MAPPING_FIRST_KEY_STATE // Expect the first key of a block mapping.
PARSE_BLOCK_MAPPING_KEY_STATE // Expect a block mapping key.
PARSE_BLOCK_MAPPING_VALUE_STATE // Expect a block mapping value.
PARSE_FLOW_SEQUENCE_FIRST_ENTRY_STATE // Expect the first entry of a flow sequence.
PARSE_FLOW_SEQUENCE_ENTRY_STATE // Expect an entry of a flow sequence.
PARSE_FLOW_SEQUENCE_ENTRY_MAPPING_KEY_STATE // Expect a key of an ordered mapping.
PARSE_FLOW_SEQUENCE_ENTRY_MAPPING_VALUE_STATE // Expect a value of an ordered mapping.
PARSE_FLOW_SEQUENCE_ENTRY_MAPPING_END_STATE // Expect the and of an ordered mapping entry.
PARSE_FLOW_MAPPING_FIRST_KEY_STATE // Expect the first key of a flow mapping.
PARSE_FLOW_MAPPING_KEY_STATE // Expect a key of a flow mapping.
PARSE_FLOW_MAPPING_VALUE_STATE // Expect a value of a flow mapping.
PARSE_FLOW_MAPPING_EMPTY_VALUE_STATE // Expect an empty value of a flow mapping.
PARSE_END_STATE // Expect nothing.
)
func (ps ParserState) String() string {
switch ps {
case PARSE_STREAM_START_STATE:
return "PARSE_STREAM_START_STATE"
case PARSE_IMPLICIT_DOCUMENT_START_STATE:
return "PARSE_IMPLICIT_DOCUMENT_START_STATE"
case PARSE_DOCUMENT_START_STATE:
return "PARSE_DOCUMENT_START_STATE"
case PARSE_DOCUMENT_CONTENT_STATE:
return "PARSE_DOCUMENT_CONTENT_STATE"
case PARSE_DOCUMENT_END_STATE:
return "PARSE_DOCUMENT_END_STATE"
case PARSE_BLOCK_NODE_STATE:
return "PARSE_BLOCK_NODE_STATE"
case PARSE_BLOCK_SEQUENCE_FIRST_ENTRY_STATE:
return "PARSE_BLOCK_SEQUENCE_FIRST_ENTRY_STATE"
case PARSE_BLOCK_SEQUENCE_ENTRY_STATE:
return "PARSE_BLOCK_SEQUENCE_ENTRY_STATE"
case PARSE_INDENTLESS_SEQUENCE_ENTRY_STATE:
return "PARSE_INDENTLESS_SEQUENCE_ENTRY_STATE"
case PARSE_BLOCK_MAPPING_FIRST_KEY_STATE:
return "PARSE_BLOCK_MAPPING_FIRST_KEY_STATE"
case PARSE_BLOCK_MAPPING_KEY_STATE:
return "PARSE_BLOCK_MAPPING_KEY_STATE"
case PARSE_BLOCK_MAPPING_VALUE_STATE:
return "PARSE_BLOCK_MAPPING_VALUE_STATE"
case PARSE_FLOW_SEQUENCE_FIRST_ENTRY_STATE:
return "PARSE_FLOW_SEQUENCE_FIRST_ENTRY_STATE"
case PARSE_FLOW_SEQUENCE_ENTRY_STATE:
return "PARSE_FLOW_SEQUENCE_ENTRY_STATE"
case PARSE_FLOW_SEQUENCE_ENTRY_MAPPING_KEY_STATE:
return "PARSE_FLOW_SEQUENCE_ENTRY_MAPPING_KEY_STATE"
case PARSE_FLOW_SEQUENCE_ENTRY_MAPPING_VALUE_STATE:
return "PARSE_FLOW_SEQUENCE_ENTRY_MAPPING_VALUE_STATE"
case PARSE_FLOW_SEQUENCE_ENTRY_MAPPING_END_STATE:
return "PARSE_FLOW_SEQUENCE_ENTRY_MAPPING_END_STATE"
case PARSE_FLOW_MAPPING_FIRST_KEY_STATE:
return "PARSE_FLOW_MAPPING_FIRST_KEY_STATE"
case PARSE_FLOW_MAPPING_KEY_STATE:
return "PARSE_FLOW_MAPPING_KEY_STATE"
case PARSE_FLOW_MAPPING_VALUE_STATE:
return "PARSE_FLOW_MAPPING_VALUE_STATE"
case PARSE_FLOW_MAPPING_EMPTY_VALUE_STATE:
return "PARSE_FLOW_MAPPING_EMPTY_VALUE_STATE"
case PARSE_END_STATE:
return "PARSE_END_STATE"
}
return "<unknown parser state>"
}
// AliasData holds information about aliases.
type AliasData struct {
anchor []byte // The anchor.
index int // The node id.
mark Mark // The anchor mark.
}
// Parser structure holds all information about the current
// state of the parser.
type Parser struct {
lastError error
// Reader stuff
read_handler ReadHandler // Read handler.
input_reader io.Reader // File input data.
input []byte // String input data.
input_pos int
eof bool // EOF flag
buffer []byte // The working buffer.
buffer_pos int // The current position of the buffer.
unread int // The number of unread characters in the buffer.
newlines int // The number of line breaks since last non-break/non-blank character
raw_buffer []byte // The raw buffer.
raw_buffer_pos int // The current position of the buffer.
encoding Encoding // The input encoding.
offset int // The offset of the current position (in bytes).
mark Mark // The mark of the current position.
// Comments
HeadComment []byte // The current head comments
LineComment []byte // The current line comments
FootComment []byte // The current foot comments
tail_comment []byte // Foot comment that happens at the end of a block.
stem_comment []byte // Comment in item preceding a nested structure (list inside list item, etc)
comments []Comment // The folded comments for all parsed tokens
comments_head int
// Scanner stuff
stream_start_produced bool // Have we started to scan the input stream?
stream_end_produced bool // Have we reached the end of the input stream?
flow_level int // The number of unclosed '[' and '{' indicators.
tokens []Token // The tokens queue.
tokens_head int // The head of the tokens queue.
tokens_parsed int // The number of tokens fetched from the queue.
token_available bool // Does the tokens queue contain a token ready for dequeueing.
indent int // The current indentation level.
indents []int // The indentation levels stack.
simple_key_allowed bool // May a simple key occur at the current position?
simple_key_possible bool // Is the current simple key possible?
simple_key SimpleKey // The current simple key.
simple_key_stack []SimpleKey // The stack of simple keys.
// Parser stuff
state ParserState // The current parser state.
states []ParserState // The parser states stack.
marks []Mark // The stack of marks.
tag_directives []TagDirective // The list of TAG directives.
// Representer stuff
aliases []AliasData // The alias data.
document *Document // The currently parsed document.
}
type Comment struct {
ScanMark Mark // Position where scanning for comments started
TokenMark Mark // Position after which tokens will be associated with this comment
StartMark Mark // Position of '#' comment mark
EndMark Mark // Position where comment terminated
Head []byte
Line []byte
Foot []byte
}
// Emitter Definitions
// WriteHandler is called when the [Emitter] needs to flush the accumulated
// characters to the output. The handler should write @a size bytes of the
// @a buffer to the output.
//
// @param[in,out] data A pointer to an application data specified by
//
// yamlEmitter.setOutput().
//
// @param[in] buffer The buffer with bytes to be written.
// @param[in] size The size of the buffer.
//
// @returns On success, the handler should return @c 1. If the handler failed,
// the returned value should be @c 0.
type WriteHandler func(emitter *Emitter, buffer []byte) error
type EmitterState int
// The emitter states.
const (
// Expect STREAM-START.
EMIT_STREAM_START_STATE EmitterState = iota
EMIT_FIRST_DOCUMENT_START_STATE // Expect the first DOCUMENT-START or STREAM-END.
EMIT_DOCUMENT_START_STATE // Expect DOCUMENT-START or STREAM-END.
EMIT_DOCUMENT_CONTENT_STATE // Expect the content of a document.
EMIT_DOCUMENT_END_STATE // Expect DOCUMENT-END.
EMIT_FLOW_SEQUENCE_FIRST_ITEM_STATE // Expect the first item of a flow sequence.
EMIT_FLOW_SEQUENCE_TRAIL_ITEM_STATE // Expect the next item of a flow sequence, with the comma already written out
EMIT_FLOW_SEQUENCE_ITEM_STATE // Expect an item of a flow sequence.
EMIT_FLOW_MAPPING_FIRST_KEY_STATE // Expect the first key of a flow mapping.
EMIT_FLOW_MAPPING_TRAIL_KEY_STATE // Expect the next key of a flow mapping, with the comma already written out
EMIT_FLOW_MAPPING_KEY_STATE // Expect a key of a flow mapping.
EMIT_FLOW_MAPPING_SIMPLE_VALUE_STATE // Expect a value for a simple key of a flow mapping.
EMIT_FLOW_MAPPING_VALUE_STATE // Expect a value of a flow mapping.
EMIT_BLOCK_SEQUENCE_FIRST_ITEM_STATE // Expect the first item of a block sequence.
EMIT_BLOCK_SEQUENCE_ITEM_STATE // Expect an item of a block sequence.
EMIT_BLOCK_MAPPING_FIRST_KEY_STATE // Expect the first key of a block mapping.
EMIT_BLOCK_MAPPING_KEY_STATE // Expect the key of a block mapping.
EMIT_BLOCK_MAPPING_SIMPLE_VALUE_STATE // Expect a value for a simple key of a block mapping.
EMIT_BLOCK_MAPPING_VALUE_STATE // Expect a value of a block mapping.
EMIT_END_STATE // Expect nothing.
)
// Emitter holds all information about the current state of the emitter.
type Emitter struct {
// Writer stuff
write_handler WriteHandler // Write handler.
output_buffer *[]byte // String output data.
output_writer io.Writer // File output data.
buffer []byte // The working buffer.
buffer_pos int // The current position of the buffer.
encoding Encoding // The stream encoding.
// Emitter stuff
canonical bool // If the output is in the canonical style?
BestIndent int // The number of indentation spaces.
best_width int // The preferred width of the output lines.
unicode bool // Allow unescaped non-ASCII characters?
line_break LineBreak // The preferred line break.
quotePreference QuoteStyle // Preferred quote style when quoting is required.
state EmitterState // The current emitter state.
states []EmitterState // The stack of states.
events []Event // The event queue.
events_head int // The head of the event queue.
indents []int // The stack of indentation levels.
tag_directives []TagDirective // The list of tag directives.
indent int // The current indentation level.
CompactSequenceIndent bool // Is '- ' is considered part of the indentation for sequence elements?
flow_level int // The current flow level.
root_context bool // Is it the document root context?
sequence_context bool // Is it a sequence context?
mapping_context bool // Is it a mapping context?
simple_key_context bool // Is it a simple mapping key context?
line int // The current line.
column int // The current column.
whitespace bool // If the last character was a whitespace?
indention bool // If the last character was an indentation character (' ', '-', '?', ':')?
OpenEnded bool // If an explicit document end is required?
space_above bool // Is there's an empty line above?
foot_indent int // The indent used to write the foot comment above, or -1 if none.
// Anchor analysis.
anchor_data struct {
anchor []byte // The anchor value.
alias bool // Is it an alias?
}
// Tag analysis.
tag_data struct {
handle []byte // The tag handle.
suffix []byte // The tag suffix.
}
// Scalar analysis.
scalar_data struct {
value []byte // The scalar value.
multiline bool // Does the scalar contain line breaks?
flow_plain_allowed bool // Can the scalar be expressed in the flow plain style?
block_plain_allowed bool // Can the scalar be expressed in the block plain style?
single_quoted_allowed bool // Can the scalar be expressed in the single quoted style?
block_allowed bool // Can the scalar be expressed in the literal or folded styles?
style ScalarStyle // The output style.
}
// Comments
HeadComment []byte
LineComment []byte
FootComment []byte
TailComment []byte
key_line_comment []byte
// Representer stuff
opened bool // If the stream was already opened?
closed bool // If the stream was already closed?
// The information associated with the document nodes.
anchors *struct {
references int // The number of references.
anchor int // The anchor id.
serialized bool // If the node has been emitted?
}
last_anchor_id int // The last assigned anchor id.
document *Document // The currently emitted document.
}
+192
View File
@@ -0,0 +1,192 @@
// Copyright 2025 The go-yaml Project Contributors
// SPDX-License-Identifier: Apache-2.0
// YAML test data loading utilities.
// Provides helper functions for loading and processing YAML test data,
// including scalar coercion.
package libyaml
import (
"errors"
"fmt"
"io"
"strings"
)
// coerceScalar converts a YAML scalar string to an appropriate Go type
func coerceScalar(value string) any {
// Try bool and null
switch value {
case "true":
return true
case "false":
return false
case "null":
return nil
}
// Try hex int (0x or 0X prefix) - needed for test data byte arrays
var intVal int
if _, err := fmt.Sscanf(strings.ToLower(value), "0x%x", &intVal); err == nil {
return intVal
}
// Try float (must check before int because %d will parse "1.5" as "1")
if strings.Contains(value, ".") {
var floatVal float64
if _, err := fmt.Sscanf(value, "%f", &floatVal); err == nil {
return floatVal
}
}
// Try decimal int - use int64 to handle large values on 32-bit systems
var int64Val int64
if _, err := fmt.Sscanf(value, "%d", &int64Val); err == nil {
// Return as int if it fits, otherwise int64
if int64Val == int64(int(int64Val)) {
return int(int64Val)
}
return int64Val
}
// Default to string
return value
}
// LoadYAML parses YAML data using the native libyaml Parser.
// This function is exported so it can be used by other packages for data-driven testing.
// It returns a generic interface{} which is typically:
// - map[string]interface{} for YAML mappings
// - []interface{} for YAML sequences
// - scalar values, resolved according to the following rules:
// - Booleans: "true" and "false" are returned as bool (true/false).
// - Nulls: "null" is returned as nil.
// - Floats: values containing "." are parsed as float64.
// - Decimal integers: values matching integer format are parsed as int.
// - All other values are returned as string.
//
// This scalar resolution behavior matches the implementation in coerceScalar.
func LoadYAML(data []byte) (any, error) {
parser := NewParser()
parser.SetInputString(data)
defer parser.Delete()
type stackEntry struct {
container any // map[string]interface{} or []interface{}
key string // for maps: current key waiting for value
}
var stack []stackEntry
var root any
for {
var event Event
if err := parser.Parse(&event); err != nil {
if errors.Is(err, io.EOF) {
break
}
return nil, err
}
switch event.Type {
case STREAM_END_EVENT:
// End of stream, we're done
return root, nil
case STREAM_START_EVENT, DOCUMENT_START_EVENT:
// Structural markers, no action needed
case MAPPING_START_EVENT:
newMap := make(map[string]any)
stack = append(stack, stackEntry{container: newMap})
case MAPPING_END_EVENT:
if len(stack) > 0 {
popped := stack[len(stack)-1]
stack = stack[:len(stack)-1]
// Add completed map to parent or set as root
if len(stack) == 0 {
root = popped.container
} else {
parent := &stack[len(stack)-1]
if m, ok := parent.container.(map[string]any); ok {
m[parent.key] = popped.container
parent.key = "" // Reset key after use
} else if s, ok := parent.container.([]any); ok {
parent.container = append(s, popped.container)
}
}
}
case SEQUENCE_START_EVENT:
newSlice := make([]any, 0)
stack = append(stack, stackEntry{container: newSlice})
case SEQUENCE_END_EVENT:
if len(stack) > 0 {
popped := stack[len(stack)-1]
stack = stack[:len(stack)-1]
// Add completed slice to parent or set as root
if len(stack) == 0 {
root = popped.container
} else {
parent := &stack[len(stack)-1]
if m, ok := parent.container.(map[string]any); ok {
m[parent.key] = popped.container
parent.key = "" // Reset key after use
} else if s, ok := parent.container.([]any); ok {
parent.container = append(s, popped.container)
}
}
}
case SCALAR_EVENT:
value := string(event.Value)
// Only coerce plain (unquoted) scalars
isQuoted := ScalarStyle(event.Style) != PLAIN_SCALAR_STYLE
if len(stack) == 0 {
// Scalar at root level
if isQuoted {
root = value
} else {
root = coerceScalar(value)
}
} else {
parent := &stack[len(stack)-1]
if m, ok := parent.container.(map[string]any); ok {
if parent.key == "" {
// This scalar is a key - keep as string, don't coerce
parent.key = value
} else {
// This scalar is a value
if isQuoted {
m[parent.key] = value
} else {
m[parent.key] = coerceScalar(value)
}
parent.key = ""
}
} else if s, ok := parent.container.([]any); ok {
// Add to sequence
if isQuoted {
parent.container = append(s, value)
} else {
parent.container = append(s, coerceScalar(value))
}
}
}
case DOCUMENT_END_EVENT:
// Document end marker, continue processing
case ALIAS_EVENT, TAIL_COMMENT_EVENT:
// For now, skip aliases and comments (not used in test data)
}
}
return root, nil
}
+249
View File
@@ -0,0 +1,249 @@
// Copyright 2006-2010 Kirill Simonov
// Copyright 2011-2019 Canonical Ltd
// Copyright 2025 The go-yaml Project Contributors
// SPDX-License-Identifier: Apache-2.0 AND MIT
// Internal constants and buffer sizes.
// Defines buffer sizes, stack sizes, and other internal configuration
// constants for libyaml.
package libyaml
const (
// The size of the input raw buffer.
input_raw_buffer_size = 512
// The size of the input buffer.
// It should be possible to decode the whole raw buffer.
input_buffer_size = input_raw_buffer_size * 3
// The size of the output buffer.
output_buffer_size = 128
// The size of other stacks and queues.
initial_stack_size = 16
initial_queue_size = 16
initial_string_size = 16
)
// Check if the character at the specified position is an alphabetical
// character, a digit, '_', or '-'.
func isAlpha(b []byte, i int) bool {
return b[i] >= '0' && b[i] <= '9' || b[i] >= 'A' && b[i] <= 'Z' ||
b[i] >= 'a' && b[i] <= 'z' || b[i] == '_' || b[i] == '-'
}
// Check if the character at the specified position is a flow indicator as
// defined by spec production [23] c-flow-indicator ::=
// c-collect-entry | c-sequence-start | c-sequence-end |
// c-mapping-start | c-mapping-end
func isFlowIndicator(b []byte, i int) bool {
return b[i] == '[' || b[i] == ']' ||
b[i] == '{' || b[i] == '}' || b[i] == ','
}
// Check if the character at the specified position is valid for anchor names
// as defined by spec production [102] ns-anchor-char ::= ns-char -
// c-flow-indicator.
// This includes all printable characters except: CR, LF, BOM, space, tab, '[',
// ']', '{', '}', ','.
// We further limit it to ascii chars only, which is a subset of the spec
// production but is usually what most people expect.
func isAnchorChar(b []byte, i int) bool {
if isColon(b, i) {
// [Go] we exclude colons from anchor/alias names.
//
// A colon is a valid anchor character according to the YAML 1.2 specification,
// but it can lead to ambiguity.
// https://github.com/yaml/go-yaml/issues/109
//
// Also, it would have been a breaking change to support it, as go.yaml.in/yaml/v3 ignores it.
// Supporting it could lead to unexpected behavior.
return false
}
return isPrintable(b, i) &&
!isLineBreak(b, i) &&
!isBlank(b, i) &&
!isBOM(b, i) &&
!isFlowIndicator(b, i) &&
isASCII(b, i)
}
// isColon checks whether the character at the specified position is a colon.
func isColon(b []byte, i int) bool {
return b[i] == ':'
}
// Check if the character at the specified position is valid in a tag URI.
//
// The set of valid characters is:
//
// '0'-'9', 'A'-'Z', 'a'-'z', '_', '-', ';', '/', '?', ':', '@', '&',
// '=', '+', '$', '.', '!', '~', '*', '\'', '(', ')', '%'.
//
// If verbatim is true, flow indicators (',', '[', ']', '{', '}') are also
// allowed.
func isTagURIChar(b []byte, i int, verbatim bool) bool {
c := b[i]
// isAlpha covers: 0-9, A-Z, a-z, _, -
if isAlpha(b, i) {
return true
}
// Check special URI characters
switch c {
case ';', '/', '?', ':', '@', '&', '=', '+', '$', '.', '!', '~', '*', '\'', '(', ')', '%':
return true
case ',', '[', ']', '{', '}':
return verbatim
}
return false
}
// Check if the character at the specified position is a digit.
func isDigit(b []byte, i int) bool {
return b[i] >= '0' && b[i] <= '9'
}
// Get the value of a digit.
func asDigit(b []byte, i int) int {
return int(b[i]) - '0'
}
// Check if the character at the specified position is a hex-digit.
func isHex(b []byte, i int) bool {
return b[i] >= '0' && b[i] <= '9' || b[i] >= 'A' && b[i] <= 'F' ||
b[i] >= 'a' && b[i] <= 'f'
}
// Get the value of a hex-digit.
func asHex(b []byte, i int) int {
bi := b[i]
if bi >= 'A' && bi <= 'F' {
return int(bi) - 'A' + 10
}
if bi >= 'a' && bi <= 'f' {
return int(bi) - 'a' + 10
}
return int(bi) - '0'
}
// Check if the character is ASCII.
func isASCII(b []byte, i int) bool {
return b[i] <= 0x7F
}
// Check if the character at the start of the buffer can be printed unescaped.
func isPrintable(b []byte, i int) bool {
return ((b[i] == 0x0A) || // . == #x0A
(b[i] >= 0x20 && b[i] <= 0x7E) || // #x20 <= . <= #x7E
(b[i] == 0xC2 && b[i+1] >= 0xA0) || // #0xA0 <= . <= #xD7FF
(b[i] > 0xC2 && b[i] < 0xED) ||
(b[i] == 0xED && b[i+1] < 0xA0) ||
(b[i] == 0xEE) ||
(b[i] == 0xEF && // #xE000 <= . <= #xFFFD
!(b[i+1] == 0xBB && b[i+2] == 0xBF) && // && . != #xFEFF
!(b[i+1] == 0xBF && (b[i+2] == 0xBE || b[i+2] == 0xBF))))
}
// Check if the character at the specified position is NUL.
func isZeroChar(b []byte, i int) bool {
return b[i] == 0x00
}
// Check if the beginning of the buffer is a BOM.
func isBOM(b []byte, i int) bool {
return b[0] == 0xEF && b[1] == 0xBB && b[2] == 0xBF
}
// Check if the character at the specified position is space.
func isSpace(b []byte, i int) bool {
return b[i] == ' '
}
// Check if the character at the specified position is tab.
func isTab(b []byte, i int) bool {
return b[i] == '\t'
}
// Check if the character at the specified position is blank (space or tab).
func isBlank(b []byte, i int) bool {
// return isSpace(b, i) || isTab(b, i)
return b[i] == ' ' || b[i] == '\t'
}
// Check if the character at the specified position is a line break.
func isLineBreak(b []byte, i int) bool {
return (b[i] == '\r' || // CR (#xD)
b[i] == '\n' || // LF (#xA)
b[i] == 0xC2 && b[i+1] == 0x85 || // NEL (#x85)
b[i] == 0xE2 && b[i+1] == 0x80 && b[i+2] == 0xA8 || // LS (#x2028)
b[i] == 0xE2 && b[i+1] == 0x80 && b[i+2] == 0xA9) // PS (#x2029)
}
func isCRLF(b []byte, i int) bool {
return b[i] == '\r' && b[i+1] == '\n'
}
// Check if the character is a line break or NUL.
func isBreakOrZero(b []byte, i int) bool {
// return isLineBreak(b, i) || isZeroChar(b, i)
return (
// isBreak:
b[i] == '\r' || // CR (#xD)
b[i] == '\n' || // LF (#xA)
b[i] == 0xC2 && b[i+1] == 0x85 || // NEL (#x85)
b[i] == 0xE2 && b[i+1] == 0x80 && b[i+2] == 0xA8 || // LS (#x2028)
b[i] == 0xE2 && b[i+1] == 0x80 && b[i+2] == 0xA9 || // PS (#x2029)
// isZeroChar:
b[i] == 0)
}
// Check if the character is a line break, space, or NUL.
func isSpaceOrZero(b []byte, i int) bool {
// return isSpace(b, i) || isBreakOrZero(b, i)
return (
// isSpace:
b[i] == ' ' ||
// isBreakOrZero:
b[i] == '\r' || // CR (#xD)
b[i] == '\n' || // LF (#xA)
b[i] == 0xC2 && b[i+1] == 0x85 || // NEL (#x85)
b[i] == 0xE2 && b[i+1] == 0x80 && b[i+2] == 0xA8 || // LS (#x2028)
b[i] == 0xE2 && b[i+1] == 0x80 && b[i+2] == 0xA9 || // PS (#x2029)
b[i] == 0)
}
// Check if the character is a line break, space, tab, or NUL.
func isBlankOrZero(b []byte, i int) bool {
// return isBlank(b, i) || isBreakOrZero(b, i)
return (
// isBlank:
b[i] == ' ' || b[i] == '\t' ||
// isBreakOrZero:
b[i] == '\r' || // CR (#xD)
b[i] == '\n' || // LF (#xA)
b[i] == 0xC2 && b[i+1] == 0x85 || // NEL (#x85)
b[i] == 0xE2 && b[i+1] == 0x80 && b[i+2] == 0xA8 || // LS (#x2028)
b[i] == 0xE2 && b[i+1] == 0x80 && b[i+2] == 0xA9 || // PS (#x2029)
b[i] == 0)
}
// Determine the width of the character.
func width(b byte) int {
// Don't replace these by a switch without first
// confirming that it is being inlined.
if b&0x80 == 0x00 {
return 1
}
if b&0xE0 == 0xC0 {
return 2
}
if b&0xF0 == 0xE0 {
return 3
}
if b&0xF8 == 0xF0 {
return 4
}
return 0
}
+231
View File
@@ -0,0 +1,231 @@
// Copyright 2025 The go-yaml Project Contributors
// SPDX-License-Identifier: Apache-2.0
// This file contains the Loader API for reading YAML documents.
//
// Primary functions:
// - Load: Decode YAML document(s) into a value (use WithAll for multi-doc)
// - NewLoader: Create a streaming loader from io.Reader
package yaml
import (
"bytes"
"errors"
"io"
"reflect"
"go.yaml.in/yaml/v4/internal/libyaml"
)
// Load decodes YAML document(s) with the given options.
//
// By default, Load requires exactly one document in the input.
// If zero documents are found, it returns an error.
// If multiple documents are found, it returns an error.
//
// Use WithAllDocuments() to load all documents into a slice:
//
// var configs []Config
// yaml.Load(multiDocYAML, &configs, yaml.WithAllDocuments())
//
// When WithAllDocuments is used, out must be a pointer to a slice.
// Each document is decoded into the slice element type.
// Zero documents results in an empty slice (no error).
//
// Maps and pointers (to a struct, string, int, etc) are accepted as out
// values. If an internal pointer within a struct is not initialized,
// the yaml package will initialize it if necessary. The out parameter
// must not be nil.
//
// The type of the decoded values should be compatible with the respective
// values in out. If one or more values cannot be decoded due to type
// mismatches, decoding continues partially until the end of the YAML
// content, and a *yaml.LoadErrors is returned with details for all
// missed values.
//
// Struct fields are only loaded if they are exported (have an upper case
// first letter), and are loaded using the field name lowercased as the
// default key. Custom keys may be defined via the "yaml" name in the field
// tag: the content preceding the first comma is used as the key, and the
// following comma-separated options control the loading and dumping behavior.
//
// For example:
//
// type T struct {
// F int `yaml:"a,omitempty"`
// B int
// }
// var t T
// yaml.Load([]byte("a: 1\nb: 2"), &t)
//
// See the documentation of Dump for the format of tags and a list of
// supported tag options.
func Load(in []byte, out any, opts ...Option) error {
o, err := libyaml.ApplyOptions(opts...)
if err != nil {
return err
}
if o.AllDocuments {
// Multi-document mode: out must be pointer to slice
return loadAll(in, out, o)
}
// Single-document mode: exactly one document required
return loadSingle(in, out, o)
}
// loadAll loads all documents into a slice
func loadAll(in []byte, out any, opts *libyaml.Options) error {
outVal := reflect.ValueOf(out)
if outVal.Kind() != reflect.Pointer || outVal.IsNil() {
return &LoadErrors{Errors: []*libyaml.ConstructError{{
Err: errors.New("yaml: WithAllDocuments requires a non-nil pointer to a slice"),
}}}
}
sliceVal := outVal.Elem()
if sliceVal.Kind() != reflect.Slice {
return &LoadErrors{Errors: []*libyaml.ConstructError{{
Err: errors.New("yaml: WithAllDocuments requires a pointer to a slice"),
}}}
}
// Create a new slice (clear existing content)
sliceVal.Set(reflect.MakeSlice(sliceVal.Type(), 0, 0))
l, err := NewLoader(bytes.NewReader(in), func(o *libyaml.Options) error {
*o = *opts // Copy options
return nil
})
if err != nil {
return err
}
elemType := sliceVal.Type().Elem()
for {
// Create new element of slice's element type
elemPtr := reflect.New(elemType)
err := l.Load(elemPtr.Interface())
if err == io.EOF {
break
}
if err != nil {
return err
}
// Append decoded element to slice
sliceVal.Set(reflect.Append(sliceVal, elemPtr.Elem()))
}
return nil
}
// loadSingle loads exactly one document (strict)
func loadSingle(in []byte, out any, opts *libyaml.Options) error {
l, err := NewLoader(bytes.NewReader(in), func(o *libyaml.Options) error {
*o = *opts // Copy options
return nil
})
if err != nil {
return err
}
// Load first document
err = l.Load(out)
if err == io.EOF {
return &LoadErrors{Errors: []*libyaml.ConstructError{{
Err: errors.New("yaml: no documents in stream"),
}}}
}
if err != nil {
return err
}
// Check for additional documents
var dummy any
err = l.Load(&dummy)
if err != io.EOF {
if err != nil {
// Some other error occurred
return err
}
// Successfully loaded a second document - this is an error in strict mode
return &LoadErrors{Errors: []*libyaml.ConstructError{{
Err: errors.New("yaml: expected single document, found multiple"),
}}}
}
return nil
}
// A Loader reads and decodes YAML values from an input stream with configurable
// options.
type Loader struct {
composer *libyaml.Composer
decoder *libyaml.Constructor
opts *libyaml.Options
docCount int
}
// NewLoader returns a new Loader that reads from r with the given options.
//
// The Loader introduces its own buffering and may read data from r beyond the
// YAML values requested.
func NewLoader(r io.Reader, opts ...Option) (*Loader, error) {
o, err := libyaml.ApplyOptions(opts...)
if err != nil {
return nil, err
}
c := libyaml.NewComposerFromReader(r)
c.SetStreamNodes(o.StreamNodes)
return &Loader{
composer: c,
decoder: libyaml.NewConstructor(o),
opts: o,
}, nil
}
// Load reads the next YAML-encoded document from its input and stores it
// in the value pointed to by v.
//
// Returns io.EOF when there are no more documents to read.
// If WithSingleDocument option was set and a document was already read,
// subsequent calls return io.EOF.
//
// Maps and pointers (to a struct, string, int, etc) are accepted as v
// values. If an internal pointer within a struct is not initialized,
// the yaml package will initialize it if necessary. The v parameter
// must not be nil.
//
// Struct fields are only loaded if they are exported (have an upper case
// first letter), and are loaded using the field name lowercased as the
// default key. Custom keys may be defined via the "yaml" name in the field
// tag: the content preceding the first comma is used as the key, and the
// following comma-separated options control the loading and dumping behavior.
//
// See the documentation of the package-level Load function for more details
// about YAML to Go conversion and tag options.
func (l *Loader) Load(v any) (err error) {
defer handleErr(&err)
if l.opts.SingleDocument && l.docCount > 0 {
return io.EOF
}
node := l.composer.Parse() // *libyaml.Node
if node == nil {
return io.EOF
}
l.docCount++
out := reflect.ValueOf(v)
if out.Kind() == reflect.Pointer && !out.IsNil() {
out = out.Elem()
}
l.decoder.Construct(node, out) // Pass libyaml.Node directly
if len(l.decoder.TypeErrors) > 0 {
typeErrors := l.decoder.TypeErrors
l.decoder.TypeErrors = nil
return &LoadErrors{Errors: typeErrors}
}
return nil
}
+732
View File
@@ -0,0 +1,732 @@
// Copyright 2011-2019 Canonical Ltd
// Copyright 2025 The go-yaml Project Contributors
// SPDX-License-Identifier: Apache-2.0
// Package yaml implements YAML support for the Go language.
//
// Source code and other details for the project are available at GitHub:
//
// https://github.com/yaml/go-yaml
//
// This file contains:
// - Version presets (V2, V3, V4)
// - Options API (WithIndent, WithKnownFields, etc.)
// - Type and constant re-exports from internal/libyaml
// - Helper functions for struct field handling
// - Classic APIs (Decoder, Encoder, Unmarshal, Marshal)
//
// For the main API, see:
// - loader.go: Load, Loader
// - dumper.go: Dump, Dumper
package yaml
import (
"errors"
"fmt"
"io"
"reflect"
"strings"
"sync"
"go.yaml.in/yaml/v4/internal/libyaml"
)
//-----------------------------------------------------------------------------
// Version presets
//-----------------------------------------------------------------------------
// Usage:
// yaml.Dump(&data, yaml.V3)
// yaml.Dump(&data, yaml.V3, yaml.WithIndent(2), yaml.WithCompactSeqIndent())
// V2 defaults:
var V2 = Options(
WithIndent(2),
WithCompactSeqIndent(false),
WithLineWidth(80),
WithUnicode(true),
WithUniqueKeys(true),
WithQuotePreference(QuoteLegacy),
)
// V3 defaults:
var V3 = Options(
WithIndent(4),
WithCompactSeqIndent(false),
WithLineWidth(80),
WithUnicode(true),
WithUniqueKeys(true),
WithQuotePreference(QuoteLegacy),
)
// V4 defaults:
var V4 = Options(
WithIndent(2),
WithCompactSeqIndent(true),
WithLineWidth(80),
WithUnicode(true),
WithUniqueKeys(true),
WithQuotePreference(QuoteSingle),
)
//-----------------------------------------------------------------------------
// Options
//-----------------------------------------------------------------------------
// Option allows configuring YAML loading and dumping operations.
// Re-exported from internal/libyaml.
type Option = libyaml.Option
var (
// WithIndent sets indentation spaces (2-9).
// See internal/libyaml.WithIndent.
WithIndent = libyaml.WithIndent
// WithCompactSeqIndent configures '- ' as part of indentation.
// See internal/libyaml.WithCompactSeqIndent.
WithCompactSeqIndent = libyaml.WithCompactSeqIndent
// WithKnownFields enables strict field checking during loading.
// See internal/libyaml.WithKnownFields.
WithKnownFields = libyaml.WithKnownFields
// WithSingleDocument only processes first document in stream.
// See internal/libyaml.WithSingleDocument.
WithSingleDocument = libyaml.WithSingleDocument
// WithStreamNodes enables stream boundary nodes when loading.
// See internal/libyaml.WithStreamNodes.
WithStreamNodes = libyaml.WithStreamNodes
// WithAllDocuments enables multi-document mode for Load and Dump.
// See internal/libyaml.WithAllDocuments.
WithAllDocuments = libyaml.WithAllDocuments
// WithLineWidth sets preferred line width for output.
// See internal/libyaml.WithLineWidth.
WithLineWidth = libyaml.WithLineWidth
// WithUnicode controls non-ASCII characters in output.
// See internal/libyaml.WithUnicode.
WithUnicode = libyaml.WithUnicode
// WithUniqueKeys enables duplicate key detection.
// See internal/libyaml.WithUniqueKeys.
WithUniqueKeys = libyaml.WithUniqueKeys
// WithCanonical forces canonical YAML output format.
// See internal/libyaml.WithCanonical.
WithCanonical = libyaml.WithCanonical
// WithLineBreak sets line ending style for output.
// See internal/libyaml.WithLineBreak.
WithLineBreak = libyaml.WithLineBreak
// WithExplicitStart controls document start markers (---).
// See internal/libyaml.WithExplicitStart.
WithExplicitStart = libyaml.WithExplicitStart
// WithExplicitEnd controls document end markers (...).
// See internal/libyaml.WithExplicitEnd.
WithExplicitEnd = libyaml.WithExplicitEnd
// WithFlowSimpleCollections controls flow style for simple collections.
// See internal/libyaml.WithFlowSimpleCollections.
WithFlowSimpleCollections = libyaml.WithFlowSimpleCollections
// WithQuotePreference sets preferred quote style when quoting is required.
// See internal/libyaml.WithQuotePreference.
WithQuotePreference = libyaml.WithQuotePreference
)
// Options combines multiple options into a single Option.
// This is useful for creating option presets or combining version defaults
// with custom options.
//
// Example:
//
// opts := yaml.Options(yaml.V4, yaml.WithIndent(3))
// yaml.Dump(&data, opts)
func Options(opts ...Option) Option {
return libyaml.CombineOptions(opts...)
}
// OptsYAML parses a YAML string containing option settings and returns
// an Option that can be combined with other options using Options().
//
// The YAML string can specify any of these fields:
// - indent (int)
// - compact-seq-indent (bool)
// - line-width (int)
// - unicode (bool)
// - canonical (bool)
// - line-break (string: ln, cr, crln)
// - explicit-start (bool)
// - explicit-end (bool)
// - flow-simple-coll (bool)
// - known-fields (bool)
// - single-document (bool)
// - unique-keys (bool)
//
// Only fields specified in the YAML will override other options when
// combined. Unspecified fields won't affect other options.
//
// Example:
//
// opts, err := yaml.OptsYAML(`
// indent: 3
// known-fields: true
// `)
// yaml.Dump(&data, yaml.Options(V4, opts))
func OptsYAML(yamlStr string) (Option, error) {
var cfg struct {
Indent *int `yaml:"indent"`
CompactSeqIndent *bool `yaml:"compact-seq-indent"`
LineWidth *int `yaml:"line-width"`
Unicode *bool `yaml:"unicode"`
Canonical *bool `yaml:"canonical"`
LineBreak *string `yaml:"line-break"`
ExplicitStart *bool `yaml:"explicit-start"`
ExplicitEnd *bool `yaml:"explicit-end"`
FlowSimpleCollections *bool `yaml:"flow-simple-coll"`
KnownFields *bool `yaml:"known-fields"`
SingleDocument *bool `yaml:"single-document"`
UniqueKeys *bool `yaml:"unique-keys"`
}
if err := Load([]byte(yamlStr), &cfg, WithKnownFields()); err != nil {
return nil, err
}
// Build options only for fields that were set
var optList []Option
if cfg.Indent != nil {
optList = append(optList, WithIndent(*cfg.Indent))
}
if cfg.CompactSeqIndent != nil {
optList = append(optList, WithCompactSeqIndent(*cfg.CompactSeqIndent))
}
if cfg.LineWidth != nil {
optList = append(optList, WithLineWidth(*cfg.LineWidth))
}
if cfg.Unicode != nil {
optList = append(optList, WithUnicode(*cfg.Unicode))
}
if cfg.ExplicitStart != nil {
optList = append(optList, WithExplicitStart(*cfg.ExplicitStart))
}
if cfg.ExplicitEnd != nil {
optList = append(optList, WithExplicitEnd(*cfg.ExplicitEnd))
}
if cfg.FlowSimpleCollections != nil {
optList = append(optList, WithFlowSimpleCollections(*cfg.FlowSimpleCollections))
}
if cfg.KnownFields != nil {
optList = append(optList, WithKnownFields(*cfg.KnownFields))
}
if cfg.SingleDocument != nil && *cfg.SingleDocument {
optList = append(optList, WithSingleDocument())
}
if cfg.UniqueKeys != nil {
optList = append(optList, WithUniqueKeys(*cfg.UniqueKeys))
}
if cfg.Canonical != nil {
optList = append(optList, WithCanonical(*cfg.Canonical))
}
if cfg.LineBreak != nil {
switch *cfg.LineBreak {
case "ln":
optList = append(optList, WithLineBreak(LineBreakLN))
case "cr":
optList = append(optList, WithLineBreak(LineBreakCR))
case "crln":
optList = append(optList, WithLineBreak(LineBreakCRLN))
default:
return nil, errors.New("yaml: invalid line-break value (use ln, cr, or crln)")
}
}
return Options(optList...), nil
}
//-----------------------------------------------------------------------------
// Type and constant re-exports
//-----------------------------------------------------------------------------
type (
// Node represents a YAML node in the document tree.
// See internal/libyaml.Node.
Node = libyaml.Node
// Kind identifies the type of a YAML node.
// See internal/libyaml.Kind.
Kind = libyaml.Kind
// Style controls the presentation of a YAML node.
// See internal/libyaml.Style.
Style = libyaml.Style
// Marshaler is implemented by types with custom YAML marshaling.
// See internal/libyaml.Marshaler.
Marshaler = libyaml.Marshaler
// IsZeroer is implemented by types that can report if they're zero.
// See internal/libyaml.IsZeroer.
IsZeroer = libyaml.IsZeroer
)
// Unmarshaler is the interface implemented by types
// that can unmarshal a YAML description of themselves.
type Unmarshaler interface {
UnmarshalYAML(node *Node) error
}
// Re-export stream-related types
type (
VersionDirective = libyaml.StreamVersionDirective
TagDirective = libyaml.StreamTagDirective
Encoding = libyaml.Encoding
)
// Re-export encoding constants
const (
EncodingAny = libyaml.ANY_ENCODING
EncodingUTF8 = libyaml.UTF8_ENCODING
EncodingUTF16LE = libyaml.UTF16LE_ENCODING
EncodingUTF16BE = libyaml.UTF16BE_ENCODING
)
// Re-export error types
type (
// LoadError represents an error encountered while decoding a YAML document.
//
// It contains details about the location in the document where the error
// occurred, as well as a descriptive message.
LoadError = libyaml.ConstructError
// LoadErrors is returned when one or more fields cannot be properly decoded.
//
// It contains multiple *[LoadError] instances with details about each error.
LoadErrors = libyaml.LoadErrors
// TypeError is an obsolete error type retained for compatibility.
//
// Deprecated: Use [LoadErrors] instead.
//
//nolint:staticcheck // we are using deprecated TypeError for compatibility
TypeError = libyaml.TypeError
)
// Re-export Kind constants
const (
DocumentNode = libyaml.DocumentNode
SequenceNode = libyaml.SequenceNode
MappingNode = libyaml.MappingNode
ScalarNode = libyaml.ScalarNode
AliasNode = libyaml.AliasNode
StreamNode = libyaml.StreamNode
)
// Re-export Style constants
const (
TaggedStyle = libyaml.TaggedStyle
DoubleQuotedStyle = libyaml.DoubleQuotedStyle
SingleQuotedStyle = libyaml.SingleQuotedStyle
LiteralStyle = libyaml.LiteralStyle
FoldedStyle = libyaml.FoldedStyle
FlowStyle = libyaml.FlowStyle
)
// LineBreak represents the line ending style for YAML output.
type LineBreak = libyaml.LineBreak
// Line break constants for different platforms.
const (
LineBreakLN = libyaml.LN_BREAK // Unix-style \n (default)
LineBreakCR = libyaml.CR_BREAK // Old Mac-style \r
LineBreakCRLN = libyaml.CRLN_BREAK // Windows-style \r\n
)
// QuoteStyle represents the quote style to use when quoting is required.
type QuoteStyle = libyaml.QuoteStyle
// Quote style constants for required quoting.
const (
QuoteSingle = libyaml.QuoteSingle // Prefer single quotes (v4 default)
QuoteDouble = libyaml.QuoteDouble // Prefer double quotes
QuoteLegacy = libyaml.QuoteLegacy // Legacy v2/v3 behavior
)
//-----------------------------------------------------------------------------
// Helper functions
//-----------------------------------------------------------------------------
// The code in this section was copied from mgo/bson.
var (
structMap = make(map[reflect.Type]*structInfo)
fieldMapMutex sync.RWMutex
unmarshalerType reflect.Type
)
// structInfo holds details for the serialization of fields of
// a given struct.
type structInfo struct {
FieldsMap map[string]fieldInfo
FieldsList []fieldInfo
// InlineMap is the number of the field in the struct that
// contains an ,inline map, or -1 if there's none.
InlineMap int
// InlineUnmarshalers holds indexes to inlined fields that
// contain unmarshaler values.
InlineUnmarshalers [][]int
}
type fieldInfo struct {
Key string
Num int
OmitEmpty bool
Flow bool
// Id holds the unique field identifier, so we can cheaply
// check for field duplicates without maintaining an extra map.
Id int
// Inline holds the field index if the field is part of an inlined struct.
Inline []int
}
func getStructInfo(st reflect.Type) (*structInfo, error) {
fieldMapMutex.RLock()
sinfo, found := structMap[st]
fieldMapMutex.RUnlock()
if found {
return sinfo, nil
}
n := st.NumField()
fieldsMap := make(map[string]fieldInfo)
fieldsList := make([]fieldInfo, 0, n)
inlineMap := -1
inlineUnmarshalers := [][]int(nil)
for i := 0; i != n; i++ {
field := st.Field(i)
if field.PkgPath != "" && !field.Anonymous {
continue // Private field
}
info := fieldInfo{Num: i}
tag := field.Tag.Get("yaml")
if tag == "" && !strings.Contains(string(field.Tag), ":") {
tag = string(field.Tag)
}
if tag == "-" {
continue
}
inline := false
fields := strings.Split(tag, ",")
if len(fields) > 1 {
for _, flag := range fields[1:] {
switch flag {
case "omitempty":
info.OmitEmpty = true
case "flow":
info.Flow = true
case "inline":
inline = true
default:
return nil, fmt.Errorf("unsupported flag %q in tag %q of type %s", flag, tag, st)
}
}
tag = fields[0]
}
if inline {
switch field.Type.Kind() {
case reflect.Map:
if inlineMap >= 0 {
return nil, errors.New("multiple ,inline maps in struct " + st.String())
}
if field.Type.Key() != reflect.TypeOf("") {
return nil, errors.New("option ,inline needs a map with string keys in struct " + st.String())
}
inlineMap = info.Num
case reflect.Struct, reflect.Pointer:
ftype := field.Type
for ftype.Kind() == reflect.Pointer {
ftype = ftype.Elem()
}
if ftype.Kind() != reflect.Struct {
return nil, errors.New("option ,inline may only be used on a struct or map field")
}
if reflect.PointerTo(ftype).Implements(unmarshalerType) {
inlineUnmarshalers = append(inlineUnmarshalers, []int{i})
} else {
sinfo, err := getStructInfo(ftype)
if err != nil {
return nil, err
}
for _, index := range sinfo.InlineUnmarshalers {
inlineUnmarshalers = append(inlineUnmarshalers, append([]int{i}, index...))
}
for _, finfo := range sinfo.FieldsList {
if _, found := fieldsMap[finfo.Key]; found {
msg := "duplicated key '" + finfo.Key + "' in struct " + st.String()
return nil, errors.New(msg)
}
if finfo.Inline == nil {
finfo.Inline = []int{i, finfo.Num}
} else {
finfo.Inline = append([]int{i}, finfo.Inline...)
}
finfo.Id = len(fieldsList)
fieldsMap[finfo.Key] = finfo
fieldsList = append(fieldsList, finfo)
}
}
default:
return nil, errors.New("option ,inline may only be used on a struct or map field")
}
continue
}
if tag != "" {
info.Key = tag
} else {
info.Key = strings.ToLower(field.Name)
}
if _, found = fieldsMap[info.Key]; found {
msg := "duplicated key '" + info.Key + "' in struct " + st.String()
return nil, errors.New(msg)
}
info.Id = len(fieldsList)
fieldsList = append(fieldsList, info)
fieldsMap[info.Key] = info
}
sinfo = &structInfo{
FieldsMap: fieldsMap,
FieldsList: fieldsList,
InlineMap: inlineMap,
InlineUnmarshalers: inlineUnmarshalers,
}
fieldMapMutex.Lock()
structMap[st] = sinfo
fieldMapMutex.Unlock()
return sinfo, nil
}
var noWriter io.Writer
func handleErr(err *error) {
if v := recover(); v != nil {
if e, ok := v.(*libyaml.YAMLError); ok {
*err = e.Err
} else {
panic(v)
}
}
}
//-----------------------------------------------------------------------------
// Classic APIs
//-----------------------------------------------------------------------------
// A Decoder reads and decodes YAML values from an input stream.
type Decoder struct {
composer *libyaml.Composer
knownFields bool
}
// NewDecoder returns a new decoder that reads from r.
//
// The decoder introduces its own buffering and may read
// data from r beyond the YAML values requested.
func NewDecoder(r io.Reader) *Decoder {
return &Decoder{
composer: libyaml.NewComposerFromReader(r),
}
}
// KnownFields ensures that the keys in decoded mappings to
// exist as fields in the struct being decoded into.
func (dec *Decoder) KnownFields(enable bool) {
dec.knownFields = enable
}
// Decode reads the next YAML-encoded value from its input
// and stores it in the value pointed to by v.
//
// See the documentation for Unmarshal for details about the
// conversion of YAML into a Go value.
func (dec *Decoder) Decode(v any) (err error) {
d := libyaml.NewConstructor(libyaml.DefaultOptions)
d.KnownFields = dec.knownFields
defer handleErr(&err)
node := dec.composer.Parse()
if node == nil {
return io.EOF
}
out := reflect.ValueOf(v)
if out.Kind() == reflect.Pointer && !out.IsNil() {
out = out.Elem()
}
d.Construct(node, out)
if len(d.TypeErrors) > 0 {
return &LoadErrors{Errors: d.TypeErrors}
}
return nil
}
// An Encoder writes YAML values to an output stream.
type Encoder struct {
encoder *libyaml.Representer
}
// NewEncoder returns a new encoder that writes to w.
// The Encoder should be closed after use to flush all data
// to w.
func NewEncoder(w io.Writer) *Encoder {
return &Encoder{
encoder: libyaml.NewRepresenter(w, libyaml.DefaultOptions),
}
}
// Encode writes the YAML encoding of v to the stream.
// If multiple items are encoded to the stream, the
// second and subsequent document will be preceded
// with a "---" document separator, but the first will not.
//
// See the documentation for Marshal for details about the conversion of Go
// values to YAML.
func (e *Encoder) Encode(v any) (err error) {
defer handleErr(&err)
e.encoder.MarshalDoc("", reflect.ValueOf(v))
return nil
}
// SetIndent changes the used indentation used when encoding.
func (e *Encoder) SetIndent(spaces int) {
if spaces < 0 {
panic("yaml: cannot indent to a negative number of spaces")
}
e.encoder.Indent = spaces
}
// CompactSeqIndent makes it so that '- ' is considered part of the indentation.
func (e *Encoder) CompactSeqIndent() {
e.encoder.Emitter.CompactSequenceIndent = true
}
// DefaultSeqIndent makes it so that '- ' is not considered part of the indentation.
func (e *Encoder) DefaultSeqIndent() {
e.encoder.Emitter.CompactSequenceIndent = false
}
// Close closes the encoder by writing any remaining data.
// It does not write a stream terminating string "...".
func (e *Encoder) Close() (err error) {
defer handleErr(&err)
e.encoder.Finish()
return nil
}
// Unmarshal decodes the first document found within the in byte slice
// and assigns decoded values into the out value.
//
// Maps and pointers (to a struct, string, int, etc) are accepted as out
// values. If an internal pointer within a struct is not initialized,
// the yaml package will initialize it if necessary for unmarshalling
// the provided data. The out parameter must not be nil.
//
// The type of the decoded values should be compatible with the respective
// values in out. If one or more values cannot be decoded due to a type
// mismatches, decoding continues partially until the end of the YAML
// content, and a *yaml.LoadErrors is returned with details for all
// missed values.
//
// Struct fields are only unmarshalled if they are exported (have an
// upper case first letter), and are unmarshalled using the field name
// lowercased as the default key. Custom keys may be defined via the
// "yaml" name in the field tag: the content preceding the first comma
// is used as the key, and the following comma-separated options are
// used to tweak the marshaling process (see Marshal).
// Conflicting names result in a runtime error.
//
// For example:
//
// type T struct {
// F int `yaml:"a,omitempty"`
// B int
// }
// var t T
// yaml.Construct([]byte("a: 1\nb: 2"), &t)
//
// See the documentation of Marshal for the format of tags and a list of
// supported tag options.
func Unmarshal(in []byte, out any) (err error) {
return unmarshal(in, out, V3)
}
func unmarshal(in []byte, out any, opts ...Option) (err error) {
defer handleErr(&err)
o, err := libyaml.ApplyOptions(opts...)
if err != nil {
return err
}
// Check if out implements yaml.Unmarshaler
if u, ok := out.(Unmarshaler); ok {
p := libyaml.NewComposer(in)
defer p.Destroy()
node := p.Parse()
if node != nil {
return u.UnmarshalYAML(node)
}
return nil
}
return libyaml.Construct(in, out, o)
}
// Marshal serializes the value provided into a YAML document. The structure
// of the generated document will reflect the structure of the value itself.
// Maps and pointers (to struct, string, int, etc) are accepted as the in value.
//
// Struct fields are only marshaled if they are exported (have an upper case
// first letter), and are marshaled using the field name lowercased as the
// default key. Custom keys may be defined via the "yaml" name in the field
// tag: the content preceding the first comma is used as the key, and the
// following comma-separated options are used to tweak the marshaling process.
// Conflicting names result in a runtime error.
//
// The field tag format accepted is:
//
// `(...) yaml:"[<key>][,<flag1>[,<flag2>]]" (...)`
//
// The following flags are currently supported:
//
// omitempty Only include the field if it's not set to the zero
// value for the type or to empty slices or maps.
// Zero valued structs will be omitted if all their public
// fields are zero, unless they implement an IsZero
// method (see the IsZeroer interface type), in which
// case the field will be excluded if IsZero returns true.
//
// flow Marshal using a flow style (useful for structs,
// sequences and maps).
//
// inline Inline the field, which must be a struct or a map,
// causing all of its fields or keys to be processed as if
// they were part of the outer struct. For maps, keys must
// not conflict with the yaml keys of other struct fields.
// See doc/inline-tags.md for detailed examples and use cases.
//
// In addition, if the key is "-", the field is ignored.
//
// For example:
//
// type T struct {
// F int `yaml:"a,omitempty"`
// B int
// }
// yaml.Marshal(&T{B: 2}) // Returns "b: 2\n"
// yaml.Marshal(&T{F: 1}} // Returns "a: 1\nb: 0\n"
func Marshal(in any) (out []byte, err error) {
defer handleErr(&err)
e := libyaml.NewRepresenter(noWriter, libyaml.DefaultOptions)
defer e.Destroy()
e.MarshalDoc("", reflect.ValueOf(in))
e.Finish()
out = e.Out
return out, err
}