updated vendor
This commit is contained in:
@@ -1,2 +1,3 @@
|
||||
* -text
|
||||
*.bin -text -diff
|
||||
*.md text eol=lf
|
||||
|
||||
+10
-1
@@ -31,6 +31,9 @@ builds:
|
||||
- mips64le
|
||||
goarm:
|
||||
- 7
|
||||
ignore:
|
||||
- goos: windows
|
||||
goarch: arm
|
||||
-
|
||||
id: "s2d"
|
||||
binary: s2d
|
||||
@@ -57,6 +60,9 @@ builds:
|
||||
- mips64le
|
||||
goarm:
|
||||
- 7
|
||||
ignore:
|
||||
- goos: windows
|
||||
goarch: arm
|
||||
-
|
||||
id: "s2sx"
|
||||
binary: s2sx
|
||||
@@ -84,6 +90,9 @@ builds:
|
||||
- mips64le
|
||||
goarm:
|
||||
- 7
|
||||
ignore:
|
||||
- goos: windows
|
||||
goarch: arm
|
||||
|
||||
archives:
|
||||
-
|
||||
@@ -91,7 +100,7 @@ archives:
|
||||
name_template: "s2-{{ .Os }}_{{ .Arch }}{{ if .Arm }}v{{ .Arm }}{{ end }}"
|
||||
format_overrides:
|
||||
- goos: windows
|
||||
format: zip
|
||||
formats: ['zip']
|
||||
files:
|
||||
- unpack/*
|
||||
- s2/LICENSE
|
||||
|
||||
+700
-693
File diff suppressed because it is too large
Load Diff
+78
-78
@@ -1,79 +1,79 @@
|
||||
# Finite State Entropy
|
||||
|
||||
This package provides Finite State Entropy encoding and decoding.
|
||||
|
||||
Finite State Entropy (also referenced as [tANS](https://en.wikipedia.org/wiki/Asymmetric_numeral_systems#tANS))
|
||||
encoding provides a fast near-optimal symbol encoding/decoding
|
||||
for byte blocks as implemented in [zstandard](https://github.com/facebook/zstd).
|
||||
|
||||
This can be used for compressing input with a lot of similar input values to the smallest number of bytes.
|
||||
This does not perform any multi-byte [dictionary coding](https://en.wikipedia.org/wiki/Dictionary_coder) as LZ coders,
|
||||
but it can be used as a secondary step to compressors (like Snappy) that does not do entropy encoding.
|
||||
|
||||
* [Godoc documentation](https://godoc.org/github.com/klauspost/compress/fse)
|
||||
|
||||
## News
|
||||
|
||||
* Feb 2018: First implementation released. Consider this beta software for now.
|
||||
|
||||
# Usage
|
||||
|
||||
This package provides a low level interface that allows to compress single independent blocks.
|
||||
|
||||
Each block is separate, and there is no built in integrity checks.
|
||||
This means that the caller should keep track of block sizes and also do checksums if needed.
|
||||
|
||||
Compressing a block is done via the [`Compress`](https://godoc.org/github.com/klauspost/compress/fse#Compress) function.
|
||||
You must provide input and will receive the output and maybe an error.
|
||||
|
||||
These error values can be returned:
|
||||
|
||||
| Error | Description |
|
||||
|---------------------|-----------------------------------------------------------------------------|
|
||||
| `<nil>` | Everything ok, output is returned |
|
||||
| `ErrIncompressible` | Returned when input is judged to be too hard to compress |
|
||||
| `ErrUseRLE` | Returned from the compressor when the input is a single byte value repeated |
|
||||
| `(error)` | An internal error occurred. |
|
||||
|
||||
As can be seen above there are errors that will be returned even under normal operation so it is important to handle these.
|
||||
|
||||
To reduce allocations you can provide a [`Scratch`](https://godoc.org/github.com/klauspost/compress/fse#Scratch) object
|
||||
that can be re-used for successive calls. Both compression and decompression accepts a `Scratch` object, and the same
|
||||
object can be used for both.
|
||||
|
||||
Be aware, that when re-using a `Scratch` object that the *output* buffer is also re-used, so if you are still using this
|
||||
you must set the `Out` field in the scratch to nil. The same buffer is used for compression and decompression output.
|
||||
|
||||
Decompressing is done by calling the [`Decompress`](https://godoc.org/github.com/klauspost/compress/fse#Decompress) function.
|
||||
You must provide the output from the compression stage, at exactly the size you got back. If you receive an error back
|
||||
your input was likely corrupted.
|
||||
|
||||
It is important to note that a successful decoding does *not* mean your output matches your original input.
|
||||
There are no integrity checks, so relying on errors from the decompressor does not assure your data is valid.
|
||||
|
||||
For more detailed usage, see examples in the [godoc documentation](https://godoc.org/github.com/klauspost/compress/fse#pkg-examples).
|
||||
|
||||
# Performance
|
||||
|
||||
A lot of factors are affecting speed. Block sizes and compressibility of the material are primary factors.
|
||||
All compression functions are currently only running on the calling goroutine so only one core will be used per block.
|
||||
|
||||
The compressor is significantly faster if symbols are kept as small as possible. The highest byte value of the input
|
||||
is used to reduce some of the processing, so if all your input is above byte value 64 for instance, it may be
|
||||
beneficial to transpose all your input values down by 64.
|
||||
|
||||
With moderate block sizes around 64k speed are typically 200MB/s per core for compression and
|
||||
around 300MB/s decompression speed.
|
||||
|
||||
The same hardware typically does Huffman (deflate) encoding at 125MB/s and decompression at 100MB/s.
|
||||
|
||||
# Plans
|
||||
|
||||
At one point, more internals will be exposed to facilitate more "expert" usage of the components.
|
||||
|
||||
A streaming interface is also likely to be implemented. Likely compatible with [FSE stream format](https://github.com/Cyan4973/FiniteStateEntropy/blob/dev/programs/fileio.c#L261).
|
||||
|
||||
# Contributing
|
||||
|
||||
Contributions are always welcome. Be aware that adding public functions will require good justification and breaking
|
||||
# Finite State Entropy
|
||||
|
||||
This package provides Finite State Entropy encoding and decoding.
|
||||
|
||||
Finite State Entropy (also referenced as [tANS](https://en.wikipedia.org/wiki/Asymmetric_numeral_systems#tANS))
|
||||
encoding provides a fast near-optimal symbol encoding/decoding
|
||||
for byte blocks as implemented in [zstandard](https://github.com/facebook/zstd).
|
||||
|
||||
This can be used for compressing input with a lot of similar input values to the smallest number of bytes.
|
||||
This does not perform any multi-byte [dictionary coding](https://en.wikipedia.org/wiki/Dictionary_coder) as LZ coders,
|
||||
but it can be used as a secondary step to compressors (like Snappy) that does not do entropy encoding.
|
||||
|
||||
* [Godoc documentation](https://godoc.org/github.com/klauspost/compress/fse)
|
||||
|
||||
## News
|
||||
|
||||
* Feb 2018: First implementation released. Consider this beta software for now.
|
||||
|
||||
# Usage
|
||||
|
||||
This package provides a low level interface that allows to compress single independent blocks.
|
||||
|
||||
Each block is separate, and there is no built in integrity checks.
|
||||
This means that the caller should keep track of block sizes and also do checksums if needed.
|
||||
|
||||
Compressing a block is done via the [`Compress`](https://godoc.org/github.com/klauspost/compress/fse#Compress) function.
|
||||
You must provide input and will receive the output and maybe an error.
|
||||
|
||||
These error values can be returned:
|
||||
|
||||
| Error | Description |
|
||||
|---------------------|-----------------------------------------------------------------------------|
|
||||
| `<nil>` | Everything ok, output is returned |
|
||||
| `ErrIncompressible` | Returned when input is judged to be too hard to compress |
|
||||
| `ErrUseRLE` | Returned from the compressor when the input is a single byte value repeated |
|
||||
| `(error)` | An internal error occurred. |
|
||||
|
||||
As can be seen above there are errors that will be returned even under normal operation so it is important to handle these.
|
||||
|
||||
To reduce allocations you can provide a [`Scratch`](https://godoc.org/github.com/klauspost/compress/fse#Scratch) object
|
||||
that can be re-used for successive calls. Both compression and decompression accepts a `Scratch` object, and the same
|
||||
object can be used for both.
|
||||
|
||||
Be aware, that when re-using a `Scratch` object that the *output* buffer is also re-used, so if you are still using this
|
||||
you must set the `Out` field in the scratch to nil. The same buffer is used for compression and decompression output.
|
||||
|
||||
Decompressing is done by calling the [`Decompress`](https://godoc.org/github.com/klauspost/compress/fse#Decompress) function.
|
||||
You must provide the output from the compression stage, at exactly the size you got back. If you receive an error back
|
||||
your input was likely corrupted.
|
||||
|
||||
It is important to note that a successful decoding does *not* mean your output matches your original input.
|
||||
There are no integrity checks, so relying on errors from the decompressor does not assure your data is valid.
|
||||
|
||||
For more detailed usage, see examples in the [godoc documentation](https://godoc.org/github.com/klauspost/compress/fse#pkg-examples).
|
||||
|
||||
# Performance
|
||||
|
||||
A lot of factors are affecting speed. Block sizes and compressibility of the material are primary factors.
|
||||
All compression functions are currently only running on the calling goroutine so only one core will be used per block.
|
||||
|
||||
The compressor is significantly faster if symbols are kept as small as possible. The highest byte value of the input
|
||||
is used to reduce some of the processing, so if all your input is above byte value 64 for instance, it may be
|
||||
beneficial to transpose all your input values down by 64.
|
||||
|
||||
With moderate block sizes around 64k speed are typically 200MB/s per core for compression and
|
||||
around 300MB/s decompression speed.
|
||||
|
||||
The same hardware typically does Huffman (deflate) encoding at 125MB/s and decompression at 100MB/s.
|
||||
|
||||
# Plans
|
||||
|
||||
At one point, more internals will be exposed to facilitate more "expert" usage of the components.
|
||||
|
||||
A streaming interface is also likely to be implemented. Likely compatible with [FSE stream format](https://github.com/Cyan4973/FiniteStateEntropy/blob/dev/programs/fileio.c#L261).
|
||||
|
||||
# Contributing
|
||||
|
||||
Contributions are always welcome. Be aware that adding public functions will require good justification and breaking
|
||||
changes will likely not be accepted. If in doubt open an issue before writing the PR.
|
||||
+89
-89
@@ -1,89 +1,89 @@
|
||||
# Huff0 entropy compression
|
||||
|
||||
This package provides Huff0 encoding and decoding as used in zstd.
|
||||
|
||||
[Huff0](https://github.com/Cyan4973/FiniteStateEntropy#new-generation-entropy-coders),
|
||||
a Huffman codec designed for modern CPU, featuring OoO (Out of Order) operations on multiple ALU
|
||||
(Arithmetic Logic Unit), achieving extremely fast compression and decompression speeds.
|
||||
|
||||
This can be used for compressing input with a lot of similar input values to the smallest number of bytes.
|
||||
This does not perform any multi-byte [dictionary coding](https://en.wikipedia.org/wiki/Dictionary_coder) as LZ coders,
|
||||
but it can be used as a secondary step to compressors (like Snappy) that does not do entropy encoding.
|
||||
|
||||
* [Godoc documentation](https://godoc.org/github.com/klauspost/compress/huff0)
|
||||
|
||||
## News
|
||||
|
||||
This is used as part of the [zstandard](https://github.com/klauspost/compress/tree/master/zstd#zstd) compression and decompression package.
|
||||
|
||||
This ensures that most functionality is well tested.
|
||||
|
||||
# Usage
|
||||
|
||||
This package provides a low level interface that allows to compress single independent blocks.
|
||||
|
||||
Each block is separate, and there is no built in integrity checks.
|
||||
This means that the caller should keep track of block sizes and also do checksums if needed.
|
||||
|
||||
Compressing a block is done via the [`Compress1X`](https://godoc.org/github.com/klauspost/compress/huff0#Compress1X) and
|
||||
[`Compress4X`](https://godoc.org/github.com/klauspost/compress/huff0#Compress4X) functions.
|
||||
You must provide input and will receive the output and maybe an error.
|
||||
|
||||
These error values can be returned:
|
||||
|
||||
| Error | Description |
|
||||
|---------------------|-----------------------------------------------------------------------------|
|
||||
| `<nil>` | Everything ok, output is returned |
|
||||
| `ErrIncompressible` | Returned when input is judged to be too hard to compress |
|
||||
| `ErrUseRLE` | Returned from the compressor when the input is a single byte value repeated |
|
||||
| `ErrTooBig` | Returned if the input block exceeds the maximum allowed size (128 Kib) |
|
||||
| `(error)` | An internal error occurred. |
|
||||
|
||||
|
||||
As can be seen above some of there are errors that will be returned even under normal operation so it is important to handle these.
|
||||
|
||||
To reduce allocations you can provide a [`Scratch`](https://godoc.org/github.com/klauspost/compress/huff0#Scratch) object
|
||||
that can be re-used for successive calls. Both compression and decompression accepts a `Scratch` object, and the same
|
||||
object can be used for both.
|
||||
|
||||
Be aware, that when re-using a `Scratch` object that the *output* buffer is also re-used, so if you are still using this
|
||||
you must set the `Out` field in the scratch to nil. The same buffer is used for compression and decompression output.
|
||||
|
||||
The `Scratch` object will retain state that allows to re-use previous tables for encoding and decoding.
|
||||
|
||||
## Tables and re-use
|
||||
|
||||
Huff0 allows for reusing tables from the previous block to save space if that is expected to give better/faster results.
|
||||
|
||||
The Scratch object allows you to set a [`ReusePolicy`](https://godoc.org/github.com/klauspost/compress/huff0#ReusePolicy)
|
||||
that controls this behaviour. See the documentation for details. This can be altered between each block.
|
||||
|
||||
Do however note that this information is *not* stored in the output block and it is up to the users of the package to
|
||||
record whether [`ReadTable`](https://godoc.org/github.com/klauspost/compress/huff0#ReadTable) should be called,
|
||||
based on the boolean reported back from the CompressXX call.
|
||||
|
||||
If you want to store the table separate from the data, you can access them as `OutData` and `OutTable` on the
|
||||
[`Scratch`](https://godoc.org/github.com/klauspost/compress/huff0#Scratch) object.
|
||||
|
||||
## Decompressing
|
||||
|
||||
The first part of decoding is to initialize the decoding table through [`ReadTable`](https://godoc.org/github.com/klauspost/compress/huff0#ReadTable).
|
||||
This will initialize the decoding tables.
|
||||
You can supply the complete block to `ReadTable` and it will return the data part of the block
|
||||
which can be given to the decompressor.
|
||||
|
||||
Decompressing is done by calling the [`Decompress1X`](https://godoc.org/github.com/klauspost/compress/huff0#Scratch.Decompress1X)
|
||||
or [`Decompress4X`](https://godoc.org/github.com/klauspost/compress/huff0#Scratch.Decompress4X) function.
|
||||
|
||||
For concurrently decompressing content with a fixed table a stateless [`Decoder`](https://godoc.org/github.com/klauspost/compress/huff0#Decoder) can be requested which will remain correct as long as the scratch is unchanged. The capacity of the provided slice indicates the expected output size.
|
||||
|
||||
You must provide the output from the compression stage, at exactly the size you got back. If you receive an error back
|
||||
your input was likely corrupted.
|
||||
|
||||
It is important to note that a successful decoding does *not* mean your output matches your original input.
|
||||
There are no integrity checks, so relying on errors from the decompressor does not assure your data is valid.
|
||||
|
||||
# Contributing
|
||||
|
||||
Contributions are always welcome. Be aware that adding public functions will require good justification and breaking
|
||||
changes will likely not be accepted. If in doubt open an issue before writing the PR.
|
||||
# Huff0 entropy compression
|
||||
|
||||
This package provides Huff0 encoding and decoding as used in zstd.
|
||||
|
||||
[Huff0](https://github.com/Cyan4973/FiniteStateEntropy#new-generation-entropy-coders),
|
||||
a Huffman codec designed for modern CPU, featuring OoO (Out of Order) operations on multiple ALU
|
||||
(Arithmetic Logic Unit), achieving extremely fast compression and decompression speeds.
|
||||
|
||||
This can be used for compressing input with a lot of similar input values to the smallest number of bytes.
|
||||
This does not perform any multi-byte [dictionary coding](https://en.wikipedia.org/wiki/Dictionary_coder) as LZ coders,
|
||||
but it can be used as a secondary step to compressors (like Snappy) that does not do entropy encoding.
|
||||
|
||||
* [Godoc documentation](https://godoc.org/github.com/klauspost/compress/huff0)
|
||||
|
||||
## News
|
||||
|
||||
This is used as part of the [zstandard](https://github.com/klauspost/compress/tree/master/zstd#zstd) compression and decompression package.
|
||||
|
||||
This ensures that most functionality is well tested.
|
||||
|
||||
# Usage
|
||||
|
||||
This package provides a low level interface that allows to compress single independent blocks.
|
||||
|
||||
Each block is separate, and there is no built in integrity checks.
|
||||
This means that the caller should keep track of block sizes and also do checksums if needed.
|
||||
|
||||
Compressing a block is done via the [`Compress1X`](https://godoc.org/github.com/klauspost/compress/huff0#Compress1X) and
|
||||
[`Compress4X`](https://godoc.org/github.com/klauspost/compress/huff0#Compress4X) functions.
|
||||
You must provide input and will receive the output and maybe an error.
|
||||
|
||||
These error values can be returned:
|
||||
|
||||
| Error | Description |
|
||||
|---------------------|-----------------------------------------------------------------------------|
|
||||
| `<nil>` | Everything ok, output is returned |
|
||||
| `ErrIncompressible` | Returned when input is judged to be too hard to compress |
|
||||
| `ErrUseRLE` | Returned from the compressor when the input is a single byte value repeated |
|
||||
| `ErrTooBig` | Returned if the input block exceeds the maximum allowed size (128 Kib) |
|
||||
| `(error)` | An internal error occurred. |
|
||||
|
||||
|
||||
As can be seen above some of there are errors that will be returned even under normal operation so it is important to handle these.
|
||||
|
||||
To reduce allocations you can provide a [`Scratch`](https://godoc.org/github.com/klauspost/compress/huff0#Scratch) object
|
||||
that can be re-used for successive calls. Both compression and decompression accepts a `Scratch` object, and the same
|
||||
object can be used for both.
|
||||
|
||||
Be aware, that when re-using a `Scratch` object that the *output* buffer is also re-used, so if you are still using this
|
||||
you must set the `Out` field in the scratch to nil. The same buffer is used for compression and decompression output.
|
||||
|
||||
The `Scratch` object will retain state that allows to re-use previous tables for encoding and decoding.
|
||||
|
||||
## Tables and re-use
|
||||
|
||||
Huff0 allows for reusing tables from the previous block to save space if that is expected to give better/faster results.
|
||||
|
||||
The Scratch object allows you to set a [`ReusePolicy`](https://godoc.org/github.com/klauspost/compress/huff0#ReusePolicy)
|
||||
that controls this behaviour. See the documentation for details. This can be altered between each block.
|
||||
|
||||
Do however note that this information is *not* stored in the output block and it is up to the users of the package to
|
||||
record whether [`ReadTable`](https://godoc.org/github.com/klauspost/compress/huff0#ReadTable) should be called,
|
||||
based on the boolean reported back from the CompressXX call.
|
||||
|
||||
If you want to store the table separate from the data, you can access them as `OutData` and `OutTable` on the
|
||||
[`Scratch`](https://godoc.org/github.com/klauspost/compress/huff0#Scratch) object.
|
||||
|
||||
## Decompressing
|
||||
|
||||
The first part of decoding is to initialize the decoding table through [`ReadTable`](https://godoc.org/github.com/klauspost/compress/huff0#ReadTable).
|
||||
This will initialize the decoding tables.
|
||||
You can supply the complete block to `ReadTable` and it will return the data part of the block
|
||||
which can be given to the decompressor.
|
||||
|
||||
Decompressing is done by calling the [`Decompress1X`](https://godoc.org/github.com/klauspost/compress/huff0#Scratch.Decompress1X)
|
||||
or [`Decompress4X`](https://godoc.org/github.com/klauspost/compress/huff0#Scratch.Decompress4X) function.
|
||||
|
||||
For concurrently decompressing content with a fixed table a stateless [`Decoder`](https://godoc.org/github.com/klauspost/compress/huff0#Decoder) can be requested which will remain correct as long as the scratch is unchanged. The capacity of the provided slice indicates the expected output size.
|
||||
|
||||
You must provide the output from the compression stage, at exactly the size you got back. If you receive an error back
|
||||
your input was likely corrupted.
|
||||
|
||||
It is important to note that a successful decoding does *not* mean your output matches your original input.
|
||||
There are no integrity checks, so relying on errors from the decompressor does not assure your data is valid.
|
||||
|
||||
# Contributing
|
||||
|
||||
Contributions are always welcome. Be aware that adding public functions will require good justification and breaking
|
||||
changes will likely not be accepted. If in doubt open an issue before writing the PR.
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
//go:build amd64 && !appengine && !noasm && gc
|
||||
// +build amd64,!appengine,!noasm,gc
|
||||
|
||||
// This file contains the specialisation of Decoder.Decompress4X
|
||||
// and Decoder.Decompress1X that use an asm implementation of thir main loops.
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
//go:build !amd64 || appengine || !gc || noasm
|
||||
// +build !amd64 appengine !gc noasm
|
||||
|
||||
// This file contains a generic implementation of Decoder.Decompress4X.
|
||||
package huff0
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
//go:build amd64 && !appengine && !noasm && gc
|
||||
// +build amd64,!appengine,!noasm,gc
|
||||
|
||||
package cpuinfo
|
||||
|
||||
|
||||
+1
@@ -78,6 +78,7 @@ func (b *blockEnc) initNewEncode() {
|
||||
b.recentOffsets = [3]uint32{1, 4, 8}
|
||||
b.litEnc.Reuse = huff0.ReusePolicyNone
|
||||
b.coders.setPrev(nil, nil, nil)
|
||||
b.dictLitEnc = nil
|
||||
}
|
||||
|
||||
// reset will reset the block for a new encode, but in the same stream,
|
||||
|
||||
+1
-1
@@ -21,7 +21,7 @@ type fastBase struct {
|
||||
crc *xxhash.Digest
|
||||
tmp [8]byte
|
||||
blk *blockEnc
|
||||
lastDictID uint32
|
||||
lastDict *dict
|
||||
lowMem bool
|
||||
}
|
||||
|
||||
|
||||
+9
-5
@@ -479,10 +479,13 @@ func (e *bestFastEncoder) Reset(d *dict, singleBlock bool) {
|
||||
if d == nil {
|
||||
return
|
||||
}
|
||||
dictChanged := d != e.lastDict
|
||||
// Init or copy dict table
|
||||
if len(e.dictTable) != len(e.table) || d.id != e.lastDictID {
|
||||
if len(e.dictTable) != len(e.table) || dictChanged {
|
||||
if len(e.dictTable) != len(e.table) {
|
||||
e.dictTable = make([]prevEntry, len(e.table))
|
||||
} else {
|
||||
clear(e.dictTable)
|
||||
}
|
||||
end := int32(len(d.content)) - 8 + e.maxMatchOff
|
||||
for i := e.maxMatchOff; i < end; i += 4 {
|
||||
@@ -510,13 +513,14 @@ func (e *bestFastEncoder) Reset(d *dict, singleBlock bool) {
|
||||
offset: i + 3,
|
||||
}
|
||||
}
|
||||
e.lastDictID = d.id
|
||||
}
|
||||
|
||||
// Init or copy dict table
|
||||
if len(e.dictLongTable) != len(e.longTable) || d.id != e.lastDictID {
|
||||
// Init or copy dict long table
|
||||
if len(e.dictLongTable) != len(e.longTable) || dictChanged {
|
||||
if len(e.dictLongTable) != len(e.longTable) {
|
||||
e.dictLongTable = make([]prevEntry, len(e.longTable))
|
||||
} else {
|
||||
clear(e.dictLongTable)
|
||||
}
|
||||
if len(d.content) >= 8 {
|
||||
cv := load6432(d.content, 0)
|
||||
@@ -538,8 +542,8 @@ func (e *bestFastEncoder) Reset(d *dict, singleBlock bool) {
|
||||
off++
|
||||
}
|
||||
}
|
||||
e.lastDictID = d.id
|
||||
}
|
||||
e.lastDict = d
|
||||
// Reset table to initial state
|
||||
copy(e.longTable[:], e.dictLongTable)
|
||||
|
||||
|
||||
+9
-5
@@ -1102,10 +1102,13 @@ func (e *betterFastEncoderDict) Reset(d *dict, singleBlock bool) {
|
||||
if d == nil {
|
||||
return
|
||||
}
|
||||
dictChanged := d != e.lastDict
|
||||
// Init or copy dict table
|
||||
if len(e.dictTable) != len(e.table) || d.id != e.lastDictID {
|
||||
if len(e.dictTable) != len(e.table) || dictChanged {
|
||||
if len(e.dictTable) != len(e.table) {
|
||||
e.dictTable = make([]tableEntry, len(e.table))
|
||||
} else {
|
||||
clear(e.dictTable)
|
||||
}
|
||||
end := int32(len(d.content)) - 8 + e.maxMatchOff
|
||||
for i := e.maxMatchOff; i < end; i += 4 {
|
||||
@@ -1133,14 +1136,15 @@ func (e *betterFastEncoderDict) Reset(d *dict, singleBlock bool) {
|
||||
offset: i + 3,
|
||||
}
|
||||
}
|
||||
e.lastDictID = d.id
|
||||
e.allDirty = true
|
||||
}
|
||||
|
||||
// Init or copy dict table
|
||||
if len(e.dictLongTable) != len(e.longTable) || d.id != e.lastDictID {
|
||||
// Init or copy dict long table
|
||||
if len(e.dictLongTable) != len(e.longTable) || dictChanged {
|
||||
if len(e.dictLongTable) != len(e.longTable) {
|
||||
e.dictLongTable = make([]prevEntry, len(e.longTable))
|
||||
} else {
|
||||
clear(e.dictLongTable)
|
||||
}
|
||||
if len(d.content) >= 8 {
|
||||
cv := load6432(d.content, 0)
|
||||
@@ -1162,9 +1166,9 @@ func (e *betterFastEncoderDict) Reset(d *dict, singleBlock bool) {
|
||||
off++
|
||||
}
|
||||
}
|
||||
e.lastDictID = d.id
|
||||
e.allDirty = true
|
||||
}
|
||||
e.lastDict = d
|
||||
|
||||
// Reset table to initial state
|
||||
{
|
||||
|
||||
+4
-2
@@ -1040,15 +1040,18 @@ func (e *doubleFastEncoder) Reset(d *dict, singleBlock bool) {
|
||||
// ResetDict will reset and set a dictionary if not nil
|
||||
func (e *doubleFastEncoderDict) Reset(d *dict, singleBlock bool) {
|
||||
allDirty := e.allDirty
|
||||
dictChanged := d != e.lastDict
|
||||
e.fastEncoderDict.Reset(d, singleBlock)
|
||||
if d == nil {
|
||||
return
|
||||
}
|
||||
|
||||
// Init or copy dict table
|
||||
if len(e.dictLongTable) != len(e.longTable) || d.id != e.lastDictID {
|
||||
if len(e.dictLongTable) != len(e.longTable) || dictChanged {
|
||||
if len(e.dictLongTable) != len(e.longTable) {
|
||||
e.dictLongTable = make([]tableEntry, len(e.longTable))
|
||||
} else {
|
||||
clear(e.dictLongTable)
|
||||
}
|
||||
if len(d.content) >= 8 {
|
||||
cv := load6432(d.content, 0)
|
||||
@@ -1065,7 +1068,6 @@ func (e *doubleFastEncoderDict) Reset(d *dict, singleBlock bool) {
|
||||
}
|
||||
}
|
||||
}
|
||||
e.lastDictID = d.id
|
||||
allDirty = true
|
||||
}
|
||||
// Reset table to initial state
|
||||
|
||||
+4
-2
@@ -805,9 +805,11 @@ func (e *fastEncoderDict) Reset(d *dict, singleBlock bool) {
|
||||
}
|
||||
|
||||
// Init or copy dict table
|
||||
if len(e.dictTable) != len(e.table) || d.id != e.lastDictID {
|
||||
if len(e.dictTable) != len(e.table) || d != e.lastDict {
|
||||
if len(e.dictTable) != len(e.table) {
|
||||
e.dictTable = make([]tableEntry, len(e.table))
|
||||
} else {
|
||||
clear(e.dictTable)
|
||||
}
|
||||
if true {
|
||||
end := e.maxMatchOff + int32(len(d.content)) - 8
|
||||
@@ -827,7 +829,7 @@ func (e *fastEncoderDict) Reset(d *dict, singleBlock bool) {
|
||||
}
|
||||
}
|
||||
}
|
||||
e.lastDictID = d.id
|
||||
e.lastDict = d
|
||||
e.allDirty = true
|
||||
}
|
||||
|
||||
|
||||
+13
@@ -138,11 +138,18 @@ func (e *Encoder) Reset(w io.Writer) {
|
||||
func (e *Encoder) ResetWithOptions(w io.Writer, opts ...EOption) error {
|
||||
e.o.resetOpt = true
|
||||
defer func() { e.o.resetOpt = false }()
|
||||
hadDict := e.o.dict != nil
|
||||
for _, o := range opts {
|
||||
if err := o(&e.o); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
hasDict := e.o.dict != nil
|
||||
if hadDict != hasDict {
|
||||
// Dict presence changed — encoder type must be recreated.
|
||||
e.state.encoder = nil
|
||||
e.init = sync.Once{}
|
||||
}
|
||||
e.Reset(w)
|
||||
return nil
|
||||
}
|
||||
@@ -448,6 +455,12 @@ func (e *Encoder) Close() error {
|
||||
if s.encoder == nil {
|
||||
return nil
|
||||
}
|
||||
if s.w == nil {
|
||||
if len(s.filling) == 0 && !s.headerWritten && !s.eofWritten && s.nInput == 0 {
|
||||
return nil
|
||||
}
|
||||
return errors.New("zstd: encoder has no writer")
|
||||
}
|
||||
err := e.nextBlock(true)
|
||||
if err != nil {
|
||||
if errors.Is(s.err, ErrEncoderClosed) {
|
||||
|
||||
+1
@@ -42,6 +42,7 @@ func (o *encoderOptions) setDefault() {
|
||||
level: SpeedDefault,
|
||||
allLitEntropy: false,
|
||||
lowMem: false,
|
||||
fullZero: true,
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
//go:build amd64 && !appengine && !noasm && gc
|
||||
// +build amd64,!appengine,!noasm,gc
|
||||
|
||||
package zstd
|
||||
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
//go:build !amd64 || appengine || !gc || noasm
|
||||
// +build !amd64 appengine !gc noasm
|
||||
|
||||
package zstd
|
||||
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
//go:build (!amd64 && !arm64) || appengine || !gc || purego || noasm
|
||||
// +build !amd64,!arm64 appengine !gc purego noasm
|
||||
|
||||
package xxhash
|
||||
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
//go:build amd64 && !appengine && !noasm && gc
|
||||
// +build amd64,!appengine,!noasm,gc
|
||||
|
||||
// Copyright 2019+ Klaus Post. All rights reserved.
|
||||
// License information can be found in the LICENSE file.
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
//go:build !amd64 || appengine || !gc || noasm
|
||||
// +build !amd64 appengine !gc noasm
|
||||
|
||||
// Copyright 2019+ Klaus Post. All rights reserved.
|
||||
// License information can be found in the LICENSE file.
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
//go:build amd64 && !appengine && !noasm && gc
|
||||
// +build amd64,!appengine,!noasm,gc
|
||||
|
||||
package zstd
|
||||
|
||||
|
||||
-1
@@ -1,5 +1,4 @@
|
||||
//go:build !amd64 || appengine || !gc || noasm
|
||||
// +build !amd64 appengine !gc noasm
|
||||
|
||||
package zstd
|
||||
|
||||
|
||||
Reference in New Issue
Block a user