whatcanGOwrong

This commit is contained in:
2024-09-19 21:38:24 -04:00
commit d0ae4d841d
17908 changed files with 4096831 additions and 0 deletions
@@ -0,0 +1,2 @@
# To prevent CRLF breakages on Windows for fragile files, like testdata.
* -text
@@ -0,0 +1 @@
github: mvdan
@@ -0,0 +1,23 @@
on: [push, pull_request]
name: Test
jobs:
test:
strategy:
matrix:
go-version: [1.20.x, 1.21.x]
os: [ubuntu-latest, macos-latest, windows-latest]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/setup-go@v3
with:
go-version: ${{ matrix.go-version }}
- uses: actions/checkout@v3
- run: go test ./...
- run: go test -race ./...
# Static checks from this point forward. Only run on one Go version and on
# Linux, since it's the fastest platform, and the tools behave the same.
- if: matrix.os == 'ubuntu-latest' && matrix.go-version == '1.21.x'
run: diff <(echo -n) <(gofmt -s -d .)
- if: matrix.os == 'ubuntu-latest' && matrix.go-version == '1.21.x'
run: go vet ./...
@@ -0,0 +1,175 @@
# Changelog
## [v0.6.0] - 2024-01-28
This release is based on Go 1.21's gofmt, and requires Go 1.20 or later.
The following changes are included:
* Support `go` version strings from newer go.mod files - [#280]
* Consider simple error checks even if they use the `=` operator - [#271]
* Ignore `//line` directives to avoid panics - [#288]
## [v0.5.0] - 2023-04-09
This release is based on Go 1.20's gofmt, and requires Go 1.19 or later.
The biggest change in this release is that we now vendor copies of the packages
`go/format`, `go/printer`, and `go/doc/comment` on top of `cmd/gofmt` itself.
This allows for each gofumpt release to format code in exactly the same way
no matter what Go version is used to build it, as Go versions can change those
three packages in ways that alter formatting behavior.
This vendoring adds a small amount of duplication when using the
`mvdan.cc/gofumpt/format` library, but it's the only way to make gofumpt
versions consistent in their behavior and formatting, just like gofmt.
The jump to Go 1.20's `go/printer` should also bring a small performance
improvement, as we contributed patches to make printing about 25% faster:
* https://go.dev/cl/412555
* https://go.dev/cl/412557
* https://go.dev/cl/424924
The following changes are included as well:
* Skip `testdata` dirs by default like we already do for `vendor` - [#260]
* Avoid inserting newlines incorrectly in some func signatures - [#235]
* Avoid joining some comments with the previous line - [#256]
* Fix `gofumpt -version` for release archives - [#253]
## [v0.4.0] - 2022-09-27
This release is based on Go 1.19's gofmt, and requires Go 1.18 or later.
We recommend building gofumpt with Go 1.19 for the best formatting results.
The jump from Go 1.18 brings diffing in pure Go, removing the need to exec `diff`,
and a small parsing speed-up thanks to `go/parser.SkipObjectResolution`.
The following formatting fixes are included as well:
* Allow grouping declarations with comments - [#212]
* Properly measure the length of case clauses - [#217]
* Fix a few crashes found by Go's native fuzzing
## [v0.3.1] - 2022-03-21
This bugfix release resolves a number of issues:
* Avoid "too many open files" error regression introduced by [v0.3.0] - [#208]
* Use the `go.mod` relative to each Go file when deriving flag defaults - [#211]
* Remove unintentional debug prints when directly formatting files
## [v0.3.0] - 2022-02-22
This is gofumpt's third major release, based on Go 1.18's gofmt.
The jump from Go 1.17's gofmt should bring a noticeable speed-up,
as the tool can now format many files concurrently.
On an 8-core laptop, formatting a large codebase is 4x as fast.
The following [formatting rules](https://github.com/mvdan/gofumpt#Added-rules) are added:
* Functions should separate `) {` where the indentation helps readability
* Field lists should not have leading or trailing empty lines
The following changes are included as well:
* Generated files are now fully formatted when given as explicit arguments
* Prepare for Go 1.18's module workspaces, which could cause errors
* Import paths sharing a prefix with the current module path are no longer
grouped with standard library imports
* `format.Options` gains a `ModulePath` field per the last bullet point
## [v0.2.1] - 2021-12-12
This bugfix release resolves a number of issues:
* Add deprecated flags `-s` and `-r` once again, now giving useful errors
* Avoid a panic with certain function declaration styles
* Don't group interface members of different kinds
* Account for leading comments in composite literals
## [v0.2.0] - 2021-11-10
This is gofumpt's second major release, based on Go 1.17's gofmt.
The jump from Go 1.15's gofmt should bring a mild speed-up,
as walking directories with `filepath.WalkDir` uses fewer syscalls.
gofumports is now removed, after being deprecated in [v0.1.0].
Its main purpose was IDE integration; it is now recommended to use gopls,
which in turn implements goimports and supports gofumpt natively.
IDEs which don't integrate with gopls (such as GoLand) implement goimports too,
so it is safe to use gofumpt as their "format on save" command.
See the [installation instructions](https://github.com/mvdan/gofumpt#Installation)
for more details.
The following [formatting rules](https://github.com/mvdan/gofumpt#Added-rules) are added:
* Composite literals should not have leading or trailing empty lines
* No empty lines following an assignment operator
* Functions using an empty line for readability should use a `) {` line instead
* Remove unnecessary empty lines from interfaces
Finally, the following changes are made to the gofumpt tool:
* Initial support for Go 1.18's type parameters is added
* The `-r` flag is removed in favor of `gofmt -r`
* The `-s` flag is removed as it is always enabled
* Vendor directories are skipped unless given as explicit arguments
* The added rules are not applied to generated Go files
* The `format` Go API now also applies the `gofmt -s` simplification
* Add support for `//gofumpt:diagnose` comments
## [v0.1.1] - 2021-03-11
This bugfix release backports fixes for a few issues:
* Keep leading empty lines in func bodies if they help readability
* Avoid breaking comment alignment on empty field lists
* Add support for `//go-sumtype:` directives
## [v0.1.0] - 2021-01-05
This is gofumpt's first release, based on Go 1.15.x. It solidifies the features
which have worked well for over a year.
This release will be the last to include `gofumports`, the fork of `goimports`
which applies `gofumpt`'s rules on top of updating the Go import lines. Users
who were relying on `goimports` in their editors or IDEs to apply both `gofumpt`
and `goimports` in a single step should switch to gopls, the official Go
language server. It is supported by many popular editors such as VS Code and
Vim, and already bundles gofumpt support. Instructions are available [in the
README](https://github.com/mvdan/gofumpt).
`gofumports` also added maintenance work and potential confusion to end users.
In the future, there will only be one way to use `gofumpt` from the command
line. We also have a [Go API](https://pkg.go.dev/mvdan.cc/gofumpt/format) for
those building programs with gofumpt.
Finally, this release adds the `-version` flag, to print the tool's own version.
The flag will work for "master" builds too.
[v0.6.0]: https://github.com/mvdan/gofumpt/releases/tag/v0.6.0
[#271]: https://github.com/mvdan/gofumpt/issues/271
[#280]: https://github.com/mvdan/gofumpt/issues/280
[#288]: https://github.com/mvdan/gofumpt/issues/288
[v0.5.0]: https://github.com/mvdan/gofumpt/releases/tag/v0.5.0
[#235]: https://github.com/mvdan/gofumpt/issues/235
[#253]: https://github.com/mvdan/gofumpt/issues/253
[#256]: https://github.com/mvdan/gofumpt/issues/256
[#260]: https://github.com/mvdan/gofumpt/issues/260
[v0.4.0]: https://github.com/mvdan/gofumpt/releases/tag/v0.4.0
[#212]: https://github.com/mvdan/gofumpt/issues/212
[#217]: https://github.com/mvdan/gofumpt/issues/217
[v0.3.1]: https://github.com/mvdan/gofumpt/releases/tag/v0.3.1
[#208]: https://github.com/mvdan/gofumpt/issues/208
[#211]: https://github.com/mvdan/gofumpt/pull/211
[v0.3.0]: https://github.com/mvdan/gofumpt/releases/tag/v0.3.0
[v0.2.1]: https://github.com/mvdan/gofumpt/releases/tag/v0.2.1
[v0.2.0]: https://github.com/mvdan/gofumpt/releases/tag/v0.2.0
[v0.1.1]: https://github.com/mvdan/gofumpt/releases/tag/v0.1.1
[v0.1.0]: https://github.com/mvdan/gofumpt/releases/tag/v0.1.0
@@ -0,0 +1,27 @@
Copyright (c) 2019, Daniel Martí. All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following disclaimer
in the documentation and/or other materials provided with the
distribution.
* Neither the name of the copyright holder nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
@@ -0,0 +1,27 @@
Copyright (c) 2009 The Go Authors. All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following disclaimer
in the documentation and/or other materials provided with the
distribution.
* Neither the name of Google Inc. nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
@@ -0,0 +1,664 @@
# gofumpt
[![Go Reference](https://pkg.go.dev/badge/mvdan.cc/gofumpt/format.svg)](https://pkg.go.dev/mvdan.cc/gofumpt/format)
go install mvdan.cc/gofumpt@latest
Enforce a stricter format than `gofmt`, while being backwards compatible.
That is, `gofumpt` is happy with a subset of the formats that `gofmt` is happy with.
The tool is a fork of `gofmt` as of Go 1.21, and requires Go 1.20 or later.
It can be used as a drop-in replacement to format your Go code,
and running `gofmt` after `gofumpt` should produce no changes.
For example:
gofumpt -l -w .
Some of the Go source files in this repository belong to the Go project.
The project includes copies of `go/printer` and `go/doc/comment` as of Go 1.21
to ensure consistent formatting independent of what Go version is being used.
The [added formatting rules](#Added-rules) are implemented in the `format` package.
`vendor` and `testdata` directories are skipped unless given as explicit arguments.
Similarly, the added rules do not apply to generated Go files unless they are
given as explicit arguments.
Finally, note that the `-r` rewrite flag is removed in favor of `gofmt -r`,
and the `-s` flag is hidden as it is always enabled.
### Added rules
**No empty lines following an assignment operator**
<details><summary><i>Example</i></summary>
```go
func foo() {
foo :=
"bar"
}
```
```go
func foo() {
foo := "bar"
}
```
</details>
**No empty lines around function bodies**
<details><summary><i>Example</i></summary>
```go
func foo() {
println("bar")
}
```
```go
func foo() {
println("bar")
}
```
</details>
**Functions should separate `) {` where the indentation helps readability**
<details><summary><i>Example</i></summary>
```go
func foo(s string,
i int) {
println("bar")
}
// With an empty line it's slightly better, but still not great.
func bar(s string,
i int) {
println("bar")
}
```
```go
func foo(s string,
i int,
) {
println("bar")
}
// With an empty line it's slightly better, but still not great.
func bar(s string,
i int,
) {
println("bar")
}
```
</details>
**No empty lines around a lone statement (or comment) in a block**
<details><summary><i>Example</i></summary>
```go
if err != nil {
return err
}
```
```go
if err != nil {
return err
}
```
</details>
**No empty lines before a simple error check**
<details><summary><i>Example</i></summary>
```go
foo, err := processFoo()
if err != nil {
return err
}
```
```go
foo, err := processFoo()
if err != nil {
return err
}
```
</details>
**Composite literals should use newlines consistently**
<details><summary><i>Example</i></summary>
```go
// A newline before or after an element requires newlines for the opening and
// closing braces.
var ints = []int{1, 2,
3, 4}
// A newline between consecutive elements requires a newline between all
// elements.
var matrix = [][]int{
{1},
{2}, {
3,
},
}
```
```go
var ints = []int{
1, 2,
3, 4,
}
var matrix = [][]int{
{1},
{2},
{
3,
},
}
```
</details>
**Empty field lists should use a single line**
<details><summary><i>Example</i></summary>
```go
var V interface {
} = 3
type T struct {
}
func F(
)
```
```go
var V interface{} = 3
type T struct{}
func F()
```
</details>
**`std` imports must be in a separate group at the top**
<details><summary><i>Example</i></summary>
```go
import (
"foo.com/bar"
"io"
"io/ioutil"
)
```
```go
import (
"io"
"io/ioutil"
"foo.com/bar"
)
```
</details>
**Short case clauses should take a single line**
<details><summary><i>Example</i></summary>
```go
switch c {
case 'a', 'b',
'c', 'd':
}
```
```go
switch c {
case 'a', 'b', 'c', 'd':
}
```
</details>
**Multiline top-level declarations must be separated by empty lines**
<details><summary><i>Example</i></summary>
```go
func foo() {
println("multiline foo")
}
func bar() {
println("multiline bar")
}
```
```go
func foo() {
println("multiline foo")
}
func bar() {
println("multiline bar")
}
```
</details>
**Single var declarations should not be grouped with parentheses**
<details><summary><i>Example</i></summary>
```go
var (
foo = "bar"
)
```
```go
var foo = "bar"
```
</details>
**Contiguous top-level declarations should be grouped together**
<details><summary><i>Example</i></summary>
```go
var nicer = "x"
var with = "y"
var alignment = "z"
```
```go
var (
nicer = "x"
with = "y"
alignment = "z"
)
```
</details>
**Simple var-declaration statements should use short assignments**
<details><summary><i>Example</i></summary>
```go
var s = "somestring"
```
```go
s := "somestring"
```
</details>
**The `-s` code simplification flag is enabled by default**
<details><summary><i>Example</i></summary>
```go
var _ = [][]int{[]int{1}}
```
```go
var _ = [][]int{{1}}
```
</details>
**Octal integer literals should use the `0o` prefix on modules using Go 1.13 and later**
<details><summary><i>Example</i></summary>
```go
const perm = 0755
```
```go
const perm = 0o755
```
</details>
**Comments which aren't Go directives should start with a whitespace**
<details><summary><i>Example</i></summary>
```go
//go:noinline
//Foo is awesome.
func Foo() {}
```
```go
//go:noinline
// Foo is awesome.
func Foo() {}
```
</details>
**Composite literals should not have leading or trailing empty lines**
<details><summary><i>Example</i></summary>
```go
var _ = []string{
"foo",
}
var _ = map[string]string{
"foo": "bar",
}
```
```go
var _ = []string{
"foo",
}
var _ = map[string]string{
"foo": "bar",
}
```
</details>
**Field lists should not have leading or trailing empty lines**
<details><summary><i>Example</i></summary>
```go
type Person interface {
Name() string
Age() int
}
type ZeroFields struct {
// No fields are needed here.
}
```
```go
type Person interface {
Name() string
Age() int
}
type ZeroFields struct {
// No fields are needed here.
}
```
</details>
#### Extra rules behind `-extra`
**Adjacent parameters with the same type should be grouped together**
<details><summary><i>Example</i></summary>
```go
func Foo(bar string, baz string) {}
```
```go
func Foo(bar, baz string) {}
```
</details>
### Installation
`gofumpt` is a replacement for `gofmt`, so you can simply `go install` it as
described at the top of this README and use it.
When using an IDE or editor with Go integration based on `gopls`,
it's best to configure the editor to use the `gofumpt` support built into `gopls`.
The instructions below show how to set up `gofumpt` for some of the
major editors out there.
#### Visual Studio Code
Enable the language server following [the official docs](https://github.com/golang/vscode-go#readme),
and then enable gopls's `gofumpt` option. Note that VS Code will complain about
the `gopls` settings, but they will still work.
```json
"go.useLanguageServer": true,
"gopls": {
"formatting.gofumpt": true,
},
```
#### GoLand
GoLand doesn't use `gopls` so it should be configured to use `gofumpt` directly.
Once `gofumpt` is installed, follow the steps below:
- Open **Settings** (File > Settings)
- Open the **Tools** section
- Find the *File Watchers* sub-section
- Click on the `+` on the right side to add a new file watcher
- Choose *Custom Template*
When a window asks for settings, you can enter the following:
* File Types: Select all .go files
* Scope: Project Files
* Program: Select your `gofumpt` executable
* Arguments: `-w $FilePath$`
* Output path to refresh: `$FilePath$`
* Working directory: `$ProjectFileDir$`
* Environment variables: `GOROOT=$GOROOT$;GOPATH=$GOPATH$;PATH=$GoBinDirs$`
To avoid unnecessary runs, you should disable all checkboxes in the *Advanced* section.
#### Vim
The configuration depends on the plugin you are using: [vim-go](https://github.com/fatih/vim-go)
or [govim](https://github.com/govim/govim).
##### vim-go
To configure `gopls` to use `gofumpt`:
```vim
let g:go_fmt_command="gopls"
let g:go_gopls_gofumpt=1
```
##### govim
To configure `gopls` to use `gofumpt`:
```vim
call govim#config#Set("Gofumpt", 1)
```
#### Neovim
When using [`lspconfig`](https://github.com/neovim/nvim-lspconfig), pass the `gofumpt` setting to `gopls`:
```lua
require('lspconfig').gopls.setup({
settings = {
gopls = {
gofumpt = true
}
}
})
```
#### Emacs
For [lsp-mode](https://emacs-lsp.github.io/lsp-mode/) users on version 8.0.0 or higher:
```elisp
(setq lsp-go-use-gofumpt t)
```
For users of `lsp-mode` before `8.0.0`:
```elisp
(lsp-register-custom-settings
'(("gopls.gofumpt" t)))
```
For [eglot](https://github.com/joaotavora/eglot) users:
```elisp
(setq-default eglot-workspace-configuration
'((:gopls . ((gofumpt . t)))))
```
#### Helix
When using the `gopls` language server, modify the Go settings in `~/.config/helix/languages.toml`:
```toml
[language-server.gopls.config]
"formatting.gofumpt" = true
```
#### Sublime Text
With ST4, install the Sublime Text LSP extension according to [the documentation](https://github.com/sublimelsp/LSP),
and enable `gopls`'s `gofumpt` option in the LSP package settings,
including setting `lsp_format_on_save` to `true`.
```json
"lsp_format_on_save": true,
"clients":
{
"gopls":
{
"enabled": true,
"initializationOptions": {
"gofumpt": true,
}
}
}
```
### Roadmap
This tool is a place to experiment. In the long term, the features that work
well might be proposed for `gofmt` itself.
The tool is also compatible with `gofmt` and is aimed to be stable, so you can
rely on it for your code as long as you pin a version of it.
### Frequently Asked Questions
> Why attempt to replace `gofmt` instead of building on top of it?
Our design is to build on top of `gofmt`, and we'll never add rules which
disagree with its formatting. So we extend `gofmt` rather than compete with it.
The tool is a modified copy of `gofmt`, for the purpose of allowing its use as a
drop-in replacement in editors and scripts.
> Why are my module imports being grouped with standard library imports?
Any import paths that don't start with a domain name like `foo.com` are
effectively [reserved by the Go toolchain](https://github.com/golang/go/issues/32819).
Third party modules should either start with a domain name,
even a local one like `foo.local`, or use [a reserved path prefix](https://github.com/golang/go/issues/37641).
For backwards compatibility with modules set up before these rules were clear,
`gofumpt` will treat any import path sharing a prefix with the current module
path as third party. For example, if the current module is `mycorp/mod1`, then
all import paths in `mycorp/...` will be considered third party.
> How can I use `gofumpt` if I already use `goimports` to replace `gofmt`?
Most editors have replaced the `goimports` program with the same functionality
provided by a language server like `gopls`. This mechanism is significantly
faster and more powerful, since the language server has more information that is
kept up to date, necessary to add missing imports.
As such, the general recommendation is to let your editor fix your imports -
either via `gopls`, such as VSCode or vim-go, or via their own custom
implementation, such as GoLand. Then follow the install instructions above to
enable the use of `gofumpt` instead of `gofmt`.
If you want to avoid integrating with `gopls`, and are OK with the overhead of
calling `goimports` from scratch on each save, you should be able to call both
tools; for example, `goimports file.go && gofumpt file.go`.
### Contributing
Issues and pull requests are welcome! Please open an issue to discuss a feature
before sending a pull request.
We also use the `#gofumpt` channel over at the
[Gophers Slack](https://invite.slack.golangbridge.org/) to chat.
When reporting a formatting bug, insert a `//gofumpt:diagnose` comment.
The comment will be rewritten to include useful debugging information.
For instance:
```
$ cat f.go
package p
//gofumpt:diagnose
$ gofumpt f.go
package p
//gofumpt:diagnose v0.1.1-0.20211103104632-bdfa3b02e50a -lang=v1.16
```
### License
Note that much of the code is copied from Go's `gofmt` command. You can tell
which files originate from the Go repository from their copyright headers. Their
license file is `LICENSE.google`.
`gofumpt`'s original source files are also under the 3-clause BSD license, with
the separate file `LICENSE`.
@@ -0,0 +1,5 @@
// Copyright (c) 2023, Daniel Martí <mvdan@mvdan.cc>
// See LICENSE for licensing information
// gofumpt enforces a stricter format than gofmt, while being backwards compatible.
package main
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,38 @@
// Copyright (c) 2021, Daniel Martí <mvdan@mvdan.cc>
// See LICENSE for licensing information
package format_test
import (
"testing"
qt "github.com/frankban/quicktest"
"mvdan.cc/gofumpt/format"
)
func TestSourceIncludesSimplify(t *testing.T) {
t.Parallel()
in := []byte(`
package p
var ()
func f() {
for _ = range v {
}
}
`[1:])
want := []byte(`
package p
func f() {
for range v {
}
}
`[1:])
got, err := format.Source(in, format.Options{})
qt.Assert(t, err, qt.IsNil)
qt.Assert(t, string(got), qt.Equals, string(want))
}
@@ -0,0 +1,64 @@
// Copyright (c) 2021, Daniel Martí <mvdan@mvdan.cc>
// See LICENSE for licensing information
package format
import (
"errors"
"fmt"
"go/scanner"
"path/filepath"
"strings"
"testing"
qt "github.com/frankban/quicktest"
"golang.org/x/tools/txtar"
)
func FuzzFormat(f *testing.F) {
// Initialize the corpus with the Go files from our test scripts.
paths, err := filepath.Glob(filepath.Join("..", "testdata", "script", "*.txtar"))
qt.Assert(f, err, qt.IsNil)
qt.Assert(f, paths, qt.Not(qt.HasLen), 0)
for _, path := range paths {
archive, err := txtar.ParseFile(path)
qt.Assert(f, err, qt.IsNil)
for _, file := range archive.Files {
f.Logf("adding %s from %s", file.Name, path)
if strings.HasSuffix(file.Name, ".go") || strings.Contains(file.Name, ".go.") {
f.Add(string(file.Data), int8(18), false) // -lang=1.18
f.Add(string(file.Data), int8(1), false) // -lang=1.1
f.Add(string(file.Data), int8(18), true) // -lang=1.18 -extra
}
}
}
f.Fuzz(func(t *testing.T, src string,
majorVersion int8, // Empty version if negative, 1.N otherwise.
extraRules bool,
) {
// TODO: also fuzz Options.ModulePath
opts := Options{ExtraRules: extraRules}
if majorVersion >= 0 {
opts.LangVersion = fmt.Sprintf("1.%d", majorVersion)
}
orig := []byte(src)
formatted, err := Source(orig, opts)
if errors.As(err, &scanner.ErrorList{}) {
return // invalid syntax from parsing
}
qt.Assert(t, err, qt.IsNil)
_ = formatted
// TODO: verify that the result is idempotent
// TODO: verify that, if the input was valid Go 1.N syntax,
// so is the output (how? go/parser lacks an option)
// TODO: check calling format.Node directly as well
qt.Assert(t, string(orig), qt.Equals, src,
qt.Commentf("input source bytes were modified"))
})
}
@@ -0,0 +1,113 @@
// Copyright 2009 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package format
import (
"go/ast"
"go/token"
"reflect"
"unicode"
"unicode/utf8"
)
// Values/types for special cases.
var (
identType = reflect.TypeOf((*ast.Ident)(nil))
objectPtrType = reflect.TypeOf((*ast.Object)(nil))
positionType = reflect.TypeOf(token.NoPos)
callExprType = reflect.TypeOf((*ast.CallExpr)(nil))
)
func isWildcard(s string) bool {
rune, size := utf8.DecodeRuneInString(s)
return size == len(s) && unicode.IsLower(rune)
}
// match reports whether pattern matches val,
// recording wildcard submatches in m.
// If m == nil, match checks whether pattern == val.
func match(m map[string]reflect.Value, pattern, val reflect.Value) bool {
// Wildcard matches any expression. If it appears multiple
// times in the pattern, it must match the same expression
// each time.
if m != nil && pattern.IsValid() && pattern.Type() == identType {
name := pattern.Interface().(*ast.Ident).Name
if isWildcard(name) && val.IsValid() {
// wildcards only match valid (non-nil) expressions.
if _, ok := val.Interface().(ast.Expr); ok && !val.IsNil() {
if old, ok := m[name]; ok {
return match(nil, old, val)
}
m[name] = val
return true
}
}
}
// Otherwise, pattern and val must match recursively.
if !pattern.IsValid() || !val.IsValid() {
return !pattern.IsValid() && !val.IsValid()
}
if pattern.Type() != val.Type() {
return false
}
// Special cases.
switch pattern.Type() {
case identType:
// For identifiers, only the names need to match
// (and none of the other *ast.Object information).
// This is a common case, handle it all here instead
// of recursing down any further via reflection.
p := pattern.Interface().(*ast.Ident)
v := val.Interface().(*ast.Ident)
return p == nil && v == nil || p != nil && v != nil && p.Name == v.Name
case objectPtrType, positionType:
// object pointers and token positions always match
return true
case callExprType:
// For calls, the Ellipsis fields (token.Position) must
// match since that is how f(x) and f(x...) are different.
// Check them here but fall through for the remaining fields.
p := pattern.Interface().(*ast.CallExpr)
v := val.Interface().(*ast.CallExpr)
if p.Ellipsis.IsValid() != v.Ellipsis.IsValid() {
return false
}
}
p := reflect.Indirect(pattern)
v := reflect.Indirect(val)
if !p.IsValid() || !v.IsValid() {
return !p.IsValid() && !v.IsValid()
}
switch p.Kind() {
case reflect.Slice:
if p.Len() != v.Len() {
return false
}
for i := 0; i < p.Len(); i++ {
if !match(m, p.Index(i), v.Index(i)) {
return false
}
}
return true
case reflect.Struct:
for i := 0; i < p.NumField(); i++ {
if !match(m, p.Field(i), v.Field(i)) {
return false
}
}
return true
case reflect.Interface:
return match(m, p.Elem(), v.Elem())
}
// Handle token integers, etc.
return p.Interface() == v.Interface()
}
@@ -0,0 +1,169 @@
// Copyright 2010 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package format
import (
"go/ast"
"go/token"
"reflect"
)
type simplifier struct{}
func (s simplifier) Visit(node ast.Node) ast.Visitor {
switch n := node.(type) {
case *ast.CompositeLit:
// array, slice, and map composite literals may be simplified
outer := n
var keyType, eltType ast.Expr
switch typ := outer.Type.(type) {
case *ast.ArrayType:
eltType = typ.Elt
case *ast.MapType:
keyType = typ.Key
eltType = typ.Value
}
if eltType != nil {
var ktyp reflect.Value
if keyType != nil {
ktyp = reflect.ValueOf(keyType)
}
typ := reflect.ValueOf(eltType)
for i, x := range outer.Elts {
px := &outer.Elts[i]
// look at value of indexed/named elements
if t, ok := x.(*ast.KeyValueExpr); ok {
if keyType != nil {
s.simplifyLiteral(ktyp, keyType, t.Key, &t.Key)
}
x = t.Value
px = &t.Value
}
s.simplifyLiteral(typ, eltType, x, px)
}
// node was simplified - stop walk (there are no subnodes to simplify)
return nil
}
case *ast.SliceExpr:
// a slice expression of the form: s[a:len(s)]
// can be simplified to: s[a:]
// if s is "simple enough" (for now we only accept identifiers)
//
// Note: This may not be correct because len may have been redeclared in
// the same package. However, this is extremely unlikely and so far
// (April 2022, after years of supporting this rewrite feature)
// has never come up, so let's keep it working as is (see also #15153).
//
// Also note that this code used to use go/ast's object tracking,
// which was removed in exchange for go/parser.Mode.SkipObjectResolution.
// False positives are extremely unlikely as described above,
// and go/ast's object tracking is incomplete in any case.
if n.Max != nil {
// - 3-index slices always require the 2nd and 3rd index
break
}
if s, _ := n.X.(*ast.Ident); s != nil {
// the array/slice object is a single identifier
if call, _ := n.High.(*ast.CallExpr); call != nil && len(call.Args) == 1 && !call.Ellipsis.IsValid() {
// the high expression is a function call with a single argument
if fun, _ := call.Fun.(*ast.Ident); fun != nil && fun.Name == "len" {
// the function called is "len"
if arg, _ := call.Args[0].(*ast.Ident); arg != nil && arg.Name == s.Name {
// the len argument is the array/slice object
n.High = nil
}
}
}
}
// Note: We could also simplify slice expressions of the form s[0:b] to s[:b]
// but we leave them as is since sometimes we want to be very explicit
// about the lower bound.
// An example where the 0 helps:
// x, y, z := b[0:2], b[2:4], b[4:6]
// An example where it does not:
// x, y := b[:n], b[n:]
case *ast.RangeStmt:
// - a range of the form: for x, _ = range v {...}
// can be simplified to: for x = range v {...}
// - a range of the form: for _ = range v {...}
// can be simplified to: for range v {...}
if isBlank(n.Value) {
n.Value = nil
}
if isBlank(n.Key) && n.Value == nil {
n.Key = nil
}
}
return s
}
func (s simplifier) simplifyLiteral(typ reflect.Value, astType, x ast.Expr, px *ast.Expr) {
ast.Walk(s, x) // simplify x
// if the element is a composite literal and its literal type
// matches the outer literal's element type exactly, the inner
// literal type may be omitted
if inner, ok := x.(*ast.CompositeLit); ok {
if match(nil, typ, reflect.ValueOf(inner.Type)) {
inner.Type = nil
}
}
// if the outer literal's element type is a pointer type *T
// and the element is & of a composite literal of type T,
// the inner &T may be omitted.
if ptr, ok := astType.(*ast.StarExpr); ok {
if addr, ok := x.(*ast.UnaryExpr); ok && addr.Op == token.AND {
if inner, ok := addr.X.(*ast.CompositeLit); ok {
if match(nil, reflect.ValueOf(ptr.X), reflect.ValueOf(inner.Type)) {
inner.Type = nil // drop T
*px = inner // drop &
}
}
}
}
}
func isBlank(x ast.Expr) bool {
ident, ok := x.(*ast.Ident)
return ok && ident.Name == "_"
}
func simplify(f *ast.File) {
// remove empty declarations such as "const ()", etc
removeEmptyDeclGroups(f)
var s simplifier
ast.Walk(s, f)
}
func removeEmptyDeclGroups(f *ast.File) {
i := 0
for _, d := range f.Decls {
if g, ok := d.(*ast.GenDecl); !ok || !isEmpty(f, g) {
f.Decls[i] = d
i++
}
}
f.Decls = f.Decls[:i]
}
func isEmpty(f *ast.File, g *ast.GenDecl) bool {
if g.Doc != nil || g.Specs != nil {
return false
}
for _, c := range f.Comments {
// if there is a comment in the declaration, it is not considered empty
if g.Pos() <= c.Pos() && c.End() <= g.End() {
return false
}
}
return true
}
@@ -0,0 +1,4 @@
go test fuzz v1
string("package A\nfunc A000000000(A000000000000,\nA00000000)(){\"\"}")
int8(62)
bool(true)
@@ -0,0 +1,4 @@
go test fuzz v1
string("package A\nvar A A\nvar A A")
int8(18)
bool(false)
@@ -0,0 +1,89 @@
// Copyright (c) 2019, Daniel Martí <mvdan@mvdan.cc>
// See LICENSE for licensing information
//go:build ignore
package main
import (
"bytes"
"encoding/json"
"io"
"os"
"os/exec"
"path"
"path/filepath"
"strings"
)
var (
modulePath = "mvdan.cc/gofumpt"
vendorDir = filepath.Join("internal", "govendor")
)
// All the packages which affect the formatting behavior.
var toVendor = []string{
"go/format",
"go/printer",
"go/doc/comment",
"internal/diff",
}
func main() {
catch(os.RemoveAll(vendorDir))
catch(os.MkdirAll(vendorDir, 0o777))
out, err := exec.Command("go", "env", "GOVERSION").Output()
catch(err)
catch(os.WriteFile(filepath.Join(vendorDir, "version.txt"), out, 0o666))
oldnew := []string{
"//go:generate", "//disabled go:generate",
}
for _, pkgPath := range toVendor {
oldnew = append(oldnew, pkgPath, path.Join(modulePath, vendorDir, pkgPath))
}
replacer := strings.NewReplacer(oldnew...)
listArgs := append([]string{"list", "-json"}, toVendor...)
out, err = exec.Command("go", listArgs...).Output()
catch(err)
type Package struct {
Dir string
ImportPath string
GoFiles []string
}
dec := json.NewDecoder(bytes.NewReader(out))
for {
var pkg Package
err := dec.Decode(&pkg)
if err == io.EOF {
break
}
catch(err)
// Otherwise we can't import it.
dstPkg := strings.TrimPrefix(pkg.ImportPath, "internal/")
dstDir := filepath.Join(vendorDir, filepath.FromSlash(dstPkg))
catch(os.MkdirAll(dstDir, 0o777))
// TODO: if the packages start using build tags like GOOS or GOARCH,
// we will need to vendor IgnoredGoFiles as well.
for _, goFile := range pkg.GoFiles {
srcBytes, err := os.ReadFile(filepath.Join(pkg.Dir, goFile))
catch(err)
src := replacer.Replace(string(srcBytes))
dst := filepath.Join(dstDir, goFile)
catch(os.WriteFile(dst, []byte(src), 0o666))
}
}
}
func catch(err error) {
if err != nil {
panic(err)
}
}
+18
View File
@@ -0,0 +1,18 @@
module mvdan.cc/gofumpt
go 1.20
require (
github.com/frankban/quicktest v1.14.6
github.com/google/go-cmp v0.6.0
github.com/rogpeppe/go-internal v1.12.0
golang.org/x/mod v0.14.0
golang.org/x/sync v0.6.0
golang.org/x/sys v0.16.0
golang.org/x/tools v0.17.0
)
require (
github.com/kr/pretty v0.3.1 // indirect
github.com/kr/text v0.2.0 // indirect
)
+22
View File
@@ -0,0 +1,22 @@
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8=
github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0=
github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e/go.mod h1:pJLUxLENpZxwdsKMEsNbx1VGcRFpLqf3715MtcvvzbA=
github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs=
github.com/rogpeppe/go-internal v1.12.0 h1:exVL4IDcn6na9z1rAb56Vxr+CgyK3nn3O+epU5NdKM8=
github.com/rogpeppe/go-internal v1.12.0/go.mod h1:E+RYuTGaKKdloAfM02xzb0FW3Paa99yedzYV+kq4uf4=
golang.org/x/mod v0.14.0 h1:dGoOF9QVLYng8IHTm7BAyWqCqSheQ5pYWGhzW00YJr0=
golang.org/x/mod v0.14.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/sync v0.6.0 h1:5BMeUDZ7vkXGfEr1x9B4bRcTH4lpkTkpdh0T/J+qjbQ=
golang.org/x/sync v0.6.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sys v0.16.0 h1:xWw16ngr6ZMtmxDyKyIgsE93KNKz5HKmMa3b8ALHidU=
golang.org/x/sys v0.16.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/tools v0.17.0 h1:FvmRgNOcs3kOa+T20R1uhfP9F6HgG2mfxDv1vrx1Htc=
golang.org/x/tools v0.17.0/go.mod h1:xsh6VxdV005rRVaS6SSAf9oiAqljS7UZUacMZ8Bnsps=
+630
View File
@@ -0,0 +1,630 @@
// Copyright 2009 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package main
import (
"bytes"
"context"
"encoding/json"
"flag"
"fmt"
"go/ast"
"go/parser"
"go/scanner"
"go/token"
"io"
"io/fs"
"os"
"os/exec"
"path/filepath"
"regexp"
"runtime"
"runtime/pprof"
"strings"
"sync"
"golang.org/x/sync/semaphore"
gformat "mvdan.cc/gofumpt/format"
"mvdan.cc/gofumpt/internal/govendor/diff"
"mvdan.cc/gofumpt/internal/govendor/go/printer"
gversion "mvdan.cc/gofumpt/internal/version"
)
//go:generate go run gen_govendor.go
//go:generate go run . -w internal/govendor
var (
// main operation modes
list = flag.Bool("l", false, "")
write = flag.Bool("w", false, "")
doDiff = flag.Bool("d", false, "")
allErrors = flag.Bool("e", false, "")
// debugging
cpuprofile = flag.String("cpuprofile", "", "")
// gofumpt's own flags
langVersion = flag.String("lang", "", "")
modulePath = flag.String("modpath", "", "")
extraRules = flag.Bool("extra", false, "")
showVersion = flag.Bool("version", false, "")
// DEPRECATED
rewriteRule = flag.String("r", "", "")
simplifyAST = flag.Bool("s", false, "")
)
var version = ""
// Keep these in sync with go/format/format.go.
const (
tabWidth = 8
printerMode = printer.UseSpaces | printer.TabIndent | printerNormalizeNumbers
// printerNormalizeNumbers means to canonicalize number literal prefixes
// and exponents while printing. See https://golang.org/doc/go1.13#gofmt.
//
// This value is defined in go/printer specifically for go/format and cmd/gofmt.
printerNormalizeNumbers = 1 << 30
)
// fdSem guards the number of concurrently-open file descriptors.
//
// For now, this is arbitrarily set to 200, based on the observation that many
// platforms default to a kernel limit of 256. Ideally, perhaps we should derive
// it from rlimit on platforms that support that system call.
//
// File descriptors opened from outside of this package are not tracked,
// so this limit may be approximate.
var fdSem = make(chan bool, 200)
var (
fileSet = token.NewFileSet() // per process FileSet
parserMode parser.Mode
)
func usage() {
fmt.Fprintf(os.Stderr, `usage: gofumpt [flags] [path ...]
-version show version and exit
-d display diffs instead of rewriting files
-e report all errors (not just the first 10 on different lines)
-l list files whose formatting differs from gofumpt's
-w write result to (source) file instead of stdout
-extra enable extra rules which should be vetted by a human
-lang str target Go version in the form "1.X" (default from go.mod)
-modpath str Go module path containing the source file (default from go.mod)
`)
}
func initParserMode() {
parserMode = parser.ParseComments | parser.SkipObjectResolution
if *allErrors {
parserMode |= parser.AllErrors
}
}
func isGoFile(f fs.DirEntry) bool {
// ignore non-Go files
name := f.Name()
return !strings.HasPrefix(name, ".") && strings.HasSuffix(name, ".go") && !f.IsDir()
}
var rxCodeGenerated = regexp.MustCompile(`^// Code generated .* DO NOT EDIT\.$`)
func isGenerated(file *ast.File) bool {
for _, cg := range file.Comments {
if cg.Pos() > file.Package {
return false
}
for _, line := range cg.List {
if rxCodeGenerated.MatchString(line.Text) {
return true
}
}
}
return false
}
// A sequencer performs concurrent tasks that may write output, but emits that
// output in a deterministic order.
type sequencer struct {
maxWeight int64
sem *semaphore.Weighted // weighted by input bytes (an approximate proxy for memory overhead)
prev <-chan *reporterState // 1-buffered
}
// newSequencer returns a sequencer that allows concurrent tasks up to maxWeight
// and writes tasks' output to out and err.
func newSequencer(maxWeight int64, out, err io.Writer) *sequencer {
sem := semaphore.NewWeighted(maxWeight)
prev := make(chan *reporterState, 1)
prev <- &reporterState{out: out, err: err}
return &sequencer{
maxWeight: maxWeight,
sem: sem,
prev: prev,
}
}
// exclusive is a weight that can be passed to a sequencer to cause
// a task to be executed without any other concurrent tasks.
const exclusive = -1
// Add blocks until the sequencer has enough weight to spare, then adds f as a
// task to be executed concurrently.
//
// If the weight is either negative or larger than the sequencer's maximum
// weight, Add blocks until all other tasks have completed, then the task
// executes exclusively (blocking all other calls to Add until it completes).
//
// f may run concurrently in a goroutine, but its output to the passed-in
// reporter will be sequential relative to the other tasks in the sequencer.
//
// If f invokes a method on the reporter, execution of that method may block
// until the previous task has finished. (To maximize concurrency, f should
// avoid invoking the reporter until it has finished any parallelizable work.)
//
// If f returns a non-nil error, that error will be reported after f's output
// (if any) and will cause a nonzero final exit code.
func (s *sequencer) Add(weight int64, f func(*reporter) error) {
if weight < 0 || weight > s.maxWeight {
weight = s.maxWeight
}
if err := s.sem.Acquire(context.TODO(), weight); err != nil {
// Change the task from "execute f" to "report err".
weight = 0
f = func(*reporter) error { return err }
}
r := &reporter{prev: s.prev}
next := make(chan *reporterState, 1)
s.prev = next
// Start f in parallel: it can run until it invokes a method on r, at which
// point it will block until the previous task releases the output state.
go func() {
if err := f(r); err != nil {
r.Report(err)
}
next <- r.getState() // Release the next task.
s.sem.Release(weight)
}()
}
// AddReport prints an error to s after the output of any previously-added
// tasks, causing the final exit code to be nonzero.
func (s *sequencer) AddReport(err error) {
s.Add(0, func(*reporter) error { return err })
}
// GetExitCode waits for all previously-added tasks to complete, then returns an
// exit code for the sequence suitable for passing to os.Exit.
func (s *sequencer) GetExitCode() int {
c := make(chan int, 1)
s.Add(0, func(r *reporter) error {
c <- r.ExitCode()
return nil
})
return <-c
}
// A reporter reports output, warnings, and errors.
type reporter struct {
prev <-chan *reporterState
state *reporterState
}
// reporterState carries the state of a reporter instance.
//
// Only one reporter at a time may have access to a reporterState.
type reporterState struct {
out, err io.Writer
exitCode int
}
// getState blocks until any prior reporters are finished with the reporter
// state, then returns the state for manipulation.
func (r *reporter) getState() *reporterState {
if r.state == nil {
r.state = <-r.prev
}
return r.state
}
// Warnf emits a warning message to the reporter's error stream,
// without changing its exit code.
func (r *reporter) Warnf(format string, args ...any) {
fmt.Fprintf(r.getState().err, format, args...)
}
// Write emits a slice to the reporter's output stream.
//
// Any error is returned to the caller, and does not otherwise affect the
// reporter's exit code.
func (r *reporter) Write(p []byte) (int, error) {
return r.getState().out.Write(p)
}
// Report emits a non-nil error to the reporter's error stream,
// changing its exit code to a nonzero value.
func (r *reporter) Report(err error) {
if err == nil {
panic("Report with nil error")
}
st := r.getState()
scanner.PrintError(st.err, err)
st.exitCode = 2
}
func (r *reporter) ExitCode() int {
return r.getState().exitCode
}
// If info == nil, we are formatting stdin instead of a file.
// If in == nil, the source is the contents of the file with the given filename.
func processFile(filename string, info fs.FileInfo, in io.Reader, r *reporter, explicit bool) error {
src, err := readFile(filename, info, in)
if err != nil {
return err
}
fileSet := token.NewFileSet()
fragmentOk := false
if info == nil {
// If we are formatting stdin, we accept a program fragment in lieu of a
// complete source file.
fragmentOk = true
}
file, sourceAdj, indentAdj, err := parse(fileSet, filename, src, fragmentOk)
if err != nil {
return err
}
ast.SortImports(fileSet, file)
// Apply gofumpt's changes before we print the code in gofumpt's format.
// If either -lang or -modpath aren't set, fetch them from go.mod.
lang := *langVersion
modpath := *modulePath
if lang == "" || modpath == "" {
dir := filepath.Dir(filename)
mod, ok := moduleCacheByDir.Load(dir)
if ok && mod != nil {
mod := mod.(*module)
if lang == "" {
lang = mod.Go
}
if modpath == "" {
modpath = mod.Module.Path
}
}
}
// We always apply the gofumpt formatting rules to explicit files, including stdin.
// Otherwise, we don't apply them on generated files.
// We also skip walking vendor directories entirely, but that happens elsewhere.
if explicit || !isGenerated(file) {
gformat.File(fileSet, file, gformat.Options{
LangVersion: lang,
ModulePath: modpath,
ExtraRules: *extraRules,
})
}
res, err := format(fileSet, file, sourceAdj, indentAdj, src, printer.Config{Mode: printerMode, Tabwidth: tabWidth})
if err != nil {
return err
}
if !bytes.Equal(src, res) {
// formatting has changed
if *list {
fmt.Fprintln(r, filename)
}
if *write {
if info == nil {
panic("-w should not have been allowed with stdin")
}
// make a temporary backup before overwriting original
perm := info.Mode().Perm()
bakname, err := backupFile(filename+".", src, perm)
if err != nil {
return err
}
fdSem <- true
err = os.WriteFile(filename, res, perm)
<-fdSem
if err != nil {
os.Rename(bakname, filename)
return err
}
err = os.Remove(bakname)
if err != nil {
return err
}
}
if *doDiff {
newName := filepath.ToSlash(filename)
oldName := newName + ".orig"
r.Write(diff.Diff(oldName, src, newName, res))
}
}
if !*list && !*write && !*doDiff {
_, err = r.Write(res)
}
return err
}
// readFile reads the contents of filename, described by info.
// If in is non-nil, readFile reads directly from it.
// Otherwise, readFile opens and reads the file itself,
// with the number of concurrently-open files limited by fdSem.
func readFile(filename string, info fs.FileInfo, in io.Reader) ([]byte, error) {
if in == nil {
fdSem <- true
var err error
f, err := os.Open(filename)
if err != nil {
return nil, err
}
in = f
defer func() {
f.Close()
<-fdSem
}()
}
// Compute the file's size and read its contents with minimal allocations.
//
// If we have the FileInfo from filepath.WalkDir, use it to make
// a buffer of the right size and avoid ReadAll's reallocations.
//
// If the size is unknown (or bogus, or overflows an int), fall back to
// a size-independent ReadAll.
size := -1
if info != nil && info.Mode().IsRegular() && int64(int(info.Size())) == info.Size() {
size = int(info.Size())
}
if size+1 <= 0 {
// The file is not known to be regular, so we don't have a reliable size for it.
var err error
src, err := io.ReadAll(in)
if err != nil {
return nil, err
}
return src, nil
}
// We try to read size+1 bytes so that we can detect modifications: if we
// read more than size bytes, then the file was modified concurrently.
// (If that happens, we could, say, append to src to finish the read, or
// proceed with a truncated buffer — but the fact that it changed at all
// indicates a possible race with someone editing the file, so we prefer to
// stop to avoid corrupting it.)
src := make([]byte, size+1)
n, err := io.ReadFull(in, src)
switch err {
case nil, io.EOF, io.ErrUnexpectedEOF:
// io.ReadFull returns io.EOF (for an empty file) or io.ErrUnexpectedEOF
// (for a non-empty file) if the file was changed unexpectedly. Continue
// with comparing file sizes in those cases.
default:
return nil, err
}
if n < size {
return nil, fmt.Errorf("error: size of %s changed during reading (from %d to %d bytes)", filename, size, n)
} else if n > size {
return nil, fmt.Errorf("error: size of %s changed during reading (from %d to >=%d bytes)", filename, size, len(src))
}
return src[:n], nil
}
func main() { os.Exit(main1()) }
func main1() int {
// Arbitrarily limit in-flight work to 2MiB times the number of threads.
//
// The actual overhead for the parse tree and output will depend on the
// specifics of the file, but this at least keeps the footprint of the process
// roughly proportional to GOMAXPROCS.
maxWeight := (2 << 20) * int64(runtime.GOMAXPROCS(0))
s := newSequencer(maxWeight, os.Stdout, os.Stderr)
// call gofmtMain in a separate function
// so that it can use defer and have them
// run before the exit.
gofmtMain(s)
return s.GetExitCode()
}
func gofmtMain(s *sequencer) {
// Ensure our parsed files never start with base 1,
// to ensure that using token.NoPos+1 will panic.
fileSet.AddFile("gofumpt_base.go", 1, 10)
flag.Usage = usage
flag.Parse()
if *simplifyAST {
fmt.Fprintf(os.Stderr, "warning: -s is deprecated as it is always enabled\n")
}
if *rewriteRule != "" {
fmt.Fprintf(os.Stderr, `the rewrite flag is no longer available; use "gofmt -r" instead`+"\n")
os.Exit(2)
}
// Print the gofumpt version if the user asks for it.
if *showVersion {
fmt.Println(gversion.String(version))
return
}
if *cpuprofile != "" {
fdSem <- true
f, err := os.Create(*cpuprofile)
if err != nil {
s.AddReport(fmt.Errorf("creating cpu profile: %s", err))
return
}
defer func() {
f.Close()
<-fdSem
}()
pprof.StartCPUProfile(f)
defer pprof.StopCPUProfile()
}
initParserMode()
args := flag.Args()
if len(args) == 0 {
if *write {
s.AddReport(fmt.Errorf("error: cannot use -w with standard input"))
return
}
s.Add(0, func(r *reporter) error {
// TODO: test explicit==true
return processFile("<standard input>", nil, os.Stdin, r, true)
})
return
}
for _, arg := range args {
switch info, err := os.Stat(arg); {
case err != nil:
s.AddReport(err)
case !info.IsDir():
// Non-directory arguments are always formatted.
arg := arg
s.Add(fileWeight(arg, info), func(r *reporter) error {
return processFile(arg, info, nil, r, true)
})
default:
// Directories are walked, ignoring non-Go files.
err := filepath.WalkDir(arg, func(path string, f fs.DirEntry, err error) error {
// vendor and testdata directories are skipped,
// unless they are explicitly passed as an argument.
base := filepath.Base(path)
if path != arg && (base == "vendor" || base == "testdata") {
return filepath.SkipDir
}
if err != nil || !isGoFile(f) {
return err
}
info, err := f.Info()
if err != nil {
s.AddReport(err)
return nil
}
s.Add(fileWeight(path, info), func(r *reporter) error {
return processFile(path, info, nil, r, false)
})
return nil
})
if err != nil {
s.AddReport(err)
}
}
}
}
type module struct {
Go string
Module struct {
Path string
}
}
func loadModuleInfo(dir string) any {
cmd := exec.Command("go", "mod", "edit", "-json")
cmd.Dir = dir
// Spawning "go mod edit" will open files by design,
// such as the named pipe to obtain stdout.
// TODO(mvdan): if we run into "too many open files" errors again in the
// future, we probably need to turn fdSem into a weighted semaphore so this
// operation can acquire a weight larger than 1.
fdSem <- true
out, err := cmd.Output()
defer func() { <-fdSem }()
if err != nil || len(out) == 0 {
return nil
}
mod := new(module)
if err := json.Unmarshal(out, mod); err != nil {
return nil
}
return mod
}
// Written to by fileWeight, read from fileWeight and processFile.
// A present but nil value means that loading the module info failed.
// Note that we don't require the keys to be absolute directories,
// so duplicates are possible. The same can happen with symlinks.
var moduleCacheByDir sync.Map // map[dirString]*module
func fileWeight(path string, info fs.FileInfo) int64 {
dir := filepath.Dir(path)
if _, ok := moduleCacheByDir.Load(dir); !ok {
moduleCacheByDir.Store(dir, loadModuleInfo(dir))
}
if info == nil {
return exclusive
}
if info.Mode().Type() == fs.ModeSymlink {
var err error
info, err = os.Stat(path)
if err != nil {
return exclusive
}
}
if !info.Mode().IsRegular() {
// For non-regular files, FileInfo.Size is system-dependent and thus not a
// reliable indicator of weight.
return exclusive
}
return info.Size()
}
const chmodSupported = runtime.GOOS != "windows"
// backupFile writes data to a new file named filename<number> with permissions perm,
// with <number randomly chosen such that the file name is unique. backupFile returns
// the chosen file name.
func backupFile(filename string, data []byte, perm fs.FileMode) (string, error) {
fdSem <- true
defer func() { <-fdSem }()
// create backup file
f, err := os.CreateTemp(filepath.Dir(filename), filepath.Base(filename))
if err != nil {
return "", err
}
bakname := f.Name()
if chmodSupported {
err = f.Chmod(perm)
if err != nil {
f.Close()
os.Remove(bakname)
return bakname, err
}
}
// write data to backup file
_, err = f.Write(data)
if err1 := f.Close(); err == nil {
err = err1
}
return bakname, err
}
@@ -0,0 +1,177 @@
// Copyright 2015 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// TODO(gri): This file and the file src/go/format/internal.go are
// the same (but for this comment and the package name). Do not modify
// one without the other. Determine if we can factor out functionality
// in a public API. See also #11844 for context.
package main
import (
"bytes"
"go/ast"
"go/parser"
"go/token"
"strings"
"mvdan.cc/gofumpt/internal/govendor/go/printer"
)
// parse parses src, which was read from the named file,
// as a Go source file, declaration, or statement list.
func parse(fset *token.FileSet, filename string, src []byte, fragmentOk bool) (
file *ast.File,
sourceAdj func(src []byte, indent int) []byte,
indentAdj int,
err error,
) {
// Try as whole source file.
file, err = parser.ParseFile(fset, filename, src, parserMode)
// If there's no error, return. If the error is that the source file didn't begin with a
// package line and source fragments are ok, fall through to
// try as a source fragment. Stop and return on any other error.
if err == nil || !fragmentOk || !strings.Contains(err.Error(), "expected 'package'") {
return
}
// If this is a declaration list, make it a source file
// by inserting a package clause.
// Insert using a ';', not a newline, so that the line numbers
// in psrc match the ones in src.
psrc := append([]byte("package p;"), src...)
file, err = parser.ParseFile(fset, filename, psrc, parserMode)
if err == nil {
sourceAdj = func(src []byte, indent int) []byte {
// Remove the package clause.
// Gofmt has turned the ';' into a '\n'.
src = src[indent+len("package p\n"):]
return bytes.TrimSpace(src)
}
return
}
// If the error is that the source file didn't begin with a
// declaration, fall through to try as a statement list.
// Stop and return on any other error.
if !strings.Contains(err.Error(), "expected declaration") {
return
}
// If this is a statement list, make it a source file
// by inserting a package clause and turning the list
// into a function body. This handles expressions too.
// Insert using a ';', not a newline, so that the line numbers
// in fsrc match the ones in src. Add an extra '\n' before the '}'
// to make sure comments are flushed before the '}'.
fsrc := append(append([]byte("package p; func _() {"), src...), '\n', '\n', '}')
file, err = parser.ParseFile(fset, filename, fsrc, parserMode)
if err == nil {
sourceAdj = func(src []byte, indent int) []byte {
// Cap adjusted indent to zero.
if indent < 0 {
indent = 0
}
// Remove the wrapping.
// Gofmt has turned the "; " into a "\n\n".
// There will be two non-blank lines with indent, hence 2*indent.
src = src[2*indent+len("package p\n\nfunc _() {"):]
// Remove only the "}\n" suffix: remaining whitespaces will be trimmed anyway
src = src[:len(src)-len("}\n")]
return bytes.TrimSpace(src)
}
// Gofmt has also indented the function body one level.
// Adjust that with indentAdj.
indentAdj = -1
}
// Succeeded, or out of options.
return
}
// format formats the given package file originally obtained from src
// and adjusts the result based on the original source via sourceAdj
// and indentAdj.
func format(
fset *token.FileSet,
file *ast.File,
sourceAdj func(src []byte, indent int) []byte,
indentAdj int,
src []byte,
cfg printer.Config,
) ([]byte, error) {
if sourceAdj == nil {
// Complete source file.
var buf bytes.Buffer
err := cfg.Fprint(&buf, fset, file)
if err != nil {
return nil, err
}
return buf.Bytes(), nil
}
// Partial source file.
// Determine and prepend leading space.
i, j := 0, 0
for j < len(src) && isSpace(src[j]) {
if src[j] == '\n' {
i = j + 1 // byte offset of last line in leading space
}
j++
}
var res []byte
res = append(res, src[:i]...)
// Determine and prepend indentation of first code line.
// Spaces are ignored unless there are no tabs,
// in which case spaces count as one tab.
indent := 0
hasSpace := false
for _, b := range src[i:j] {
switch b {
case ' ':
hasSpace = true
case '\t':
indent++
}
}
if indent == 0 && hasSpace {
indent = 1
}
for i := 0; i < indent; i++ {
res = append(res, '\t')
}
// Format the source.
// Write it without any leading and trailing space.
cfg.Indent = indent + indentAdj
var buf bytes.Buffer
err := cfg.Fprint(&buf, fset, file)
if err != nil {
return nil, err
}
out := sourceAdj(buf.Bytes(), cfg.Indent)
// If the adjusted output is empty, the source
// was empty but (possibly) for white space.
// The result is the incoming source.
if len(out) == 0 {
return src, nil
}
// Otherwise, append output to leading space.
res = append(res, out...)
// Determine and append trailing space.
i = len(src)
for i > 0 && isSpace(src[i-1]) {
i--
}
return append(res, src[i:]...), nil
}
// isSpace reports whether the byte is a space character.
// isSpace defines a space as being among the following bytes: ' ', '\t', '\n' and '\r'.
func isSpace(b byte) bool {
return b == ' ' || b == '\t' || b == '\n' || b == '\r'
}
@@ -0,0 +1,261 @@
// Copyright 2022 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package diff
import (
"bytes"
"fmt"
"sort"
"strings"
)
// A pair is a pair of values tracked for both the x and y side of a diff.
// It is typically a pair of line indexes.
type pair struct{ x, y int }
// Diff returns an anchored diff of the two texts old and new
// in the “unified diff” format. If old and new are identical,
// Diff returns a nil slice (no output).
//
// Unix diff implementations typically look for a diff with
// the smallest number of lines inserted and removed,
// which can in the worst case take time quadratic in the
// number of lines in the texts. As a result, many implementations
// either can be made to run for a long time or cut off the search
// after a predetermined amount of work.
//
// In contrast, this implementation looks for a diff with the
// smallest number of “unique” lines inserted and removed,
// where unique means a line that appears just once in both old and new.
// We call this an “anchored diff” because the unique lines anchor
// the chosen matching regions. An anchored diff is usually clearer
// than a standard diff, because the algorithm does not try to
// reuse unrelated blank lines or closing braces.
// The algorithm also guarantees to run in O(n log n) time
// instead of the standard O(n²) time.
//
// Some systems call this approach a “patience diff,” named for
// the “patience sorting” algorithm, itself named for a solitaire card game.
// We avoid that name for two reasons. First, the name has been used
// for a few different variants of the algorithm, so it is imprecise.
// Second, the name is frequently interpreted as meaning that you have
// to wait longer (to be patient) for the diff, meaning that it is a slower algorithm,
// when in fact the algorithm is faster than the standard one.
func Diff(oldName string, old []byte, newName string, new []byte) []byte {
if bytes.Equal(old, new) {
return nil
}
x := lines(old)
y := lines(new)
// Print diff header.
var out bytes.Buffer
fmt.Fprintf(&out, "diff %s %s\n", oldName, newName)
fmt.Fprintf(&out, "--- %s\n", oldName)
fmt.Fprintf(&out, "+++ %s\n", newName)
// Loop over matches to consider,
// expanding each match to include surrounding lines,
// and then printing diff chunks.
// To avoid setup/teardown cases outside the loop,
// tgs returns a leading {0,0} and trailing {len(x), len(y)} pair
// in the sequence of matches.
var (
done pair // printed up to x[:done.x] and y[:done.y]
chunk pair // start lines of current chunk
count pair // number of lines from each side in current chunk
ctext []string // lines for current chunk
)
for _, m := range tgs(x, y) {
if m.x < done.x {
// Already handled scanning forward from earlier match.
continue
}
// Expand matching lines as far possible,
// establishing that x[start.x:end.x] == y[start.y:end.y].
// Note that on the first (or last) iteration we may (or definitely do)
// have an empty match: start.x==end.x and start.y==end.y.
start := m
for start.x > done.x && start.y > done.y && x[start.x-1] == y[start.y-1] {
start.x--
start.y--
}
end := m
for end.x < len(x) && end.y < len(y) && x[end.x] == y[end.y] {
end.x++
end.y++
}
// Emit the mismatched lines before start into this chunk.
// (No effect on first sentinel iteration, when start = {0,0}.)
for _, s := range x[done.x:start.x] {
ctext = append(ctext, "-"+s)
count.x++
}
for _, s := range y[done.y:start.y] {
ctext = append(ctext, "+"+s)
count.y++
}
// If we're not at EOF and have too few common lines,
// the chunk includes all the common lines and continues.
const C = 3 // number of context lines
if (end.x < len(x) || end.y < len(y)) &&
(end.x-start.x < C || (len(ctext) > 0 && end.x-start.x < 2*C)) {
for _, s := range x[start.x:end.x] {
ctext = append(ctext, " "+s)
count.x++
count.y++
}
done = end
continue
}
// End chunk with common lines for context.
if len(ctext) > 0 {
n := end.x - start.x
if n > C {
n = C
}
for _, s := range x[start.x : start.x+n] {
ctext = append(ctext, " "+s)
count.x++
count.y++
}
done = pair{start.x + n, start.y + n}
// Format and emit chunk.
// Convert line numbers to 1-indexed.
// Special case: empty file shows up as 0,0 not 1,0.
if count.x > 0 {
chunk.x++
}
if count.y > 0 {
chunk.y++
}
fmt.Fprintf(&out, "@@ -%d,%d +%d,%d @@\n", chunk.x, count.x, chunk.y, count.y)
for _, s := range ctext {
out.WriteString(s)
}
count.x = 0
count.y = 0
ctext = ctext[:0]
}
// If we reached EOF, we're done.
if end.x >= len(x) && end.y >= len(y) {
break
}
// Otherwise start a new chunk.
chunk = pair{end.x - C, end.y - C}
for _, s := range x[chunk.x:end.x] {
ctext = append(ctext, " "+s)
count.x++
count.y++
}
done = end
}
return out.Bytes()
}
// lines returns the lines in the file x, including newlines.
// If the file does not end in a newline, one is supplied
// along with a warning about the missing newline.
func lines(x []byte) []string {
l := strings.SplitAfter(string(x), "\n")
if l[len(l)-1] == "" {
l = l[:len(l)-1]
} else {
// Treat last line as having a message about the missing newline attached,
// using the same text as BSD/GNU diff (including the leading backslash).
l[len(l)-1] += "\n\\ No newline at end of file\n"
}
return l
}
// tgs returns the pairs of indexes of the longest common subsequence
// of unique lines in x and y, where a unique line is one that appears
// once in x and once in y.
//
// The longest common subsequence algorithm is as described in
// Thomas G. Szymanski, “A Special Case of the Maximal Common
// Subsequence Problem,” Princeton TR #170 (January 1975),
// available at https://research.swtch.com/tgs170.pdf.
func tgs(x, y []string) []pair {
// Count the number of times each string appears in a and b.
// We only care about 0, 1, many, counted as 0, -1, -2
// for the x side and 0, -4, -8 for the y side.
// Using negative numbers now lets us distinguish positive line numbers later.
m := make(map[string]int)
for _, s := range x {
if c := m[s]; c > -2 {
m[s] = c - 1
}
}
for _, s := range y {
if c := m[s]; c > -8 {
m[s] = c - 4
}
}
// Now unique strings can be identified by m[s] = -1+-4.
//
// Gather the indexes of those strings in x and y, building:
// xi[i] = increasing indexes of unique strings in x.
// yi[i] = increasing indexes of unique strings in y.
// inv[i] = index j such that x[xi[i]] = y[yi[j]].
var xi, yi, inv []int
for i, s := range y {
if m[s] == -1+-4 {
m[s] = len(yi)
yi = append(yi, i)
}
}
for i, s := range x {
if j, ok := m[s]; ok && j >= 0 {
xi = append(xi, i)
inv = append(inv, j)
}
}
// Apply Algorithm A from Szymanski's paper.
// In those terms, A = J = inv and B = [0, n).
// We add sentinel pairs {0,0}, and {len(x),len(y)}
// to the returned sequence, to help the processing loop.
J := inv
n := len(xi)
T := make([]int, n)
L := make([]int, n)
for i := range T {
T[i] = n + 1
}
for i := 0; i < n; i++ {
k := sort.Search(n, func(k int) bool {
return T[k] >= J[i]
})
T[k] = J[i]
L[i] = k + 1
}
k := 0
for _, v := range L {
if k < v {
k = v
}
}
seq := make([]pair, 2+k)
seq[1+k] = pair{len(x), len(y)} // sentinel at end
lastj := n
for i := n - 1; i >= 0; i-- {
if L[i] == k && J[i] < lastj {
seq[k] = pair{xi[i], yi[J[i]]}
k--
}
}
seq[0] = pair{0, 0} // sentinel at start
return seq
}
@@ -0,0 +1,36 @@
// Copyright 2022 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
/*
Package comment implements parsing and reformatting of Go doc comments,
(documentation comments), which are comments that immediately precede
a top-level declaration of a package, const, func, type, or var.
Go doc comment syntax is a simplified subset of Markdown that supports
links, headings, paragraphs, lists (without nesting), and preformatted text blocks.
The details of the syntax are documented at https://go.dev/doc/comment.
To parse the text associated with a doc comment (after removing comment markers),
use a [Parser]:
var p comment.Parser
doc := p.Parse(text)
The result is a [*Doc].
To reformat it as a doc comment, HTML, Markdown, or plain text,
use a [Printer]:
var pr comment.Printer
os.Stdout.Write(pr.Text(doc))
The [Parser] and [Printer] types are structs whose fields can be
modified to customize the operations.
For details, see the documentation for those types.
Use cases that need additional control over reformatting can
implement their own logic by inspecting the parsed syntax itself.
See the documentation for [Doc], [Block], [Text] for an overview
and links to additional types.
*/
package comment
@@ -0,0 +1,169 @@
// Copyright 2022 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package comment
import (
"bytes"
"fmt"
"strconv"
)
// An htmlPrinter holds the state needed for printing a Doc as HTML.
type htmlPrinter struct {
*Printer
tight bool
}
// HTML returns an HTML formatting of the Doc.
// See the [Printer] documentation for ways to customize the HTML output.
func (p *Printer) HTML(d *Doc) []byte {
hp := &htmlPrinter{Printer: p}
var out bytes.Buffer
for _, x := range d.Content {
hp.block(&out, x)
}
return out.Bytes()
}
// block prints the block x to out.
func (p *htmlPrinter) block(out *bytes.Buffer, x Block) {
switch x := x.(type) {
default:
fmt.Fprintf(out, "?%T", x)
case *Paragraph:
if !p.tight {
out.WriteString("<p>")
}
p.text(out, x.Text)
out.WriteString("\n")
case *Heading:
out.WriteString("<h")
h := strconv.Itoa(p.headingLevel())
out.WriteString(h)
if id := p.headingID(x); id != "" {
out.WriteString(` id="`)
p.escape(out, id)
out.WriteString(`"`)
}
out.WriteString(">")
p.text(out, x.Text)
out.WriteString("</h")
out.WriteString(h)
out.WriteString(">\n")
case *Code:
out.WriteString("<pre>")
p.escape(out, x.Text)
out.WriteString("</pre>\n")
case *List:
kind := "ol>\n"
if x.Items[0].Number == "" {
kind = "ul>\n"
}
out.WriteString("<")
out.WriteString(kind)
next := "1"
for _, item := range x.Items {
out.WriteString("<li")
if n := item.Number; n != "" {
if n != next {
out.WriteString(` value="`)
out.WriteString(n)
out.WriteString(`"`)
next = n
}
next = inc(next)
}
out.WriteString(">")
p.tight = !x.BlankBetween()
for _, blk := range item.Content {
p.block(out, blk)
}
p.tight = false
}
out.WriteString("</")
out.WriteString(kind)
}
}
// inc increments the decimal string s.
// For example, inc("1199") == "1200".
func inc(s string) string {
b := []byte(s)
for i := len(b) - 1; i >= 0; i-- {
if b[i] < '9' {
b[i]++
return string(b)
}
b[i] = '0'
}
return "1" + string(b)
}
// text prints the text sequence x to out.
func (p *htmlPrinter) text(out *bytes.Buffer, x []Text) {
for _, t := range x {
switch t := t.(type) {
case Plain:
p.escape(out, string(t))
case Italic:
out.WriteString("<i>")
p.escape(out, string(t))
out.WriteString("</i>")
case *Link:
out.WriteString(`<a href="`)
p.escape(out, t.URL)
out.WriteString(`">`)
p.text(out, t.Text)
out.WriteString("</a>")
case *DocLink:
url := p.docLinkURL(t)
if url != "" {
out.WriteString(`<a href="`)
p.escape(out, url)
out.WriteString(`">`)
}
p.text(out, t.Text)
if url != "" {
out.WriteString("</a>")
}
}
}
}
// escape prints s to out as plain text,
// escaping < & " ' and > to avoid being misinterpreted
// in larger HTML constructs.
func (p *htmlPrinter) escape(out *bytes.Buffer, s string) {
start := 0
for i := 0; i < len(s); i++ {
switch s[i] {
case '<':
out.WriteString(s[start:i])
out.WriteString("&lt;")
start = i + 1
case '&':
out.WriteString(s[start:i])
out.WriteString("&amp;")
start = i + 1
case '"':
out.WriteString(s[start:i])
out.WriteString("&quot;")
start = i + 1
case '\'':
out.WriteString(s[start:i])
out.WriteString("&apos;")
start = i + 1
case '>':
out.WriteString(s[start:i])
out.WriteString("&gt;")
start = i + 1
}
}
out.WriteString(s[start:])
}
@@ -0,0 +1,188 @@
// Copyright 2022 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package comment
import (
"bytes"
"fmt"
"strings"
)
// An mdPrinter holds the state needed for printing a Doc as Markdown.
type mdPrinter struct {
*Printer
headingPrefix string
raw bytes.Buffer
}
// Markdown returns a Markdown formatting of the Doc.
// See the [Printer] documentation for ways to customize the Markdown output.
func (p *Printer) Markdown(d *Doc) []byte {
mp := &mdPrinter{
Printer: p,
headingPrefix: strings.Repeat("#", p.headingLevel()) + " ",
}
var out bytes.Buffer
for i, x := range d.Content {
if i > 0 {
out.WriteByte('\n')
}
mp.block(&out, x)
}
return out.Bytes()
}
// block prints the block x to out.
func (p *mdPrinter) block(out *bytes.Buffer, x Block) {
switch x := x.(type) {
default:
fmt.Fprintf(out, "?%T", x)
case *Paragraph:
p.text(out, x.Text)
out.WriteString("\n")
case *Heading:
out.WriteString(p.headingPrefix)
p.text(out, x.Text)
if id := p.headingID(x); id != "" {
out.WriteString(" {#")
out.WriteString(id)
out.WriteString("}")
}
out.WriteString("\n")
case *Code:
md := x.Text
for md != "" {
var line string
line, md, _ = strings.Cut(md, "\n")
if line != "" {
out.WriteString("\t")
out.WriteString(line)
}
out.WriteString("\n")
}
case *List:
loose := x.BlankBetween()
for i, item := range x.Items {
if i > 0 && loose {
out.WriteString("\n")
}
if n := item.Number; n != "" {
out.WriteString(" ")
out.WriteString(n)
out.WriteString(". ")
} else {
out.WriteString(" - ") // SP SP - SP
}
for i, blk := range item.Content {
const fourSpace = " "
if i > 0 {
out.WriteString("\n" + fourSpace)
}
p.text(out, blk.(*Paragraph).Text)
out.WriteString("\n")
}
}
}
}
// text prints the text sequence x to out.
func (p *mdPrinter) text(out *bytes.Buffer, x []Text) {
p.raw.Reset()
p.rawText(&p.raw, x)
line := bytes.TrimSpace(p.raw.Bytes())
if len(line) == 0 {
return
}
switch line[0] {
case '+', '-', '*', '#':
// Escape what would be the start of an unordered list or heading.
out.WriteByte('\\')
case '0', '1', '2', '3', '4', '5', '6', '7', '8', '9':
i := 1
for i < len(line) && '0' <= line[i] && line[i] <= '9' {
i++
}
if i < len(line) && (line[i] == '.' || line[i] == ')') {
// Escape what would be the start of an ordered list.
out.Write(line[:i])
out.WriteByte('\\')
line = line[i:]
}
}
out.Write(line)
}
// rawText prints the text sequence x to out,
// without worrying about escaping characters
// that have special meaning at the start of a Markdown line.
func (p *mdPrinter) rawText(out *bytes.Buffer, x []Text) {
for _, t := range x {
switch t := t.(type) {
case Plain:
p.escape(out, string(t))
case Italic:
out.WriteString("*")
p.escape(out, string(t))
out.WriteString("*")
case *Link:
out.WriteString("[")
p.rawText(out, t.Text)
out.WriteString("](")
out.WriteString(t.URL)
out.WriteString(")")
case *DocLink:
url := p.docLinkURL(t)
if url != "" {
out.WriteString("[")
}
p.rawText(out, t.Text)
if url != "" {
out.WriteString("](")
url = strings.ReplaceAll(url, "(", "%28")
url = strings.ReplaceAll(url, ")", "%29")
out.WriteString(url)
out.WriteString(")")
}
}
}
}
// escape prints s to out as plain text,
// escaping special characters to avoid being misinterpreted
// as Markdown markup sequences.
func (p *mdPrinter) escape(out *bytes.Buffer, s string) {
start := 0
for i := 0; i < len(s); i++ {
switch s[i] {
case '\n':
// Turn all \n into spaces, for a few reasons:
// - Avoid introducing paragraph breaks accidentally.
// - Avoid the need to reindent after the newline.
// - Avoid problems with Markdown renderers treating
// every mid-paragraph newline as a <br>.
out.WriteString(s[start:i])
out.WriteByte(' ')
start = i + 1
continue
case '`', '_', '*', '[', '<', '\\':
// Not all of these need to be escaped all the time,
// but is valid and easy to do so.
// We assume the Markdown is being passed to a
// Markdown renderer, not edited by a person,
// so it's fine to have escapes that are not strictly
// necessary in some cases.
out.WriteString(s[start:i])
out.WriteByte('\\')
out.WriteByte(s[i])
start = i + 1
}
}
out.WriteString(s[start:])
}
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,288 @@
// Copyright 2022 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package comment
import (
"bytes"
"fmt"
"strings"
)
// A Printer is a doc comment printer.
// The fields in the struct can be filled in before calling
// any of the printing methods
// in order to customize the details of the printing process.
type Printer struct {
// HeadingLevel is the nesting level used for
// HTML and Markdown headings.
// If HeadingLevel is zero, it defaults to level 3,
// meaning to use <h3> and ###.
HeadingLevel int
// HeadingID is a function that computes the heading ID
// (anchor tag) to use for the heading h when generating
// HTML and Markdown. If HeadingID returns an empty string,
// then the heading ID is omitted.
// If HeadingID is nil, h.DefaultID is used.
HeadingID func(h *Heading) string
// DocLinkURL is a function that computes the URL for the given DocLink.
// If DocLinkURL is nil, then link.DefaultURL(p.DocLinkBaseURL) is used.
DocLinkURL func(link *DocLink) string
// DocLinkBaseURL is used when DocLinkURL is nil,
// passed to [DocLink.DefaultURL] to construct a DocLink's URL.
// See that method's documentation for details.
DocLinkBaseURL string
// TextPrefix is a prefix to print at the start of every line
// when generating text output using the Text method.
TextPrefix string
// TextCodePrefix is the prefix to print at the start of each
// preformatted (code block) line when generating text output,
// instead of (not in addition to) TextPrefix.
// If TextCodePrefix is the empty string, it defaults to TextPrefix+"\t".
TextCodePrefix string
// TextWidth is the maximum width text line to generate,
// measured in Unicode code points,
// excluding TextPrefix and the newline character.
// If TextWidth is zero, it defaults to 80 minus the number of code points in TextPrefix.
// If TextWidth is negative, there is no limit.
TextWidth int
}
func (p *Printer) headingLevel() int {
if p.HeadingLevel <= 0 {
return 3
}
return p.HeadingLevel
}
func (p *Printer) headingID(h *Heading) string {
if p.HeadingID == nil {
return h.DefaultID()
}
return p.HeadingID(h)
}
func (p *Printer) docLinkURL(link *DocLink) string {
if p.DocLinkURL != nil {
return p.DocLinkURL(link)
}
return link.DefaultURL(p.DocLinkBaseURL)
}
// DefaultURL constructs and returns the documentation URL for l,
// using baseURL as a prefix for links to other packages.
//
// The possible forms returned by DefaultURL are:
// - baseURL/ImportPath, for a link to another package
// - baseURL/ImportPath#Name, for a link to a const, func, type, or var in another package
// - baseURL/ImportPath#Recv.Name, for a link to a method in another package
// - #Name, for a link to a const, func, type, or var in this package
// - #Recv.Name, for a link to a method in this package
//
// If baseURL ends in a trailing slash, then DefaultURL inserts
// a slash between ImportPath and # in the anchored forms.
// For example, here are some baseURL values and URLs they can generate:
//
// "/pkg/" → "/pkg/math/#Sqrt"
// "/pkg" → "/pkg/math#Sqrt"
// "/" → "/math/#Sqrt"
// "" → "/math#Sqrt"
func (l *DocLink) DefaultURL(baseURL string) string {
if l.ImportPath != "" {
slash := ""
if strings.HasSuffix(baseURL, "/") {
slash = "/"
} else {
baseURL += "/"
}
switch {
case l.Name == "":
return baseURL + l.ImportPath + slash
case l.Recv != "":
return baseURL + l.ImportPath + slash + "#" + l.Recv + "." + l.Name
default:
return baseURL + l.ImportPath + slash + "#" + l.Name
}
}
if l.Recv != "" {
return "#" + l.Recv + "." + l.Name
}
return "#" + l.Name
}
// DefaultID returns the default anchor ID for the heading h.
//
// The default anchor ID is constructed by converting every
// rune that is not alphanumeric ASCII to an underscore
// and then adding the prefix “hdr-”.
// For example, if the heading text is “Go Doc Comments”,
// the default ID is “hdr-Go_Doc_Comments”.
func (h *Heading) DefaultID() string {
// Note: The “hdr-” prefix is important to avoid DOM clobbering attacks.
// See https://pkg.go.dev/github.com/google/safehtml#Identifier.
var out strings.Builder
var p textPrinter
p.oneLongLine(&out, h.Text)
s := strings.TrimSpace(out.String())
if s == "" {
return ""
}
out.Reset()
out.WriteString("hdr-")
for _, r := range s {
if r < 0x80 && isIdentASCII(byte(r)) {
out.WriteByte(byte(r))
} else {
out.WriteByte('_')
}
}
return out.String()
}
type commentPrinter struct {
*Printer
}
// Comment returns the standard Go formatting of the Doc,
// without any comment markers.
func (p *Printer) Comment(d *Doc) []byte {
cp := &commentPrinter{Printer: p}
var out bytes.Buffer
for i, x := range d.Content {
if i > 0 && blankBefore(x) {
out.WriteString("\n")
}
cp.block(&out, x)
}
// Print one block containing all the link definitions that were used,
// and then a second block containing all the unused ones.
// This makes it easy to clean up the unused ones: gofmt and
// delete the final block. And it's a nice visual signal without
// affecting the way the comment formats for users.
for i := 0; i < 2; i++ {
used := i == 0
first := true
for _, def := range d.Links {
if def.Used == used {
if first {
out.WriteString("\n")
first = false
}
out.WriteString("[")
out.WriteString(def.Text)
out.WriteString("]: ")
out.WriteString(def.URL)
out.WriteString("\n")
}
}
}
return out.Bytes()
}
// blankBefore reports whether the block x requires a blank line before it.
// All blocks do, except for Lists that return false from x.BlankBefore().
func blankBefore(x Block) bool {
if x, ok := x.(*List); ok {
return x.BlankBefore()
}
return true
}
// block prints the block x to out.
func (p *commentPrinter) block(out *bytes.Buffer, x Block) {
switch x := x.(type) {
default:
fmt.Fprintf(out, "?%T", x)
case *Paragraph:
p.text(out, "", x.Text)
out.WriteString("\n")
case *Heading:
out.WriteString("# ")
p.text(out, "", x.Text)
out.WriteString("\n")
case *Code:
md := x.Text
for md != "" {
var line string
line, md, _ = strings.Cut(md, "\n")
if line != "" {
out.WriteString("\t")
out.WriteString(line)
}
out.WriteString("\n")
}
case *List:
loose := x.BlankBetween()
for i, item := range x.Items {
if i > 0 && loose {
out.WriteString("\n")
}
out.WriteString(" ")
if item.Number == "" {
out.WriteString(" - ")
} else {
out.WriteString(item.Number)
out.WriteString(". ")
}
for i, blk := range item.Content {
const fourSpace = " "
if i > 0 {
out.WriteString("\n" + fourSpace)
}
p.text(out, fourSpace, blk.(*Paragraph).Text)
out.WriteString("\n")
}
}
}
}
// text prints the text sequence x to out.
func (p *commentPrinter) text(out *bytes.Buffer, indent string, x []Text) {
for _, t := range x {
switch t := t.(type) {
case Plain:
p.indent(out, indent, string(t))
case Italic:
p.indent(out, indent, string(t))
case *Link:
if t.Auto {
p.text(out, indent, t.Text)
} else {
out.WriteString("[")
p.text(out, indent, t.Text)
out.WriteString("]")
}
case *DocLink:
out.WriteString("[")
p.text(out, indent, t.Text)
out.WriteString("]")
}
}
}
// indent prints s to out, indenting with the indent string
// after each newline in s.
func (p *commentPrinter) indent(out *bytes.Buffer, indent, s string) {
for s != "" {
line, rest, ok := strings.Cut(s, "\n")
out.WriteString(line)
if ok {
out.WriteString("\n")
out.WriteString(indent)
}
s = rest
}
}
@@ -0,0 +1,47 @@
// Copyright 2022 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Code generated by 'go generate' DO NOT EDIT.
//disabled go:generate ./mkstd.sh
package comment
var stdPkgs = []string{
"bufio",
"bytes",
"cmp",
"context",
"crypto",
"embed",
"encoding",
"errors",
"expvar",
"flag",
"fmt",
"hash",
"html",
"image",
"io",
"log",
"maps",
"math",
"mime",
"net",
"os",
"path",
"plugin",
"reflect",
"regexp",
"runtime",
"slices",
"sort",
"strconv",
"strings",
"sync",
"syscall",
"testing",
"time",
"unicode",
"unsafe",
}
@@ -0,0 +1,337 @@
// Copyright 2022 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package comment
import (
"bytes"
"fmt"
"sort"
"strings"
"unicode/utf8"
)
// A textPrinter holds the state needed for printing a Doc as plain text.
type textPrinter struct {
*Printer
long strings.Builder
prefix string
codePrefix string
width int
}
// Text returns a textual formatting of the Doc.
// See the [Printer] documentation for ways to customize the text output.
func (p *Printer) Text(d *Doc) []byte {
tp := &textPrinter{
Printer: p,
prefix: p.TextPrefix,
codePrefix: p.TextCodePrefix,
width: p.TextWidth,
}
if tp.codePrefix == "" {
tp.codePrefix = p.TextPrefix + "\t"
}
if tp.width == 0 {
tp.width = 80 - utf8.RuneCountInString(tp.prefix)
}
var out bytes.Buffer
for i, x := range d.Content {
if i > 0 && blankBefore(x) {
out.WriteString(tp.prefix)
writeNL(&out)
}
tp.block(&out, x)
}
anyUsed := false
for _, def := range d.Links {
if def.Used {
anyUsed = true
break
}
}
if anyUsed {
writeNL(&out)
for _, def := range d.Links {
if def.Used {
fmt.Fprintf(&out, "[%s]: %s\n", def.Text, def.URL)
}
}
}
return out.Bytes()
}
// writeNL calls out.WriteByte('\n')
// but first trims trailing spaces on the previous line.
func writeNL(out *bytes.Buffer) {
// Trim trailing spaces.
data := out.Bytes()
n := 0
for n < len(data) && (data[len(data)-n-1] == ' ' || data[len(data)-n-1] == '\t') {
n++
}
if n > 0 {
out.Truncate(len(data) - n)
}
out.WriteByte('\n')
}
// block prints the block x to out.
func (p *textPrinter) block(out *bytes.Buffer, x Block) {
switch x := x.(type) {
default:
fmt.Fprintf(out, "?%T\n", x)
case *Paragraph:
out.WriteString(p.prefix)
p.text(out, "", x.Text)
case *Heading:
out.WriteString(p.prefix)
out.WriteString("# ")
p.text(out, "", x.Text)
case *Code:
text := x.Text
for text != "" {
var line string
line, text, _ = strings.Cut(text, "\n")
if line != "" {
out.WriteString(p.codePrefix)
out.WriteString(line)
}
writeNL(out)
}
case *List:
loose := x.BlankBetween()
for i, item := range x.Items {
if i > 0 && loose {
out.WriteString(p.prefix)
writeNL(out)
}
out.WriteString(p.prefix)
out.WriteString(" ")
if item.Number == "" {
out.WriteString(" - ")
} else {
out.WriteString(item.Number)
out.WriteString(". ")
}
for i, blk := range item.Content {
const fourSpace = " "
if i > 0 {
writeNL(out)
out.WriteString(p.prefix)
out.WriteString(fourSpace)
}
p.text(out, fourSpace, blk.(*Paragraph).Text)
}
}
}
}
// text prints the text sequence x to out.
func (p *textPrinter) text(out *bytes.Buffer, indent string, x []Text) {
p.oneLongLine(&p.long, x)
words := strings.Fields(p.long.String())
p.long.Reset()
var seq []int
if p.width < 0 || len(words) == 0 {
seq = []int{0, len(words)} // one long line
} else {
seq = wrap(words, p.width-utf8.RuneCountInString(indent))
}
for i := 0; i+1 < len(seq); i++ {
if i > 0 {
out.WriteString(p.prefix)
out.WriteString(indent)
}
for j, w := range words[seq[i]:seq[i+1]] {
if j > 0 {
out.WriteString(" ")
}
out.WriteString(w)
}
writeNL(out)
}
}
// oneLongLine prints the text sequence x to out as one long line,
// without worrying about line wrapping.
// Explicit links have the [ ] dropped to improve readability.
func (p *textPrinter) oneLongLine(out *strings.Builder, x []Text) {
for _, t := range x {
switch t := t.(type) {
case Plain:
out.WriteString(string(t))
case Italic:
out.WriteString(string(t))
case *Link:
p.oneLongLine(out, t.Text)
case *DocLink:
p.oneLongLine(out, t.Text)
}
}
}
// wrap wraps words into lines of at most max runes,
// minimizing the sum of the squares of the leftover lengths
// at the end of each line (except the last, of course),
// with a preference for ending lines at punctuation (.,:;).
//
// The returned slice gives the indexes of the first words
// on each line in the wrapped text with a final entry of len(words).
// Thus the lines are words[seq[0]:seq[1]], words[seq[1]:seq[2]],
// ..., words[seq[len(seq)-2]:seq[len(seq)-1]].
//
// The implementation runs in O(n log n) time, where n = len(words),
// using the algorithm described in D. S. Hirschberg and L. L. Larmore,
// “[The least weight subsequence problem],” FOCS 1985, pp. 137-143.
//
// [The least weight subsequence problem]: https://doi.org/10.1109/SFCS.1985.60
func wrap(words []string, max int) (seq []int) {
// The algorithm requires that our scoring function be concave,
// meaning that for all i₀ ≤ i₁ < j₀ ≤ j₁,
// weight(i₀, j₀) + weight(i₁, j₁) ≤ weight(i₀, j₁) + weight(i₁, j₀).
//
// Our weights are two-element pairs [hi, lo]
// ordered by elementwise comparison.
// The hi entry counts the weight for lines that are longer than max,
// and the lo entry counts the weight for lines that are not.
// This forces the algorithm to first minimize the number of lines
// that are longer than max, which correspond to lines with
// single very long words. Having done that, it can move on to
// minimizing the lo score, which is more interesting.
//
// The lo score is the sum for each line of the square of the
// number of spaces remaining at the end of the line and a
// penalty of 64 given out for not ending the line in a
// punctuation character (.,:;).
// The penalty is somewhat arbitrarily chosen by trying
// different amounts and judging how nice the wrapped text looks.
// Roughly speaking, using 64 means that we are willing to
// end a line with eight blank spaces in order to end at a
// punctuation character, even if the next word would fit in
// those spaces.
//
// We care about ending in punctuation characters because
// it makes the text easier to skim if not too many sentences
// or phrases begin with a single word on the previous line.
// A score is the score (also called weight) for a given line.
// add and cmp add and compare scores.
type score struct {
hi int64
lo int64
}
add := func(s, t score) score { return score{s.hi + t.hi, s.lo + t.lo} }
cmp := func(s, t score) int {
switch {
case s.hi < t.hi:
return -1
case s.hi > t.hi:
return +1
case s.lo < t.lo:
return -1
case s.lo > t.lo:
return +1
}
return 0
}
// total[j] is the total number of runes
// (including separating spaces) in words[:j].
total := make([]int, len(words)+1)
total[0] = 0
for i, s := range words {
total[1+i] = total[i] + utf8.RuneCountInString(s) + 1
}
// weight returns weight(i, j).
weight := func(i, j int) score {
// On the last line, there is zero weight for being too short.
n := total[j] - 1 - total[i]
if j == len(words) && n <= max {
return score{0, 0}
}
// Otherwise the weight is the penalty plus the square of the number of
// characters remaining on the line or by which the line goes over.
// In the latter case, that value goes in the hi part of the score.
// (See note above.)
p := wrapPenalty(words[j-1])
v := int64(max-n) * int64(max-n)
if n > max {
return score{v, p}
}
return score{0, v + p}
}
// The rest of this function is “The Basic Algorithm” from
// Hirschberg and Larmore's conference paper,
// using the same names as in the paper.
f := []score{{0, 0}}
g := func(i, j int) score { return add(f[i], weight(i, j)) }
bridge := func(a, b, c int) bool {
k := c + sort.Search(len(words)+1-c, func(k int) bool {
k += c
return cmp(g(a, k), g(b, k)) > 0
})
if k > len(words) {
return true
}
return cmp(g(c, k), g(b, k)) <= 0
}
// d is a one-ended deque implemented as a slice.
d := make([]int, 1, len(words))
d[0] = 0
bestleft := make([]int, 1, len(words))
bestleft[0] = -1
for m := 1; m < len(words); m++ {
f = append(f, g(d[0], m))
bestleft = append(bestleft, d[0])
for len(d) > 1 && cmp(g(d[1], m+1), g(d[0], m+1)) <= 0 {
d = d[1:] // “Retire”
}
for len(d) > 1 && bridge(d[len(d)-2], d[len(d)-1], m) {
d = d[:len(d)-1] // “Fire”
}
if cmp(g(m, len(words)), g(d[len(d)-1], len(words))) < 0 {
d = append(d, m) // “Hire”
// The next few lines are not in the paper but are necessary
// to handle two-word inputs correctly. It appears to be
// just a bug in the paper's pseudocode.
if len(d) == 2 && cmp(g(d[1], m+1), g(d[0], m+1)) <= 0 {
d = d[1:]
}
}
}
bestleft = append(bestleft, d[0])
// Recover least weight sequence from bestleft.
n := 1
for m := len(words); m > 0; m = bestleft[m] {
n++
}
seq = make([]int, n)
for m := len(words); m > 0; m = bestleft[m] {
n--
seq[n] = m
}
return seq
}
// wrapPenalty is the penalty for inserting a line break after word s.
func wrapPenalty(s string) int64 {
switch s[len(s)-1] {
case '.', ',', ':', ';':
return 0
}
return 64
}
@@ -0,0 +1,134 @@
// Copyright 2012 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Package format implements standard formatting of Go source.
//
// Note that formatting of Go source code changes over time, so tools relying on
// consistent formatting should execute a specific version of the gofmt binary
// instead of using this package. That way, the formatting will be stable, and
// the tools won't need to be recompiled each time gofmt changes.
//
// For example, pre-submit checks that use this package directly would behave
// differently depending on what Go version each developer uses, causing the
// check to be inherently fragile.
package format
import (
"bytes"
"fmt"
"go/ast"
"go/parser"
"go/token"
"io"
"mvdan.cc/gofumpt/internal/govendor/go/printer"
)
// Keep these in sync with cmd/gofmt/gofmt.go.
const (
tabWidth = 8
printerMode = printer.UseSpaces | printer.TabIndent | printerNormalizeNumbers
// printerNormalizeNumbers means to canonicalize number literal prefixes
// and exponents while printing. See https://golang.org/doc/go1.13#gofmt.
//
// This value is defined in mvdan.cc/gofumpt/internal/govendor/go/printer specifically for mvdan.cc/gofumpt/internal/govendor/go/format and cmd/gofmt.
printerNormalizeNumbers = 1 << 30
)
var config = printer.Config{Mode: printerMode, Tabwidth: tabWidth}
const parserMode = parser.ParseComments | parser.SkipObjectResolution
// Node formats node in canonical gofmt style and writes the result to dst.
//
// The node type must be *ast.File, *printer.CommentedNode, []ast.Decl,
// []ast.Stmt, or assignment-compatible to ast.Expr, ast.Decl, ast.Spec,
// or ast.Stmt. Node does not modify node. Imports are not sorted for
// nodes representing partial source files (for instance, if the node is
// not an *ast.File or a *printer.CommentedNode not wrapping an *ast.File).
//
// The function may return early (before the entire result is written)
// and return a formatting error, for instance due to an incorrect AST.
func Node(dst io.Writer, fset *token.FileSet, node any) error {
// Determine if we have a complete source file (file != nil).
var file *ast.File
var cnode *printer.CommentedNode
switch n := node.(type) {
case *ast.File:
file = n
case *printer.CommentedNode:
if f, ok := n.Node.(*ast.File); ok {
file = f
cnode = n
}
}
// Sort imports if necessary.
if file != nil && hasUnsortedImports(file) {
// Make a copy of the AST because ast.SortImports is destructive.
// TODO(gri) Do this more efficiently.
var buf bytes.Buffer
err := config.Fprint(&buf, fset, file)
if err != nil {
return err
}
file, err = parser.ParseFile(fset, "", buf.Bytes(), parserMode)
if err != nil {
// We should never get here. If we do, provide good diagnostic.
return fmt.Errorf("format.Node internal error (%s)", err)
}
ast.SortImports(fset, file)
// Use new file with sorted imports.
node = file
if cnode != nil {
node = &printer.CommentedNode{Node: file, Comments: cnode.Comments}
}
}
return config.Fprint(dst, fset, node)
}
// Source formats src in canonical gofmt style and returns the result
// or an (I/O or syntax) error. src is expected to be a syntactically
// correct Go source file, or a list of Go declarations or statements.
//
// If src is a partial source file, the leading and trailing space of src
// is applied to the result (such that it has the same leading and trailing
// space as src), and the result is indented by the same amount as the first
// line of src containing code. Imports are not sorted for partial source files.
func Source(src []byte) ([]byte, error) {
fset := token.NewFileSet()
file, sourceAdj, indentAdj, err := parse(fset, "", src, true)
if err != nil {
return nil, err
}
if sourceAdj == nil {
// Complete source file.
// TODO(gri) consider doing this always.
ast.SortImports(fset, file)
}
return format(fset, file, sourceAdj, indentAdj, src, config)
}
func hasUnsortedImports(file *ast.File) bool {
for _, d := range file.Decls {
d, ok := d.(*ast.GenDecl)
if !ok || d.Tok != token.IMPORT {
// Not an import declaration, so we're done.
// Imports are always first.
return false
}
if d.Lparen.IsValid() {
// For now assume all grouped imports are unsorted.
// TODO(gri) Should check if they are sorted already.
return true
}
// Ungrouped imports are sorted by default.
}
return false
}
@@ -0,0 +1,177 @@
// Copyright 2015 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// TODO(gri): This file and the file src/cmd/gofmt/internal.go are
// the same (but for this comment and the package name). Do not modify
// one without the other. Determine if we can factor out functionality
// in a public API. See also #11844 for context.
package format
import (
"bytes"
"go/ast"
"go/parser"
"go/token"
"strings"
"mvdan.cc/gofumpt/internal/govendor/go/printer"
)
// parse parses src, which was read from the named file,
// as a Go source file, declaration, or statement list.
func parse(fset *token.FileSet, filename string, src []byte, fragmentOk bool) (
file *ast.File,
sourceAdj func(src []byte, indent int) []byte,
indentAdj int,
err error,
) {
// Try as whole source file.
file, err = parser.ParseFile(fset, filename, src, parserMode)
// If there's no error, return. If the error is that the source file didn't begin with a
// package line and source fragments are ok, fall through to
// try as a source fragment. Stop and return on any other error.
if err == nil || !fragmentOk || !strings.Contains(err.Error(), "expected 'package'") {
return
}
// If this is a declaration list, make it a source file
// by inserting a package clause.
// Insert using a ';', not a newline, so that the line numbers
// in psrc match the ones in src.
psrc := append([]byte("package p;"), src...)
file, err = parser.ParseFile(fset, filename, psrc, parserMode)
if err == nil {
sourceAdj = func(src []byte, indent int) []byte {
// Remove the package clause.
// Gofmt has turned the ';' into a '\n'.
src = src[indent+len("package p\n"):]
return bytes.TrimSpace(src)
}
return
}
// If the error is that the source file didn't begin with a
// declaration, fall through to try as a statement list.
// Stop and return on any other error.
if !strings.Contains(err.Error(), "expected declaration") {
return
}
// If this is a statement list, make it a source file
// by inserting a package clause and turning the list
// into a function body. This handles expressions too.
// Insert using a ';', not a newline, so that the line numbers
// in fsrc match the ones in src. Add an extra '\n' before the '}'
// to make sure comments are flushed before the '}'.
fsrc := append(append([]byte("package p; func _() {"), src...), '\n', '\n', '}')
file, err = parser.ParseFile(fset, filename, fsrc, parserMode)
if err == nil {
sourceAdj = func(src []byte, indent int) []byte {
// Cap adjusted indent to zero.
if indent < 0 {
indent = 0
}
// Remove the wrapping.
// Gofmt has turned the "; " into a "\n\n".
// There will be two non-blank lines with indent, hence 2*indent.
src = src[2*indent+len("package p\n\nfunc _() {"):]
// Remove only the "}\n" suffix: remaining whitespaces will be trimmed anyway
src = src[:len(src)-len("}\n")]
return bytes.TrimSpace(src)
}
// Gofmt has also indented the function body one level.
// Adjust that with indentAdj.
indentAdj = -1
}
// Succeeded, or out of options.
return
}
// format formats the given package file originally obtained from src
// and adjusts the result based on the original source via sourceAdj
// and indentAdj.
func format(
fset *token.FileSet,
file *ast.File,
sourceAdj func(src []byte, indent int) []byte,
indentAdj int,
src []byte,
cfg printer.Config,
) ([]byte, error) {
if sourceAdj == nil {
// Complete source file.
var buf bytes.Buffer
err := cfg.Fprint(&buf, fset, file)
if err != nil {
return nil, err
}
return buf.Bytes(), nil
}
// Partial source file.
// Determine and prepend leading space.
i, j := 0, 0
for j < len(src) && isSpace(src[j]) {
if src[j] == '\n' {
i = j + 1 // byte offset of last line in leading space
}
j++
}
var res []byte
res = append(res, src[:i]...)
// Determine and prepend indentation of first code line.
// Spaces are ignored unless there are no tabs,
// in which case spaces count as one tab.
indent := 0
hasSpace := false
for _, b := range src[i:j] {
switch b {
case ' ':
hasSpace = true
case '\t':
indent++
}
}
if indent == 0 && hasSpace {
indent = 1
}
for i := 0; i < indent; i++ {
res = append(res, '\t')
}
// Format the source.
// Write it without any leading and trailing space.
cfg.Indent = indent + indentAdj
var buf bytes.Buffer
err := cfg.Fprint(&buf, fset, file)
if err != nil {
return nil, err
}
out := sourceAdj(buf.Bytes(), cfg.Indent)
// If the adjusted output is empty, the source
// was empty but (possibly) for white space.
// The result is the incoming source.
if len(out) == 0 {
return src, nil
}
// Otherwise, append output to leading space.
res = append(res, out...)
// Determine and append trailing space.
i = len(src)
for i > 0 && isSpace(src[i-1]) {
i--
}
return append(res, src[i:]...), nil
}
// isSpace reports whether the byte is a space character.
// isSpace defines a space as being among the following bytes: ' ', '\t', '\n' and '\r'.
func isSpace(b byte) bool {
return b == ' ' || b == '\t' || b == '\n' || b == '\r'
}
@@ -0,0 +1,156 @@
// Copyright 2022 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package printer
import (
"go/ast"
"strings"
"mvdan.cc/gofumpt/internal/govendor/go/doc/comment"
)
// formatDocComment reformats the doc comment list,
// returning the canonical formatting.
func formatDocComment(list []*ast.Comment) []*ast.Comment {
// Extract comment text (removing comment markers).
var kind, text string
var directives []*ast.Comment
if len(list) == 1 && strings.HasPrefix(list[0].Text, "/*") {
kind = "/*"
text = list[0].Text
if !strings.Contains(text, "\n") || allStars(text) {
// Single-line /* .. */ comment in doc comment position,
// or multiline old-style comment like
// /*
// * Comment
// * text here.
// */
// Should not happen, since it will not work well as a
// doc comment, but if it does, just ignore:
// reformatting it will only make the situation worse.
return list
}
text = text[2 : len(text)-2] // cut /* and */
} else if strings.HasPrefix(list[0].Text, "//") {
kind = "//"
var b strings.Builder
for _, c := range list {
after, found := strings.CutPrefix(c.Text, "//")
if !found {
return list
}
// Accumulate //go:build etc lines separately.
if isDirective(after) {
directives = append(directives, c)
continue
}
b.WriteString(strings.TrimPrefix(after, " "))
b.WriteString("\n")
}
text = b.String()
} else {
// Not sure what this is, so leave alone.
return list
}
if text == "" {
return list
}
// Parse comment and reformat as text.
var p comment.Parser
d := p.Parse(text)
var pr comment.Printer
text = string(pr.Comment(d))
// For /* */ comment, return one big comment with text inside.
slash := list[0].Slash
if kind == "/*" {
c := &ast.Comment{
Slash: slash,
Text: "/*\n" + text + "*/",
}
return []*ast.Comment{c}
}
// For // comment, return sequence of // lines.
var out []*ast.Comment
for text != "" {
var line string
line, text, _ = strings.Cut(text, "\n")
if line == "" {
line = "//"
} else if strings.HasPrefix(line, "\t") {
line = "//" + line
} else {
line = "// " + line
}
out = append(out, &ast.Comment{
Slash: slash,
Text: line,
})
}
if len(directives) > 0 {
out = append(out, &ast.Comment{
Slash: slash,
Text: "//",
})
for _, c := range directives {
out = append(out, &ast.Comment{
Slash: slash,
Text: c.Text,
})
}
}
return out
}
// isDirective reports whether c is a comment directive.
// See go.dev/issue/37974.
// This code is also in go/ast.
func isDirective(c string) bool {
// "//line " is a line directive.
// "//extern " is for gccgo.
// "//export " is for cgo.
// (The // has been removed.)
if strings.HasPrefix(c, "line ") || strings.HasPrefix(c, "extern ") || strings.HasPrefix(c, "export ") {
return true
}
// "//[a-z0-9]+:[a-z0-9]"
// (The // has been removed.)
colon := strings.Index(c, ":")
if colon <= 0 || colon+1 >= len(c) {
return false
}
for i := 0; i <= colon+1; i++ {
if i == colon {
continue
}
b := c[i]
if !('a' <= b && b <= 'z' || '0' <= b && b <= '9') {
return false
}
}
return true
}
// allStars reports whether text is the interior of an
// old-style /* */ comment with a star at the start of each line.
func allStars(text string) bool {
for i := 0; i < len(text); i++ {
if text[i] == '\n' {
j := i + 1
for j < len(text) && (text[j] == ' ' || text[j] == '\t') {
j++
}
if j < len(text) && text[j] != '*' {
return false
}
}
}
return true
}
@@ -0,0 +1,170 @@
// Copyright 2020 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package printer
import (
"go/build/constraint"
"sort"
"text/tabwriter"
)
func (p *printer) fixGoBuildLines() {
if len(p.goBuild)+len(p.plusBuild) == 0 {
return
}
// Find latest possible placement of //go:build and // +build comments.
// That's just after the last blank line before we find a non-comment.
// (We'll add another blank line after our comment block.)
// When we start dropping // +build comments, we can skip over /* */ comments too.
// Note that we are processing tabwriter input, so every comment
// begins and ends with a tabwriter.Escape byte.
// And some newlines have turned into \f bytes.
insert := 0
for pos := 0; ; {
// Skip leading space at beginning of line.
blank := true
for pos < len(p.output) && (p.output[pos] == ' ' || p.output[pos] == '\t') {
pos++
}
// Skip over // comment if any.
if pos+3 < len(p.output) && p.output[pos] == tabwriter.Escape && p.output[pos+1] == '/' && p.output[pos+2] == '/' {
blank = false
for pos < len(p.output) && !isNL(p.output[pos]) {
pos++
}
}
// Skip over \n at end of line.
if pos >= len(p.output) || !isNL(p.output[pos]) {
break
}
pos++
if blank {
insert = pos
}
}
// If there is a //go:build comment before the place we identified,
// use that point instead. (Earlier in the file is always fine.)
if len(p.goBuild) > 0 && p.goBuild[0] < insert {
insert = p.goBuild[0]
} else if len(p.plusBuild) > 0 && p.plusBuild[0] < insert {
insert = p.plusBuild[0]
}
var x constraint.Expr
switch len(p.goBuild) {
case 0:
// Synthesize //go:build expression from // +build lines.
for _, pos := range p.plusBuild {
y, err := constraint.Parse(p.commentTextAt(pos))
if err != nil {
x = nil
break
}
if x == nil {
x = y
} else {
x = &constraint.AndExpr{X: x, Y: y}
}
}
case 1:
// Parse //go:build expression.
x, _ = constraint.Parse(p.commentTextAt(p.goBuild[0]))
}
var block []byte
if x == nil {
// Don't have a valid //go:build expression to treat as truth.
// Bring all the lines together but leave them alone.
// Note that these are already tabwriter-escaped.
for _, pos := range p.goBuild {
block = append(block, p.lineAt(pos)...)
}
for _, pos := range p.plusBuild {
block = append(block, p.lineAt(pos)...)
}
} else {
block = append(block, tabwriter.Escape)
block = append(block, "//go:build "...)
block = append(block, x.String()...)
block = append(block, tabwriter.Escape, '\n')
if len(p.plusBuild) > 0 {
lines, err := constraint.PlusBuildLines(x)
if err != nil {
lines = []string{"// +build error: " + err.Error()}
}
for _, line := range lines {
block = append(block, tabwriter.Escape)
block = append(block, line...)
block = append(block, tabwriter.Escape, '\n')
}
}
}
block = append(block, '\n')
// Build sorted list of lines to delete from remainder of output.
toDelete := append(p.goBuild, p.plusBuild...)
sort.Ints(toDelete)
// Collect output after insertion point, with lines deleted, into after.
var after []byte
start := insert
for _, end := range toDelete {
if end < start {
continue
}
after = appendLines(after, p.output[start:end])
start = end + len(p.lineAt(end))
}
after = appendLines(after, p.output[start:])
if n := len(after); n >= 2 && isNL(after[n-1]) && isNL(after[n-2]) {
after = after[:n-1]
}
p.output = p.output[:insert]
p.output = append(p.output, block...)
p.output = append(p.output, after...)
}
// appendLines is like append(x, y...)
// but it avoids creating doubled blank lines,
// which would not be gofmt-standard output.
// It assumes that only whole blocks of lines are being appended,
// not line fragments.
func appendLines(x, y []byte) []byte {
if len(y) > 0 && isNL(y[0]) && // y starts in blank line
(len(x) == 0 || len(x) >= 2 && isNL(x[len(x)-1]) && isNL(x[len(x)-2])) { // x is empty or ends in blank line
y = y[1:] // delete y's leading blank line
}
return append(x, y...)
}
func (p *printer) lineAt(start int) []byte {
pos := start
for pos < len(p.output) && !isNL(p.output[pos]) {
pos++
}
if pos < len(p.output) {
pos++
}
return p.output[start:pos]
}
func (p *printer) commentTextAt(start int) string {
if start < len(p.output) && p.output[start] == tabwriter.Escape {
start++
}
pos := start
for pos < len(p.output) && p.output[pos] != tabwriter.Escape && !isNL(p.output[pos]) {
pos++
}
return string(p.output[start:pos])
}
func isNL(b byte) bool {
return b == '\n' || b == '\f'
}
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -0,0 +1 @@
go1.21.0
@@ -0,0 +1,105 @@
// Copyright (c) 2020, Daniel Martí <mvdan@mvdan.cc>
// See LICENSE for licensing information
package version
import (
"encoding/json"
"fmt"
"os"
"runtime"
"runtime/debug"
"time"
"golang.org/x/mod/module"
)
// Note that this is not a main package, so a "var version" will not work with
// our go-cross script which uses -ldflags=main.version=xxx.
const ourModulePath = "mvdan.cc/gofumpt"
const fallbackVersion = "(devel)" // to match the default from runtime/debug
func findModule(info *debug.BuildInfo, modulePath string) *debug.Module {
if info.Main.Path == modulePath {
return &info.Main
}
for _, dep := range info.Deps {
if dep.Path == modulePath {
return dep
}
}
return nil
}
func gofumptVersion() string {
info, ok := debug.ReadBuildInfo()
if !ok {
return fallbackVersion // no build info available
}
// Note that gofumpt may be used as a library via the format package,
// so we cannot assume it is the main module in the build.
mod := findModule(info, ourModulePath)
if mod == nil {
return fallbackVersion // not found?
}
if mod.Replace != nil {
mod = mod.Replace
}
// If we found a meaningful version, we are done.
// If gofumpt is not the main module, stop as well,
// as VCS info is only for the main module.
if mod.Version != "(devel)" || mod != &info.Main {
return mod.Version
}
// Fall back to trying to use VCS information.
// Until https://github.com/golang/go/issues/50603 is implemented,
// manually construct something like a pseudo-version.
// TODO: remove when this code is dead, hopefully in Go 1.20.
// For the tests, as we don't want the VCS information to change over time.
if v := os.Getenv("GARBLE_TEST_BUILDSETTINGS"); v != "" {
var extra []debug.BuildSetting
if err := json.Unmarshal([]byte(v), &extra); err != nil {
panic(err)
}
info.Settings = append(info.Settings, extra...)
}
var vcsTime time.Time
var vcsRevision string
for _, setting := range info.Settings {
switch setting.Key {
case "vcs.time":
// If the format is invalid, we'll print a zero timestamp.
vcsTime, _ = time.Parse(time.RFC3339Nano, setting.Value)
case "vcs.revision":
vcsRevision = setting.Value
if len(vcsRevision) > 12 {
vcsRevision = vcsRevision[:12]
}
}
}
if vcsRevision != "" {
return module.PseudoVersion("", "", vcsTime, vcsRevision)
}
return fallbackVersion
}
func goVersion() string {
// For the tests, as we don't want the Go version to change over time.
if testVersion := os.Getenv("GO_VERSION_TEST"); testVersion != "" {
return testVersion
}
return runtime.Version()
}
func String(injected string) string {
if injected != "" {
return fmt.Sprintf("%s (%s)", injected, goVersion())
}
return fmt.Sprintf("%s (%s)", gofumptVersion(), goVersion())
}
@@ -0,0 +1,58 @@
// Copyright (c) 2019, Daniel Martí <mvdan@mvdan.cc>
// See LICENSE for licensing information
package main
import (
"encoding/json"
"flag"
"os"
"os/exec"
"path/filepath"
"testing"
qt "github.com/frankban/quicktest"
"github.com/rogpeppe/go-internal/gotooltest"
"github.com/rogpeppe/go-internal/testscript"
)
func TestMain(m *testing.M) {
os.Exit(testscript.RunMain(m, map[string]func() int{
"gofumpt": main1,
}))
}
var update = flag.Bool("u", false, "update testscript output files")
func TestScript(t *testing.T) {
t.Parallel()
var goEnv struct {
GOCACHE string
GOMODCACHE string
GOMOD string
}
out, err := exec.Command("go", "env", "-json").CombinedOutput()
if err != nil {
t.Fatal(err)
}
if err := json.Unmarshal(out, &goEnv); err != nil {
t.Fatal(err)
}
p := testscript.Params{
Dir: filepath.Join("testdata", "script"),
UpdateScripts: *update,
RequireExplicitExec: true,
Setup: func(env *testscript.Env) error {
env.Setenv("GOCACHE", goEnv.GOCACHE)
env.Setenv("GOMODCACHE", goEnv.GOMODCACHE)
env.Setenv("GOMOD_DIR", filepath.Dir(goEnv.GOMOD))
return nil
},
}
err = gotooltest.Setup(&p)
qt.Assert(t, err, qt.IsNil)
testscript.Run(t, p)
}
@@ -0,0 +1,60 @@
exec gofumpt -w foo.go
cmp foo.go foo.go.golden
exec gofumpt -d foo.go.golden
! stdout .
-- foo.go --
package p
func f() {
foo :=
"bar"
foo :=
"bar"
_, _ =
0,
1
_, _ = 0,
1
_ =
`
foo
`
_ = /* inline */
"foo"
_ = // inline
"foo"
}
-- foo.go.golden --
package p
func f() {
foo := "bar"
foo := "bar"
_, _ = 0,
1
_, _ = 0,
1
_ = `
foo
`
_ = /* inline */ "foo"
_ = // inline
"foo"
}
@@ -0,0 +1,85 @@
exec gofumpt -w foo.go
cmp foo.go foo.go.golden
exec gofumpt -d foo.go.golden
! stdout .
-- foo.go --
package p
func f() {
if true {
// lone comment
}
{
}
{
// lone comment
}
type S struct {
// lone comment
}
type I interface {
// lone comment
}
}
type SOut struct {
// lone comment
}
type IOut interface {
// lone comment
}
-- foo.go.golden --
package p
func f() {
if true {
// lone comment
}
{
}
{
// lone comment
}
type S struct {
// lone comment
}
type I interface {
// lone comment
}
}
type SOut struct {
// lone comment
}
type IOut interface {
// lone comment
}
@@ -0,0 +1,118 @@
exec gofumpt -w foo.go
cmp foo.go foo.go.golden
exec gofumpt -d foo.go.golden
! stdout .
-- foo.go --
package p
func f() {
if true {
println()
}
for true {
println()
}
{
println(1, 2,
3, 4, `foo
bar`)
}
{
// comment directly before
println()
// comment after
}
{
// comment before
println()
// comment directly after
}
// For readability; the empty line helps separate the multi-line
// condition from the body.
if true &&
true {
println()
}
for true &&
true {
println()
}
if true &&
true {
// documented single statement
println()
}
}
-- foo.go.golden --
package p
func f() {
if true {
println()
}
for true {
println()
}
{
println(1, 2,
3, 4, `foo
bar`)
}
{
// comment directly before
println()
// comment after
}
{
// comment before
println()
// comment directly after
}
// For readability; the empty line helps separate the multi-line
// condition from the body.
if true &&
true {
println()
}
for true &&
true {
println()
}
if true &&
true {
// documented single statement
println()
}
}
@@ -0,0 +1,47 @@
exec gofumpt -w foo.go
cmp foo.go foo.go.golden
exec gofumpt -d foo.go.golden
! stdout .
-- foo.go --
package p
import "C"
import "os"
import `C`
import "os"
import "C"
import (
"io"
"utf8"
)
import `C`
import (
"io"
"utf8"
)
-- foo.go.golden --
package p
import "C"
import "os"
import "C"
import "os"
import "C"
import (
"io"
"utf8"
)
import "C"
import (
"io"
"utf8"
)
@@ -0,0 +1,140 @@
exec gofumpt -w foo.go
cmp foo.go foo.go.golden
exec gofumpt -d foo.go.golden
! stdout .
-- foo.go --
//go:build tag
// +build tag
package p
//go:generate some command
//go:unknowndirective
//lint:disablefoo
//go-sumtype:decl Foo
//nolint
//nolint // explanation
//nolint:somelinter // explanation
//NOSONAR
//NOSONAR // explanation
//noinspection ALL
//noinspection foo,bar
//not actually: a directive
//https://just.one/url
//TODO: do something
//export CgoFunc
//extern open
func c_open(name *byte, mode int, perm int) int
//line 123
//sys Unlink(path string) (err error)
//sysnb Getpid() (pid int)
//foo is foo.
type foo int
// comment with a tab.
// comment with many spaces
//comment group
//123 numbers too
// comment group
//123 numbers too
//{
//this is probably code
//}
////////////
// ascii art
//----------
//
-- foo.go.golden --
//go:build tag
// +build tag
package p
//go:generate some command
//go:unknowndirective
//lint:disablefoo
//go-sumtype:decl Foo
//nolint
//nolint // explanation
//nolint:somelinter // explanation
//NOSONAR
//NOSONAR // explanation
//noinspection ALL
//noinspection foo,bar
// not actually: a directive
// https://just.one/url
// TODO: do something
//export CgoFunc
//extern open
func c_open(name *byte, mode int, perm int) int
//line 123
//sys Unlink(path string) (err error)
//sysnb Getpid() (pid int)
// foo is foo.
type foo int
// comment with a tab.
// comment with many spaces
// comment group
// 123 numbers too
// comment group
// 123 numbers too
//{
//this is probably code
//}
////////////
// ascii art
//----------
//
@@ -0,0 +1,109 @@
exec gofumpt -w foo.go
cmp foo.go foo.go.golden
exec gofumpt -d foo.go.golden
! stdout .
-- foo.go --
package p
var _ = []string{
"foo",
}
var _ = []string{
"foo",
}
var _ = []string{
// joint comment
"foo",
}
var _ = []string{
// separate comment
"foo",
}
var _ = map[string]string{
"foo": "bar",
}
var _ = map[string]string{
"foo": "bar",
}
var _ = map[string]string{
// joint comment
"foo": "bar",
}
var _ = map[string]string{
// separate comment
"foo": "bar",
}
var _ = map[string]string{
/*
joint comment
*/
"foo": "bar",
}
-- foo.go.golden --
package p
var _ = []string{
"foo",
}
var _ = []string{
"foo",
}
var _ = []string{
// joint comment
"foo",
}
var _ = []string{
// separate comment
"foo",
}
var _ = map[string]string{
"foo": "bar",
}
var _ = map[string]string{
"foo": "bar",
}
var _ = map[string]string{
// joint comment
"foo": "bar",
}
var _ = map[string]string{
// separate comment
"foo": "bar",
}
var _ = map[string]string{
/*
joint comment
*/
"foo": "bar",
}
@@ -0,0 +1,127 @@
exec gofumpt -w foo.go
cmp foo.go foo.go.golden
exec gofumpt -d foo.go.golden
! stdout .
-- foo.go --
package p
var _ = []int{}
var _ = []int{
}
var _ = []int{1, 2,
3, 4}
var _ = []int{
1, 2, 3, 4}
var _ = [][]string{{
"no need for more newlines",
"if wrapping a single expression",
}}
var _ = []string{`
no need for newlines
`, `
if no elements are surrounded by newlines
`}
var _ = []struct{ a int }{
{ // consistent
a: 1,
},
{
a: 2,
}, { // inconsistent
a: 3,
},
}
var _ = []struct{ a int }{{
a: 1,
}, {
a: 2,
}, {
a: 3,
}}
var _ interface{
}
func _(struct{
})
var _ = []interface {
}{1, 2, 3}
func _(
)
type T struct {
Foo // comment
Bar struct { // comment
}
}
-- foo.go.golden --
package p
var _ = []int{}
var _ = []int{}
var _ = []int{
1, 2,
3, 4,
}
var _ = []int{
1, 2, 3, 4,
}
var _ = [][]string{{
"no need for more newlines",
"if wrapping a single expression",
}}
var _ = []string{`
no need for newlines
`, `
if no elements are surrounded by newlines
`}
var _ = []struct{ a int }{
{ // consistent
a: 1,
},
{
a: 2,
},
{ // inconsistent
a: 3,
},
}
var _ = []struct{ a int }{{
a: 1,
}, {
a: 2,
}, {
a: 3,
}}
var _ interface{}
func _(struct{})
var _ = []interface{}{1, 2, 3}
func _()
type T struct {
Foo // comment
Bar struct { // comment
}
}
@@ -0,0 +1,99 @@
exec gofumpt -w foo.go
cmp foo.go foo.go.golden
exec gofumpt -d foo.go.golden
! stdout .
-- foo.go --
package p
var single = "foo"
var another = "bar"
const one = 'q'
const two = 'w'
const three = 'e'
const four = 'r'
var not = 'a'
var v1 = 's'
//go:embed hello.txt
var v2 = 'd'
var v1 = 's'
// comment line 1
// comment line 2
var v2 = 'd'
var v1 = "mixed"
const c1 = "mixed"
//go:embed hello.txt
var v1 = 's'
var v2 = 'd'
var v3 = 'd'
// comment
var v1 = 's'
var v2 = 'd'
/* comment */
var v3 = 'd'
const inline1 = "s1" // c1
const inline2 = "s2" // c2
const inline3 = "s3" // c3
-- foo.go.golden --
package p
var (
single = "foo"
another = "bar"
)
const (
one = 'q'
two = 'w'
three = 'e'
four = 'r'
)
var not = 'a'
var v1 = 's'
//go:embed hello.txt
var v2 = 'd'
var (
v1 = 's'
// comment line 1
// comment line 2
v2 = 'd'
)
var v1 = "mixed"
const c1 = "mixed"
//go:embed hello.txt
var v1 = 's'
var (
v2 = 'd'
v3 = 'd'
)
// comment
var (
v1 = 's'
v2 = 'd'
/* comment */
v3 = 'd'
)
const (
inline1 = "s1" // c1
inline2 = "s2" // c2
inline3 = "s3" // c3
)
@@ -0,0 +1,100 @@
exec gofumpt -w f1.go f2.go
cmp f1.go f1.go.golden
cmp f2.go f2.go.golden
exec gofumpt -d f1.go.golden f2.go.golden
! stdout .
-- f1.go --
package p
import "non-grouped"
import (
"grouped"
)
var single = "foo"
var (
// verbose is verbose.
verbose = "bar"
)
// This entire block has a comment.
var (
groupComment = "bar"
)
var (
multiple1 string
multiple2 string
)
const (
first = iota
)
var (
multiline = []string{
"foo",
"bar",
}
)
var (
foo = "foo"
// bar = "bar"
// baz = "baz"
)
-- f1.go.golden --
package p
import "non-grouped"
import (
"grouped"
)
var single = "foo"
// verbose is verbose.
var verbose = "bar"
// This entire block has a comment.
var (
groupComment = "bar"
)
var (
multiple1 string
multiple2 string
)
const (
first = iota
)
var multiline = []string{
"foo",
"bar",
}
var foo = "foo"
// bar = "bar"
// baz = "baz"
-- f2.go --
package p
func _() {
var (
_ int
)
}
-- f2.go.golden --
package p
func _() {
var _ int
}
@@ -0,0 +1,70 @@
exec gofumpt -w foo.go
cmp foo.go foo.go.golden
exec gofumpt -d foo.go.golden
! stdout .
-- foo.go --
package p
func f1() { println("single line") }
func f2() { println("single line") }
func f3() {
println("multiline")
}
func f4() {
println("multiline")
}
// l1 is a var.
var l1 = []int{
1, 2,
}
// l2 is a var.
var l2 = []int{
3, 4,
}
var (
s3 = `
ok if grouped together
`
s4 = `
ok if grouped together
`
)
var _ = "ok if either isn't multiline"
-- foo.go.golden --
package p
func f1() { println("single line") }
func f2() { println("single line") }
func f3() {
println("multiline")
}
func f4() {
println("multiline")
}
// l1 is a var.
var l1 = []int{
1, 2,
}
// l2 is a var.
var l2 = []int{
3, 4,
}
var (
s3 = `
ok if grouped together
`
s4 = `
ok if grouped together
`
)
var _ = "ok if either isn't multiline"
@@ -0,0 +1,26 @@
cp foo.orig.go foo.go
! exec gofumpt -w -r foo foo.go
stderr 'the rewrite flag is no longer available; use "gofmt -r" instead\n'
cmp foo.orig.go foo.go
exec gofumpt -w -s foo.go
stderr 'warning: -s is deprecated as it is always enabled\n'
cmp foo.go foo.go.golden
exec gofumpt -d foo.go.golden
! stdout .
-- foo.orig.go --
package p
func f() {
println("foo")
}
-- foo.go.golden --
package p
func f() {
println("foo")
}
@@ -0,0 +1,102 @@
env GO_VERSION_TEST=go1.18.29
# First, test a local build of gofumpt resulting from 'git clone'.
# Its version will be inferred from VCS, but since we want a stable test,
# we mock the VCS information. Note that test binaries do not have VCS info.
# Data obtained from a real build while developing.
env GARBLE_TEST_BUILDSETTINGS='[{"Key":"vcs","Value":"git"},{"Key":"vcs.revision","Value":"8dda8068d9f339047fc1777b688afb66a0a0db17"},{"Key":"vcs.time","Value":"2022-07-27T15:58:40Z"},{"Key":"vcs.modified","Value":"true"}]'
exec gofumpt foo.go
cmp stdout foo.go.golden
exec gofumpt outdated.go
cmp stdout foo.go.golden
exec gofumpt -extra foo.go
cmp stdout foo.go.golden-extra
exec gofumpt -lang=1.0 foo.go
cmp stdout foo.go.golden-lang
exec gofumpt -d nochange.go
! stdout .
exec gofumpt -d foo.go.golden
! stdout .
exec gofumpt -d -extra foo.go.golden-extra
! stdout .
# A local build without VCS information will result in a missing version.
env GARBLE_TEST_BUILDSETTINGS='[]'
exec gofumpt foo.go
cmp stdout foo.go.golden-devel
[short] stop 'the rest of this test builds gofumpt binaries'
# We want a published version of gofumpt on the public module proxies,
# because that's the only way that its module version will be included.
# Using a directory replace directive will not work.
# This means that any change in how gofumpt reports its own version
# will require two pull requests, the second one updating the test script.
# We could consider using go-internal/goproxytest, but then we would need to
# manually run something like go-internal/cmd/txtar-addmod reguarly.
# Or teach goproxytest to serve a mock version of gofumpt from its local checkout.
# Either way, both are relatively overkill for now.
env GOBIN=${WORK}/bin
env GOFUMPT_PUBLISHED_VERSION=v0.3.2-0.20220627183521-8dda8068d9f3
# TODO: update these once the library fix hits master
# gofumpt as the main binary with a real module version.
go install mvdan.cc/gofumpt@${GOFUMPT_PUBLISHED_VERSION}
exec ${GOBIN}/gofumpt foo.go
cmp stdout foo.go.golden-released
# gofumpt as a library with a real module version.
cd ${GOMOD_DIR}/testdata/gofumpt-external
go install .
cd ${WORK}
stdin foo.go
exec ${GOBIN}/gofumpt-external
cmp stdout foo.go.golden-external
-- go.mod --
module test
go 1.16
-- foo.go --
package p
//gofumpt:diagnose
-- outdated.go --
package p
//gofumpt:diagnose v0.1.0
-- nochange.go --
package p
//gofumpt:diagnosefoobar
-- foo.go.golden --
package p
//gofumpt:diagnose version: v0.0.0-20220727155840-8dda8068d9f3 (go1.18.29) flags: -lang=v1.16 -modpath=test
-- foo.go.golden-devel --
package p
//gofumpt:diagnose version: (devel) (go1.18.29) flags: -lang=v1.16 -modpath=test
-- foo.go.golden-extra --
package p
//gofumpt:diagnose version: v0.0.0-20220727155840-8dda8068d9f3 (go1.18.29) flags: -lang=v1.16 -modpath=test -extra
-- foo.go.golden-lang --
package p
//gofumpt:diagnose version: v0.0.0-20220727155840-8dda8068d9f3 (go1.18.29) flags: -lang=v1.0 -modpath=test
-- foo.go.golden-released --
package p
//gofumpt:diagnose v0.3.2-0.20220627183521-8dda8068d9f3 -lang=v1.16 -modpath=test
-- foo.go.golden-external --
package p
//gofumpt:diagnose (devel) -lang=v1.16 -modpath=
@@ -0,0 +1,95 @@
# By default, this rule isn't enabled.
exec gofumpt foo.go
cmp stdout foo.go
# It's run with -extra.
exec gofumpt -extra foo.go
cmp stdout foo.go.golden
exec gofumpt -d foo.go.golden
! stdout .
-- foo.go --
package p
type f func(x int, y int) int
type i interface {
add(x int, y int)
}
type s struct {
x int
y int
}
func mergeAdjacent(x int, y int) {}
func mergeThreeAdjacent(x int, y int, z int) {}
func mergeOneWithTwo(x, y int, z int) {}
func mergeTwoWithOne(x int, y, z int) {}
func mergeWithComment(
x int, y int, // comment
)
func mergeAllSyntax(x chan []*foo.Bar, y chan []*foo.Bar) {}
func dontMergeAnonymousParams(int, int) {}
func dontMergeMultipleLines(
x int,
y int,
) {
}
func dontMergeMultipleLines2(
x,
y int,
z int,
) {
}
-- foo.go.golden --
package p
type f func(x, y int) int
type i interface {
add(x, y int)
}
type s struct {
x int
y int
}
func mergeAdjacent(x, y int) {}
func mergeThreeAdjacent(x, y, z int) {}
func mergeOneWithTwo(x, y, z int) {}
func mergeTwoWithOne(x, y, z int) {}
func mergeWithComment(
x, y int, // comment
)
func mergeAllSyntax(x, y chan []*foo.Bar) {}
func dontMergeAnonymousParams(int, int) {}
func dontMergeMultipleLines(
x int,
y int,
) {
}
func dontMergeMultipleLines2(
x,
y int,
z int,
) {
}
@@ -0,0 +1,503 @@
exec gofumpt -w foo.go
cmp foo.go foo.go.golden
exec gofumpt -d foo.go.golden
! stdout .
-- foo.go --
package p
func f1() {
println("multiple")
println("statements")
}
func f2() {
// comment directly before
println()
// comment after
}
func _() {
f3 := func() {
println()
}
}
func multilineParams(p1 string,
p2 string) {
println("body")
}
func multilineParamsUnambiguous(p1 string,
p2 string,
) {
println("body")
}
func multilineParamsListNoReturn(
p1 string,
p2 string,
) {
println("body")
}
func multilineParamsListReturningNamedSingleValue(
p1 string,
p2 string,
) (err error) {
println("body")
return err
}
func multilineParamsListReturningSingleValue(
p1 string,
p2 string,
) error {
println("body")
return nil
}
func multilineParamsListReturningNamedMultiValues(
p1 string,
p2 string,
) (s string, err error) {
println("body")
return s, err
}
func multilineParamsListReturningMultiValues(
p1 string,
p2 string,
) (string, error) {
println("body")
return "", nil
}
func multilineParamsListReturningNamedMultiLineValuesList(
p1 string,
p2 string,
) (
s string,
err error,
) {
println("body")
return s, err
}
func multilineParamsListReturningMultiLineValues(
p1 string,
p2 string,
) (
string,
error,
) {
println("body")
return "", nil
}
func multilineParamsOneParamNoReturn(
p1 string,
) {
println("body")
}
func multilineParamsOneParamReturningNamedSingleValue(
p1 string,
) (err error) {
println("body")
return err
}
func multilineParamsOneParamReturningSingleValue(
p1 string,
) error {
println("body")
return nil
}
func multilineParamsOneParamReturningNamedMultiValues(
p1 string,
) (s string, err error) {
println("body")
return s, err
}
func multilineParamsOneParamReturningMultiValues(
p1 string,
) (string, error) {
println("body")
return "", nil
}
func multilineParamsOneParamReturningNamedMultiLineValuesList(
p1 string,
) (
s string,
err error,
) {
println("body")
return s, err
}
func multilineParamsOneParamReturningMultiLineValues(
p1 string,
) (
string,
error,
) {
println("body")
return "", nil
}
func multilineResults() (p1 string,
p2 string) {
println("body")
}
func multilineResultsUnambiguous() (p1 string,
p2 string,
) {
println("body")
}
func multilineNoFields(
) {
println("body")
}
func f(
foo int,
bar string,
/* baz */) {
body()
}
func f2(
foo int,
bar string,
) (
string,
error,
/* baz */) {
return "", nil
}
func multilineResultsMultipleEmptyLines() (p1 string,
p2 string) {
println("body")
}
func multilineParamsWithoutEmptyLine(p1 string,
p2 string) {
println("body")
}
func multilineParamsWithoutEmptyLineWithComment(p1 string,
p2 string) {
// comment
println("body")
}
// Same as the others above, but with a single result parameter without
// parentheses. This used to cause token.File.Offset crashes.
func f(p1 string,
p2 string) int {
println("body")
return 0
}
func a() {
f := func(s string,
b bool,
) {
// foo
}
}
func f(p1 string,
p2 string) (int, string,
/* baz */) {
println("body")
return 0, ""
}
-- foo.go.golden --
package p
func f1() {
println("multiple")
println("statements")
}
func f2() {
// comment directly before
println()
// comment after
}
func _() {
f3 := func() {
println()
}
}
func multilineParams(p1 string,
p2 string,
) {
println("body")
}
func multilineParamsUnambiguous(p1 string,
p2 string,
) {
println("body")
}
func multilineParamsListNoReturn(
p1 string,
p2 string,
) {
println("body")
}
func multilineParamsListReturningNamedSingleValue(
p1 string,
p2 string,
) (err error) {
println("body")
return err
}
func multilineParamsListReturningSingleValue(
p1 string,
p2 string,
) error {
println("body")
return nil
}
func multilineParamsListReturningNamedMultiValues(
p1 string,
p2 string,
) (s string, err error) {
println("body")
return s, err
}
func multilineParamsListReturningMultiValues(
p1 string,
p2 string,
) (string, error) {
println("body")
return "", nil
}
func multilineParamsListReturningNamedMultiLineValuesList(
p1 string,
p2 string,
) (
s string,
err error,
) {
println("body")
return s, err
}
func multilineParamsListReturningMultiLineValues(
p1 string,
p2 string,
) (
string,
error,
) {
println("body")
return "", nil
}
func multilineParamsOneParamNoReturn(
p1 string,
) {
println("body")
}
func multilineParamsOneParamReturningNamedSingleValue(
p1 string,
) (err error) {
println("body")
return err
}
func multilineParamsOneParamReturningSingleValue(
p1 string,
) error {
println("body")
return nil
}
func multilineParamsOneParamReturningNamedMultiValues(
p1 string,
) (s string, err error) {
println("body")
return s, err
}
func multilineParamsOneParamReturningMultiValues(
p1 string,
) (string, error) {
println("body")
return "", nil
}
func multilineParamsOneParamReturningNamedMultiLineValuesList(
p1 string,
) (
s string,
err error,
) {
println("body")
return s, err
}
func multilineParamsOneParamReturningMultiLineValues(
p1 string,
) (
string,
error,
) {
println("body")
return "", nil
}
func multilineResults() (p1 string,
p2 string,
) {
println("body")
}
func multilineResultsUnambiguous() (p1 string,
p2 string,
) {
println("body")
}
func multilineNoFields() {
println("body")
}
func f(
foo int,
bar string,
/* baz */
) {
body()
}
func f2(
foo int,
bar string,
) (
string,
error,
/* baz */
) {
return "", nil
}
func multilineResultsMultipleEmptyLines() (p1 string,
p2 string,
) {
println("body")
}
func multilineParamsWithoutEmptyLine(p1 string,
p2 string,
) {
println("body")
}
func multilineParamsWithoutEmptyLineWithComment(p1 string,
p2 string,
) {
// comment
println("body")
}
// Same as the others above, but with a single result parameter without
// parentheses. This used to cause token.File.Offset crashes.
func f(p1 string,
p2 string,
) int {
println("body")
return 0
}
func a() {
f := func(s string,
b bool,
) {
// foo
}
}
func f(p1 string,
p2 string) (int, string,
/* baz */) {
println("body")
return 0, ""
}
@@ -0,0 +1,45 @@
# Explicitly given generated files are formatted with our rules.
exec gofumpt foo.go
cmp stdout foo.go.golden
# stdin is still considered an explicit file.
stdin foo.go
exec gofumpt
cmp stdout foo.go.golden
# Implicitly walked generated files get formatted without the added rules.
exec gofumpt -l .
stdout -count=1 '^badgofmt.go$'
! stdout '^foo.go$'
! stderr .
-- badgofmt.go --
// Code generated by foo. DO NOT EDIT.
package foo
func f() {
println("body")
}
-- foo.go --
// foo is a package about bar.
// Code generated by foo. DO NOT EDIT.
package foo
func f() {
println("body")
}
-- foo.go.golden --
// foo is a package about bar.
// Code generated by foo. DO NOT EDIT.
package foo
func f() {
println("body")
}
@@ -0,0 +1,27 @@
# Test various edge cases with go.mod files.
[!go1.21] skip 'Go 1.20 or older cannot parse these go.mod files'
exec gofumpt toolchain-stable/a.go
stdout '//gofumpt:diagnose.* -lang=v1.21'
exec gofumpt toolchain-unstable/a.go
stdout '//gofumpt:diagnose.* -lang=v1.21'
-- toolchain-stable/go.mod --
module a
go 1.21.2
-- toolchain-stable/a.go --
package a
//gofumpt:diagnose
-- toolchain-unstable/go.mod --
module a
go 1.21rc3
-- toolchain-unstable/a.go --
package a
//gofumpt:diagnose
@@ -0,0 +1,55 @@
exec gofumpt orig.go.golden
cp stdout formatted.go.golden
mkdir -p vendor/foo testdata/foo
cp orig.go.golden vendor/foo/foo.go
cp orig.go.golden testdata/foo/foo.go
# format explicit dirs
exec gofumpt -l vendor testdata
stdout -count=1 'vendor[/\\]foo[/\\]foo.go'
stdout -count=1 'testdata[/\\]foo[/\\]foo.go'
! stderr .
# format explicit files
exec gofumpt -l vendor/foo/foo.go testdata/foo/foo.go
stdout -count=1 'vendor[/\\]foo[/\\]foo.go'
stdout -count=1 'testdata[/\\]foo[/\\]foo.go'
! stderr .
# ignore implicit dirs via fs walking
exec gofumpt -l .
! stdout .
! stderr .
# format explicit pkg while ignoring rest
mkdir vendor/ignore testdata/ignore
cp orig.go.golden vendor/ignore/ignore.go
cp orig.go.golden testdata/ignore/ignore.go
exec gofumpt -l vendor/foo testdata/foo .
stdout -count=1 'vendor[/\\]foo[/\\]foo.go'
stdout -count=1 'testdata[/\\]foo[/\\]foo.go'
! stderr .
# format explicit dirs without clean paths
exec gofumpt -l $WORK//vendor ./testdata/./
stdout -count=1 'vendor[/\\]foo[/\\]foo.go'
stdout -count=1 'testdata[/\\]foo[/\\]foo.go'
! stderr .
-- orig.go.golden --
package p
func f() {
if true {
// lone comment
}
{
}
{
// lone comment
}
}
@@ -0,0 +1,208 @@
exec gofumpt -w foo.go
cmp foo.go foo.go.golden
exec gofumpt -d foo.go.golden
! stdout .
-- foo.go --
package p
type i1 interface {
a(x int) int
b(x int) int
c(x int) int
D()
E()
f()
}
type i2 interface {
// comment for a
a(x int) int
// comment between a and b
// comment for b
b(x int) int
// comment between b and c
c(x int) int
d(x int) int
// comment for e
e(x int) int
}
type i3 interface {
a(x int) int
// standalone comment
b(x int) int
}
type leadingLine1 interface {
a(x int) int
}
type leadingLine2 interface {
a(x int) int
}
type leadingLine3 interface {
// comment
a(x int) int
}
type leadingLine4 interface {
// comment
a(x int) int
}
type leadingLine5 interface {
// comment
// comment for a
a(x int) int
}
type leadingLine6 interface {
// comment
// comment for a
a(x int) int
}
type leadingLine7 interface {
// comment
// comment for a
a(x int) int
}
type leadingLine8 interface {
// comment
}
type ii1 interface {
DoA()
DoB()
UndoA()
UndoB()
}
-- foo.go.golden --
package p
type i1 interface {
a(x int) int
b(x int) int
c(x int) int
D()
E()
f()
}
type i2 interface {
// comment for a
a(x int) int
// comment between a and b
// comment for b
b(x int) int
// comment between b and c
c(x int) int
d(x int) int
// comment for e
e(x int) int
}
type i3 interface {
a(x int) int
// standalone comment
b(x int) int
}
type leadingLine1 interface {
a(x int) int
}
type leadingLine2 interface {
a(x int) int
}
type leadingLine3 interface {
// comment
a(x int) int
}
type leadingLine4 interface {
// comment
a(x int) int
}
type leadingLine5 interface {
// comment
// comment for a
a(x int) int
}
type leadingLine6 interface {
// comment
// comment for a
a(x int) int
}
type leadingLine7 interface {
// comment
// comment for a
a(x int) int
}
type leadingLine8 interface {
// comment
}
type ii1 interface {
DoA()
DoB()
UndoA()
UndoB()
}
@@ -0,0 +1,173 @@
# Line directives can throw off our use of MergeLines.
# We should ignore them entirely when calculating line numbers.
# The file below is borrowed from Go's test/dwarf/linedirectives.go.
exec gofumpt -w foo.go
cmp foo.go foo.go.golden
-- foo.go --
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
//line foo/bar.y:4
package main
//line foo/bar.y:60
func main() {
//line foo/bar.y:297
f, l := 0, 0
//line yacctab:1
f, l = 1, 1
//line yaccpar:1
f, l = 2, 1
//line foo/bar.y:82
f, l = 3, 82
//line foo/bar.y:90
f, l = 3, 90
//line foo/bar.y:92
f, l = 3, 92
//line foo/bar.y:100
f, l = 3, 100
//line foo/bar.y:104
l = 104
//line foo/bar.y:112
l = 112
//line foo/bar.y:117
l = 117
//line foo/bar.y:121
l = 121
//line foo/bar.y:125
l = 125
//line foo/bar.y:133
l = 133
//line foo/bar.y:146
l = 146
//line foo/bar.y:148
//line foo/bar.y:153
//line foo/bar.y:155
l = 155
//line foo/bar.y:160
//line foo/bar.y:164
//line foo/bar.y:173
//line foo/bar.y:178
//line foo/bar.y:180
//line foo/bar.y:185
//line foo/bar.y:195
//line foo/bar.y:197
//line foo/bar.y:202
//line foo/bar.y:204
//line foo/bar.y:208
//line foo/bar.y:211
//line foo/bar.y:213
//line foo/bar.y:215
//line foo/bar.y:217
//line foo/bar.y:221
//line foo/bar.y:229
//line foo/bar.y:236
//line foo/bar.y:238
//line foo/bar.y:240
//line foo/bar.y:244
//line foo/bar.y:249
//line foo/bar.y:253
//line foo/bar.y:257
//line foo/bar.y:262
//line foo/bar.y:267
//line foo/bar.y:272
if l == f {
//line foo/bar.y:277
panic("aie!")
//line foo/bar.y:281
}
//line foo/bar.y:285
return
//line foo/bar.y:288
//line foo/bar.y:290
}
//line foo/bar.y:293
//line foo/bar.y:295
-- foo.go.golden --
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
//line foo/bar.y:4
package main
//line foo/bar.y:60
func main() {
//line foo/bar.y:297
f, l := 0, 0
//line yacctab:1
f, l = 1, 1
//line yaccpar:1
f, l = 2, 1
//line foo/bar.y:82
f, l = 3, 82
//line foo/bar.y:90
f, l = 3, 90
//line foo/bar.y:92
f, l = 3, 92
//line foo/bar.y:100
f, l = 3, 100
//line foo/bar.y:104
l = 104
//line foo/bar.y:112
l = 112
//line foo/bar.y:117
l = 117
//line foo/bar.y:121
l = 121
//line foo/bar.y:125
l = 125
//line foo/bar.y:133
l = 133
//line foo/bar.y:146
l = 146
//line foo/bar.y:148
//line foo/bar.y:153
//line foo/bar.y:155
l = 155
//line foo/bar.y:160
//line foo/bar.y:164
//line foo/bar.y:173
//line foo/bar.y:178
//line foo/bar.y:180
//line foo/bar.y:185
//line foo/bar.y:195
//line foo/bar.y:197
//line foo/bar.y:202
//line foo/bar.y:204
//line foo/bar.y:208
//line foo/bar.y:211
//line foo/bar.y:213
//line foo/bar.y:215
//line foo/bar.y:217
//line foo/bar.y:221
//line foo/bar.y:229
//line foo/bar.y:236
//line foo/bar.y:238
//line foo/bar.y:240
//line foo/bar.y:244
//line foo/bar.y:249
//line foo/bar.y:253
//line foo/bar.y:257
//line foo/bar.y:262
//line foo/bar.y:267
//line foo/bar.y:272
if l == f {
//line foo/bar.y:277
panic("aie!")
//line foo/bar.y:281
}
//line foo/bar.y:285
return
//line foo/bar.y:288
//line foo/bar.y:290
}
//line foo/bar.y:293
//line foo/bar.y:295
@@ -0,0 +1,157 @@
cp foo.go foo.go.orig
exec gofumpt -w foo.go
cmp foo.go foo.go.orig
env GOFUMPT_SPLIT_LONG_LINES=on
exec gofumpt -w foo.go
cmp foo.go foo.go.golden
exec gofumpt -d foo.go.golden
! stdout .
-- foo.go --
package p
func _() {
if err := f(argument1, argument2, argument3, argument4, argument5, argument6, argument7, argument8, argument9, argument10); err != nil {
panic(err)
}
// Tiny arguments to ensure the length calculation is right.
if err := f(a, b, c, d, e, f, g, h, i, j, k, l, m, n, o, p, q, r, s, t, u, v, w, x, y, z, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10); err != nil {
panic(err)
}
// These wouldn't take significantly less horizontal space if split.
f(x, "one single very very very very very very very very very very very very very very very very long literal")
if err := f(x, "one single very very very very very very very very very very very very very very very very long literal"); err != nil {
panic(err)
}
{
{
{
{
println("first", "one single very very very very very very very very very very very very very long literal")
}
}
}
}
// Allow splitting at the start of sub-lists too.
if err := f(argument1, argument2, argument3, argument4, someComplex{argument5, argument6, argument7, argument8, argument9, argument10}); err != nil {
panic(err)
}
if err := f(argument1, argument2, argument3, argument4, &someComplex{argument5, argument6, argument7, argument8, argument9, argument10}); err != nil {
panic(err)
}
if err := f(argument1, argument2, argument3, argument4, []someSlice{argument5, argument6, argument7, argument8, argument9, argument10}); err != nil {
panic(err)
}
// Allow splitting "lists" of binary expressions.
if boolean1 && boolean2 && boolean3 && boolean4 && boolean5 && boolean6 && boolean7 && boolean8 && boolean9 && boolean10 && boolean11 {
}
// Over 100, and we split in a way that doesn't break "len(" off.
if boolean1 || boolean2 || boolean3 || boolean4 || len(someVeryLongVarName.SomeVeryLongSelector) > 0 {
}
}
// Note that function declarations have a higher limit of 120.
// This line goes beyond the limit of 120, but splitting it would leave the
// following line with just 20 non-indentation characters. Not worth it.
func LongButNotWorthSplitting(argument1, argument2, argument3, argument4, argument5, argument6, argument7, argument8, argument9 int) bool {
}
// This line goes well past the limit and it should be split.
// Note that it has a nested func type in a parameter.
func TooLongWithFuncParam(fn func(int) (int, error), argument1, argument2, argument3, argument4, argument5, argument6, argument7, argument8, argument9, argument10 int) bool {
}
// This is like LongButNotWorthSplitting, but with a func parameter.
func LongButNotWorthSplitting2(fn func(int) (int, error), argument3, argument4, argument5, argument6, argument7, argument8, argument9 int) bool {
}
// Never split result parameter lists, as that could easily add confusion with
// extra input parameters.
func NeverSplitResults(argument1, argument2, argument3, argument4, argument5 int) (result1 int, result2, result3, result4, result5, result6, result7, result8 bool) {
}
-- foo.go.golden --
package p
func _() {
if err := f(argument1, argument2, argument3, argument4, argument5, argument6, argument7,
argument8, argument9, argument10); err != nil {
panic(err)
}
// Tiny arguments to ensure the length calculation is right.
if err := f(a, b, c, d, e, f, g, h, i, j, k, l, m, n, o, p, q, r, s, t, u, v, w, x, y, z, 0,
1, 2, 3, 4, 5, 6, 7, 8, 9, 10); err != nil {
panic(err)
}
// These wouldn't take significantly less horizontal space if split.
f(x, "one single very very very very very very very very very very very very very very very very long literal")
if err := f(x, "one single very very very very very very very very very very very very very very very very long literal"); err != nil {
panic(err)
}
{
{
{
{
println("first", "one single very very very very very very very very very very very very very long literal")
}
}
}
}
// Allow splitting at the start of sub-lists too.
if err := f(argument1, argument2, argument3, argument4, someComplex{
argument5, argument6, argument7, argument8, argument9, argument10,
}); err != nil {
panic(err)
}
if err := f(argument1, argument2, argument3, argument4, &someComplex{
argument5, argument6, argument7, argument8, argument9, argument10,
}); err != nil {
panic(err)
}
if err := f(argument1, argument2, argument3, argument4, []someSlice{
argument5, argument6, argument7, argument8, argument9, argument10,
}); err != nil {
panic(err)
}
// Allow splitting "lists" of binary expressions.
if boolean1 && boolean2 && boolean3 && boolean4 && boolean5 && boolean6 && boolean7 &&
boolean8 && boolean9 && boolean10 && boolean11 {
}
// Over 100, and we split in a way that doesn't break "len(" off.
if boolean1 || boolean2 || boolean3 || boolean4 ||
len(someVeryLongVarName.SomeVeryLongSelector) > 0 {
}
}
// Note that function declarations have a higher limit of 120.
// This line goes beyond the limit of 120, but splitting it would leave the
// following line with just 20 non-indentation characters. Not worth it.
func LongButNotWorthSplitting(argument1, argument2, argument3, argument4, argument5, argument6, argument7, argument8, argument9 int) bool {
}
// This line goes well past the limit and it should be split.
// Note that it has a nested func type in a parameter.
func TooLongWithFuncParam(fn func(int) (int, error), argument1, argument2, argument3, argument4,
argument5, argument6, argument7, argument8, argument9, argument10 int) bool {
}
// This is like LongButNotWorthSplitting, but with a func parameter.
func LongButNotWorthSplitting2(fn func(int) (int, error), argument3, argument4, argument5, argument6, argument7, argument8, argument9 int) bool {
}
// Never split result parameter lists, as that could easily add confusion with
// extra input parameters.
func NeverSplitResults(argument1, argument2, argument3, argument4, argument5 int) (result1 int, result2, result3, result4, result5, result6, result7, result8 bool) {
}
@@ -0,0 +1,8 @@
# A missing import shouldn't matter nor be fixed by gofumpt.
exec gofumpt foo.go
cmp stdout foo.go
-- foo.go --
package p
var _ bytes.Buffer
@@ -0,0 +1,115 @@
exec gofumpt -w foo.go
cmp foo.go foo.go.golden
exec gofumpt -d foo.go.golden
! stdout .
-- foo.go --
package p
var Do1 func() error
var Do2 func() (int, error)
func f() {
n1, err := Do2()
if err != nil {
panic(err)
}
if n2, err := Do2(); err != nil {
panic(err)
}
n3, err := Do2()
if err != nil {
panic(err)
}
select {
default:
err := Do1()
if err != nil {
panic(err)
}
}
n4, err := Do2()
if err != nil && err.Error() == "complex condition" {
panic(err)
}
err1 := Do1()
if err != nil {
panic(err)
}
{
if err != nil {
panic(err)
}
}
n5, err = Do2()
if err != nil {
panic(err)
}
}
-- foo.go.golden --
package p
var Do1 func() error
var Do2 func() (int, error)
func f() {
n1, err := Do2()
if err != nil {
panic(err)
}
if n2, err := Do2(); err != nil {
panic(err)
}
n3, err := Do2()
if err != nil {
panic(err)
}
select {
default:
err := Do1()
if err != nil {
panic(err)
}
}
n4, err := Do2()
if err != nil && err.Error() == "complex condition" {
panic(err)
}
err1 := Do1()
if err != nil {
panic(err)
}
{
if err != nil {
panic(err)
}
}
n5, err = Do2()
if err != nil {
panic(err)
}
}
@@ -0,0 +1,60 @@
# Initially, the Go language version is too low.
exec gofumpt -l .
! stdout .
# We can give an explicitly newer version.
exec gofumpt -lang=1.13 -l .
stdout -count=1 'foo\.go'
stdout -count=1 'nested[/\\]nested\.go'
# If we bump the version in go.mod, it should be picked up.
exec go mod edit -go=1.13
exec gofumpt -l .
stdout -count=1 'foo\.go'
! stdout 'nested'
# Ensure we produce the output we expect, and that it's stable.
exec gofumpt foo.go
cmp stdout foo.go.golden
exec gofumpt -d foo.go.golden
! stdout .
# We can give an explicitly older version, too
exec gofumpt -lang=1.0 -l .
! stdout .
-- go.mod --
module test
go 1.12
-- foo.go --
package p
const (
i = 0
j = 022
k = 0o_7_5_5
l = 1022
)
-- foo.go.golden --
package p
const (
i = 0
j = 0o22
k = 0o_7_5_5
l = 1022
)
-- nested/go.mod --
module nested
go 1.11
-- nested/nested.go --
package p
const (
i = 0
j = 022
k = 0o_7_5_5
l = 1022
)
@@ -0,0 +1,157 @@
exec gofumpt -w foo.go
cmp foo.go foo.go.golden
exec gofumpt -d foo.go.golden
! stdout .
-- foo.go --
package p
func f(r rune) {
switch r {
case 'a',
'b',
'c':
case 'd', 'e', 'f':
case 'a', 'b',
'c':
case 'v', 'e', 'r', 'y', 'l', 'o', 'n', 'g',
'l', 'i', 's', 't', '.', '.', '.':
// before
case 'a',
'b': // inline
// after
case 'a', // middle
'b':
case 'a', 'b', 'c', 'd', 'e', 'f',
'g': // very very long inline comment at the end
case 'a', 'b', 'c',
'd': // short comment
}
{
{
{
{
{
switch r {
case 'i', 'n', 'd', 'e',
'n', 't', 'e', 'd':
}
}
}
}
}
}
}
func s(x int) {
switch x {
case
shortConstant1,
shortConstant2:
// A comment.
fmt.Println(x)
case
shortConstant3,
shortConstant4:
// Do nothing.
default:
// Another comment.
fmt.Println(x * 2)
}
}
func s(x int) {
switch x {
case
longerConstantName1,
longerConstantName2:
// A comment.
fmt.Println(x)
case
longerConstantName3,
longerConstantName4:
// Do nothing.
default:
// Another comment.
fmt.Println(x * 2)
}
}
-- foo.go.golden --
package p
func f(r rune) {
switch r {
case 'a', 'b', 'c':
case 'd', 'e', 'f':
case 'a', 'b', 'c':
case 'v', 'e', 'r', 'y', 'l', 'o', 'n', 'g',
'l', 'i', 's', 't', '.', '.', '.':
// before
case 'a', 'b': // inline
// after
case 'a', // middle
'b':
case 'a', 'b', 'c', 'd', 'e', 'f',
'g': // very very long inline comment at the end
case 'a', 'b', 'c', 'd': // short comment
}
{
{
{
{
{
switch r {
case 'i', 'n', 'd', 'e',
'n', 't', 'e', 'd':
}
}
}
}
}
}
}
func s(x int) {
switch x {
case shortConstant1, shortConstant2:
// A comment.
fmt.Println(x)
case shortConstant3, shortConstant4:
// Do nothing.
default:
// Another comment.
fmt.Println(x * 2)
}
}
func s(x int) {
switch x {
case
longerConstantName1,
longerConstantName2:
// A comment.
fmt.Println(x)
case
longerConstantName3,
longerConstantName4:
// Do nothing.
default:
// Another comment.
fmt.Println(x * 2)
}
}
@@ -0,0 +1,48 @@
exec gofumpt -w foo.go
cmp foo.go foo.go.golden
exec gofumpt -d foo.go.golden
! stdout .
-- foo.go --
package p
var global = x
func f() {
var local = x
var local2, local3 = x, y
var onlyType T
var typeAndVar T = x
var _ = unused
var (
aligned = x
vars = y
here = y
)
}
-- foo.go.golden --
package p
var global = x
func f() {
local := x
local2, local3 := x, y
var onlyType T
var typeAndVar T = x
_ = unused
var (
aligned = x
vars = y
here = y
)
}
@@ -0,0 +1,42 @@
# gofumpt changes -s to default to true.
exec gofumpt foo.go
cmp stdout foo.go.golden
-- foo.go --
package p
const ()
const (
// Comment
)
type ()
type (
// Comment
)
var ()
var (
// Comment
)
var _ = [][]int{[]int{1}}
-- foo.go.golden --
package p
const (
// Comment
)
type (
// Comment
)
var (
// Comment
)
var _ = [][]int{{1}}
@@ -0,0 +1,193 @@
exec gofumpt -w foo.go
cmp foo.go foo.go.golden
exec gofumpt -d foo.go.golden
! stdout .
-- go.mod --
module nodomainmod/mod1
go 1.16
-- foo.go --
package p
import (
"io"
"io/ioutil" // if the user keeps them in the top group, obey that
_ "io/ioutil"
_ "image/png"
"bufio" // the above is for a side effect; this one has a comment
)
import (
"os"
"foo.local/one"
bytes_ "bytes"
"io"
)
import (
"foo.local/two"
"fmt"
)
// If they are in order, but with extra newlines, join them.
import (
"more"
"std"
)
// We need to split std vs non-std in this case too.
import (
"foo.local"
"foo.local/three"
math "math"
)
import (
"x"
// don't mess up this comment
"y"
// or many
// of them
"z"
)
// This used to crash gofumpt, as there's no space to insert an extra newline.
import (
"std"
"non.std/pkg"
)
// All of the extra imports below are known to not belong in std.
// For example/ and test/, see https://golang.org/issue/37641.
import (
"io"
"example/foo"
"internal/bar"
"test/baz"
)
import (
"io"
"nodomainmod"
"nodomainmod/mod1/pkg1"
"nodomainmod/mod2"
"nodomainmodextra"
)
import (
"io"
"nodomainother/mod.withdot/pkg1"
)
// TODO: fix issue 225.
import (
"path/filepath"
"time"
"github.com/tinkerbell/tink/pkg/apis/core/v1alpha1"
"k8s.io/apimachinery/pkg/types"
"sigs.k8s.io/yaml"
)
-- foo.go.golden --
package p
import (
"io"
"io/ioutil" // if the user keeps them in the top group, obey that
_ "io/ioutil"
_ "image/png"
"bufio" // the above is for a side effect; this one has a comment
)
import (
"io"
"os"
"foo.local/one"
bytes_ "bytes"
)
import (
"fmt"
"foo.local/two"
)
// If they are in order, but with extra newlines, join them.
import (
"more"
"std"
)
// We need to split std vs non-std in this case too.
import (
math "math"
"foo.local"
"foo.local/three"
)
import (
"x"
// don't mess up this comment
"y"
// or many
// of them
"z"
)
// This used to crash gofumpt, as there's no space to insert an extra newline.
import (
"std"
"non.std/pkg"
)
// All of the extra imports below are known to not belong in std.
// For example/ and test/, see https://golang.org/issue/37641.
import (
"io"
"example/foo"
"internal/bar"
"test/baz"
)
import (
"io"
"nodomainmodextra"
"nodomainmod"
"nodomainmod/mod1/pkg1"
"nodomainmod/mod2"
)
import (
"io"
"nodomainother/mod.withdot/pkg1"
)
// TODO: fix issue 225.
import (
"path/filepath"
"time"
"github.com/tinkerbell/tink/pkg/apis/core/v1alpha1"
"k8s.io/apimachinery/pkg/types"
"sigs.k8s.io/yaml"
)
@@ -0,0 +1,98 @@
exec gofumpt foo.go
cmp stdout foo.go.golden
-- go.mod --
module test
go 1.18
-- foo.go --
package p
func Foo[A, B any](x A, y B) {}
type Vector[T any] []T
var v Vector[int ]
type PredeclaredSignedInteger interface {
int | int8 | int16 | int32 | int64
}
type StringableSignedInteger interface {
~int | ~int8 | ~int16 | ~int32 | ~int64
String() string
}
type CombineEmbeds interface {
fmt.Stringer
comparable | io.Reader
Foo()
}
func Caller() {
Foo[int,int](1,2)
}
func Issue235[K interface {
comparable
constraints.Ordered
}, V any](m map[K]V) []K {
keys := maps.Keys(m)
slices.Sort(keys)
return keys
}
func multilineParams[V any](p1 V,
p2 V) {
println("body")
}
-- foo.go.golden --
package p
func Foo[A, B any](x A, y B) {}
type Vector[T any] []T
var v Vector[int]
type PredeclaredSignedInteger interface {
int | int8 | int16 | int32 | int64
}
type StringableSignedInteger interface {
~int | ~int8 | ~int16 | ~int32 | ~int64
String() string
}
type CombineEmbeds interface {
fmt.Stringer
comparable | io.Reader
Foo()
}
func Caller() {
Foo[int, int](1, 2)
}
func Issue235[K interface {
comparable
constraints.Ordered
}, V any](m map[K]V) []K {
keys := maps.Keys(m)
slices.Sort(keys)
return keys
}
func multilineParams[V any](p1 V,
p2 V,
) {
println("body")
}
@@ -0,0 +1,33 @@
# Whether we run gofumpt from inside or outside a module,
# we should always use the information from its go.mod.
# We also test that we don't get confused by the presence of go.work.
exec gofumpt a/go112.go
cmp stdout a/go113.go
cd a
exec gofumpt go112.go
cmp stdout go113.go
-- go.work --
go 1.18
use ./a
use ./b
-- a/go.mod --
module a
go 1.18
-- a/a.go --
package a
-- a/go112.go --
package main
const x = 0777
-- a/go113.go --
package main
const x = 0o777
-- b/go.mod --
module b
go 1.18
-- b/b.go --
package b
@@ -0,0 +1,91 @@
// Copyright (c) 2019, Daniel Martí <mvdan@mvdan.cc>
// See LICENSE for licensing information
package main
import (
"bytes"
"fmt"
"os"
"os/exec"
"path/filepath"
"strconv"
"testing"
qt "github.com/frankban/quicktest"
"golang.org/x/sys/unix"
)
func init() {
// Here rather than in TestMain, to reuse the unix build tag.
if limit := os.Getenv("TEST_WITH_FILE_LIMIT"); limit != "" {
n, err := strconv.ParseUint(limit, 10, 64)
if err != nil {
panic(err)
}
rlimit := unix.Rlimit{Cur: n, Max: n}
if err := unix.Setrlimit(unix.RLIMIT_NOFILE, &rlimit); err != nil {
panic(err)
}
os.Exit(main1())
}
}
func TestWithLowOpenFileLimit(t *testing.T) {
// Safe to run in parallel, as we only change the limit for child processes.
t.Parallel()
tempDir := t.TempDir()
testBinary, err := os.Executable()
qt.Assert(t, err, qt.IsNil)
const (
// Enough directories to run into the ulimit.
// Enough number of files in total to run into the ulimit.
numberDirs = 500
numberFilesPerDir = 20
numberFilesTotal = numberDirs * numberFilesPerDir
)
t.Logf("writing %d tiny Go files", numberFilesTotal)
var allGoFiles []string
for i := 0; i < numberDirs; i++ {
// Prefix "p", so the package name is a valid identifier.
// Add one go.mod file per directory as well,
// which will help catch data races when loading module info.
dirName := fmt.Sprintf("p%03d", i)
dirPath := filepath.Join(tempDir, dirName)
err := os.MkdirAll(dirPath, 0o777)
qt.Assert(t, err, qt.IsNil)
err = os.WriteFile(filepath.Join(dirPath, "go.mod"),
[]byte(fmt.Sprintf("module %s\n\ngo 1.16", dirName)), 0o666)
qt.Assert(t, err, qt.IsNil)
for j := 0; j < numberFilesPerDir; j++ {
filePath := filepath.Join(dirPath, fmt.Sprintf("%03d.go", j))
err := os.WriteFile(filePath,
// Extra newlines so that "-l" prints all paths.
[]byte(fmt.Sprintf("package %s\n\n\n", dirName)), 0o666)
qt.Assert(t, err, qt.IsNil)
allGoFiles = append(allGoFiles, filePath)
}
}
if len(allGoFiles) != numberFilesTotal {
panic("allGoFiles doesn't have the expected number of files?")
}
runGofmt := func(paths ...string) {
t.Logf("running with %d paths", len(paths))
cmd := exec.Command(testBinary, append([]string{"-l"}, paths...)...)
// 256 is a relatively common low limit, e.g. on Mac.
cmd.Env = append(os.Environ(), "TEST_WITH_FILE_LIMIT=256")
out, err := cmd.Output()
var stderr []byte
if err, _ := err.(*exec.ExitError); err != nil {
stderr = err.Stderr
}
qt.Assert(t, err, qt.IsNil, qt.Commentf("stderr:\n%s", stderr))
qt.Assert(t, bytes.Count(out, []byte("\n")), qt.Equals, len(allGoFiles))
}
runGofmt(tempDir)
runGofmt(allGoFiles...)
}
@@ -0,0 +1,2 @@
# To prevent CRLF breakages on Windows for fragile files, like testdata.
* -text
@@ -0,0 +1 @@
github: mvdan
@@ -0,0 +1,23 @@
on: [push, pull_request]
name: Test
jobs:
test:
strategy:
matrix:
go-version: [1.22.x, 1.23.x]
os: [ubuntu-latest, macos-latest, windows-latest]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/setup-go@v5
with:
go-version: ${{ matrix.go-version }}
- uses: actions/checkout@v4
- run: go test ./...
- run: go test -race ./...
# Static checks from this point forward. Only run on one Go version and on
# Linux, since it's the fastest platform, and the tools behave the same.
- if: matrix.os == 'ubuntu-latest' && matrix.go-version == '1.23.x'
run: diff <(echo -n) <(gofmt -s -d .)
- if: matrix.os == 'ubuntu-latest' && matrix.go-version == '1.23.x'
run: go vet ./...
@@ -0,0 +1,185 @@
# Changelog
## [v0.7.0] - 2024-08-16
This release is based on Go 1.23.0's gofmt, and requires Go 1.22 or later.
The following changes are included:
* Group `internal/...` imported packages as standard library - #307
## [v0.6.0] - 2024-01-28
This release is based on Go 1.21's gofmt, and requires Go 1.20 or later.
The following changes are included:
* Support `go` version strings from newer go.mod files - [#280]
* Consider simple error checks even if they use the `=` operator - [#271]
* Ignore `//line` directives to avoid panics - [#288]
## [v0.5.0] - 2023-04-09
This release is based on Go 1.20's gofmt, and requires Go 1.19 or later.
The biggest change in this release is that we now vendor copies of the packages
`go/format`, `go/printer`, and `go/doc/comment` on top of `cmd/gofmt` itself.
This allows for each gofumpt release to format code in exactly the same way
no matter what Go version is used to build it, as Go versions can change those
three packages in ways that alter formatting behavior.
This vendoring adds a small amount of duplication when using the
`mvdan.cc/gofumpt/format` library, but it's the only way to make gofumpt
versions consistent in their behavior and formatting, just like gofmt.
The jump to Go 1.20's `go/printer` should also bring a small performance
improvement, as we contributed patches to make printing about 25% faster:
* https://go.dev/cl/412555
* https://go.dev/cl/412557
* https://go.dev/cl/424924
The following changes are included as well:
* Skip `testdata` dirs by default like we already do for `vendor` - [#260]
* Avoid inserting newlines incorrectly in some func signatures - [#235]
* Avoid joining some comments with the previous line - [#256]
* Fix `gofumpt -version` for release archives - [#253]
## [v0.4.0] - 2022-09-27
This release is based on Go 1.19's gofmt, and requires Go 1.18 or later.
We recommend building gofumpt with Go 1.19 for the best formatting results.
The jump from Go 1.18 brings diffing in pure Go, removing the need to exec `diff`,
and a small parsing speed-up thanks to `go/parser.SkipObjectResolution`.
The following formatting fixes are included as well:
* Allow grouping declarations with comments - [#212]
* Properly measure the length of case clauses - [#217]
* Fix a few crashes found by Go's native fuzzing
## [v0.3.1] - 2022-03-21
This bugfix release resolves a number of issues:
* Avoid "too many open files" error regression introduced by [v0.3.0] - [#208]
* Use the `go.mod` relative to each Go file when deriving flag defaults - [#211]
* Remove unintentional debug prints when directly formatting files
## [v0.3.0] - 2022-02-22
This is gofumpt's third major release, based on Go 1.18's gofmt.
The jump from Go 1.17's gofmt should bring a noticeable speed-up,
as the tool can now format many files concurrently.
On an 8-core laptop, formatting a large codebase is 4x as fast.
The following [formatting rules](https://github.com/mvdan/gofumpt#Added-rules) are added:
* Functions should separate `) {` where the indentation helps readability
* Field lists should not have leading or trailing empty lines
The following changes are included as well:
* Generated files are now fully formatted when given as explicit arguments
* Prepare for Go 1.18's module workspaces, which could cause errors
* Import paths sharing a prefix with the current module path are no longer
grouped with standard library imports
* `format.Options` gains a `ModulePath` field per the last bullet point
## [v0.2.1] - 2021-12-12
This bugfix release resolves a number of issues:
* Add deprecated flags `-s` and `-r` once again, now giving useful errors
* Avoid a panic with certain function declaration styles
* Don't group interface members of different kinds
* Account for leading comments in composite literals
## [v0.2.0] - 2021-11-10
This is gofumpt's second major release, based on Go 1.17's gofmt.
The jump from Go 1.15's gofmt should bring a mild speed-up,
as walking directories with `filepath.WalkDir` uses fewer syscalls.
gofumports is now removed, after being deprecated in [v0.1.0].
Its main purpose was IDE integration; it is now recommended to use gopls,
which in turn implements goimports and supports gofumpt natively.
IDEs which don't integrate with gopls (such as GoLand) implement goimports too,
so it is safe to use gofumpt as their "format on save" command.
See the [installation instructions](https://github.com/mvdan/gofumpt#Installation)
for more details.
The following [formatting rules](https://github.com/mvdan/gofumpt#Added-rules) are added:
* Composite literals should not have leading or trailing empty lines
* No empty lines following an assignment operator
* Functions using an empty line for readability should use a `) {` line instead
* Remove unnecessary empty lines from interfaces
Finally, the following changes are made to the gofumpt tool:
* Initial support for Go 1.18's type parameters is added
* The `-r` flag is removed in favor of `gofmt -r`
* The `-s` flag is removed as it is always enabled
* Vendor directories are skipped unless given as explicit arguments
* The added rules are not applied to generated Go files
* The `format` Go API now also applies the `gofmt -s` simplification
* Add support for `//gofumpt:diagnose` comments
## [v0.1.1] - 2021-03-11
This bugfix release backports fixes for a few issues:
* Keep leading empty lines in func bodies if they help readability
* Avoid breaking comment alignment on empty field lists
* Add support for `//go-sumtype:` directives
## [v0.1.0] - 2021-01-05
This is gofumpt's first release, based on Go 1.15.x. It solidifies the features
which have worked well for over a year.
This release will be the last to include `gofumports`, the fork of `goimports`
which applies `gofumpt`'s rules on top of updating the Go import lines. Users
who were relying on `goimports` in their editors or IDEs to apply both `gofumpt`
and `goimports` in a single step should switch to gopls, the official Go
language server. It is supported by many popular editors such as VS Code and
Vim, and already bundles gofumpt support. Instructions are available [in the
README](https://github.com/mvdan/gofumpt).
`gofumports` also added maintenance work and potential confusion to end users.
In the future, there will only be one way to use `gofumpt` from the command
line. We also have a [Go API](https://pkg.go.dev/mvdan.cc/gofumpt/format) for
those building programs with gofumpt.
Finally, this release adds the `-version` flag, to print the tool's own version.
The flag will work for "master" builds too.
[v0.7.0]: https://github.com/mvdan/gofumpt/releases/tag/v0.7.0
[v0.6.0]: https://github.com/mvdan/gofumpt/releases/tag/v0.6.0
[#271]: https://github.com/mvdan/gofumpt/issues/271
[#280]: https://github.com/mvdan/gofumpt/issues/280
[#288]: https://github.com/mvdan/gofumpt/issues/288
[v0.5.0]: https://github.com/mvdan/gofumpt/releases/tag/v0.5.0
[#235]: https://github.com/mvdan/gofumpt/issues/235
[#253]: https://github.com/mvdan/gofumpt/issues/253
[#256]: https://github.com/mvdan/gofumpt/issues/256
[#260]: https://github.com/mvdan/gofumpt/issues/260
[v0.4.0]: https://github.com/mvdan/gofumpt/releases/tag/v0.4.0
[#212]: https://github.com/mvdan/gofumpt/issues/212
[#217]: https://github.com/mvdan/gofumpt/issues/217
[v0.3.1]: https://github.com/mvdan/gofumpt/releases/tag/v0.3.1
[#208]: https://github.com/mvdan/gofumpt/issues/208
[#211]: https://github.com/mvdan/gofumpt/pull/211
[v0.3.0]: https://github.com/mvdan/gofumpt/releases/tag/v0.3.0
[v0.2.1]: https://github.com/mvdan/gofumpt/releases/tag/v0.2.1
[v0.2.0]: https://github.com/mvdan/gofumpt/releases/tag/v0.2.0
[v0.1.1]: https://github.com/mvdan/gofumpt/releases/tag/v0.1.1
[v0.1.0]: https://github.com/mvdan/gofumpt/releases/tag/v0.1.0
@@ -0,0 +1,27 @@
Copyright (c) 2019, Daniel Martí. All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following disclaimer
in the documentation and/or other materials provided with the
distribution.
* Neither the name of the copyright holder nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
@@ -0,0 +1,27 @@
Copyright (c) 2009 The Go Authors. All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following disclaimer
in the documentation and/or other materials provided with the
distribution.
* Neither the name of Google Inc. nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
@@ -0,0 +1,664 @@
# gofumpt
[![Go Reference](https://pkg.go.dev/badge/mvdan.cc/gofumpt/format.svg)](https://pkg.go.dev/mvdan.cc/gofumpt/format)
go install mvdan.cc/gofumpt@latest
Enforce a stricter format than `gofmt`, while being backwards compatible.
That is, `gofumpt` is happy with a subset of the formats that `gofmt` is happy with.
The tool is a fork of `gofmt` as of Go 1.23.0, and requires Go 1.22 or later.
It can be used as a drop-in replacement to format your Go code,
and running `gofmt` after `gofumpt` should produce no changes.
For example:
gofumpt -l -w .
Some of the Go source files in this repository belong to the Go project.
The project includes copies of `go/printer` and `go/doc/comment` as of Go 1.23.0
to ensure consistent formatting independent of what Go version is being used.
The [added formatting rules](#Added-rules) are implemented in the `format` package.
`vendor` and `testdata` directories are skipped unless given as explicit arguments.
Similarly, the added rules do not apply to generated Go files unless they are
given as explicit arguments.
Finally, note that the `-r` rewrite flag is removed in favor of `gofmt -r`,
and the `-s` flag is hidden as it is always enabled.
### Added rules
**No empty lines following an assignment operator**
<details><summary><i>Example</i></summary>
```go
func foo() {
foo :=
"bar"
}
```
```go
func foo() {
foo := "bar"
}
```
</details>
**No empty lines around function bodies**
<details><summary><i>Example</i></summary>
```go
func foo() {
println("bar")
}
```
```go
func foo() {
println("bar")
}
```
</details>
**Functions should separate `) {` where the indentation helps readability**
<details><summary><i>Example</i></summary>
```go
func foo(s string,
i int) {
println("bar")
}
// With an empty line it's slightly better, but still not great.
func bar(s string,
i int) {
println("bar")
}
```
```go
func foo(s string,
i int,
) {
println("bar")
}
// With an empty line it's slightly better, but still not great.
func bar(s string,
i int,
) {
println("bar")
}
```
</details>
**No empty lines around a lone statement (or comment) in a block**
<details><summary><i>Example</i></summary>
```go
if err != nil {
return err
}
```
```go
if err != nil {
return err
}
```
</details>
**No empty lines before a simple error check**
<details><summary><i>Example</i></summary>
```go
foo, err := processFoo()
if err != nil {
return err
}
```
```go
foo, err := processFoo()
if err != nil {
return err
}
```
</details>
**Composite literals should use newlines consistently**
<details><summary><i>Example</i></summary>
```go
// A newline before or after an element requires newlines for the opening and
// closing braces.
var ints = []int{1, 2,
3, 4}
// A newline between consecutive elements requires a newline between all
// elements.
var matrix = [][]int{
{1},
{2}, {
3,
},
}
```
```go
var ints = []int{
1, 2,
3, 4,
}
var matrix = [][]int{
{1},
{2},
{
3,
},
}
```
</details>
**Empty field lists should use a single line**
<details><summary><i>Example</i></summary>
```go
var V interface {
} = 3
type T struct {
}
func F(
)
```
```go
var V interface{} = 3
type T struct{}
func F()
```
</details>
**`std` imports must be in a separate group at the top**
<details><summary><i>Example</i></summary>
```go
import (
"foo.com/bar"
"io"
"io/ioutil"
)
```
```go
import (
"io"
"io/ioutil"
"foo.com/bar"
)
```
</details>
**Short case clauses should take a single line**
<details><summary><i>Example</i></summary>
```go
switch c {
case 'a', 'b',
'c', 'd':
}
```
```go
switch c {
case 'a', 'b', 'c', 'd':
}
```
</details>
**Multiline top-level declarations must be separated by empty lines**
<details><summary><i>Example</i></summary>
```go
func foo() {
println("multiline foo")
}
func bar() {
println("multiline bar")
}
```
```go
func foo() {
println("multiline foo")
}
func bar() {
println("multiline bar")
}
```
</details>
**Single var declarations should not be grouped with parentheses**
<details><summary><i>Example</i></summary>
```go
var (
foo = "bar"
)
```
```go
var foo = "bar"
```
</details>
**Contiguous top-level declarations should be grouped together**
<details><summary><i>Example</i></summary>
```go
var nicer = "x"
var with = "y"
var alignment = "z"
```
```go
var (
nicer = "x"
with = "y"
alignment = "z"
)
```
</details>
**Simple var-declaration statements should use short assignments**
<details><summary><i>Example</i></summary>
```go
var s = "somestring"
```
```go
s := "somestring"
```
</details>
**The `-s` code simplification flag is enabled by default**
<details><summary><i>Example</i></summary>
```go
var _ = [][]int{[]int{1}}
```
```go
var _ = [][]int{{1}}
```
</details>
**Octal integer literals should use the `0o` prefix on modules using Go 1.13 and later**
<details><summary><i>Example</i></summary>
```go
const perm = 0755
```
```go
const perm = 0o755
```
</details>
**Comments which aren't Go directives should start with a whitespace**
<details><summary><i>Example</i></summary>
```go
//go:noinline
//Foo is awesome.
func Foo() {}
```
```go
//go:noinline
// Foo is awesome.
func Foo() {}
```
</details>
**Composite literals should not have leading or trailing empty lines**
<details><summary><i>Example</i></summary>
```go
var _ = []string{
"foo",
}
var _ = map[string]string{
"foo": "bar",
}
```
```go
var _ = []string{
"foo",
}
var _ = map[string]string{
"foo": "bar",
}
```
</details>
**Field lists should not have leading or trailing empty lines**
<details><summary><i>Example</i></summary>
```go
type Person interface {
Name() string
Age() int
}
type ZeroFields struct {
// No fields are needed here.
}
```
```go
type Person interface {
Name() string
Age() int
}
type ZeroFields struct {
// No fields are needed here.
}
```
</details>
#### Extra rules behind `-extra`
**Adjacent parameters with the same type should be grouped together**
<details><summary><i>Example</i></summary>
```go
func Foo(bar string, baz string) {}
```
```go
func Foo(bar, baz string) {}
```
</details>
### Installation
`gofumpt` is a replacement for `gofmt`, so you can simply `go install` it as
described at the top of this README and use it.
When using an IDE or editor with Go integration based on `gopls`,
it's best to configure the editor to use the `gofumpt` support built into `gopls`.
The instructions below show how to set up `gofumpt` for some of the
major editors out there.
#### Visual Studio Code
Enable the language server following [the official docs](https://github.com/golang/vscode-go#readme),
and then enable gopls's `gofumpt` option. Note that VS Code will complain about
the `gopls` settings, but they will still work.
```json
"go.useLanguageServer": true,
"gopls": {
"formatting.gofumpt": true,
},
```
#### GoLand
GoLand doesn't use `gopls` so it should be configured to use `gofumpt` directly.
Once `gofumpt` is installed, follow the steps below:
- Open **Settings** (File > Settings)
- Open the **Tools** section
- Find the *File Watchers* sub-section
- Click on the `+` on the right side to add a new file watcher
- Choose *Custom Template*
When a window asks for settings, you can enter the following:
* File Types: Select all .go files
* Scope: Project Files
* Program: Select your `gofumpt` executable
* Arguments: `-w $FilePath$`
* Output path to refresh: `$FilePath$`
* Working directory: `$ProjectFileDir$`
* Environment variables: `GOROOT=$GOROOT$;GOPATH=$GOPATH$;PATH=$GoBinDirs$`
To avoid unnecessary runs, you should disable all checkboxes in the *Advanced* section.
#### Vim
The configuration depends on the plugin you are using: [vim-go](https://github.com/fatih/vim-go)
or [govim](https://github.com/govim/govim).
##### vim-go
To configure `gopls` to use `gofumpt`:
```vim
let g:go_fmt_command="gopls"
let g:go_gopls_gofumpt=1
```
##### govim
To configure `gopls` to use `gofumpt`:
```vim
call govim#config#Set("Gofumpt", 1)
```
#### Neovim
When using [`lspconfig`](https://github.com/neovim/nvim-lspconfig), pass the `gofumpt` setting to `gopls`:
```lua
require('lspconfig').gopls.setup({
settings = {
gopls = {
gofumpt = true
}
}
})
```
#### Emacs
For [lsp-mode](https://emacs-lsp.github.io/lsp-mode/) users on version 8.0.0 or higher:
```elisp
(setq lsp-go-use-gofumpt t)
```
For users of `lsp-mode` before `8.0.0`:
```elisp
(lsp-register-custom-settings
'(("gopls.gofumpt" t)))
```
For [eglot](https://github.com/joaotavora/eglot) users:
```elisp
(setq-default eglot-workspace-configuration
'((:gopls . ((gofumpt . t)))))
```
#### Helix
When using the `gopls` language server, modify the Go settings in `~/.config/helix/languages.toml`:
```toml
[language-server.gopls.config]
"formatting.gofumpt" = true
```
#### Sublime Text
With ST4, install the Sublime Text LSP extension according to [the documentation](https://github.com/sublimelsp/LSP),
and enable `gopls`'s `gofumpt` option in the LSP package settings,
including setting `lsp_format_on_save` to `true`.
```json
"lsp_format_on_save": true,
"clients":
{
"gopls":
{
"enabled": true,
"initializationOptions": {
"gofumpt": true,
}
}
}
```
### Roadmap
This tool is a place to experiment. In the long term, the features that work
well might be proposed for `gofmt` itself.
The tool is also compatible with `gofmt` and is aimed to be stable, so you can
rely on it for your code as long as you pin a version of it.
### Frequently Asked Questions
> Why attempt to replace `gofmt` instead of building on top of it?
Our design is to build on top of `gofmt`, and we'll never add rules which
disagree with its formatting. So we extend `gofmt` rather than compete with it.
The tool is a modified copy of `gofmt`, for the purpose of allowing its use as a
drop-in replacement in editors and scripts.
> Why are my module imports being grouped with standard library imports?
Any import paths that don't start with a domain name like `foo.com` are
effectively [reserved by the Go toolchain](https://github.com/golang/go/issues/32819).
Third party modules should either start with a domain name,
even a local one like `foo.local`, or use [a reserved path prefix](https://github.com/golang/go/issues/37641).
For backwards compatibility with modules set up before these rules were clear,
`gofumpt` will treat any import path sharing a prefix with the current module
path as third party. For example, if the current module is `mycorp/mod1`, then
all import paths in `mycorp/...` will be considered third party.
> How can I use `gofumpt` if I already use `goimports` to replace `gofmt`?
Most editors have replaced the `goimports` program with the same functionality
provided by a language server like `gopls`. This mechanism is significantly
faster and more powerful, since the language server has more information that is
kept up to date, necessary to add missing imports.
As such, the general recommendation is to let your editor fix your imports -
either via `gopls`, such as VSCode or vim-go, or via their own custom
implementation, such as GoLand. Then follow the install instructions above to
enable the use of `gofumpt` instead of `gofmt`.
If you want to avoid integrating with `gopls`, and are OK with the overhead of
calling `goimports` from scratch on each save, you should be able to call both
tools; for example, `goimports file.go && gofumpt file.go`.
### Contributing
Issues and pull requests are welcome! Please open an issue to discuss a feature
before sending a pull request.
We also use the `#gofumpt` channel over at the
[Gophers Slack](https://invite.slack.golangbridge.org/) to chat.
When reporting a formatting bug, insert a `//gofumpt:diagnose` comment.
The comment will be rewritten to include useful debugging information.
For instance:
```
$ cat f.go
package p
//gofumpt:diagnose
$ gofumpt f.go
package p
//gofumpt:diagnose v0.1.1-0.20211103104632-bdfa3b02e50a -lang=go1.16
```
### License
Note that much of the code is copied from Go's `gofmt` command. You can tell
which files originate from the Go repository from their copyright headers. Their
license file is `LICENSE.google`.
`gofumpt`'s original source files are also under the 3-clause BSD license, with
the separate file `LICENSE`.
@@ -0,0 +1,5 @@
// Copyright (c) 2023, Daniel Martí <mvdan@mvdan.cc>
// See LICENSE for licensing information
// gofumpt enforces a stricter format than gofmt, while being backwards compatible.
package main
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,38 @@
// Copyright (c) 2021, Daniel Martí <mvdan@mvdan.cc>
// See LICENSE for licensing information
package format_test
import (
"testing"
"github.com/go-quicktest/qt"
"mvdan.cc/gofumpt/format"
)
func TestSourceIncludesSimplify(t *testing.T) {
t.Parallel()
in := []byte(`
package p
var ()
func f() {
for _ = range v {
}
}
`[1:])
want := []byte(`
package p
func f() {
for range v {
}
}
`[1:])
got, err := format.Source(in, format.Options{})
qt.Assert(t, qt.IsNil(err))
qt.Assert(t, qt.Equals(string(got), string(want)))
}
@@ -0,0 +1,64 @@
// Copyright (c) 2021, Daniel Martí <mvdan@mvdan.cc>
// See LICENSE for licensing information
package format
import (
"errors"
"fmt"
"go/scanner"
"path/filepath"
"strings"
"testing"
"github.com/go-quicktest/qt"
"golang.org/x/tools/txtar"
)
func FuzzFormat(f *testing.F) {
// Initialize the corpus with the Go files from our test scripts.
paths, err := filepath.Glob(filepath.Join("..", "testdata", "script", "*.txtar"))
qt.Assert(f, qt.IsNil(err))
qt.Assert(f, qt.Not(qt.HasLen(paths, 0)))
for _, path := range paths {
archive, err := txtar.ParseFile(path)
qt.Assert(f, qt.IsNil(err))
for _, file := range archive.Files {
f.Logf("adding %s from %s", file.Name, path)
if strings.HasSuffix(file.Name, ".go") || strings.Contains(file.Name, ".go.") {
f.Add(string(file.Data), int8(18), false) // -lang=go1.18
f.Add(string(file.Data), int8(1), false) // -lang=go1.1
f.Add(string(file.Data), int8(18), true) // -lang=go1.18 -extra
}
}
}
f.Fuzz(func(t *testing.T, src string,
majorVersion int8, // Empty version if negative, 1.N otherwise.
extraRules bool,
) {
// TODO: also fuzz Options.ModulePath
opts := Options{ExtraRules: extraRules}
if majorVersion >= 0 {
opts.LangVersion = fmt.Sprintf("go1.%d", majorVersion)
}
orig := []byte(src)
formatted, err := Source(orig, opts)
if errors.As(err, &scanner.ErrorList{}) {
return // invalid syntax from parsing
}
qt.Assert(t, qt.IsNil(err))
_ = formatted
// TODO: verify that the result is idempotent
// TODO: verify that, if the input was valid Go 1.N syntax,
// so is the output (how? go/parser lacks an option)
// TODO: check calling format.Node directly as well
qt.Assert(t, qt.Equals(string(orig), src),
qt.Commentf("input source bytes were modified"))
})
}
@@ -0,0 +1,113 @@
// Copyright 2009 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package format
import (
"go/ast"
"go/token"
"reflect"
"unicode"
"unicode/utf8"
)
// Values/types for special cases.
var (
identType = reflect.TypeOf((*ast.Ident)(nil))
objectPtrType = reflect.TypeOf((*ast.Object)(nil))
positionType = reflect.TypeOf(token.NoPos)
callExprType = reflect.TypeOf((*ast.CallExpr)(nil))
)
func isWildcard(s string) bool {
rune, size := utf8.DecodeRuneInString(s)
return size == len(s) && unicode.IsLower(rune)
}
// match reports whether pattern matches val,
// recording wildcard submatches in m.
// If m == nil, match checks whether pattern == val.
func match(m map[string]reflect.Value, pattern, val reflect.Value) bool {
// Wildcard matches any expression. If it appears multiple
// times in the pattern, it must match the same expression
// each time.
if m != nil && pattern.IsValid() && pattern.Type() == identType {
name := pattern.Interface().(*ast.Ident).Name
if isWildcard(name) && val.IsValid() {
// wildcards only match valid (non-nil) expressions.
if _, ok := val.Interface().(ast.Expr); ok && !val.IsNil() {
if old, ok := m[name]; ok {
return match(nil, old, val)
}
m[name] = val
return true
}
}
}
// Otherwise, pattern and val must match recursively.
if !pattern.IsValid() || !val.IsValid() {
return !pattern.IsValid() && !val.IsValid()
}
if pattern.Type() != val.Type() {
return false
}
// Special cases.
switch pattern.Type() {
case identType:
// For identifiers, only the names need to match
// (and none of the other *ast.Object information).
// This is a common case, handle it all here instead
// of recursing down any further via reflection.
p := pattern.Interface().(*ast.Ident)
v := val.Interface().(*ast.Ident)
return p == nil && v == nil || p != nil && v != nil && p.Name == v.Name
case objectPtrType, positionType:
// object pointers and token positions always match
return true
case callExprType:
// For calls, the Ellipsis fields (token.Position) must
// match since that is how f(x) and f(x...) are different.
// Check them here but fall through for the remaining fields.
p := pattern.Interface().(*ast.CallExpr)
v := val.Interface().(*ast.CallExpr)
if p.Ellipsis.IsValid() != v.Ellipsis.IsValid() {
return false
}
}
p := reflect.Indirect(pattern)
v := reflect.Indirect(val)
if !p.IsValid() || !v.IsValid() {
return !p.IsValid() && !v.IsValid()
}
switch p.Kind() {
case reflect.Slice:
if p.Len() != v.Len() {
return false
}
for i := 0; i < p.Len(); i++ {
if !match(m, p.Index(i), v.Index(i)) {
return false
}
}
return true
case reflect.Struct:
for i := 0; i < p.NumField(); i++ {
if !match(m, p.Field(i), v.Field(i)) {
return false
}
}
return true
case reflect.Interface:
return match(m, p.Elem(), v.Elem())
}
// Handle token integers, etc.
return p.Interface() == v.Interface()
}
@@ -0,0 +1,169 @@
// Copyright 2010 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package format
import (
"go/ast"
"go/token"
"reflect"
)
type simplifier struct{}
func (s simplifier) Visit(node ast.Node) ast.Visitor {
switch n := node.(type) {
case *ast.CompositeLit:
// array, slice, and map composite literals may be simplified
outer := n
var keyType, eltType ast.Expr
switch typ := outer.Type.(type) {
case *ast.ArrayType:
eltType = typ.Elt
case *ast.MapType:
keyType = typ.Key
eltType = typ.Value
}
if eltType != nil {
var ktyp reflect.Value
if keyType != nil {
ktyp = reflect.ValueOf(keyType)
}
typ := reflect.ValueOf(eltType)
for i, x := range outer.Elts {
px := &outer.Elts[i]
// look at value of indexed/named elements
if t, ok := x.(*ast.KeyValueExpr); ok {
if keyType != nil {
s.simplifyLiteral(ktyp, keyType, t.Key, &t.Key)
}
x = t.Value
px = &t.Value
}
s.simplifyLiteral(typ, eltType, x, px)
}
// node was simplified - stop walk (there are no subnodes to simplify)
return nil
}
case *ast.SliceExpr:
// a slice expression of the form: s[a:len(s)]
// can be simplified to: s[a:]
// if s is "simple enough" (for now we only accept identifiers)
//
// Note: This may not be correct because len may have been redeclared in
// the same package. However, this is extremely unlikely and so far
// (April 2022, after years of supporting this rewrite feature)
// has never come up, so let's keep it working as is (see also #15153).
//
// Also note that this code used to use go/ast's object tracking,
// which was removed in exchange for go/parser.Mode.SkipObjectResolution.
// False positives are extremely unlikely as described above,
// and go/ast's object tracking is incomplete in any case.
if n.Max != nil {
// - 3-index slices always require the 2nd and 3rd index
break
}
if s, _ := n.X.(*ast.Ident); s != nil {
// the array/slice object is a single identifier
if call, _ := n.High.(*ast.CallExpr); call != nil && len(call.Args) == 1 && !call.Ellipsis.IsValid() {
// the high expression is a function call with a single argument
if fun, _ := call.Fun.(*ast.Ident); fun != nil && fun.Name == "len" {
// the function called is "len"
if arg, _ := call.Args[0].(*ast.Ident); arg != nil && arg.Name == s.Name {
// the len argument is the array/slice object
n.High = nil
}
}
}
}
// Note: We could also simplify slice expressions of the form s[0:b] to s[:b]
// but we leave them as is since sometimes we want to be very explicit
// about the lower bound.
// An example where the 0 helps:
// x, y, z := b[0:2], b[2:4], b[4:6]
// An example where it does not:
// x, y := b[:n], b[n:]
case *ast.RangeStmt:
// - a range of the form: for x, _ = range v {...}
// can be simplified to: for x = range v {...}
// - a range of the form: for _ = range v {...}
// can be simplified to: for range v {...}
if isBlank(n.Value) {
n.Value = nil
}
if isBlank(n.Key) && n.Value == nil {
n.Key = nil
}
}
return s
}
func (s simplifier) simplifyLiteral(typ reflect.Value, astType, x ast.Expr, px *ast.Expr) {
ast.Walk(s, x) // simplify x
// if the element is a composite literal and its literal type
// matches the outer literal's element type exactly, the inner
// literal type may be omitted
if inner, ok := x.(*ast.CompositeLit); ok {
if match(nil, typ, reflect.ValueOf(inner.Type)) {
inner.Type = nil
}
}
// if the outer literal's element type is a pointer type *T
// and the element is & of a composite literal of type T,
// the inner &T may be omitted.
if ptr, ok := astType.(*ast.StarExpr); ok {
if addr, ok := x.(*ast.UnaryExpr); ok && addr.Op == token.AND {
if inner, ok := addr.X.(*ast.CompositeLit); ok {
if match(nil, reflect.ValueOf(ptr.X), reflect.ValueOf(inner.Type)) {
inner.Type = nil // drop T
*px = inner // drop &
}
}
}
}
}
func isBlank(x ast.Expr) bool {
ident, ok := x.(*ast.Ident)
return ok && ident.Name == "_"
}
func simplify(f *ast.File) {
// remove empty declarations such as "const ()", etc
removeEmptyDeclGroups(f)
var s simplifier
ast.Walk(s, f)
}
func removeEmptyDeclGroups(f *ast.File) {
i := 0
for _, d := range f.Decls {
if g, ok := d.(*ast.GenDecl); !ok || !isEmpty(f, g) {
f.Decls[i] = d
i++
}
}
f.Decls = f.Decls[:i]
}
func isEmpty(f *ast.File, g *ast.GenDecl) bool {
if g.Doc != nil || g.Specs != nil {
return false
}
for _, c := range f.Comments {
// if there is a comment in the declaration, it is not considered empty
if g.Pos() <= c.Pos() && c.End() <= g.End() {
return false
}
}
return true
}
@@ -0,0 +1,4 @@
go test fuzz v1
string("package A\nfunc A000000000(A000000000000,\nA00000000)(){\"\"}")
int8(62)
bool(true)
@@ -0,0 +1,4 @@
go test fuzz v1
string("package A\nvar A A\nvar A A")
int8(18)
bool(false)
@@ -0,0 +1,89 @@
// Copyright (c) 2019, Daniel Martí <mvdan@mvdan.cc>
// See LICENSE for licensing information
//go:build ignore
package main
import (
"bytes"
"encoding/json"
"io"
"os"
"os/exec"
"path"
"path/filepath"
"strings"
)
var (
modulePath = "mvdan.cc/gofumpt"
vendorDir = filepath.Join("internal", "govendor")
)
// All the packages which affect the formatting behavior.
var toVendor = []string{
"go/format",
"go/printer",
"go/doc/comment",
"internal/diff",
}
func main() {
catch(os.RemoveAll(vendorDir))
catch(os.MkdirAll(vendorDir, 0o777))
out, err := exec.Command("go", "env", "GOVERSION").Output()
catch(err)
catch(os.WriteFile(filepath.Join(vendorDir, "version.txt"), out, 0o666))
oldnew := []string{
"//go:generate", "//disabled go:generate",
}
for _, pkgPath := range toVendor {
oldnew = append(oldnew, pkgPath, path.Join(modulePath, vendorDir, pkgPath))
}
replacer := strings.NewReplacer(oldnew...)
listArgs := append([]string{"list", "-json"}, toVendor...)
out, err = exec.Command("go", listArgs...).Output()
catch(err)
type Package struct {
Dir string
ImportPath string
GoFiles []string
}
dec := json.NewDecoder(bytes.NewReader(out))
for {
var pkg Package
err := dec.Decode(&pkg)
if err == io.EOF {
break
}
catch(err)
// Otherwise we can't import it.
dstPkg := strings.TrimPrefix(pkg.ImportPath, "internal/")
dstDir := filepath.Join(vendorDir, filepath.FromSlash(dstPkg))
catch(os.MkdirAll(dstDir, 0o777))
// TODO: if the packages start using build tags like GOOS or GOARCH,
// we will need to vendor IgnoredGoFiles as well.
for _, goFile := range pkg.GoFiles {
srcBytes, err := os.ReadFile(filepath.Join(pkg.Dir, goFile))
catch(err)
src := replacer.Replace(string(srcBytes))
dst := filepath.Join(dstDir, goFile)
catch(os.WriteFile(dst, []byte(src), 0o666))
}
}
}
func catch(err error) {
if err != nil {
panic(err)
}
}
+18
View File
@@ -0,0 +1,18 @@
module mvdan.cc/gofumpt
go 1.22
require (
github.com/go-quicktest/qt v1.101.0
github.com/google/go-cmp v0.6.0
github.com/rogpeppe/go-internal v1.12.0
golang.org/x/mod v0.14.0
golang.org/x/sync v0.6.0
golang.org/x/sys v0.16.0
golang.org/x/tools v0.17.0
)
require (
github.com/kr/pretty v0.3.1 // indirect
github.com/kr/text v0.2.0 // indirect
)
+21
View File
@@ -0,0 +1,21 @@
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
github.com/go-quicktest/qt v1.101.0 h1:O1K29Txy5P2OK0dGo59b7b0LR6wKfIhttaAhHUyn7eI=
github.com/go-quicktest/qt v1.101.0/go.mod h1:14Bz/f7NwaXPtdYEgzsx46kqSxVwTbzVZsDC26tQJow=
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e/go.mod h1:pJLUxLENpZxwdsKMEsNbx1VGcRFpLqf3715MtcvvzbA=
github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs=
github.com/rogpeppe/go-internal v1.12.0 h1:exVL4IDcn6na9z1rAb56Vxr+CgyK3nn3O+epU5NdKM8=
github.com/rogpeppe/go-internal v1.12.0/go.mod h1:E+RYuTGaKKdloAfM02xzb0FW3Paa99yedzYV+kq4uf4=
golang.org/x/mod v0.14.0 h1:dGoOF9QVLYng8IHTm7BAyWqCqSheQ5pYWGhzW00YJr0=
golang.org/x/mod v0.14.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/sync v0.6.0 h1:5BMeUDZ7vkXGfEr1x9B4bRcTH4lpkTkpdh0T/J+qjbQ=
golang.org/x/sync v0.6.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sys v0.16.0 h1:xWw16ngr6ZMtmxDyKyIgsE93KNKz5HKmMa3b8ALHidU=
golang.org/x/sys v0.16.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/tools v0.17.0 h1:FvmRgNOcs3kOa+T20R1uhfP9F6HgG2mfxDv1vrx1Htc=
golang.org/x/tools v0.17.0/go.mod h1:xsh6VxdV005rRVaS6SSAf9oiAqljS7UZUacMZ8Bnsps=
+630
View File
@@ -0,0 +1,630 @@
// Copyright 2009 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package main
import (
"bytes"
"context"
"encoding/json"
"flag"
"fmt"
"go/ast"
"go/parser"
"go/scanner"
"go/token"
"io"
"io/fs"
"os"
"os/exec"
"path/filepath"
"regexp"
"runtime"
"runtime/pprof"
"strings"
"sync"
"golang.org/x/sync/semaphore"
gformat "mvdan.cc/gofumpt/format"
"mvdan.cc/gofumpt/internal/govendor/diff"
"mvdan.cc/gofumpt/internal/govendor/go/printer"
gversion "mvdan.cc/gofumpt/internal/version"
)
//go:generate go run gen_govendor.go
//go:generate go run . -w internal/govendor
var (
// main operation modes
list = flag.Bool("l", false, "")
write = flag.Bool("w", false, "")
doDiff = flag.Bool("d", false, "")
allErrors = flag.Bool("e", false, "")
// debugging
cpuprofile = flag.String("cpuprofile", "", "")
// gofumpt's own flags
langVersion = flag.String("lang", "", "")
modulePath = flag.String("modpath", "", "")
extraRules = flag.Bool("extra", false, "")
showVersion = flag.Bool("version", false, "")
// DEPRECATED
rewriteRule = flag.String("r", "", "")
simplifyAST = flag.Bool("s", false, "")
)
var version = ""
// Keep these in sync with go/format/format.go.
const (
tabWidth = 8
printerMode = printer.UseSpaces | printer.TabIndent | printerNormalizeNumbers
// printerNormalizeNumbers means to canonicalize number literal prefixes
// and exponents while printing. See https://golang.org/doc/go1.13#gofmt.
//
// This value is defined in go/printer specifically for go/format and cmd/gofmt.
printerNormalizeNumbers = 1 << 30
)
// fdSem guards the number of concurrently-open file descriptors.
//
// For now, this is arbitrarily set to 200, based on the observation that many
// platforms default to a kernel limit of 256. Ideally, perhaps we should derive
// it from rlimit on platforms that support that system call.
//
// File descriptors opened from outside of this package are not tracked,
// so this limit may be approximate.
var fdSem = make(chan bool, 200)
var (
fileSet = token.NewFileSet() // per process FileSet
parserMode parser.Mode
)
func usage() {
fmt.Fprintf(os.Stderr, `usage: gofumpt [flags] [path ...]
-version show version and exit
-d display diffs instead of rewriting files
-e report all errors (not just the first 10 on different lines)
-l list files whose formatting differs from gofumpt's
-w write result to (source) file instead of stdout
-extra enable extra rules which should be vetted by a human
-lang str target Go version in the form "go1.X" (default from go.mod)
-modpath str Go module path containing the source file (default from go.mod)
`)
}
func initParserMode() {
parserMode = parser.ParseComments | parser.SkipObjectResolution
if *allErrors {
parserMode |= parser.AllErrors
}
}
func isGoFile(f fs.DirEntry) bool {
// ignore non-Go files
name := f.Name()
return !strings.HasPrefix(name, ".") && strings.HasSuffix(name, ".go") && !f.IsDir()
}
var rxCodeGenerated = regexp.MustCompile(`^// Code generated .* DO NOT EDIT\.$`)
func isGenerated(file *ast.File) bool {
for _, cg := range file.Comments {
if cg.Pos() > file.Package {
return false
}
for _, line := range cg.List {
if rxCodeGenerated.MatchString(line.Text) {
return true
}
}
}
return false
}
// A sequencer performs concurrent tasks that may write output, but emits that
// output in a deterministic order.
type sequencer struct {
maxWeight int64
sem *semaphore.Weighted // weighted by input bytes (an approximate proxy for memory overhead)
prev <-chan *reporterState // 1-buffered
}
// newSequencer returns a sequencer that allows concurrent tasks up to maxWeight
// and writes tasks' output to out and err.
func newSequencer(maxWeight int64, out, err io.Writer) *sequencer {
sem := semaphore.NewWeighted(maxWeight)
prev := make(chan *reporterState, 1)
prev <- &reporterState{out: out, err: err}
return &sequencer{
maxWeight: maxWeight,
sem: sem,
prev: prev,
}
}
// exclusive is a weight that can be passed to a sequencer to cause
// a task to be executed without any other concurrent tasks.
const exclusive = -1
// Add blocks until the sequencer has enough weight to spare, then adds f as a
// task to be executed concurrently.
//
// If the weight is either negative or larger than the sequencer's maximum
// weight, Add blocks until all other tasks have completed, then the task
// executes exclusively (blocking all other calls to Add until it completes).
//
// f may run concurrently in a goroutine, but its output to the passed-in
// reporter will be sequential relative to the other tasks in the sequencer.
//
// If f invokes a method on the reporter, execution of that method may block
// until the previous task has finished. (To maximize concurrency, f should
// avoid invoking the reporter until it has finished any parallelizable work.)
//
// If f returns a non-nil error, that error will be reported after f's output
// (if any) and will cause a nonzero final exit code.
func (s *sequencer) Add(weight int64, f func(*reporter) error) {
if weight < 0 || weight > s.maxWeight {
weight = s.maxWeight
}
if err := s.sem.Acquire(context.TODO(), weight); err != nil {
// Change the task from "execute f" to "report err".
weight = 0
f = func(*reporter) error { return err }
}
r := &reporter{prev: s.prev}
next := make(chan *reporterState, 1)
s.prev = next
// Start f in parallel: it can run until it invokes a method on r, at which
// point it will block until the previous task releases the output state.
go func() {
if err := f(r); err != nil {
r.Report(err)
}
next <- r.getState() // Release the next task.
s.sem.Release(weight)
}()
}
// AddReport prints an error to s after the output of any previously-added
// tasks, causing the final exit code to be nonzero.
func (s *sequencer) AddReport(err error) {
s.Add(0, func(*reporter) error { return err })
}
// GetExitCode waits for all previously-added tasks to complete, then returns an
// exit code for the sequence suitable for passing to os.Exit.
func (s *sequencer) GetExitCode() int {
c := make(chan int, 1)
s.Add(0, func(r *reporter) error {
c <- r.ExitCode()
return nil
})
return <-c
}
// A reporter reports output, warnings, and errors.
type reporter struct {
prev <-chan *reporterState
state *reporterState
}
// reporterState carries the state of a reporter instance.
//
// Only one reporter at a time may have access to a reporterState.
type reporterState struct {
out, err io.Writer
exitCode int
}
// getState blocks until any prior reporters are finished with the reporter
// state, then returns the state for manipulation.
func (r *reporter) getState() *reporterState {
if r.state == nil {
r.state = <-r.prev
}
return r.state
}
// Warnf emits a warning message to the reporter's error stream,
// without changing its exit code.
func (r *reporter) Warnf(format string, args ...any) {
fmt.Fprintf(r.getState().err, format, args...)
}
// Write emits a slice to the reporter's output stream.
//
// Any error is returned to the caller, and does not otherwise affect the
// reporter's exit code.
func (r *reporter) Write(p []byte) (int, error) {
return r.getState().out.Write(p)
}
// Report emits a non-nil error to the reporter's error stream,
// changing its exit code to a nonzero value.
func (r *reporter) Report(err error) {
if err == nil {
panic("Report with nil error")
}
st := r.getState()
scanner.PrintError(st.err, err)
st.exitCode = 2
}
func (r *reporter) ExitCode() int {
return r.getState().exitCode
}
// If info == nil, we are formatting stdin instead of a file.
// If in == nil, the source is the contents of the file with the given filename.
func processFile(filename string, info fs.FileInfo, in io.Reader, r *reporter, explicit bool) error {
src, err := readFile(filename, info, in)
if err != nil {
return err
}
fileSet := token.NewFileSet()
fragmentOk := false
if info == nil {
// If we are formatting stdin, we accept a program fragment in lieu of a
// complete source file.
fragmentOk = true
}
file, sourceAdj, indentAdj, err := parse(fileSet, filename, src, fragmentOk)
if err != nil {
return err
}
ast.SortImports(fileSet, file)
// Apply gofumpt's changes before we print the code in gofumpt's format.
// If either -lang or -modpath aren't set, fetch them from go.mod.
lang := *langVersion
modpath := *modulePath
if lang == "" || modpath == "" {
dir := filepath.Dir(filename)
mod, ok := moduleCacheByDir.Load(dir)
if ok && mod != nil {
mod := mod.(*module)
if lang == "" {
lang = "go" + mod.Go
}
if modpath == "" {
modpath = mod.Module.Path
}
}
}
// We always apply the gofumpt formatting rules to explicit files, including stdin.
// Otherwise, we don't apply them on generated files.
// We also skip walking vendor directories entirely, but that happens elsewhere.
if explicit || !isGenerated(file) {
gformat.File(fileSet, file, gformat.Options{
LangVersion: lang,
ModulePath: modpath,
ExtraRules: *extraRules,
})
}
res, err := format(fileSet, file, sourceAdj, indentAdj, src, printer.Config{Mode: printerMode, Tabwidth: tabWidth})
if err != nil {
return err
}
if !bytes.Equal(src, res) {
// formatting has changed
if *list {
fmt.Fprintln(r, filename)
}
if *write {
if info == nil {
panic("-w should not have been allowed with stdin")
}
// make a temporary backup before overwriting original
perm := info.Mode().Perm()
bakname, err := backupFile(filename+".", src, perm)
if err != nil {
return err
}
fdSem <- true
err = os.WriteFile(filename, res, perm)
<-fdSem
if err != nil {
os.Rename(bakname, filename)
return err
}
err = os.Remove(bakname)
if err != nil {
return err
}
}
if *doDiff {
newName := filepath.ToSlash(filename)
oldName := newName + ".orig"
r.Write(diff.Diff(oldName, src, newName, res))
}
}
if !*list && !*write && !*doDiff {
_, err = r.Write(res)
}
return err
}
// readFile reads the contents of filename, described by info.
// If in is non-nil, readFile reads directly from it.
// Otherwise, readFile opens and reads the file itself,
// with the number of concurrently-open files limited by fdSem.
func readFile(filename string, info fs.FileInfo, in io.Reader) ([]byte, error) {
if in == nil {
fdSem <- true
var err error
f, err := os.Open(filename)
if err != nil {
return nil, err
}
in = f
defer func() {
f.Close()
<-fdSem
}()
}
// Compute the file's size and read its contents with minimal allocations.
//
// If we have the FileInfo from filepath.WalkDir, use it to make
// a buffer of the right size and avoid ReadAll's reallocations.
//
// If the size is unknown (or bogus, or overflows an int), fall back to
// a size-independent ReadAll.
size := -1
if info != nil && info.Mode().IsRegular() && int64(int(info.Size())) == info.Size() {
size = int(info.Size())
}
if size+1 <= 0 {
// The file is not known to be regular, so we don't have a reliable size for it.
var err error
src, err := io.ReadAll(in)
if err != nil {
return nil, err
}
return src, nil
}
// We try to read size+1 bytes so that we can detect modifications: if we
// read more than size bytes, then the file was modified concurrently.
// (If that happens, we could, say, append to src to finish the read, or
// proceed with a truncated buffer — but the fact that it changed at all
// indicates a possible race with someone editing the file, so we prefer to
// stop to avoid corrupting it.)
src := make([]byte, size+1)
n, err := io.ReadFull(in, src)
switch err {
case nil, io.EOF, io.ErrUnexpectedEOF:
// io.ReadFull returns io.EOF (for an empty file) or io.ErrUnexpectedEOF
// (for a non-empty file) if the file was changed unexpectedly. Continue
// with comparing file sizes in those cases.
default:
return nil, err
}
if n < size {
return nil, fmt.Errorf("error: size of %s changed during reading (from %d to %d bytes)", filename, size, n)
} else if n > size {
return nil, fmt.Errorf("error: size of %s changed during reading (from %d to >=%d bytes)", filename, size, len(src))
}
return src[:n], nil
}
func main() { os.Exit(main1()) }
func main1() int {
// Arbitrarily limit in-flight work to 2MiB times the number of threads.
//
// The actual overhead for the parse tree and output will depend on the
// specifics of the file, but this at least keeps the footprint of the process
// roughly proportional to GOMAXPROCS.
maxWeight := (2 << 20) * int64(runtime.GOMAXPROCS(0))
s := newSequencer(maxWeight, os.Stdout, os.Stderr)
// call gofmtMain in a separate function
// so that it can use defer and have them
// run before the exit.
gofmtMain(s)
return s.GetExitCode()
}
func gofmtMain(s *sequencer) {
// Ensure our parsed files never start with base 1,
// to ensure that using token.NoPos+1 will panic.
fileSet.AddFile("gofumpt_base.go", 1, 10)
flag.Usage = usage
flag.Parse()
if *simplifyAST {
fmt.Fprintf(os.Stderr, "warning: -s is deprecated as it is always enabled\n")
}
if *rewriteRule != "" {
fmt.Fprintf(os.Stderr, `the rewrite flag is no longer available; use "gofmt -r" instead`+"\n")
os.Exit(2)
}
// Print the gofumpt version if the user asks for it.
if *showVersion {
fmt.Println(gversion.String(version))
return
}
if *cpuprofile != "" {
fdSem <- true
f, err := os.Create(*cpuprofile)
if err != nil {
s.AddReport(fmt.Errorf("creating cpu profile: %s", err))
return
}
defer func() {
f.Close()
<-fdSem
}()
pprof.StartCPUProfile(f)
defer pprof.StopCPUProfile()
}
initParserMode()
args := flag.Args()
if len(args) == 0 {
if *write {
s.AddReport(fmt.Errorf("error: cannot use -w with standard input"))
return
}
s.Add(0, func(r *reporter) error {
// TODO: test explicit==true
return processFile("<standard input>", nil, os.Stdin, r, true)
})
return
}
for _, arg := range args {
switch info, err := os.Stat(arg); {
case err != nil:
s.AddReport(err)
case !info.IsDir():
// Non-directory arguments are always formatted.
arg := arg
s.Add(fileWeight(arg, info), func(r *reporter) error {
return processFile(arg, info, nil, r, true)
})
default:
// Directories are walked, ignoring non-Go files.
err := filepath.WalkDir(arg, func(path string, f fs.DirEntry, err error) error {
// vendor and testdata directories are skipped,
// unless they are explicitly passed as an argument.
base := filepath.Base(path)
if path != arg && (base == "vendor" || base == "testdata") {
return filepath.SkipDir
}
if err != nil || !isGoFile(f) {
return err
}
info, err := f.Info()
if err != nil {
s.AddReport(err)
return nil
}
s.Add(fileWeight(path, info), func(r *reporter) error {
return processFile(path, info, nil, r, false)
})
return nil
})
if err != nil {
s.AddReport(err)
}
}
}
}
type module struct {
Go string
Module struct {
Path string
}
}
func loadModuleInfo(dir string) any {
cmd := exec.Command("go", "mod", "edit", "-json")
cmd.Dir = dir
// Spawning "go mod edit" will open files by design,
// such as the named pipe to obtain stdout.
// TODO(mvdan): if we run into "too many open files" errors again in the
// future, we probably need to turn fdSem into a weighted semaphore so this
// operation can acquire a weight larger than 1.
fdSem <- true
out, err := cmd.Output()
defer func() { <-fdSem }()
if err != nil || len(out) == 0 {
return nil
}
mod := new(module)
if err := json.Unmarshal(out, mod); err != nil {
return nil
}
return mod
}
// Written to by fileWeight, read from fileWeight and processFile.
// A present but nil value means that loading the module info failed.
// Note that we don't require the keys to be absolute directories,
// so duplicates are possible. The same can happen with symlinks.
var moduleCacheByDir sync.Map // map[dirString]*module
func fileWeight(path string, info fs.FileInfo) int64 {
dir := filepath.Dir(path)
if _, ok := moduleCacheByDir.Load(dir); !ok {
moduleCacheByDir.Store(dir, loadModuleInfo(dir))
}
if info == nil {
return exclusive
}
if info.Mode().Type() == fs.ModeSymlink {
var err error
info, err = os.Stat(path)
if err != nil {
return exclusive
}
}
if !info.Mode().IsRegular() {
// For non-regular files, FileInfo.Size is system-dependent and thus not a
// reliable indicator of weight.
return exclusive
}
return info.Size()
}
const chmodSupported = runtime.GOOS != "windows"
// backupFile writes data to a new file named filename<number> with permissions perm,
// with <number randomly chosen such that the file name is unique. backupFile returns
// the chosen file name.
func backupFile(filename string, data []byte, perm fs.FileMode) (string, error) {
fdSem <- true
defer func() { <-fdSem }()
// create backup file
f, err := os.CreateTemp(filepath.Dir(filename), filepath.Base(filename))
if err != nil {
return "", err
}
bakname := f.Name()
if chmodSupported {
err = f.Chmod(perm)
if err != nil {
f.Close()
os.Remove(bakname)
return bakname, err
}
}
// write data to backup file
_, err = f.Write(data)
if err1 := f.Close(); err == nil {
err = err1
}
return bakname, err
}
@@ -0,0 +1,177 @@
// Copyright 2015 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// TODO(gri): This file and the file src/go/format/internal.go are
// the same (but for this comment and the package name). Do not modify
// one without the other. Determine if we can factor out functionality
// in a public API. See also #11844 for context.
package main
import (
"bytes"
"go/ast"
"go/parser"
"go/token"
"strings"
"mvdan.cc/gofumpt/internal/govendor/go/printer"
)
// parse parses src, which was read from the named file,
// as a Go source file, declaration, or statement list.
func parse(fset *token.FileSet, filename string, src []byte, fragmentOk bool) (
file *ast.File,
sourceAdj func(src []byte, indent int) []byte,
indentAdj int,
err error,
) {
// Try as whole source file.
file, err = parser.ParseFile(fset, filename, src, parserMode)
// If there's no error, return. If the error is that the source file didn't begin with a
// package line and source fragments are ok, fall through to
// try as a source fragment. Stop and return on any other error.
if err == nil || !fragmentOk || !strings.Contains(err.Error(), "expected 'package'") {
return
}
// If this is a declaration list, make it a source file
// by inserting a package clause.
// Insert using a ';', not a newline, so that the line numbers
// in psrc match the ones in src.
psrc := append([]byte("package p;"), src...)
file, err = parser.ParseFile(fset, filename, psrc, parserMode)
if err == nil {
sourceAdj = func(src []byte, indent int) []byte {
// Remove the package clause.
// Gofmt has turned the ';' into a '\n'.
src = src[indent+len("package p\n"):]
return bytes.TrimSpace(src)
}
return
}
// If the error is that the source file didn't begin with a
// declaration, fall through to try as a statement list.
// Stop and return on any other error.
if !strings.Contains(err.Error(), "expected declaration") {
return
}
// If this is a statement list, make it a source file
// by inserting a package clause and turning the list
// into a function body. This handles expressions too.
// Insert using a ';', not a newline, so that the line numbers
// in fsrc match the ones in src. Add an extra '\n' before the '}'
// to make sure comments are flushed before the '}'.
fsrc := append(append([]byte("package p; func _() {"), src...), '\n', '\n', '}')
file, err = parser.ParseFile(fset, filename, fsrc, parserMode)
if err == nil {
sourceAdj = func(src []byte, indent int) []byte {
// Cap adjusted indent to zero.
if indent < 0 {
indent = 0
}
// Remove the wrapping.
// Gofmt has turned the "; " into a "\n\n".
// There will be two non-blank lines with indent, hence 2*indent.
src = src[2*indent+len("package p\n\nfunc _() {"):]
// Remove only the "}\n" suffix: remaining whitespaces will be trimmed anyway
src = src[:len(src)-len("}\n")]
return bytes.TrimSpace(src)
}
// Gofmt has also indented the function body one level.
// Adjust that with indentAdj.
indentAdj = -1
}
// Succeeded, or out of options.
return
}
// format formats the given package file originally obtained from src
// and adjusts the result based on the original source via sourceAdj
// and indentAdj.
func format(
fset *token.FileSet,
file *ast.File,
sourceAdj func(src []byte, indent int) []byte,
indentAdj int,
src []byte,
cfg printer.Config,
) ([]byte, error) {
if sourceAdj == nil {
// Complete source file.
var buf bytes.Buffer
err := cfg.Fprint(&buf, fset, file)
if err != nil {
return nil, err
}
return buf.Bytes(), nil
}
// Partial source file.
// Determine and prepend leading space.
i, j := 0, 0
for j < len(src) && isSpace(src[j]) {
if src[j] == '\n' {
i = j + 1 // byte offset of last line in leading space
}
j++
}
var res []byte
res = append(res, src[:i]...)
// Determine and prepend indentation of first code line.
// Spaces are ignored unless there are no tabs,
// in which case spaces count as one tab.
indent := 0
hasSpace := false
for _, b := range src[i:j] {
switch b {
case ' ':
hasSpace = true
case '\t':
indent++
}
}
if indent == 0 && hasSpace {
indent = 1
}
for i := 0; i < indent; i++ {
res = append(res, '\t')
}
// Format the source.
// Write it without any leading and trailing space.
cfg.Indent = indent + indentAdj
var buf bytes.Buffer
err := cfg.Fprint(&buf, fset, file)
if err != nil {
return nil, err
}
out := sourceAdj(buf.Bytes(), cfg.Indent)
// If the adjusted output is empty, the source
// was empty but (possibly) for white space.
// The result is the incoming source.
if len(out) == 0 {
return src, nil
}
// Otherwise, append output to leading space.
res = append(res, out...)
// Determine and append trailing space.
i = len(src)
for i > 0 && isSpace(src[i-1]) {
i--
}
return append(res, src[i:]...), nil
}
// isSpace reports whether the byte is a space character.
// isSpace defines a space as being among the following bytes: ' ', '\t', '\n' and '\r'.
func isSpace(b byte) bool {
return b == ' ' || b == '\t' || b == '\n' || b == '\r'
}
@@ -0,0 +1,261 @@
// Copyright 2022 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package diff
import (
"bytes"
"fmt"
"sort"
"strings"
)
// A pair is a pair of values tracked for both the x and y side of a diff.
// It is typically a pair of line indexes.
type pair struct{ x, y int }
// Diff returns an anchored diff of the two texts old and new
// in the “unified diff” format. If old and new are identical,
// Diff returns a nil slice (no output).
//
// Unix diff implementations typically look for a diff with
// the smallest number of lines inserted and removed,
// which can in the worst case take time quadratic in the
// number of lines in the texts. As a result, many implementations
// either can be made to run for a long time or cut off the search
// after a predetermined amount of work.
//
// In contrast, this implementation looks for a diff with the
// smallest number of “unique” lines inserted and removed,
// where unique means a line that appears just once in both old and new.
// We call this an “anchored diff” because the unique lines anchor
// the chosen matching regions. An anchored diff is usually clearer
// than a standard diff, because the algorithm does not try to
// reuse unrelated blank lines or closing braces.
// The algorithm also guarantees to run in O(n log n) time
// instead of the standard O(n²) time.
//
// Some systems call this approach a “patience diff,” named for
// the “patience sorting” algorithm, itself named for a solitaire card game.
// We avoid that name for two reasons. First, the name has been used
// for a few different variants of the algorithm, so it is imprecise.
// Second, the name is frequently interpreted as meaning that you have
// to wait longer (to be patient) for the diff, meaning that it is a slower algorithm,
// when in fact the algorithm is faster than the standard one.
func Diff(oldName string, old []byte, newName string, new []byte) []byte {
if bytes.Equal(old, new) {
return nil
}
x := lines(old)
y := lines(new)
// Print diff header.
var out bytes.Buffer
fmt.Fprintf(&out, "diff %s %s\n", oldName, newName)
fmt.Fprintf(&out, "--- %s\n", oldName)
fmt.Fprintf(&out, "+++ %s\n", newName)
// Loop over matches to consider,
// expanding each match to include surrounding lines,
// and then printing diff chunks.
// To avoid setup/teardown cases outside the loop,
// tgs returns a leading {0,0} and trailing {len(x), len(y)} pair
// in the sequence of matches.
var (
done pair // printed up to x[:done.x] and y[:done.y]
chunk pair // start lines of current chunk
count pair // number of lines from each side in current chunk
ctext []string // lines for current chunk
)
for _, m := range tgs(x, y) {
if m.x < done.x {
// Already handled scanning forward from earlier match.
continue
}
// Expand matching lines as far as possible,
// establishing that x[start.x:end.x] == y[start.y:end.y].
// Note that on the first (or last) iteration we may (or definitely do)
// have an empty match: start.x==end.x and start.y==end.y.
start := m
for start.x > done.x && start.y > done.y && x[start.x-1] == y[start.y-1] {
start.x--
start.y--
}
end := m
for end.x < len(x) && end.y < len(y) && x[end.x] == y[end.y] {
end.x++
end.y++
}
// Emit the mismatched lines before start into this chunk.
// (No effect on first sentinel iteration, when start = {0,0}.)
for _, s := range x[done.x:start.x] {
ctext = append(ctext, "-"+s)
count.x++
}
for _, s := range y[done.y:start.y] {
ctext = append(ctext, "+"+s)
count.y++
}
// If we're not at EOF and have too few common lines,
// the chunk includes all the common lines and continues.
const C = 3 // number of context lines
if (end.x < len(x) || end.y < len(y)) &&
(end.x-start.x < C || (len(ctext) > 0 && end.x-start.x < 2*C)) {
for _, s := range x[start.x:end.x] {
ctext = append(ctext, " "+s)
count.x++
count.y++
}
done = end
continue
}
// End chunk with common lines for context.
if len(ctext) > 0 {
n := end.x - start.x
if n > C {
n = C
}
for _, s := range x[start.x : start.x+n] {
ctext = append(ctext, " "+s)
count.x++
count.y++
}
done = pair{start.x + n, start.y + n}
// Format and emit chunk.
// Convert line numbers to 1-indexed.
// Special case: empty file shows up as 0,0 not 1,0.
if count.x > 0 {
chunk.x++
}
if count.y > 0 {
chunk.y++
}
fmt.Fprintf(&out, "@@ -%d,%d +%d,%d @@\n", chunk.x, count.x, chunk.y, count.y)
for _, s := range ctext {
out.WriteString(s)
}
count.x = 0
count.y = 0
ctext = ctext[:0]
}
// If we reached EOF, we're done.
if end.x >= len(x) && end.y >= len(y) {
break
}
// Otherwise start a new chunk.
chunk = pair{end.x - C, end.y - C}
for _, s := range x[chunk.x:end.x] {
ctext = append(ctext, " "+s)
count.x++
count.y++
}
done = end
}
return out.Bytes()
}
// lines returns the lines in the file x, including newlines.
// If the file does not end in a newline, one is supplied
// along with a warning about the missing newline.
func lines(x []byte) []string {
l := strings.SplitAfter(string(x), "\n")
if l[len(l)-1] == "" {
l = l[:len(l)-1]
} else {
// Treat last line as having a message about the missing newline attached,
// using the same text as BSD/GNU diff (including the leading backslash).
l[len(l)-1] += "\n\\ No newline at end of file\n"
}
return l
}
// tgs returns the pairs of indexes of the longest common subsequence
// of unique lines in x and y, where a unique line is one that appears
// once in x and once in y.
//
// The longest common subsequence algorithm is as described in
// Thomas G. Szymanski, “A Special Case of the Maximal Common
// Subsequence Problem,” Princeton TR #170 (January 1975),
// available at https://research.swtch.com/tgs170.pdf.
func tgs(x, y []string) []pair {
// Count the number of times each string appears in a and b.
// We only care about 0, 1, many, counted as 0, -1, -2
// for the x side and 0, -4, -8 for the y side.
// Using negative numbers now lets us distinguish positive line numbers later.
m := make(map[string]int)
for _, s := range x {
if c := m[s]; c > -2 {
m[s] = c - 1
}
}
for _, s := range y {
if c := m[s]; c > -8 {
m[s] = c - 4
}
}
// Now unique strings can be identified by m[s] = -1+-4.
//
// Gather the indexes of those strings in x and y, building:
// xi[i] = increasing indexes of unique strings in x.
// yi[i] = increasing indexes of unique strings in y.
// inv[i] = index j such that x[xi[i]] = y[yi[j]].
var xi, yi, inv []int
for i, s := range y {
if m[s] == -1+-4 {
m[s] = len(yi)
yi = append(yi, i)
}
}
for i, s := range x {
if j, ok := m[s]; ok && j >= 0 {
xi = append(xi, i)
inv = append(inv, j)
}
}
// Apply Algorithm A from Szymanski's paper.
// In those terms, A = J = inv and B = [0, n).
// We add sentinel pairs {0,0}, and {len(x),len(y)}
// to the returned sequence, to help the processing loop.
J := inv
n := len(xi)
T := make([]int, n)
L := make([]int, n)
for i := range T {
T[i] = n + 1
}
for i := 0; i < n; i++ {
k := sort.Search(n, func(k int) bool {
return T[k] >= J[i]
})
T[k] = J[i]
L[i] = k + 1
}
k := 0
for _, v := range L {
if k < v {
k = v
}
}
seq := make([]pair, 2+k)
seq[1+k] = pair{len(x), len(y)} // sentinel at end
lastj := n
for i := n - 1; i >= 0; i-- {
if L[i] == k && J[i] < lastj {
seq[k] = pair{xi[i], yi[J[i]]}
k--
}
}
seq[0] = pair{0, 0} // sentinel at start
return seq
}
@@ -0,0 +1,36 @@
// Copyright 2022 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
/*
Package comment implements parsing and reformatting of Go doc comments,
(documentation comments), which are comments that immediately precede
a top-level declaration of a package, const, func, type, or var.
Go doc comment syntax is a simplified subset of Markdown that supports
links, headings, paragraphs, lists (without nesting), and preformatted text blocks.
The details of the syntax are documented at https://go.dev/doc/comment.
To parse the text associated with a doc comment (after removing comment markers),
use a [Parser]:
var p comment.Parser
doc := p.Parse(text)
The result is a [*Doc].
To reformat it as a doc comment, HTML, Markdown, or plain text,
use a [Printer]:
var pr comment.Printer
os.Stdout.Write(pr.Text(doc))
The [Parser] and [Printer] types are structs whose fields can be
modified to customize the operations.
For details, see the documentation for those types.
Use cases that need additional control over reformatting can
implement their own logic by inspecting the parsed syntax itself.
See the documentation for [Doc], [Block], [Text] for an overview
and links to additional types.
*/
package comment
@@ -0,0 +1,169 @@
// Copyright 2022 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package comment
import (
"bytes"
"fmt"
"strconv"
)
// An htmlPrinter holds the state needed for printing a [Doc] as HTML.
type htmlPrinter struct {
*Printer
tight bool
}
// HTML returns an HTML formatting of the [Doc].
// See the [Printer] documentation for ways to customize the HTML output.
func (p *Printer) HTML(d *Doc) []byte {
hp := &htmlPrinter{Printer: p}
var out bytes.Buffer
for _, x := range d.Content {
hp.block(&out, x)
}
return out.Bytes()
}
// block prints the block x to out.
func (p *htmlPrinter) block(out *bytes.Buffer, x Block) {
switch x := x.(type) {
default:
fmt.Fprintf(out, "?%T", x)
case *Paragraph:
if !p.tight {
out.WriteString("<p>")
}
p.text(out, x.Text)
out.WriteString("\n")
case *Heading:
out.WriteString("<h")
h := strconv.Itoa(p.headingLevel())
out.WriteString(h)
if id := p.headingID(x); id != "" {
out.WriteString(` id="`)
p.escape(out, id)
out.WriteString(`"`)
}
out.WriteString(">")
p.text(out, x.Text)
out.WriteString("</h")
out.WriteString(h)
out.WriteString(">\n")
case *Code:
out.WriteString("<pre>")
p.escape(out, x.Text)
out.WriteString("</pre>\n")
case *List:
kind := "ol>\n"
if x.Items[0].Number == "" {
kind = "ul>\n"
}
out.WriteString("<")
out.WriteString(kind)
next := "1"
for _, item := range x.Items {
out.WriteString("<li")
if n := item.Number; n != "" {
if n != next {
out.WriteString(` value="`)
out.WriteString(n)
out.WriteString(`"`)
next = n
}
next = inc(next)
}
out.WriteString(">")
p.tight = !x.BlankBetween()
for _, blk := range item.Content {
p.block(out, blk)
}
p.tight = false
}
out.WriteString("</")
out.WriteString(kind)
}
}
// inc increments the decimal string s.
// For example, inc("1199") == "1200".
func inc(s string) string {
b := []byte(s)
for i := len(b) - 1; i >= 0; i-- {
if b[i] < '9' {
b[i]++
return string(b)
}
b[i] = '0'
}
return "1" + string(b)
}
// text prints the text sequence x to out.
func (p *htmlPrinter) text(out *bytes.Buffer, x []Text) {
for _, t := range x {
switch t := t.(type) {
case Plain:
p.escape(out, string(t))
case Italic:
out.WriteString("<i>")
p.escape(out, string(t))
out.WriteString("</i>")
case *Link:
out.WriteString(`<a href="`)
p.escape(out, t.URL)
out.WriteString(`">`)
p.text(out, t.Text)
out.WriteString("</a>")
case *DocLink:
url := p.docLinkURL(t)
if url != "" {
out.WriteString(`<a href="`)
p.escape(out, url)
out.WriteString(`">`)
}
p.text(out, t.Text)
if url != "" {
out.WriteString("</a>")
}
}
}
}
// escape prints s to out as plain text,
// escaping < & " ' and > to avoid being misinterpreted
// in larger HTML constructs.
func (p *htmlPrinter) escape(out *bytes.Buffer, s string) {
start := 0
for i := 0; i < len(s); i++ {
switch s[i] {
case '<':
out.WriteString(s[start:i])
out.WriteString("&lt;")
start = i + 1
case '&':
out.WriteString(s[start:i])
out.WriteString("&amp;")
start = i + 1
case '"':
out.WriteString(s[start:i])
out.WriteString("&quot;")
start = i + 1
case '\'':
out.WriteString(s[start:i])
out.WriteString("&apos;")
start = i + 1
case '>':
out.WriteString(s[start:i])
out.WriteString("&gt;")
start = i + 1
}
}
out.WriteString(s[start:])
}
@@ -0,0 +1,188 @@
// Copyright 2022 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package comment
import (
"bytes"
"fmt"
"strings"
)
// An mdPrinter holds the state needed for printing a Doc as Markdown.
type mdPrinter struct {
*Printer
headingPrefix string
raw bytes.Buffer
}
// Markdown returns a Markdown formatting of the Doc.
// See the [Printer] documentation for ways to customize the Markdown output.
func (p *Printer) Markdown(d *Doc) []byte {
mp := &mdPrinter{
Printer: p,
headingPrefix: strings.Repeat("#", p.headingLevel()) + " ",
}
var out bytes.Buffer
for i, x := range d.Content {
if i > 0 {
out.WriteByte('\n')
}
mp.block(&out, x)
}
return out.Bytes()
}
// block prints the block x to out.
func (p *mdPrinter) block(out *bytes.Buffer, x Block) {
switch x := x.(type) {
default:
fmt.Fprintf(out, "?%T", x)
case *Paragraph:
p.text(out, x.Text)
out.WriteString("\n")
case *Heading:
out.WriteString(p.headingPrefix)
p.text(out, x.Text)
if id := p.headingID(x); id != "" {
out.WriteString(" {#")
out.WriteString(id)
out.WriteString("}")
}
out.WriteString("\n")
case *Code:
md := x.Text
for md != "" {
var line string
line, md, _ = strings.Cut(md, "\n")
if line != "" {
out.WriteString("\t")
out.WriteString(line)
}
out.WriteString("\n")
}
case *List:
loose := x.BlankBetween()
for i, item := range x.Items {
if i > 0 && loose {
out.WriteString("\n")
}
if n := item.Number; n != "" {
out.WriteString(" ")
out.WriteString(n)
out.WriteString(". ")
} else {
out.WriteString(" - ") // SP SP - SP
}
for i, blk := range item.Content {
const fourSpace = " "
if i > 0 {
out.WriteString("\n" + fourSpace)
}
p.text(out, blk.(*Paragraph).Text)
out.WriteString("\n")
}
}
}
}
// text prints the text sequence x to out.
func (p *mdPrinter) text(out *bytes.Buffer, x []Text) {
p.raw.Reset()
p.rawText(&p.raw, x)
line := bytes.TrimSpace(p.raw.Bytes())
if len(line) == 0 {
return
}
switch line[0] {
case '+', '-', '*', '#':
// Escape what would be the start of an unordered list or heading.
out.WriteByte('\\')
case '0', '1', '2', '3', '4', '5', '6', '7', '8', '9':
i := 1
for i < len(line) && '0' <= line[i] && line[i] <= '9' {
i++
}
if i < len(line) && (line[i] == '.' || line[i] == ')') {
// Escape what would be the start of an ordered list.
out.Write(line[:i])
out.WriteByte('\\')
line = line[i:]
}
}
out.Write(line)
}
// rawText prints the text sequence x to out,
// without worrying about escaping characters
// that have special meaning at the start of a Markdown line.
func (p *mdPrinter) rawText(out *bytes.Buffer, x []Text) {
for _, t := range x {
switch t := t.(type) {
case Plain:
p.escape(out, string(t))
case Italic:
out.WriteString("*")
p.escape(out, string(t))
out.WriteString("*")
case *Link:
out.WriteString("[")
p.rawText(out, t.Text)
out.WriteString("](")
out.WriteString(t.URL)
out.WriteString(")")
case *DocLink:
url := p.docLinkURL(t)
if url != "" {
out.WriteString("[")
}
p.rawText(out, t.Text)
if url != "" {
out.WriteString("](")
url = strings.ReplaceAll(url, "(", "%28")
url = strings.ReplaceAll(url, ")", "%29")
out.WriteString(url)
out.WriteString(")")
}
}
}
}
// escape prints s to out as plain text,
// escaping special characters to avoid being misinterpreted
// as Markdown markup sequences.
func (p *mdPrinter) escape(out *bytes.Buffer, s string) {
start := 0
for i := 0; i < len(s); i++ {
switch s[i] {
case '\n':
// Turn all \n into spaces, for a few reasons:
// - Avoid introducing paragraph breaks accidentally.
// - Avoid the need to reindent after the newline.
// - Avoid problems with Markdown renderers treating
// every mid-paragraph newline as a <br>.
out.WriteString(s[start:i])
out.WriteByte(' ')
start = i + 1
continue
case '`', '_', '*', '[', '<', '\\':
// Not all of these need to be escaped all the time,
// but is valid and easy to do so.
// We assume the Markdown is being passed to a
// Markdown renderer, not edited by a person,
// so it's fine to have escapes that are not strictly
// necessary in some cases.
out.WriteString(s[start:i])
out.WriteByte('\\')
out.WriteByte(s[i])
start = i + 1
}
}
out.WriteString(s[start:])
}
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,288 @@
// Copyright 2022 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package comment
import (
"bytes"
"fmt"
"strings"
)
// A Printer is a doc comment printer.
// The fields in the struct can be filled in before calling
// any of the printing methods
// in order to customize the details of the printing process.
type Printer struct {
// HeadingLevel is the nesting level used for
// HTML and Markdown headings.
// If HeadingLevel is zero, it defaults to level 3,
// meaning to use <h3> and ###.
HeadingLevel int
// HeadingID is a function that computes the heading ID
// (anchor tag) to use for the heading h when generating
// HTML and Markdown. If HeadingID returns an empty string,
// then the heading ID is omitted.
// If HeadingID is nil, h.DefaultID is used.
HeadingID func(h *Heading) string
// DocLinkURL is a function that computes the URL for the given DocLink.
// If DocLinkURL is nil, then link.DefaultURL(p.DocLinkBaseURL) is used.
DocLinkURL func(link *DocLink) string
// DocLinkBaseURL is used when DocLinkURL is nil,
// passed to [DocLink.DefaultURL] to construct a DocLink's URL.
// See that method's documentation for details.
DocLinkBaseURL string
// TextPrefix is a prefix to print at the start of every line
// when generating text output using the Text method.
TextPrefix string
// TextCodePrefix is the prefix to print at the start of each
// preformatted (code block) line when generating text output,
// instead of (not in addition to) TextPrefix.
// If TextCodePrefix is the empty string, it defaults to TextPrefix+"\t".
TextCodePrefix string
// TextWidth is the maximum width text line to generate,
// measured in Unicode code points,
// excluding TextPrefix and the newline character.
// If TextWidth is zero, it defaults to 80 minus the number of code points in TextPrefix.
// If TextWidth is negative, there is no limit.
TextWidth int
}
func (p *Printer) headingLevel() int {
if p.HeadingLevel <= 0 {
return 3
}
return p.HeadingLevel
}
func (p *Printer) headingID(h *Heading) string {
if p.HeadingID == nil {
return h.DefaultID()
}
return p.HeadingID(h)
}
func (p *Printer) docLinkURL(link *DocLink) string {
if p.DocLinkURL != nil {
return p.DocLinkURL(link)
}
return link.DefaultURL(p.DocLinkBaseURL)
}
// DefaultURL constructs and returns the documentation URL for l,
// using baseURL as a prefix for links to other packages.
//
// The possible forms returned by DefaultURL are:
// - baseURL/ImportPath, for a link to another package
// - baseURL/ImportPath#Name, for a link to a const, func, type, or var in another package
// - baseURL/ImportPath#Recv.Name, for a link to a method in another package
// - #Name, for a link to a const, func, type, or var in this package
// - #Recv.Name, for a link to a method in this package
//
// If baseURL ends in a trailing slash, then DefaultURL inserts
// a slash between ImportPath and # in the anchored forms.
// For example, here are some baseURL values and URLs they can generate:
//
// "/pkg/" → "/pkg/math/#Sqrt"
// "/pkg" → "/pkg/math#Sqrt"
// "/" → "/math/#Sqrt"
// "" → "/math#Sqrt"
func (l *DocLink) DefaultURL(baseURL string) string {
if l.ImportPath != "" {
slash := ""
if strings.HasSuffix(baseURL, "/") {
slash = "/"
} else {
baseURL += "/"
}
switch {
case l.Name == "":
return baseURL + l.ImportPath + slash
case l.Recv != "":
return baseURL + l.ImportPath + slash + "#" + l.Recv + "." + l.Name
default:
return baseURL + l.ImportPath + slash + "#" + l.Name
}
}
if l.Recv != "" {
return "#" + l.Recv + "." + l.Name
}
return "#" + l.Name
}
// DefaultID returns the default anchor ID for the heading h.
//
// The default anchor ID is constructed by converting every
// rune that is not alphanumeric ASCII to an underscore
// and then adding the prefix “hdr-”.
// For example, if the heading text is “Go Doc Comments”,
// the default ID is “hdr-Go_Doc_Comments”.
func (h *Heading) DefaultID() string {
// Note: The “hdr-” prefix is important to avoid DOM clobbering attacks.
// See https://pkg.go.dev/github.com/google/safehtml#Identifier.
var out strings.Builder
var p textPrinter
p.oneLongLine(&out, h.Text)
s := strings.TrimSpace(out.String())
if s == "" {
return ""
}
out.Reset()
out.WriteString("hdr-")
for _, r := range s {
if r < 0x80 && isIdentASCII(byte(r)) {
out.WriteByte(byte(r))
} else {
out.WriteByte('_')
}
}
return out.String()
}
type commentPrinter struct {
*Printer
}
// Comment returns the standard Go formatting of the [Doc],
// without any comment markers.
func (p *Printer) Comment(d *Doc) []byte {
cp := &commentPrinter{Printer: p}
var out bytes.Buffer
for i, x := range d.Content {
if i > 0 && blankBefore(x) {
out.WriteString("\n")
}
cp.block(&out, x)
}
// Print one block containing all the link definitions that were used,
// and then a second block containing all the unused ones.
// This makes it easy to clean up the unused ones: gofmt and
// delete the final block. And it's a nice visual signal without
// affecting the way the comment formats for users.
for i := 0; i < 2; i++ {
used := i == 0
first := true
for _, def := range d.Links {
if def.Used == used {
if first {
out.WriteString("\n")
first = false
}
out.WriteString("[")
out.WriteString(def.Text)
out.WriteString("]: ")
out.WriteString(def.URL)
out.WriteString("\n")
}
}
}
return out.Bytes()
}
// blankBefore reports whether the block x requires a blank line before it.
// All blocks do, except for Lists that return false from x.BlankBefore().
func blankBefore(x Block) bool {
if x, ok := x.(*List); ok {
return x.BlankBefore()
}
return true
}
// block prints the block x to out.
func (p *commentPrinter) block(out *bytes.Buffer, x Block) {
switch x := x.(type) {
default:
fmt.Fprintf(out, "?%T", x)
case *Paragraph:
p.text(out, "", x.Text)
out.WriteString("\n")
case *Heading:
out.WriteString("# ")
p.text(out, "", x.Text)
out.WriteString("\n")
case *Code:
md := x.Text
for md != "" {
var line string
line, md, _ = strings.Cut(md, "\n")
if line != "" {
out.WriteString("\t")
out.WriteString(line)
}
out.WriteString("\n")
}
case *List:
loose := x.BlankBetween()
for i, item := range x.Items {
if i > 0 && loose {
out.WriteString("\n")
}
out.WriteString(" ")
if item.Number == "" {
out.WriteString(" - ")
} else {
out.WriteString(item.Number)
out.WriteString(". ")
}
for i, blk := range item.Content {
const fourSpace = " "
if i > 0 {
out.WriteString("\n" + fourSpace)
}
p.text(out, fourSpace, blk.(*Paragraph).Text)
out.WriteString("\n")
}
}
}
}
// text prints the text sequence x to out.
func (p *commentPrinter) text(out *bytes.Buffer, indent string, x []Text) {
for _, t := range x {
switch t := t.(type) {
case Plain:
p.indent(out, indent, string(t))
case Italic:
p.indent(out, indent, string(t))
case *Link:
if t.Auto {
p.text(out, indent, t.Text)
} else {
out.WriteString("[")
p.text(out, indent, t.Text)
out.WriteString("]")
}
case *DocLink:
out.WriteString("[")
p.text(out, indent, t.Text)
out.WriteString("]")
}
}
}
// indent prints s to out, indenting with the indent string
// after each newline in s.
func (p *commentPrinter) indent(out *bytes.Buffer, indent, s string) {
for s != "" {
line, rest, ok := strings.Cut(s, "\n")
out.WriteString(line)
if ok {
out.WriteString("\n")
out.WriteString(indent)
}
s = rest
}
}
@@ -0,0 +1,50 @@
// Copyright 2022 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Code generated by 'go generate' DO NOT EDIT.
//disabled go:generate ./mkstd.sh
package comment
var stdPkgs = []string{
"bufio",
"bytes",
"cmp",
"context",
"crypto",
"embed",
"encoding",
"errors",
"expvar",
"flag",
"fmt",
"hash",
"html",
"image",
"io",
"iter",
"log",
"maps",
"math",
"mime",
"net",
"os",
"path",
"plugin",
"reflect",
"regexp",
"runtime",
"slices",
"sort",
"strconv",
"strings",
"structs",
"sync",
"syscall",
"testing",
"time",
"unicode",
"unique",
"unsafe",
}
@@ -0,0 +1,337 @@
// Copyright 2022 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package comment
import (
"bytes"
"fmt"
"sort"
"strings"
"unicode/utf8"
)
// A textPrinter holds the state needed for printing a Doc as plain text.
type textPrinter struct {
*Printer
long strings.Builder
prefix string
codePrefix string
width int
}
// Text returns a textual formatting of the [Doc].
// See the [Printer] documentation for ways to customize the text output.
func (p *Printer) Text(d *Doc) []byte {
tp := &textPrinter{
Printer: p,
prefix: p.TextPrefix,
codePrefix: p.TextCodePrefix,
width: p.TextWidth,
}
if tp.codePrefix == "" {
tp.codePrefix = p.TextPrefix + "\t"
}
if tp.width == 0 {
tp.width = 80 - utf8.RuneCountInString(tp.prefix)
}
var out bytes.Buffer
for i, x := range d.Content {
if i > 0 && blankBefore(x) {
out.WriteString(tp.prefix)
writeNL(&out)
}
tp.block(&out, x)
}
anyUsed := false
for _, def := range d.Links {
if def.Used {
anyUsed = true
break
}
}
if anyUsed {
writeNL(&out)
for _, def := range d.Links {
if def.Used {
fmt.Fprintf(&out, "[%s]: %s\n", def.Text, def.URL)
}
}
}
return out.Bytes()
}
// writeNL calls out.WriteByte('\n')
// but first trims trailing spaces on the previous line.
func writeNL(out *bytes.Buffer) {
// Trim trailing spaces.
data := out.Bytes()
n := 0
for n < len(data) && (data[len(data)-n-1] == ' ' || data[len(data)-n-1] == '\t') {
n++
}
if n > 0 {
out.Truncate(len(data) - n)
}
out.WriteByte('\n')
}
// block prints the block x to out.
func (p *textPrinter) block(out *bytes.Buffer, x Block) {
switch x := x.(type) {
default:
fmt.Fprintf(out, "?%T\n", x)
case *Paragraph:
out.WriteString(p.prefix)
p.text(out, "", x.Text)
case *Heading:
out.WriteString(p.prefix)
out.WriteString("# ")
p.text(out, "", x.Text)
case *Code:
text := x.Text
for text != "" {
var line string
line, text, _ = strings.Cut(text, "\n")
if line != "" {
out.WriteString(p.codePrefix)
out.WriteString(line)
}
writeNL(out)
}
case *List:
loose := x.BlankBetween()
for i, item := range x.Items {
if i > 0 && loose {
out.WriteString(p.prefix)
writeNL(out)
}
out.WriteString(p.prefix)
out.WriteString(" ")
if item.Number == "" {
out.WriteString(" - ")
} else {
out.WriteString(item.Number)
out.WriteString(". ")
}
for i, blk := range item.Content {
const fourSpace = " "
if i > 0 {
writeNL(out)
out.WriteString(p.prefix)
out.WriteString(fourSpace)
}
p.text(out, fourSpace, blk.(*Paragraph).Text)
}
}
}
}
// text prints the text sequence x to out.
func (p *textPrinter) text(out *bytes.Buffer, indent string, x []Text) {
p.oneLongLine(&p.long, x)
words := strings.Fields(p.long.String())
p.long.Reset()
var seq []int
if p.width < 0 || len(words) == 0 {
seq = []int{0, len(words)} // one long line
} else {
seq = wrap(words, p.width-utf8.RuneCountInString(indent))
}
for i := 0; i+1 < len(seq); i++ {
if i > 0 {
out.WriteString(p.prefix)
out.WriteString(indent)
}
for j, w := range words[seq[i]:seq[i+1]] {
if j > 0 {
out.WriteString(" ")
}
out.WriteString(w)
}
writeNL(out)
}
}
// oneLongLine prints the text sequence x to out as one long line,
// without worrying about line wrapping.
// Explicit links have the [ ] dropped to improve readability.
func (p *textPrinter) oneLongLine(out *strings.Builder, x []Text) {
for _, t := range x {
switch t := t.(type) {
case Plain:
out.WriteString(string(t))
case Italic:
out.WriteString(string(t))
case *Link:
p.oneLongLine(out, t.Text)
case *DocLink:
p.oneLongLine(out, t.Text)
}
}
}
// wrap wraps words into lines of at most max runes,
// minimizing the sum of the squares of the leftover lengths
// at the end of each line (except the last, of course),
// with a preference for ending lines at punctuation (.,:;).
//
// The returned slice gives the indexes of the first words
// on each line in the wrapped text with a final entry of len(words).
// Thus the lines are words[seq[0]:seq[1]], words[seq[1]:seq[2]],
// ..., words[seq[len(seq)-2]:seq[len(seq)-1]].
//
// The implementation runs in O(n log n) time, where n = len(words),
// using the algorithm described in D. S. Hirschberg and L. L. Larmore,
// “[The least weight subsequence problem],” FOCS 1985, pp. 137-143.
//
// [The least weight subsequence problem]: https://doi.org/10.1109/SFCS.1985.60
func wrap(words []string, max int) (seq []int) {
// The algorithm requires that our scoring function be concave,
// meaning that for all i₀ ≤ i₁ < j₀ ≤ j₁,
// weight(i₀, j₀) + weight(i₁, j₁) ≤ weight(i₀, j₁) + weight(i₁, j₀).
//
// Our weights are two-element pairs [hi, lo]
// ordered by elementwise comparison.
// The hi entry counts the weight for lines that are longer than max,
// and the lo entry counts the weight for lines that are not.
// This forces the algorithm to first minimize the number of lines
// that are longer than max, which correspond to lines with
// single very long words. Having done that, it can move on to
// minimizing the lo score, which is more interesting.
//
// The lo score is the sum for each line of the square of the
// number of spaces remaining at the end of the line and a
// penalty of 64 given out for not ending the line in a
// punctuation character (.,:;).
// The penalty is somewhat arbitrarily chosen by trying
// different amounts and judging how nice the wrapped text looks.
// Roughly speaking, using 64 means that we are willing to
// end a line with eight blank spaces in order to end at a
// punctuation character, even if the next word would fit in
// those spaces.
//
// We care about ending in punctuation characters because
// it makes the text easier to skim if not too many sentences
// or phrases begin with a single word on the previous line.
// A score is the score (also called weight) for a given line.
// add and cmp add and compare scores.
type score struct {
hi int64
lo int64
}
add := func(s, t score) score { return score{s.hi + t.hi, s.lo + t.lo} }
cmp := func(s, t score) int {
switch {
case s.hi < t.hi:
return -1
case s.hi > t.hi:
return +1
case s.lo < t.lo:
return -1
case s.lo > t.lo:
return +1
}
return 0
}
// total[j] is the total number of runes
// (including separating spaces) in words[:j].
total := make([]int, len(words)+1)
total[0] = 0
for i, s := range words {
total[1+i] = total[i] + utf8.RuneCountInString(s) + 1
}
// weight returns weight(i, j).
weight := func(i, j int) score {
// On the last line, there is zero weight for being too short.
n := total[j] - 1 - total[i]
if j == len(words) && n <= max {
return score{0, 0}
}
// Otherwise the weight is the penalty plus the square of the number of
// characters remaining on the line or by which the line goes over.
// In the latter case, that value goes in the hi part of the score.
// (See note above.)
p := wrapPenalty(words[j-1])
v := int64(max-n) * int64(max-n)
if n > max {
return score{v, p}
}
return score{0, v + p}
}
// The rest of this function is “The Basic Algorithm” from
// Hirschberg and Larmore's conference paper,
// using the same names as in the paper.
f := []score{{0, 0}}
g := func(i, j int) score { return add(f[i], weight(i, j)) }
bridge := func(a, b, c int) bool {
k := c + sort.Search(len(words)+1-c, func(k int) bool {
k += c
return cmp(g(a, k), g(b, k)) > 0
})
if k > len(words) {
return true
}
return cmp(g(c, k), g(b, k)) <= 0
}
// d is a one-ended deque implemented as a slice.
d := make([]int, 1, len(words))
d[0] = 0
bestleft := make([]int, 1, len(words))
bestleft[0] = -1
for m := 1; m < len(words); m++ {
f = append(f, g(d[0], m))
bestleft = append(bestleft, d[0])
for len(d) > 1 && cmp(g(d[1], m+1), g(d[0], m+1)) <= 0 {
d = d[1:] // “Retire”
}
for len(d) > 1 && bridge(d[len(d)-2], d[len(d)-1], m) {
d = d[:len(d)-1] // “Fire”
}
if cmp(g(m, len(words)), g(d[len(d)-1], len(words))) < 0 {
d = append(d, m) // “Hire”
// The next few lines are not in the paper but are necessary
// to handle two-word inputs correctly. It appears to be
// just a bug in the paper's pseudocode.
if len(d) == 2 && cmp(g(d[1], m+1), g(d[0], m+1)) <= 0 {
d = d[1:]
}
}
}
bestleft = append(bestleft, d[0])
// Recover least weight sequence from bestleft.
n := 1
for m := len(words); m > 0; m = bestleft[m] {
n++
}
seq = make([]int, n)
for m := len(words); m > 0; m = bestleft[m] {
n--
seq[n] = m
}
return seq
}
// wrapPenalty is the penalty for inserting a line break after word s.
func wrapPenalty(s string) int64 {
switch s[len(s)-1] {
case '.', ',', ':', ';':
return 0
}
return 64
}
@@ -0,0 +1,134 @@
// Copyright 2012 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Package format implements standard formatting of Go source.
//
// Note that formatting of Go source code changes over time, so tools relying on
// consistent formatting should execute a specific version of the gofmt binary
// instead of using this package. That way, the formatting will be stable, and
// the tools won't need to be recompiled each time gofmt changes.
//
// For example, pre-submit checks that use this package directly would behave
// differently depending on what Go version each developer uses, causing the
// check to be inherently fragile.
package format
import (
"bytes"
"fmt"
"go/ast"
"go/parser"
"go/token"
"io"
"mvdan.cc/gofumpt/internal/govendor/go/printer"
)
// Keep these in sync with cmd/gofmt/gofmt.go.
const (
tabWidth = 8
printerMode = printer.UseSpaces | printer.TabIndent | printerNormalizeNumbers
// printerNormalizeNumbers means to canonicalize number literal prefixes
// and exponents while printing. See https://golang.org/doc/go1.13#gofmt.
//
// This value is defined in mvdan.cc/gofumpt/internal/govendor/go/printer specifically for mvdan.cc/gofumpt/internal/govendor/go/format and cmd/gofmt.
printerNormalizeNumbers = 1 << 30
)
var config = printer.Config{Mode: printerMode, Tabwidth: tabWidth}
const parserMode = parser.ParseComments | parser.SkipObjectResolution
// Node formats node in canonical gofmt style and writes the result to dst.
//
// The node type must be *[ast.File], *[printer.CommentedNode], [][ast.Decl],
// [][ast.Stmt], or assignment-compatible to [ast.Expr], [ast.Decl], [ast.Spec],
// or [ast.Stmt]. Node does not modify node. Imports are not sorted for
// nodes representing partial source files (for instance, if the node is
// not an *[ast.File] or a *[printer.CommentedNode] not wrapping an *[ast.File]).
//
// The function may return early (before the entire result is written)
// and return a formatting error, for instance due to an incorrect AST.
func Node(dst io.Writer, fset *token.FileSet, node any) error {
// Determine if we have a complete source file (file != nil).
var file *ast.File
var cnode *printer.CommentedNode
switch n := node.(type) {
case *ast.File:
file = n
case *printer.CommentedNode:
if f, ok := n.Node.(*ast.File); ok {
file = f
cnode = n
}
}
// Sort imports if necessary.
if file != nil && hasUnsortedImports(file) {
// Make a copy of the AST because ast.SortImports is destructive.
// TODO(gri) Do this more efficiently.
var buf bytes.Buffer
err := config.Fprint(&buf, fset, file)
if err != nil {
return err
}
file, err = parser.ParseFile(fset, "", buf.Bytes(), parserMode)
if err != nil {
// We should never get here. If we do, provide good diagnostic.
return fmt.Errorf("format.Node internal error (%s)", err)
}
ast.SortImports(fset, file)
// Use new file with sorted imports.
node = file
if cnode != nil {
node = &printer.CommentedNode{Node: file, Comments: cnode.Comments}
}
}
return config.Fprint(dst, fset, node)
}
// Source formats src in canonical gofmt style and returns the result
// or an (I/O or syntax) error. src is expected to be a syntactically
// correct Go source file, or a list of Go declarations or statements.
//
// If src is a partial source file, the leading and trailing space of src
// is applied to the result (such that it has the same leading and trailing
// space as src), and the result is indented by the same amount as the first
// line of src containing code. Imports are not sorted for partial source files.
func Source(src []byte) ([]byte, error) {
fset := token.NewFileSet()
file, sourceAdj, indentAdj, err := parse(fset, "", src, true)
if err != nil {
return nil, err
}
if sourceAdj == nil {
// Complete source file.
// TODO(gri) consider doing this always.
ast.SortImports(fset, file)
}
return format(fset, file, sourceAdj, indentAdj, src, config)
}
func hasUnsortedImports(file *ast.File) bool {
for _, d := range file.Decls {
d, ok := d.(*ast.GenDecl)
if !ok || d.Tok != token.IMPORT {
// Not an import declaration, so we're done.
// Imports are always first.
return false
}
if d.Lparen.IsValid() {
// For now assume all grouped imports are unsorted.
// TODO(gri) Should check if they are sorted already.
return true
}
// Ungrouped imports are sorted by default.
}
return false
}
@@ -0,0 +1,177 @@
// Copyright 2015 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// TODO(gri): This file and the file src/cmd/gofmt/internal.go are
// the same (but for this comment and the package name). Do not modify
// one without the other. Determine if we can factor out functionality
// in a public API. See also #11844 for context.
package format
import (
"bytes"
"go/ast"
"go/parser"
"go/token"
"strings"
"mvdan.cc/gofumpt/internal/govendor/go/printer"
)
// parse parses src, which was read from the named file,
// as a Go source file, declaration, or statement list.
func parse(fset *token.FileSet, filename string, src []byte, fragmentOk bool) (
file *ast.File,
sourceAdj func(src []byte, indent int) []byte,
indentAdj int,
err error,
) {
// Try as whole source file.
file, err = parser.ParseFile(fset, filename, src, parserMode)
// If there's no error, return. If the error is that the source file didn't begin with a
// package line and source fragments are ok, fall through to
// try as a source fragment. Stop and return on any other error.
if err == nil || !fragmentOk || !strings.Contains(err.Error(), "expected 'package'") {
return
}
// If this is a declaration list, make it a source file
// by inserting a package clause.
// Insert using a ';', not a newline, so that the line numbers
// in psrc match the ones in src.
psrc := append([]byte("package p;"), src...)
file, err = parser.ParseFile(fset, filename, psrc, parserMode)
if err == nil {
sourceAdj = func(src []byte, indent int) []byte {
// Remove the package clause.
// Gofmt has turned the ';' into a '\n'.
src = src[indent+len("package p\n"):]
return bytes.TrimSpace(src)
}
return
}
// If the error is that the source file didn't begin with a
// declaration, fall through to try as a statement list.
// Stop and return on any other error.
if !strings.Contains(err.Error(), "expected declaration") {
return
}
// If this is a statement list, make it a source file
// by inserting a package clause and turning the list
// into a function body. This handles expressions too.
// Insert using a ';', not a newline, so that the line numbers
// in fsrc match the ones in src. Add an extra '\n' before the '}'
// to make sure comments are flushed before the '}'.
fsrc := append(append([]byte("package p; func _() {"), src...), '\n', '\n', '}')
file, err = parser.ParseFile(fset, filename, fsrc, parserMode)
if err == nil {
sourceAdj = func(src []byte, indent int) []byte {
// Cap adjusted indent to zero.
if indent < 0 {
indent = 0
}
// Remove the wrapping.
// Gofmt has turned the "; " into a "\n\n".
// There will be two non-blank lines with indent, hence 2*indent.
src = src[2*indent+len("package p\n\nfunc _() {"):]
// Remove only the "}\n" suffix: remaining whitespaces will be trimmed anyway
src = src[:len(src)-len("}\n")]
return bytes.TrimSpace(src)
}
// Gofmt has also indented the function body one level.
// Adjust that with indentAdj.
indentAdj = -1
}
// Succeeded, or out of options.
return
}
// format formats the given package file originally obtained from src
// and adjusts the result based on the original source via sourceAdj
// and indentAdj.
func format(
fset *token.FileSet,
file *ast.File,
sourceAdj func(src []byte, indent int) []byte,
indentAdj int,
src []byte,
cfg printer.Config,
) ([]byte, error) {
if sourceAdj == nil {
// Complete source file.
var buf bytes.Buffer
err := cfg.Fprint(&buf, fset, file)
if err != nil {
return nil, err
}
return buf.Bytes(), nil
}
// Partial source file.
// Determine and prepend leading space.
i, j := 0, 0
for j < len(src) && isSpace(src[j]) {
if src[j] == '\n' {
i = j + 1 // byte offset of last line in leading space
}
j++
}
var res []byte
res = append(res, src[:i]...)
// Determine and prepend indentation of first code line.
// Spaces are ignored unless there are no tabs,
// in which case spaces count as one tab.
indent := 0
hasSpace := false
for _, b := range src[i:j] {
switch b {
case ' ':
hasSpace = true
case '\t':
indent++
}
}
if indent == 0 && hasSpace {
indent = 1
}
for i := 0; i < indent; i++ {
res = append(res, '\t')
}
// Format the source.
// Write it without any leading and trailing space.
cfg.Indent = indent + indentAdj
var buf bytes.Buffer
err := cfg.Fprint(&buf, fset, file)
if err != nil {
return nil, err
}
out := sourceAdj(buf.Bytes(), cfg.Indent)
// If the adjusted output is empty, the source
// was empty but (possibly) for white space.
// The result is the incoming source.
if len(out) == 0 {
return src, nil
}
// Otherwise, append output to leading space.
res = append(res, out...)
// Determine and append trailing space.
i = len(src)
for i > 0 && isSpace(src[i-1]) {
i--
}
return append(res, src[i:]...), nil
}
// isSpace reports whether the byte is a space character.
// isSpace defines a space as being among the following bytes: ' ', '\t', '\n' and '\r'.
func isSpace(b byte) bool {
return b == ' ' || b == '\t' || b == '\n' || b == '\r'
}
@@ -0,0 +1,156 @@
// Copyright 2022 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package printer
import (
"go/ast"
"strings"
"mvdan.cc/gofumpt/internal/govendor/go/doc/comment"
)
// formatDocComment reformats the doc comment list,
// returning the canonical formatting.
func formatDocComment(list []*ast.Comment) []*ast.Comment {
// Extract comment text (removing comment markers).
var kind, text string
var directives []*ast.Comment
if len(list) == 1 && strings.HasPrefix(list[0].Text, "/*") {
kind = "/*"
text = list[0].Text
if !strings.Contains(text, "\n") || allStars(text) {
// Single-line /* .. */ comment in doc comment position,
// or multiline old-style comment like
// /*
// * Comment
// * text here.
// */
// Should not happen, since it will not work well as a
// doc comment, but if it does, just ignore:
// reformatting it will only make the situation worse.
return list
}
text = text[2 : len(text)-2] // cut /* and */
} else if strings.HasPrefix(list[0].Text, "//") {
kind = "//"
var b strings.Builder
for _, c := range list {
after, found := strings.CutPrefix(c.Text, "//")
if !found {
return list
}
// Accumulate //go:build etc lines separately.
if isDirective(after) {
directives = append(directives, c)
continue
}
b.WriteString(strings.TrimPrefix(after, " "))
b.WriteString("\n")
}
text = b.String()
} else {
// Not sure what this is, so leave alone.
return list
}
if text == "" {
return list
}
// Parse comment and reformat as text.
var p comment.Parser
d := p.Parse(text)
var pr comment.Printer
text = string(pr.Comment(d))
// For /* */ comment, return one big comment with text inside.
slash := list[0].Slash
if kind == "/*" {
c := &ast.Comment{
Slash: slash,
Text: "/*\n" + text + "*/",
}
return []*ast.Comment{c}
}
// For // comment, return sequence of // lines.
var out []*ast.Comment
for text != "" {
var line string
line, text, _ = strings.Cut(text, "\n")
if line == "" {
line = "//"
} else if strings.HasPrefix(line, "\t") {
line = "//" + line
} else {
line = "// " + line
}
out = append(out, &ast.Comment{
Slash: slash,
Text: line,
})
}
if len(directives) > 0 {
out = append(out, &ast.Comment{
Slash: slash,
Text: "//",
})
for _, c := range directives {
out = append(out, &ast.Comment{
Slash: slash,
Text: c.Text,
})
}
}
return out
}
// isDirective reports whether c is a comment directive.
// See go.dev/issue/37974.
// This code is also in go/ast.
func isDirective(c string) bool {
// "//line " is a line directive.
// "//extern " is for gccgo.
// "//export " is for cgo.
// (The // has been removed.)
if strings.HasPrefix(c, "line ") || strings.HasPrefix(c, "extern ") || strings.HasPrefix(c, "export ") {
return true
}
// "//[a-z0-9]+:[a-z0-9]"
// (The // has been removed.)
colon := strings.Index(c, ":")
if colon <= 0 || colon+1 >= len(c) {
return false
}
for i := 0; i <= colon+1; i++ {
if i == colon {
continue
}
b := c[i]
if !('a' <= b && b <= 'z' || '0' <= b && b <= '9') {
return false
}
}
return true
}
// allStars reports whether text is the interior of an
// old-style /* */ comment with a star at the start of each line.
func allStars(text string) bool {
for i := 0; i < len(text); i++ {
if text[i] == '\n' {
j := i + 1
for j < len(text) && (text[j] == ' ' || text[j] == '\t') {
j++
}
if j < len(text) && text[j] != '*' {
return false
}
}
}
return true
}
@@ -0,0 +1,170 @@
// Copyright 2020 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package printer
import (
"go/build/constraint"
"slices"
"text/tabwriter"
)
func (p *printer) fixGoBuildLines() {
if len(p.goBuild)+len(p.plusBuild) == 0 {
return
}
// Find latest possible placement of //go:build and // +build comments.
// That's just after the last blank line before we find a non-comment.
// (We'll add another blank line after our comment block.)
// When we start dropping // +build comments, we can skip over /* */ comments too.
// Note that we are processing tabwriter input, so every comment
// begins and ends with a tabwriter.Escape byte.
// And some newlines have turned into \f bytes.
insert := 0
for pos := 0; ; {
// Skip leading space at beginning of line.
blank := true
for pos < len(p.output) && (p.output[pos] == ' ' || p.output[pos] == '\t') {
pos++
}
// Skip over // comment if any.
if pos+3 < len(p.output) && p.output[pos] == tabwriter.Escape && p.output[pos+1] == '/' && p.output[pos+2] == '/' {
blank = false
for pos < len(p.output) && !isNL(p.output[pos]) {
pos++
}
}
// Skip over \n at end of line.
if pos >= len(p.output) || !isNL(p.output[pos]) {
break
}
pos++
if blank {
insert = pos
}
}
// If there is a //go:build comment before the place we identified,
// use that point instead. (Earlier in the file is always fine.)
if len(p.goBuild) > 0 && p.goBuild[0] < insert {
insert = p.goBuild[0]
} else if len(p.plusBuild) > 0 && p.plusBuild[0] < insert {
insert = p.plusBuild[0]
}
var x constraint.Expr
switch len(p.goBuild) {
case 0:
// Synthesize //go:build expression from // +build lines.
for _, pos := range p.plusBuild {
y, err := constraint.Parse(p.commentTextAt(pos))
if err != nil {
x = nil
break
}
if x == nil {
x = y
} else {
x = &constraint.AndExpr{X: x, Y: y}
}
}
case 1:
// Parse //go:build expression.
x, _ = constraint.Parse(p.commentTextAt(p.goBuild[0]))
}
var block []byte
if x == nil {
// Don't have a valid //go:build expression to treat as truth.
// Bring all the lines together but leave them alone.
// Note that these are already tabwriter-escaped.
for _, pos := range p.goBuild {
block = append(block, p.lineAt(pos)...)
}
for _, pos := range p.plusBuild {
block = append(block, p.lineAt(pos)...)
}
} else {
block = append(block, tabwriter.Escape)
block = append(block, "//go:build "...)
block = append(block, x.String()...)
block = append(block, tabwriter.Escape, '\n')
if len(p.plusBuild) > 0 {
lines, err := constraint.PlusBuildLines(x)
if err != nil {
lines = []string{"// +build error: " + err.Error()}
}
for _, line := range lines {
block = append(block, tabwriter.Escape)
block = append(block, line...)
block = append(block, tabwriter.Escape, '\n')
}
}
}
block = append(block, '\n')
// Build sorted list of lines to delete from remainder of output.
toDelete := append(p.goBuild, p.plusBuild...)
slices.Sort(toDelete)
// Collect output after insertion point, with lines deleted, into after.
var after []byte
start := insert
for _, end := range toDelete {
if end < start {
continue
}
after = appendLines(after, p.output[start:end])
start = end + len(p.lineAt(end))
}
after = appendLines(after, p.output[start:])
if n := len(after); n >= 2 && isNL(after[n-1]) && isNL(after[n-2]) {
after = after[:n-1]
}
p.output = p.output[:insert]
p.output = append(p.output, block...)
p.output = append(p.output, after...)
}
// appendLines is like append(x, y...)
// but it avoids creating doubled blank lines,
// which would not be gofmt-standard output.
// It assumes that only whole blocks of lines are being appended,
// not line fragments.
func appendLines(x, y []byte) []byte {
if len(y) > 0 && isNL(y[0]) && // y starts in blank line
(len(x) == 0 || len(x) >= 2 && isNL(x[len(x)-1]) && isNL(x[len(x)-2])) { // x is empty or ends in blank line
y = y[1:] // delete y's leading blank line
}
return append(x, y...)
}
func (p *printer) lineAt(start int) []byte {
pos := start
for pos < len(p.output) && !isNL(p.output[pos]) {
pos++
}
if pos < len(p.output) {
pos++
}
return p.output[start:pos]
}
func (p *printer) commentTextAt(start int) string {
if start < len(p.output) && p.output[start] == tabwriter.Escape {
start++
}
pos := start
for pos < len(p.output) && p.output[pos] != tabwriter.Escape && !isNL(p.output[pos]) {
pos++
}
return string(p.output[start:pos])
}
func isNL(b byte) bool {
return b == '\n' || b == '\f'
}
File diff suppressed because it is too large Load Diff

Some files were not shown because too many files have changed in this diff Show More