aboutsummaryrefslogtreecommitdiff
path: root/docs
diff options
context:
space:
mode:
Diffstat (limited to 'docs')
-rw-r--r--docs/dev/README.md88
-rw-r--r--docs/dev/architecture.md79
-rw-r--r--docs/dev/debugging.md2
3 files changed, 83 insertions, 86 deletions
diff --git a/docs/dev/README.md b/docs/dev/README.md
index 2f6215d6b..d30727786 100644
--- a/docs/dev/README.md
+++ b/docs/dev/README.md
@@ -26,15 +26,6 @@ Discussion happens in this Zulip stream:
26 26
27https://rust-lang.zulipchat.com/#narrow/stream/185405-t-compiler.2Fwg-rls-2.2E0 27https://rust-lang.zulipchat.com/#narrow/stream/185405-t-compiler.2Fwg-rls-2.2E0
28 28
29# Work List
30
31We have this "work list" paper document:
32
33https://paper.dropbox.com/doc/RLS-2.0-work-list--AZ3BgHKKCtqszbsi3gi6sjchAQ-42vbnxzuKq2lKwW0mkn8Y
34
35It shows what everyone is working on right now. If you want to (this is not
36mandatory), add yourself to the list!
37
38# Issue Labels 29# Issue Labels
39 30
40* [good-first-issue](https://github.com/rust-analyzer/rust-analyzer/labels/good%20first%20issue) 31* [good-first-issue](https://github.com/rust-analyzer/rust-analyzer/labels/good%20first%20issue)
@@ -50,10 +41,12 @@ mandatory), add yourself to the list!
50 41
51# CI 42# CI
52 43
53We use Travis for CI. Most of the things, including formatting, are checked by 44We use GitHub Actions for CI. Most of the things, including formatting, are checked by
54`cargo test` so, if `cargo test` passes locally, that's a good sign that CI will 45`cargo test` so, if `cargo test` passes locally, that's a good sign that CI will
55be green as well. We use bors-ng to enforce the [not rocket 46be green as well. The only exception is that long-running by default a skipped locally.
56science](https://graydon2.dreamwidth.org/1597.html) rule. 47Use `env RUN_SLOW_TESTS=1 cargo test` to run the full suite.
48
49We use bors-ng to enforce the [not rocket science](https://graydon2.dreamwidth.org/1597.html) rule.
57 50
58You can run `cargo xtask install-pre-commit-hook` to install git-hook to run rustfmt on commit. 51You can run `cargo xtask install-pre-commit-hook` to install git-hook to run rustfmt on commit.
59 52
@@ -81,42 +74,41 @@ relevant test and execute it (VS Code includes an action for running a single
81test). 74test).
82 75
83However, launching a VS Code instance with locally build language server is 76However, launching a VS Code instance with locally build language server is
84possible. There's even a VS Code task for this, so just <kbd>F5</kbd> should 77possible. There's "Run Extension (Dev Server)" launch configuration for this.
85work (thanks, [@andrew-w-ross](https://github.com/andrew-w-ross)!). 78
86 79In general, I use one of the following workflows for fixing bugs and
87I often just install development version with `cargo xtask install --server --jemalloc` and 80implementing features.
88restart the host VS Code. 81
89 82If the problem concerns only internal parts of rust-analyzer (ie, I don't need
90See [./debugging.md](./debugging.md) for how to attach to rust-analyzer with 83to touch `ra_lsp_server` crate or typescript code), there is a unit-test for it.
91debugger, and don't forget that rust-analyzer has useful `pd` snippet and `dbg` 84So, I use **Rust Analyzer: Run** action in VS Code to run this single test, and
92postfix completion for printf debugging :-) 85then just do printf-driven development/debugging. As a sanity check after I'm
93 86done, I use `cargo xtask install --server` and **Reload Window** action in VS
94# Working With VS Code Extension 87Code to sanity check that the thing works as I expect.
95 88
96To work on the VS Code extension, launch code inside `editors/code` and use `F5` 89If the problem concerns only the VS Code extension, I use **Run Extension**
97to launch/debug. To automatically apply formatter and linter suggestions, use 90launch configuration from `launch.json`. Notably, this uses the usual
98`npm run fix`. 91`ra_lsp_server` binary from `PATH`. After I am done with the fix, I use `cargo
99 92xtask install --client-code` to try the new extension for real.
100Tests are located inside `src/test` and are named `*.test.ts`. They use the 93
101[Mocha](https://mochajs.org) test framework and the builtin Node 94If I need to fix something in the `ra_lsp_server` crate, I feel sad because it's
102[assert](https://nodejs.org/api/assert.html) module. Unlike normal Node tests 95on the boundary between the two processes, and working there is slow. I usually
103they must be hosted inside a VS Code instance. This can be done in one of two 96just `cargo xtask install --server` and poke changes from my live environment.
104ways: 97Note that this uses `--release`, which is usually faster overall, because
105 98loading stdlib into debug version of rust-analyzer takes a lot of time. To speed
1061. When `F5` debugging in VS Code select the `Extension Tests` configuration 99things up, sometimes I open a temporary hello-world project which has
107 from the drop-down at the top of the Debug View. This will launch a temporary 100`"rust-analyzer.withSysroot": false` in `.code/settings.json`. This flag causes
108 instance of VS Code. The test results will appear in the "Debug Console" tab 101rust-analyzer to skip loading the sysroot, which greatly reduces the amount of
109 of the primary VS Code instance. 102things rust-analyzer needs to do, and makes printf's more useful. Note that you
110 103should only use `eprint!` family of macros for debugging: stdout is used for LSP
1112. Run `npm test` from the command line. Although this is initiated from the 104communication, and `print!` would break it.
112 command line it is not headless; it will also launch a temporary instance of 105
113 VS Code. 106If I need to fix something simultaneously in the server and in the client, I
114 107feel even more sad. I don't have a specific workflow for this case.
115Due to the requirements of running the tests inside VS Code they are **not run 108
116on CI**. When making changes to the extension please ensure the tests are not 109Additionally, I use `cargo run --release -p ra_cli -- analysis-stats
117broken locally before opening a Pull Request. 110path/to/some/rust/crate` to run a batch analysis. This is primaraly useful for
118 111performance optimiations, or for bug minimization.
119To install **only** the VS Code extension, use `cargo xtask install --client-code`.
120 112
121# Logging 113# Logging
122 114
diff --git a/docs/dev/architecture.md b/docs/dev/architecture.md
index 629645757..9675ed0b6 100644
--- a/docs/dev/architecture.md
+++ b/docs/dev/architecture.md
@@ -12,6 +12,9 @@ analyzer:
12 12
13https://www.youtube.com/playlist?list=PL85XCvVPmGQho7MZkdW-wtPtuJcFpzycE 13https://www.youtube.com/playlist?list=PL85XCvVPmGQho7MZkdW-wtPtuJcFpzycE
14 14
15Note that the guide and videos are pretty dated, this document should be in
16generally fresher.
17
15## The Big Picture 18## The Big Picture
16 19
17![](https://user-images.githubusercontent.com/1711539/50114578-e8a34280-0255-11e9-902c-7cfc70747966.png) 20![](https://user-images.githubusercontent.com/1711539/50114578-e8a34280-0255-11e9-902c-7cfc70747966.png)
@@ -20,13 +23,12 @@ On the highest level, rust-analyzer is a thing which accepts input source code
20from the client and produces a structured semantic model of the code. 23from the client and produces a structured semantic model of the code.
21 24
22More specifically, input data consists of a set of test files (`(PathBuf, 25More specifically, input data consists of a set of test files (`(PathBuf,
23String)` pairs) and information about project structure, captured in the so called 26String)` pairs) and information about project structure, captured in the so
24`CrateGraph`. The crate graph specifies which files are crate roots, which cfg 27called `CrateGraph`. The crate graph specifies which files are crate roots,
25flags are specified for each crate (TODO: actually implement this) and what 28which cfg flags are specified for each crate and what dependencies exist between
26dependencies exist between the crates. The analyzer keeps all this input data in 29the crates. The analyzer keeps all this input data in memory and never does any
27memory and never does any IO. Because the input data is source code, which 30IO. Because the input data are source code, which typically measures in tens of
28typically measures in tens of megabytes at most, keeping all input data in 31megabytes at most, keeping everything in memory is OK.
29memory is OK.
30 32
31A "structured semantic model" is basically an object-oriented representation of 33A "structured semantic model" is basically an object-oriented representation of
32modules, functions and types which appear in the source code. This representation 34modules, functions and types which appear in the source code. This representation
@@ -43,37 +45,39 @@ can be quickly updated for small modifications.
43## Code generation 45## Code generation
44 46
45Some of the components of this repository are generated through automatic 47Some of the components of this repository are generated through automatic
46processes. These are outlined below: 48processes. `cargo xtask codegen` runs all generation tasks. Generated code is
49commited to the git repository.
50
51In particular, `cargo xtask codegen` generates:
52
531. [`syntax_kind/generated`](https://github.com/rust-analyzer/rust-analyzer/blob/a0be39296d2925972cacd9fbf8b5fb258fad6947/crates/ra_parser/src/syntax_kind/generated.rs)
54 -- the set of terminals and non-terminals of rust grammar.
47 55
48- `cargo xtask codegen`: The kinds of tokens that are reused in several places, so a generator 562. [`ast/generated`](https://github.com/rust-analyzer/rust-analyzer/blob/a0be39296d2925972cacd9fbf8b5fb258fad6947/crates/ra_syntax/src/ast/generated.rs)
49 is used. We use `quote!` macro to generate the files listed below, based on 57 -- AST data structure.
50 the grammar described in [grammar.ron]:
51 - [ast/generated.rs][ast generated]
52 - [syntax_kind/generated.rs][syntax_kind generated]
53 58
54[grammar.ron]: ../../crates/ra_syntax/src/grammar.ron 59.3 [`doc_tests/generated`](https://github.com/rust-analyzer/rust-analyzer/blob/a0be39296d2925972cacd9fbf8b5fb258fad6947/crates/ra_assists/src/doc_tests/generated.rs),
55[ast generated]: ../../crates/ra_syntax/src/ast/generated.rs 60 [`test_data/parser/inline`](https://github.com/rust-analyzer/rust-analyzer/tree/a0be39296d2925972cacd9fbf8b5fb258fad6947/crates/ra_syntax/test_data/parser/inline)
56[syntax_kind generated]: ../../crates/ra_parser/src/syntax_kind/generated.rs 61 -- tests for assists and the parser.
62
63The source for 1 and 2 is in [`ast_src.rs`](https://github.com/rust-analyzer/rust-analyzer/blob/a0be39296d2925972cacd9fbf8b5fb258fad6947/xtask/src/ast_src.rs).
57 64
58## Code Walk-Through 65## Code Walk-Through
59 66
60### `crates/ra_syntax`, `crates/ra_parser` 67### `crates/ra_syntax`, `crates/ra_parser`
61 68
62Rust syntax tree structure and parser. See 69Rust syntax tree structure and parser. See
63[RFC](https://github.com/rust-lang/rfcs/pull/2256) for some design notes. 70[RFC](https://github.com/rust-lang/rfcs/pull/2256) and [./syntax.md](./syntax.md) for some design notes.
64 71
65- [rowan](https://github.com/rust-analyzer/rowan) library is used for constructing syntax trees. 72- [rowan](https://github.com/rust-analyzer/rowan) library is used for constructing syntax trees.
66- `grammar` module is the actual parser. It is a hand-written recursive descent parser, which 73- `grammar` module is the actual parser. It is a hand-written recursive descent parser, which
67 produces a sequence of events like "start node X", "finish node Y". It works similarly to [kotlin's parser](https://github.com/JetBrains/kotlin/blob/4d951de616b20feca92f3e9cc9679b2de9e65195/compiler/frontend/src/org/jetbrains/kotlin/parsing/KotlinParsing.java), 74 produces a sequence of events like "start node X", "finish node Y". It works similarly to [kotlin's parser](https://github.com/JetBrains/kotlin/blob/4d951de616b20feca92f3e9cc9679b2de9e65195/compiler/frontend/src/org/jetbrains/kotlin/parsing/KotlinParsing.java),
68 which is a good source of inspiration for dealing with syntax errors and incomplete input. Original [libsyntax parser](https://github.com/rust-lang/rust/blob/6b99adeb11313197f409b4f7c4083c2ceca8a4fe/src/libsyntax/parse/parser.rs) 75 which is a good source of inspiration for dealing with syntax errors and incomplete input. Original [libsyntax parser](https://github.com/rust-lang/rust/blob/6b99adeb11313197f409b4f7c4083c2ceca8a4fe/src/libsyntax/parse/parser.rs)
69 is what we use for the definition of the Rust language. 76 is what we use for the definition of the Rust language.
70- `parser_api/parser_impl` bridges the tree-agnostic parser from `grammar` with `rowan` trees. 77- `TreeSink` and `TokenSource` traits bridge the tree-agnostic parser from `grammar` with `rowan` trees.
71 This is the thing that turns a flat list of events into a tree (see `EventProcessor`)
72- `ast` provides a type safe API on top of the raw `rowan` tree. 78- `ast` provides a type safe API on top of the raw `rowan` tree.
73- `grammar.ron` RON description of the grammar, which is used to 79- `ast_src` description of the grammar, which is used to generate `syntax_kinds`
74 generate `syntax_kinds` and `ast` modules, using `cargo xtask codegen` command. 80 and `ast` modules, using `cargo xtask codegen` command.
75- `algo`: generic tree algorithms, including `walk` for O(1) stack
76 space tree traversal (this is cool).
77 81
78Tests for ra_syntax are mostly data-driven: `test_data/parser` contains subdirectories with a bunch of `.rs` 82Tests for ra_syntax are mostly data-driven: `test_data/parser` contains subdirectories with a bunch of `.rs`
79(test vectors) and `.txt` files with corresponding syntax trees. During testing, we check 83(test vectors) and `.txt` files with corresponding syntax trees. During testing, we check
@@ -81,6 +85,10 @@ Tests for ra_syntax are mostly data-driven: `test_data/parser` contains subdirec
81tests). Additionally, running `cargo xtask codegen` will walk the grammar module and collect 85tests). Additionally, running `cargo xtask codegen` will walk the grammar module and collect
82all `// test test_name` comments into files inside `test_data/parser/inline` directory. 86all `// test test_name` comments into files inside `test_data/parser/inline` directory.
83 87
88Note
89[`api_walkthrough`](https://github.com/rust-analyzer/rust-analyzer/blob/2fb6af89eb794f775de60b82afe56b6f986c2a40/crates/ra_syntax/src/lib.rs#L190-L348)
90in particular: it shows off various methods of working with syntax tree.
91
84See [#93](https://github.com/rust-analyzer/rust-analyzer/pull/93) for an example PR which 92See [#93](https://github.com/rust-analyzer/rust-analyzer/pull/93) for an example PR which
85fixes a bug in the grammar. 93fixes a bug in the grammar.
86 94
@@ -94,18 +102,22 @@ defines most of the "input" queries: facts supplied by the client of the
94analyzer. Reading the docs of the `ra_db::input` module should be useful: 102analyzer. Reading the docs of the `ra_db::input` module should be useful:
95everything else is strictly derived from those inputs. 103everything else is strictly derived from those inputs.
96 104
97### `crates/ra_hir` 105### `crates/ra_hir*` crates
98 106
99HIR provides high-level "object oriented" access to Rust code. 107HIR provides high-level "object oriented" access to Rust code.
100 108
101The principal difference between HIR and syntax trees is that HIR is bound to a 109The principal difference between HIR and syntax trees is that HIR is bound to a
102particular crate instance. That is, it has cfg flags and features applied (in 110particular crate instance. That is, it has cfg flags and features applied. So,
103theory, in practice this is to be implemented). So, the relation between 111the relation between syntax and HIR is many-to-one. The `source_binder` module
104syntax and HIR is many-to-one. The `source_binder` module is responsible for 112is responsible for guessing a HIR for a particular source position.
105guessing a HIR for a particular source position.
106 113
107Underneath, HIR works on top of salsa, using a `HirDatabase` trait. 114Underneath, HIR works on top of salsa, using a `HirDatabase` trait.
108 115
116`ra_hir_xxx` crates have a strong ECS flavor, in that they work with raw ids and
117directly query the databse.
118
119The top-level `ra_hir` façade crate wraps ids into a more OO-flavored API.
120
109### `crates/ra_ide` 121### `crates/ra_ide`
110 122
111A stateful library for analyzing many Rust files as they change. `AnalysisHost` 123A stateful library for analyzing many Rust files as they change. `AnalysisHost`
@@ -135,18 +147,9 @@ different from data on disk. This is more or less the single really
135platform-dependent component, so it lives in a separate repository and has an 147platform-dependent component, so it lives in a separate repository and has an
136extensive cross-platform CI testing. 148extensive cross-platform CI testing.
137 149
138### `crates/gen_lsp_server`
139
140A language server scaffold, exposing a synchronous crossbeam-channel based API.
141This crate handles protocol handshaking and parsing messages, while you
142control the message dispatch loop yourself.
143
144Run with `RUST_LOG=sync_lsp_server=debug` to see all the messages.
145
146### `crates/ra_cli` 150### `crates/ra_cli`
147 151
148A CLI interface to rust-analyzer. 152A CLI interface to rust-analyzer, mainly for testing.
149
150 153
151## Testing Infrastructure 154## Testing Infrastructure
152 155
diff --git a/docs/dev/debugging.md b/docs/dev/debugging.md
index f868e6998..1ccf4dca2 100644
--- a/docs/dev/debugging.md
+++ b/docs/dev/debugging.md
@@ -1,5 +1,7 @@
1# Debugging vs Code plugin and the Language Server 1# Debugging vs Code plugin and the Language Server
2 2
3**NOTE:** the information here is mostly obsolete
4
3Install [LLDB](https://lldb.llvm.org/) and the [LLDB Extension](https://marketplace.visualstudio.com/items?itemName=vadimcn.vscode-lldb). 5Install [LLDB](https://lldb.llvm.org/) and the [LLDB Extension](https://marketplace.visualstudio.com/items?itemName=vadimcn.vscode-lldb).
4 6
5Checkout rust rust-analyzer and open it in vscode. 7Checkout rust rust-analyzer and open it in vscode.