diff options
Diffstat (limited to 'docs')
-rw-r--r-- | docs/dev/architecture.md | 46 |
1 files changed, 23 insertions, 23 deletions
diff --git a/docs/dev/architecture.md b/docs/dev/architecture.md index e97e082fc..2ba8b7967 100644 --- a/docs/dev/architecture.md +++ b/docs/dev/architecture.md | |||
@@ -9,9 +9,9 @@ Yet another resource is this playlist with videos about various parts of the ana | |||
9 | 9 | ||
10 | https://www.youtube.com/playlist?list=PL85XCvVPmGQho7MZkdW-wtPtuJcFpzycE | 10 | https://www.youtube.com/playlist?list=PL85XCvVPmGQho7MZkdW-wtPtuJcFpzycE |
11 | 11 | ||
12 | Note that the guide and videos are pretty dated, this document should be in generally fresher. | 12 | Note that the guide and videos are pretty dated, this document should be, in general, fresher. |
13 | 13 | ||
14 | See also this implementation-oriented blog posts: | 14 | See also these implementation-related blog posts: |
15 | 15 | ||
16 | * https://rust-analyzer.github.io/blog/2019/11/13/find-usages.html | 16 | * https://rust-analyzer.github.io/blog/2019/11/13/find-usages.html |
17 | * https://rust-analyzer.github.io/blog/2020/07/20/three-architectures-for-responsive-ide.html | 17 | * https://rust-analyzer.github.io/blog/2020/07/20/three-architectures-for-responsive-ide.html |
@@ -27,9 +27,9 @@ On the highest level, rust-analyzer is a thing which accepts input source code f | |||
27 | 27 | ||
28 | More specifically, input data consists of a set of test files (`(PathBuf, String)` pairs) and information about project structure, captured in the so called `CrateGraph`. | 28 | More specifically, input data consists of a set of test files (`(PathBuf, String)` pairs) and information about project structure, captured in the so called `CrateGraph`. |
29 | The crate graph specifies which files are crate roots, which cfg flags are specified for each crate and what dependencies exist between the crates. | 29 | The crate graph specifies which files are crate roots, which cfg flags are specified for each crate and what dependencies exist between the crates. |
30 | This the input (ground) state. | 30 | This is the input (ground) state. |
31 | The analyzer keeps all this input data in memory and never does any IO. | 31 | The analyzer keeps all this input data in memory and never does any IO. |
32 | Because the input data are source code, which typically measures in tens of megabytes at most, keeping everything in memory is OK. | 32 | Because the input data is source code, which typically measures in tens of megabytes at most, keeping everything in memory is OK. |
33 | 33 | ||
34 | A "structured semantic model" is basically an object-oriented representation of modules, functions and types which appear in the source code. | 34 | A "structured semantic model" is basically an object-oriented representation of modules, functions and types which appear in the source code. |
35 | This representation is fully "resolved": all expressions have types, all references are bound to declarations, etc. | 35 | This representation is fully "resolved": all expressions have types, all references are bound to declarations, etc. |
@@ -42,7 +42,7 @@ The underlying engine makes sure that model is computed lazily (on-demand) and c | |||
42 | 42 | ||
43 | ## Code Map | 43 | ## Code Map |
44 | 44 | ||
45 | This section talks briefly about various important directories an data structures. | 45 | This section talks briefly about various important directories and data structures. |
46 | Pay attention to the **Architecture Invariant** sections. | 46 | Pay attention to the **Architecture Invariant** sections. |
47 | They often talk about things which are deliberately absent in the source code. | 47 | They often talk about things which are deliberately absent in the source code. |
48 | 48 | ||
@@ -62,7 +62,7 @@ VS Code plugin. | |||
62 | ### `libs/` | 62 | ### `libs/` |
63 | 63 | ||
64 | rust-analyzer independent libraries which we publish to crates.io. | 64 | rust-analyzer independent libraries which we publish to crates.io. |
65 | It not heavily utilized at the moment. | 65 | It's not heavily utilized at the moment. |
66 | 66 | ||
67 | ### `crates/parser` | 67 | ### `crates/parser` |
68 | 68 | ||
@@ -110,8 +110,8 @@ in particular: it shows off various methods of working with syntax tree. | |||
110 | 110 | ||
111 | See [#93](https://github.com/rust-analyzer/rust-analyzer/pull/93) for an example PR which fixes a bug in the grammar. | 111 | See [#93](https://github.com/rust-analyzer/rust-analyzer/pull/93) for an example PR which fixes a bug in the grammar. |
112 | 112 | ||
113 | **Architecture Invariant:** `syntax` crate is completely independent from the rest of rust-analyzer, it knows nothing about salsa or LSP. | 113 | **Architecture Invariant:** `syntax` crate is completely independent from the rest of rust-analyzer. It knows nothing about salsa or LSP. |
114 | This is important because it is possible to useful tooling using only syntax tree. | 114 | This is important because it is possible to make useful tooling using only the syntax tree. |
115 | Without semantic information, you don't need to be able to _build_ code, which makes the tooling more robust. | 115 | Without semantic information, you don't need to be able to _build_ code, which makes the tooling more robust. |
116 | See also https://web.stanford.edu/~mlfbrown/paper.pdf. | 116 | See also https://web.stanford.edu/~mlfbrown/paper.pdf. |
117 | You can view the `syntax` crate as an entry point to rust-analyzer. | 117 | You can view the `syntax` crate as an entry point to rust-analyzer. |
@@ -122,7 +122,7 @@ The tree is fully determined by the contents of its syntax nodes, it doesn't nee | |||
122 | Using the tree as a store for semantic info is convenient in traditional compilers, but doesn't work nicely in the IDE. | 122 | Using the tree as a store for semantic info is convenient in traditional compilers, but doesn't work nicely in the IDE. |
123 | Specifically, assists and refactors require transforming syntax trees, and that becomes awkward if you need to do something with the semantic info. | 123 | Specifically, assists and refactors require transforming syntax trees, and that becomes awkward if you need to do something with the semantic info. |
124 | 124 | ||
125 | **Architecture Invariant:** syntax tree is build for a single file. | 125 | **Architecture Invariant:** syntax tree is built for a single file. |
126 | This is to enable parallel parsing of all files. | 126 | This is to enable parallel parsing of all files. |
127 | 127 | ||
128 | **Architecture Invariant:** Syntax trees are by design incomplete and do not enforce well-formedness. | 128 | **Architecture Invariant:** Syntax trees are by design incomplete and do not enforce well-formedness. |
@@ -131,7 +131,7 @@ If an AST method returns an `Option`, it *can* be `None` at runtime, even if thi | |||
131 | ### `crates/base_db` | 131 | ### `crates/base_db` |
132 | 132 | ||
133 | We use the [salsa](https://github.com/salsa-rs/salsa) crate for incremental and on-demand computation. | 133 | We use the [salsa](https://github.com/salsa-rs/salsa) crate for incremental and on-demand computation. |
134 | Roughly, you can think of salsa as a key-value store, but it also can compute derived values using specified functions. The `base_db` crate provides basic infrastructure for interacting with salsa. | 134 | Roughly, you can think of salsa as a key-value store, but it can also compute derived values using specified functions. The `base_db` crate provides basic infrastructure for interacting with salsa. |
135 | Crucially, it defines most of the "input" queries: facts supplied by the client of the analyzer. | 135 | Crucially, it defines most of the "input" queries: facts supplied by the client of the analyzer. |
136 | Reading the docs of the `base_db::input` module should be useful: everything else is strictly derived from those inputs. | 136 | Reading the docs of the `base_db::input` module should be useful: everything else is strictly derived from those inputs. |
137 | 137 | ||
@@ -160,11 +160,11 @@ These crates also define various intermediate representations of the core. | |||
160 | 160 | ||
161 | `Body` stores information about expressions. | 161 | `Body` stores information about expressions. |
162 | 162 | ||
163 | **Architecture Invariant:** this crates are not, and will never be, an api boundary. | 163 | **Architecture Invariant:** these crates are not, and will never be, an api boundary. |
164 | 164 | ||
165 | **Architecture Invariant:** these creates explicitly care about being incremental. | 165 | **Architecture Invariant:** these crates explicitly care about being incremental. |
166 | The core invariant we maintain is "typing inside a function's body never invalidates global derived data". | 166 | The core invariant we maintain is "typing inside a function's body never invalidates global derived data". |
167 | Ie, if you change body of `foo`, all facts about `bar` should remain intact. | 167 | i.e., if you change the body of `foo`, all facts about `bar` should remain intact. |
168 | 168 | ||
169 | **Architecture Invariant:** hir exists only in context of particular crate instance with specific CFG flags. | 169 | **Architecture Invariant:** hir exists only in context of particular crate instance with specific CFG flags. |
170 | The same syntax may produce several instances of HIR if the crate participates in the crate graph more than once. | 170 | The same syntax may produce several instances of HIR if the crate participates in the crate graph more than once. |
@@ -188,12 +188,12 @@ We first resolve the parent _syntax_ node to the parent _hir_ element. | |||
188 | Then we ask the _hir_ parent what _syntax_ children does it have. | 188 | Then we ask the _hir_ parent what _syntax_ children does it have. |
189 | Then we look for our node in the set of children. | 189 | Then we look for our node in the set of children. |
190 | 190 | ||
191 | This is the heart of many IDE features, like goto definition, which start with figuring out a hir node at the cursor. | 191 | This is the heart of many IDE features, like goto definition, which start with figuring out the hir node at the cursor. |
192 | This is some kind of (yet unnamed) uber-IDE pattern, as it is present in Roslyn and Kotlin as well. | 192 | This is some kind of (yet unnamed) uber-IDE pattern, as it is present in Roslyn and Kotlin as well. |
193 | 193 | ||
194 | ### `crates/ide` | 194 | ### `crates/ide` |
195 | 195 | ||
196 | The `ide` crate build's on top of `hir` semantic model to provide high-level IDE features like completion or goto definition. | 196 | The `ide` crate builds on top of `hir` semantic model to provide high-level IDE features like completion or goto definition. |
197 | It is an **API Boundary**. | 197 | It is an **API Boundary**. |
198 | If you want to use IDE parts of rust-analyzer via LSP, custom flatbuffers-based protocol or just as a library in your text editor, this is the right API. | 198 | If you want to use IDE parts of rust-analyzer via LSP, custom flatbuffers-based protocol or just as a library in your text editor, this is the right API. |
199 | 199 | ||
@@ -232,7 +232,7 @@ All other requests are processed in background. | |||
232 | 232 | ||
233 | **Architecture Invariant:** the server is stateless, a-la HTTP. | 233 | **Architecture Invariant:** the server is stateless, a-la HTTP. |
234 | Sometimes state needs to be preserved between requests. | 234 | Sometimes state needs to be preserved between requests. |
235 | For example, "what is the `edit` for the fifth's completion item of the last completion edit?". | 235 | For example, "what is the `edit` for the fifth completion item of the last completion edit?". |
236 | For this, the second request should include enough info to re-create the context from scratch. | 236 | For this, the second request should include enough info to re-create the context from scratch. |
237 | This generally means including all the parameters of the original request. | 237 | This generally means including all the parameters of the original request. |
238 | 238 | ||
@@ -249,7 +249,7 @@ These crates deal with invoking `cargo` to learn about project structure and get | |||
249 | They use `crates/path` heavily instead of `std::path`. | 249 | They use `crates/path` heavily instead of `std::path`. |
250 | A single `rust-analyzer` process can serve many projects, so it is important that server's current directory does not leak. | 250 | A single `rust-analyzer` process can serve many projects, so it is important that server's current directory does not leak. |
251 | 251 | ||
252 | ### `crates/mbe`, `crated/tt`, `crates/proc_macro_api`, `crates/proc_macro_srv` | 252 | ### `crates/mbe`, `crates/tt`, `crates/proc_macro_api`, `crates/proc_macro_srv` |
253 | 253 | ||
254 | These crates implement macros as token tree -> token tree transforms. | 254 | These crates implement macros as token tree -> token tree transforms. |
255 | They are independent from the rest of the code. | 255 | They are independent from the rest of the code. |
@@ -260,11 +260,11 @@ This crate is responsible for parsing, evaluation and general definition of `cfg | |||
260 | 260 | ||
261 | ### `crates/vfs`, `crates/vfs-notify` | 261 | ### `crates/vfs`, `crates/vfs-notify` |
262 | 262 | ||
263 | These crates implement a virtual fils system. | 263 | These crates implement a virtual file system. |
264 | They provide consistent snapshots of the underlying file system and insulate messy OS paths. | 264 | They provide consistent snapshots of the underlying file system and insulate messy OS paths. |
265 | 265 | ||
266 | **Architecture Invariant:** vfs doesn't assume a single unified file system. | 266 | **Architecture Invariant:** vfs doesn't assume a single unified file system. |
267 | IE, a single rust-analyzer process can act as a remote server for two different machines, where the same `/tmp/foo.rs` path points to different files. | 267 | i.e., a single rust-analyzer process can act as a remote server for two different machines, where the same `/tmp/foo.rs` path points to different files. |
268 | For this reason, all path APIs generally take some existing path as a "file system witness". | 268 | For this reason, all path APIs generally take some existing path as a "file system witness". |
269 | 269 | ||
270 | ### `crates/stdx` | 270 | ### `crates/stdx` |
@@ -319,7 +319,7 @@ That is, rust-analyzer requires unwinding. | |||
319 | 319 | ||
320 | ### Testing | 320 | ### Testing |
321 | 321 | ||
322 | Rust Analyzer has three interesting [systems boundaries](https://www.tedinski.com/2018/04/10/making-tests-a-positive-influence-on-design.html) to concentrate tests on. | 322 | Rust Analyzer has three interesting [system boundaries](https://www.tedinski.com/2018/04/10/making-tests-a-positive-influence-on-design.html) to concentrate tests on. |
323 | 323 | ||
324 | The outermost boundary is the `rust-analyzer` crate, which defines an LSP interface in terms of stdio. | 324 | The outermost boundary is the `rust-analyzer` crate, which defines an LSP interface in terms of stdio. |
325 | We do integration testing of this component, by feeding it with a stream of LSP requests and checking responses. | 325 | We do integration testing of this component, by feeding it with a stream of LSP requests and checking responses. |
@@ -328,8 +328,8 @@ For this reason, we try to avoid writing too many tests on this boundary: in a s | |||
328 | Heavy tests are only run when `RUN_SLOW_TESTS` env var is set. | 328 | Heavy tests are only run when `RUN_SLOW_TESTS` env var is set. |
329 | 329 | ||
330 | The middle, and most important, boundary is `ide`. | 330 | The middle, and most important, boundary is `ide`. |
331 | Unlike `rust-analyzer`, which exposes API, `ide` uses Rust API and is intended to use by various tools. | 331 | Unlike `rust-analyzer`, which exposes API, `ide` uses Rust API and is intended for use by various tools. |
332 | Typical test creates an `AnalysisHost`, calls some `Analysis` functions and compares the results against expectation. | 332 | A typical test creates an `AnalysisHost`, calls some `Analysis` functions and compares the results against expectation. |
333 | 333 | ||
334 | The innermost and most elaborate boundary is `hir`. | 334 | The innermost and most elaborate boundary is `hir`. |
335 | It has a much richer vocabulary of types than `ide`, but the basic testing setup is the same: we create a database, run some queries, assert result. | 335 | It has a much richer vocabulary of types than `ide`, but the basic testing setup is the same: we create a database, run some queries, assert result. |
@@ -343,7 +343,7 @@ See the `marks` module in the `test_utils` crate for more. | |||
343 | All required library code must be a part of the tests. | 343 | All required library code must be a part of the tests. |
344 | This ensures fast test execution. | 344 | This ensures fast test execution. |
345 | 345 | ||
346 | **Architecture Invariant:** tests are data driven and do not test API. | 346 | **Architecture Invariant:** tests are data driven and do not test the API. |
347 | Tests which directly call various API functions are a liability, because they make refactoring the API significantly more complicated. | 347 | Tests which directly call various API functions are a liability, because they make refactoring the API significantly more complicated. |
348 | So most of the tests look like this: | 348 | So most of the tests look like this: |
349 | 349 | ||