aboutsummaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* Merge #2911bors[bot]2020-02-03160-172/+798
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 2911: Implement collecting errors while tokenizing r=matklad a=Veetaha Now we are collecting errors from `rustc_lexer` and returning them in `ParsedToken { token, error }` and `ParsedTokens { tokens, errors }` structures **([UPD]: this is now simplified, see updates bellow)**. The main changes are introduced in `ra_syntax/parsing/lexer.rs`. It now exposes the following functions and types: ```rust pub fn tokenize(text: &str) -> ParsedTokens; pub fn tokenize_append(text: &str, parsed_tokens_to_append_to: &mut ParsedTokens); pub fn first_token(text: &str) -> Option<ParsedToken>; // allows any number of tokens in text pub fn single_token(text: &str) -> Option<ParsedToken>; // allows only a single token in text pub struct ParsedToken { pub token: Token, pub error: Option<SyntaxError> } pub struct ParsedTokens { pub tokens: Vec<Token>, pub errors: Vec<SyntaxError> } pub enum TokenizeError { /* Simple enum which reflects rustc_lexer tokenization errors */ } ``` In the first commit I implemented it with iterators, but then decided that since this crate is ad hoc for `rust-analyzer` and we clearly see the places of its usage it would be better to simplify it to vectors. This is currently WIP, because I want to add tests for error messages generated by the lexer. I'd like to listen to you thoughts how to define these tests in `ra_syntax/test-data` dir. Related issues: #223 **[UPD]** After the PR review the API was simplified: ```rust pub fn tokenize(text: &str) -> (Vec<Token>, Vec<SyntaxError>); // Both lex functions do not check for unescape errors pub fn lex_single_syntax_kind(text: &str) -> Option<(SyntaxKind, Option<SyntaxError>)>; pub fn lex_single_valid_syntax_kind(text: &str) -> Option<SyntaxKind>; // This will be removed in the next PR in favour of simlifying `SyntaxError` to `(String, TextRange)` pub enum TokenizeError { /* Simple enum which reflects rustc_lexer tokenization errors */ } // this is private, but may be made public if such demand would exist in future (least privilege principle) fn lex_first_token(text: &str) -> Option<(Token, Option<SyntaxError>)>; ``` Co-authored-by: Veetaha <[email protected]>
| * ra_syntax: added tests for tokenization errorsVeetaha2020-02-03150-49/+427
| |
| * ra_syntax: add backticks around tokens specimenVeetaha2020-02-031-12/+23
| |
| * ra_syntax: removed unnecessary init statement from reparsing testsVeetaha2020-02-031-3/+0
| |
| * ra_syntax: rename first_token() -> lex_first_token()Veetaha2020-02-031-3/+3
| |
| * ra_syntax: fixed a typo in doc commentVeetaha2020-02-031-2/+4
| |
| * ra_syntax: refactored the lexer design as per @matklad and @kiljacken PR reviewVeetaha2020-02-039-178/+199
| |
| * ra_syntax: remove backticks from TokenizeError message since that is not ↵Veetaha2020-02-031-9/+9
| | | | | | | | Markdown ;(
| * add better docs for tokenize errorsVeetaha2020-02-031-2/+2
| |
| * ra_syntax: moved ParsedToken derive attribute under the doc commentVeetaha2020-02-031-1/+1
| |
| * ra_syntax: fixed doc commentVeetaha2020-02-031-1/+1
| |
| * Reimplemented lexer with vectors instead of iteratorsVeetaha2020-02-0310-196/+250
| |
| * ra_syntax: changed added diagnostics information returned from tokenize() ↵Veetaha2020-02-034-74/+237
|/ | | | (implemented with iterators)
* Merge #3003bors[bot]2020-02-033-16/+3
|\ | | | | | | | | | | | | | | | | | | | | 3003: Remove rollup-typescript r=matklad a=matklad It seems like just calling typescript directly is simpler and more reliable? @Veetaha what do you think about this approach? Co-authored-by: Aleksey Kladov <[email protected]>
| * Remove unnecessary flagsAleksey Kladov2020-02-031-2/+2
| |
| * Remove rollup-typescriptAleksey Kladov2020-02-033-15/+2
| | | | | | | | It seems like just calling typescript directly is simpler and more reliable?
* | Merge #3002bors[bot]2020-02-032-29/+32
|\| | | | | | | | | | | | | | | 3002: Update some rollup packages r=matklad a=matklad Co-authored-by: Aleksey Kladov <[email protected]>
| * Update some rollup packagesAleksey Kladov2020-02-032-29/+32
| |
* | Merge #3001bors[bot]2020-02-034-18/+23
|\| | | | | | | | | | | | | | | 3001: Use simple prng instead of a dependency r=matklad a=matklad Co-authored-by: Aleksey Kladov <[email protected]>
| * Use simple prng instead of a dependencyAleksey Kladov2020-02-034-18/+23
|/ | | | closes #2999
* Merge #2997bors[bot]2020-02-031-10/+6
|\ | | | | | | | | | | | | | | 2997: Use proper import name in the label r=matklad a=SomeoneToIgnore Co-authored-by: Kirill Bulatov <[email protected]>
| * Use proper import name in the labelKirill Bulatov2020-02-031-10/+6
| |
* | Merge #2998bors[bot]2020-02-033-15/+14
|\ \ | |/ |/| | | | | | | | | | | 2998: Remove recent improvements to the build script r=matklad a=matklad Co-authored-by: Aleksey Kladov <[email protected]>
| * Remove recent improvements to the build scriptAleksey Kladov2020-02-033-15/+14
|/ | | | | | | | tslib as a dev dependency and commonjs modules are definitely *wrong* in the ideal world, **but** in the real world that's the only combination that works. See https://rust-lang.zulipchat.com/#narrow/stream/185405-t-compiler.2Fwg-rls-2.2E0/topic/Problems.20with.20TypeScript.20build
* Merge #2994bors[bot]2020-02-032-12/+11
|\ | | | | | | | | | | | | | | 2994: Small cleanup r=matklad a=SomeoneToIgnore A follow-up to https://github.com/rust-analyzer/rust-analyzer/pull/2990#discussion_r374044482 Co-authored-by: Kirill Bulatov <[email protected]>
| * Simplify paths searchesKirill Bulatov2020-02-031-4/+3
| |
| * Fix inlay hints test snippet compilationKirill Bulatov2020-02-031-8/+8
| |
* | Merge #2995bors[bot]2020-02-031-0/+1
|\ \ | | | | | | | | | | | | | | | | | | | | | 2995: Fix build of typscript extension r=matklad a=matklad Co-authored-by: Aleksey Kladov <[email protected]>
| * | Fix build of typscript extensionAleksey Kladov2020-02-031-0/+1
|/ /
* | Merge #2959bors[bot]2020-02-0314-247/+244
|\ \ | |/ |/| | | | | | | | | | | | | | | | | | | 2959: Rework how we send diagnostics to client r=matklad a=kiljacken The previous way of sending from the thread pool suffered from stale diagnostics due to being canceled before we could clear the old ones. The key change is moving to sending diagnostics from the main loop thread, but doing all the hard work in the thread pool. This should provide the best of both worlds, with little to no of the downsides. This should hopefully fix a lot of issues, but we'll need testing in each individual issue to be sure. Co-authored-by: Emil Lauridsen <[email protected]>
| * Update snapshot tests due to removed SuggestedFixEmil Lauridsen2020-02-037-39/+67
| |
| * Remove stray todoEmil Lauridsen2020-02-031-1/+0
| |
| * Rework how we send diagnostics to client.Emil Lauridsen2020-02-037-208/+178
| | | | | | | | | | | | | | | | | | | | | | | | The previous way of sending from the thread pool suffered from stale diagnostics due to being canceled before we could clear the old ones. The key change is moving to sending diagnostics from the main loop thread, but doing all the hard work in the thread pool. This should provide the best of both worlds, with little to no of the downsides. This should hopefully fix a lot of issues, but we'll need testing in each individual issue to be sure.
* | Merge #2990bors[bot]2020-02-031-6/+16
|\ \ | |/ |/| | | | | | | | | | | 2990: Use name only when searching for an import candidate r=me a=SomeoneToIgnore Co-authored-by: Kirill Bulatov <[email protected]>
| * Use name only when searching for an import candidateKirill Bulatov2020-02-021-6/+16
| |
* | Merge #2993bors[bot]2020-02-031-1/+1
|\ \ | | | | | | | | | | | | | | | | | | | | | 2993: vscode: fix bundling by switching to es2015 target modules system r=matklad a=Veetaha Quick fix Co-authored-by: Veetaha <[email protected]>
| * | vscode: fix bundling by switching to es2015 target modules systemVeetaha2020-02-031-1/+1
| | |
* | | Merge #2991bors[bot]2020-02-022-6/+6
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 2991: vscode: updated rollup typescript so it typechecks the bundle r=Veetaha a=Veetaha See [this happy update](https://github.com/rollup/plugins/blob/master/packages/typescript/CHANGELOG.md#v300) from `@rollup/typescript-plugin` changelog) I also added a utility script to view the latest dependencies versions (`dry-run` variant) and update them in batch. Beware, that it bumps versions even if the major version of them has changed (for updating only within one major version there is a cli option, but I didn't want add a whole bunch of scripts) Some of the dependencies major versions are out of date: ``` ~/my/projects/rust-analyzer/editors/code (feature/refactoring-vscode-ext) $ npm run bump-deps:dry-run > [email protected] bump-deps:dry-run /home/veetaha/my/projects/rust-analyzer/editors/code > npm-check-updates Checking /home/veetaha/my/projects/rust-analyzer/editors/code/package.json [====================] 16/16 100% jsonc-parser ^2.1.0 → ^2.2.0 @rollup/plugin-commonjs ^11.0.1 → ^11.0.2 @rollup/plugin-node-resolve ^6.1.0 → ^7.1.0 @types/node ^12.12.25 → ^13.7.0 rollup ^1.30.1 → ^1.31.0 tslint ^5.20.1 → ^6.0.0 vsce ^1.71.0 → ^1.72.0 ``` Co-authored-by: Veetaha <[email protected]>
| * | vscode: dropped npm-check-updates scriptVeetaha2020-02-022-2014/+1
| | |
| * | vscode: updated rollup typescript so it typechecks the bundleVeetaha2020-02-022-7/+2020
|/ /
* | Merge #2989bors[bot]2020-02-025-11/+15
|\ \ | |/ |/| | | | | | | | | | | | | 2989: vscode extension: migrate from any to unknown where possible r=Veetaha a=Veetaha `unknown` type is the stricter version of `any` and it should always be prefered (like `const` over `let`). It lets you assign any value to it, but doesn't let you carry out arbitrary operations on them without an explicit type check (like `typeof unknownValue === 'string'`). Co-authored-by: Veetaha <[email protected]>
| * vscode: use void where possibleVeetaha2020-02-023-5/+9
| |
| * vscode: fix, fallback to any for cmd typeVeetaha2020-02-021-1/+1
| |
| * vscode extension: migrate from any to unknown where possibleVeetaha2020-02-024-9/+9
| |
* | Merge #2987bors[bot]2020-02-023-33/+18
|\| | | | | | | | | | | | | | | 2987: vscode refactoring: use more laconic export snytax, split huge string literal r=matklad a=Veetaha Co-authored-by: Veetaha <[email protected]>
| * vscode: fix typeVeetaha2020-02-021-1/+1
| |
| * vscode refactoring: use more laconic export snytax, split huge string to ↵Veetaha2020-02-023-33/+18
|/ | | | several lines
* Merge #2986bors[bot]2020-02-026-10/+11
|\ | | | | | | | | | | | | | | 2986: vscode extension cleanup: migrate to prefer-const tslint rule r=matklad a=Veetaha Co-authored-by: Veetaha <[email protected]>
| * vscode extension cleanup: migrate to prefer-const tslint ruleVeetaha2020-02-026-10/+11
|/
* Merge #2985bors[bot]2020-02-023-33/+34
|\ | | | | | | | | | | | | | | 2985: Avoid premature pessimization r=matklad a=matklad Co-authored-by: Aleksey Kladov <[email protected]>
generated by cgit v1.2.3 (git 2.25.1) at 2024-12-03 20:00:37 +0000