Merge #2061

2061: Theme loading and "editor.tokenColorCustomizations" support. r=matklad a=seivan

Fixes: [Issue#1294](https://github.com/rust-analyzer/rust-analyzer/issues/1294#issuecomment-497450325)

TODO: 
- [x] Load themes
- [x] Load existing `ralsp`-prefixed overrides from `"workbench.colorCustomizations"`.
- [x] Load overrides from `"editor.tokenColorCustomizations.textMateRules"`.
- [x] Use RA tags to load `vscode.DecorationRenderOptions` (colors) from theme & overrides.
- [x] Map RA tags to common TextMate scopes before loading colors.
- [x] Add default scope mappings in extension.
- [x] Cache mappings between settings updates. 
- [x] Add scope mapping configuration manifest in `package.json`
- [x] Load configurable scope mappings from settings.
- [x] Load JSON Scheme for text mate scope rules in settings.
- [x] Update [Readme](https://github.com/seivan/rust-analyzer/blob/feature/themes/docs/user/README.md#settings).

Borrowed the theme loading (`scopes.ts`) from `Tree Sitter` with some modifications to reading `"editor.tokenColorCustomizations"` for merging with loaded themes and had to remove the async portions to be able to load it from settings updates. 

~Just a PoC and an idea I toyed around with a lot of room for improvement.~
For starters, certain keywords aren't part of the standard TextMate grammar, so it still reads colors from the `ralsp` prefixed values in `"workbench.colorCustomizations"`. 

But I think there's more value making the extension work with existing themes by maping some of the decoration tags to existing key or keys. 

<img width="453" alt="Screenshot 2019-11-09 at 17 43 18" src="https://user-images.githubusercontent.com/55424/68531968-71b4e380-0318-11ea-924e-cdbb8d5eae06.png">
<img width="780" alt="Screenshot 2019-11-09 at 17 41 45" src="https://user-images.githubusercontent.com/55424/68531950-4b8f4380-0318-11ea-8f85-24a84efaf23b.png">
<img width="468" alt="Screenshot 2019-11-09 at 17 40 29" src="https://user-images.githubusercontent.com/55424/68531952-51852480-0318-11ea-800a-6ae9215f5368.png">


These will merge with the default ones coming with the extension, so you don't have to implement all of them and works well with overrides defined in settings. 

```jsonc
    "editor.tokenColorCustomizations": {
        "textMateRules": [
            {
                "scope": "keyword",
                "settings": {
                    "fontStyle": "bold",
                }
            },
        ]
    },
```


Edit: The idea is to work with 90% of the themes out there by working within existing scopes available that are generally styled. It's not to say I want to erase the custom Rust scopes - those should still remain and eventually worked into a custom grammar bundle for Rust specific themes that target those, I just want to make it work with generic themes offered on the market place for now. 

A custom grammar bundle and themes for Rust specific scopes is out of... scope for this PR. 
We'll make another round to tackle those issues. 


Current fallbacks implemented

```typescript
    [
        'comment',
        [
            'comment',
            'comment.block',
            'comment.line',
            'comment.block.documentation'
        ]
    ],
    ['string', ['string']],
    ['keyword', ['keyword']],
    ['keyword.control', ['keyword.control', 'keyword', 'keyword.other']],
    [
        'keyword.unsafe',
        ['storage.modifier', 'keyword.other', 'keyword.control', 'keyword']
    ],
    ['function', ['entity.name.function']],
    ['parameter', ['variable.parameter']],
    ['constant', ['constant', 'variable']],
    ['type', ['entity.name.type']],
    ['builtin', ['variable.language', 'support.type', 'support.type']],
    ['text', ['string', 'string.quoted', 'string.regexp']],
    ['attribute', ['keyword']],
    ['literal', ['string', 'string.quoted', 'string.regexp']],
    ['macro', ['support.other']],
    ['variable', ['variable']],
    ['variable.mut', ['variable', 'storage.modifier']],
    [
        'field',
        [
            'variable.object.property',
            'meta.field.declaration',
            'meta.definition.property',
            'variable.other'
        ]
    ],
    ['module', ['entity.name.section', 'entity.other']]
```


Co-authored-by: Seivan Heidari <[email protected]>

1 files changed, 11 insertions, 2 deletions

diff --git a/docs/user/README.md b/docs/user/README.md
index 9cdabfd42..968d2e34c 100644
--- a/docs/user/README.md
+++ b/docs/user/README.md
@@ -87,7 +87,16 @@ host.
 
 ### Settings
 
-* `rust-analyzer.highlightingOn`: enables experimental syntax highlighting
+* `rust-analyzer.highlightingOn`: enables experimental syntax highlighting.
+* `rust-analyzer.scopeMappings` -- a scheme backed JSON object to tweak Rust Analyzer scopes to TextMate scopes.
+   ```jsonc
+    {
+        //Will autocomplete keys to available RA scopes. 
+        "keyword.unsafe": ["keyword", "keyword.control"],
+        //Values are string | TextMateScope | [string | TextMateScope]
+        "variable.mut": "variable"
+    }
+   ```
 * `rust-analyzer.enableEnhancedTyping`: by default, rust-analyzer intercepts
   `Enter` key to make it easier to continue comments. Note that it may conflict with VIM emulation plugin.
 * `rust-analyzer.raLspServerPath`: path to `ra_lsp_server` executable
@@ -106,7 +115,7 @@ host.
 * `rust-analyzer.trace.cargo-watch`: enables cargo-watch logging
 * `RUST_SRC_PATH`: environment variable that overwrites the sysroot
 * `rust-analyzer.featureFlags` -- a JSON object to tweak fine-grained behavior:
-   ```js
+   ```jsonc
    {
        // Show diagnostics produced by rust-analyzer itself.
        "lsp.diagnostics": true,