Merge #2061

2061: Theme loading and "editor.tokenColorCustomizations" support. r=matklad a=seivan

Fixes: [Issue#1294](https://github.com/rust-analyzer/rust-analyzer/issues/1294#issuecomment-497450325)

TODO: 
- [x] Load themes
- [x] Load existing `ralsp`-prefixed overrides from `"workbench.colorCustomizations"`.
- [x] Load overrides from `"editor.tokenColorCustomizations.textMateRules"`.
- [x] Use RA tags to load `vscode.DecorationRenderOptions` (colors) from theme & overrides.
- [x] Map RA tags to common TextMate scopes before loading colors.
- [x] Add default scope mappings in extension.
- [x] Cache mappings between settings updates. 
- [x] Add scope mapping configuration manifest in `package.json`
- [x] Load configurable scope mappings from settings.
- [x] Load JSON Scheme for text mate scope rules in settings.
- [x] Update [Readme](https://github.com/seivan/rust-analyzer/blob/feature/themes/docs/user/README.md#settings).

Borrowed the theme loading (`scopes.ts`) from `Tree Sitter` with some modifications to reading `"editor.tokenColorCustomizations"` for merging with loaded themes and had to remove the async portions to be able to load it from settings updates. 

~Just a PoC and an idea I toyed around with a lot of room for improvement.~
For starters, certain keywords aren't part of the standard TextMate grammar, so it still reads colors from the `ralsp` prefixed values in `"workbench.colorCustomizations"`. 

But I think there's more value making the extension work with existing themes by maping some of the decoration tags to existing key or keys. 

<img width="453" alt="Screenshot 2019-11-09 at 17 43 18" src="https://user-images.githubusercontent.com/55424/68531968-71b4e380-0318-11ea-924e-cdbb8d5eae06.png">
<img width="780" alt="Screenshot 2019-11-09 at 17 41 45" src="https://user-images.githubusercontent.com/55424/68531950-4b8f4380-0318-11ea-8f85-24a84efaf23b.png">
<img width="468" alt="Screenshot 2019-11-09 at 17 40 29" src="https://user-images.githubusercontent.com/55424/68531952-51852480-0318-11ea-800a-6ae9215f5368.png">


These will merge with the default ones coming with the extension, so you don't have to implement all of them and works well with overrides defined in settings. 

```jsonc
    "editor.tokenColorCustomizations": {
        "textMateRules": [
            {
                "scope": "keyword",
                "settings": {
                    "fontStyle": "bold",
                }
            },
        ]
    },
```


Edit: The idea is to work with 90% of the themes out there by working within existing scopes available that are generally styled. It's not to say I want to erase the custom Rust scopes - those should still remain and eventually worked into a custom grammar bundle for Rust specific themes that target those, I just want to make it work with generic themes offered on the market place for now. 

A custom grammar bundle and themes for Rust specific scopes is out of... scope for this PR. 
We'll make another round to tackle those issues. 


Current fallbacks implemented

```typescript
    [
        'comment',
        [
            'comment',
            'comment.block',
            'comment.line',
            'comment.block.documentation'
        ]
    ],
    ['string', ['string']],
    ['keyword', ['keyword']],
    ['keyword.control', ['keyword.control', 'keyword', 'keyword.other']],
    [
        'keyword.unsafe',
        ['storage.modifier', 'keyword.other', 'keyword.control', 'keyword']
    ],
    ['function', ['entity.name.function']],
    ['parameter', ['variable.parameter']],
    ['constant', ['constant', 'variable']],
    ['type', ['entity.name.type']],
    ['builtin', ['variable.language', 'support.type', 'support.type']],
    ['text', ['string', 'string.quoted', 'string.regexp']],
    ['attribute', ['keyword']],
    ['literal', ['string', 'string.quoted', 'string.regexp']],
    ['macro', ['support.other']],
    ['variable', ['variable']],
    ['variable.mut', ['variable', 'storage.modifier']],
    [
        'field',
        [
            'variable.object.property',
            'meta.field.declaration',
            'meta.definition.property',
            'variable.other'
        ]
    ],
    ['module', ['entity.name.section', 'entity.other']]
```


Co-authored-by: Seivan Heidari <[email protected]>

1 files changed, 9 insertions, 5 deletions

diff --git a/editors/code/src/config.ts b/editors/code/src/config.ts
index 4b388b80c..a88be6e35 100644
--- a/editors/code/src/config.ts
+++ b/editors/code/src/config.ts
@@ -1,5 +1,6 @@
 import * as vscode from 'vscode';
-
+import * as scopes from './scopes';
+import * as scopesMapper from './scopes_mapper';
 import { Server } from './server';
 
 const RA_LSP_DEBUG = process.env.__RA_LSP_SERVER_DEBUG;
@@ -54,10 +55,17 @@ export class Config {
 
     public userConfigChanged() {
         const config = vscode.workspace.getConfiguration('rust-analyzer');
+
+        Server.highlighter.removeHighlights();
+
         let requireReloadMessage = null;
 
         if (config.has('highlightingOn')) {
             this.highlightingOn = config.get('highlightingOn') as boolean;
+            if (this.highlightingOn) {
+                scopes.load();
+                scopesMapper.load();
+            }
         }
 
         if (config.has('rainbowHighlightingOn')) {
@@ -66,10 +74,6 @@ export class Config {
             ) as boolean;
         }
 
-        if (!this.highlightingOn && Server) {
-            Server.highlighter.removeHighlights();
-        }
-
         if (config.has('enableEnhancedTyping')) {
             this.enableEnhancedTyping = config.get(
                 'enableEnhancedTyping',