Merge #5101

5101: Add expect -- a light-weight alternative to insta r=matklad a=matklad

This PR implements a small snapshot-testing library. Snapshot updating is done by setting an env var, or by using editor feature (which runs  a test with env-var set). 

Here's workflow for updating a failing test:

![expect](https://user-images.githubusercontent.com/1711539/85926956-28afa080-b8a3-11ea-9260-c6d0d8914d0b.gif)

Here's workflow for adding a new test:

![expect-fresh](https://user-images.githubusercontent.com/1711539/85926961-306f4500-b8a3-11ea-9369-f2373e327a3f.gif)

Note that colorized diffs are not implemented in this PR, but should be easy to add (we already use them in test_utils). 

Main differences from insta (which is essential for rust-analyzer development, thanks @mitsuhiko!):
* self-updating tests, no need for a separate tool
* fewer features (only inline snapshots, no redactions)
* fewer deps (no yaml, no persistence)
* tighter integration with editor
* first-class snapshot object, which can be used to write test functions (as opposed to testing macros)
* trivial to tweak for rust-analyzer needs, by virtue of being a workspace member. 

I think eventually we should converge to a single snapshot testing library, but I am not sure that `expect` is exactly right, so I suggest rolling with both insta and expect for some time (if folks agree that expect might be better in the first place!). 

# Editor Integration Implementation 

The thing I am most excited about is the ability to update a specific snapshot from the editor. I want this to be available to other snapshot-testing libraries (cc @mitsuhiko, @aaronabramov), so I want to document how this works. 

The ideal UI here would be a code action (:bulb:). Unfortunately, it seems like it is impossible to implement without some kind of persistence (if you save test failures into some kind of a database, like insta does, than you can read the database from the editor plugin). Note that it is possible to highlight error by outputing error message in rustc's format. Unfortunately, one can't use the same trick to implement a quick fix. 

For this reason, expect makes use of another rust-analyzer feature -- ability to run a single test at the cursor position. This does need some expect-specific code in rust-analyzer unfortunately. Specifically, if rust-analyzer notices that the cursor is on `expect!` macro, it adds a special flag to runnable's JSON. However, given #5017 it is possible to approximate this well-enough without rust-analyzer integration. Specifically, an extension can register a special runner which checks (using regexes) if rust-anlyzer runnable covers text with specific macro invocation and do special magic in that case. 

closes #3835 


Co-authored-by: Aleksey Kladov <[email protected]>

2 files changed, 6 insertions, 1 deletions

diff --git a/editors/code/src/lsp_ext.ts b/editors/code/src/lsp_ext.ts
index e16ea799c..fdb99956b 100644
--- a/editors/code/src/lsp_ext.ts
+++ b/editors/code/src/lsp_ext.ts
@@ -60,6 +60,7 @@ export interface Runnable {
         workspaceRoot?: string;
         cargoArgs: string[];
         executableArgs: string[];
+        expectTest?: boolean;
     };
 }
 export const runnables = new lc.RequestType<RunnablesParams, Runnable[], void>("experimental/runnables");
diff --git a/editors/code/src/run.ts b/editors/code/src/run.ts
index 766b05112..e1430e31f 100644
--- a/editors/code/src/run.ts
+++ b/editors/code/src/run.ts
@@ -108,12 +108,16 @@ export async function createTask(runnable: ra.Runnable, config: Config): Promise
     if (runnable.args.executableArgs.length > 0) {
         args.push('--', ...runnable.args.executableArgs);
     }
+    const env: { [key: string]: string } = { "RUST_BACKTRACE": "short" };
+    if (runnable.args.expectTest) {
+        env["UPDATE_EXPECT"] = "1";
+    }
     const definition: tasks.CargoTaskDefinition = {
         type: tasks.TASK_TYPE,
         command: args[0], // run, test, etc...
         args: args.slice(1),
         cwd: runnable.args.workspaceRoot,
-        env: Object.assign({}, process.env as { [key: string]: string }, { "RUST_BACKTRACE": "short" }),
+        env: Object.assign({}, process.env as { [key: string]: string }, env),
     };
 
     const target = vscode.workspace.workspaceFolders![0]; // safe, see main activate()