Merge #5101

5101: Add expect -- a light-weight alternative to insta r=matklad a=matklad

This PR implements a small snapshot-testing library. Snapshot updating is done by setting an env var, or by using editor feature (which runs  a test with env-var set). 

Here's workflow for updating a failing test:

![expect](https://user-images.githubusercontent.com/1711539/85926956-28afa080-b8a3-11ea-9260-c6d0d8914d0b.gif)

Here's workflow for adding a new test:

![expect-fresh](https://user-images.githubusercontent.com/1711539/85926961-306f4500-b8a3-11ea-9369-f2373e327a3f.gif)

Note that colorized diffs are not implemented in this PR, but should be easy to add (we already use them in test_utils). 

Main differences from insta (which is essential for rust-analyzer development, thanks @mitsuhiko!):
* self-updating tests, no need for a separate tool
* fewer features (only inline snapshots, no redactions)
* fewer deps (no yaml, no persistence)
* tighter integration with editor
* first-class snapshot object, which can be used to write test functions (as opposed to testing macros)
* trivial to tweak for rust-analyzer needs, by virtue of being a workspace member. 

I think eventually we should converge to a single snapshot testing library, but I am not sure that `expect` is exactly right, so I suggest rolling with both insta and expect for some time (if folks agree that expect might be better in the first place!). 

# Editor Integration Implementation 

The thing I am most excited about is the ability to update a specific snapshot from the editor. I want this to be available to other snapshot-testing libraries (cc @mitsuhiko, @aaronabramov), so I want to document how this works. 

The ideal UI here would be a code action (:bulb:). Unfortunately, it seems like it is impossible to implement without some kind of persistence (if you save test failures into some kind of a database, like insta does, than you can read the database from the editor plugin). Note that it is possible to highlight error by outputing error message in rustc's format. Unfortunately, one can't use the same trick to implement a quick fix. 

For this reason, expect makes use of another rust-analyzer feature -- ability to run a single test at the cursor position. This does need some expect-specific code in rust-analyzer unfortunately. Specifically, if rust-analyzer notices that the cursor is on `expect!` macro, it adds a special flag to runnable's JSON. However, given #5017 it is possible to approximate this well-enough without rust-analyzer integration. Specifically, an extension can register a special runner which checks (using regexes) if rust-anlyzer runnable covers text with specific macro invocation and do special magic in that case. 

closes #3835 


Co-authored-by: Aleksey Kladov <[email protected]>

7 files changed, 361 insertions, 12 deletions

diff --git a/crates/expect/Cargo.toml b/crates/expect/Cargo.toml
new file mode 100644
index 000000000..caee43106
--- /dev/null
+++ b/crates/expect/Cargo.toml
@@ -0,0 +1,10 @@
+[package]
+name = "expect"
+version = "0.1.0"
+authors = ["rust-analyzer developers"]
+edition = "2018"
+
+[dependencies]
+once_cell = "1"
+difference = "2"
+stdx = { path = "../stdx" }
diff --git a/crates/expect/src/lib.rs b/crates/expect/src/lib.rs
new file mode 100644
index 000000000..dd7b96aab
--- /dev/null
+++ b/crates/expect/src/lib.rs
@@ -0,0 +1,293 @@
+//! Snapshot testing library, see
+//! https://github.com/rust-analyzer/rust-analyzer/pull/5101
+use std::{
+    collections::HashMap,
+    env, fmt, fs,
+    ops::Range,
+    panic,
+    path::{Path, PathBuf},
+    sync::Mutex,
+};
+
+use difference::Changeset;
+use once_cell::sync::Lazy;
+use stdx::{lines_with_ends, trim_indent};
+
+const HELP: &str = "
+You can update all `expect![[]]` tests by:
+
+    env UPDATE_EXPECT=1 cargo test
+
+To update a single test, place the cursor on `expect` token and use `run` feature of rust-analyzer.
+";
+
+fn update_expect() -> bool {
+    env::var("UPDATE_EXPECT").is_ok()
+}
+
+/// expect![[""]]
+#[macro_export]
+macro_rules! expect {
+    [[$lit:literal]] => {$crate::Expect {
+        file: file!(),
+        line: line!(),
+        column: column!(),
+        data: $lit,
+    }};
+    [[]] => { $crate::expect![[""]] };
+}
+
+#[derive(Debug)]
+pub struct Expect {
+    pub file: &'static str,
+    pub line: u32,
+    pub column: u32,
+    pub data: &'static str,
+}
+
+impl Expect {
+    pub fn assert_eq(&self, actual: &str) {
+        let trimmed = self.trimmed();
+        if &trimmed == actual {
+            return;
+        }
+        Runtime::fail(self, &trimmed, actual);
+    }
+    pub fn assert_debug_eq(&self, actual: &impl fmt::Debug) {
+        let actual = format!("{:#?}\n", actual);
+        self.assert_eq(&actual)
+    }
+
+    fn trimmed(&self) -> String {
+        if !self.data.contains('\n') {
+            return self.data.to_string();
+        }
+        trim_indent(self.data)
+    }
+
+    fn locate(&self, file: &str) -> Location {
+        let mut target_line = None;
+        let mut line_start = 0;
+        for (i, line) in lines_with_ends(file).enumerate() {
+            if i == self.line as usize - 1 {
+                let pat = "expect![[";
+                let offset = line.find(pat).unwrap();
+                let literal_start = line_start + offset + pat.len();
+                let indent = line.chars().take_while(|&it| it == ' ').count();
+                target_line = Some((literal_start, indent));
+                break;
+            }
+            line_start += line.len();
+        }
+        let (literal_start, line_indent) = target_line.unwrap();
+        let literal_length =
+            file[literal_start..].find("]]").expect("Couldn't find matching `]]` for `expect![[`.");
+        let literal_range = literal_start..literal_start + literal_length;
+        Location { line_indent, literal_range }
+    }
+}
+
+#[derive(Default)]
+struct Runtime {
+    help_printed: bool,
+    per_file: HashMap<&'static str, FileRuntime>,
+}
+static RT: Lazy<Mutex<Runtime>> = Lazy::new(Default::default);
+
+impl Runtime {
+    fn fail(expect: &Expect, expected: &str, actual: &str) {
+        let mut rt = RT.lock().unwrap_or_else(|poisoned| poisoned.into_inner());
+        let mut updated = "";
+        if update_expect() {
+            updated = " (updated)";
+            rt.per_file
+                .entry(expect.file)
+                .or_insert_with(|| FileRuntime::new(expect))
+                .update(expect, actual);
+        }
+        let print_help = !rt.help_printed && !update_expect();
+        rt.help_printed = true;
+
+        let help = if print_help { HELP } else { "" };
+
+        let diff = Changeset::new(actual, expected, "\n");
+
+        println!(
+            "\n
+\x1b[1m\x1b[91merror\x1b[97m: expect test failed\x1b[0m{}
+   \x1b[1m\x1b[34m-->\x1b[0m {}:{}:{}
+{}
+\x1b[1mExpect\x1b[0m:
+----
+{}
+----
+
+\x1b[1mActual\x1b[0m:
+----
+{}
+----
+
+\x1b[1mDiff\x1b[0m:
+----
+{}
+----
+",
+            updated, expect.file, expect.line, expect.column, help, expected, actual, diff
+        );
+        // Use resume_unwind instead of panic!() to prevent a backtrace, which is unnecessary noise.
+        panic::resume_unwind(Box::new(()));
+    }
+}
+
+struct FileRuntime {
+    path: PathBuf,
+    original_text: String,
+    patchwork: Patchwork,
+}
+
+impl FileRuntime {
+    fn new(expect: &Expect) -> FileRuntime {
+        let path = workspace_root().join(expect.file);
+        let original_text = fs::read_to_string(&path).unwrap();
+        let patchwork = Patchwork::new(original_text.clone());
+        FileRuntime { path, original_text, patchwork }
+    }
+    fn update(&mut self, expect: &Expect, actual: &str) {
+        let loc = expect.locate(&self.original_text);
+        let patch = format_patch(loc.line_indent.clone(), actual);
+        self.patchwork.patch(loc.literal_range, &patch);
+        fs::write(&self.path, &self.patchwork.text).unwrap()
+    }
+}
+
+#[derive(Debug)]
+struct Location {
+    line_indent: usize,
+    literal_range: Range<usize>,
+}
+
+#[derive(Debug)]
+struct Patchwork {
+    text: String,
+    indels: Vec<(Range<usize>, usize)>,
+}
+
+impl Patchwork {
+    fn new(text: String) -> Patchwork {
+        Patchwork { text, indels: Vec::new() }
+    }
+    fn patch(&mut self, mut range: Range<usize>, patch: &str) {
+        self.indels.push((range.clone(), patch.len()));
+        self.indels.sort_by_key(|(delete, _insert)| delete.start);
+
+        let (delete, insert) = self
+            .indels
+            .iter()
+            .take_while(|(delete, _)| delete.start < range.start)
+            .map(|(delete, insert)| (delete.end - delete.start, insert))
+            .fold((0usize, 0usize), |(x1, y1), (x2, y2)| (x1 + x2, y1 + y2));
+
+        for pos in &mut [&mut range.start, &mut range.end] {
+            **pos -= delete;
+            **pos += insert;
+        }
+
+        self.text.replace_range(range, &patch);
+    }
+}
+
+fn format_patch(line_indent: usize, patch: &str) -> String {
+    let mut max_hashes = 0;
+    let mut cur_hashes = 0;
+    for byte in patch.bytes() {
+        if byte != b'#' {
+            cur_hashes = 0;
+            continue;
+        }
+        cur_hashes += 1;
+        max_hashes = max_hashes.max(cur_hashes);
+    }
+    let hashes = &"#".repeat(max_hashes + 1);
+    let indent = &" ".repeat(line_indent);
+    let is_multiline = patch.contains('\n');
+
+    let mut buf = String::new();
+    buf.push('r');
+    buf.push_str(hashes);
+    buf.push('"');
+    if is_multiline {
+        buf.push('\n');
+    }
+    let mut final_newline = false;
+    for line in lines_with_ends(patch) {
+        if is_multiline {
+            buf.push_str(indent);
+            buf.push_str("    ");
+        }
+        buf.push_str(line);
+        final_newline = line.ends_with('\n');
+    }
+    if final_newline {
+        buf.push_str(indent);
+    }
+    buf.push('"');
+    buf.push_str(hashes);
+    buf
+}
+
+fn workspace_root() -> PathBuf {
+    Path::new(
+        &env::var("CARGO_MANIFEST_DIR").unwrap_or_else(|_| env!("CARGO_MANIFEST_DIR").to_owned()),
+    )
+    .ancestors()
+    .nth(2)
+    .unwrap()
+    .to_path_buf()
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn test_format_patch() {
+        let patch = format_patch(0, "hello\nworld\n");
+        expect![[r##"
+            r#"
+                hello
+                world
+            "#"##]]
+        .assert_eq(&patch);
+
+        let patch = format_patch(4, "single line");
+        expect![[r##"r#"single line"#"##]].assert_eq(&patch);
+    }
+
+    #[test]
+    fn test_patchwork() {
+        let mut patchwork = Patchwork::new("one two three".to_string());
+        patchwork.patch(4..7, "zwei");
+        patchwork.patch(0..3, "один");
+        patchwork.patch(8..13, "3");
+        expect![[r#"
+            Patchwork {
+                text: "один zwei 3",
+                indels: [
+                    (
+                        0..3,
+                        8,
+                    ),
+                    (
+                        4..7,
+                        4,
+                    ),
+                    (
+                        8..13,
+                        1,
+                    ),
+                ],
+            }
+        "#]]
+        .assert_debug_eq(&patchwork);
+    }
+}
diff --git a/crates/ra_ide/Cargo.toml b/crates/ra_ide/Cargo.toml
index bbc6a5c9b..8e8892309 100644
--- a/crates/ra_ide/Cargo.toml
+++ b/crates/ra_ide/Cargo.toml
@@ -28,6 +28,7 @@ ra_cfg = { path = "../ra_cfg" }
 ra_fmt = { path = "../ra_fmt" }
 ra_prof = { path = "../ra_prof" }
 test_utils = { path = "../test_utils" }
+expect = { path = "../expect" }
 ra_assists = { path = "../ra_assists" }
 ra_ssr = { path = "../ra_ssr" }
 
diff --git a/crates/ra_ide/src/goto_definition.rs b/crates/ra_ide/src/goto_definition.rs
index bea7fbfa7..969d5e0ff 100644
--- a/crates/ra_ide/src/goto_definition.rs
+++ b/crates/ra_ide/src/goto_definition.rs
@@ -103,6 +103,7 @@ pub(crate) fn reference_definition(
 
 #[cfg(test)]
 mod tests {
+    use expect::{expect, Expect};
     use test_utils::assert_eq_text;
 
     use crate::mock_analysis::analysis_and_position;
@@ -142,16 +143,40 @@ mod tests {
         nav.assert_match(expected);
     }
 
+    fn check(ra_fixture: &str, expect: Expect) {
+        let (analysis, pos) = analysis_and_position(ra_fixture);
+
+        let mut navs = analysis.goto_definition(pos).unwrap().unwrap().info;
+        if navs.len() == 0 {
+            panic!("unresolved reference")
+        }
+        assert_eq!(navs.len(), 1);
+
+        let nav = navs.pop().unwrap();
+        let file_text = analysis.file_text(nav.file_id()).unwrap();
+
+        let mut actual = nav.debug_render();
+        actual += "\n";
+        actual += &file_text[nav.full_range()].to_string();
+        if let Some(focus) = nav.focus_range() {
+            actual += "|";
+            actual += &file_text[focus];
+            actual += "\n";
+        }
+        expect.assert_eq(&actual);
+    }
+
     #[test]
     fn goto_def_in_items() {
-        check_goto(
-            "
-            //- /lib.rs
-            struct Foo;
-            enum E { X(Foo<|>) }
-            ",
-            "Foo STRUCT_DEF FileId(1) 0..11 7..10",
-            "struct Foo;|Foo",
+        check(
+            r#"
+struct Foo;
+enum E { X(Foo<|>) }
+"#,
+            expect![[r#"
+                Foo STRUCT_DEF FileId(1) 0..11 7..10
+                struct Foo;|Foo
+            "#]],
         );
     }
 
diff --git a/crates/rust-analyzer/src/handlers.rs b/crates/rust-analyzer/src/handlers.rs
index e35a5e846..0940fcc28 100644
--- a/crates/rust-analyzer/src/handlers.rs
+++ b/crates/rust-analyzer/src/handlers.rs
@@ -23,7 +23,7 @@ use ra_ide::{
 };
 use ra_prof::profile;
 use ra_project_model::TargetKind;
-use ra_syntax::{AstNode, SyntaxKind, TextRange, TextSize};
+use ra_syntax::{algo, ast, AstNode, SyntaxKind, TextRange, TextSize};
 use serde::{Deserialize, Serialize};
 use serde_json::to_value;
 use stdx::{format_to, split_delim};
@@ -407,8 +407,19 @@ pub(crate) fn handle_runnables(
     let file_id = from_proto::file_id(&snap, &params.text_document.uri)?;
     let line_index = snap.analysis.file_line_index(file_id)?;
     let offset = params.position.map(|it| from_proto::offset(&line_index, it));
-    let mut res = Vec::new();
     let cargo_spec = CargoTargetSpec::for_file(&snap, file_id)?;
+
+    let expect_test = match offset {
+        Some(offset) => {
+            let source_file = snap.analysis.parse(file_id)?;
+            algo::find_node_at_offset::<ast::MacroCall>(source_file.syntax(), offset)
+                .and_then(|it| it.path()?.segment()?.name_ref())
+                .map_or(false, |it| it.text() == "expect")
+        }
+        None => false,
+    };
+
+    let mut res = Vec::new();
     for runnable in snap.analysis.runnables(file_id)? {
         if let Some(offset) = offset {
             if !runnable.nav.full_range().contains_inclusive(offset) {
@@ -418,8 +429,12 @@ pub(crate) fn handle_runnables(
         if should_skip_target(&runnable, cargo_spec.as_ref()) {
             continue;
         }
-
-        res.push(to_proto::runnable(&snap, file_id, runnable)?);
+        let mut runnable = to_proto::runnable(&snap, file_id, runnable)?;
+        if expect_test {
+            runnable.label = format!("{} + expect", runnable.label);
+            runnable.args.expect_test = Some(true);
+        }
+        res.push(runnable);
     }
 
     // Add `cargo check` and `cargo test` for the whole package
@@ -438,6 +453,7 @@ pub(crate) fn handle_runnables(
                             spec.package.clone(),
                         ],
                         executable_args: Vec::new(),
+                        expect_test: None,
                     },
                 })
             }
@@ -451,6 +467,7 @@ pub(crate) fn handle_runnables(
                     workspace_root: None,
                     cargo_args: vec!["check".to_string(), "--workspace".to_string()],
                     executable_args: Vec::new(),
+                    expect_test: None,
                 },
             });
         }
diff --git a/crates/rust-analyzer/src/lsp_ext.rs b/crates/rust-analyzer/src/lsp_ext.rs
index 1371f6cb4..1befe678c 100644
--- a/crates/rust-analyzer/src/lsp_ext.rs
+++ b/crates/rust-analyzer/src/lsp_ext.rs
@@ -161,6 +161,8 @@ pub struct CargoRunnable {
     pub cargo_args: Vec<String>,
     // stuff after --
     pub executable_args: Vec<String>,
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub expect_test: Option<bool>,
 }
 
 pub enum InlayHints {}
diff --git a/crates/rust-analyzer/src/to_proto.rs b/crates/rust-analyzer/src/to_proto.rs
index f6cb8e4bb..a03222ae9 100644
--- a/crates/rust-analyzer/src/to_proto.rs
+++ b/crates/rust-analyzer/src/to_proto.rs
@@ -666,6 +666,7 @@ pub(crate) fn runnable(
             workspace_root: workspace_root.map(|it| it.into()),
             cargo_args,
             executable_args,
+            expect_test: None,
         },
     })
 }