Merge #1213

1213: Make lexer produce only single character puncts r=matklad a=edwin0cheng

As discussed in Zulip, this PR change `lexer` to produce only single char punct.

* Remove producing `DOTDOTDOT, DOTDOTEQ, DOTDOT, COLONCOLON, EQEQ, FAT_ARROW, NEQ, THIN_ARROW` in lexer.
* Add required code in parser to make sure everythings works fine.
* Change some tests (Mainly because the `ast::token_tree` is different)

Note: i think the use of `COLON` in rust is too overloaded :)


Co-authored-by: Edwin Cheng <[email protected]>

1 files changed, 1 insertions, 1 deletions

diff --git a/crates/ra_parser/src/grammar/items.rs b/crates/ra_parser/src/grammar/items.rs
index 318fd69a1..97f8122a9 100644
--- a/crates/ra_parser/src/grammar/items.rs
+++ b/crates/ra_parser/src/grammar/items.rs
@@ -383,7 +383,7 @@ pub(crate) fn token_tree(p: &mut Parser) {
                 return;
             }
             R_PAREN | R_BRACK => p.err_and_bump("unmatched brace"),
-            _ => p.bump(),
+            _ => p.bump_raw(),
         }
     }
     p.expect(closing_paren_kind);