1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
|
<!DOCTYPE html>
<html lang="en">
<head>
<link rel="stylesheet" href="/style.css">
<link rel="stylesheet" href="/syntax.css">
<meta charset="UTF-8">
<meta name="viewport" content="initial-scale=1">
<meta content="#ffffff" name="theme-color">
<meta name="HandheldFriendly" content="true">
<meta property="og:title" content="Introducing Tablespoon">
<meta property="og:type" content="website">
<meta property="og:description" content="a static site {for, by, about} me ">
<meta property="og:url" content="https://peppe.rs">
<link rel="icon" type="image/x-icon" href="/favicon.png">
<title>Introducing Tablespoon · peppe.rs</title>
<body>
<div class="posts">
<div class="post">
<a href="/" class="post-end-link">Home</a>
<span>/</span>
<a href="/posts" class="post-end-link">Posts</a>
<span>/</span>
<a class="post-end-link">Introducing Tablespoon</a>
<a class="stats post-end-link" href="https://git.peppe.rs/web/site/plain/posts/introducing_tablespoon.md
">View Raw</a>
<div class="separator"></div>
<div class="date">
02/08 — 2024
<div class="stats">
<span class="stats-number">
72.33
</span>
<span class="stats-unit">cm</span>
 
<span class="stats-number">
4.5
</span>
<span class="stats-unit">min</span>
</div>
</div>
<h1>
Introducing Tablespoon
</h1>
<div class="post-text">
<p><a href="https://git.peppe.rs/languages/tbsp">tbsp</a> (tree-based
source-processing language) is an awk-like language that operates on
tree-sitter syntax trees. To motivate the need for such a program, we
could begin by writing a markdown-to-html converter using
<code>tbsp</code> and <a
href="https://github.com/tree-sitter-grammars/tree-sitter-markdown">tree-sitter-md</a>.
We need some markdown to begin with:</p>
<pre><code># 1 heading
content of first paragraph
## 1.1 heading
content of nested paragraph</code></pre>
<p>For future reference, this markdown is parsed like so by
tree-sitter-md (visualization generated by <a
href="https://git.peppe.rs/cli/tree-viz">tree-viz</a>):</p>
<pre><code>document
| section
| | atx_heading
| | | atx_h1_marker "#"
| | | heading_content inline "1 heading"
| | paragraph
| | | inline "content of first paragraph"
| | section
| | | atx_heading
| | | | atx_h2_marker "##"
| | | | heading_content inline "1.1 heading"
| | | paragraph
| | | | inline "content of nested paragraph"</code></pre>
<p>Onto the converter itself. Every <code>tbsp</code> program is written
as a collection of stanzas. Typically, we start with a stanza like
so:</p>
<pre><code>BEGIN {
int depth = 0;
print("<html>\n");
print("<body>\n");
}</code></pre>
<p>The stanza begins with a “pattern”, in this case, <code>BEGIN</code>,
and is followed a block of code. This block specifically, is executed
right at the beginning, before traversing the parse tree. In this
stanza, we set a “depth” variable to keep track of nesting of markdown
headers, and begin our html document by printing the
<code><html></code> and <code><body></code> tags.</p>
<p>We can follow this stanza with an <code>END</code> stanza, that is
executed after the traversal:</p>
<pre><code>END {
print("</body>\n");
print("</html>\n");
}</code></pre>
<p>In this stanza, we close off the tags we opened at the start of the
document. We can move onto the interesting bits of the conversion
now:</p>
<pre><code>enter section {
depth += 1;
}
leave section {
depth -= 1;
}</code></pre>
<p>The above stanzas begin with <code>enter</code> and
<code>leave</code> clauses, followed by the name of a tree-sitter node
kind: <code>section</code>. The <code>section</code> identifier is
visible in the tree-visualization above, it encompasses a
markdown-section, and is created for every markdown header. To
understand how <code>tbsp</code> executes above stanzas:</p>
<pre><code>document ... depth = 0
| section <-------- enter section (1) ... depth = 1
| | atx_heading
| | | inline
| | paragraph
| | | inline
| | section <----- enter section (2) ... depth = 2
| | | atx_heading
| | | | inline
| | | paragraph
| | | | inline
| | | <----------- leave section (2) ... depth = 1
| | <-------------- leave section (1) ... depth = 0 </code></pre>
<p>The following stanzas should be self-explanatory now:</p>
<pre><code>enter atx_heading {
print("<h");
print(depth);
print(">");
}
leave atx_heading {
print("</h");
print(depth);
print(">\n");
}
enter inline {
print(text(node));
}</code></pre>
<p>But an explanation is included nonetheless:</p>
<pre><code>document ... depth = 0
| section <-------- enter section (1) ... depth = 1
| | atx_heading <- enter atx_heading ... print "<h1>"
| | | inline <--- enter inline ... print ..
| | | <----------- leave atx_heading ... print "</h1>"
| | paragraph
| | | inline <--- enter inline ... print ..
| | section <----- enter section (2) ... depth = 2
| | | atx_heading enter atx_heading ... print "<h2>"
| | | | inline <- enter inline ... print ..
| | | | <-------- leave atx_heading ... print "</h2>"
| | | paragraph
| | | | inline <- enter inline ... print ..
| | | <----------- leave section (2) ... depth = 1
| | <-------------- leave section (1) ... depth = 0 </code></pre>
<p>The <a
href="https://git.peppe.rs/languages/tbsp/tree/examples">examples</a>
directory contains a complete markdown-to-html converter, along with a
few other motivating examples.</p>
<h3 id="usage">Usage</h3>
<p>The <code>tbsp</code> evaluator is written in rust, use cargo to
build and run:</p>
<pre><code>cargo build --release
./target/release/tbsp --help</code></pre>
<p><code>tbsp</code> requires three inputs:</p>
<ul>
<li>a <code>tbsp</code> program, referred to as “program file”</li>
<li>a language</li>
<li>an input file or some input text at stdin</li>
</ul>
<p>You can run the interpreter like so (this program prints an overview
of a rust file):</p>
<pre><code>$ ./target/release/tbsp \
-f./examples/code-overview/overview.tbsp \
-l rust \
src/main.rs
module
└╴struct Cli
└╴trait Cli
└╴fn program
└╴fn language
└╴fn file
└╴fn try_consume_stdin
└╴fn main</code></pre>
</div>
<div class="intro">
Hi.
<div class="hot-links">
<a href="/index.xml" class="feed-button">Subscribe</a>
</div>
<p>I'm Akshay, programmer and pixel-artist.
I write <a href="https://git.peppe.rs">open-source stuff</a>.
I also design fonts:
<a href="https://git.peppe.rs/fonts/scientifica/about">scientifica</a>,
<a href="https://git.peppe.rs/fonts/curie/about">curie</a>.
</p>
<p>Reach out at [email protected].</p>
</div>
<a href="/" class="post-end-link">Home</a>
<span>/</span>
<a href="/posts" class="post-end-link">Posts</a>
<span>/</span>
<a class="post-end-link">Introducing Tablespoon</a>
<a class="stats post-end-link" href="https://git.peppe.rs/web/site/plain/posts/introducing_tablespoon.md
">View Raw</a>
</div>
</div>
</body>
</html>
|
