Blog

Reading time:

Rewrite my toy language interpreter in Rust

Im rewriting Grotsky (my toy programming language) in Rust, the previous implementation was done in Go. The goal of the rewrite is to improve my Rust skills, and to improve the performance of Grotsky, by at least 10x.

As of this writting the Rust implementation is 4x faster than the Golang implementation.

I've rewritten the Lexer, Parser and part of the Runtime. Enough for run this code and measure how long it takes for each implementation to finish:

let a = 1
while a < 100000000 {
    a = a + 1
}

I was inspired by Mitchel Hashimoto's post: My Approach to Building Large Technical Projects. And I want to create a roadmap of little projects to reach my goal of having a full-fledged interpreter in Rust.

Goal

Until now Grotsky has been running on a Tree-based interpreter. My goal is that at the end of the rewrite I will have a Bytecode interpreter.

First I want to rewrite the Tree-based interpreter in Rust and achieve at least 10x performance improvement.

Then figure out if I want to use a Register based or Stack based bytecode interpreter.

Also, I would like to have a stable bytecode representation to be able to compile programs to a binary format that can be shipped as is or packaged into a binary.

Finally, it's time Grotsky gets a REPL.

Roadmap

Believe it or not, Grotsky it's pretty big. It has support for reading and writting files, sockets and more on the stdlib. Which means I have a huge task ahead of me.

First thing I want to do is to have some sort of automated testing setup that can compare Rust vs Go implementation. Right now all test are written as Go unit test, I need to make them agnostic of language backend.

  • Jun 9: Have a complete setup of automated tests for correctness and performance.
  • Jun 16: Language runtime (without stdlib) rewritten in Rust.
  • Jun 23: Finish migrating stdlib and publish results (new blog post). Use new implementation for blog engine.
  • Jun 30: Decide which kind of bytecode interpreter to use, then make a design and plan for the implementation.
  • Jul 7: Have a working bytecode interpreter that is able to run the program shown in this post ^. Just a simple while loop. Compare performance and share progress.
  • Jul 14: Add support for functions and closures.
  • Jul 21: Finish runtime without stdlib in bytecode interpreter.
  • Jul 28: Implement stdlib in bytecode interpreter. Share results.
  • Aug 5: Add ability to compile to bytecode and run from bytecode.
  • Aug 12: Add REPL and finish up project.

Im gonna have lots of fun with this project and Im sure next time a post here I'll be a changed man. Surely gonna learn a lot, Im excited about what lies ahead.