Compiler logic

Our goal in this chapter is to develop confidence with parsing and interpreting command-line arguments, describing how a markdown compiler works, and implementing control blocks to make it possible. We are also going to introduce vectors to get a head start on the next chapter.

# How the Markdown compiler works

Now is a good time to start thinking about how we want to interact with our tiny markdown compiler. At its very core, a command-line markdown compiler should take in the name of a markdown file and then turn it into valid HTML file. Our tool is going to be very naive: it will accept only one argument (the name of the markdown file), it will expect that file to end in .md, and it will only convert first-order headings (# for the <h1> tag) and paragraphs.

By the end of this tutorial, though, you will have all the knowledge necessary to expand on this tool and make it less naive. I will leave that to you as a challenge for once you’re finished.

Until then, let’s start thinking about how we want to interact with our tool, and some of our expectations when we use it.

Calling our tool should be as simple as:

$ tinymd somefile.md

When we call the tool from the command line, we will intitiate the tool’s lifecycle. An application’s lifecycle is the set of steps it goes through from the start of execution to the end of execution.

Our compiler’s lifecycle will look something like this:

Given a call to tinymd...
  When I pass a markdown file as an argument, it should:
    1. open the file
    2. parse the file line by line into a buffer
    3. export the buffer to a new html file
  When I pass anything else OR no argument at all, it should:
    1. show the banner

As I mentioned before, this tool isn’t going to be very smart. We will expect only two possible outcomes from invoking it: either we will print the banner, because we have not passed a single argument that is a valid markdown file, or we will parse a valid markdown file, because we passed a valid markdown file as the sole argument.

Knowing all the ways our tool will be invoked helps to guide how we will develop the main logic that will call the appropriate functions based on the given arguments (or lack thereof).

The following table illustrates all the possible ways someone could invoke the tinymd tool, and what we need to plan for in terms of execution:

Essentially, any command that does not have a single valid markdown file as its sole argument will just call usage(), which outputs the banner.

Now that we know all the ways our took can be invoked, we are ready to think about how we want our tool to parse a Markdown file.

To define a tool that parses Markdown, we need to know what a Markdown file looks like. For the sake of this tutorial, we are only concerned with two types of Markdown syntax: headings and paragraphs. A heading in Markdown is denoted with a #. Paragraphs are plain text with no special characters at the start of the line.

For example, let’s say we have a markdown file called favorite_song.md with the following contents:

# My Favorite Song

My favorite song is In The Air Tonight by Phil Collins.

Here is the HTML equivalent:

This type of translation is exactly what the compiler is going to do.

Now we know how our tool will be invoked, and when it is, how we want the Markdown inside to be translated into HTML. Next, we’ll start building the command-line argument parsing logic so that we can pass a Markdown file to our tool.

# Parsing command-line arguments

The idea behind parsing command line arguments is fairly straight forward. Given an argument, perform some operation based on what that argument is. In the case of our Markdown compiler, we really only care about one argument–and we hope that it’s the name of a Markdown file.

The way Rust parses arguments is to collect them all up and store them in an iterable object. This is also how most other languages do it, too; once the collection happens, you will generally want to do a for loop or some kind of switch statement that will parse the arguments first by the number of arguments (since the number of arguments can trigger certain operations, if we wanted) and then by the value of those arguments.

Rust has a special kind of switch statement called match, which does exactly what it sounds like: it matches values with associated logic. The way we use these to parse arguments is by collecting the arguments into an iterable object, then looping through them to match their values with some associated operations.

Rust has a concept called collections, which you can probably guess are akin to vectors or lists. A collection in Rust is something that can be iterated; anything that can be looped through and acted on is generally considered a collection. When values (like command-line arguments) can be stuffed into a single object and iterated, we say that they are collected.

To do this in Rust, it’s actually very straightforward: we will create a variable to hold our arguments, which will be a vector of Strings, pull out our arguments, then collect them into a single, iterable object.

A vector in Rust is a type denoted by the keyword Vec, followed by < and >, with the variable type enclosed in the brackets.

Since we want to create a String vector–where each argument given will be a different element of the vector–we will declare it as a type Vec<String>.

Let’s create a variable called args that is a String vector, and assign it the value of our collected arguments:

// Collect all arguments in a vector
    let args: Vec<String> = std::env::args().collect();

You might recognize this chaining-style syntax from other languages. Rust has a lot of features behind the scenes that make it feel like a higher-level language, while at the same time requiring a closer-to-the-metal mindset. Here, the expression std::env::args() exposes an evironment variable of type std::env::Args, and then collects the arguments into a vector–to be used as the return value.

The .collect() part of this takes the arguments and converts them into a std::iter::Iterator<std::string::String>–which, as you can see, is a generic Iterator type (from which a Vec is derived) of type std::string::String. If you guessed that Vec<std::string::String> and Vec<String> are the same thing, then you guessed right!

The value we give args is a collection of all the arguments. To get this, Rust gives us a standard library of tools called std. From std, we are given the env, which holds specific environment variables. To get those variables, we use env’s member function args() to pull them out–then chain the function collect() to it–which collects the arguments into an iterable object.

At this point, args is an iterable vector of strings, where each element is an argument passed from the command line. If you’ve ever dealt with command-line tools before, you will know that the first element in the list is always going to be the name of the program (i.e., tinymd). Rust arrays start at zero, so the value at &args[0] is going to be tinymd.

Notice how we use the reference operator & to grab the value at args[0]. Rust has a very unique way of treating variable assignment and ownership. To explain, consider the following two pseudo statements:

1) left = right;
2) left = &right;

In most other languages, the first statement means that the value of left is identical now to the value of right. In other words, if we try to access the value of left or right later on, both will be the same value. In Rust, however, the right-hand expression is moved to the left, which means only the left-hand expression contains the value now.

If you want to keep the value in the right-hand expression whiel also capturing it elsewhere, you need to use the reference (&) operator. The second example in the snippet above (left = &right;) is how you would borrow right’s value to create the value for left. The way you borrow the value is by creating a reference to it.

So if we wanted to access the second element in the args vector (which is the name of the Markdown file we are supposed to be passing), we would use &args[1] and not args[1], since the former will provide a reference to the data and the latter will trigger a move of that value–which we do not want.

Let’s go ahead and collect up the arguments in our program. Go to your main() function and delete everything inside. Next, let’s add our args declaration:

fn main() {

    let args: Vec<String> = std::env::args().collect();

}

Great! Our main() function is now going to collect up the arguments and, in the next section, we’ll see how it matches the correct number of arguments with the appropriate operations. If the program has two arguments, that means we have the program name (element 0) and the Markdown file name (element 1). Any other number of arguments should be rejected by the program. In the next section, we will build this logic out.

# The match block

With the arguments collected into args, we now want to make sure that there are only two of them. Recall that our goal is to accept just one argument: the name of a Markdown file to parse. This means that we have two total elements in the args vector: the name of the program, and the name of the Markdown file.

We only handle invocation that includes two arguments, so the length of the args vector should be 2. To check this, we’ll use a match block to match the length of the vector with either 2 (the length we want) or not 2:

fn main() {

    let args: Vec<String> = std::env::args().collect();

    match args.len() {
      // ...
    }

}

Here you can see that we grab the length of the vector by calling .len() on it. The result is an integer value. To match values in this match block, we declare them like this: left => right. If args.len() equals 2, then we want to call parse_markdown_file():

match args.len() {
  2 => parse_markdown_file(),

You’ll notice the comma at the end; the match rule statements (left=>right) are comma separated. If we wanted to put one before, and call usage() if no file was passed, we might do something like this:

match args.len() {
  1 => usage(),
  2 => parse_markdown_file(),
  // etc. etc. 

In this case, checking for one argument or more than two arguments is redundant; instead of checking twice for a number of arguments not equal to 2, we can use the default case for match, which is a _ (underscore):

match args.len() {
  2 => parse_markdown_file(),
  _ => usage()
}

The default match case (_) will trigger if no other match case triggers.

The match cases don’t have to be a call to a single function. They can also be a block themselves:

match args.len() {
  2 => parse_markdown_file(),
  _ => { 
    println!("[ ERROR ] Invalid invocation (you done goofed!)");
    usage();
  }
}

At this point, our main function can check whether there are two arguments–the name of the program (arg 1) and the Markdown file to be parsed (arg 2), and call the appropriate function (parse_markdown_file()) if there are only two arguments.

# Passing arguments to functions

The last thing we need to do in the main() function is to pass the second argument into the function that parses the markdown file–parse_markdown_file(). To do this, we need to change the declaration of parse_markdown_file() to accept a single argument: a string slice that is the filename to parse.

To add an argument to a function, you declare it the same way you would a regular variable, except this time you can omit the let. Let’s modify the function to accept a string slice argument named _filename:

fn parse_markdown_file(_filename: &str) {

}

I’m using an underscore (_) here in the filename variable to remind me that this is coming from a function parameter. Feel free to name it whatever you want.

Let’s also go ahead and put some placeholder text in there to help us see when this function is called:

fn parse_markdown_file(_filename: &str) {
  print_short_banner();
  println!("[ INFO ] Trying to parse {}...", _filename);
}

Here I have it outputting the short banner and then the informational message.

Now, if we were to invoke the tool with cargo run -q afile.md, it should tell is [ INFO ] Trying to parse afile.md.... We will continue fleshing out this function in the next chapter.

Finally, we have to actually pass the filename to the function, back down in the match block we had finished earlier. When args.len() is 2, we want to pass the second element in the args vector to parse_markdown_file(). To do that, we will simply pass it as a string slice (since the function accepts a &str as the argument):

// ...
match args.len() {
  2 => parse_markdown_file(&args[1]),
  //...
}

Notice how we access the 2nd element of args the same way we might do it in other languages. Also note that we are using the reference operator here; we don’t want to pass args[1] in, since that would move the value into the function–causing args[1] to be null. Remember: in Rust, assignment is a move, not a copy!

Go ahead and build and run your project: cargo run -q. Here’s what mine says:

$ cargo run -q
[ ERROR ] You forgot to specify the markdown file to parse!
tinymd (v0.1.0), A tiny markdown compiler based on Jesse's tutorials.
Written by: Jesse Lawson <[email protected]>
Homepage: https://jesselawson.org/rust
Usage: tinymd <somefile>.md

Next, let’s pass it the name of a fake file to see if we setup our logic correctly:

$ cargo run -q test.md
tinymd (v0.1.0), A tiny markdown compiler based on Jesse's tutorials.
[ INFO ] Trying to parse test.md...

In this chapter, we learned about how a markdown compiler works, and we started working more closely with strings and string slices while piecing together functionality with conditional logic.

Next, we’ll implement our Markdown compiler logic and open a file, read it line-by-line, translate it into HTML, and write the HTML to a new file.

It’s going to be fun!

“If you can’t fly, then run, if you can’t run then walk, if you can’t walk then crawl, but whatever you do you have to keep moving forward.”

— Martin Luther King, Jr.

📦 All the code up to this point is available on GitHub.