Merge branch 'main' of https://github.com/cozis/Noja
This commit is contained in:
Francesco Cozzuto
@@ -0,0 +1,21 @@
|
||||
# The Noja language
|
||||
|
||||
## Introduction
|
||||
|
||||
This language was written as a personal study of how interpreters and compilers work. For this reason, the language is very basic. One of the main inspirations was the CPython's source code since it's extremely readable and has a very simple and clean architecture.
|
||||
|
||||
## Implementation overview
|
||||
|
||||
The interpreter works by compiling the provided source to a bytecode
|
||||
format and executing it. The bytecode is very high level since it does things like:
|
||||
|
||||
- explicitly referring to variables by name.
|
||||
|
||||
- treating values as atomic things: from the perspective of the
|
||||
bytecode, a list and an integer occupy the same space on the
|
||||
stack, which is 1.
|
||||
|
||||
- referring to instructions by their index.
|
||||
|
||||
All values (objects) are allocated on a garbage-collected heap. All variables are simply references to these objects. The garbage collection algorithm is a copy-and-compact. It behaves as a bump-pointer allocator until there is space left, and when space runs out, it creates a new heap, copies all of the alive object into it, calls the destructors of the dead objects and frees the old one.
|
||||
|
||||
@@ -0,0 +1,320 @@
|
||||
|
||||
# The Noja language
|
||||
|
||||
This file was intended for people who already program in other high level languages (such as Python, Javascript, Ruby) and don't need to be introduced to basic programming concepts (variables, expressions and branches). This way, there is more space for the comparison of the language's features with the mainstream languages.
|
||||
|
||||
## Table of contents
|
||||
3. [The first program](#the-first-program)
|
||||
4. [Expressions](#expressions)
|
||||
5. [Branches](#branches)
|
||||
6. [Loops](#loops)
|
||||
7. [Functions](#functions)
|
||||
|
||||
## The first program
|
||||
|
||||
The sintax is similar to Python's but is more C-like. A Noja script
|
||||
is a list of statements that can be of multiple kinds:
|
||||
|
||||
- function declaractions
|
||||
- expressions
|
||||
- if-else branches
|
||||
- while loops
|
||||
- do-while loops
|
||||
- return statements
|
||||
- composit statements
|
||||
|
||||
In general, unless it's inside strings, whitespace is ignored and
|
||||
comments start with the `#` character.
|
||||
|
||||
The most basic yet interesting program is:
|
||||
|
||||
```py
|
||||
print('Hello, world!\n');
|
||||
```
|
||||
|
||||
as in other languages, this kind of statement is an expression.
|
||||
Expression statements require a ';' to determine their end.
|
||||
|
||||
The print function can take any number of arguments of any type
|
||||
and doesn't add any spaces or newlines to the output.
|
||||
|
||||
```py
|
||||
print(1, 2, 3, true, '\n');
|
||||
```
|
||||
|
||||
## Expressions
|
||||
|
||||
You can set variables without declaring them first by using the
|
||||
assignment operator:
|
||||
```py
|
||||
a = 5;
|
||||
```
|
||||
which is similar to Python's assignment, but is a little different.
|
||||
In this language, assignments are considered as expressions, in fact
|
||||
you can do things like
|
||||
```py
|
||||
a = (b = 1) + 1;
|
||||
|
||||
# The value resulting from an assignment is the assigned value.
|
||||
# After this expression, b's value is 1 and a's value is 2.
|
||||
|
||||
print('b = ', b, '\n'); # b = 1
|
||||
print('a = ', a, '\n'); # a = 2
|
||||
```
|
||||
all of the basic arithmetic operators are available:
|
||||
```py
|
||||
x = 1 + 1;
|
||||
y = 1 - 2;
|
||||
z = 3 * 2;
|
||||
w = 10 / 3;
|
||||
|
||||
print('x = ', x, '\n'); # x = 2
|
||||
print('y = ', y, '\n'); # y = -1
|
||||
print('z = ', z, '\n'); # z = 6
|
||||
print('w = ', w, '\n'); # w = 3
|
||||
```
|
||||
Note how the division returns the rounded down version of the result.
|
||||
This is because the division was performed on integers. By making one
|
||||
of the operands a floating point value, also a floating point result
|
||||
is returned:
|
||||
```py
|
||||
w = 10 / 3.0;
|
||||
|
||||
print('w = ', w, '\n');
|
||||
```
|
||||
Arithmetic operators are only available for numeric types of objects.
|
||||
If you try to apply them on other kinds of types, you get a runtime
|
||||
error.
|
||||
|
||||
Relational operators are also available:
|
||||
```py
|
||||
print(1 < 2, '\n'); # true
|
||||
print(1 > 2, '\n'); # false
|
||||
|
||||
print(1 >= 0, '\n'); # true
|
||||
print(1 <= 0, '\n'); # false
|
||||
|
||||
print(1 == 5, '\n'); # false
|
||||
print(6 == 6, '\n'); # true
|
||||
|
||||
print(1 != 5, '\n'); # true
|
||||
print(6 != 6, '\n'); # false
|
||||
```
|
||||
The equal and not equal operators are available on every type of object,
|
||||
while the others are only available for numeric types.
|
||||
|
||||
## Branches
|
||||
|
||||
It's possible to make the execution of a statement optional, based on the
|
||||
result of an expression. Like in other languages, you do this using if-else
|
||||
statements:
|
||||
|
||||
```py
|
||||
if 1 < 2:
|
||||
print('Took the branch!\n'); # This is executed!
|
||||
|
||||
if 1 > 2:
|
||||
print('Didn\'t take the branch\n'); # This isn't!
|
||||
```
|
||||
..or you can specify an alternative branch, which is executed when the
|
||||
condition isn't true:
|
||||
|
||||
```py
|
||||
if 1 > 2:
|
||||
print('Not executed..\n');
|
||||
else
|
||||
print('Executed!\n');
|
||||
```
|
||||
You can have multiple statements inside a branch by having them inside a
|
||||
compound statement. Compound statements are statement lists wrapped inside
|
||||
curly brackets, like this:
|
||||
|
||||
```py
|
||||
{ print('Hello from a '); print('compound statement!\n'); }
|
||||
```
|
||||
This way they count as one statement.
|
||||
|
||||
```py
|
||||
if 1 == 1:
|
||||
{
|
||||
print('Executed\n');
|
||||
print('Also executed\n');
|
||||
}
|
||||
```
|
||||
|
||||
Variables defined inside an if-else statement's branch are defined
|
||||
in the parent's context. This implies that variables may or may not
|
||||
be defined when you access them, based on which branch is taken.
|
||||
|
||||
```py
|
||||
a = 1;
|
||||
|
||||
if a < 2:
|
||||
x = 100;
|
||||
|
||||
# Now x is defined, but if "a" were to be higher or equal to 2, it
|
||||
# wouldn't be defined and the runtime would return an error.
|
||||
```
|
||||
|
||||
## Loops
|
||||
|
||||
Looping constructs are available in the form of while and do-while
|
||||
statements. The while statement checks the condition before each
|
||||
iteration:
|
||||
|
||||
```py
|
||||
i = 0;
|
||||
while i < 10:
|
||||
i = i + 1;
|
||||
```
|
||||
|
||||
This loop runs for 10 times. As for the if-else statement, a single
|
||||
statement is expected as the body of the while statement. You can
|
||||
provide it a compound statement tho.
|
||||
|
||||
```py
|
||||
i = 0;
|
||||
while i < 10:
|
||||
{
|
||||
print('While iteration no. ', i, '\n');
|
||||
i = i + 1;
|
||||
}
|
||||
```
|
||||
|
||||
The do-while statement checks the condition at the end of each
|
||||
iteration. This means that at least one iteration is performed!
|
||||
|
||||
```py
|
||||
i = 0;
|
||||
do
|
||||
{
|
||||
print('Do-while iteration no. ', i, '\n');
|
||||
i = i + 1;
|
||||
}
|
||||
while i < 10;
|
||||
```
|
||||
|
||||
Like for if-else statements, variables defined inside the loop
|
||||
body are shared with the parent's context.
|
||||
|
||||
## Functions
|
||||
|
||||
Functions can be defined using the following syntax:
|
||||
|
||||
```py
|
||||
# Define it
|
||||
fun say_hello_to(name)
|
||||
print('Hello, ', name, '!\n\n');
|
||||
|
||||
# .. and then call it.
|
||||
say_hello_to('Francesco');
|
||||
```
|
||||
|
||||
Functions can have an arbitrary amount of arguments. If the function is
|
||||
called with more arguments than it expected, the extra values are thrown
|
||||
away. If the function is called with less arguments than it expected,
|
||||
the argument set if filled up with none values.
|
||||
|
||||
```py
|
||||
fun test_func(a, b, c)
|
||||
{
|
||||
print('a = ', a, '\n');
|
||||
print('b = ', b, '\n');
|
||||
print('c = ', c, '\n\n');
|
||||
}
|
||||
|
||||
test_func();
|
||||
# a = none
|
||||
# b = none
|
||||
# c = none
|
||||
|
||||
test_func(1, 2);
|
||||
# a = 1
|
||||
# b = 2
|
||||
# c = none
|
||||
|
||||
test_func(1, 2, 3);
|
||||
# a = 1
|
||||
# b = 2
|
||||
# c = 3
|
||||
|
||||
test_func(1, 2, 3, 4);
|
||||
# a = 1
|
||||
# b = 2
|
||||
# c = 3
|
||||
```
|
||||
|
||||
Functions are actually variables like the ones that are be defined using
|
||||
the assignment operator. In fact, you can reassign them new values if you
|
||||
want.
|
||||
|
||||
```py
|
||||
test_func = 5;
|
||||
|
||||
# The following line, if executed, returns an error because the test_func
|
||||
# identifier is now associated to 5, which is not a function.
|
||||
|
||||
test_func(); # Error!!
|
||||
```
|
||||
|
||||
Functions can return values exactly like in other languages:
|
||||
|
||||
```py
|
||||
fun multiply(x, y)
|
||||
return x * y;
|
||||
|
||||
p = 4;
|
||||
q = 7;
|
||||
r = multiply(p, q);
|
||||
|
||||
print(p, ' * ', q, ' = ', r, '\n');
|
||||
```
|
||||
|
||||
If the function doesn't return any values, then the `none` value is returned.
|
||||
As an example, the `print` function always returns `none`
|
||||
|
||||
```py
|
||||
print(print()); # none
|
||||
```
|
||||
|
||||
Functions are always "pure", in the sense that the only values that the
|
||||
function body can access are the ones provided as arguments. Usually in
|
||||
other languages, functions can access the global scope and the parent
|
||||
scope (closures). There's no such mechanism in this language (at the
|
||||
moment).
|
||||
|
||||
The only exception is made for the "built in" variables, which are
|
||||
provided by the runtime of the language and can't be modified by the
|
||||
user. The print function is one of these variables. One may override
|
||||
these variables but the effect only lasts for the lifetame of the
|
||||
context local to the assignment.
|
||||
|
||||
```py
|
||||
# Overwrite the print variable inside the global
|
||||
# scope..
|
||||
print = 5;
|
||||
|
||||
# The reference to the print function is lost
|
||||
# withing this scope.
|
||||
|
||||
fun test()
|
||||
{
|
||||
# If the previous assignment were to overwrite the
|
||||
# print function globally, the next statement would
|
||||
# fail because the value 5 isn't a function. But
|
||||
# it doesn't fail!
|
||||
print('Not overwritten here!\n');
|
||||
}
|
||||
|
||||
test();
|
||||
|
||||
# We can take the reference to the print function
|
||||
# by taking it from a function!
|
||||
|
||||
fun get_print_back()
|
||||
return print;
|
||||
|
||||
print = get_print_back();
|
||||
|
||||
print('Hei! Print is back!\n');
|
||||
```
|
||||
@@ -1,71 +0,0 @@
|
||||
|
||||
# ------------------------------------------------------------------------- #
|
||||
# --- Introduction -------------------------------------------------------- #
|
||||
#
|
||||
# This language was written as a personal study of how interpreters
|
||||
# and compilers work. For this reason, the language is very basic.
|
||||
# One of the main inspirations was the CPython's source code since
|
||||
# it's extremely readable and has a very simple and clean architecture.
|
||||
#
|
||||
# This file was intended for people who already program in other
|
||||
# high level languages (such as Python, Javascript, Ruby) and don't
|
||||
# need to be introduced to basic programming concepts (variables,
|
||||
# expressions and branches). This way, there is more space for the
|
||||
# comparison of the language's features with the mainstream languages.
|
||||
#
|
||||
# ------------------------------------------------------------------------- #
|
||||
# --- Implementation ------------------------------------------------------ #
|
||||
#
|
||||
# The interpreter works by compiling the provided source to a bytecode
|
||||
# format and executing it. The bytecode is very high level since it
|
||||
# does things like:
|
||||
#
|
||||
# - explicitly referring to variables by name.
|
||||
#
|
||||
# - treating values as atomic things: from the perspective of the
|
||||
# bytecode, a list and an integer occupy the same space on the
|
||||
# stack, which is 1.
|
||||
#
|
||||
# - referring to instructions by their index.
|
||||
#
|
||||
# For example, by compiling the following snippet
|
||||
|
||||
define = true;
|
||||
|
||||
if define:
|
||||
a = 33;
|
||||
|
||||
print(a, '\n');
|
||||
|
||||
# one would obtain the following bytecode:
|
||||
#
|
||||
# 0: PUSHTRU
|
||||
# 1: ASS "define"
|
||||
# 2: POP 1
|
||||
# 3: PUSHVAR "define"
|
||||
# 4: JUMPIFNOTANDPOP 8
|
||||
# 5: PUSHINT 33
|
||||
# 6: ASS "a"
|
||||
# 7: POP 1
|
||||
# 8: PUSHSTR "\n"
|
||||
# 9: PUSHVAR "a"
|
||||
# 10: PUSHVAR "print"
|
||||
# 11: CALL 2
|
||||
# 12: POP 1
|
||||
# 13: RETURN
|
||||
#
|
||||
# as you can see, there are instructions like ASS and PUSHVAR that
|
||||
# assign to and read from variables by specifying names, and jumps
|
||||
# that refer to other points of the "executable" by specifying indices
|
||||
# (like JUMPIFNOTANDPOP) instead of raw addresses.
|
||||
#
|
||||
# All values (objects) are allocated on a garbage-collected heap.
|
||||
# For this reason all variables are simply references to these objects.
|
||||
# The garbage collection algorithm is a copy-and-compact one. It
|
||||
# behaves as a bump-pointer allocator until there is space left,
|
||||
# and when space runs out, it creates a new heap, copies all of the
|
||||
# alive object into it, calls the destructors of the dead objects
|
||||
# and frees the old one.
|
||||
#
|
||||
# ------------------------------------------------------------------------- #
|
||||
# ------------------------------------------------------------------------- #
|
||||
@@ -1,112 +0,0 @@
|
||||
|
||||
# ------------------------------------------------------------------------- #
|
||||
# --- The first program --------------------------------------------------- #
|
||||
#
|
||||
# The sintax is similar to Python's but is more C-like. A Noja script
|
||||
# is a list of statements that can be of multiple kinds:
|
||||
#
|
||||
# - function declaractions
|
||||
# - expressions
|
||||
# - if-else branches
|
||||
# - while loops
|
||||
# - do-while loops
|
||||
# - return statements
|
||||
# - composit statements
|
||||
#
|
||||
# In general, unless it's inside strings, whitespace is ignored and
|
||||
# comments start with the # character.
|
||||
#
|
||||
# The most basic yet interesting program is:
|
||||
|
||||
print('Hello, world!\n');
|
||||
|
||||
# as in other languages, this kind of statement is an expression.
|
||||
# Expression statements require a ';' to determine their end.
|
||||
#
|
||||
# The print function can take any number of arguments of any type
|
||||
# and doesn't add any spaces or newlines to the output.
|
||||
|
||||
print(1, 2, 3, '\n');
|
||||
|
||||
#
|
||||
# ------------------------------------------------------------------------- #
|
||||
# --- Variables and expressions ------------------------------------------- #
|
||||
#
|
||||
# You can set variables without declaring them first by using the
|
||||
# assignment operator:
|
||||
|
||||
a = 5;
|
||||
|
||||
# which is similar to Python's assignment, but is a little different.
|
||||
# In this language, assignments are considered as expressions, in fact
|
||||
# you can do things like
|
||||
|
||||
a = (b = 1) + 1;
|
||||
|
||||
# The value resulting from an assignment is the assigned value.
|
||||
# After this expression, b's value is 1 and a's value is 2.
|
||||
|
||||
print('b = ', b, '\n'); # b = 1
|
||||
print('a = ', a, '\n'); # a = 2
|
||||
|
||||
# all of the basic arithmetic operators are available:
|
||||
|
||||
x = 1 + 1;
|
||||
y = 1 - 2;
|
||||
z = 3 * 2;
|
||||
w = 10 / 3;
|
||||
|
||||
print('x = ', x, '\n'); # x = 2
|
||||
print('y = ', y, '\n'); # y = -1
|
||||
print('z = ', z, '\n'); # z = 6
|
||||
print('w = ', w, '\n'); # w = 3
|
||||
|
||||
# Note how the division returns the rounded down version of the result.
|
||||
# This is because the division was performed on integers. By making one
|
||||
# of the operands a floating point value, also a floating point result
|
||||
# is returned:
|
||||
|
||||
w = 10 / 3.0;
|
||||
|
||||
print('w = ', w, '\n');
|
||||
|
||||
# Arithmetic operators are only available for numeric types of objects.
|
||||
# If you try to apply them on other kinds of types, you get a runtime
|
||||
# error:
|
||||
|
||||
# (Uncomment the following line and run this file to get the error)
|
||||
# p = 5 + 'hello';
|
||||
|
||||
# And relational operators are also available:
|
||||
|
||||
print(1 < 2, '\n'); # true
|
||||
print(1 > 2, '\n'); # false
|
||||
|
||||
print(1 >= 0, '\n'); # true
|
||||
print(1 <= 0, '\n'); # false
|
||||
|
||||
print(1 == 5, '\n'); # false
|
||||
print(6 == 6, '\n'); # true
|
||||
|
||||
print(1 != 5, '\n'); # true
|
||||
print(6 != 6, '\n'); # false
|
||||
|
||||
# The equal and not equal operators are available on every type of object,
|
||||
# while the others are only available for numeric types.
|
||||
#
|
||||
# ------------------------------------------------------------------------- #
|
||||
# --- The boolean type ---------------------------------------------------- #
|
||||
#
|
||||
|
||||
# TODO
|
||||
|
||||
#
|
||||
# ------------------------------------------------------------------------- #
|
||||
# --- The none value ------------------------------------------------------ #
|
||||
#
|
||||
|
||||
# TODO
|
||||
|
||||
#
|
||||
# ------------------------------------------------------------------------- #
|
||||
# ------------------------------------------------------------------------- #
|
||||
@@ -1,50 +0,0 @@
|
||||
|
||||
# ------------------------------------------------------------------------- #
|
||||
# --- Branches ------------------------------------------------------------ #
|
||||
#
|
||||
# It's possible to make the execution of a statement optional, based on the
|
||||
# result of an expression. Like in other languages, you do this using if-else
|
||||
# statements:
|
||||
|
||||
if 1 < 2:
|
||||
print('Took the branch!\n'); # This is executed!
|
||||
|
||||
if 1 > 2:
|
||||
print('Didn\'t take the branch\n'); # This isn't!
|
||||
|
||||
# or you can specify an alternative branch, which is executed when the
|
||||
# condition isn't true:
|
||||
|
||||
if 1 > 2:
|
||||
print('Not executed..\n');
|
||||
else
|
||||
print('Executed!\n');
|
||||
|
||||
# You can have multiple statements inside a branch by having them inside a
|
||||
# compound statement. Compound statements are statement lists wrapped inside
|
||||
# curly brackets, like this:
|
||||
|
||||
{ print('Hello from a '); print('compound statement!\n'); }
|
||||
|
||||
# This way they count as one statement.
|
||||
|
||||
if 1 == 1:
|
||||
{
|
||||
print('Executed\n');
|
||||
print('Also executed\n');
|
||||
}
|
||||
|
||||
# Variables defined inside an if-else statement's branch are defined
|
||||
# in the parent's context. This implies that variables may or may not
|
||||
# be defined when you access them, based on which branch is taken.
|
||||
|
||||
a = 1;
|
||||
|
||||
if a < 2:
|
||||
x = 100;
|
||||
|
||||
# Now x is defined, but if "a" were to be higher or equal to 2, it
|
||||
# wouldn't be defined and the runtime would return an error.
|
||||
#
|
||||
# ------------------------------------------------------------------------- #
|
||||
# ------------------------------------------------------------------------- #
|
||||
@@ -1,39 +0,0 @@
|
||||
|
||||
# ------------------------------------------------------------------------- #
|
||||
# --- Loops --------------------------------------------------------------- #
|
||||
#
|
||||
# Looping constructs are available in the form of while and do-while
|
||||
# statements. The while statement checks the condition before each
|
||||
# iteration:
|
||||
|
||||
i = 0;
|
||||
while i < 10:
|
||||
i = i + 1;
|
||||
|
||||
# This loop runs for 10 times. As for the if-else statement, a single
|
||||
# statement is expected as the body of the while statement. You can
|
||||
# provide it a compound statement tho.
|
||||
|
||||
i = 0;
|
||||
while i < 10:
|
||||
{
|
||||
print('While iteration no. ', i, '\n');
|
||||
i = i + 1;
|
||||
}
|
||||
|
||||
# The do-while statement checks the condition at the end of each
|
||||
# iteration. This means that at least one iteration is performed!
|
||||
|
||||
i = 0;
|
||||
do
|
||||
{
|
||||
print('Do-while iteration no. ', i, '\n');
|
||||
i = i + 1;
|
||||
}
|
||||
while i < 10;
|
||||
|
||||
# Like for if-else statements, variables defined inside the loop
|
||||
# body are shared with the parent's context.
|
||||
#
|
||||
# ------------------------------------------------------------------------- #
|
||||
# ------------------------------------------------------------------------- #
|
||||
@@ -24,3 +24,5 @@ while i < 3;
|
||||
|
||||
# ------------------------------------- #
|
||||
# ------------------------------------- #
|
||||
|
||||
print(count(1, 2));
|
||||
@@ -0,0 +1,17 @@
|
||||
|
||||
The source files directly contained by this folder expose to the user through the command line the functionalities implemented inside the subfolders. The `main.c` file implements the entry of the program and, based on the options provided by the user, executes, parses or disassembles the noja source code. In `debug.c` is implemented the callback that `main.c` provides to the runtime if the user asks for an execution in debug mode that makes it possible to expose the internal state of the interpreter during the execution.
|
||||
|
||||
* The `utils` folder contains the implementations of general purpose data structures and definitions that are useful through all of the codebase. Some of the more used data structures implemented in it are:
|
||||
|
||||
* `BPAlloc`: A bump-pointer allocator. It's useful during the compilation phase because lots of small objects need to be allocated and then freed at the same time when the final executable is produced.
|
||||
* `Error`: A structure useful to report errors to function callers.
|
||||
* `Source`: A string object that is used in place of raw strings to move source code around.
|
||||
|
||||
* The `common` folder implements the `Executable` data structure, which contains the result of a source's compilation. It can be though about as an array of bytecode instructions that can be directly executed.
|
||||
|
||||
* The `compiler` folder implements the compiler of the interpreter. The main routine that is exported from here is `compile`, which transforms a `Source` into an `Executable`. Other functions are exported like `serialize` that transforms an `AST` to a JSON string. This subfolder is the only part of the codebase that should be able to access the `AST` nodes.
|
||||
|
||||
* The `objects` folder implements the object model. In the context of this language, an object is a virtual class that implements a given set of methods. This folder exports functions that transform "raw" data types into objects, functions that do the inverse transformation and functions that trigger the virtual methods. This folder also contains the implementation of the heap and the garbage collector that needs to be tightly coupled with the object model.
|
||||
|
||||
* The `runtime` folder implements the routines that run the `Executable`. It depends heavily on `objects`. It basically iterates over the executable's bytecode and applies the described actions to the objects.
|
||||
It also implements a couple of objects that can't be implemented inside `objects` because they depend on the runtime internals.
|
||||
@@ -1,3 +1,21 @@
|
||||
|
||||
/* WHAT IS THIS FILE?
|
||||
**
|
||||
** This file implements the routines that transform the AST
|
||||
** into a list of bytecodes. The functionalities of this file
|
||||
** are exposed through the `compile` function, that takes as
|
||||
** input an `AST` and outputs an `Executable`.
|
||||
**
|
||||
** The function that does the heavy lifting is `emit_instr_for_node`
|
||||
** which walks the tree and writes instructions to the `ExeBuilder`.
|
||||
**
|
||||
** Some semantic errors are catched at this phase, in which
|
||||
** case, they are reported by filling out the `error` structure
|
||||
** and aborting. It's also possible that the compilation fails
|
||||
** bacause of internal errors (which usually means "out of memory").
|
||||
**
|
||||
*/
|
||||
|
||||
#include <assert.h>
|
||||
#include <setjmp.h>
|
||||
#include <stdlib.h>
|
||||
@@ -490,6 +508,26 @@ static _Bool emit_instr_for_node(ExeBuilder *exeb, Node *node, Error *error)
|
||||
return 0;
|
||||
}
|
||||
|
||||
/* Symbol: compile
|
||||
*
|
||||
* Serializes an AST into bytecode format.
|
||||
*
|
||||
*
|
||||
* Arguments:
|
||||
*
|
||||
* ast: The AST to be serialized.
|
||||
* alloc: The allocator that will be used to get new
|
||||
* memory. (optional)
|
||||
* error: Error information structure that is filled out if
|
||||
* an error occurres.
|
||||
*
|
||||
*
|
||||
* Returns:
|
||||
* A pointer to an `Executable` that is the object that
|
||||
* contains the bytecode. If an error occurres, NULL is
|
||||
* returned and the `error` structure is filled out.
|
||||
*
|
||||
*/
|
||||
Executable *compile(AST *ast, BPAlloc *alloc, Error *error)
|
||||
{
|
||||
assert(ast != NULL);
|
||||
|
||||
+103
-11
@@ -1,3 +1,31 @@
|
||||
|
||||
/* WHAT IS THIS FILE?
|
||||
**
|
||||
** This file implements the parser of the language, that transforms
|
||||
** `Source` objects into `AST` objects. The functionalities of this
|
||||
** file are exposed throigh the `parse` function.
|
||||
**
|
||||
** It's mainly composed by routines that can each parse specific
|
||||
** parts of a noja source string. For example, `parse_expression`
|
||||
** parses expressions and `parse_while_statement` parses while statements.
|
||||
** These functions call each other recursively to parse the source
|
||||
** and build the abstract syntax tree (AST) that can be then compiled
|
||||
** into bytecode. If at any point the parsing fails because of an
|
||||
** external or internal error, then the error is reported and the parsing
|
||||
** is aborted.
|
||||
**
|
||||
** Since the nodes of the AST always have the same lifetime (they're
|
||||
** allocated at the same time and die all together), the allocator
|
||||
** scheme of choise is a bump-pointer allocator. This way each of the
|
||||
** parsing routines can allocate memory if it need it but doesn't need
|
||||
** to free it if an error occurres.
|
||||
**
|
||||
** The parsing routines don't operate directly on the source text, but
|
||||
** on the tokenized version of it. Before parsing a linked list of
|
||||
** tokens is produced through the `tokenize` function.
|
||||
**
|
||||
*/
|
||||
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
#include <assert.h>
|
||||
@@ -81,21 +109,36 @@ static inline _Bool isoper(char c)
|
||||
c == '=';
|
||||
}
|
||||
|
||||
AST *parse(Source *src, BPAlloc *alloc, Error *error)
|
||||
/* Symbol: tokenize
|
||||
*
|
||||
* Build a list of tokens that represents the
|
||||
* provided source code.
|
||||
*
|
||||
*
|
||||
* Arguments:
|
||||
*
|
||||
* src: The source code to be tokenized.
|
||||
* alloc: The allocator that will contain all of the
|
||||
* generated tokens.
|
||||
* error: Error information structure that is filled out if
|
||||
* an error occurres.
|
||||
*
|
||||
* None of the arguments are optional.
|
||||
*
|
||||
*
|
||||
* Returns:
|
||||
* A pointer to the first node of a linked list of tokens.
|
||||
* If an error occurres, NULL is returned and the `error`
|
||||
* structure is filled out.
|
||||
*
|
||||
*/
|
||||
static Token *tokenize(Source *src, BPAlloc *alloc, Error *error)
|
||||
{
|
||||
assert(src != NULL);
|
||||
assert(alloc != NULL);
|
||||
|
||||
const char *str = Source_GetBody(src);
|
||||
int len = Source_GetSize(src);
|
||||
assert(str != NULL);
|
||||
assert(len >= 0);
|
||||
|
||||
AST *ast = BPAlloc_Malloc(alloc, sizeof(AST));
|
||||
|
||||
if(ast == NULL)
|
||||
return NULL;
|
||||
|
||||
Token *head = NULL,
|
||||
*tail = NULL;
|
||||
int i = 0;
|
||||
@@ -343,9 +386,58 @@ AST *parse(Source *src, BPAlloc *alloc, Error *error)
|
||||
tail = tok;
|
||||
}
|
||||
|
||||
return head;
|
||||
}
|
||||
|
||||
/* Symbol: parse
|
||||
*
|
||||
* Build an AST that represents the provided source code.
|
||||
*
|
||||
*
|
||||
* Arguments:
|
||||
*
|
||||
* src: The source code to be parsed.
|
||||
* alloc: The allocator that will contain all of the garbage
|
||||
* the function needs and the final AST.
|
||||
* error: Error information structure that is filled out if
|
||||
* an error occurres.
|
||||
*
|
||||
* None of the arguments are optional.
|
||||
*
|
||||
*
|
||||
* Returns:
|
||||
*
|
||||
* A pointer to the generated AST object. The AST object and
|
||||
* all of the stuff that's referenced by it will be stored
|
||||
* onto the provided allocator, therefore the AST will have
|
||||
* the same lifetime of the allocator. If an error occurres,
|
||||
* NULL is returned and the `error` structure is filled out.
|
||||
*
|
||||
* Notes:
|
||||
* The AST structure holds a weak reference to the source
|
||||
* object, therefore it will be invalidated if the source
|
||||
* is freed before the AST.
|
||||
*
|
||||
*/
|
||||
AST *parse(Source *src, BPAlloc *alloc, Error *error)
|
||||
{
|
||||
assert(src != NULL);
|
||||
assert(alloc != NULL);
|
||||
assert(error != NULL);
|
||||
|
||||
AST *ast = BPAlloc_Malloc(alloc, sizeof(AST));
|
||||
|
||||
if(ast == NULL)
|
||||
return NULL;
|
||||
|
||||
Token *tokens = tokenize(src, alloc, error);
|
||||
|
||||
if(tokens == NULL)
|
||||
return NULL;
|
||||
|
||||
Context ctx;
|
||||
ctx.src = str;
|
||||
ctx.token = head;
|
||||
ctx.src = Source_GetBody(src);
|
||||
ctx.token = tokens;
|
||||
ctx.alloc = alloc;
|
||||
ctx.error = error;
|
||||
|
||||
|
||||
@@ -1,3 +1,22 @@
|
||||
|
||||
/* WHAT IS THIS FILE?
|
||||
**
|
||||
** This file implements the routines that serialize the AST
|
||||
** into JSON format. The JSON manipulation is handled by the
|
||||
** third party library xJSON (written by me, still).
|
||||
**
|
||||
** The serialization functionality is exposed through the
|
||||
** `serialize` function, that takes as an `AST` as argument
|
||||
** and outputs a string of valid JSON. Therefore the xJSON
|
||||
** dependency isn't exposed to the caller and can be regarded
|
||||
** as an implementation detail.
|
||||
**
|
||||
** The way the serialization occurres is by converting the
|
||||
** AST's representation native to the compiler to one native
|
||||
** to xJSON, an then calling xj_encode on the converted AST.
|
||||
**
|
||||
*/
|
||||
|
||||
#include <assert.h>
|
||||
#include <xjson.h>
|
||||
#include "serialize.h"
|
||||
|
||||
@@ -116,8 +116,6 @@ static Object *select(Object *self, Object *key, Heap *heap, Error *error)
|
||||
// Not the one we wanted.
|
||||
}
|
||||
|
||||
int old_i = i;
|
||||
|
||||
pert >>= 5;
|
||||
i = (i * 5 + pert + 1) & mask;
|
||||
}
|
||||
@@ -249,8 +247,6 @@ static _Bool insert(Object *self, Object *key, Object *val, Heap *heap, Error *e
|
||||
// Collision.
|
||||
}
|
||||
|
||||
int old_i = i;
|
||||
|
||||
pert >>= 5;
|
||||
i = (i * 5 + pert + 1) & mask;
|
||||
}
|
||||
|
||||
@@ -11,6 +11,8 @@ static Object *bin_print(Runtime *runtime, Object **argv, unsigned int argc, Err
|
||||
|
||||
static Object *bin_count(Runtime *runtime, Object **argv, unsigned int argc, Error *error)
|
||||
{
|
||||
assert(argc == 1);
|
||||
|
||||
int n = Object_Count(argv[0], error);
|
||||
|
||||
if(error->occurred)
|
||||
|
||||
@@ -55,7 +55,8 @@ static Object *call(Object *self, Object **argv, unsigned int argc, Heap *heap,
|
||||
{
|
||||
// Some arguments are missing.
|
||||
argv2 = malloc(sizeof(Object*) * expected_argc);
|
||||
|
||||
argc2 = expected_argc;
|
||||
|
||||
if(argv2 == NULL)
|
||||
{
|
||||
Error_Report(error, 1, "No memory");
|
||||
|
||||
Reference in New Issue
Block a user