bug fix on sqlblock and documentations

README.md

@@ -233,6 +233,28 @@ SELECT * FROM table 1 UNION ALL SELECT * FROM table 2
 ## Delete Data:
 - Use a query like `DELETE FROM <table_name> [WHERE <conditions>]` to delete rows from a table that matches the conditions.
   
+## Performance Measurement 
+- Execution time can be recorded using the `stats` command described above.
+  - `stats` command without any argument will show the execution time of all queries executed so far.
+  - `stats reset` will reset the timer for total execution time printed by `stats` command above.
+  - `stats on` will show execution time for every following query until a `stats off` command is received.
+
+## MonetDB Passthrough for Hybrid Engine 
+AQuery++ supports MonetDB passthrough for hybrid engine. Simply put standard SQL queries inside a \<sql> \</sql> block. <br>
+
+Each query inside an sql block must be separated by a semicolon. And they will be sent to MonetDB directly which means they should be written in MonetDB dialect instead of AQuery dialect. Please refer to the [MonetDB documentation](https://www.monetdb.org/documentation-Sep2022/user-guide/sql-summary/) for more information.
+
+For example:
+```
+CREATE TABLE my_table (c1 INT, c2 INT, c3 STRING)
+INSERT INTO my_table VALUES(10, 20, "example"), (20, 30, "example2")
+<sql>
+INSERT INTO my_table VALUES(10, 20, "example3");
+CREATE INDEX idx1 ON my_table(c1);
+</sql>
+SELECT * FROM my_table WHERE c1 > 10
+```
+
 ## Built-in functions: 
 - `avg[s]`: average of a column. `avgs(col), avgs(w, col)` is rolling and moving average with window `w` of the column `col`.
 - `var[s]`, `stddev[s]`: [moving/rolling] **population** variance, standard deviation.
@@ -254,7 +276,7 @@ SELECT * FROM table 1 UNION ALL SELECT * FROM table 2
 - AQuery++ supports different execution engines thanks to the decoupled compiler structure.
 - Hybrid Execution Engine: decouples the query into two parts. The sql-compliant part is executed by an Embedded version of Monetdb and everything else is executed by a post-process module which is generated by AQuery++ Compiler in C++ and then compiled and executed.
 - AQuery Library: A set of header based libraries that provide column arithmetic and operations inspired by array programming languages like kdb. This library is used by C++ post-processor code which can significantly reduce the complexity of generated code, reducing compile time while maintaining the best performance. The set of libraries can also be used by UDFs as well as User modules which makes it easier for users to write simple but powerful extensions. 
-  
+
 # Roadmap
 - [x] SQL Parser -> AQuery Parser (Front End)
 - [x] AQuery-C++ Compiler (Back End)

reconstruct/ast.py

@@ -1037,6 +1037,7 @@ class insert(ast_node):
                 if isinstance(v, dict):
                     keys = v.keys()
                     v = list(v.values())
+                v = [f"'{vv}'" if type(vv) is str else vv for vv in v]
                 _vals.append(v)
             values = _vals
             
@@ -1547,7 +1548,7 @@ class passthru_sql(ast_node):
         for sql in sqls:
             sq = sql.strip(' \t\n\r;')
             if sq:
-                context.queries.append('Q' + sql + ';')
+                context.queries.append('Q' + sql.strip('\r\n\t ;') + ';')
                 lq = sq.lower()
                 if lq.startswith('select'):
                     context.queries.append('O')

server/server.cpp

@@ -207,21 +207,27 @@ void print_monetdb_results(Server* srv, const char* sep = " ", const char* end =
         uint8_t* szs = static_cast<uint8_t*>(alloca(ncols));
         std::string header_string = "";
         const char* err_msg = nullptr;
+        const size_t l_sep = strlen(sep);
+        const size_t l_end = strlen(end);
+        char* _buffer = buffer;
+
         for(uint32_t i = 0; i < ncols; ++i){
             err_msg = monetdbe_result_fetch(_res, &cols[i], i);
-            if(err_msg) { free(cols); return; }
+            if(err_msg) { goto cleanup; }
             col_data[i] = static_cast<char *>(cols[i]->data);
             prtfns[i] = monetdbe_prtfns[cols[i]->type];
             szs [i] = monetdbe_type_szs[cols[i]->type];
             header_string = header_string + cols[i]->name + sep + '|' + sep;
         }
-        const size_t l_sep = strlen(sep);
-        const size_t l_end = strlen(end);
+
+        if(l_sep > 512 || l_end > 512) {
+            puts("Error: separator or end string too long");
+            goto cleanup;
+        }
 		if (header_string.size() - l_sep - 1>= 0)
 			header_string.resize(header_string.size() - l_sep - 1);
         header_string += end + std::string(header_string.size(), '=') + end;
         fputs(header_string.c_str(), stdout);
-        char* _buffer = buffer;
         for(uint64_t i = 0; i < srv->cnt; ++i){
             for(uint32_t j = 0; j < ncols; ++j){
                 //copy the field to buf
@@ -239,8 +245,11 @@ void print_monetdb_results(Server* srv, const char* sep = " ", const char* end =
                 _buffer = buffer;
             }
         }
+        memcpy(_buffer, end, l_end);
+        _buffer += l_end;
         if (_buffer != buffer)
             fwrite(buffer, 1, _buffer - buffer, stdout);
+cleanup:        
         free(cols);
     }
 }

tests/sqlblock.a

@@ -0,0 +1,9 @@
+CREATE TABLE my_table (c1 INT, c2 INT, c3 STRING)
+INSERT INTO my_table VALUES(10, 20, "example"), (20, 30, "example2")
+<sql>
+INSERT INTO my_table VALUES(14, 24, 'example3');
+CREATE INDEX idx1 ON my_table(c1);
+SELECT * FROM my_table WHERE c1 < 15;
+
+</sql>
+SELECT * FROM my_table WHERE c1 > 15