Bill
a464029d14
|
2 years ago | |
---|---|---|
aquery_parser | 2 years ago | |
data | 2 years ago | |
docs | 2 years ago | |
engine | 2 years ago | |
lib | 2 years ago | |
monetdb | 2 years ago | |
msc-plugin | 2 years ago | |
msvs-py | 2 years ago | |
reconstruct | 2 years ago | |
sdk | 2 years ago | |
server | 2 years ago | |
tests | 2 years ago | |
.gitignore | 2 years ago | |
Dockerfile | 2 years ago | |
LICENSE | 2 years ago | |
Makefile | 2 years ago | |
README.md | 2 years ago | |
aquery_config.py | 2 years ago | |
build_instructions.txt | 2 years ago | |
csv.h | 2 years ago | |
dbconn.py | 2 years ago | |
header.cxx | 2 years ago | |
mmw.cpp | 2 years ago | |
prompt.py | 2 years ago | |
requirements.txt | 2 years ago | |
sample_ast.json | 2 years ago | |
test.aquery | 2 years ago |
README.md
AQuery++ Database
Introduction
AQuery++ Database is a cross-platform, In-Memory Column-Store Database that incorporates compiled query execution.
Architecture
AQuery Compiler
- The query is first processed by the AQuery Compiler which is composed of a frontend that parses the query into AST and a backend that generates target code that delivers the query.
- Front end of AQuery++ Compiler is built on top of mo-sql-parsing with modifications to handle AQuery dialect and extension.
- Backend of AQuery++ Compiler generates target code dependent on the Execution Engine. It can either be the C++ code for AQuery Execution Engine or sql and C++ post-processor for Hybrid Engine or k9 for the k9 Engine.
Execution Engines
- AQuery++ supports different execution engines thanks to the decoupled compiler structure.
- AQuery Execution Engine: executes query by compiling the query plan to C++ code. Doesn't support joins and udf functions.
- Hybrid Execution Engine: decouples the query into two parts. The sql-compliant part is executed by an Embedded version of Monetdb and everything else is executed by a post-process module which is generated by AQuery++ Compiler in C++ and then compiled and executed.
- K9 Execution Engine: (discontinued).
Roadmap
- SQL Parser -> AQuery Parser (Front End)
- AQuery-C++ Compiler (Back End)
- Schema and Data Model
- Data acquisition/output from/to csv file
- Execution Engine
- Projections and single-group Aggregations
- Group by Aggregations
- Filters
- Order by
- Assumption
- Flatten
- UDFs (Hybrid Engine only)
- User Module
- Triggers
- Join (Hybrid Engine only)
- Subqueries
- Query Optimization
- Selection/Order by push-down
- Join Optimization
TODO:
- User Module load syntax parsing (fn definition/registration)
- User Module initialize location
- User Module test
- Interval based triggers
- Optimize Compilation Process, using static libraries, hot reloading server binary
- Bug fixes: type deduction misaligned in Hybrid Engine
- Investigation: Using postproc only for q1 in Hybrid Engine (make is_special always on)
- Limitation: putting ColRefs back to monetdb.
- Limitation: String operations and Date/Time data type.
- C++ Meta-Programming: Eliminate template recursions as much as possible.
Installation
Requirements
-
Recent version of Linux, Windows or MacOS, with recent C++ compiler that has C++17 (1z) support.
- GCC: 9.0 or above (g++ 7.x, 8.x fail to handle variadic template expansion due to a compiler bug)
- Clang: 5.0 or above (Recommended)
- MSVC: 2017 or later
-
Monetdb for Hybrid Engine
- On windows, the required libraries and headers are already included in the repo.
- On Linux, see Monetdb Easy Setup for instructions.
- On MacOS, Monetdb can be easily installed in homebrew
brew install monetdb
.
-
Python 3.6 or above and install required packages in requirements.txt by
python3 -m pip install -r requirements.txt
Usage
python3 prompt.py
will launch the interactive command prompt. The server binary will be autometically rebuilt and started.
Commands:
-
<sql statement>
: parse sql statement -
f <filename>
: parse all sql statements in file -
dbg
start debugging session -
print
: printout parsed sql statements -
exec
: execute last parsed statement(s) with AQuery Execution Engine. AQuery Execution Engine executes query by compiling it to C++ code and then executing it. -
xexec
: execute last parsed statement(s) with Hybrid Execution Engine. Hybrid Execution Engine decouples the query into two parts. The sql-compliant part is executed by an Embedded version of Monetdb and everything else is executed by a post-process module which is generated by AQuery++ Compiler in C++ and then compiled and executed. -
r
: run the last generated code snippet -
save <OPTIONAL: filename>
: save current code snippet. will use random filename if not specified. -
exit
: quit the prompt
Example:
f moving_avg.a
xexec