AQuery Database
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
 
 
Bill a464029d14
Added select into
2 years ago
aquery_parser Added select into 2 years ago
data bug fixes, restructure, user module parsing 2 years ago
docs regression: nested aggregation support 2 years ago
engine regression: nested aggregation support 2 years ago
lib clean trash 2 years ago
monetdb bug fixes 2 years ago
msc-plugin bug fixes 2 years ago
msvs-py fix gitw 2 years ago
reconstruct Added select into 2 years ago
sdk added user module 2 years ago
server Added select into 2 years ago
tests regression: nested aggregation support 2 years ago
.gitignore added user module 2 years ago
Dockerfile restructure 2 years ago
LICENSE fix gitw 2 years ago
Makefile Added select into 2 years ago
README.md Added select into 2 years ago
aquery_config.py Added select into 2 years ago
build_instructions.txt restructure 2 years ago
csv.h fix gitw 2 years ago
dbconn.py fix gitw 2 years ago
header.cxx fix gitw 2 years ago
mmw.cpp fix gitw 2 years ago
prompt.py Added select into 2 years ago
requirements.txt restructure 2 years ago
sample_ast.json Assumption, outfile, bugfixes on type deduction 2 years ago
test.aquery added user module 2 years ago

README.md

AQuery++ Database

Introduction

AQuery++ Database is a cross-platform, In-Memory Column-Store Database that incorporates compiled query execution.

Architecture

Architecture

AQuery Compiler

  • The query is first processed by the AQuery Compiler which is composed of a frontend that parses the query into AST and a backend that generates target code that delivers the query.
  • Front end of AQuery++ Compiler is built on top of mo-sql-parsing with modifications to handle AQuery dialect and extension.
  • Backend of AQuery++ Compiler generates target code dependent on the Execution Engine. It can either be the C++ code for AQuery Execution Engine or sql and C++ post-processor for Hybrid Engine or k9 for the k9 Engine.

Execution Engines

  • AQuery++ supports different execution engines thanks to the decoupled compiler structure.
  • AQuery Execution Engine: executes query by compiling the query plan to C++ code. Doesn't support joins and udf functions.
  • Hybrid Execution Engine: decouples the query into two parts. The sql-compliant part is executed by an Embedded version of Monetdb and everything else is executed by a post-process module which is generated by AQuery++ Compiler in C++ and then compiled and executed.
  • K9 Execution Engine: (discontinued).

Roadmap

  • SQL Parser -> AQuery Parser (Front End)
  • AQuery-C++ Compiler (Back End)
    • Schema and Data Model
    • Data acquisition/output from/to csv file
  • Execution Engine
    • Projections and single-group Aggregations
    • Group by Aggregations
    • Filters
    • Order by
    • Assumption
    • Flatten
    • UDFs (Hybrid Engine only)
    • User Module
    • Triggers
    • Join (Hybrid Engine only)
    • Subqueries
  • Query Optimization
    • Selection/Order by push-down
    • Join Optimization

TODO:

  • User Module load syntax parsing (fn definition/registration)
  • User Module initialize location
  • User Module test
  • Interval based triggers
  • Optimize Compilation Process, using static libraries, hot reloading server binary
  • Bug fixes: type deduction misaligned in Hybrid Engine
  • Investigation: Using postproc only for q1 in Hybrid Engine (make is_special always on)
  • Limitation: putting ColRefs back to monetdb.
  • Limitation: String operations and Date/Time data type.
  • C++ Meta-Programming: Eliminate template recursions as much as possible.

Installation

Requirements

  1. Recent version of Linux, Windows or MacOS, with recent C++ compiler that has C++17 (1z) support.

    • GCC: 9.0 or above (g++ 7.x, 8.x fail to handle variadic template expansion due to a compiler bug)
    • Clang: 5.0 or above (Recommended)
    • MSVC: 2017 or later
  2. Monetdb for Hybrid Engine

    • On windows, the required libraries and headers are already included in the repo.
    • On Linux, see Monetdb Easy Setup for instructions.
    • On MacOS, Monetdb can be easily installed in homebrew brew install monetdb.
  3. Python 3.6 or above and install required packages in requirements.txt by python3 -m pip install -r requirements.txt

Usage

python3 prompt.py will launch the interactive command prompt. The server binary will be autometically rebuilt and started.

Commands:

  • <sql statement>: parse sql statement

  • f <filename>: parse all sql statements in file

  • dbg start debugging session

  • print: printout parsed sql statements

  • exec: execute last parsed statement(s) with AQuery Execution Engine. AQuery Execution Engine executes query by compiling it to C++ code and then executing it.

  • xexec: execute last parsed statement(s) with Hybrid Execution Engine. Hybrid Execution Engine decouples the query into two parts. The sql-compliant part is executed by an Embedded version of Monetdb and everything else is executed by a post-process module which is generated by AQuery++ Compiler in C++ and then compiled and executed.

  • r: run the last generated code snippet

  • save <OPTIONAL: filename>: save current code snippet. will use random filename if not specified.

  • exit: quit the prompt

Example:

f moving_avg.a
xexec