AQuery++ Database is a cross-platform, In-Memory Column-Store Database that incorporates compiled query execution.
AQuery++ Database is a cross-platform, In-Memory Column-Store Database that incorporates compiled query execution.
# Installation
## Requirements
## Requirements
1. Recent version of Linux, Windows or MacOS, with recent C++ compiler that has C++17 (1z) support. (however c++20 is recommended if available for heterogeneous lookup on unordered containers)
1. Recent version of Linux, Windows or MacOS, with recent C++ compiler that has C++17 (1z) support. (however c++20 is recommended if available for heterogeneous lookup on unordered containers)
- GCC: 9.0 or above (g++ 7.x, 8.x fail to handle fold-expressions due to a compiler bug)
- GCC: 9.0 or above (g++ 7.x, 8.x fail to handle fold-expressions due to a compiler bug)
@ -18,7 +17,46 @@ AQuery++ Database is a cross-platform, In-Memory Column-Store Database that inco
3. Python 3.6 or above and install required packages in requirements.txt by `python3 -m pip install -r requirements.txt`
3. Python 3.6 or above and install required packages in requirements.txt by `python3 -m pip install -r requirements.txt`
## DOCKER:
## Installation
AQuery is tested on mainstream operating systems such as Windows, macOS and Linux
### Windows
There're multiple options to run AQuery on Windows. You can use the native toolchain from Microsoft Visual Studio or gcc from Cygwin/MinGW or run it under Windows Subsystem for Linux.
- For WSL, Docker or linux virtual machines, see Linux, Docker sections below
- For Visual Studio:
1. Install python3.6 or above from [official website](https://www.python.org/downloads/windows/) or Microsoft Store.
2. Install Microsoft Visual Studio 2022 or later with **Desktop development with C++** selected.
3. Clone AQuery repo from [Github](https://github.com/sunyinqi0508/AQuery2)
5. Change the build driver from aquery_config.py to "MSBuild"
6. The libraries and headers for Monetdb are already included in msc-plugins, however you can also choose to download them from [Monetdb Easy Setup](https://www.monetdb.org/easy-setup/) and put them in the same place.
- For MinGW:
1. Install gcc and python3 using the package manager. (For Msys2, `pacman -S gcc python3`)
2. Clone AQuery repo from Github
3. Install python requirements
4. The prebuilt binaries are included in ./lib directory. However, you could also rebuild them from [source](https://github.com/MonetDB/MonetDB).
- Note that it might be possible to use python from python.org or Microsoft store with gcc from MinGW. However, it might not work because of ABI breakage. So the better way is to use gcc with MinGW python from pacman or use clang/MSVC instead.
### macOS
- Install a package manager such as [homebrew](https://brew.sh)
- Install python3 and monetdb using homebrew `brew install python3 monetdb`
- Install C++ compiler come with Xcode commandline tool by `xcode-select --install` or from homebrew
- If you have multiple C++ compilers on the system. Specify C++ compiler by setting the **CXX** environment variable. e.g. `export CXX=clang`
- Install python packages from **requirements.txt**
**for arm64 macOS users**
- In theory, AQuery++ could work on both native arm64 and x86_64 through Rosetta. But for maximum performance, running native is preferred.
- However, they can't be mixed up, i.e. make sure every component, `python` binary, `C++ compiler`, `monetdb` library and system commandline utilities such as `uname` should have the same architecture.
- Because I can't get access to an arm-based mac to fully test this setup, there might still be issues. Please open an issue if you encounter any problems.
### Linux
- Install monetdb, see [Monetdb Easy Setup](https://www.monetdb.org/easy-setup/) for instructions.
- Install python3, C++ compiler and git. (For Ubuntu, run `apt update && apt install -y python3 python3-pip clang-14 libmonetdbe-dev git `)
- If you have multiple C++ compilers on the system. Specify C++ compiler by setting the **CXX** environment variable. e.g. `export CXX=clang-14`
### Docker:
- Alternatively, you can also use docker to run AQuery.
- Alternatively, you can also use docker to run AQuery.
- Type `make docker` to build the docker image from scratch.
- Type `make docker` to build the docker image from scratch.
- For Arm-based Mac users, you would need to build and run the x86_64 docker image because MonetDB doesn't offer official binaries for arm64 Linux.
- For Arm-based Mac users, you would need to build and run the x86_64 docker image because MonetDB doesn't offer official binaries for arm64 Linux.
@ -33,7 +71,6 @@ AQuery++ Database is a cross-platform, In-Memory Column-Store Database that inco
- `xexec`: execute last parsed statement(s) with Hybrid Execution Engine. Hybrid Execution Engine decouples the query into two parts. The standard SQL (MonetDB dialect) part is executed by an Embedded version of Monetdb and everything else is executed by a post-process module which is generated by AQuery++ Compiler in C++ and then compiled and executed.
- `xexec`: execute last parsed statement(s) with Hybrid Execution Engine. Hybrid Execution Engine decouples the query into two parts. The standard SQL (MonetDB dialect) part is executed by an Embedded version of Monetdb and everything else is executed by a post-process module which is generated by AQuery++ Compiler in C++ and then compiled and executed.
- `save <OPTIONAL: filename>`: save current code snippet. will use random filename if not specified.
- `save <OPTIONAL: filename>`: save current code snippet. will use random filename if not specified.
- `exit`: quit the prompt
- `exit`: quit the prompt
- `exec`: execute last parsed statement(s) with AQuery Execution Engine (Old). AQuery Execution Engine executes query by compiling it to C++ code and then executing it.
- `r`: run the last generated code snippet
- `r`: run the last generated code snippet
### Example:
### Example:
`f moving_avg.a`<br>
`f moving_avg.a`<br>
@ -41,10 +78,6 @@ AQuery++ Database is a cross-platform, In-Memory Column-Store Database that inco
See ./tests/ for more examples.
See ./tests/ for more examples.
## Notes for arm64 macOS users
- In theory, AQuery++ could work on both native arm64 and x86_64 through Rosetta. But for maximum performance, running native is preferred.
- However, they can't be mixed up, i.e. make sure every component, `python` binary, `C++ compiler`, `monetdb` library and system commandline utilities such as `uname` should have the same architecture.
- Because I can't get access to an arm-based mac to fully test this setup, there might still be issues. Please open an issue if you encounter any problems.
## Architecture
## Architecture
@ -86,9 +119,8 @@ See ./tests/ for more examples.
- [ ] Hot reloading server binary
- [ ] Hot reloading server binary
- [x] Bug fixes: type deduction misaligned in Hybrid Engine
- [x] Bug fixes: type deduction misaligned in Hybrid Engine
- [ ] Investigation: Using postproc only for q1 in Hybrid Engine (make is_special always on)
- [ ] Investigation: Using postproc only for q1 in Hybrid Engine (make is_special always on)
- [x] Limitation: putting ColRefs back to monetdb. (Comparison)
- [ ] C++ Meta-Programming: Eliminate template recursions as much as possible.
- [ ] C++ Meta-Programming: Eliminate template recursions as much as possible.
- [ ] Functionality: Basic helper functions in aquery
- [ ] Functionality: Basic helper functions in aquery
- [] Improvement: More DDLs, e.g. drop table, update table, etc.
- [x] Improvement: More DDLs, e.g. drop table, update table, etc.
