The Cortex platform consists of aNoSQL database (CortexDB), a universal application (CortexUniplex), data import tools (CortexImplex and CortexJSAutoimport) and a collection of tools - eg CortexUniplexAPI (JSON HTTP API) enables developers and users to quickly and easily implement new applications, data analytics, and other requirements. This applies both to the department as well as to developers who require fast and easy database access without having to deal with the actual database technology.
In addition to the database server, the overall concept also includes tools based on it for immediate deployment in order to
- easily configure data models for the prefered application
- ensure data imports from other systems
- grant via developer interface (CortexUniplexAPI) a simple a database access.
It is irrelevant for the realization phase whether you want to create an individual application for a few thousand datasets or whether you need a database for large and/or complex structures (for example, big data).
For general information, please visit our website: http://www.cortex-ag.com
The CortexDB Database¶
The CortexDB is the core of the CortexPlatform. It is a database management system (DBMS) developed entirely in C/C++ without any third-party libraries. The CortexDB is a multi-model database that uses the concept of a document store for the storage of data records, transfers all fields and field contents redundancy-free into a multi-dimensional key/value store (6th normal form / 6NF), and can store the transaction time and validity time of information bitemporally. In addition, the one-to-one connection between data records is possible as a reference structure ("GraphDB").
By combining the different database concepts, CortexDB stores data records as a container (document store) and at the same time transfers them into the 6th normal form (key/value store). This represents the entire database and the original data records are thus preserved. Selections are carried out exclusively in the redundancy-free 6th normal form and only the result is read from the document store containers if the user rights permit this. This means that in addition to read and write rights to data records, a selection right is also available here.
In contrast to NoSQL and SQL databases, the CortexDB is the only database that combines the use of a schema-less data store with simultaneous and complete normalisation of all contents into the highest normal form.
The following diagram illustrates the entire process from integration from a relational model (or also csv files) to storage in the CortexDB as a document store and its simultaneous storage in the key/value store, to selection and the final reading of the results.
Scheme for the 6th normal form (Transmission and query)¶
6th normal form (index)¶
The 6th normal form (i.e. the index structure) of the CortexDB differs considerably from the structures and possibilities of all previously available databases. This means not only greater speed and simpler queries, but also completely new approaches.
It should be noted that the index maintenance is taken over fully automatically by the database and that no database and no adjustments are necessary.
General database structures and index¶
Databases differ primarily in their database schema. Relational databases organise data in tables and columns, using the same column contents to link tables together (via so-called foreign keys). This is often documented in the form of entity relationship diagrams (ERD).
In general, all databases currently in use (SQL and also NoSQL) are distinguished by this database schema, whether Key/Value, Document Store, GraphDB and others. Only the applications organise data based on the underlying database schema. The database index is a structure separate from the data to speed up and search database content.
All databases use flat index structures with references to data records to enable data access in sorted order. Differences are formed by special indexes in which the databases differ, although in all cases these are flat indexes (e.g. bitmap index).
Field index in the application¶
Within the application CortexUniplex the index for each field can be displayed via the so-called field index. This shows the used content of the field (about all dataset types) and the frequency (number) of contents in the database. Behind every number, therefore, is the above mentioned list of contents.
For each entry in this field index, the result can be displayed in a separate list. This allows direct access to datasets with the corresponding content. For further selections for combining the various contents via the field index, additional functions are available in the application.
Due to the quantity combination of different field indexes, the queries within the database are made in this way. Through the average, difference, symmetric difference and total amount only the occurrences are combined with each other, and only when the output occurs, the datasets are being read.
Because the CortexDB tracks every content and field in an index list, queries are made using simple set combinations.
Example of the combination of names and locations:
Meyer: ID1; ID2; ID3; ID5; ID8; ID13 Hamburg: ID2; ID4; ID6; ID8; ID10 Average = [Meyer] ∩ [Hamburg] = ID2; ID8
The intersection of the IDs of the fields Name for Meyer and Place for Hamburg results in a new ID list with two contents. Only after this combination has been performed does the display or further processing of the contents of affected datasets take place. Before this time, it is not necessary to read the datasets, only to read the ID lists (ie the index).
The configuration of such and further selections takes place in the standard application and can be used via the corresponding API (CortexUniplexAPI).
The CortexDB offers the possibility to use different database functions as part of the schemaless structure.
It is thus possible, for example, to construct structures via reference fields, as can be found in graph databases. Furthermore, individual fields (optional) may be used as often as desired in a data set (e.g., for bank details or e-mail addresses);Similarly, past, current and future values can be stored per field of a data set (bitemporal database). In addition, the 6th normal form is stored internally similar to key/ value stores (multi-dimensional key/value store).
The example shows the dataset of a person with a reference to another person and its description over time.
Each of these options can also be combined with the others (for example, for the temporal change of multiple references).
For a dataset, the CortexDB stores the transaction time of changes. In addition, each content of all fields can be individually provided with a validity period within a data dataset (attribute time stamping). Therefore, for each dataset not only the state of a dataset in the last transaction is visible, but also the complete "life cycle" (historization), in which case we speak of bitemporal database.
In this case, the transaction time is automatically (implicitly) maintained by the server and the validity of a value is maintained by the user (explicitly) or by automatic mechanisms such as data import or interfaces. If no validity has been maintained, then "unknown" applies to this value until it is overwritten with a new value (a transaction overwrites this value) or another value is added with a validity.
Information about transactions can be displayed within a dataset.
In contrast, the history information (or * history processing *) shows the validity of a value in a field. These can be displayed for each field and maintained explicitly. If the validity is unknown for a value, the value is permanently valid until it is overwritten by a new value. The standard view of a dataset always shows only the currently valid value.