Advantages and disadvantages of traditional database and new database

(1) Traditional indexes are not suitable for massive data.

The index of traditional row storage database needs to be manually set, which is not completely transparent to the application and needs to be constantly adjusted with the changes of scenarios and requirements, and the manual maintenance cost is very high. Moreover, the traditional index occupies a lot of storage space, even higher than the data itself, which leads to the decline of query efficiency.

(2) slow data loading speed

Because the index needs to be recreated, the loading performance will become poor. In order to solve these problems, the analysis architecture system must minimize disk I/O, improve query efficiency and reduce manual maintenance costs. GBase8a (hereinafter referred to as GBase 8a), a general analysis database of NTU, has effectively solved the above problems through column storage mode, data compression, intelligent indexing, parallel processing, concurrency control and efficient query optimizer. The following sections will describe how the innovative architecture of GBase 8a achieves these goals.

Second, the new database.

The new database adopts distributed parallel computing architecture and is deployed on X86 general server, which meets the real-time transaction requirements of big data with low cost and high scalability and breaks through the performance bottleneck of traditional databases.

Technical innovation of distributed non-relational database

NoSQL, a non-relational database, abandons the complex functions of relational database, such as relational operation and transaction processing, and only provides simple Key and Value data storage and query in exchange for high scalability and high performance, so as to meet the simple operation requirements of massive data in Internet application scenarios such as forums, blogs, SNS and Weibo. The main technological innovations are:

(1) Simple data manipulation for efficient response. NoSQL only supports key storage and query values (data), and does not support efficient query of non-key data columns; Because the data operation is simple, there is generally no need for associated operation between data, so the system can support high concurrency and fast response speed.

(2) A variety of consistency strategies meet business needs. Unlike traditional relational databases that only support strong consistency, NoSQL also supports weak consistency and final consistency, which can be configured according to application scenarios. For example, for applications with frequent writing operations but not strict requirements for reading the latest version of data, such as the storage and analysis of Internet webpage data, the final consistency strategy can be adopted; For the application of subscription relational storage, strong consistency strategy must be used to ensure that the latest version of data is always read.