The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced today Apache® IoTDB™ as a Top-Level Project (TLP).
Apache IoTDB is an Open Source IoT database designed to meet the rigorous data, storage, and analytics requirements of large-scale Internet of Things (IoT) and Industrial Internet of Things (IIoT) applications. The project was first developed as a research project at Tsinghua University and entered the Apache Incubator in November 2018.
“The Internet of Things, especially Industrial IoT, has swept the globe with unimaginable volumes of data,” said Xiangdong Huang, Vice President of Apache IoTDB. “To date, both Relational and Key Value-based database solutions struggle to meet the demands of IoT data management. Apache IoTDB is the missing link between current IoT data and IoT applications, and is redefining how IoT data is managed, both in the cloud and on the edge. We are proud to graduate as an Apache Top-Level Project, which is an important milestone in our project’s maturity.”
Apache IoTDB provides a compact and time series optimized columnar data file, which is able to efficiently store and access time series data. The database engine is specially optimized for time series-oriented operations, such as aggregations query, down-sampling, and time alignment query. Due to its lightweight structure, high performance, and deep integration with Apache Big Data ecosystem projects (such as Flink, Hadoop, and Spark), Apache IoTDB easily meets the requirements of storing massive data sets, ingesting high-speed data, and analyzing complex data, both on the edge and the cloud. Features include:
- High-throughput read and write: supports high-speed write access for millions of low-power and intelligently networked devices, and provides lightning-quick read access for retrieving data on billions of data points.
- Efficient directory structure: organizes complex metadata structure from IoT devices and large scale time series data, with fuzzy searching strategy for complex directory of time series data.
- Rich query semantics: supports time alignment for time series data across devices and sensors, computation in time series field, and abundant aggregation functions in time dimension.
- Flexible deployment: supports running on the edge (e.g., running on a Raspberry Pi), as well as forming a cluster in the cloud. It also provides a bridge tool between cloud platforms and data synchronization on premise machines.
- Deep integration with Open Source Big Data projects: supports analysis ecosystems, including Apache Flink, Hadoop, PLC4X and Spark, as well as other Open Source applications.
- Low hardware cost: reaches a high compression ratio of disk storage.
Apache IoTDB is in use at dozens of organizations that include ArcelorMittal AMERICA, BONC Ltd., the China Meteorological Administration, Datang Xianyi, Goldwind, Haier, Lenovo, NAVINFO, pragmatic industries GMBH, Shanghai Metro, Tsinghua University, Yangtze Optical Fiber and Cable Company, and more.
“IoTDB has attained Apache Top Level project status at a time of confluence of database, IoT and AI technologies in conjunction with a wider adoption of Industry 4.0 and automation approaches to further enable remote work and increased efficiencies,” said Prof. C. Mohan, recently retired IBM Fellow, Former Chief Scientist of IBM India, and a member of the US National Academy of Engineering. “I am excited since this is the first Chinese University originated open-source project to reach this status. While I have been associated with the researchers behind IoTDB as a Distinguished Visiting Professor of the School of Software at China’s prestigious Tsinghua University, I have seen this project reach maturity and build up a vibrant OSS community around it. It has a bright future ahead of it and I plan to collaborate on it.”
“Apache IoTDB is a perfect fit for edge computing,” said Dr. Julian Feinauer, CEO at pragmatic industries GmbH. “The high compression helps to use the (limited) amount of memory we have very efficiently. IoTDB is a perfect fit, especially in IIoT use cases, where network and compute capabilities are limited on the edge.”
“Apache IoTDB was initially launched by a Chinese University and then incubated successfully in the Apache Community,” said Prof. Hong Mei, an academician of the Chinese Academy of Sciences. “Following the Apache Way, it has created a healthy and active international open source community. It is a successful practice of open source education and culture advancement in China.”
“Apache IoTDB has made many optimizations for different runtime environments, operating systems, and workloads in both the edge and the cloud. As a core infrastructure software in Industrial Internet, it innovates a series of IoT data management and analysis techniques,” said Prof. Xiangke Liao, an academician of the Chinese Academy of Engineering. “Through the open source model, Apache IoTDB shares its creative techniques to the world.”
“With the continuous growth of intelligent devices, machine-generated data is growing day by day, which poses extraordinary challenges on storing process, query speed, and storage space,” said Dawei Liu, architect at AutoAI Inc., a subsidiary of NAVINFO, and member of the Apache IoTDB Project Management Committee. “We tried and tested a variety of solutions and finally chose IoTDB as our core database for its high performance, openness to the enterprise, and its active community. We built our Wecloud platform based on Apache IoTDB, which has served well for BMW, Toyota, and Great Wall Motors, among other auto manufacturers. The project deeply attracted me to become a part of the community. The coolest thing is that I finally became an IoTDB committer and now share our ideas to the community.”
“Apache IoTDB is an open source project and software technology innovation developed for the need of AIoT Big Data applications,” said Prof. Jianmin Wang, Dean of the Tsinghua University School of Software, who originally decided to donate the project to the ASF. “It is also a very beneficial attempt for training leading talents. There will be a long way to go and the future is promising.”
“Apache IoTDB is on its way to becoming a standard IoT data management and analysis solution, and we’re excited to build upon our work thus far,” added Huang. “We believe Apache IoTDB will help more users and companies to solve their real problems. The process to achieve the goal is exciting and honorable, and we invite more contributors to join us. Following the Apache Way, let’s bring this interesting, meaningful, and powerful software to the whole world.”
A published paper on Apache IoTDB written by members of the Apache IoTDB Project Management Committee is available at http://www.vldb.org/pvldb/vol13/p2901-wang.pdf.
Availability and Oversight
Apache IoTDB software is released under the Apache License v2.0 and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project’s day-to-day operations, including community development and product releases.