DataStax, an expert in artificial intelligence (AI) focused NoSQL databases, has introduced its Hyper-Converged Data Platform (HCDP) as a pre-packaged solution for enterprise clients looking to develop in-house generative AI (GenAI) vector databases.
Vectors are mathematical representations used by GenAI systems to analyze and compare datasets, enabling valuable insights. HCDP enables organizations to build GenAI platforms within their own data centers, leveraging the technology on their private data. The release includes Nvidia microservices and retrieval augmented generation (RAG) capability.
DataStax emphasizes that HCDP is not a hardware appliance or combined storage and server software. Instead, it is intended to be implemented virtually in the customer’s environment, similar to cloud deployment. Founded in 2010, DataStax is deeply rooted in the NoSQL database industry. In addition to HCDP, it also offers Astra DB, a cloud-based database-as-a-service, and DataStax Enterprise (DSE), which caters to on-premise deployments. Both Astra DB and DSE are based on the Apache Cassandra NoSQL database. The launch of HCDP coincides with the release of DSE version 6.9.
Bill McLane, Chief Technology Officer for cloud at DataStax, stated that HCDP targets clients who want to establish on-premise GenAI infrastructure, allowing them complete control over data utilization. He also highlighted that with HCDP, companies can leverage generative AI using their own data and language models without surrendering control to third parties.
HCDP utilizes the capabilities of OpenSearch for search and visualization and Apache Pulsar as a messaging platform for building data pipelines and managing data distribution. McLane emphasized that vectors are crucial to HCDP and DataStax’s functionality in GenAI. He explained that generative AI systems utilize vector search queries to gather relevant data, which are then compared with the existing vector data set of the company. The information is then employed to generate responses.
Any type of data, including product catalogs, customer histories, or unstructured data records, can be transformed into vectors and stored for future searches. For clients who want to use their own data in a GenAI system alongside their transactional database, HCDP enables streaming of new data to create and update vector data.
This offering is particularly useful for companies concerned about compliance and security or those with large database installations who prefer to avoid migrating their data to a cloud environment due to cost reasons.