Federal agencies are looking to gain actionable intelligence and information from disparate data sources in a secure, scalable, and efficient manner. An emerging technology known as a big data fabric could provide those agencies with a unified platform “that accelerates insights by automating ingestion, curation, discovery, preparation, and integration from data silos,” according to Forrester Research.
Forrester recently identified the 15 most significant big data fabric vendors in “The Forrester Wave: Big Data Fabric, Q2 2018” by Noel Yuhanna and other Forrester analysts. The big data fabric market is growing because more enterprise architects see big data fabric as a way to increase agility and minimize the complexity that Federal agencies are struggling with today.
The leaders in this field support a broad set of use cases, enhanced artificial intelligence and machine learning capabilities, and scalability features. The strong performers are offering more data management features and deployment options to compete with incumbent leaders, and a growing group of new contenders are ramping up their offerings to expand core functionality, according to the report.
The 15 companies include Cambridge Semantics, Cloudera, Denodo Technologies, Hitachi Vantara, Hortonworks, IBM, Informatica, Oracle, Paxata, Podium Data, SAP, Syncsort, Talend, TIBCO Software, and Trifacta, which were researched, analyzed, and scored against 26 criteria.
Big data fabric can support many types of use cases, including real-time insights, machine learning, streaming analytics, and advanced analytics. “Big data fabric offers the ability to efficiently store, process, and access large volumes of IoT [Internet of Things] data from sensors, devices, and switches in data lakes through automation and machine learning technologies. It enables analytics by streaming data from other big data platforms and integrating with data lakes to deliver operational insights. Big data fabric handles the whole process of ingestion, integration, processing, preparation, and security,” the report states.
This is good news for agency managers who want to make more informed decisions, in some cases in near real-time, to run more efficient operations and deliver better citizen-centric services. According to the MeriTalk/Teradata study, “Acing the Big Data Test,” 72 percent of Federal IT managers surveyed in January 2017 said they were using big data to improve mission outcomes. But only 57 percent, at the time, had a workable plan to accommodate the influx of IoT data.
Forty-five percent of the Federal managers said they are using big data to improve operational efficiency, 44 percent are executing cybersecurity analytics, and 41 percent are establishing performance tracking metrics.
Respondents also said their priorities frequently were cybersecurity analytics, financial data management, logistics management, and fraud detection. Additionally, the Federal managers said IoT data and big data analytics could help maximize productivity among existing IT applications, rectify IT infrastructure flaws, and improve user experience of IT services in the next three to five years.
To be included in the Forrester survey, companies offering big data fabric solutions must provide functionality such as data access, data discovery, data transformation, data integration, data preparation, data security, data governance, and data orchestration of data sources to support various big data fabric workloads and use cases. “The solution must be able to ingest, process, and curate large amounts of structured, semi-structured, and unstructured data stored in big data platforms such as Apache Hadoop, MPP EDW [Massively Parallel Processing Enterprise Data Warehouses], NoSQL, Apache Spark, in-memory technologies, and other related commercial and open source projects.”
A market leader such as Oracle offers a broad array of big data capabilities, and its strengths lie in its security, governance capabilities, highly scalable data movement, and transformations that can be done in real-time streaming environments, the report states. IBM, which is leveraging AI and machine learning capabilities, is a good fit for enterprises “that have voluminous and complex legacy data that must be integrated with other sources, require strong security, and want to leverage a hybrid cloud data fabric.” Meanwhile, a strong contender such as Hortonworks is building an open source platform for managing, securing, and governing big data fabrics.
The bottom line is that big data fabric platforms have the potential to help agencies turn data into insight with agility, speed, scale, and security.