MENLO PARK, Calif.--(BUSINESS WIRE)--Ascend.io, the Data Automation Cloud, today announced results from its third annual research study, The DataAware Pulse Survey, about the work capacity and priorities of data teams. Findings from more than 500 U.S.-based data scientists, data engineers, data analysts, enterprise architects, and—new this year—chief data officers (CDOs) reveal that despite 81% of respondents indicating that their team’s overall productivity has improved in the last 12 months, 95% of teams are still at or over capacity—just a 1% decrease from the 2021 study. The study also found that data automation is emerging as the most promising path to increase data team capacity and productivity, with a majority (85%) planning to implement automation technologies in the next year even though only 3.5% of the same respondents reported currently have automation technologies in place.
Data Initiatives Are Ballooning Beyond Team Capacity
Nearly all data teams (93%) anticipate the number of data pipelines in their organization to increase between now and the end of the year—with 57% projecting an increase of 50% or greater. Amid the rising number of data pipelines across their organization, nearly three in four respondents (72%) indicated that the need for data products is growing faster than their team size. This was especially true among data engineer respondents, of which 82% stated that the need for data products was increasing at a faster rate than their team size.
“Data team productivity remains the single biggest threat to the success of data projects and workloads,” said Sean Knapp, CEO and founder of Ascend.io. “In fact, data team capacity has only marginally improved year over year, yet the demands on these teams continue to grow exponentially—far beyond what teams can feasibly keep up with.”
Team Backlogs Have Emerged Across the Data Lifecycle
One major roadblock to data team productivity remains fast access to data. When asked how much time they spend trying to gain access to the data they need to do their job, respondents said they spend an astounding 18.91 hours on average per week. Data scientists spend the most time trying to gain access to data each week at 24.6 hours, followed by data engineers at 19.1 hours.
However, data access is not the only roadblock. When it comes to the other top bottlenecks for team productivity, 66% cited team size or hiring constraints as their biggest productivity roadblock, followed by technology limitations (42%).
When asked which activities or tasks in their organization’s data ecosystem are the most backlogged, respondents were split. However, data scientists, data engineers, data analysts, and enterprise architects all agree to disagree—they each were all more likely to identify their own function as the most backlogged or resource-demanding compared to their peers.
- Data scientists are 3.3 times more likely to say data science is the most bottlenecked
- Data engineers are 2 times more likely to indicate data engineering
- Data analysts are 1.9 times more likely to say data analysis
- Enterprise architects are 1.5 times more likely to indicate data architecture
Data Teams Look to Automation, Flex Code, and Data Mesh to Increase Productivity
As data teams look for ways to overcome bandwidth limitations, many data professionals are turning to automation to improve data workload efficiency and productivity. In fact, while only 3.5% currently use them, 85% of respondents indicated that their team will likely implement data automation technologies in the next 12 months.
As data teams assess new solutions, many are considering low-code tools and data mesh frameworks to unlock greater team efficiency and business value. Respondents indicated a strong interest in low-code tools that provided greater flexibility (i.e., flex code), with the majority (81%) saying they would be more inclined to use a no-code or low-code tool if it offered the ability to use their preferred programming languages, up from 73% in 2021.
Respondents also cited a strong interest in data mesh frameworks, with 76% planning to implement a data mesh in the next 12 to 24 months. The majority (86%) of data teams believe a data mesh will enable their business to make the most of their existing data architectures and resources. A striking 90% of CDOs agree that a data mesh will enable the business to make the most of their data investments.
“The numbers don’t lie—data teams must find a way to dramatically accelerate their productivity, and the overwhelming majority are looking to automation as the answer,” said Knapp. “Data leaders are increasingly finding that leveraging automation in conjunction with flex code and data mesh technologies significantly increases productivity and amplifies the impact of some of their most talented resources.”
To learn more about Ascend.io’s 2022 DataAware Pulse Survey, download the full infographic or check out the automation-focused highlights.
Additional Resources
- Schedule a demo with one of Ascend.io’s data engineers
- Follow Ascend.io on Twitter and LinkedIn
- Read the Ascend.io blog
About Ascend.io
Ascend.io, the Data Automation Cloud named a 2021 Gartner Cool Vendor, provides the most advanced automation for data and analytics engineering workloads. Ascend unifies the core capabilities of data engineering—data ingestion, transformation, delivery, orchestration, and observability—into a single platform so that data teams deliver 10x faster. With 96% of data teams already at or over capacity, engineering productivity is a top priority for enterprises. Ascend’s Flex-code user interface empowers any member of the data team—from data engineers to data scientists to data analysts—to quickly and easily build and deliver on the data and analytics workloads they need. And with Ascend’s DataAware™ intelligence, data teams no longer spend hours carefully orchestrating brittle data workloads and instead rely on advanced automation to optimize the entire data lifecycle. Ascend runs natively on data lakes and warehouses and in AWS, Google Cloud and Microsoft Azure.
