Implementing DevOps: Navigating the Top 4 Challenges

This is the second part of our DevOps Journey Series. In this article, we'll dive into industry research to present a data-backed perspective on some of the challenges organizations face as they adopt DevOps practices.


Based on our interactions with over 200 organizations, our previous article concluded that very few have streamlined DevOps and that challenges with implementation are a common occurrence.

In the middle of their DevOps implementation journey, Developer and Ops teams are getting overwhelmed with firefighting. As a result of this, teams are getting distracted from focusing on and solving core business problems, leading to fewer features being developed or deployed.

For starters, even though 83% of IT leaders report their organization uses DevOps approaches, 78% say their teams are performing below expectations, according to a study by Puppet1. Gartner predicted something similar in 2019, stating that "75% of DevOps initiatives will fail to meet expectations."

In this article, we will investigate the research to determine the extent of these challenges. We have reviewed multiple whitepapers published by the industry's leading voices to gain insight into these challenges and are sharing our findings. Let's begin.

Impact on Developer and Ops productivity

According to Haystack2 and Survation, 74% of developers find themselves “working on operations in some form, even if only as part of their job.” And, only 26% "solely" work in Product Development.

Impact on Developer and Ops productivity | Facets | Implementing DevOps: Navigating the Top 4 Challenges
Study to understand the impact of COVID-19 on Software Engineers, Haystack

This was not surprising to us. While building new products or deploying new features, speed matters. And as many iterations take place, handoffs happen between Developer and Ops teams.

These handoffs, over time, become much likelier and responsible for information and team productivity loss6. To reduce these handoffs, we introduce automation. But, deployment automation is complex and requires the Ops team to use multiple tools correctly and work with the Development teams, decreasing their bandwidth to work on innovations instead.

Another piece of evidence we found that impacts Developer and Ops productivity was lacking visibility into incoming "unplanned work."

According to Jellyfish3, 45% of engineering managers think that making sure everyone is focused on the highest priority work is the greatest challenge in 2022.

Top challenges faced by engineering managers | Facets | Implementing DevOps: Navigating the Top 4 Challenges
2022 State of Engineering Management, Jellyfish

The same analysis also finds the average percentage of time a developer team spends on unplanned work. It increased by three percentage points in 2021 (22%) from 2020 (19%).

"Teams spent 22% of time on unplanned work in 2021, slightly more than they spent in 2020 at 19%. This sudden increase in unplanned work may signal that management is unaware of certain types of work that are taking time away from established priorities." - According to the report

Now that’s a pain point! Developers’ productivity suffers, resulting in slower release cycles.

In organizations where DevOps implementation is incomplete, or a cultural Shift-left is nonexistent, inefficiencies are widespread due to exceeding cognitive load on the Ops team. This is happening because keeping up with the best practices and managing multiple tools and upgrades is challenging.

An example of this is Treebo, a well-known hotel chain in the premium-budget segment from India. As Kadam Jeet Singh, CTO & Co-founder of Treebo tells in his own words:

"The biggest challenge for the Ops team was that they would spend a good 70-80% of their time in solving production issues or helping the development teams out in debugging, this was frustrating for both teams. The turnaround times of the team would inevitably be higher because of this."

That said, another challenge organizations encounter is a lack of talent with an eye for detail and a skillset to reduce toil.

DevOps skillset shortage

The DevOps market crossed $7 billion in 2021 and is predicted to expand at a CAGR of more than 20% from 2022 to 2028, reaching a value of more than $30 billion.

And as a result, job opportunities are also increasing.

That's just the United States! It's no longer a mystery that DevOps engineers are in high demand, with over 112K new job ads on LinkedIn.

But this question sticks around: Is the availability of individuals with DevOps skills ample as well? The answer is No. Very few keep the know-how and are efficient in building automation to reduce toil.

According to the trends captured in the Puppet report1, shortage of skills, at 33%, is the single largest contributing factor to why organizations struggle and face DevOps challenges.

"Armed with a clearer sense of the task at hand and having begun to automate more, mid-level teams cite a shortage of skills (33%), legacy architecture (29%), organizational resistance to change (21%), and limited or lack of automation (19%) as the primary blockers to better DevOps practices." - According to the report

It's not the ideal case. With increased organizational efficiency as a byproduct, your reliance on DevOps engineers should decrease.

Environmental inconsistencies leading to unpredictable outcomes

According to CSA4, Lack of Internal Guidance (33%) followed by Insecure Default Settings (18%) and Negligence (16%), are the major contributing factors to misconfigurations. We'll call them Environmental Inconsistencies.

Primary causes of misconfigurations in organisations | Facets | Implementing DevOps: Navigating the Top 4 Challenges
Secure DevOps and Misconfigurations Survey Report, CSA

Tracking changes in your cloud workload becomes increasingly difficult as the size of your team grows. It's a perfect recipe for what we'll discuss next.

Environment drifts are common in organizations where new code releases happen numerous times daily. That said, the consequences of poorly managed drifts can be far-reaching — from growing frustration in engineers due to delayed deployments to unplanned downtimes and security vulnerabilities costing real dollars.

Environmental inconsistencies are manageable in general. However, substantial internal advice is required. The CSA Report4 emphasizes this:

"The primary reason cited for these misconfigurations was flawed or lacking internal guidance (33%). This indicates that the guidance organizations are developing internally is ineffective for preventing misconfiguration." - According to the report

The need of the hour is to have single-click environments with collaborative workflows. Faster issue discovery will then result in faster code deployments, and less time to go live with the changes.

Underestimated or wasted Cloud Spend

Gartner forecasts public cloud spending will reach roughly $600 billion in 2023 — a 43% spending increase from 2021. Big spending, indeed. But, what about spending predictions or wasteful spending? According to Flexera5, wasted cloud spending has increased to 32% in 2022, from 30% in 2021.

"Spend is likely less efficient and likely even higher on average, as many organizations tend to underestimate their amount of waste." - According to the report

According to the same study, 66% of respondents report an increase in cloud spending that was "higher than initially planned this year."

Underestimated or wasted Cloud Spend | Facets | Implementing DevOps: Navigating the Top 4 Challenges
State of the Cloud Report, Flexera

Cloud cost optimization challenges best of the best. It requires a by-design cost-optimization approach to DevOps.

Let's Summarize

Scalability, reliability, and cost productivity are essential in today's fast-paced industry. Maintaining a competitive edge requires quick deployment times, upgrades, and new features, as well as maturity in dealing with DevOps challenges.

However, with new innovations, massive disruption in the DevOps value chain will continue in the coming years, and the trend toward innovation-led processes is no surprise. Platform Engineering proposes new approaches to accelerate and rethink DevOps. In our future articles, we'll go over this topic in greater depth.

Sources

  1. The 2021 State of DevOps Report, Puppet
  2. Study to understand the impact of COVID-19 on Software Engineers, Haystack
  3. 2022 State of Engineering Management, Jellyfish
  4. Secure DevOps and Misconfigurations Survey Report, CSA
  5. State of the Cloud Report, Flexera
  6. The Art of Lean Software Development

table of contents

SHARE

Back

Get Your Facets Developer Control Plane

Consult our experts for your DevOps needs by booking a demo