The tools you work with have a lot of impact on what you can accomplish, and the more sophisticated the tools the better — especially in software. My company, Beagle Research, just completed a study into the use of a DevOps strategy with the Salesforce Lightning Platform, sponsored by DevOps solutions provider Copado.
DevOps is a strategy for building, changing and deploying enterprise software that can be used with a Scrum or Agile methodology, as well as others. More than concentrating on code and coding, DevOps takes a holistic look at culture and infrastructure in its broadest manifestation.
Even if you’ve never built systems, you can surmise that planning, developing, assessing, testing and deploying software are all critical milestones, and they’re often spread across technical departments of IT like development and operations, hence the name.
It can be challenging to compare enterprise software strategies. For instance, using an on-premises hardware and software stack has been common for decades, but with the development of cloud computing, users found they could eliminate having to care about a good deal of their development environments, leaving it all to the cloud vendor.
How do you compare overhead and costs between cloud and on-premises? What are the effects on speed-to-market, reliability, security? To control for some variables and enable us to make an apples-to-apples comparison, we chose to research only companies developing and maintaining systems using Salesforce Lightning and a DevOps strategy.
Value for Most
Companies ranged in size from fewer than 100 employees to more than 10,000. There were similar measures for number of Salesforce users and number of developers, as well as number of production orgs. Two-thirds, or 67 percent, said they ran between two and seven production orgs. Most of the respondents were C-level executives (48 percent) or upper management (40 percent).
We found that DevOps has been delivering value for most of its users, though the larger organizations have greater challenges — more on that in a moment. Seventeen percent claimed more than US$5 million in benefits from using a DevOps strategy.
Those people have a good understanding that software flexibility drives business agility. Impressively, 54 percent said their lead time for making changes to their Salesforce orgs was between one day and one week. Compare that to a more traditional process that typically takes weeks or months.
We also identified an elite group that operates even faster — 21 percent said their lead time for making changes was less than a day, and 8 percent said it took less than an hour. Taken together, 83 percent could make changes in a week or less.
In other recent research I’ve been involved in, the delivery of running, tested and deployable code was much slower. Clearly, if a business depends on its ability to make quick changes to meet changing market demands, this is where you want to be.
On the Other Hand
As you might expect, though, the benefits of a DevOps strategy were not distributed evenly across all users. Generally, smaller businesses with smaller development groups did better overall at establishing DevOps programs and at excelling within them.
The most successful businesses using DevOps were those that used a well-integrated set of tools to move through development and deployment. Many organizations, especially smaller ones, used a combination of in-house developed and open source management tools. At best, the great variety of tool choices suggested to me that some best practices were still being worked out.
Even with Salesforce Lightning and a DevOps approach, you still can have issues. Almost everyone had the experience of deploying a release to a production org and having a service degradation.
A plurality of respondents — 43 percent — said a problem occurred up to 15 percent of the time. However, the vast majority — 86 percent — said service degradations happened less than half of the time.
This is an important snapshot of the state of the industry. Speed of delivery slightly exceeds stability of releases, indicating a need to bring the two metrics more in alignment.
Some Best Practices Considerations
A strong majority — 60 percent — said each developer in the business had a private development environment.
Also, 77 percent said they used version control to store code and click-based Salesforce customizations.
Most synchronized their development environments with the latest changes from other teams, with 41 percent doing it on-demand, or at most once per day, and 42 percent saying they did it between once a day and once a week.
Seventy-five percent said changes made in version control triggered automation tests.
Eighty-seven percent expressed confidence that when automated tests pass, the software is ready for release. However, meta-analysis of the data strongly suggests that the greater a team’s confidence in their tests, the higher their change failure rate.
Skeptics who were neutral on this question experienced a 40 percent lower change fail rate than those who expressed strong confidence in their tests.
It’s good to be skeptical.
Some Analysis
The allure of the digital disruption, in part, is having the capacity to change a business process to take advantage of changing market conditions. Many businesses already are having that experience.
Big data and analytics tell us what needs attention, but then we still need to change our systems’ behaviors. Flexible software contributes to a business’ agility, and that’s good, but that speed and flexibility need to be balanced by security and what I can only call the “bulletproofness” of the new or changed code.
The businesses most able to reap the rewards of DevOps tend to be smaller, though large enterprises have their bragging points. While larger businesses already see benefits from a DevOps strategy, they are the ones with the greatest potential to do more. What’s holding them back?
In any organization, size breeds complexity, which causes business friction. We don’t have all the data to say so unequivocally, but it seems that bigger organizations have more walls to break down.
It looks to me like the development tools are pretty good. Not enough businesses have well-integrated management suites to handle the complexity, and it also seems like culture forms stovepipes, which causes less-stellar performance.
If that’s so, there’s still some cultural work to be done enhancing communications within and between developer groups and the business. DevOps tools can be a big part of that help, but as with the psychiatrist trying to change a lightbulb, the bulb still has to want to change.
Loading ...
