Here at Scale, we lead a series of meetups for our portfolio companies, bringing together executives to dive into a specific topic and share ideas. It is useful for the individuals to accumulate that collective wisdom and gives us insight into pain points and successes emerging in the technology landscape today. We recently kicked off our first VP, Engineering and VP, Operations community meetup, and our first topic: New Demands on the Software Delivery Pipeline.
As consumers, we are the drivers for the change, demanding web and mobile applications to be of the highest quality, to render on-page or in-app with lightning speed, and to cater to our unique likes and behaviors and not those of the generic masses. It is not enough to deliver the same static content throughout the day, let alone for several days.
But, what is going on in the background to affect this rapid evolution? This very question is what motivated us to invite Adam Jacob, the CTO and co-founder of Chef and Roy Rapoport, DevOps guru and Manager of Insight Engineering at Netflix, to spearhead discussion about the quickening pace of the software supply chain. Present to discuss were heads of Eng and Ops from JFrog, CircleCI, Stormpath, Pantheon, Treasure Data, PubNub and Agari, amongst others.
What followed was not quite what I expected. Tech press would have you believe that everyone is after #GIFEE, standardizing on Docker, orchestrated by Kubernetes, and worry later about any workloads requiring state. This was distinctly not part of the discussion that ensued. What follows is a snapshot of the core ideas discussed:
- Software’s ability to drive change is a bonafide macro-level trend and business leaders are motivated to use software to drive automation. It used to be that software vendors (i.e. Chef) had to explain the merits of automation before explaining how their software could help. Today, enterprises with a digital presence are cognizant they have to move fast or risk irrelevance. “DevOps” is still not universally understood but the outcome from adopting a devops culture is understood. Maybe this is enough?
- The constructs are vastly different whether you are starting from scratch today vs. improving on an existing software delivery pipeline. You can liken this to the Google vs. Tesla approach to self-driving cars: from scratch vs. incremental. If we could wipe out all vehicles on the road today, Google’s very quantified approach to self-driving cars would be the optimal choice. In the absence of that option, Tesla’s incremental approach, fitting into the current ecosystem and adding another 1M miles of driving every 10 hours, works quite well. This is a very real consideration in devops today. If you started from scratch, yes, adopt all of the shiny new objects. If you’re mid-stream and looking for incremental gains, the interdependencies get more complicated.
- Lots of the nuance in DevOps comes from cultural and behavioral changes in addition to the technology changes. Many articles, even books, have been written about this, but it remains true. If starting an organization from scratch, or spinning out a focused team on DevOps and microservices, think about the behaviors you want to reinforce and build the software pipeline around that, be it code review, thorough testing, or avoidance of the ad hoc. Decide in advance the tradeoffs: is the speed to deploy more important than comprehensive testing? What is the metric by which you measure?
- Netflix is notorious for being a snowflake and tending toward building vs. buying, but Roy shared that the build vs. buy decision will be – and should be – different nearly anywhere else. Netflix has originated many OSS projects, Spinnaker and Aminator to name a few, but that was an output of custom software they wrote internally to fit their exact needs. On the infrastructure side they are the opposite, standardized on AWS to eliminate the need to manage infrastructure. Most in the group agreed that the argument to ‘buy’ is getting stronger as the tools available are improving.
- The limitations of releasing code with velocity are well documented for monolithic applications; what emerged was microservices. Microservice architecture does ameliorate the velocity problem but over time the spread of microservices surfaces other problems. Adam and Roy brought to bear several problems that emerge in an organization over time: How many microservices are too many to handle? What happens when the author of a discrete microservice leaves the company unexpectedly, without leaving behind the related wisdom? Do existing monitoring tools deliver enough granularity or do you have to augment the monitoring tooling?
- The mention of AWS returned simultaneous reactions of love + hate. In the eyes of many developers, the rise to $10B+ is now the very reason to question. Amazon provides a much-needed and very efficient service, but is it too costly and is the slippery slope into lock-in too real? But, are any competitors yet at parity, to induce the conversation to switch? Amazon continues to release new functionality to great fanfare in the market (see Lambda for details), but the advice was clear: be sure to conduct a thorough POC before using in production.
We addressed pertinent questions but for each answered, two more arose. We look forward to hosting more meetups with this insightful crew and tackle more issues facing DevOps today.
