How to turn DevOps into DevSecOps

DevSecOps

Information systems require that development and operation be closely linked and that a high level of security be maintained.

Figure 1. What is DevSecOps ?

In order to maintain a high level of security in modern times, high adaptability based on high aspirations is essential.
While the points about Development x Operation are left to other articles, this article explains the two axes: Development × Security and Operation × Security.

Development × Security

Development × Security can be rephrased as creating secure deliverables in a secure development environment using a secure development method.
A secure development environment is, for example, a development terminal built with a zero-trust philosophy and a build pipeline that enables SLSA and CIS SSCS.
A secure development method refers to a safe development process, such as a source code merge flow.
By secure deliverables, we mean that the resulting system is secure.

Secure Development Environment

The development environment is created using multiple SaaS, including a source code repository, library repository, CI/CD pipeline, ticket management system, observability tool, security checking tool, and automated testing tool.
It is important to integrate these SaaS accounts using OIDC and SAML and to establish a mechanism to always check them.
In other words, deploying MDM, EDR, and Zero-Trust Network Agents to constantly check that users, devices, and communications are in a secure state and using SLSA tools and reports, CIS Supply Chain Guide and benchmarks to check that your build pipelines are in a secure state will enable a secure development environment.
It is difficult to prepare these from scratch for each project, so it is desirable to prepare them in common as an internal infrastructure.

Secure Development Method

In the development process, there has been a shift from the era of development resource efficiency to the era of development flow efficiency.
As a result, it has become more important to provide reviewers and other experts with extra capacity, in other words, to eliminate bottlenecks.
While confirmation by experts is effective in improving quality, the need for experts is increasing, and it is becoming increasingly difficult to secure experts.
How to reduce the viewpoints that experts must confirm and how to make confirmation easier are very important when considering flow efficiency. In other words, it is important to automate as much as possible where it can be automated, consolidate necessary information, and ensure quality that does not rely on human resources as much as possible.

Another important idea is multilayering. To paraphrase, there are always holes in human behavior, so perfect protection and perfect reporting cannot be expected by a single means. This does not only mean that security products should be combined and defended in multiple layers, but it also means that there are holes in reviews and diagnoses by experts and specialized tools. As development processes have changed to focus on flow efficiency and accept uncertainty, it is increasingly important to accept uncertainty and work on security in multiple layers in both defense and process aspects.

Secure Deliverables

A major change in recent years is that development deliverables are now commonly connected to other systems.
In modern times, systems do not stand alone. It is common to use various OSSs, SaaS, and API linkages, and supply chains are becoming more complex, making it difficult to control the entire system.
Until now, when seeds of vulnerability, which are vulnerabilities that do not appear, were found, they have been overlooked as having no immediate problem.
With the changing times, uncertainty has increased, and it has become impossible to predict when and how seeds of vulnerability would blossom into a critical vulnerability due to changes in circumstances outside our control. The small seeds of vulnerability can quickly become large just like a snowball rolling down a hill and become a hindrance to progress later on.
Therefore, we recommend that you create a mechanism that cannot be ignored by setting up the build pipeline to abend, etc., and that all warnings reported by various automation tools can be resolved while they are still small.

In addition, to improve the security quality of a system, it is very important not only to take measures such as writing a security design document, introducing SAST/DAST/IAST/agent-based vulnerability management tools, and undergoing security diagnostics, but also to enrich test codes to check the impact of revisions. Therefore, we recommend that you steadily do implementations recommended in DevOps, such as implementing TDD/BDD.

Figure 2. Points of Development x Security

Operations × Security

The primary purpose of Operations × Security is to continue providing secure systems.
Every day, systems continue to grow in complexity outside the developer's control, vulnerabilities continue to be discovered, and attacks continue to escalate and become more sophisticated.
In the IT industry, we hear the screams of "I didn't do anything and it broke", but security is always broken if you don't do anything.
In the modern IT scene, doing nothing does not mean maintaining the status quo. It means being swept downstream at an unstoppable speed.

In order to continue providing secure systems, it goes without saying that security operations designed with reference to frameworks such as CSF are important. Here, we consider security patching as a representative example of common concerns.

What are the problems when trying to apply security patches?

• Do not know what kind of security patches to apply
• Unaware of what kind of tests should be applied for security patches to be used in a commercial environment

First, introducing agent-based vulnerability scanners, SCA/SBOM, and moving away from the idea of identifying the minimum required security patches to the idea of applying all the mechanically extracted security patches can be a big improvement. This eliminates human judgment and can greatly improve the workload.

The second problem is difficult to solve. It is important to note that many security improvements, including applying security patches, can cause problems that are often found only in integration and system tests. Generally, test cases for integration and system tests are created from designs and requirements. However, since security improvement does not include design and requirement changes, IT engineers have a very hard time creating sufficient test cases.

There are two points to address this problem.

The first point is to automate not only unit tests but also integration and system tests. By automating not only the test run but also the setup of test data, many integration and system tests can be automated.

The other point is to provide a period of rest except for emergency security fix. For example, the overall safety accuracy can be improved by patching a staging environment two weeks to a month before patching a commercial environment and having the person in charge of the application or the ordering company conduct daily tests in that environment. These can be easily achieved by distributing a security fix applied OS image or by using a snapshot repository.

By using these ideas and methods, non-functional improvement work and application development work are relatively loosely coupled. As a result, security patching can be performed in a high cycle, and as a side effect, maintenance work such as refactoring can be safely performed.

Figure 3. Points of Operation x Security

Summary

As we have explained, the following five points are important in order to continue to provide secure systems in a safe cycle.

• Organizational preparation of zero-trust development terminals and secure CI/CD pipelines, and to continue to raise the bar in terms of governance, security, and cost.
• At a time when the emphasis is on development flow efficiency, it is important to design processes and organizations from a security perspective, taking care that these are designed to eliminate bottlenecks in the flow.
• A mechanism to passively detect vulnerabilities in various layers such as applications, dependent libraries, OS, cloud settings, etc. should be introduced to minimize the effort required for detection.
• Design the overall process with the assumption that refactoring/patching and other maintenance work will be done in a high cycle.
• Design and agree in advance what risks are acceptable and to what degree, and what regression tests must be passed before environment changes or deployments can be made.

In order to continually provide a secure system, it is important to ensure repeatability and continuity. Conversely, it is important to eliminate elements that prevent repeatability and continuity as much as possible.

I hope this article will be of help to you in the stable operation of your system.
Thank you for reading to the end.