Category Archives: DevSecOps

Introduction to the Linux Foundation’s SLSA project

Posted on by Leave a comment

Supply Chain Security is a hot topic these days. And more and more, we as developers are dealing with this daily. But what does this mean for us, and how is this influencing our job? I want to give an overview of common attacks against the Software Supply Chain from the developer’s view and will introduce the Open Source project SLSA from the Linux Foundation. 

a) Who is the project SLSA

Various experts from the security field started this project to share their knowledge, leading to the mentioned project. There is no company or governmental organisation behind this project. It is a pure Open Source project under the umbrella of the Linux Foundation. 

b) What is the goal of this project?

The SLSA project is open-source and will not provide its own source code. It is, therefore, not a classic open-source project to publish a specific solution. Instead, it is a documentation project to process knowledge about supply chain security in software development and make it freely accessible. The structure contains examples of where the events or attacks mentioned have been successfully taken and which countermeasures should be taken. We’ll look at the security levels provided a little later. The aim here is to allow the reader to prepare for these threats with specific steps slowly. 

After all, based on your options and the current environment, you have to decide which next steps and measures make sense to implement. 

c) What is the current status of the project?

The project is currently (beginning of 2022) still in the alpha phase. There is a lot of documentation, but some places still provide references to future content. So it’s always worth taking a look inside. Since this is an open-source project, you can, of course, also contribute yourself. Here you have the opportunity to make your expertise available to others. 

SLSA security levels?

a) Why do we need these security levels?

When you look around the SLSA project website for the first time, the description of the security levels quickly catches your eye. But what are these levels all about? With these levels, you want to achieve two things. First, it should help to enable a self-assessment. This first consideration serves to classify the own existing environment and to identify the primary weak points. The second goal of this level is to help you plan the following purposes that should be implemented to increase your protection significantly. These recommendations are based on evaluating many projects and environments and on the security experts’ knowledge. This enables you to use the available means and opportunities to increase your security. 

b) Level 0

The first level mentioned is marked with the number zero. This is the state in which no action has yet been taken. The documentation that describes the entire build process must be available and up-to-date to reach this level. This enables an initial analysis of the environment to be secured. It can also be seen as a way of taking stock.

c) Level 1

The level with the number one requires first actions on the environment. The point here is to list all the components used to create a binary file. Not only are the dependencies used listed, but all other metadata is recorded if possible. There is the term SBOM, Software Bills Of Materials in the industry. However, the term can be broadened considerably here. Within JFrog Artifactory, there is “build info”, which means a superset of SBOM. All sorts of metadata are recorded here, such as the operating system and version of the build machine, environment variables and their contents, etc. I have created my video for this.    

In this video, I’ll go over the details of SBOM and Build Info.

Within the SLSA project, the term SBOM or Build Info is not explicitly mentioned. Instead, the requirement is described more generally, and it is also clearly pointed out that this level does not yet help against the compromise itself. It is only the basis for the follow-up and the comprehensive risk assessment. Based on this level, one can start with vulnerability management.

d) Level 2

Level two describes code versioning software in combination with a hosted build service. With this combination, you want to trust the build service and thus trust the origin of the generated files. Here you must take a critical look at where and who will offer these services. The fundamental consideration is based on the assumption that a specialized provider has a better grip on the issue of security within the build infrastructure than you can guarantee yourself. However, the manipulation of protection of the build service can be increased in various ways.

e) Level 3

With level three, auditors are now included in the considerations. Here, trained security specialists are used to check the existing build environment for possible vulnerabilities. As part of an accreditation process, the auditors determine whether specific standards are met to ensure the verifiability of the source and the integrity of the provenance (origin). This level offers better protection than all previous levels to ward off well-known threats. Preventing cross-build contamination is an example.

f) Level 4

In the SLSA project, level four is the highest currently achieved. To reach this level, the following conditions are set. The first requirement describes the necessity that two people constantly carry out all changes. This does not only refer to the changes made to the source code. The term is broader here and also includes all changes to the system and the configurations. This ensures that no intentional or unintentional compromise can occur at any point. Such a process has prevented most bugs, vulnerabilities and compromise attempts.

The second requirement relates to the technical environment. It is assumed here that it is a hermetic and reproducible creation process. However, a distinction should be made between the requirement for hermetic builds and reproducible builds. The former guarantees that all required dependencies are known and checked for vulnerabilities. Here, of course, it plays a role from which sources these dependencies were obtained. On the other hand, reproducible builds help detect changes and conduct subsequent analyses. However, the reproducibility of a build is not a mandatory part of the fourth level.

g) Limitations

Even if many supply chain threads are addressed with the SLSA project, there are some limitations that I would like to list here.

1) The list of all dependencies in a project can simply become too large to be able to fully record and evaluate them. That can depend on different things. The point that shows this most clearly is based on the possibility of some dependency management systems not only specifying explicit dependency versions. In some scenarios, you can also define a version range. This now results in an exponential increase in all possible version combinations.

2) The number of all components to be backed up can exceed the available capacities. The teams then have to decide which elements are subjected to safety analysis.

3) The SLSA security levels are not transitive. This means that this component itself consists of components that have a lower security level. So here, you have to pay very close attention to the details.

4) Even if automation is the solution in many areas, there may be areas where this will not be possible. Again, this refers in many areas to closed-source projects. Here, some analyzes are simply not feasible. It must then be decided how this dependency will be classified on a case-by-case basis.

What are the requirements? 

All the requirements mentioned here are a little more detailed in reality. An overview of the requirements was provided to guide what exactly is meant by each point. Here, you can see which condition belongs to which level and how this is understood. Then the details are discussed, such as what difference it makes if you only use a building service or if you also use the “Build as Code” strategy. All the details are too extensive here, so I refer to the section on the project’s website.  

Supply Chain Threads

a) Overview 

To better understand the project and its limitations, you should look at the individual types of attacks on a typical software development environment. A distinction is generally made between source, image and dependency threads. The SLSA project also delimits precisely what it includes and what it does not.

The following overview comes directly from the project and will be further developed if necessary. At this point, one must not forget that this project is only in the alpha phase (early 2022). However, these types of attacks mentioned are timeless and apply to almost every development environment.

Let’s get to the source threads. This type of attack aims to compromise the source code itself. 

b.1) Bypass Code Review

The attack called “Bypass Code Review” describes a scenario in which an attempt is made to bypass the control mechanisms and directly add compromised source code to the project. Here public paths are taken to reach the destination. Such attacks were carried out on the Linux kernel, for example, via the mailing list. The best and probably simplest way to thwart such attacks is to have all source code changes checked independently by at least two people. However, at this point, one must also find ways to prevent mutual “covering up”.

b.2) Compromised Source Control System

If importing compromised source code does not work, you can try to attack the CVS itself. Attempts are being made here to make changes to the source code on the server, bypassing the official channels. A prominent example is a successful attack on the source code of PHP. The self-managed system was attacked, and the changes were imported directly as a commit. Only securing the system helps against this type of attack. To manage such a system yourself, you must have the necessary resources and knowledge. In this case, it is often easier and cheaper to use a managed system with Github.

Next, we come to the build threads.

b.3) Modified Source Code after Source Control

The source code cannot only be changed where it is stored. When a Build process is started, the source code is fetched from the specified source. The source text on the build page can also be modified before the translation process starts. One attack in which this approach was used is the “Webmin” project. Here, additional and modified source texts were injected into the translation process. The agents responsible for carrying out the individual build steps must be secured to prevent this attack. Here, too, one has to weigh up how much of this work is carried out by oneself since how safe such an environment depends primarily on one’s own abilities. 

b.4) Compromised Build Platform

Since the built environment is part of the value chain, this part must also be secured classically. Here, too, it is again the case that one’s own abilities in ​​safety must be assessed. A well-known representative of this type of attack is the Solarwinds Hack. Here, the build route was modified to compromise the newly created binary with each run. 

b.5) ​​Using Bad Dependencies – Dependency Thread

At this point, I would like to insert the dependency threads briefly. Dependencies are specifically modified and then offered. The security of the repositories plays a central role here. I’ll come back to that in a moment. However, this attack pattern refers to external locations where dependencies are fetched to process them. Particular attention should be paid to assessing the integrity of caches and mirrors of known official bodies. Often the “official” mirror servers of publicly accessible repositories do not have the same financial resources as the original. This causes attacks to be placed there.

You may be offered modified versions of known commonly used dependencies when you dock at such a place.

b.6) Bypassed CI/CD

Let’s get back to the build threads. Sometimes it is easier to bypass the CI/CD environment at an appropriate point. Here, the topics within a development path are selected where a transfer between the build server and the associated repositories is not sufficiently secured. The attack involves offering a recipient such as a repository a modified binary to make it look like it is from the CI environment. Malicious code is injected parallel to its own build infrastructure. 

b.7) Compromised package repo

A component repository is also part of the infrastructure and can fall victim to an attack. As with all other elements, protection is necessary here. Attackers try to gain access to the repositories to replace the binaries of known and frequently used dependencies with modified versions. Depending on your ability to harden such a system against attacks, you have to decide whether an externally managed alternative will increase the security standard of your value chain.

b.8) Using a bad package

We come to the last attack variant mentioned here. In this scenario, known dependencies are taken as a template and then their own modified versions are offered in a public repository. Here, an attempt is made to give the own variant a name close to the original name and depicts typical spelling errors. This dependency can still be resolved if a misspelling of the original name is used in a version definition. Only in this case a modified version of it will be loaded.

The only way to protect yourself against this attack is to carefully check how each individual dependency has been defined in your own projects. 

Project Persia Overview

We have seen which general types of attacks on a classic software development environment are possible. These are always generic attack patterns that can be used independently of the current products. Others also saw this as an idea to develop a general strategy to counter this. The language here is from the open-source project Persia, which is currently (early 2022) in the alpha phase. We are close to officially accepting it as an incubator in the CD Foundation. The project website is https://pyrsia.io.

The basic idea of ​​this project is that there will be a network of trust for open-source projects. A P2P network consisting of build and distribution nodes should exist where the sources are fetched, built and offered decentrally. In addition to the classic advantages such as reliability and better utilisation of the network resources of a P2P network, increased security should also be achieved with the binaries themselves. The types of attacks on source code management software are excluded. But all subsequent attack patterns can be repelled in this way. 

Unfortunately, it is not possible to fully describe the project. But at this point, I would like to draw attention to my video about this project. (https://pyrsia.io/) 

Cheers Sven

The Power of #JFrog Build Info (Build Metadata)

Posted on by Leave a comment

Intro

This article will take a detailed look at what the term build-info is all about and why it will help us protect against attacks such as the Solarwinds Hack. 

What is the concept behind the term – build-info?

Let’s start at the very beginning and clarify the basic principle behind the term build-info. The term build-info has been coined for many years by the company JFrog, among others. This is a particular type of repository.

This repository stores the information that describes the context that led to the creation of a binary file. With this information, you can now achieve a wide variety of things.

What components does build-info consist of?

The content of a build-info is not strictly defined. Instead, the approach that applies is that the more, the better. Of course, you have to proceed with caution here too. All possible parameters are collected. In addition to the date and time, the system on which the process was run, which operating system was used in which patch level, to active environment variables, compiler switches and library versions.

The challenge is actually that it is not known which information will later be helpful and expedient. For this reason, more rather than less should be saved.

Why do we actually need a build-info?

The task of a build-info is to enable the observation, or rather, the analysis of a past situation. There can be a variety of reasons for this. For example, it can be used to improve quality, or it can be the basis for reconstructing a cyber attack that has taken place. And with that, we come straight to the event that got everything rolling in the recent past.

Trigger – SolarWinds Hack

One of the others will have heard or read something about it. We are talking about one of the most significant cyberattacks that have ever taken place. It’s the SolarWinds Hack. Here it was not the final target that was attacked directly, but a point in the supply chain. SolarWinds is a software company that provides a product for managing network infrastructure. With just over 300,000 customers worldwide, this software’s automatic update process has been the target of the attack. It was not the update process itself that was compromised, but the creation of the binaries that will be distributed with this update process. The attack took place on the company’s CI route to immediately infect the newly created binaries with each build. Here the CI route was manipulated so that another component was added to the binary to be generated. This component can be thought of as a kind of initial charge. As soon as this has been ignited or activated, further components are dynamically reloaded. As a result, each infection had different forms. These files were then offered to all customers by means of an automatic update. Thus, over 15,000 systems were infiltrated within a short time.

Reaction – Executive Order of Cybersecurity

Since there were many well-known US companies, US organizations and US government institutions among the victims, the question arose of how to counter such a threat from the US state in the future. The US government has decided that one begins with the complete cataloguing of all software components in use, including all their constituent parts. This obligation to provide evidence was formulated in the “Executive Order of Cybersecurity”. However, when I first heard about an executive order, I wasn’t sure what that actually meant.

What is an executive order?

An executive order is a decree of the US President that regulates or changes internal affairs within the state apparatus. You can think of it as a US president like a managing director of a company who can influence his company’s internal processes and procedures. In doing so, no applicable law can be circumvented or changed. With such an executive order, no law can be changed, stopped or restricted. But it can change the internal processes of the US authorities very drastically. And that’s exactly what happened with this Executive Order of Cybersecurity, which has directly impacted the US economy. Every company that works directly or indirectly for the state must meet these requirements to continue doing business with the US authorities.

What is the key message of the Executive Order?

The Executive Order of Cybersecurity contains a little more text, the content of which I would like to shorten here and reproduce without guaranteeing legal correctness.

This arrangement aims to record the software operated by or for the US authorities in its entirety. This means that all components of the software used must be documented. This can be thought of as follows; If you want to bake a cake, you need a list of ingredients. However, listing these would not meet the requirements, as knowledge of all elements down to the last link is required. In our example, an ingredient that consists of more than one component would have to provide a list of all existing parts. This is how you get all the components used that are needed for the cake. If we now transfer this to the software, it is necessary to record all direct and indirect dependencies. The list of all components is then called – SBOM (Software Bills Of Material) 

And what does that mean for me?

Now I am not directly working for an organ of the US government. But it will still reach me sooner or later. Because through the indirection, all possible economic sectors worldwide will be affected. It can also affect me in the private sector. If I have an open-source project that is used indirectly in this environment, it can only continue if I prepare the project accordingly. Long story short; It will come our way in whatever way it will happen.

What are the general requirements for a build-info?

Let’s get back to the build-info. A build-info is a superset of an SBOM (Software Bills Of Material). So the dependencies are catalogued, and additional information from the runtime environment in which the binary is generated is recorded. However, there are also some requirements for such build-info. By this, I mean properties that must be present in order to be able to deal with this information in a meaningful way.

Accessibility:

The information that is collected must be quickly and easily accessible and therefore usable. A central storage location such as Artifactory is suitable here, as all binaries and dependencies are also stored here. You also have all the meta-information of a respective binary file at this location.

Immutability:

When information is collected, it makes sense to store it in such a way that it can no longer be changed. This promotes the acceptance of the data situation and gives particular security when evaluating a case.

Combinability:

The static data are also suitable for enriching them with current secondary data. In this case, I am talking about the vulnerabilities that can be found in the dependencies used. In contrast to the static information, this image has to be constantly updated. In practical use, this means that the data must be stored in a machine-readable manner in order to enable the connection of other data sources. In the case of Artifactory, it is the data from JFrog Xray that is displayed here.

How can a build-info be generated?

It would be best if you had the following things.

First, the JFrog CLI generates the build-info and second, it is possible to save this information permanently. You can do this very quickly with the Build-Info Repositories from Artifactory. But one step at a time.

The process for generating a build-info is as follows.

In the following example, we assume that we are going to build a Java project with maven. Here you have to get involved in the build process. You can do that quite easily with the JFrog CLI tools. These represent a wrapper for the build process used in each case. In our example, it is maven. To install the JFrog CLI Tools, you can go on the website https://jfrog.com/getcli/ choose the installer that is suitable for it. After installing and configuring the JFrog CLI Tools, you can start the build via the CLI on the command line. After a successful build, you can find the build-info as a JSON file in the target directory. This file can be viewed with a text editor to get a feel for the wealth of information that is stored here. This file is regenerated with each build. This means that there is a 1: 1 relationship between the build cycle and the binaries generated in the process. Using the CLI, you can then transfer this information to the Artifactory build repository and evaluate it there using a graphical user interface.

How can I link vulnerability scans to the build-info?

If you are now within Artifactory (https://bit.ly/SvenYouTube ), look at the build-information; you also get the current information regarding the known vulnerabilities. This is an example of how the build-information can be enriched with further system information.

Since there is not enough space at this point to clarify the generation and evaluation, I refer to a demo project in which I have stored the steps in the README, including detailed explanations of the individual practical steps.

If you want, you can also register for one of my free DevSecOps workshops. You are also welcome to speak to me directly if you’re going to do this in a free community or company event.

The URL for the demo project is: 

https://github.com/Java-Workshops/JFrog-FreeTier-JVM

Have fun trying it out.

Cheers, Sven! 

SolarWinds hack and the Executive Order from Mr Biden — And now?

Posted on by Leave a comment

In the past two years, we have had to learn a lot about cybersecurity. The new attack vectors are becoming more and more sophisticated and are directed more and more against the value chain in general. But what does that mean for us? What can be done about it, and what reactions have the state already taken?

Let’s start with the story that got all of this rolling and made sure that the general attention was drawn to the vulnerabilities of the available IT infrastructure. We’re talking about the SolarWinds Hack. What happened here, and what is more critical; What are the lessons learned from this incident?

It is essential to know that the SolarWinds company produces software that is used to manage network infrastructure. With the name “Orion Platform, ” this software should help manage and administer the network components efficiently. If you look at the product description on the SolarWinds website, you can read that the software can monitor, analyse, and manage the network infrastructure. And that’s exactly what the customers did. The company itself has around 300,000 customers worldwide, including individual companies and government organisations and corporations from a wide variety of areas. To always stay up to date, the software platform includes an automatic update mechanism. And that was exactly what the attackers were after. They saw in the SolarWinds company a multiplier for their own activities. But how can you go about this? The attackers obtained the necessary software tools by breaking into the FireEye company and infiltrating the SolarWinds network. The goal was the software development department in which the binaries of the Orion platform are created. Here, the CI routes have been manipulated to include compromised binaries in every build of the software. As a result, the company produced these binaries and put them into circulation through the automatic update process. In this way, around 18,000 targets could be infiltrated and compromised within a short time.

What does that mean for us now?

There are two angles here. The first is from the consumer’s point of view. However, for your own production, it must be ensured that no compromised binaries are used. That means that all, and here I mean all binaries used, must be checked. These are the dependencies of the application and the tools used in the production process. This includes the tools such as the CI server used, e.g. Jenkins, or the operating system components of the infrastructure and Docker images.

The second perspective represents the point of view that one has as a software distributor. This means anyone who distributes binaries in any form. This includes customers in the classic sense and other departments that may be considered consumers of the binaries created. The associated loss of trust can cause a company severe financial damage.

What can we do to protect ourselves?

If you look at the different approaches to arm yourself against these threats, you can see two fundamental differences.

DAST – Dynamic Application Security Testing

An application can be subjected to a black box test. This is called DAST, Dynamic Application Security Testing. Here you go the way a hacker would do it. The application is attacked via the available interaction points. Attack vectors based on the most common vulnerabilities are usually used here. SQL injection can be cited as an example. These attack patterns are initially independent of the technology used and represent typical security holes in web applications.

This approach has advantages as well as disadvantages. A serious disadvantage is that the tests cannot be started until the application has been developed. This means that the measures to eliminate weak points found are more costly than is the case with other approaches. On the other hand, a significant advantage of this approach is that the dynamic context can also be checked when testing the production system. Examples include weak configurations, open ports, and more.

SAST – Static Application Security Testing

The counterpart to the dynamic tests is the static tests. These tests analyse the individual components of the infrastructure involved. This includes not only the applications created but also the components involved in operating and creating the application.

A disadvantage of the SAST method is that it can only search for known security vulnerabilities. However, this point is put into perspective in that the number of known security gaps increases steadily. Still, it is also assumed that the number of known security gaps exceeds the number of unknown security gaps. Another advantage of the SAST method is that it checks the licenses used. This area also contributes to the security of the company, albeit in a legal sense.

So what has the most significant effect when you start with the topic of security?

When looking at the different approaches, it becomes clear that the proportion of directly or indirectly used binaries makes up the most significant amount in a technology stack. This means that you should also start securing precisely these parts. Unfortunately, there is only one way to ensure with SAST that it is really 100% checked directly. All other methods do not achieve this coverage, or if they do, then only in sporadic exceptional cases.

However, it should be noted that one understands the interdependencies. This is the only way to identify and extract the possible attack vector. Here it is of essential importance that not only the binaries themselves are examined, but also the associated meta-information is evaluated. Within software development, there are two points for these analyses where such an analysis fits very well. The first point that most people will think of is the CI segment. Here the CI server can carry out the analysis during the execution of a build pipeline and all the checks and also take over the reporting. If the violations are too massive, the build is cancelled. The second position, which is ideally suited for such an analysis, is directly in the IDE. Here it can already be determined during the definition whether the license used is permitted. The known vulnerabilities can also be extracted and thus provide enough information to decide whether this version of this dependency may be used.

Lesson Learned – what we learned from the SolarWinds Hack.

We learned the following things from SolarWinds. First, the attacks are no longer aimed at the final targets themselves but against the suppliers. Here one tries to reach multipliers who massively increase the effectiveness of this attack. Likewise, you no longer only have to protect yourself against your use of compromised binaries. The newly added component means that the self-generated binaries must also be secured. Nobody wants to become a distributor of infected software. It follows that all parts involved must be subjected to permanent checks.

Reactions – The Executive Order from Mr Biden

One or the other has undoubtedly noticed that the US President gave this Executive Order of Cybersecurity. But what exactly does that mean, and who can even do something like that under what conditions?

In the United States, the president – the “executive branch” of the US government – can issue what is known as an executive order. Much like a CEO in a company, the executive branch has the power to make some unilateral decisions that affect the behaviour of the US government. These orders must be limited to government conduct and policies and not to general laws or statutes that encroach on the rights of any individual or state. If the executive transgresses in this area, the US Supreme Court or other legal remedies can appeal and revoke all measures. Only the president – due to his status as head of state – can issue an executive order. No other officer is allowed to do this.

Significantly, executive orders are mostly limited to how the US government operates internally. For example, an executive order might not simply resolve to raise taxes in a state or issue far-reaching commercial guidelines. However, executive orders are legally binding on the behaviour of the US government if they do not violate fundamental rights or make something unconstitutional. Because of this, these commands can be found to change the policy from the president to the president as, like a CEO, they set some of the government’s agendas. In this case, the Executive Ordinance refers to how the US government monitors and regulates its software usage and sets out requirements for how that software must be documented to be sold to the government. It doesn’t dictate trading on a broad scale.

Since the US government can define its way of working, it was decided that the SBOM requirements (in preparation) will be the only way to approve software purchases – to get an idea of ​​the volume, you have to know it is one amount in the tens of billions per year. This means that any company that wants to sell to US government agencies, likely state and local agencies, and anyone who works with resellers who sell to the government and more, must meet all of these new standards.

When the European Union agreed on comprehensive data protection and data security laws with the GDPR, the effect was felt worldwide and quickly. In a globally networked economy, decisions by the world’s largest GDP producers can set standards for other areas. With the GDPR, companies in the US and elsewhere had to immediately revise and update software and processes to continue doing business in the EU. It became a worldwide rule so as not to lose EU treaties. Likewise, the US cybersecurity order is believed to be reversing, with U.S.-based global corporations influencing global software security behaviour.

Wow – what now?

What does that mean for me as a software developer? Well, the effects at this point are not as severe as you might want to imagine. In the end, it is said that a complete list of all contained dependencies must be created. This means the direct and indirect dependencies. To obtain such a dependency graph, one can, for example, use the tools that already do this. It means the tools that can generate a complete dependency graph, for example. For this, you need a logical point through which all dependencies are fetched. A tool such as Artifactory is best, as this is where the knowledge of all dependency managers used comes together. Devices such as the vulnerability scanner JFrog Xray can then access this information, scan for known vulnerabilities and licenses used, and extract the entire dependency graph. This information can thus be generated as build information by the CI route (e.g. JFrog Pipelines) and stored in a build repository within Artifactory.

Conclusion

We have seen that the new qualities of attacks on software systems, in general, will lead to advanced countermeasures. What is meant here is a complete analysis of all artefacts involved and produced. The tools required for this must combine the information into aggregated impact graphs across technology boundaries. One point within software development that is ideal for this is the logical point via which all binaries are fed into the system. However, the components alone are insufficient since all indirect elements can only be reached and checked by understanding the associated meta-information.

The tools used for this can then also be used to create the SBOMs.

In my opinion, it is only a matter of time before we also have to meet this requirement if we do not want to suffer economic disadvantages from the Executive Order of Cybersecurity.

If you want to find out more about the topic or test the tools briefly mentioned here, don’t hesitate to contact me.

In addition to further information, we at JFrog offer free workshops to prepare for these challenges in a purely practical manner.

And if you want to know more immediately, you are welcome to go to my YouTube channel. There I have other videos on this topic.

It would be nice to leave a subscription on my YouTube channel as a small thank you. https://www.youtube.com/channel/UCNkQKejDX-pQM9-lKZRpEwA

Cheers Sven

What is the difference between SAST, DAST, IAST and RASP?

Posted on by Leave a comment

Intro:

In this post, we’re going to look at the differences between the various cybersecurity defence techniques. Here you can identify four main groups, which we will go through briefly one after another to illustrate the advantages and disadvantages.

SAST – Static Application Security Testing

SAST describes the process in which the components of an application are subjected to a static analysis. This approach not only searches for security gaps but also determines the licenses for the individual elements. In the following, however, I will only deal with the consideration of vulnerabilities in this post.

The term static comes from the fact that only the static context is used for this evaluation. All dynamic or context-related assessments are not possible with this method. SAST can be used to analyze the entire tech stack of an application if access to these components is possible, which is also one of the advantages of SAST compared to other analysis methods. The only exception is the IAST procedure, which we will come back to later.

What exactly happens now with the SAST procedure? SAST is available in two forms. The first that I would like to mention relates to checking your source code. Here the attempt is made to determine whether there are patterns that make a later vulnerability easier or even possible in the first place. For example, it searches for calls to critical libraries or identifies unsafe handling of resources. However, one honestly has to admit at this point that, first of all, your source code will be by far the smallest part of the overall application in the vast majority of projects. Second, the analysis methods in this area are still at a very early stage of development.

When using SAST, it makes a lot more sense to deal with all the other components first. Therefore, all binary files of the entire application environment are meant here. This also includes all elements that play a role in the development, such as the CI server and the other tools used.

DAST – Dynamic Application Security Testing

The DAST analysis method does precisely the opposite in comparison to the SAST method. Here the application is viewed as a black box. As soon as an executable version of the application is available, it is attacked through automatically executed cyber attacks. In most cases, the general attack patterns are based on the Most Common Vulnerabilities defined by the OSWAP. Defining your attack vectors is either not provided for in the tools I know or requires very detailed knowledge in this area. For developers who do not use these tools daily and intensively, their use is limited to the predefined attack vectors. It is a good thing with this procedure that unknown security gaps can also be identified, as long as they are based on the Most Common Vulnerabilities. Some manufacturers extend this approach with AI or ML techniques. This approach using AI and ML techniques is still in a very early stage of development, and the currently available potential cannot be foreseen.

Unfortunately, the method can only be used sensibly against the production environment, as this is the only way to identify all gaps based on configuration deficiencies. Therefore, in contrast to the SAST process, it makes no sense to use it within the CI route.

IAST – Interactive Application Security Testing

IAST uses software tools to evaluate application performance and identify vulnerabilities. IAST takes an “agent-like” approach; H. Agents and sensors run to continuously analyze application functions during automated tests, manual tests, or a mixture of both.

The process and feedback occur in real-time in the IDE, Continuous Integration (CI) environment or quality assurance or during production. The sensors have access to:

  • All source code
  • Data and control flow
  • System configuration data
  • Web components
  • Backend connection data

The main difference between IAST, SAST, and DAST is that it runs inside the application.

Access to all static components as well as the runtime information enables a very comprehensive picture.

It is a combination of static and dynamic analysis. However, the part of the dynamic analysis is not a pure black-box test as it is implemented at DAST.

Some reasons for using IAST:

Potential problems are identified earlier, so IAST minimizes the cost of eliminating potential costs and delays.

This is due to a “shift left” approach, meaning it is carried out in the early stages of the project life cycle.

Similar to SAST, the IAST analysis provides complete lines of data-rich code so that security teams can immediately lookout for a specific bug.

With the wealth of information that the tool has access to, the source of vulnerabilities can be precisely identified.

Unlike other dynamic software tests, IAST can be easily integrated into Continuous Integration and Deployment (CI / CD) pipelines.

The evaluations take place in real-time in the production environment.

On the other hand:

IAST tools can slow down the operation of the application.

This is based on the fact that the agents change the bytecode themselves. This leads to a lower performance of the overall system.

The change itself can, of course, also lead to problems in the production environment.

The use of agents naturally also represents a potential source of danger since these agents can also be compromised.

– See Solarwinds Hack

RASP – Runtime Application Self Protection

RASP is the approach to secure the application from within.

The backup takes place at runtime and generally consists of looking for suspicious commands when they are executed.

What does that mean now?

With the RASP approach, you can examine the entire application context on the production machine and real-time.

Here all commands that are processed are examined for possible attack patterns.

Therefore, this procedure aims to identify existing security gaps and attack patterns and those that are not yet known.

Here it goes very clearly into the use of AI and ML techniques.

RASP tools can usually be used in two operating modes.

The first operating mode (monitoring) is limited to observing and reporting possible attacks.

The second operating mode (protection) then includes implementing defensive measures in real-time and directly on the production environment.

RASP aims to fill the gap left by application security testing and network perimeter controls.

The two approaches (SAST and DAST) do not have sufficient visibility into real-time data and event flows to prevent vulnerabilities from sliding through the verification process or block new threats that were overlooked during development.

RASP is similar to Interactive Application Security Testing (IAST); the main difference is that IAST focuses on identifying vulnerabilities in the applications, and RASPs focus on protecting against cybersecurity attacks that can exploit these vulnerabilities or other attack vectors.

The RASP technology has the following advantages:

First, RASP complements SAST and DAST with an additional layer of protection after the application is started (usually in production).

It can be easily applied with faster development cycles.

Unexpected entries are checked and checked.

It enables you to react quickly to an attack by providing comprehensive analysis and information about the possible vulnerabilities.

However, RASP tools have certain disadvantages:

By sitting on the application server, RASP tools can adversely affect application performance.

In addition, the technology (RASP) may not be compliant with regulations or internal guidelines,

which allows the installation of other software or

the automatic blocking of services is permitted.

The use of this technology can also lead to the people involved believing themselves to be false security.

The application must also be switched offline until the vulnerability is eliminated.

RASP is not a substitute for application security testing because it cannot provide comprehensive protection.

While RASP and IAST have similar methods and uses, RASP does not perform extensive scans but instead runs as part of the application to examine traffic and activity. Both report attacks as soon as they occur; with IAST, this happens at the time of the test, while with RASP, it takes place at runtime in production.

Conclusion

All approaches result in a wide range of options for arming yourself against known and unknown security gaps. Here it is essential to reconcile your own needs and those of the company for the future direction.

Conclusion RASP:

We have seen that with RASP, there is an approach in which the application can protect itself against attacks at runtime. The permanent monitoring of your activities and the data transferred to the application enable an analysis based on the runtime environment. Here you can choose between pure monitoring or alerting and active self-protection. However, software components are added to the runtime environment with RASP approaches to manipulate the system independently. This has an impact on performance. With this approach, RASP concentrates on the detection and defence of current cyber attacks. So it analyzes the data and user behaviour in order to identify suspicious activities.

Conclusion IAST:

The IAST approach combines the SAST and DAST approach and is already used within the SDLC, i.e. within the development itself. This means that the IAST tools are already further “to the left” compared to the RASP tools. Another difference to the RASP tools is that IAST consists of static, dynamic and manual tests. Here it also becomes clear that IAST is more in the development phase. The combination of dynamic, static and manual tests promises a comprehensive security solution. However, one should not underestimate the complexity of the manual and dynamic security tests at this point.

Conclusion DAST:

The DAST approach focuses on how a hacker would approach the system. The overall system is understood like a black box, and the attacks occur without knowing the technologies used. The point here is to harden the production system against the Most Common Vulnerabilities. However, one must not forget at this point that this technology can only be used at the end of the production cycle.

Conclusion SAST:

If you have access to all system components, the SAST approach can be used very effectively against known security gaps and license problems. This procedure is the only guarantee that the entire tech stack can be subjected to direct control. The focus of the SAST approach is on static semantics and, in turn, is completely blind to security holes in the dynamic context. A huge advantage is that this approach can be used with the first line of source code.

My personal opinion:

My personal opinion on these different approaches is as follows:

If you start with DevSecOps or security in IT in general, the SAST approach makes the most sense. This is where the greatest potential threat can be eliminated with minimal effort. It is also a process that can be used in all steps of the production line. Only when all components in the system are secured against known security gaps do the following methods show their highest potential. After introducing SAST, I would raise the IAST approach and, finally, the RASP approach. This also ensures that the respective teams can grow with the task and that there are no obstacles or delays in production.

Cheers Sven

The Lifeline of a Vulnerability

Posted on by Leave a comment

Intro

Again and again, we read something in the IT news about security gaps that have been found. The more severe the classification of this loophole, the more attention this information will get in the general press. Most of the time, you don’t even hear or read anything about all the security holes found that are not as well known as the SolarWinds Hack, for example. But what is the typical lifeline of such a security gap?

Vulnerability was generated until found 

Let’s start with the birth of a vulnerability. This birth can be done in two differently motivated ways. On the one hand, it can happen to any developer that he creates a security hole by an unfortunate combination of source code pieces. On the other hand, it can also be based on targeted manipulation. However, this has essentially no effect on the further course of the lifeline of a security vulnerability. In the following, we assume that a security hole has been created and that it is now active in some software. These can be executable programs or libraries offered that are integrated into other software projects as a dependency.

Found until Public available

In most cases, it is not possible to understand precisely when a security hole was created. But let’s assume that there is a security hole and that it will be found. It clearly depends on which person or which team finds this weak point. This has a severe impact on the subsequent course of history. Let’s start with the fact that this vulnerability has been found by people who are interested in using this vulnerability themselves or having it exploited by other parties. Here the information is either kept under lock and key or offered for purchase at relevant places on the Internet. There are primarily financial or political motives here, which I do not want to go into here. However, at this point, it can be clearly seen that the information is passed on at this point in channels that are not generally available to the public.

However, if the security gap is found by people or groups who are interested in making the knowledge about it available to the general public, various mechanisms now take effect. However, one must not forget that commercial interests will also play a role in most cases. However, the motivation is different. If the company or the project itself is affected by this vulnerability, there is usually an interest in presenting the information relatively harmless. The feared damage can even lead to the security gap being fixed, but knowledge about it further is hidden. This approach is to be viewed as critical, as it must be assumed that there will also be other groups or people who will gain this knowledge.

But let’s assume that people who are not directly involved in the affected components found the information about the vulnerability. In most cases, this is the motivation to sell knowledge of the vulnerability. In addition to the affected projects or products, there are also providers of the vulnerability databases. These companies have a direct and obvious interest in acquiring this knowledge. But to which company will the finder of the vulnerability sell this knowledge? Here it can be assumed that there is a very high probability that it will be the company that pays the better price. This has another side effect that affects the classification of the vulnerability. Many vulnerabilities are given an assessment using the CVSS. Here, the base value is made by different people. Different people will have other personal interests here, which will then also be reflected in this value. 

Here I refer to the blog post about CVSS – explained. https://svenruppert.com/2021/04/07/cvss-explained-the-basics/

Regardless of the detours via which the knowledge comes to the vulnerabilities databases. Only when the information has reached one of these points can one assume that this knowledge will be available to the general public over time.

Public available bis consumable

However, one fact can be seen very clearly at this point. Regardless of which provider of vulnerabilities you choose, there will only ever be a subset of all known vulnerabilities in this data set. As an end consumer, there is only one sensible way to go here. Instead of contacting the providers directly, you should rely on integrators. This refers to services that integrate various sources themselves and then offer them processed and merged. It is also essential that the findings are processed so that further processing by machines is possible. This means that the meta-information such as the CVE or the CVSS value is supplied. This is the only way other programs can work with this information. The CVSS value is given as an example. This is used, for example, in CI environments to interrupt further processing when a particular threshold value is reached. Only when the information is prepared in this way and is available to the end-user can this information be consumable. Since the information generally represents a considerable financial value, it can be assumed in the vast majority of cases that the commercial providers of such data sets will have access to updated information more quickly than freely available data collections.

Consumable bis running in Production

If the information can now be consumed, i.e. processed with the tools used in software development, the storyline begins in your projects. Whichever provider you have decided on, this information is available from a particular point in time, and you can now react to it yourself. The requirement is that the necessary changes are activated in production as quickly as possible. This is the only way to avoid the potential damage that can result from this security vulnerability. This results in various requirements for your software development processes. The most obvious need is the throughput times. Only those who have implemented a high degree of automation can enable short response times in the delivery processes. It is also an advantage if the team concerned can make the necessary decisions themselves and quickly. Lengthy approval processes are annoying at this point and can also cause extensive damage to the company.

Another point that can release high potential is the provision of safety-critical information in all production stages involved. The earlier the data is taken into account in production, the lower the cost of removing security gaps. We’ll come back to that in more detail when the shift-left topic is discussed.

Another question that arises is that of the effective mechanisms against vulnerabilities.

TestCoverage is your safety-belt; try Mutation Testing

The best knowledge of security gaps is of no use if this knowledge cannot be put to use. But what tools do you have in software development to take efficient action against known security gaps? Here I would like to highlight one metric in particular: the test coverage of your own source code parts. If you have strong test coverage, you can make changes to the system and rely on the test suite. If a smooth test of all affected system components has taken place, nothing stands in the way of making the software available from a technical point of view.

But let’s take a step back and take a closer look at the situation. In most cases, changing the version of the same dependency in use will remove known vulnerabilities. This means that efficient version management gives you the agility you need to be able to react quickly. In very few cases, the affected components have to be replaced by semantic equivalents from other manufacturers. And to classify the new composition of versions of the same ingredients as valid, strong test coverage is required. Manual tests would go far beyond the time frame and cannot be carried out with the same quality in every run. But what is strong test coverage?

I use the technique of mutation testing. This gives you more concrete test coverage than is usually the case with the conventional line or branch coverage. Unfortunately, a complete description of this procedure is not possible at this point.

However, if you want to learn more about mutation testing, visit the following URL. This video explains the theoretical and practical part of the mutation test for Java and Kotlin.

The need for a single point that understands all repo-types

If we now assume that we want to search for known security vulnerabilities in the development and the operation of the software, we need a place where we can carry out the search processes. Different areas are suitable here. However, there is a requirement that enables an efficient scan across all technologies used. We are talking about a logical central point through which all binaries must go. I don’t just mean the jar files declared as dependencies, but also all other files such as the Debian packages or Docker images. Artifactory is suitable for this as a central hub as it supports pretty much all package managers at one point. Because you have knowledge of the individual files and have the metadata, the following things can also be evaluated.

First of all, it is not only possible to capture the direct dependencies. Knowing the structures of the package managers used means that all indirect dependencies are also known. Second, it is possible to receive cross-technology evaluations. This means the full impact graph to capture the practical meaning of the individual vulnerabilities. The tool from JFrog that can give an overview here is JFrog Xray, and we are directly connected to JFrog Artifactory. Whichever tool you choose, it is crucial that you don’t just scan one technology layer. Only with a comprehensive look at the entire tech stack can one guarantee that there are no known security gaps, either directly or indirectly, in production.

Conclusion

Now we come to a conclusion. We have seen that we have little influence on most sections of a typical lifeline of an IT security vulnerability. Actually, there are only two sections that we can influence directly. On the one hand, it is the quickest and most comprehensive possible access to reliable security databases. Here it is essential that you not only entrust yourself to one provider but rather concentrate on so-called “mergers” or “aggregators”. The use of such supersets can compensate for the economically motivated vagueness of the individual providers. I named JFrog Xray as an example of such an aggregator. The second section of a typical lifeline is in your own home. That means, as soon as you know about a security gap, you have to act yourself—robust automation and a well-coordinated DevSecOps team help here. We will deal with precisely this section from the “security” perspective in another blogpost. Here, however, we had already seen that strong test coverage is one of the critical elements in the fight against vulnerabilities. Here I would like to refer again to the test method “Mutation Testing”, which is a very effective tool in TDD.

And what can I do right now?

Anyone who would like to try out such a DevSecOps approach is welcome to register for one of my free workshops. (https://jfrog.com/resources/upcoming-webinars/) These are offered in German and English. There I will use JFrog Artifactory and JFrog Xray to show how one can practically proceed against known vulnerabilities. And if you don’t want to wait that long, you can, of course, take a look at my YouTube channel and find out a little more about the topic there. I would be delighted to welcome you as my new subscriber. Thanks!

Happy Coding

Sven

CVSS – explained – the Basics

Posted on by 4 comments

Intro

What is the Common Vulnerability Scoring System short called CVSS, who is behind it, what are we doing with it and what a CVSS Value means for you? I will explain how a CVSS Score is calculated, what the different elements of it mean and what are the differences between the different CVSS versions.

The Basic Idea Of CVSS

The basic idea behind CVSS is to provide a general classification of the severity of a security vulnerability. This is about the classification and evaluation of weak points. But what does the abbreviation CVSS mean?

What does the abbreviation CVSS mean?

The letters stand for the words: Common Vulnerability Scoring System. That means something like a general vulnerability rating system. Here, the weak points found are evaluated from various points of view. These elements are weighted against each other so that a standardized number between 0 and 10 is obtained at the end.

For what do you need such a rating system?

A rating system that provides us with a standardized number allows us to evaluate different weak points abstractly and derive follow-up actions from them. The focus here is on standardizing the handling of these weak points. As a result, you can define actions based on the value ranges. Here I mean processes in the value creation sources that are affected by this weak point.

What is the basic structure of this assessment?

In principle, CVSS can be described so that the probability and the maximum possible damage are related using predefined factors. The basic formula for this is: risk = probability of occurrence x damage

The Basic Values from 0..10

The evaluation in the CVSS is based on various criteria and is called “metrics”. For each metric, one or more values selected from a firmly defined selection option. This selection then results in a value between 0.0 and 10.0. Where 0 is the lowest and 10 is the highest risk value. The entire range of values ​​is then subdivided into groups and are labelled “None“, “Low“, “Medium“, “High“, and “Critical“. These metrics are divided into three areas that are weighted differently from one another. These are the areas “Basic Metrics”, “Temporal Metrics”, and “Environmental Metrics”. Here, different aspects are queried in each area, which must be assigned a single value. The weighting among each other and the subsequent composition of the three group values ​​gives the final result.

However, all component values ​​that lead to this result are always supplied. This behaviour ensures that there is transparency at all times about how these values ​​originally came about. Next, the three sub-areas will be explained individually in detail.

The Basic Metrics

The basic metrics form the foundation of this rating system. The aim is to record the technical details of the vulnerability that will not change over time. You can imagine it to be an assessment that is independent of other changing elements. Different parties can carry out the calculation of the base value. It can be done by the discoverer, the manufacturer of the project or product concerned or by a party (CERT) charged with eliminating this weak point. One can imagine that, based on this initial decision, the value itself will turn out differently since the individual groups pursue different goals.

necessary requirements:

The base value evaluates the prerequisites that are necessary for a successful attack via this security gap. This is, for example, the distinction between whether a user account must be available on the target system for an attack that is used in the course of the attack or whether the system can be compromised without the knowledge about a system user. It also plays a significant role in whether a system is vulnerable over the Internet or whether physical access to the affected component is required.

Complexity of the attack:

The base value should also reflect how complex the attack is to carry out. In this case, the complexity relates to the necessary technical steps and includes assessing whether the interaction with a regular user is essential. Is it sufficient to encourage any user to interact, or does this user have to belong to a specific system group (e.g. administrator)? At this point, it is already evident that the assessment of a new vulnerability requires exact knowledge of this vulnerability and the systems concerned. The correct classification is not a trivial process.

Assessment of the damage:

The basic metrics also take into account the damage that this attack could cause to the affected component. This means the possibilities to extract the data from the system, manipulate it, or completely prevent the system’s use. One speaks here of the three areas;

  • Confidentiality
  • Integrity
  • Availability

However, you have to be careful here concerning the weighting of these possibilities. In one case, it can be worse when data is stolen than it is changed. In another case, the unusability of a component can be the worst damage to be assumed.

Scope-Metric:

The scope metric has also been available since CVSS version 3.0. This metric looks at the effects of an affected component on other system components. For example, one can imagine that a compromised element in a virtualized environment enables access to the carrier system. A successful change of this scope represents a greater risk for the overall system and is therefore also evaluated using this factor. This point alone clearly shows that the interpretation of the values also requires adjusting to one’s situation. And so we come to the “temporal” and “environment” metrics.

The Temporal Metrics

The time-dependent components of the vulnerability assessment are brought together in the group of temporal metrics. The peculiarity at this point is that the base value can be reduced by the temporal components only. The initial rating is intended to represent the worst-case scenario.

This has both advantages and disadvantages if you bear in mind that it is during the initial assessment of a vulnerability that can give very different interests. At this point, there are two things that need to be highlighted;

1) Which factors influence the temporal metrics? 

The elements that change over time influence the “Temporal Metrics”.

On the one hand, this refers to changes concerning the availability of tools that support the exploitation of the vulnerability. These can be exploits or step-by-step instructions. A distinction must be made whether a chess point is theoretical or whether a manufacturer has officially confirmed it. All of these events change the base value.

2) Influence of the initial evaluation?

The influence on the initial evaluation comes about through external framework conditions. These take place over an undefined time frame and are not relevant for the actual basic assessment. Even if an exploit is already in circulation during the base values survey, this knowledge will not be included in the primary assessment. However, the base value can only be reduced by the temporal metrics. This approach takes a little getting used to and is often the subject of criticism. The reason why you decided on it is understandable from the theoretical point of view. The base value is intended to denote the most excellent possible damage.

And this is where a conflict arises. The person or group who has found a security gap tries to set the base value as high as possible. A highly critical loophole is better to sell and better exploited in the media. The reputation of the person/group who found this gap increases as a result. The affected company or the affected project is interested in exactly the opposite assessment. It, therefore, depends on who finds the security gap, how the recycling should take place and by which body the first evaluation is carried out. The only offsetting component is the Environmental Metrics.

The Environmental Metrics

In the case of environmental metrics, the own system landscape is set in relation to the security gap. This means that the evaluation is adjusted based on the real situation. In contrast to Temporal Metrics, Environmental Metrics can correct the base value in both directions. The environment can therefore lead to a higher classification and must also be constantly adapted to your own changes. The combination now gives rise to purely practical questions. Let’s assume that there is a patch from the manufacturer for a security hole. The mere presence of this modification leads to a lowering of the total value in the Temporal Metrics. However, as long as the patch has not been activated in your own systems, the overall value must be drastically corrected upwards again via the Environmental Metrics. Why did I say at this point that the value has to be increased drastically? As soon as a patch is available, this patch can be used to better understand the security gap and its effects. The attacker has more and more detailed information that can be used. This reduces the resistance of the not yet hardened systems.

The Final Score

At the end of an evaluation, the final score is obtained, calculated from the three previously mentioned values. At this point, I will not explain the calculation details; I have a separate post on this. The resulting value is then assigned to a value group. But there is one more point that is very important to me personally. In many cases, I see that the final score is simply carried over. The individual adjustments utilizing the environmental score do not take place. Instead, the value one to one is adopted as the risk assessment. In many cases, this behaviour leads to a dangerous evaluation which is incorrect for the overall system concerned.

Conclusion

We come to the management summary. With the CVSS we have a value system for evaluating security gaps in software. Since there are no alternatives, the system has been in use worldwide for over ten years and is constantly being developed, it is a defacto standard. The evaluation consists of three components. First, there is the basic score, which is there to depict a purely technical worst-case scenario. The second component is the evaluation of the time-dependent corrections based on external influences. This is about the evaluation of whether there are further findings, tools or patches for this security gap. The peculiarity of this point is that the base score can only be reduced with this value. The third component is the assessment of your own system environment with regard to this weak point. With this consideration, the security gap is adjusted in relation to the real situation on site. This value can lower or higher the base rating. Last but not least, an overall evaluation is made from these three values, which results in a number between 0.0 to 10.0.

This final value can be used to control your own actions to defend against the security gap in the overall context. At first glance, everything is quite abstract. It takes some practice to get a feel for the figures it contains.

The CVSS can only develop its full effect if it deals more deeply with this matter within your own system.

I will explain the underlying calculations in detail in further posts.

A Challenge of the Software Distribution

Posted on by Leave a comment

The four factors that are working against us

Software development is more and more dependent on Dependencies and the frequency of deployments is increasing. Both trends together are pushing themselves higher. Another element that turns the delivery of software into a network bottleneck is the usage of compounded artefacts. And the last trend that is working against us, is the exploding amount of edges or better-called edge nodes.All four trends together are a challenge for the infrastructure.But what we could do against it?

Edge-Computing

Before we look at the acceleration strategies I will explain a bit the term “Edge” or better “Edge-Computing” because this is often used in this context. 


What is Edge or better edge computing?

The principle of edge computing states that data processing takes place at the Edge of the network. Which device is ultimately responsible for processing the data can differ depending on the application and the implementation of the concept.
An edge device is a device on the network periphery that generates, processes or forwards data itself. Examples of edge devices are smartphones, autonomous vehicles, sensors or IoT devices such as fire alarms.
An edge gateway is installed between the edge device and the network. It receives data from edge devices that do not have to be processed in real-time, processes specific data locally or selectively, sends the data to other services or central data centers. Edge gateways have wireless or wired interfaces to the edge devices and the communication networks for private or public clouds.


Pros of Edge Computing

The data processing takes place in the vicinity of the data source, minimising transmission and response times. Communication is possible almost in real-time. Simultaneously, the data throughput and the bandwidth usage reduction in the network, since only specific data that are not to be processed locally need to be transmitted to central data centres. Many functions can also be maintained even if the network or parts of the network fail—the performance of edge computing scales by providing more intelligent devices at the network periphery.

Cons of Edge Computing

Edge computing offers more security due to the locally limited data storage, but this is only the case if appropriate security concepts are available for the decentralised devices, due to the heterogeneity and many different devices, the effort involved in implementing the security concepts increases.

Fog Computing

Edge computing and fog computing are both decentralised data processing concepts. Fog Computing inserts another layer with the so-called Fog Nodes between the edge devices and the cloud. These are small, local data centres in the access areas of the cloud. These fog nodes collect the data from the edge devices. You select the data to be processed locally or decentrally and forward it to central servers or process it directly yourself. 
Selecting the best of both worlds means we are combining both principles of Edge- and Fog-Computing.

What are the acceleration options for SW Distribution?

There are different strategies to scale the distribution of binaries, and every solution suits a specific use-case. We will not have a few on cloud solutions only because companies are operating worldwide and have to deal with different governmental regulations and restrictions. Additionally, to these restrictions, I want to highlight the need for hybrid solutions as well. Hybrid solutions are including on-prem resources as well as air gaped infrastructure used for high-security environments.

a) Custom Solution based on replication or scaling servers

One possibility to scale inside your network/architecture is scaling hardware and working with direct replication. Implementing this by yourself will most-likely consume a higher budget of workforce, knowledge, time and money based on the fact that this is not a trivial project. At the same time, this approach is bound into the borders of the infrastructure you have access to.

b) P2P Networks

Peer to Peer networks is based on equal nodes that are sharing the binaries between all nodes.The peer to peer approach implies that you will have a bunch of copies of your files. If you are downloading a file from the network, all nodes can serve parts independently. This approach of splitting up files and delivering from different nodes simultaneously to the requesting node leads to constant and efficient network usage and reduced download times.

c) CDN – Content Delivery Network

CDN’s are optimised to deliver large files over regions. The network itself is build-out of a huge number of nodes that are caching files for regional delivery. With this strategy, the original server will not be overloaded.

Check out on my Youtube Channel the video with the title "DevSecOps - the Low hanging fruits".This video describes the balance between writing the code itself or adding a dependency in each Cloud-Native App layer. The question is, what does this mean for DevSecOps?

JFrog Solution

With the three mentioned techniques you can build up huge and powerful architecture that fit´s to your needs. But the integration of all these technologies and implementing products is not easy. We faced this challenge as well and over the years we found solutions that we integrated into a single DevSecOps Platform called “The JFrog Platform“. I don´t want to give an overview of all components, for this check out my youtube channel. Here I want to focus on the components that are responsible for the Distribution of the binaries only.

JFrog Distribution

With the JFrog Distribution, the knowledge about the content of the repositories and the corresponding metadata is used to provide a replication strategy. The replication solution is designed for internal and external repositories to bring the binaries all the way down to the place where it is needed. The infrastructure can be built in a hybrid model, including on-prem and cloud nodes.Even air-gapped solutions are possible with import/export mechanisms. In this scenario, we are focussing on a scalable caching mechanism that is optimised for reads.

What is a Release Bundle?

A Release bundle is a composition of binaries. These binaries can be of different types, like maven, Debian or Docker. The Release Bundle can be seen as a Bills Of Materials (BOM).The content and well as the Release Bundles itself are immutable. This immutability makes it possible to implement efficient caching and replication mechanisms across different networks and regions.

What is an Edge Node in this context?

An Edge Node in our context is a node that will provide the functionality of a read-only Artifactory.With this Edge Node, the delivery process is optimised, and we will see that replication is done in a transactional way. The difference to the original meaning of an Edge Node is that this instance is not the consuming or producing element. This can be seen as a Fog-Node, that is the first layer above the real edge nodes layer.

P2P Download

The P2P solution focuses on environments that need to handle download bursts inside the same network or region.This download bursts could be scenarios like “updating a server farm” or “updating a Microservice Mesh”. The usage is unidirectional, which means that the consumer is not updating from their side. They are just waiting for a new version and all consumer updating at the same time.This behaviour is a perfect case for the P2P solution. Artifactory, or an Edge Node in the same network or region, is influencing an update of all P2P Nodes with a new version of a binary. The consumer itself will request the binary from the P2P node and not from the Artifactory instance anymore.The responsible Artifactory instance manages the P2P nodes, which leads to zero maintenance on the user side. Have in mind, that the RBAC is active at the P2P nodes as well. 

CDN Distribution

The CDN Solution is optimised to deliver binaries to different parts of the world. We have it in two flavours. One is for the public and mostly used to distribute SDK’s, Drivers or other free available binaries. The other flavour is focussing on the private distribution.Whatever solution you are using, the RBAC defined inside the Access Module is respected, including solutions with Authentication and Authorisation and unique links including Access Tokens.

Conclusion

Ok, it is time for the conclusion.What we discussed today;
With the increasing amount of dependencies, a higher frequency of deployments and the constantly growing number of applications and edge-nodes, we are facing scalability challenges.
We had a look at three ways you could go to increase your delivery speed.The discussed solution based on

a) JFrog Distribution helps you build up a strong replication strategy inside your hybrid infrastructure to speed up the development cycle.
b) JFrog P2P that will allow you to handle massive download bursts inside a network or region. This solution fits tasks that need to distribute binaries to a high number of consumers concurrently during download bursts.
c) JFrog CDN to deliver binaries worldwide into regional data centres to make the experience for the consumer as best as possible.


All this is bundled into the JFrog DevSecOps Platform. 


Cheers Sven

DevSecOps – Be Independent Again

Posted on by Leave a comment

What do the effects the news of the last few months can have to do with risk management and the presumption of storage, and why is it an elementary component of DevSecOps?

If you want to see this Post as a video, check-out the following from me Youtube Channel

What Has Happened So Far

Again and again, changes have happened that set things in motion that were considered to have been set. In some cases, services or products have been freely available for many years, or the type of restriction has not changed. I am taking one of the last changes as an occasion to show the resulting behavior and to formulate solutions that help you deal with it.

In software development, repositories are one of the central elements that enable you to efficiently deal with the abundance of dependencies in a software environment. A wide variety of types and associated technologies have evolved over the decades. But there is a common approach mostly resulted in a global central authority that is seen as an essential reference.

I examined the topic of the repository from a generic point of view in a little more detail on youtube. 

As an example, I would like to briefly show what a minimal technology stack can look like today. Java is used for the application itself, the dependencies of which are defined using maven. For this, we need access to maven repositories. Debian repositories [Why Debian Repos are mission-critical..] used for the operating system on which the application is running. The components that then packaged into Docker images use Docker registries, and finally, the applications orchestrated in a composition of Docker images using Kubernetes. Here alone, we are dealing with four different repository types. At this point, I have left out the need for generic repositories to provide the required tools used within the DevSecOps pipeline.

DockerHub And Its Dominance

The example that inspired me to write this article was DockerHub’s announcements. Access to this service was free, and there were no further restrictions on storage space and storage duration for freely available Docker images. This fact has led to a large number of open source projects using this repository for their purposes. Over the years a whole network of dependencies between these images has built up.

Docker Hub was in the news recently for two reasons.

Storage Restrictions

Previously, Docker images were stored indefinitely on Dockerhub. On the one hand, this meant that nobody cared about the storage space of the Docker images. On the other hand, pretty much everyone has been counting on it not to change again. Unfortunately, that has now changed. The retention period for inactive Docker images has been reduced to six months. What doesn’t sound particularly critical at first turns out to be quite uncomfortable in detail.

Download Throttling

Docker has limited the download rate to 100 pulls per six hours for anonymous users, and 200 pulls per six hours for free accounts. Number 200 sounds pretty bearable. However, it makes sense to take a more detailed look here. 200 inquiries / 6h are 200 inquiries / 360min. We’re talking about 0.55 queries/minute at a constant query rate. First, many systems do more than one build and therefore requests, every 2 minutes. Second, if the limit is reached, it can take more than half a business day to regain access. The latter is to be viewed as very critical. As a rule, limit values given per hour, which then only leads to a delay of a little less than an hour. Six hours is a different order of magnitude.

Maven and MavenCentral

If you look at the different technologies, a similar monoculture emerges in the Maven area. Here is the maven-central a singular point operated by one company. A larger company bought this company. What does this mean for the future of this repository? I don’t know. However, it is not uncommon for costs to be optimized after a takeover by another company. A legitimate question arises here; What economic advantage does the operator of such a central, free-of-charge infrastructure have?

JDKs

There have been so many structural changes here that I’m not even sure what the official name is. But there is one thing I observe with eagle eyes in projects. Different versions, platforms and providers of the JDKs result in a source of joy in LTS projects that should not be underestimated. Here, too, it is not guaranteed how long the providers will keep the respective builds of a JDK for a platform. What is planned today can be optimized tomorrow. Here, too, you should take a look at the JDKs that are not only used internally but also by customers. Who has all the installers for the JDKs in use in stock? Are these JDKs also used within your own CI route, or do you trust the availability of specific Docker images?

Moderate Independence

How can this be countered now? The answer is straightforward. You get everything you need just once and then save it in your systems. And so we are running against the efforts of the last few years. As in most other cases, moderate use of this approach is recommended. More important than ever is the sensible use of freely available resources. It can help if a stringent retention tactic is used. Not everything has to be kept indefinitely. Many elements that are held in the caches are no longer needed after a while. Sophisticated handling of repositories and the nesting of resources helps here. Unfortunately, I cannot go into too much detail here, but it can be noted in short form.

The structure of the respective repositories enables, on the one hand, to create concrete compositions and, on the other hand, to carry out very efficient maintenance. Sources must be kept in individual repositories and then merged using virtual repositories. This process can be built up so efficiently that it can even drastically reduce the number of build cycles. 

DevSecOps – Risk Minimization

There is another advantage in dealing with the subject of “independence”. Because all files that are kept in their structures can be analyzed with regard to vulnerabilities and compliance, now when these elements are in one place, in a repository manager, I have a central location where I can scan them. The result is a complete dependency graph that includes the dependencies of an application but also the associated workbench. That, in turn, is one of the critical statements when you turn to the topic of DevSecOps. Security is like quality! It’s not just a tool; it’s not just a person responsible for it. It is a philosophy that has to run through the entire value chain.

Happy Coding,

Sven Ruppert 

The quick Wins of DevSecOps

Posted on by Leave a comment

Hello and welcome to my DevSecOps post. Here in Germany, it’s winter right now, and the forests are quiet. The snow slows down everything, and it’s a beautiful time to move undisturbed through the woods.

Here you can pursue your thoughts, and I had to think about a subject that customers or participants at conferences ask me repeatedly.

The question is almost always:

What are the quick wins or low hanging fruits if you want to deal more with the topic of security in software development?

And I want to answer this question right now!

For the lazy ones, you can see it as youtube video as well

Let’s start with the definition of a phrase that often used in the business world.

Make Or Buy

Even as a software developer, you will often hear this phrase during meetings with the company’s management and sales part.

The phrase is called; “Make or Buy“. Typically, we have to decide if we want to do something ourselves or spend money to buy the requested functionality. It could be less or more functionality or different so that we have to adjust ourself to use it in our context.

But as a software developer, we have to deal with the same question every day. I am talking about dependencies. Should we write the source code by ourselves or just adding the next dependencies? Who will be responsible for removing bugs, and what is the total cost of this decision? But first, let’s take a look at the make-or-buy association inside the full tech-stack.

Diff between Make / Buy on all layers.

If we are looking at all layers of a cloud-native stack to compare the value of “make” to “buy” we will see that the component “buy” is in all layers the bigger one. But first things first.

Alt Text

The first step is the development of the application itself.

Assuming that we are working with Java and using maven as a dependency manager, we are most likely adding more lines of code indirectly as dependency compared to the number of lines we are writing by ourselves. The dependencies are the more prominent part, and third parties develop them. We have to be carefully, and it is good advice to check these external binaries for known vulnerabilities.

We should have the same behaviour regarding compliance and license usage. The next layer will be the operating system, in our case Linux.

And again, we are adding some configuration files and the rest are existing binaries.

The result is an application running inside the operating system that is a composition of external binaries based on our configuration.

The two following layers, Docker and Kubernetes, are leading us to the same result. Until now, we are not looking at the tool-stack for the production line itself.

All programs and utilities that are directly or indirectly used under the hood called DevSecOps are some dependencies.

All layers’ dependencies are the most significant part by far.

Checking these binaries against known Vulnerabilities is the first logical step.

one time and recurring efforts for Compliance/Vulnerabilities

Comparing the effort of scanning against known Vulnerabilities and for Compliance Issues, we see a few differences.

Let’s start with the Compliance issues.

Compliance issues:

The first step will be defining what licenses are allowed at what part of the production line. This definition of allowed license includes the dependencies during the coding time and the usage of tools and runtime environments. Defining the non-critical license types should be checked by a specialised lawyer. With this list of white labelled license types, we can start using the machine to scan on a regular base the full tool stack. After the machine found a violation, we have to remove this element, and it must be replaced by another that is licensed under a white-labelled one.

Vulnerabilities:

The recurrent effort on this site is low compared to the amount of work that vulnerabilities are producing. A slightly different workflow is needed for the handling of found vulnerabilities. Without more significant preparations, the machine can do the work on a regular base as well. The identification of a vulnerability will trigger the workflow that includes human interaction. The vulnerability must be classified internally that leads to the decisions what the following action will be.

Compliance Issues: just singular points in your full-stack

There is one other difference between Compliance Issues and Vulnerabilities. If there is a compliance issue, it is a singular point inside the overall environment. Just this single part is a defect and is not influencing other elements of the environment.

Alt Text

Vulnerabilities: can be combined into different attack vectors.

Vulnerabilities are a bit different. They do not only exist at the point where they are located. Additionally, they can be combined with other existing vulnerabilities in any additional layer of the environment. Vulnerabilities can be combined into different attack vectors. Every possible attack vector itself must be seen and evaluated. A set of minor vulnerabilities in different layers of the application can be combined into a highly critical risk.

Alt Text

Vulnerabilities: timeline from found until active in the production

I want to have an eye on as next is the timeline from vulnerability is found until the fix is in production. After a vulnerability is existing in a binary, we have nearly no control over the time until this is found. It depends on the person itself if the vulnerability is reported to the creator of the binary, a commercial security service, a government or it will be sold on a darknet marketplace. But, assuming that the information is reported to the binary creator itself, it will take some time until the data is publicly available. We have no control over the duration from finding the vulnerability to the time that the information is publicly available. The next period is based on the commercial aspect of this issue.

As a consumer, we can only get the information as soon as possible is spending money.
This state of affairs is not nice, but mostly the truth.

Nevertheless, at some point, the information is consumable for us. If you are using JFrog Xray, from the free tier, for example, you will get the information very fast. JFrog is consuming different security information resources and merging all information into a single vulnerability database. After this database is fed with new information, all JFrog Xray instances are updated. After this stage is reached, you can act.

Alt Text

Test-Coverage is your safety-belt; try Mutation Testing.

Until now, the only thing you can do to speed up the information flow is spending money for professional security information aggregator. But as soon as the information is consumable for you, the timer runs. It depends on your environment how fast this security fix will be up and running in production. To minimise the amount of time a full automated CI Pipeline ist one of the critical factors.

But even more critical is excellent and robust test coverage.

Good test coverage will allow you, to switch dependency versions immediately and push this change after a green test run into production. I recommend using a more substantial test coverage as pure line-coverages. The technique called “mutation test coverage” is a powerful one.

Mutation Test Coverage
If you want to know more about this on, check out my YouTube channel. I have a video that explains the theoretical part and the practical one for Java and Kotlin.

The need for a single point that understands all repo-types
To get a picture of the full impact graph based on all known vulnerabilities, it is crucial to understand all package managers included by the dependencies. Focussing on just one layer in the tech-stack is by far not enough.

JFrog Artifactory provides information, including the vendor-specific metadata that is part of the package managers.

JFrog Xray can consume all this knowledge and can scan all binaries that are hosted inside the repositories that are managed by Artifactory.

Alt Text

Vulnerabilities – IDE plugin

Shift Left means that Vulnerabilities must be eliminated as early as possible inside the production pipeline. One early-stage after the concept phase is the coding itself. At the moment you start adding dependencies to your project you are possibly adding Vulnerabilities as well.

The fastest way to get feedback regarding your dependencies is the JFrog IDE Plugin. This plugin will connect your IDE to your JFrog Xray Instance. The free tier will give you access to Vulnerability scanning. The Plugin is OpenSource and available for IntelliJ, VS-Code, Eclipse,… If you need some additional features, make a feature request on GitHub or fork the Repository add your changes and make a merge request.

Try it out by yourself – JFrog Free Tier

How to use the IDE plugin?

If you add a dependency to your project, the IDE Plugin can understand this information based on the used package manager. The IDE Plugin is connected to your JFrog Xray instance and will be queried if there is a change inside your project’s dependency definition. The information provided by Xray includes the known vulnerabilities of the added dependency. If there is a fixed version of the dependency available, the new version number will be shown.

If you want to see the IDE Plugin in Action without registering

for a Free Tier, have a look at my youtube video.

Conclusion

With the JFrog Free Tier, you have the tools in your hands to practice Shift Left and pushing it into your IDE.

Create repositories for all included technologies, use Artifactory as a proxy for your binaries and let Xray scan the full stack.

With this, you have a complete impact graph based on your full-stack and the pieces of information about known Vulnerabilities as early as possible inside your production line.

You don’t have to wait until your CI Pipeline starts complaining. This will save a lot of your time.