Part two: Improving cyber-attack response capabilities with MISP!
There are many organizations that use IoC (Indicator of Compromise) as a measure against cyber-attacks, which is easy to use among threat intelligence. On the other hand, there are many information sources that distribute IoC, and it is difficult to judge from which information source to acquire IoC. This paper explains the index for evaluating information sources created by NTTDATA-CERT.
Introduction
NTTDATA-CERT collects information called IoC (Indicator of Compromise) for the prevention and early detection of security incidents. Among IoC, IP addresses and domains can be applied to the detection rules of security devices such as Firewalls and SIEMs to block and detect illegal communications, which is expected to improve security.
MISP (Malware Information Sharing Platform) (*1) is used as the management platform of IoC, and the efficiency and labor saving of IoC utilization have been continued for several years. One of the issues in the continuous operation of MISP is to establish a method for selecting IoC information sources to improve the security of an organization. There are many information sources in the world, and the type of threat related IoC that is distributed varies depending on the information source. In addition, how the distributed IoC is utilized is also different in each organization. Therefore, it is necessary for each organization to create an index to evaluate and select information sources. This paper introduces the evaluation index of information sources created by NTTDATA-CERT and explains each viewpoint.
Evaluation index of information sources in NTTDATA-CERT
When we created the evaluation index, we clarified the viewpoints that can be considered from the information sources themselves, such as the amount of IoC distributed by information sources and the frequency of distribution. We also interviewed the operation team of NTTDATA-CERT and incorporated limitations and requests in the operation as viewpoints. The evaluation index created is shown in Table 1.
| Perspective | Acceptance criteria |
|---|---|
| (1) Compatibility with operations | Information sources that only deliver types of IoC that are not used in operations will not be adopted |
| (2) Operator | Information sources that cannot be confirmed by the operator/organization of the information source will not be adopted |
| (3) Types of Threats | Information sources that cannot be confirmed about what threats/attack methods IoC is delivered will not be adopted |
| (4) Automated | Do not adopt information sources whose distribution format changes every time and for which automatic processing is difficult |
| (5) Context | Do not adopt information sources that only distribute IoC values without providing timestamps or attacker information |
| (6) Shareable Range | Do not adopt information sources that only distribute IoC that cannot be shared with group companies due to organizational mission |
| (7) Distribution Amount | Do not adopt information sources that cannot be narrowed down to an operable amount |
| (8) False Blocking | Do not adopt information sources that have many IoCs that match the canonical site list |
| (9) Degree of Overlap | Information sources with a large amount of IoC distributed by other information sources will not be adopted |
| (10) Delivery Speed | Information sources that often distribute the same IoC later than other information sources will not be adopted |
As long as the management policy of the information source does not change, Viewpoint 1-6 confirms the content that does not change regardless of the timing of confirmation. Therefore, when an information source is evaluated, if even one of the viewpoints 1-6 fails to meet the acceptance or rejection criteria, the information source is not adopted as an operational target. On the other hand, the evaluation result of viewpoint 7-10 may change depending on the IoC delivered at the time of the evaluation. Therefore, the decision of rejection is not made immediately even if the criteria is not met. In addition to the results of the evaluation of the other viewpoints, the acceptance or rejection criteria are discussed. Since this indicator is created only for NTTDATA-CERT, the acceptance or rejection criteria should be adjusted to suit each organization by referring to the explanation in Chapter 3.
Details of Each Perspective
In this chapter we will describe the intent and details of each perspective included in the evaluation index.
Compatibility with Operations
In order to acquire an IoC suitable for the operation of the organization, confirm the types of IoC distributed by the information source. Types of IoC include domains, IP addresses, URLs, and Hashes. When determining acceptance criteria, ask the operation team of the security device about the types of IoC that can be handled.
Operator
For information sources whose operator is unknown, check whether the name of the organization or an individual operating the information source is listed, and check if the reliability and accuracy of the IoC distributed to them are unknown and unreliable. The location varies depending on the information source, but you should look for the "About us" page or linked SNS profiles.
Types of Threats
In order to adopt an information source that distributes IoC that matches your organization's security measures, check for what threats your information source distributes IoC. The location varies depending on the information source, but you should judge from the name of the information source and check the "About us" page. In this case, if you can confirm the type of threats your information source distributes IoC for, you should adopt it. However, if you have a more specific understanding of your organization's security measures, you should set the information source that distributes IoC about specific threats and attack methods as the acceptance criteria.
Automated
In order to minimize manual operation, check how IoC can be submitted to MISP from the information source. Check whether there is an API for obtaining IoC or whether IoC is distributed as structured data (CSV or JSON).
Context
In addition to the IoC itself (domain and IP address), in order to obtain information that can be used for filtering when the amount of distribution is large or for analysis of SOC, the presence or absence of additional information added when the information source distributes the IoC is checked. For example, if First Seen is added, the search range can be narrowed, and efficient analysis can be performed when searching whether communication including the IoC has occurred in the SOC. In addition to First Seen, there are names of the attacker group, steps of the cyber kill chain, etc. However, in reality, the information added by the information source is small, and if the criteria are tightened, all the information sources may be rejected.
Shareable Range
In order to share the acquired IoC with other organizations, check the shareable range of the IoC distributed by the information source. The location varies depending on the information source, but check the "About us" page and the terms of use. Basically, information sources published on the Internet are accessible by anyone, so there are often no restrictions on sharing. However, it is necessary to be careful that IoC acquired from a closed community is often restricted from sharing. NTTDATA-CERT shares the IoC acquired with the group companies, and since it has a mission to improve the security of the entire NTTDATA Group, it adopts information sources that can be shared.
Distribution Amount
If an excessive number of IoC is injected into the security equipment, the security equipment cannot check the communication in real time, so it checks the amount of IoC delivered in a certain period. Since there are some information sources whose distribution frequency is biased, the security equipment checks not only the amount delivered per day but also the amount delivered per month. As a point, if there is an information source whose distribution amount is too much, we should check the information attached to the IoC to narrow down the amount so that it can be operated.
False Blocking
In order to check whether the information source distributes the sites used for business as IoC, we check the percentage of IoC distributed over a certain period that matches the canonical site list. The canonical site list needs to contain the sites used for business and the sites that provide canonical services. However, there are so many canonical sites in the world that it is difficult for an organization to keep track of all of them and keep updating the site list. Furthermore, due to the spread of cloud storage and hosting services, even if these services themselves are not malicious, the stored content may be malicious. Therefore, even if an IoC matches a legitimate site list, it is premature to conclude that the IoC causes a false blocking.
Therefore, NTTDATA-CERT uses a collection of legitimate site lists called misp-warninglists (*2). The misp-warninglists contains a list of access rankings and a list of IP address ranges owned by various cloud services, and most of the lists are updated regularly. When misp-warninglists is used, the IoC is checked at the URL level, as shown in Figure 1, to determine whether a matched IoC actually causes a false blocking.
Figure 1: Notes on the IoC that matches misp-warninglists
After this confirmation, the percentage of IoCs that cause false interruptions among the IoCs delivered by the information sources targeted for evaluation is calculated to reduce the risk that information sources that should have been adopted would be rejected.
Degree of Overlap
In order to obtain rarer IoCs, check whether the IoC distributed by one information source is also distributed by another information source. Since it is difficult to check all published information sources, we check whether the IoC of a new information source overlaps with that of an already adopted information source. If there are overlapping IoCs, we check which information source distributes the IoC faster in terms of distribution speed below.
Distribution Speed
When there are multiple information sources that distribute the same IoC, we check which information source distributes the IoC fastest in order to obtain the IoC faster. Normally, we can make an accurate comparison by using the time stamps such as First Seen and Last Seen that are assigned to each IoC, but if there is no time stamp, we compare by the date and time that the information source distributes the IoC.
Results of Actual Use of Evaluation Indicators
Four new information sources were evaluated using the evaluation indicators in Table 1. The results shown in Table 2 were obtained, and sources C and D were selected.
| SourcesViewpoint | Sources A | Sources B | Sources C | Sources D |
|---|---|---|---|---|
| (1)-(6) | No problem | It is a list of IoC and there is no time stamp. | No problem | Compared to other information sources, it is expected to improve the security of the local organization in particular (see Point of View (3) Types of threats). |
| (7) Amount of Delivery | No problem | - | No problem | No problem |
| (8) False Blocking | No problem | - | No problem | High match rate with misp-warninglists |
| (9) Degree of overlap | 80% of IoCs overlap with other sources | - | No problem | No problem |
| (10) Delivery speed | Delivery timing is slower than other sources | - | - | - |
| Adoption/rejection | Rejected from viewpoints (9) and (10) | Rejected from context perspective | Adoption | Adoption |
Normally, we would refrain from adopting information source D, which contains many IoCs with the possibility of false blocking. However, when we checked from the viewpoint of "types of threats," we found that this information source particularly delivers IoCs that improve the security of our own organization compared to other information sources, and we found that this information source should be actively adopted. The concern grasped from the "viewpoint of false blocking" was resolved by incorporating an operation to exclude inappropriate IoC by misp-warninglists, and only IoC necessary for the organization was left. By taking an overall view of information sources based on the evaluation index, it was possible to adopt high-quality information sources without being dragged by one fault, and the security was improved.
Conclusion
The information source evaluation index created by NTTDATA-CERT was introduced, and the details of each viewpoint were explained. Since each organization has different ways of utilizing IoC and requires various information sources, I would like you to arrange and use each viewpoint by referring to the explanation. I hope this article will enable each organization to adopt information sources that improve security.
Hirosawa Tatsunori
NTT DATA Group Corporation
After joining NTT DATA, he became a member of NTTDATA-CERT and has been engaged in OSINT operations centering on the management and operation of threat information, as well as handling incidents that occur within the group. He is actively working as the secretary of the Digital Forensic Study Group and the chief of the Young People's Activity WG.
