One of the important topics in fighting fraud and crime is data logistics, or the movement of IOCs, observations, and intelligence betwen cooperating entities. If you receive a phishing message and want to tell others about it to reduce the number of potential victims – you are using data logistics. If you receive so-called threat feeds, you are using data logistics. This process of collection, storage, and distribution is always hung up on a few main issues: 1) agreeing to common data formats; 2) convincing others of the benefits of sharing data, and 3) ensuring that the data sharing operations meet secrity, privacy, and disclosure regulations.
The APWG created a specific term, e-crime event data records, to identify the data pieces, streams, or records we collect and share using data logistics. Why make a new term? Technicrats can be pedantic. If we’re discussing indicators-of-compromise (IOCs), malicious scanning data, account takeover info (ATO) or anything else, we’re really talking about an event composed of some type of data record. Our focus is on data related to electronic crime so it’s e-crime event data records. No matter what detailed event data we’re discussing, it can be generally identified as e-crime event data , even if new data or crime type appear. Simplicity wins. Discussions with non-technocrats like governments and law enforcement agents also become easier with general terms.
Issue #1: Agree on Common Data Formats
The collected and shared data must be in a format that makes it easy for the collector and understandable by the reciever. In many cases, the data collector has one chance to do the collection. For example, a network operator has one chance to capture network traffic as it only comes by once. Historically, e-crime event data was exchanged via the simple csv format. Although a sheet full of csv data is easy to distribute, updating and correction is painful as one may have to reshare the entire data set. We have strived since 2004 to make data sharing easy and automated as manual processing does not scale well with large data sets.
We at the APWG have been pursuing a strategy that allows our members to collect and submit data in various formats, such as the IETF IODEF, that include critical elements and investigative hints. We suport other formats as necessary to ensure maximal data comprehension for the data reciver and conveys the critical information easily.
Issue #2: Convincing Others of the Benefits of Sharing Their Data Legally
Most people like it when others supply ecrime event data records. Many people have silly reasons why they can’t share their own data. (This is known as the 1/4/95 rule: One percent of people share their data and take others’ data; 4% just take others’ data and use it; 95% of the people take the data and don’t know what to do with it.)
The APWG helps our members explain the benefits of data sharing to their organizations and government regulators. We also host symposiums on identifying the reasons our members think they cannot share their ecrime event data records with us and their fellow crime fighters. Once identified, we help define procedures, policies, or guidance to reduce those impediments and increase ther amount of data sharing that we provide.
Issue #3: Ensuring our data sharing operations meet secrity, privacy, and disclosure regulations
The APWG, as an ecrime event data clearinghouse, endeavours to allow our members and other parties to collect and share data that helps the fight on crime. We help our members by defining common processes and expectations so data can be collected and shared anywhere in the world. A part of this effort is discussion and interaction with governments and international treaty partners to ensure we collect and share data in a consitent and legal manner. Our recent efforts include making our data sets EU GDPR compliant; working with ICANN to remove malicious domains from the DNS system; supporting efforts in the Council of Europe and the United Nations on data movement for the pursuit of crminal activity; and offering metrics and experiences to guide governmental thought. We plan to continue to be a leader in innovative way to solve the legal issues in data logistics.
Our data logistics blog contains our thoughts and plans on these, and other, data logistics and data sharing issues.