Security Testing: From TARA to test coverage and regression strategies
Automotive security has gained huge relevance in short time-frame. The annual Vector client survey has pushed its relevance to the top challenge within 12 months. One reason: Safety needs security as a mandatory condition, which means that any safety-critical system as a minimum must also be protected for cybersecurity. While functional safety today is well established, cybersecurity still is in its infancy for many companies. Security must be integrated early in the design phase of a vehicle to understand the threats and risks to car functions. Our article looks towards security testing, and how to make it efficient, yet effective.
Traditional security verification and validation uses a variety of techniques from static analysis to fuzzing and PenTest. While brute-force testing might sound appealing and easy to apply for detection of weakness at any place, it is expensive, inefficient and time consuming. Therefore, we have developed above given grey-box security testing suite where we conduct a mini-TARA and on this basis, we identify the attack vectors and focus our testing based on identified assets and risks. By this way we provide to the system owners/manufacturers efficient, effective and foreseeable results in shorter time range. For the penetration testing we propose to start with a limited scope of the first step: Namely, asset definition. For risk-based security testing we propose to start with a quick analysis from assets and assets classifications (CIAAG) and their possible impact categories (SOFP) towards attack vectors. Together with the system owner/manufacturer (which can be an OEM or system supplier) we go through the system architecture, interfaces, available safety and security mechanisms to identify the assets in the system. After the initial phase of system and architecture walkthroughs a list of “Asset Candidates” are provided to the system owner/manufacturer. During one or two different workshop sessions we create an agreed list of the system assets, names, descriptions and additionally protection categories and impact categories and create the systematic documentation of those.
On the one side the definition of the assets, protection and impact categories brings the white box aspects to the testing whereas also we follow the black-box security testing approach. For this reason, we name it as grey box Penetration Testing as it increases efficiency. For instance, a specific architecture or protocol – when known – invites specific attacks, such as CAN with DOS attacks. The described validation methodology provides several advantages such as
- Risk-based testing with a tailored and thus efficient grey-box methodology
- Easy to understand, asset related results with a clear structure
- Prioritized list of findings based on the impact categories
In this industry practice paper, we will evaluate different formats of PenTesting and show how to make practical usage. Examples from various companies on PenTesting for automotive OEMs and tier-1 suppliers illustrate the PenTesting approach. In recent evaluations, the grey-box paradigm has showed its advantages, specifically when approached on a true “competition” between different PenTesting teams. In a specific show-case in early 2019, we tested for two tier-1 suppliers their ECU. The grey-box method needed substantially lower effort and was still more effective.
Safety & Security