LTL Model Checking of Self Modifying Code - CNRS - Centre national de la recherche scientifique Accéder directement au contenu
Article Dans Une Revue Formal Methods in System Design Année : 2022

LTL Model Checking of Self Modifying Code

Tayssir Touili
Xin Ye
  • Fonction : Auteur

Résumé

Self modifying code is code that can modify its own instructions during the execution of the program. It is extensively used by malware writers to obfuscate their malicious code. Thus, analysing self modifying code is nowadays a big challenge. In this paper, we consider the LTL model-checking problem of self modifying code. We model such programs using self-modifying pushdown systems (SM-PDS), an extension of pushdown systems that can modify its own set of transitions during execution. We reduce the LTL model-checking problem to the emptiness problem of self-modifying Büchi pushdown systems (SM-BPDS). We implemented our techniques in a tool that we successfully applied for the detection of several self-modifying malware. Our tool was also able to detect several malwares that well-known antiviruses such as BitDefender, Kinsoft, Avira, eScan, Kaspersky, Qihoo-360, Baidu, Avast, and Symantec failed to detect.
Fichier principal
Vignette du fichier
main.pdf (3.48 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03863361 , version 1 (21-11-2022)

Identifiants

  • HAL Id : hal-03863361 , version 1

Citer

Tayssir Touili, Xin Ye. LTL Model Checking of Self Modifying Code. Formal Methods in System Design, 2022, 60 (2). ⟨hal-03863361⟩
10 Consultations
21 Téléchargements

Partager

Gmail Facebook X LinkedIn More