Cuckoo Sandbox Customization (V2) | Create Module • Adlice Software

Cuckoo Sandbox is a neat open source project used by many people around the world to test malware into a secure environment, to understand how they work and what they do. Cuckoo is written in a modular way, with python language. It’s really easy to customize, and this is what I’m going to show you here.

This post is a rewrite of the previous post, that was about Cuckoo V1, updated for Cuckoo V2.

Cuckoo Sandbox

You can download and install Cuckoo Sandbox here, and here’s a tutorial on how to configure it. We will not cover the installation of cuckoo itself because the guide is really well done.

Customization, the idea

Let’s study a real example that we’ve done for our own purpose, here at Adlice Labs. We wanted an easy way to test for malware removal with our software, RogueKiller. The idea is to get a removal report for malware that we send to the sandbox, let’s take a look.

The Cuckoo team made a guide for customization as well, this will be our reference.
In our templates below, for consistency all our file modules will be named “custom.py”. Just adapt to your case.

In the below, I will reference 2 different folders by the following:

$$CUCKOO: /usr/local/lib/python2.7/dist-packages/cuckoo/ (cuckoo installation path)
$$CWD: Cuckoo working directory, depends on you

Auxiliary (guest) module

The auxiliary guest modules are located into $$CWD/analyzer/windows/modules/auxiliary.
They are executed before the malware, on the guest machine. Therefore they can be useful to log some initial state machine information.

They don’t need to be activated by a config section, once the file is here it gets executed.

import logging
import json

from common.abstracts import Auxiliary
from common.results import NetlogFile

log = logging.getLogger(__name__)

class Custom(Auxiliary):
    """Gather custom data"""

    def __init__(self, options={}, analyzer=None):
        Auxiliary.__init__(self, options, analyzer)

    def start(self):
        log.info("Starting my Custom auxiliary module")
        nf = NetlogFile("logs/initial.json")
        nf.send(json.dumps(['foo', {'bar': ('baz', None, 1.0, 2, False)}]))

As a result, an initial.json file is created into /logs.

Package module

The package modules are located into $$CWD/analyzer/windows/modules/packages.
They are responsible for executing and injecting the malware for analysis, they are executed on the guest.
You can define different execution routines, depending on the type of malware (exe, swf, pdf, …)
I have implemented the finish method to make some post execution actions, we will see later for a concrete example of this.

They don’t need to be activated by a config section, once the file is here it gets executed.

import os
import shlex
import json
import logging

from common.abstracts import Package
from common.results import NetlogFile

log = logging.getLogger("analyzer")

class CustomExe(Package):
    """Custom analysis package."""

    def start(self, path):
        args = self.options.get("arguments", "")

        name, ext = os.path.splitext(path)
        if not ext:
            new_path = name + ".exe"
            os.rename(path, new_path)
            path = new_path

        return self.execute(path, args=shlex.split(args))

    # Post execution
    def finish(self):
        nf = NetlogFile("logs/post.json")
        nf.send(json.dumps(['foo', {'bar': ('baz', None, 1.0, 2, False)}]))
        return True

As a result, a post.json file is created into /logs.

Processing module

The processing modules are located into $$CUCKOO/processing.
They are executed on the host, to append data generated by the modules above into the global container. That way, the data will be available by all the next modules for processing and reporting.

To enable a new processing module, you need to add the section below into $$CWD/conf/processing.conf.

[custom]
enabled = yes

You also need to add a field to the dictionary defined in $$CUCKOO/common/config.py.

"processing": {
            "analysisinfo": {
                "enabled": Boolean(True),
            },
            "apkinfo": {
                "enabled": Boolean(False),
                "decompilation_threshold": Int(5000000),
            },
            "baseline": {
                "enabled": Boolean(False),
            },
            "behavior": {
                "enabled": Boolean(True),
            },
            "custom": {
                "enabled": Boolean(True),
            },
            ...
},

And finally add your module into $$CUCKOO/processing.

import os
import json

from cuckoo.common.abstracts import Processing
from cuckoo.common.exceptions import CuckooProcessingError

class Custom(Processing):
    """Analysis custom information."""

    def run(self):
        """Run debug analysis.
        @return: debug information dict.
        """
        self.key = "custom"
        data = {}
        try:
          #initial
          custom_log = os.path.join(self.logs_path, "initial.json")
          with open(custom_log) as json_file:          
            data["initial"] = json.load(json_file)
        except Exception, e:
          raise CuckooProcessingError(str(e))

        try:
          #post
          custom_log = os.path.join(self.logs_path, "post.json")
          with open(custom_log) as json_file:          
            data["post"] = json.load(json_file)
        except Exception, e:
          raise CuckooProcessingError(str(e))

        return data

As a result, data from initial.json and post.json are appended into the global container. If the Json reporting module (builtin) is enabled, you will retrieve their content into it, under the “custom” key.

Reporting module

The reporting modules are located into $$CUCKOO/reporting.
They are executed on the host, to translate data from the global container into a different format. It could be a HTML page, json file, even a PDF or something else.

To enable a new reporting module, you need to add the section below into $$CWD/conf/reporting.conf.

[custom]
enabled = yes

You also need to add a field to the dictionary defined in $$CUCKOO/common/config.py.

 "reporting": {
            "feedback": {
                "enabled": Boolean(False),
            },
            "jsondump": {
                "enabled": Boolean(True),
                "indent": Int(4),
                "calls": Boolean(True),
            },
            "custom": {
                "enabled": Boolean(True),
            },
            ...
},

And finally add your module into $$CUCKOO/reporting.

import os
import json
import codecs

from cuckoo.common.abstracts import Report
from cuckoo.common.exceptions import CuckooReportError

class Custom(Report):
    """Saves custom results in JSON format."""   
    def run(self, results):
        """Writes report.
        @param results: Cuckoo results dict.
        @raise CuckooReportError: if fails to write report.
        """

        try:
            path = os.path.join(self.reports_path, "custom.json")

            with codecs.open(path, "w", "utf-8") as report:
                json.dump(results["custom"], report, sort_keys=False, indent=4)
        except (UnicodeError, TypeError, IOError) as e:
            raise CuckooReportError("Failed to generate JSON report: %s" % e)

As a result, data from initial.json and post.json that was stored in the global container is returning back to a single “custom.json” file. But we could have easily done a HTML, a PDF or something else.

Add new section to HTML report

If you use the web server, it may be useful to get the information displayed for your new module. This can be achieved by defining new templates and adding routes.

First create your template into $$CUCKOO/web/templates/analysis/pages/custom/index.html. This is the file that you will need to modify for your needs.

{% extends "base.html" %}
{% load staticfiles %}
{% load analysis_tags %}
{% block content %}
    
    <div class="flex-nav">
        {% include "analysis/pages/nav-sidebar.html" %}
        <section class="flex-nav__body cuckoo-analysis" tabindex="0">

            <header class="page-header cuckoo-analysis__header">
                <h2><i class="fa fa-eye"></i> My Custom Data</h2>
            </header>

            <div class="container-fluid">
                <div class="row">
                    <div class="col-md-12">
                        <div class="tabbable tabs">
                            <pre>{{report.analysis.custom}}</pre>
                        </div> 
                    </div>
                </div>
            </div>

            <!-- footer replacement to avoid double scrollbars -->
            <footer class="flex-grid__footer center-left">
                <p class="footnote">
                    &copy;2010-2017 <a href="http://www.cuckoosandbox.org" target="_blank">Cuckoo Sandbox</a>
                </p>
                <div class="logo">
                    <img src="{% static "graphic/cuckoo_inverse.png" %}" alt="Cuckoo Malware Analysis Sandbox" />
                    <a href="#">Back to Top</a>
                </div>
            </footer>
            
        </section>
    </div>

{% endblock %}

Then you need to update the sidebars to add your section to the navigation. The 2 files to be modified are the following, and are quite similar:

$$CUCKOO/web/templates/analysis/pages/sidebar.html
$$CUCKOO/web/templates/analysis/pages/nav-sidebar.html

{% if report.analysis.custom %}
<li>
	<a href="{% url "analysis" report.analysis.info.id "custom" %}"
	{% if env.view_kwargs.page == 'custom' %} class="active" {% endif %}>
		<i class="fa fa-eye"></i> 
		<span>My Custom Data</span>
	</a>
</li>
{% endif %}

...

{% if report.analysis.custom %}
<li>
	<a href="{% url "analysis" report.analysis.info.id "custom" %}">
		<i class="glyphicon glyphicon-eye"></i>My Custom Data
	</a>
</li>
{% endif %}

To finish, you need to add the route for your section in $$CUCKOO/web/controllers/analysis/routes.py:

pages = {
	"summary": "summary/index",
	"static": "static/index",
	"behavior": "behavior/index",
	"network": "network/index",
	"custom": "custom/index",
	...
}

Concrete use case: Removal report

The idea is to get a removal report for a malware with a given anti-malware scanner, RogueKiller.
To do so, we will use a new auxiliary module, for which we will write a finish routine that runs and injects our antimalware:

#!/usr/bin/env python
import logging
import os
import shlex
import json
import logging
import urllib2
import tempfile

from common.abstracts import Auxiliary
from common.results import NetlogFile
from common.results import upload_to_host
from api.process import Process
from common.defines import KERNEL32

log = logging.getLogger(__name__)

class RogueKiller(Auxiliary):
    """Gather custom data"""

    def __init__(self, options={}, analyzer=None):
        Auxiliary.__init__(self, options, analyzer)
        
    def start(self):
        pass

    def finish(self):        
        # removal scanner
        log.info("RogueKiller: Starting removal")
        try: 
          #download
          response = urllib2.urlopen("http://link_to_roguekillercmd.exe")
          f = tempfile.NamedTemporaryFile(delete=False)           
          data = response.read()   
          f.write(data)
           
          #rename
          path = f.name + ".exe"          
          f.close()
          os.rename(f.name, path)
          log.info("Downloaded remover program to %s", path)        
          

          #execute (without injection, too many problems)
          args = "-scan \"-pupismalware -pumismalware -autodelete -portable-license C:\\rk_config.ini -reportpath C:\\rkcmd.log -reportformat txt\" -dont_ask"
          #pid  = self.execute(path, args=shlex.split(args))
          p = Process()
          
          # we need to lock the analyzer while we start and exclude our remover process
          self.analyzer.process_lock.acquire()
          
          if not p.execute(path=path, args=shlex.split(args), free=True):
            raise CuckooPackageError("Unable to execute the initial process, ""analysis aborted.")
          pid = p.pid  
          
          # add to monitored list, so that it will never tried to be injected
          self.analyzer.process_list.add_pid(pid)
          
          # unlock
          self.analyzer.process_lock.release()
          
          log.info("Executing remover program with args: %s", args)

          #wait for end
          while Process(pid=pid).is_alive():
              KERNEL32.Sleep(1000)
          
          #get report
          upload_to_host("C:\\rkcmd.log", "logs/removal.log")
          log.info("Executed remover program with args: %s", args)

        except Exception, e:
          log.exception("Error while loading the remover program")

After the analysis is done, finish() is called. In this method we download in a temporary file our anti-malware (in CLI version), then we run it outside of cuckoo injection, to be faster and not put junk into the report.

Once the report is generated we upload it back to the host to attach it to our analysis. Then the cuckoo report is generated and we can compare what the malware did, and what the anti-malware was able to catch and remove. Simple as that!

In our example below, the malware was a fake putty binary, notice the RogueKillerCMD scanner running.

We then define a processing module, that will put the generated data into the global container:

import os
import json
import io

from cuckoo.common.abstracts import Processing
from cuckoo.common.exceptions import CuckooProcessingError

class RogueKiller(Processing):
    """Analysis custom information."""

    def run(self):
        """Run debug analysis.
        @return: debug information dict.
        """
        self.key = "roguekiller"
        try:
          custom_log = os.path.join(self.logs_path, "removal.log")
          with io.open(custom_log, 'r', encoding='utf-16-le') as report_file:          
            #data = json.load(report_file)
            data = report_file.read()
        except Exception, e:
          raise CuckooProcessingError(str(e))

        return data

To finish, we have added our section in the HTML templates (as described above). This is the result: