Archives for September 2014

It appears that Red Hat assurances regarding updates released thus far to fix bash vulnerabilities have unleashed an Lernaean Hydra effect, spawning and spurring the discovery of several more bugs.

Exploit Mechanics and Attack Vectors

To recap, the bash bug, now around 22 years old dating back to version 1.13, allows attackers to interact with a host operating system by exploiting a vulnerable web server (Or any bash linked application/service/client). The attacker aims to inject data into system environmental variables. Normally these variables are used by the operating system as a resource for knowledge about where to find required files, configurations for its own function as well as the programs running on it. The bash vulnerability allows malformed data inside these seemingly benign environmental variable scripts. These scripts can contain commands which bash is tricked into interpreting.   In the case of a webserver this can lead to the attacker being able to run commands with the same administrator privileges as the webserver itself, leading to full control of the compromised system.

What does this mean in terms of exploitation possibilities? This vulnerability could open the door to attackers gaining remote access and control of a system to then exploit the system to spread malware, deface system hosted websites, create botnets to launch DDoS attacks, steal personal data, send spam and phishing emails and the perpetration crude but effective destructive wiping attacks. The first Shellshock botnet, “wopbot”, has been used to attack Akamai and US DoD networks.

Security Alerts Issued

The vulnerability is being taken seriously as the ICO, concerned about the disclosure of personal data, urges businesses to install the latest patches for this vulnerability as they become available. CERT-UK, the Computer Emergency Response Team established in March 2014 by the National Cyber Security Strategy, have also issued an alert. CERT-UK suggests that the threat of Shellshock, which affects Unix, Linux, MacOS and many embedded architecture devices, may be bigger than the password revealing, Heartbleed bug, which only affected OpenSSL.

The US National Vulnerability Databases has rated the severity of the bug as 10/10, with an easy exploitation complexity rating of ‘low’. US financial regulators have urged financial institutions to act quickly to identify and fix all of their vulnerable systems and software, including third party applications, to ensure “Shellshock” does not expose them to fraud.

Why is Bash Still Vulnerable?

Several developers have reported that the same vulnerabilities that existed after the first patch (CVE-2014-7169) still exist after the second patch (CVE-2014-6271). These developers are urging a complete rethink in the approach to future patches. Computing Scientist, David A. Wheeler asserts that until bash stops automatically parsing environment variables, it will continue to be vulnerable. An open source developer, Norihiro Tanaka, demonstrated that he was able to bypass the second patch and pass through executable commands by using the name “cat” for the environmental variable, a Unix utility of the same name exists that is used to concatenate files.

Detection

Due to the nature of the attack vectors associated with the Shellshock bug and its prolonged existence, there is a need for organisations to conduct an internal review of their systems and ensure the necessary assurance work is conducted to identify compromises to security and gain assurance.

Shellshock Timeline

Timeline of notable Shellshock bash patches and attacks

Patches Released

On Sept 25^th the Free Software Foundation released bash patches. These patches are reported to fix the original flaw CVE-2014-6271 and an additional flaw, discovered by Tavis Ormandy, which spawned from the incomplete CVE-2014-7169.

Several vendors, including Oracle, have begun releasing patches to fix their products that are liked to bash. Furthermore, Florian Weimar has released an unofficial patch that puts function exporting in a separate namespace. The fix isolates function-parsing code from attacker-controlled strings in several identified use cases.

Proposed Future Solution

So while bash is not yet completely fixed, there is a clear pathway currently under development that aims to fix problems with bash. The approach is twofold, firstly Florain Wiemar, a security expert, proposed the adding of a prefix to bash functions. Adding the prefix aims to prevent functions from having the same name as system variables. Secondly, Dr. Christos Zoulas proposes only allowing the importing of environmental variables via bash when explicitly requested. Explicit requests stop bash command parsing from inadvertently executing malicious code residing in surplus environmental variables.

Solution Implications

An unfortunate caveat of any workable solution is the dissolution of backward compatibility for bash. While Wheeler does not believe the majority of bash-dependent systems would suffer, many software systems that rely on bash may need their code tested and if needs be, modified. However, just as Wheeler has shut down several of his websites until he has confidence the problem has been fixed, the same may need to follow for older services to ensure that assurance via patching and testing can be completed to provide operational confidence.

For now though, it is important for organisations for patch vulnerable systems, establish effective protective network monitoring and to ensure their end users systems are also kept up to date to ensure they are part of the solution.

Bashing Linux Security – The Shellshock Exploit

On Wednesday the world awoke to a substantial vulnerability in the Bourne again shell, otherwise known as bash.  Bash is a UNIX like shell which was created in 1980.  It is now a far cry from the simple terminal based command interpreter it used to be, having grown arms and legs in terms of functionality and has become one of the most installed utilities in Linux systems worldwide.

Exploit Intro

The vulnerability was first discovered by Stephane Chazelas, listed as advisory CVE-2014-7169, this has graduated to CVE-2014-6271 as the first fix released by Red Hat was unsuccessful.  The problem stems from bash allowing arbitrary code execution.  Several services and applications are vulnerable to unauthenticated attackers being able to provide environment variables remotely.   These environment variables are important because they can directly influence software behaviour.

The bash shell is often used remotely by users, for example using ssh to administer a system or service, many programmers run the bash shell in the background and bash provides parsing for services such as Apache and has limited command execution support.

Impact

Attackers exploit this vulnerability by creating environment variables with specific values before summoning the bash shell.  Variables can possess any name and could contain malicious code.  This code is then executed when the bash shell is called.  This will raise many eyebrows as the many ways of calling bash via applications makes this issue perilous.  It is not uncommon for an application to execute binary code by invoking Bash.  The widespread use of bash makes this vulnerability serious and remediation must be fast and effective.

Ease of Exploit

We are aware of exploit code in the wild, which is trivial to use.

Major attack vectors reported so far are HTTP requests and CGI scripts.   AcceptEnv variables in OpenSSH are also open to attack, including TERM and SSH_ORIGINAL_COMMAND.  Environmental variables, named arbitrarily, can contain vicious payloads that seek to exploit the network.

Web applications in PHP, Python, C++ or Java may also be vulnerable if they happen to use libcalls, which are backed by calls to libraries-linked to bash. For example, popen() or system() are both linked to calls to bash.

Test your Bash

Test your version of bash for this vulnerability by running the following command:

$env x='() { :;}; echo I am vulnerable to ShellShock'  bash -c "echo This is a test for CVE-2014-7169"

If the command output as follows:

I am vulnerable to ShellShock
This is a test for CVE-2014-7169

Then your version of bash is vulnerable.

If your version of bash is not vulnerable then your output should be similar to:

bash: warning: x: ignoring function definition attempt bash: error importing function definition for `x' this is a test for CVE-2014-7169

Mitigation

Unfortunately the patch released by the maintainer of bash will not fix the vulnerability.  Its attempts to sanitise code have been shown to be unsuccessful.

There are bash workarounds which may mitigate the vulnerability, however due to the limited testing of these solutions you may find that your fixes have unintended consequences.  However for those that can’t afford to wait, the fix involves upgrading to a new version of bash and imposing access limitations for vulnerable services.  Additionally, filtering the input to vulnerable services such as HTTP is advisable, a Python script that tests URL’s for vulnerabilities can be found on GitHub, contributed by Andres Riancho.

Alternatively, whilst it may not be a practical solution for many, you can replace bash with a different shell.

Update

26-09-2014

Patched versions of bash that fix CVE-2014-7169, have been released by Red Hat. As reports grow of Shellshock being actively exploited, it is strongly recommended that this update be applied as soon as possible.

The fixes apply to Red Hat Enterprise Linux 5, 6 and 7. However it is important to note that updates are applied after you have made changes to your applications that create bash functions as environment variables.

The patch that fixes Shellshock changes how bash evaluates environment variables. This means that any application which creates bash functions as environment variables will need to be updated in line with the patch changes.

To see if your bash is still vulnerable run this command:

env 'x=() { :;}; echo vulnerable' 'BASH_FUNC_x()=() { :;}; echo vulnerable' bash -c "echo test"

If your version of bash is still vulnerable the above command will result in the output “vulnerable”, this stems from the function name needing to be stored in an environment of the same name.

This means that if a function is named “compute” it would be stored in an environment variable named “compute”. After applying the update, the aforementioned function would be named “BASH_FUNC_compute()”. There are now two pairs of parentheses in the environment string, as in “BASH_FUNC_compute()=() {}”.

If after running the test above, you have found your system is vulnerable and are certain your associated applications are updated then update to the most recent version of bash by running:

# yum update bash

You will now need to restart your bash-linked applications to ensure that future environment variables are created using the new version of bash. A reboot of your system and services is not required as the vulnerability stems from initial imports of the kernel process environment. If the update is successfully installed then new processes will use the new code and Red Hat asserts that it will not be vulnerable.

If you believe your system may have been compromised by Shellshock then it is advisable to reboot your system after updating bash. Additionally security checks should be analysed to identify suspicious activity.

Links

https://rhn.redhat.com/errata/RHSA-2014-1306.html

https://access.redhat.com/articles/1200223

PACK AND PIPAL – A COMPARISON

During testing we frequently come across passwords and in most cases, though not all, they are encrypted.  Like in the real world, we will attempt to decrypt these passwords using password cracking tools.  There are many options for password cracking tools but we have recently done some analysis on two of the more common tools in use to figure out how to get the best out of them.  This blog takes a look at “Password Analysis and Cracking Kit” aka PACK and PIPAL for analysing passwords.

It’s what you do with it that counts

It is worth highlighting at the start that both PACK and PIPAL state that the aim of their tool is to help the user in cracking passwords.  They can’t crack passwords on their own, the user needs to direct the tools to get the best results.  Understanding how the tools work is therefore crucial to getting the most out of them.

Overview of the tools

PACK

PACK is developed in Python, and its main functionality is to enhance password cracking through pattern detection of masks, rules, character-sets and other password characteristics.  For interaction and functionality with PACK, the tool consists of 4 main files.

“statsgen.py” – Provides basic statistics such as password length, character-sets used, password complexity, simple masks, advanced masks.  A wordlist is required as an argument for this file.

“maskgen.py” – Provides the ability to craft pattern-based mask attacks.  This file will display output for masks generated, masks coverage, masks runtime.  A list of compatible masks is required, that can be generated from “statsgen.py”.

“policygen.py” – Generates masks that follow the password complexity rules (i.e. domain password policy).  It does not require any previous files, only the configurable options that are available.

“rulegen.py” – Analyses a password/wordlist to automatically detect rules.  Spell checking is provided by using spell-checking engines such as Aspell, MySpell etc.  When analysing a wordlist, the top 10 rules and words are displayed and the following four output base files are created, “analysis.word”, “analysis-sorted.word”, analysis.rule”, “analysis-sorted.rule”, These provide either unsorted and ununique words or rules, or occurrence sorted and unique words or rules.

PIPAL

Pipal is developed in Ruby, and is only compatible with Ruby version(s) 1.9.x.  PIPAL functions slightly differently to PACK.  PIPAL has options to be configured, however they are in the form of symbolic link files, rather than options for each file like PACK.  The user interacts with the following three parts.

“pipal.rb” – This is the main Ruby file used to run the PIPAL tool.

“checkers” – These are files that perform the analysis on passwords. Symbolic link(s) are required.

“splitters” – These are files that take each line in the file and apply a custom splitter.  Symbolic link(s) are required.

The user enables “checkers” by creating a symbolic link(s) within a directory named “checkers_enabled” to the files that are available in the “checkers_available” directory.  This provides a modular and almost “plugin” approach to the tool for analysis.  Each “checker” is a part of the overall analysis and the tool was redesigned to function in this manner. To use a splitter, you have to create a symbolic link of the file you want to use to a file called “custom_splitter.rb” in the main PIPAL directory.  For example, the “pipe_pass_user” splitter can be used if the lines contain the format – “password|username”.  At the moment, only a single splitter can be used at once.

Comparison on functionality

Time Limited Engagements

Frequently our engagements are time limited and therefore the amount of time available to us to crack a password is limited.  PIPAL requires a long time to parse large wordlists.  For example, the “rockyou” wordlist will take at least a few days for PIPAL to analyse on a basic laptop specification. PACK however, is faster at generating output, regardless of the input size.  In addition, PACK has a time management feature built in.  When using the maskgen.py file you can specify a target time for each mask using the ‘targettime’ option.

User Customisation

For PIPAL there are no options to customise processing power but the user is able to create custom masks.  PACK however, has a number of options for tailoring the use of the tool. The tool does not use your current system hardware as an indicator for the analysis, it uses a “pps” option (Passwords per Second) for calculating results.  By tweaking the functionality of this option and the ‘targettime’ option you can gather more accurate output from the tool, and this is key to the functionality of PACK, otherwise the results bare less meaning.  We found that this can be done by editing the default “pps” value in “maskgen.py” (line 39) to something more suitable. If you are going to use PACK often, then you can generate your own benchmarks for the “pps” value by using Hashcat on your system. This will provide you with the most accurate output from the ‘targettime’.

In PACK the user is able to filter the individual masks, such as the minimum and maximum password length.  The “checkmasks” option is extremely useful for a quick analysis of a mask you wish to use against your hash list.  You can supply an argument in the form of a mask and determine which type of coverage (passwords found) it has on the wordlist.

Custom Masks

Creating custom masks allows the user to control the attack and reduce the amount of processing time required.  Both PIPAL and PACK are good for creating custom masks, but in different ways.  PIPAL is more useful for creating custom masks that contain text, not just plain brute-forcing options. PACK is great for creating custom rules based on wordlists or a sample set of cracked passwords.  However, it requires more intelligent pattern detections for custom mask attacks.

Modularity

The code for both tools is modular based and provides many options.  The tools have a “plugin” feel to them (especially PIPAL with its “checkers”).  This greatly enhances the functionality, and allows the tools to be further built upon without disrupting the overall source code.  Some of the features can be enabled and disabled to customise output.

Support and compatibility

Both tools use common scripting languages, “Python” and “Ruby”. This provides compatibility with most operating systems. Included with both tools are valid input files for Hashcat, which is a real bonus for brute-force attacks and rule based attacks.

Pattern Matching

The patterns within PIPAL are more intelligent and as a result, provide better information for decision making than those within PACK.

Rule Analysis

For analysing rules in PACK, the user can take advantage of the “threads” option that will customise the processing power.  The user can then change the output base file names using the “basename” option for relevancy. PACK is great for creating custom rules based on wordlists or a sample set of cracked passwords. The rule generation in PACK though can take a while if there is a large wordlist.

Password Lists

PIPAL includes passwords that match the default Active Directory complexity, which is a useful feature.  It also has username and email checkers built in that can check whether usernames exist in passwords. This uses the Levenshtein string metric between the two sequences.

However, PIPAL requires a long time to parse large wordlists.  For example, the “rockyou” wordlist will take at least a few days for PIPAL to analyse on a basic laptop specification. PIPAL also has encoding problems parsing special characters, in practice it skips each word it finds with a special character and can be a limitation on the accuracy of later results and success.

In addition, the zip codes used within PIPAL are only relevant to USA derived data sets, and can be classed as false positives when dealing with non USA data. This is due to the understanding of how the expression works.  They also take up a lot of time whilst processing lists.

For PACK the top 10 Passwords and base words are not included.  This is a basic feature that should ideally be included. However, the PACK policy generator is great for identifying masks that match a company’s password policy.

Conclusion

As discussed at the start, it is the application of the tool that holds true value not the tool itself. In our opinion, identifying the use case first and then matching this to the best tool to meet that requirement will be of most value. Take time to understand how PIPAL and PACK work and customise their configuration to meet your own needs as this will increase the value and accuracy of the output and directly increase your overall success rate in terms of cracking passwords. At the end of the day this is the aim of using such tools. In our short analysis we found that PIPAL is a more intelligent tool. However, the length of time it can take can be a real disadvantage in a time limited engagement.  PACK’s functionality is more practical to actively being involved in the cracking process, as you can create custom masks based on certain rules, and actively calculate your time more efficiently.  In reality of course, both have their strengths and weaknesses and as a result, the circumstances will dictate the use of the tool.