1008 Commits

Author SHA1 Message Date
rthomas
db4e52dd90 Only wheel files matter 2019-11-24 08:33:17 +01:00
rthomas
fe9315f42d Update changelog 2019-11-23 14:20:38 +01:00
rthomas
bf9e4a259b Fix travis CI 2019-11-23 12:04:12 +01:00
rthomas
42ee4173cd Disable Python 3.8 in Appveyor because of error 2019-11-23 06:53:35 +01:00
Laszlo Kiss-Kollar
5f865a8980 Run build on Python 3.8
Enable Travis and AppVeyor builds on 3.8.
2019-11-23 06:53:35 +01:00
Romain
4744ab12a0
Merge pull request #359 from majin42/fix/pe_missing_offset
[PE] Fix missing offset in resource parsing
2019-11-23 06:32:33 +01:00
rthomas
d838f523bb Trigger Docker on OSX build instead of Linux 2019-11-23 06:31:31 +01:00
majin42
ed2e775e77 reintroduce missing offset parameter in PE resource data parsing 2019-11-22 10:40:49 +01:00
rthomas
934d84ffea Move to Pybind11 v2.4.3 and fix enum conflicts 2019-11-19 06:41:11 +01:00
rthomas
d400ca429d Add blog post about kaitai and LIEF 2019-11-08 16:05:59 +01:00
rthomas
4105c31f61 Missing import 2019-11-08 16:05:59 +01:00
Romain
657d589fae
Merge pull request #352 from recvfrom/master
Expose raw bytes of Authenticode-related structures and other improvements
2019-10-08 10:10:38 +02:00
Andrew
08d31be932 Authenticode-related bug fixes and improvements
Part of Authenticode verification consists of:
 - Comparing the computed Authenticode hash to the digest
   stored in the ContentInfo section
 - Comparing hash(ContentInfo) to the digest stored in the
   AuthenticatedAttributes section
 - Verifying signed(hash(AuthenticatedAttributes)) using a
   certificate identified by the issuer and serial number
   specified in the SignerInfo section

This commit makes it so that the raw bytes needed to
calculate hash(ContentInfo) and hash(AuthenticatedAttributes)
are available for use.

============================================================

Allow missing [0] in SpcSpOpusInfo

Some executables have MoreInfo but not a ProgramName (and the documentation
lists both as OPTIONAL), so handle this case correctly.

Example:

```
01416b1730218454c99b13592650cb170402b86742b4bab971565903b841829b

SEQUENCE(2 elem)
OBJECT IDENTIFIER1.3.6.1.4.1.311.2.1.12spcSpOpusInfo(Microsoft code signing)
SET(1 elem)
  SEQUENCE(1 elem)
  [1](1 elem)
   [0]http://www.mozilla.com
```

============================================================

Improve consistency of parsed serial numbers

When parsing the issuer serial number, call mbedtls_x509_get_serial instead of
parsing it as an integer directly with mbedtls_asn1_get_mpi. These two functions
differ in how they treat serial numbers prepended with '00' to prevent them from
being negative (the former preserves the '00', and the latter discards it). The
embedded certs are parsed via a call to mbedtls_x509_crt_parse_der, which uses
mbedtls_x509_get_serial behind the scenes, so there was an inconsistency between
lief_obj.signature.signer_info.issuer[1] and
lief_obj.signature.certificates[x].serial_number.  Example:

8bf57d97dd917c4f823659266caaa33e7398406daf11ba8318e3f7414ee3fb24

============================================================

Handle SpcLink and SpcString CHOICEs in SpcSpOpusInfo

The Authenticode spec doc says that these can be CHOICES, so
handle the easy ones and safely skip the others.

============================================================

Allow Authenticode sig to be parsed even if cert parsing fails

By default, mbedtls doesn't support MD2 certs, which are fairly
common in older signed executables.  Ex:

1cb16f94cebdcad7dd05c8537375a6ff6379fcdb08528fc83889f26efaa84e2a

============================================================

Enable mbed TLS MD2 and MD4 support; add Unix debug options

By default, mbedtls doesn't support MD2 certs, which are fairly
common in older signed executables.  Ex:

1cb16f94cebdcad7dd05c8537375a6ff6379fcdb08528fc83889f26efaa84e2a

============================================================

Set MBEDTLS_X509_ALLOW_UNSUPPORTED_CRITICAL_EXTENSION flag

Some older signed executables use certs with the SpcSpAgencyInfo
Critical Extension, which mbed TLS doesn't support, so set
MBEDTLS_X509_ALLOW_UNSUPPORTED_CRITICAL_EXTENSION to have it
skip this extension.  Example:

781ca31416ec708a587851dafd90c661b86f244ab8b8475c4185e958e54ff838

============================================================

Support accessing non-utf8 issuer names via Python

For a few signatures where the issuer name contained non-utf8
characters, accessing the issuer name field in Python would raise
a UnicodeDecodeError exception. Now this field is handled the
same way the names in the individual certs are (I'm not sure if
they get represented 100% correctly, but at least they are
consistent, which is good enough for me). Example:

048f91b9302c88380fc66adac1e314d82733089ef3a31eadca5f0cb4169b195f
2019-10-07 00:58:40 -04:00
rthomas
208264cb66 Remove unused files
Resolve #348
2019-10-02 08:13:12 +02:00
rthomas
e563d682bf Update changelog 2019-10-02 07:55:04 +02:00
Laszlo Kiss-Kollar
3550e45f18 Build manylinux1-compliant wheels in Travis
* Temporarily disable Melkor in test suite
The Makefile uses the -executable flag which doesn't work on RHEL5.
Disabling until I find a workaround for this.

* Patch Makefile in Melkor fuzzer
This replaces the `-executable` flag which is not supported by `find` on
CentOS 5.

* Respect CC environment variable
Several tests hard coded /usr/bin/cc which might not exist in some
environments. We first check the CC environment variable and fall back
to the hard coded path if CC is unset.

* Skip tests on GLIBC < 2.17
Some test binaries were linked against GLIBC 2.17. Skip tests which use
this binary if the platform does not have the required GLIBC version.

* Enable ccache in Docker in Travis builds

* Run `auditwheel repair` on the produced wheels
This will vendor the needed external shared libraries into the wheel and
tag it as manylinux1.

* Install ccache in Docker image

* Avoid using bind mount volume in Docker build

The bind mount volume wrote files as 'root' which causes issues with the
deploy script in Travis. Copying the source code into the image and
retrieving the built wheels instead of mounting the source tree fixes
this issue.

* Fix missing build folder when building with Docker

After finishing the build inside Docker we need the build directory from
the container to be able to deploy the built artifacts with deploy.sh.

* Use the right Python interpreter for Linux builds

The Dockerized .travis.yml builds attempt to invoke the interpreter in
the PYTHON_BINARY environment variable, which is only valid inside the
Docker image. To fix this, override the variable on Linux for tasks
which require the host's Python interpreter.

* Fix missing pip installation in Travis

The Ubuntu image in Travis does not come with `pip` preinstalled for
Python 3.

* Remove .git directory from .dockerignore

As `setup.py` uses `git` to determine the version number we need to copy
the contents of `.git` into the image.
2019-10-02 07:49:45 +02:00
Romain Thomas
e50436164e Fix missing code-block 2019-09-09 17:28:43 +02:00
Romain Thomas
27666e4f02 Add Android x86/x86-64 cmake scripts 2019-09-09 07:44:59 +02:00
Romain Thomas
f6cc8f8ff6 Improve debug build on Windows 2019-08-29 08:24:39 +02:00
Romain Thomas
6cf1e2f9df Fix doc about compilation 2019-08-28 07:16:28 +02:00
Romain Thomas
4d4f96d268 Fix local variable life-time.
Resolve #331
2019-08-27 07:54:33 +02:00
Romain Thomas
273f001895 Add script for AArch64 2019-08-24 17:28:34 +02:00
Romain Thomas
40570e6522 Enhance LIEF modules configuration 2019-08-23 08:07:40 +02:00
Romain Thomas
088951ddb7 Resolve #329 2019-08-23 07:54:23 +02:00
Chris
2cc22a792f Update 09_frida_lief.rst (#330)
Fix typo
2019-08-23 07:25:32 +02:00
Romain Thomas
04037644af Update README 2019-08-21 07:36:05 +02:00
wisk
df2d56c532 Fix PE available sections space
(cherry picked from commit fb311c28f2f6dd682d195a235ebddf549618b84b)
2019-07-29 09:37:54 +02:00
wisk
2b1214b770 Use C++ header for Mach-O
(cherry picked from commit e477afad1f0f13acc8cbca8a80d1bc7935c0923f)
2019-07-29 09:37:54 +02:00
Andrew Williams
fcb9c76be6 Expose the raw bytes backing each PE Authenticode cert (#320) 2019-07-26 19:26:13 +02:00
Romain Thomas
1b1a616ac7 Add missing entry 2019-07-26 09:09:07 +02:00
Romain Thomas
715448d3b8 Merge branch 'fix/cmake' 2019-07-25 13:09:59 +02:00
Andrew Williams
20f46e74a3 Unify SignerInfo issuer field format (#319)
* Unify SignerInfo issuer field format

This makes it so that the SignerInfo issuer field has the same
format as the issuer fields in each x509 cert, so the two can
be more easily compared.

Also, this commit adds '0x' in front of the Data Directory
RVAs and sizes to make it more clear that the values are printed
in hex.

* Add missing include in SignerInfo.cpp

* Address Codacy feedback and minor change to text

The serial number of the signing certificate will now be
printed as 'Serial Number' instead of 'Issuer Serial Number'
since the latter is a bit confusing (the serial number in
the SignerInfo section is the serial number of the certificate
that actually signed the executable)

* Update python pe_reader example based on issuer changes
2019-07-25 11:38:42 +02:00
Romain Thomas
d4ad532c58 Downgrade cmake_minimum_required to 3.5 2019-07-24 14:56:25 +02:00
Romain Thomas
3a11b24798 Add alias 2019-07-10 06:59:41 +02:00
Romain
bab1c1d5b7
Merge pull request #317 from recvfrom/master
Update the PE Authenticode parsing code
2019-07-10 06:54:08 +02:00
Andrew
535623de3a Update the PE Authenticode parsing code
- Allow UTF8String and IA5String types when parsing the Issuer fields
   (previously only PrintableString types were handled).  Handling these
   three types should be sufficient, based on the default types indicated
   at https://github.com/ARMmbed/mbedtls/blob/master/library/x509_create.c#L52.
   The approach used is similar to what's done in mbedtls's internal
   x509_get_attr_type_value function.

   Examples:
```
     8a364e0881fd7201cd6f0a0ff747451c9b93182d5699afb28ad8466f7f726660:
     SEQUENCE (4 elem)
       SET (1 elem)
         SEQUENCE (2 elem)
           OBJECT IDENTIFIER 2.5.4.6 countryName (X.520 DN component)
           PrintableString PL
       SET (1 elem)
         SEQUENCE (2 elem)
           OBJECT IDENTIFIER 2.5.4.10 organizationName (X.520 DN component)
           UTF8String Unizeto Technologies S.A.
       SET (1 elem)
         SEQUENCE (2 elem)
           OBJECT IDENTIFIER 2.5.4.11 organizationalUnitName (X.520 DN component)
           UTF8String Certum Certification Authority
       SET (1 elem)
         SEQUENCE (2 elem)
           OBJECT IDENTIFIER 2.5.4.3 commonName (X.520 DN component)
           UTF8String Certum Code Signing CA SHA2

     From a test binary compiled with osslsigncode:
     SEQUENCE (6 elem)
       SET (1 elem)
         SEQUENCE (2 elem)
           OBJECT IDENTIFIER 2.5.4.6 countryName (X.520 DN component)
           PrintableString US
       SET (1 elem)
         SEQUENCE (2 elem)
           OBJECT IDENTIFIER 2.5.4.8 stateOrProvinceName (X.520 DN component)
           UTF8String State
       SET (1 elem)
         SEQUENCE (2 elem)
           OBJECT IDENTIFIER 2.5.4.7 localityName (X.520 DN component)
           UTF8String City
       SET (1 elem)
         SEQUENCE (2 elem)
           OBJECT IDENTIFIER 2.5.4.10 organizationName (X.520 DN component)
           UTF8String Cisco Talos
       SET (1 elem)
         SEQUENCE (2 elem)
           OBJECT IDENTIFIER 2.5.4.11 organizationalUnitName (X.520 DN component)
           UTF8String Test CA 1704a8ea9e24d8ed
       SET (1 elem)
         SEQUENCE (2 elem)
           OBJECT IDENTIFIER 1.2.840.113549.1.9.1 emailAddress (PKCS #9. Deprecated, use an altName extension instead)
           IA5String rfc2606@example.net
```

 - Allow any order for the AuthenticatedAttributes, and ignore ones that
   aren't recognized. The code was looking for a strict ordering of the
   attributes, but Windows seems to be OK with any order.
   Example:
```
     From f91e258ea71dcbfc82371b2ee3e20852e45bef0cb946223d1141a6ef1dfb793f:
     SEQUENCE (2 elem)
       OBJECT IDENTIFIER 1.3.6.1.4.1.311.2.1.12 spcSpOpusInfo (Microsoft code signing)
       SET (1 elem)
         SEQUENCE (0 elem)
     SEQUENCE (2 elem)
       OBJECT IDENTIFIER 1.2.840.113549.1.9.3 contentType (PKCS #9)
       SET (1 elem)
         OBJECT IDENTIFIER 1.3.6.1.4.1.311.2.1.4 spcIndirectDataContext (Microsoft code signing)
     SEQUENCE (2 elem)
       OBJECT IDENTIFIER 1.3.6.1.4.1.311.2.1.11 spcStatementType (Microsoft code signing)
       SET (1 elem)
         SEQUENCE (1 elem)
           OBJECT IDENTIFIER 1.3.6.1.4.1.311.2.1.21 individualCodeSigning (Microsoft)
     SEQUENCE (2 elem)
       OBJECT IDENTIFIER 1.2.840.113549.1.9.4 messageDigest (PKCS #9)
       SET (1 elem)
         OCTET STRING (20 byte) 7C87D331C6E62C0EC840BC23CA63FBC2CE68586F
```

 - Allow the program name and/or more info to be missing from SpcSpOpusInfo.
   Examples:
```
     8a364e0881fd7201cd6f0a0ff747451c9b93182d5699afb28ad8466f7f726660:
     SEQUENCE (2 elem)
       OBJECT IDENTIFIER 1.3.6.1.4.1.311.2.1.12 spcSpOpusInfo (Microsoft code signing)
       SET (1 elem)
         SEQUENCE (0 elem)

     0059fb3f225c5784789622eeccb97197d591972851b63d59f5bd107ddfdb7a21:
     SEQUENCE (2 elem)
       OBJECT IDENTIFIER 1.3.6.1.4.1.311.2.1.12 spcSpOpusInfo (Microsoft code signing)
       SET (1 elem)
         SEQUENCE (1 elem)
           [0] (1 elem)
             [0] (22 byte) 0047006F0054006F0020004F00700065006E00650072
```

 - I removed the #if 0 ContentInfo from being populated, and the code worked
   with all of the test samples I used... Is there another reason this code
   is commented out?

 - The authenticated_attribytes content_name_ member now gets populated. This
   was commented out with a TODO by it... Is there a reason this field wasn't
   being populated?

 - When the SignerInfo is printed, it will now show 'N/A' if no value was
   parsed out from the executable
2019-07-09 13:51:20 -04:00
Romain Thomas
91eb4783c3 Resolve #313 2019-07-08 08:40:10 +02:00
Romain Thomas
8f2770506d Remove unecessary file 2019-07-08 08:14:20 +02:00
Romain
826965be1b
Merge pull request #308 from jlewallen/master
Fix performance issue parsing relocations.
2019-06-26 09:18:15 +02:00
Romain Thomas
bd9be9a765 Add notice regarding LIEF imphash implementation
Related to #299
2019-06-26 09:10:35 +02:00
Jacob Lewallen
3be83b7b22 Fix performance issue parsing relocations.
Uses a map to track added relocations.
2019-06-24 19:53:15 -07:00
Romain Thomas
7ff0859879 Fix terminal columns 2019-06-23 10:20:21 +02:00
Romain Thomas
778763de88 Fix Python return value policy for `get_symbol()`
Resolve #305
2019-06-23 10:19:14 +02:00
Romain
9dd9ded098
Merge pull request #301 from mackncheesiest/LIEF-300
Fixes issues with printing/serializing notes
2019-06-08 15:38:16 +02:00
Joshua Mack
89c7afa680 Adds regression check for Note::dump segfault 2019-06-07 18:04:08 -07:00
Joshua Mack
d188fbdd23 Assigns alignment of note segments that are added by lief user
Adds support for serialization of hwcap and unknown notes through
Builder::build()

Gives unknown note types a default ".note" namespace rather than
potentially alias a ".note.unknown" type in the future
2019-06-07 14:37:34 -07:00
Joshua Mack
c9c49bb4b8 Adds ability to clone ELF NoteDetails objects
Subsequently, calling print(mybin) on an ELF binary that has a new note
added it no longer segfaults
2019-06-06 21:34:03 -07:00
Romain Thomas
fa3195b1bd Add API to check if a section associated with an Off/VA exists 2019-05-28 09:47:33 +02:00
Romain Thomas
8fb74bec60 Fix ELF `patch_address` function for object files (.o) 2019-05-20 11:01:40 +02:00
Romain Thomas
22344e764e Fix conflict when the builder tries to resolve the DT_XXX_ARRAY section 2019-05-20 10:59:39 +02:00