diff --git a/NOTES-PERL.md b/NOTES-PERL.md index 13565dea6c..dbaae0d40e 100644 --- a/NOTES-PERL.md +++ b/NOTES-PERL.md @@ -1,125 +1,133 @@ -TOC -=== +Notes on Perl +============= - - Notes on Perl - - Notes on Perl on Windows - - Notes on Perl modules we use - - Notes on installing a perl module + - [General Notes](#general-notes) + - [Perl on Windows](#perl-on-windows) + - [Perl on VMS](#perl-on-vms) + - [Required Perl modules](#required-perl-modules) + - [Notes on installing a Perl module](#notes-on-installing-a-perl-module]) - Notes on Perl - ------------- - For our scripts, we rely quite a bit on Perl, and increasingly on - some core Perl modules. These Perl modules are part of the Perl - source, so if you build Perl on your own, you should be set. +General Notes +------------- - However, if you install Perl as binary packages, the outcome might - differ, and you may have to check that you do get the core modules - installed properly. We do not claim to know them all, but experience - has told us the following: +For our scripts, we rely quite a bit on Perl, and increasingly on +some core Perl modules. These Perl modules are part of the Perl +source, so if you build Perl on your own, you should be set. + +However, if you install Perl as binary packages, the outcome might +differ, and you may have to check that you do get the core modules +installed properly. We do not claim to know them all, but experience +has told us the following: - on Linux distributions based on Debian, the package `perl` will install the core Perl modules as well, so you will be fine. - on Linux distributions based on RPMs, you will need to install `perl-core` rather than just `perl`. - You MUST have at least Perl version 5.10.0 installed. This minimum - requirement is due to our use of regexp backslash sequence \R among - other features that didn't exist in core Perl before that version. +You MUST have at least Perl version 5.10.0 installed. This minimum +requirement is due to our use of regexp backslash sequence \R among +other features that didn't exist in core Perl before that version. - Notes on Perl on Windows - ------------------------ +Perl on Windows +--------------- - There are a number of build targets that can be viewed as "Windows". - Indeed, there are `VC-*` configs targeting VisualStudio C, as well as - MinGW and Cygwin. The key recommendation is to use "matching" Perl, - one that matches build environment. For example, if you will build - on Cygwin be sure to use the Cygwin package manager to install Perl. - For MSYS builds use the MSYS provided Perl. - For VC-* builds we recommend Strawberry Perl, from . - An alternative is ActiveState Perl, from - for which you may need to explicitly select the Perl module Win32/Console.pm - available via . +There are a number of build targets that can be viewed as "Windows". +Indeed, there are `VC-*` configs targeting VisualStudio C, as well as +MinGW and Cygwin. The key recommendation is to use a Perl installation +that matches the build environment. For example, if you will build +on Cygwin be sure to use the Cygwin package manager to install Perl. +For MSYS builds use the MSYS provided Perl. +For VC-* builds we recommend Strawberry Perl, from . +An alternative is ActiveState Perl, from +for which you may need to explicitly select the Perl module Win32/Console.pm +available via . - Notes on Perl on VMS - -------------------- +Perl on VMS +----------- - You will need to install Perl separately. One way to do so is to - download the source from , unpacking it, reading - `README-VMS.md` and follow the instructions. Another way is to download a - `.PCSI` file from and install it using the - POLYCENTER install tool. +You will need to install Perl separately. One way to do so is to +download the source from , unpacking it, reading +`README-VMS.md` and follow the instructions. Another way is to download a +`.PCSI` file from and install it using the +POLYCENTER install tool. - Notes on Perl modules we use - ---------------------------- +Required Perl modules +--------------------- - We make increasing use of Perl modules, and do our best to limit - ourselves to core Perl modules to keep the requirements down. There - are just a few exceptions: +We do our best to limit ourselves to core Perl modules to keep the +requirements down. There are just a few exceptions. - * `Test::More` - We require the minimum version to be 0.96, which - appeared in Perl 5.13.4, because that version was - the first to have all the features we're using. - This module is required for testing only! - If you don't plan on running the tests, - you don't need to bother with this one. +## For Building - * `Text::Template` + * `Text::Template` - This module is not part of the core Perl modules. - As a matter of fact, the core Perl modules do not - include any templating module to date. - This module is absolutely needed, - configuration depends on it. + This module is not part of the core Perl modules. + As a matter of fact, the core Perl modules do not + include any templating module to date. + This module is absolutely needed, + configuration depends on it. - To avoid unnecessary initial hurdles, we have bundled a copy of the - following modules in our source. They will work as fallbacks if - these modules aren't already installed on the system. +## For Testing - Text::Template + * `Test::More` - Notes on installing a perl module - --------------------------------- + We require the minimum version to be 0.96, which + appeared in Perl 5.13.4, because that version was + the first to have all the features we're using. + This module is required for testing only! + If you don't plan on running the tests, + you don't need to bother with this one. - There are a number of ways to install a perl module. In all - descriptions below, `Text::Template` will serve as an example. - 1. for Linux users, the easiest is to install with the use of your - favorite package manager. Usually, all you need to do is search - for the module name and to install the package that comes up. - On Debian based Linux distributions, it would go like this: +To avoid unnecessary initial hurdles, we have bundled a copy of the +following modules in our source. They will work as fallbacks if +these modules aren't already installed on the system. - $ apt-cache search Text::Template - ... - libtext-template-perl - perl module to process text templates - $ sudo apt-get install libtext-template-perl + Text::Template - Perl modules in Debian based distributions use package names like - the name of the module in question, with "lib" prepended and - "-perl" appended. +Notes on installing a Perl module +--------------------------------- - 2. Install using CPAN. This is very easy, but usually requires root - access: +There are a number of ways to install a perl module. In all +descriptions below, `Text::Template` will serve as an example. - $ cpan -i Text::Template +1. for Linux users, the easiest is to install with the use of your + favorite package manager. Usually, all you need to do is search + for the module name and to install the package that comes up. - Note that this runs all the tests that the module to be installed - comes with. This is usually a smooth operation, but there are - platforms where a failure is indicated even though the actual tests - were successful. Should that happen, you can force an - installation regardless (that should be safe since you've already - seen the tests succeed!): + On Debian based Linux distributions, it would go like this: - $ cpan -f -i Text::Template + $ apt-cache search Text::Template + ... + libtext-template-perl - perl module to process text templates + $ sudo apt-get install libtext-template-perl - Note: on VMS, you must quote any argument that contains upper case - characters, so the lines above would be: + Perl modules in Debian based distributions use package names like + the name of the module in question, with "lib" prepended and + "-perl" appended. - $ cpan -i "Text::Template" +2. Install using CPAN. This is very easy, but usually requires root + access: - and: + $ cpan -i Text::Template - $ cpan -f -i "Text::Template" + Note that this runs all the tests that the module to be installed + comes with. This is usually a smooth operation, but there are + platforms where a failure is indicated even though the actual tests + were successful. Should that happen, you can force an + installation regardless (that should be safe since you've already + seen the tests succeed!): + + $ cpan -f -i Text::Template + + Note: on VMS, you must quote any argument that contains upper case + characters, so the lines above would be: + + $ cpan -i "Text::Template" + + and: + + $ cpan -f -i "Text::Template" diff --git a/NOTES-VMS.md b/NOTES-VMS.md index ebb1e8e152..02e6cbcb8d 100644 --- a/NOTES-VMS.md +++ b/NOTES-VMS.md @@ -1,102 +1,110 @@ -NOTES FOR THE OPENVMS PLATFORM +Notes for the OpenVMS platform ============================== - Requirement details - ------------------- + - [Requirement details](#requirement-details) + - [About ANSI C compiler](#about-ansi-c-compiler) + - [About ODS-5 directory names and Perl](#about-ods-5-directory-names-and-perl) + - [About MMS and DCL](#about-mms-and-dcl) + - [About debugging](#about-debugging) + - [Checking the distribution](#checking-the-distribution) - In addition to the requirements and instructions listed - in [INSTALL.md](INSTALL.md), this are required as well: + +Requirement details +------------------- + +In addition to the requirements and instructions listed +in [INSTALL.md](INSTALL.md), this are required as well: * At least ODS-5 disk organization for source and build. Installation can be done on any existing disk organization. - About ANSI C compiler - --------------------- +About ANSI C compiler +--------------------- - An ANSI C compiled is needed among other things. This means that - VAX C is not and will not be supported. +An ANSI C compiled is needed among other things. This means that +VAX C is not and will not be supported. - We have only tested with DEC C (aka HP VMS C / VSI C) and require - version 7.1 or later. Compiling with a different ANSI C compiler may - require some work. +We have only tested with DEC C (aka HP VMS C / VSI C) and require +version 7.1 or later. Compiling with a different ANSI C compiler may +require some work. - Please avoid using C RTL feature logical names `DECC$*` when building - and testing OpenSSL. Most of all, they can be disruptive when - running the tests, as they affect the Perl interpreter. +Please avoid using C RTL feature logical names `DECC$*` when building +and testing OpenSSL. Most of all, they can be disruptive when +running the tests, as they affect the Perl interpreter. - About ODS-5 directory names and Perl - ------------------------------------ +About ODS-5 directory names and Perl +------------------------------------ - It seems that the perl function canonpath() in the `File::Spec` module - doesn't treat file specifications where the last directory name - contains periods very well. Unfortunately, some versions of VMS tar - will keep the periods in the OpenSSL source directory instead of - converting them to underscore, thereby leaving your source in - something like `[.openssl-1^.1^.0]`. This will lead to issues when - configuring and building OpenSSL. +It seems that the perl function canonpath() in the `File::Spec` module +doesn't treat file specifications where the last directory name +contains periods very well. Unfortunately, some versions of VMS tar +will keep the periods in the OpenSSL source directory instead of +converting them to underscore, thereby leaving your source in +something like `[.openssl-1^.1^.0]`. This will lead to issues when +configuring and building OpenSSL. - We have no replacement for Perl's canonpath(), so the best workaround - for now is to rename the OpenSSL source directory, as follows (please - adjust for the actual source directory name you have): +We have no replacement for Perl's canonpath(), so the best workaround +for now is to rename the OpenSSL source directory, as follows (please +adjust for the actual source directory name you have): $ rename openssl-1^.1^.0.DIR openssl-1_1_0.DIR - About MMS and DCL - ----------------- +About MMS and DCL +----------------- - MMS has certain limitations when it comes to line length, and DCL has - certain limitations when it comes to total command length. We do - what we can to mitigate, but there is the possibility that it's not - enough. Should you run into issues, a very simple solution is to set - yourself up a few logical names for the directory trees you're going - to use. +MMS has certain limitations when it comes to line length, and DCL has +certain limitations when it comes to total command length. We do +what we can to mitigate, but there is the possibility that it's not +enough. Should you run into issues, a very simple solution is to set +yourself up a few logical names for the directory trees you're going +to use. - About debugging - --------------- +About debugging +--------------- - If you build for debugging, the default on VMS is that image - activation starts the debugger automatically, giving you a debug - prompt. Unfortunately, this disrupts all other uses, such as running - test programs in the test framework. +If you build for debugging, the default on VMS is that image +activation starts the debugger automatically, giving you a debug +prompt. Unfortunately, this disrupts all other uses, such as running +test programs in the test framework. - Generally speaking, if you build for debugging, only use the programs - directly for debugging. Do not try to use them from a script, such - as running the test suite. +Generally speaking, if you build for debugging, only use the programs +directly for debugging. Do not try to use them from a script, such +as running the test suite. - ### The following is not available on Alpha +### The following is not available on Alpha - As a compromise, we're turning off the flag that makes the debugger - start automatically. If there is a program that you need to debug, - you need to turn that flag back on first, for example: +As a compromise, we're turning off the flag that makes the debugger +start automatically. If there is a program that you need to debug, +you need to turn that flag back on first, for example: $ set image /flag=call_debug [.test]evp_test.exe - Then just run it and you will find yourself in a debugging session. - When done, we recommend that you turn that flag back off: +Then just run it and you will find yourself in a debugging session. +When done, we recommend that you turn that flag back off: $ set image /flag=nocall_debug [.test]evp_test.exe - Checking the distribution - ------------------------- +Checking the distribution +------------------------- - There have been reports of places where the distribution didn't quite - get through, for example if you've copied the tree from a NFS-mounted - Unix mount point. +There have been reports of places where the distribution didn't quite +get through, for example if you've copied the tree from a NFS-mounted +Unix mount point. - The easiest way to check if everything got through as it should is to - check that this file exists: +The easiest way to check if everything got through as it should is to +check that this file exists: [.include.openssl]configuration^.h.in - The best way to get a correct distribution is to download the gzipped - tar file from ftp://ftp.openssl.org/source/, use `GZIP -d` to uncompress - it and `VMSTAR` to unpack the resulting tar file. +The best way to get a correct distribution is to download the gzipped +tar file from ftp://ftp.openssl.org/source/, use `GZIP -d` to uncompress +it and `VMSTAR` to unpack the resulting tar file. - Gzip and VMSTAR are available here: +Gzip and VMSTAR are available here: - Should you need it, you can find UnZip for VMS here: +Should you need it, you can find UnZip for VMS here: diff --git a/NOTES-WINDOWS.md b/NOTES-WINDOWS.md index cb62e1d9bd..9acee570a1 100644 --- a/NOTES-WINDOWS.md +++ b/NOTES-WINDOWS.md @@ -1,46 +1,57 @@ -NOTES FOR WINDOWS PLATFORMS +Notes for Windows platforms =========================== - There are various options to build and run OpenSSL on the Windows platforms. + - [Native builds using Visual C++](#native-builds-using-visual-c++) + - [Native builds using MinGW](#native-builds-using-mingw) + - [Linking native applications](#linking-native-applications) + - [Hosted builds using Cygwin](#hosted-builds-using-cygwin) - "Native" OpenSSL uses the Windows APIs directly at run time. - To build a native OpenSSL you can either use: - Microsoft Visual C++ (MSVC) C compiler on the command line - or - MinGW cross compiler - run on the GNU-like development environment MSYS2 - or run on Linux or Cygwin +There are various options to build and run OpenSSL on the Windows platforms. - "Hosted" OpenSSL relies on an external POSIX compatibility layer - for building (using GNU/Unix shell, compiler, and tools) and at run time. - For this option you can use Cygwin. +"Native" OpenSSL uses the Windows APIs directly at run time. +To build a native OpenSSL you can either use: - Visual C++ native builds, aka VC-* - ===================================== + Microsoft Visual C++ (MSVC) C compiler on the command line +or + MinGW cross compiler + run on the GNU-like development environment MSYS2 + or run on Linux or Cygwin - Requirement details - ------------------- +"Hosted" OpenSSL relies on an external POSIX compatibility layer +for building (using GNU/Unix shell, compiler, and tools) and at run time. +For this option you can use Cygwin. - In addition to the requirements and instructions listed in INSTALL.md, - these are required as well: +Native builds using Visual C++ +============================== - - Perl. - We recommend Strawberry Perl, available from - Please read NOTES.PERL for more information, including the use of CPAN. - An alternative is ActiveState Perl, - for which you may need to explicitly build the Perl module Win32/Console.pm - via and then download it. +The native builds using Visual C++ have a VC-* prefix. - - Microsoft Visual C compiler. - Since these are proprietary and ever-changing we cannot test them all. - Older versions may not work. Use a recent version wherever possible. +Requirement details +------------------- - - Netwide Assembler (NASM), available from - Note that NASM is the only supported assembler. +In addition to the requirements and instructions listed in INSTALL.md, +these are required as well: - Quick start - ----------- +### Perl + +We recommend Strawberry Perl, available from +Please read NOTES.PERL for more information, including the use of CPAN. +An alternative is ActiveState Perl, +for which you may need to explicitly build the Perl module Win32/Console.pm +via and then download it. + +### Microsoft Visual C compiler. + +Since these are proprietary and ever-changing we cannot test them all. +Older versions may not work. Use a recent version wherever possible. + +### Netwide Assembler (NASM) + +NASM is the only supported assembler. It is available from . + +Quick start +----------- 1. Install Perl @@ -67,40 +78,40 @@ NOTES FOR WINDOWS PLATFORMS 8. nmake install - For the full installation instructions, or if anything goes wrong at any stage, - check the INSTALL.md file. +For the full installation instructions, or if anything goes wrong at any stage, +check the INSTALL.md file. - Installation directories - ------------------------ +Installation directories +------------------------ - The default installation directories are derived from environment - variables. +The default installation directories are derived from environment +variables. - For VC-WIN32, the following defaults are use: +For VC-WIN32, the following defaults are use: - PREFIX: %ProgramFiles(86)%\OpenSSL - OPENSSLDIR: %CommonProgramFiles(86)%\SSL + PREFIX: %ProgramFiles(86)%\OpenSSL + OPENSSLDIR: %CommonProgramFiles(86)%\SSL - For VC-WIN64, the following defaults are use: +For VC-WIN64, the following defaults are use: - PREFIX: %ProgramW6432%\OpenSSL - OPENSSLDIR: %CommonProgramW6432%\SSL + PREFIX: %ProgramW6432%\OpenSSL + OPENSSLDIR: %CommonProgramW6432%\SSL - Should those environment variables not exist (on a pure Win32 - installation for examples), these fallbacks are used: +Should those environment variables not exist (on a pure Win32 +installation for examples), these fallbacks are used: - PREFIX: %ProgramFiles%\OpenSSL - OPENSSLDIR: %CommonProgramFiles%\SSL + PREFIX: %ProgramFiles%\OpenSSL + OPENSSLDIR: %CommonProgramFiles%\SSL - ALSO NOTE that those directories are usually write protected, even if - your account is in the Administrators group. To work around that, - start the command prompt by right-clicking on it and choosing "Run as - Administrator" before running 'nmake install'. The other solution - is, of course, to choose a different set of directories by using - --prefix and --openssldir when configuring. +ALSO NOTE that those directories are usually write protected, even if +your account is in the Administrators group. To work around that, +start the command prompt by right-clicking on it and choosing "Run as +Administrator" before running 'nmake install'. The other solution +is, of course, to choose a different set of directories by using +--prefix and --openssldir when configuring. - Special notes for Universal Windows Platform builds, aka VC-*-UWP - -------------------------------------------------------------------- +Special notes for Universal Windows Platform builds, aka VC-*-UWP +-------------------------------------------------------------------- - UWP targets only support building the static and dynamic libraries. @@ -108,10 +119,10 @@ NOTES FOR WINDOWS PLATFORMS "vcvarsall.bat" before you compile. For example, if you want to build "arm64" builds, you should run "vcvarsall.bat x86_arm64 uwp". - Native OpenSSL built using MinGW - ================================ +Native builds using MinGW +========================= - MinGW offers an alternative way to build native OpenSSL, by cross compilation. +MinGW offers an alternative way to build native OpenSSL, by cross compilation. * Usually the build is done on Windows in a GNU-like environment called MSYS2. @@ -135,11 +146,12 @@ NOTES FOR WINDOWS PLATFORMS In the MSYS2 shell do the configuration depending on the target architecture: - ./Configure mingw ... + ./Configure mingw ... or - ./Configure mingw64 ... + ./Configure mingw64 ... or - ./Configure ... + ./Configure ... + for the default architecture. Apart from that, follow the Unix / Linux instructions in INSTALL.md. @@ -149,55 +161,55 @@ NOTES FOR WINDOWS PLATFORMS In this case configure with the corresponding --cross-compile-prefix= option. For example - ./Configure mingw --cross-compile-prefix=i686-w64-mingw32- ... + ./Configure mingw --cross-compile-prefix=i686-w64-mingw32- ... or - ./Configure mingw64 --cross-compile-prefix=x86_64-w64-mingw32- ... + ./Configure mingw64 --cross-compile-prefix=x86_64-w64-mingw32- ... This requires that you've installed the necessary add-on packages for mingw[64] cross compilation. - Linking your application - ======================== +Linking native applications +=========================== - This section applies to all "native" builds. +This section applies to all native builds. - If you link with static OpenSSL libraries then you're expected to - additionally link your application with WS2_32.LIB, GDI32.LIB, - ADVAPI32.LIB, CRYPT32.LIB and USER32.LIB. Those developing - non-interactive service applications might feel concerned about - linking with GDI32.LIB and USER32.LIB, as they are justly associated - with interactive desktop, which is not available to service - processes. The toolkit is designed to detect in which context it's - currently executed, GUI, console app or service, and act accordingly, - namely whether or not to actually make GUI calls. Additionally those - who wish to /DELAYLOAD:GDI32.DLL and /DELAYLOAD:USER32.DLL and - actually keep them off service process should consider implementing - and exporting from .exe image in question own _OPENSSL_isservice not - relying on USER32.DLL. E.g., on Windows Vista and later you could: +If you link with static OpenSSL libraries then you're expected to +additionally link your application with WS2_32.LIB, GDI32.LIB, +ADVAPI32.LIB, CRYPT32.LIB and USER32.LIB. Those developing +non-interactive service applications might feel concerned about +linking with GDI32.LIB and USER32.LIB, as they are justly associated +with interactive desktop, which is not available to service +processes. The toolkit is designed to detect in which context it's +currently executed, GUI, console app or service, and act accordingly, +namely whether or not to actually make GUI calls. Additionally those +who wish to /DELAYLOAD:GDI32.DLL and /DELAYLOAD:USER32.DLL and +actually keep them off service process should consider implementing +and exporting from .exe image in question own _OPENSSL_isservice not +relying on USER32.DLL. E.g., on Windows Vista and later you could: - __declspec(dllexport) __cdecl BOOL _OPENSSL_isservice(void) - { - DWORD sess; + __declspec(dllexport) __cdecl BOOL _OPENSSL_isservice(void) + { + DWORD sess; - if (ProcessIdToSessionId(GetCurrentProcessId(), &sess)) - return sess == 0; - return FALSE; - } + if (ProcessIdToSessionId(GetCurrentProcessId(), &sess)) + return sess == 0; + return FALSE; + } - If you link with OpenSSL .DLLs, then you're expected to include into - your application code a small "shim" snippet, which provides - the glue between the OpenSSL BIO layer and your compiler run-time. - See also the OPENSSL_Applink manual page. +If you link with OpenSSL .DLLs, then you're expected to include into +your application code a small "shim" snippet, which provides +the glue between the OpenSSL BIO layer and your compiler run-time. +See also the OPENSSL_Applink manual page. - Hosted OpenSSL built using Cygwin - ================================= +Hosted builds using Cygwin +========================== - Cygwin implements a POSIX/Unix runtime system (cygwin1.dll) on top of the - Windows subsystem and provides a Bash shell and GNU tools environment. - Consequently, a build of OpenSSL with Cygwin is virtually identical to the - Unix procedure. +Cygwin implements a POSIX/Unix runtime system (cygwin1.dll) on top of the +Windows subsystem and provides a Bash shell and GNU tools environment. +Consequently, a build of OpenSSL with Cygwin is virtually identical to the +Unix procedure. - To build OpenSSL using Cygwin, you need to: +To build OpenSSL using Cygwin, you need to: * Install Cygwin, see @@ -206,9 +218,9 @@ NOTES FOR WINDOWS PLATFORMS * Run the Cygwin Bash shell - Apart from that, follow the Unix / Linux instructions in INSTALL.md. +Apart from that, follow the Unix / Linux instructions in INSTALL.md. - NOTE: "make test" and normal file operations may fail in directories - mounted as text (i.e. mount -t c:\somewhere /home) due to Cygwin - stripping of carriage returns. To avoid this ensure that a binary - mount is used, e.g. mount -b c:\somewhere /home. +NOTE: "make test" and normal file operations may fail in directories +mounted as text (i.e. mount -t c:\somewhere /home) due to Cygwin +stripping of carriage returns. To avoid this ensure that a binary +mount is used, e.g. mount -b c:\somewhere /home. diff --git a/README-ENGINES.md b/README-ENGINES.md index 2fc4e40a2b..9783a211a5 100644 --- a/README-ENGINES.md +++ b/README-ENGINES.md @@ -1,308 +1,308 @@ ENGINES ======= - With OpenSSL 0.9.6, a new component was added to support alternative - cryptography implementations, most commonly for interfacing with external - crypto devices (eg. accelerator cards). This component is called ENGINE, - and its presence in OpenSSL 0.9.6 (and subsequent bug-fix releases) - caused a little confusion as 0.9.6** releases were rolled in two - versions, a "standard" and an "engine" version. In development for 0.9.7, - the ENGINE code has been merged into the main branch and will be present - in the standard releases from 0.9.7 forwards. +With OpenSSL 0.9.6, a new component was added to support alternative +cryptography implementations, most commonly for interfacing with external +crypto devices (eg. accelerator cards). This component is called ENGINE, +and its presence in OpenSSL 0.9.6 (and subsequent bug-fix releases) +caused a little confusion as 0.9.6** releases were rolled in two +versions, a "standard" and an "engine" version. In development for 0.9.7, +the ENGINE code has been merged into the main branch and will be present +in the standard releases from 0.9.7 forwards. - There are currently built-in ENGINE implementations for the following - crypto devices: +There are currently built-in ENGINE implementations for the following +crypto devices: * Microsoft CryptoAPI * VIA Padlock * nCipher CHIL - In addition, dynamic binding to external ENGINE implementations is now - provided by a special ENGINE called "dynamic". See the "DYNAMIC ENGINE" - section below for details. +In addition, dynamic binding to external ENGINE implementations is now +provided by a special ENGINE called "dynamic". See the "DYNAMIC ENGINE" +section below for details. - At this stage, a number of things are still needed and are being worked on: +At this stage, a number of things are still needed and are being worked on: 1. Integration of EVP support. 2. Configuration support. 3. Documentation! - Integration of EVP support - -------------------------- +Integration of EVP support +-------------------------- - With respect to EVP, this relates to support for ciphers and digests in - the ENGINE model so that alternative implementations of existing - algorithms/modes (or previously unimplemented ones) can be provided by - ENGINE implementations. +With respect to EVP, this relates to support for ciphers and digests in +the ENGINE model so that alternative implementations of existing +algorithms/modes (or previously unimplemented ones) can be provided by +ENGINE implementations. - Configuration support - --------------------- +Configuration support +--------------------- - Configuration support currently exists in the ENGINE API itself, in the - form of "control commands". These allow an application to expose to the - user/admin the set of commands and parameter types a given ENGINE - implementation supports, and for an application to directly feed string - based input to those ENGINEs, in the form of name-value pairs. This is an - extensible way for ENGINEs to define their own "configuration" mechanisms - that are specific to a given ENGINE (eg. for a particular hardware - device) but that should be consistent across *all* OpenSSL-based - applications when they use that ENGINE. Work is in progress (or at least - in planning) for supporting these control commands from the CONF (or - NCONF) code so that applications using OpenSSL's existing configuration - file format can have ENGINE settings specified in much the same way. - Presently however, applications must use the ENGINE API itself to provide - such functionality. To see first hand the types of commands available - with the various compiled-in ENGINEs (see further down for dynamic - ENGINEs), use the "engine" openssl utility with full verbosity, i.e.: +Configuration support currently exists in the ENGINE API itself, in the +form of "control commands". These allow an application to expose to the +user/admin the set of commands and parameter types a given ENGINE +implementation supports, and for an application to directly feed string +based input to those ENGINEs, in the form of name-value pairs. This is an +extensible way for ENGINEs to define their own "configuration" mechanisms +that are specific to a given ENGINE (eg. for a particular hardware +device) but that should be consistent across *all* OpenSSL-based +applications when they use that ENGINE. Work is in progress (or at least +in planning) for supporting these control commands from the CONF (or +NCONF) code so that applications using OpenSSL's existing configuration +file format can have ENGINE settings specified in much the same way. +Presently however, applications must use the ENGINE API itself to provide +such functionality. To see first hand the types of commands available +with the various compiled-in ENGINEs (see further down for dynamic +ENGINEs), use the "engine" openssl utility with full verbosity, i.e.: - openssl engine -vvvv + openssl engine -vvvv - Documentation - ------------- +Documentation +------------- - Documentation? Volunteers welcome! The source code is reasonably well - self-documenting, but some summaries and usage instructions are needed - - moreover, they are needed in the same POD format the existing OpenSSL - documentation is provided in. Any complete or incomplete contributions - would help make this happen. +Documentation? Volunteers welcome! The source code is reasonably well +self-documenting, but some summaries and usage instructions are needed - +moreover, they are needed in the same POD format the existing OpenSSL +documentation is provided in. Any complete or incomplete contributions +would help make this happen. - STABILITY & BUG-REPORTS - ======================= +STABILITY & BUG-REPORTS +======================= - What already exists is fairly stable as far as it has been tested, but - the test base has been a bit small most of the time. For the most part, - the vendors of the devices these ENGINEs support have contributed to the - development and/or testing of the implementations, and *usually* (with no - guarantees) have experience in using the ENGINE support to drive their - devices from common OpenSSL-based applications. Bugs and/or inexplicable - behaviour in using a specific ENGINE implementation should be sent to the - author of that implementation (if it is mentioned in the corresponding C - file), and in the case of implementations for commercial hardware - devices, also through whatever vendor support channels are available. If - none of this is possible, or the problem seems to be something about the - ENGINE API itself (ie. not necessarily specific to a particular ENGINE - implementation) then you should mail complete details to the relevant - OpenSSL mailing list. For a definition of "complete details", refer to - the OpenSSL "README" file. As for which list to send it to: +What already exists is fairly stable as far as it has been tested, but +the test base has been a bit small most of the time. For the most part, +the vendors of the devices these ENGINEs support have contributed to the +development and/or testing of the implementations, and *usually* (with no +guarantees) have experience in using the ENGINE support to drive their +devices from common OpenSSL-based applications. Bugs and/or inexplicable +behaviour in using a specific ENGINE implementation should be sent to the +author of that implementation (if it is mentioned in the corresponding C +file), and in the case of implementations for commercial hardware +devices, also through whatever vendor support channels are available. If +none of this is possible, or the problem seems to be something about the +ENGINE API itself (ie. not necessarily specific to a particular ENGINE +implementation) then you should mail complete details to the relevant +OpenSSL mailing list. For a definition of "complete details", refer to +the OpenSSL "README" file. As for which list to send it to: * openssl-users: if you are *using* the ENGINE abstraction, either in an pre-compiled application or in your own application code. * openssl-dev: if you are discussing problems with OpenSSL source code. - USAGE - ===== +USAGE +===== - The default "openssl" ENGINE is always chosen when performing crypto - operations unless you specify otherwise. You must actively tell the - openssl utility commands to use anything else through a new command line - switch called "-engine". Also, if you want to use the ENGINE support in - your own code to do something similar, you must likewise explicitly - select the ENGINE implementation you want. +The default "openssl" ENGINE is always chosen when performing crypto +operations unless you specify otherwise. You must actively tell the +openssl utility commands to use anything else through a new command line +switch called "-engine". Also, if you want to use the ENGINE support in +your own code to do something similar, you must likewise explicitly +select the ENGINE implementation you want. - Depending on the type of hardware, system, and configuration, "settings" - may need to be applied to an ENGINE for it to function as expected/hoped. - The recommended way of doing this is for the application to support - ENGINE "control commands" so that each ENGINE implementation can provide - whatever configuration primitives it might require and the application - can allow the user/admin (and thus the hardware vendor's support desk - also) to provide any such input directly to the ENGINE implementation. - This way, applications do not need to know anything specific to any - device, they only need to provide the means to carry such user/admin - input through to the ENGINE in question. Ie. this connects *you* (and - your helpdesk) to the specific ENGINE implementation (and device), and - allows application authors to not get buried in hassle supporting - arbitrary devices they know (and care) nothing about. +Depending on the type of hardware, system, and configuration, "settings" +may need to be applied to an ENGINE for it to function as expected/hoped. +The recommended way of doing this is for the application to support +ENGINE "control commands" so that each ENGINE implementation can provide +whatever configuration primitives it might require and the application +can allow the user/admin (and thus the hardware vendor's support desk +also) to provide any such input directly to the ENGINE implementation. +This way, applications do not need to know anything specific to any +device, they only need to provide the means to carry such user/admin +input through to the ENGINE in question. Ie. this connects *you* (and +your helpdesk) to the specific ENGINE implementation (and device), and +allows application authors to not get buried in hassle supporting +arbitrary devices they know (and care) nothing about. - A new "openssl" utility, "openssl engine", has been added in that allows - for testing and examination of ENGINE implementations. Basic usage - instructions are available by specifying the "-?" command line switch. +A new "openssl" utility, "openssl engine", has been added in that allows +for testing and examination of ENGINE implementations. Basic usage +instructions are available by specifying the "-?" command line switch. - DYNAMIC ENGINES - =============== +DYNAMIC ENGINES +=============== - The new "dynamic" ENGINE provides a low-overhead way to support ENGINE - implementations that aren't pre-compiled and linked into OpenSSL-based - applications. This could be because existing compiled-in implementations - have known problems and you wish to use a newer version with an existing - application. It could equally be because the application (or OpenSSL - library) you are using simply doesn't have support for the ENGINE you - wish to use, and the ENGINE provider (eg. hardware vendor) is providing - you with a self-contained implementation in the form of a shared-library. - The other use-case for "dynamic" is with applications that wish to - maintain the smallest foot-print possible and so do not link in various - ENGINE implementations from OpenSSL, but instead leaves you to provide - them, if you want them, in the form of "dynamic"-loadable - shared-libraries. It should be possible for hardware vendors to provide - their own shared-libraries to support arbitrary hardware to work with - applications based on OpenSSL 0.9.7 or later. If you're using an - application based on 0.9.7 (or later) and the support you desire is only - announced for versions later than the one you need, ask the vendor to - backport their ENGINE to the version you need. +The new "dynamic" ENGINE provides a low-overhead way to support ENGINE +implementations that aren't pre-compiled and linked into OpenSSL-based +applications. This could be because existing compiled-in implementations +have known problems and you wish to use a newer version with an existing +application. It could equally be because the application (or OpenSSL +library) you are using simply doesn't have support for the ENGINE you +wish to use, and the ENGINE provider (eg. hardware vendor) is providing +you with a self-contained implementation in the form of a shared-library. +The other use-case for "dynamic" is with applications that wish to +maintain the smallest foot-print possible and so do not link in various +ENGINE implementations from OpenSSL, but instead leaves you to provide +them, if you want them, in the form of "dynamic"-loadable +shared-libraries. It should be possible for hardware vendors to provide +their own shared-libraries to support arbitrary hardware to work with +applications based on OpenSSL 0.9.7 or later. If you're using an +application based on 0.9.7 (or later) and the support you desire is only +announced for versions later than the one you need, ask the vendor to +backport their ENGINE to the version you need. - How does "dynamic" work? - ------------------------ +How does "dynamic" work? +------------------------ - The dynamic ENGINE has a special flag in its implementation such that - every time application code asks for the 'dynamic' ENGINE, it in fact - gets its own copy of it. As such, multi-threaded code (or code that - multiplexes multiple uses of 'dynamic' in a single application in any - way at all) does not get confused by 'dynamic' being used to do many - independent things. Other ENGINEs typically don't do this so there is - only ever 1 ENGINE structure of its type (and reference counts are used - to keep order). The dynamic ENGINE itself provides absolutely no - cryptographic functionality, and any attempt to "initialise" the ENGINE - automatically fails. All it does provide are a few "control commands" - that can be used to control how it will load an external ENGINE - implementation from a shared-library. To see these control commands, - use the command-line; +The dynamic ENGINE has a special flag in its implementation such that +every time application code asks for the 'dynamic' ENGINE, it in fact +gets its own copy of it. As such, multi-threaded code (or code that +multiplexes multiple uses of 'dynamic' in a single application in any +way at all) does not get confused by 'dynamic' being used to do many +independent things. Other ENGINEs typically don't do this so there is +only ever 1 ENGINE structure of its type (and reference counts are used +to keep order). The dynamic ENGINE itself provides absolutely no +cryptographic functionality, and any attempt to "initialise" the ENGINE +automatically fails. All it does provide are a few "control commands" +that can be used to control how it will load an external ENGINE +implementation from a shared-library. To see these control commands, +use the command-line; openssl engine -vvvv dynamic - The "SO_PATH" control command should be used to identify the - shared-library that contains the ENGINE implementation, and "NO_VCHECK" - might possibly be useful if there is a minor version conflict and you - (or a vendor helpdesk) is convinced you can safely ignore it. - "ID" is probably only needed if a shared-library implements - multiple ENGINEs, but if you know the engine id you expect to be using, - it doesn't hurt to specify it (and this provides a sanity check if - nothing else). "LIST_ADD" is only required if you actually wish the - loaded ENGINE to be discoverable by application code later on using the - ENGINE's "id". For most applications, this isn't necessary - but some - application authors may have nifty reasons for using it. The "LOAD" - command is the only one that takes no parameters and is the command - that uses the settings from any previous commands to actually *load* - the shared-library ENGINE implementation. If this command succeeds, the - (copy of the) 'dynamic' ENGINE will magically morph into the ENGINE - that has been loaded from the shared-library. As such, any control - commands supported by the loaded ENGINE could then be executed as per - normal. Eg. if ENGINE "foo" is implemented in the shared-library - "libfoo.so" and it supports some special control command "CMD_FOO", the - following code would load and use it (NB: obviously this code has no - error checking); +The "SO_PATH" control command should be used to identify the +shared-library that contains the ENGINE implementation, and "NO_VCHECK" +might possibly be useful if there is a minor version conflict and you +(or a vendor helpdesk) is convinced you can safely ignore it. +"ID" is probably only needed if a shared-library implements +multiple ENGINEs, but if you know the engine id you expect to be using, +it doesn't hurt to specify it (and this provides a sanity check if +nothing else). "LIST_ADD" is only required if you actually wish the +loaded ENGINE to be discoverable by application code later on using the +ENGINE's "id". For most applications, this isn't necessary - but some +application authors may have nifty reasons for using it. The "LOAD" +command is the only one that takes no parameters and is the command +that uses the settings from any previous commands to actually *load* +the shared-library ENGINE implementation. If this command succeeds, the +(copy of the) 'dynamic' ENGINE will magically morph into the ENGINE +that has been loaded from the shared-library. As such, any control +commands supported by the loaded ENGINE could then be executed as per +normal. Eg. if ENGINE "foo" is implemented in the shared-library +"libfoo.so" and it supports some special control command "CMD_FOO", the +following code would load and use it (NB: obviously this code has no +error checking); - ENGINE *e = ENGINE_by_id("dynamic"); - ENGINE_ctrl_cmd_string(e, "SO_PATH", "/lib/libfoo.so", 0); - ENGINE_ctrl_cmd_string(e, "ID", "foo", 0); - ENGINE_ctrl_cmd_string(e, "LOAD", NULL, 0); - ENGINE_ctrl_cmd_string(e, "CMD_FOO", "some input data", 0); + ENGINE *e = ENGINE_by_id("dynamic"); + ENGINE_ctrl_cmd_string(e, "SO_PATH", "/lib/libfoo.so", 0); + ENGINE_ctrl_cmd_string(e, "ID", "foo", 0); + ENGINE_ctrl_cmd_string(e, "LOAD", NULL, 0); + ENGINE_ctrl_cmd_string(e, "CMD_FOO", "some input data", 0); - For testing, the "openssl engine" utility can be useful for this sort - of thing. For example the above code excerpt would achieve much the - same result as; +For testing, the "openssl engine" utility can be useful for this sort +of thing. For example the above code excerpt would achieve much the +same result as; - openssl engine dynamic \ - -pre SO_PATH:/lib/libfoo.so \ - -pre ID:foo \ - -pre LOAD \ - -pre "CMD_FOO:some input data" + openssl engine dynamic \ + -pre SO_PATH:/lib/libfoo.so \ + -pre ID:foo \ + -pre LOAD \ + -pre "CMD_FOO:some input data" - Or to simply see the list of commands supported by the "foo" ENGINE; +Or to simply see the list of commands supported by the "foo" ENGINE; - openssl engine -vvvv dynamic \ - -pre SO_PATH:/lib/libfoo.so \ - -pre ID:foo \ - -pre LOAD + openssl engine -vvvv dynamic \ + -pre SO_PATH:/lib/libfoo.so \ + -pre ID:foo \ + -pre LOAD - Applications that support the ENGINE API and more specifically, the - "control commands" mechanism, will provide some way for you to pass - such commands through to ENGINEs. As such, you would select "dynamic" - as the ENGINE to use, and the parameters/commands you pass would - control the *actual* ENGINE used. Each command is actually a name-value - pair and the value can sometimes be omitted (eg. the "LOAD" command). - Whilst the syntax demonstrated in "openssl engine" uses a colon to - separate the command name from the value, applications may provide - their own syntax for making that separation (eg. a win32 registry - key-value pair may be used by some applications). The reason for the - "-pre" syntax in the "openssl engine" utility is that some commands - might be issued to an ENGINE *after* it has been initialised for use. - Eg. if an ENGINE implementation requires a smart-card to be inserted - during initialisation (or a PIN to be typed, or whatever), there may be - a control command you can issue afterwards to "forget" the smart-card - so that additional initialisation is no longer possible. In - applications such as web-servers, where potentially volatile code may - run on the same host system, this may provide some arguable security - value. In such a case, the command would be passed to the ENGINE after - it has been initialised for use, and so the "-post" switch would be - used instead. Applications may provide a different syntax for - supporting this distinction, and some may simply not provide it at all - ("-pre" is almost always what you're after, in reality). +Applications that support the ENGINE API and more specifically, the +"control commands" mechanism, will provide some way for you to pass +such commands through to ENGINEs. As such, you would select "dynamic" +as the ENGINE to use, and the parameters/commands you pass would +control the *actual* ENGINE used. Each command is actually a name-value +pair and the value can sometimes be omitted (eg. the "LOAD" command). +Whilst the syntax demonstrated in "openssl engine" uses a colon to +separate the command name from the value, applications may provide +their own syntax for making that separation (eg. a win32 registry +key-value pair may be used by some applications). The reason for the +"-pre" syntax in the "openssl engine" utility is that some commands +might be issued to an ENGINE *after* it has been initialised for use. +Eg. if an ENGINE implementation requires a smart-card to be inserted +during initialisation (or a PIN to be typed, or whatever), there may be +a control command you can issue afterwards to "forget" the smart-card +so that additional initialisation is no longer possible. In +applications such as web-servers, where potentially volatile code may +run on the same host system, this may provide some arguable security +value. In such a case, the command would be passed to the ENGINE after +it has been initialised for use, and so the "-post" switch would be +used instead. Applications may provide a different syntax for +supporting this distinction, and some may simply not provide it at all +("-pre" is almost always what you're after, in reality). - How do I build a "dynamic" ENGINE? - ---------------------------------- +How do I build a "dynamic" ENGINE? +---------------------------------- - This question is trickier - currently OpenSSL bundles various ENGINE - implementations that are statically built in, and any application that - calls the "ENGINE_load_builtin_engines()" function will automatically - have all such ENGINEs available (and occupying memory). Applications - that don't call that function have no ENGINEs available like that and - would have to use "dynamic" to load any such ENGINE - but on the other - hand such applications would only have the memory footprint of any - ENGINEs explicitly loaded using user/admin provided control commands. - The main advantage of not statically linking ENGINEs and only using - "dynamic" for hardware support is that any installation using no - "external" ENGINE suffers no unnecessary memory footprint from unused - ENGINEs. Likewise, installations that do require an ENGINE incur the - overheads from only *that* ENGINE once it has been loaded. +This question is trickier - currently OpenSSL bundles various ENGINE +implementations that are statically built in, and any application that +calls the "ENGINE_load_builtin_engines()" function will automatically +have all such ENGINEs available (and occupying memory). Applications +that don't call that function have no ENGINEs available like that and +would have to use "dynamic" to load any such ENGINE - but on the other +hand such applications would only have the memory footprint of any +ENGINEs explicitly loaded using user/admin provided control commands. +The main advantage of not statically linking ENGINEs and only using +"dynamic" for hardware support is that any installation using no +"external" ENGINE suffers no unnecessary memory footprint from unused +ENGINEs. Likewise, installations that do require an ENGINE incur the +overheads from only *that* ENGINE once it has been loaded. - Sounds good? Maybe, but currently building an ENGINE implementation as - a shared-library that can be loaded by "dynamic" isn't automated in - OpenSSL's build process. It can be done manually quite easily however. - Such a shared-library can either be built with any OpenSSL code it - needs statically linked in, or it can link dynamically against OpenSSL - if OpenSSL itself is built as a shared library. The instructions are - the same in each case, but in the former (statically linked any - dependencies on OpenSSL) you must ensure OpenSSL is built with - position-independent code ("PIC"). The default OpenSSL compilation may - already specify the relevant flags to do this, but you should consult - with your compiler documentation if you are in any doubt. +Sounds good? Maybe, but currently building an ENGINE implementation as +a shared-library that can be loaded by "dynamic" isn't automated in +OpenSSL's build process. It can be done manually quite easily however. +Such a shared-library can either be built with any OpenSSL code it +needs statically linked in, or it can link dynamically against OpenSSL +if OpenSSL itself is built as a shared library. The instructions are +the same in each case, but in the former (statically linked any +dependencies on OpenSSL) you must ensure OpenSSL is built with +position-independent code ("PIC"). The default OpenSSL compilation may +already specify the relevant flags to do this, but you should consult +with your compiler documentation if you are in any doubt. - This example will show building the "atalla" ENGINE in the - crypto/engine/ directory as a shared-library for use via the "dynamic" - ENGINE. +This example will show building the "atalla" ENGINE in the +crypto/engine/ directory as a shared-library for use via the "dynamic" +ENGINE. 1. "cd" to the crypto/engine/ directory of a pre-compiled OpenSSL - source tree. + source tree. 2. Recompile at least one source file so you can see all the compiler - flags (and syntax) being used to build normally. Eg; + flags (and syntax) being used to build normally. Eg; - touch hw_atalla.c ; make + touch hw_atalla.c ; make - will rebuild "hw_atalla.o" using all such flags. + will rebuild "hw_atalla.o" using all such flags. 3. Manually enter the same compilation line to compile the - "hw_atalla.c" file but with the following two changes; - * add "-DENGINE_DYNAMIC_SUPPORT" to the command line switches, - * change the output file from "hw_atalla.o" to something new, - eg. "tmp_atalla.o" + "hw_atalla.c" file but with the following two changes; + * add "-DENGINE_DYNAMIC_SUPPORT" to the command line switches, + * change the output file from "hw_atalla.o" to something new, + eg. "tmp_atalla.o" 4. Link "tmp_atalla.o" into a shared-library using the top-level - OpenSSL libraries to resolve any dependencies. The syntax for doing - this depends heavily on your system/compiler and is a nightmare - known well to anyone who has worked with shared-library portability - before. 'gcc' on Linux, for example, would use the following syntax; + OpenSSL libraries to resolve any dependencies. The syntax for doing + this depends heavily on your system/compiler and is a nightmare + known well to anyone who has worked with shared-library portability + before. 'gcc' on Linux, for example, would use the following syntax; - gcc -shared -o dyn_atalla.so tmp_atalla.o -L../.. -lcrypto + gcc -shared -o dyn_atalla.so tmp_atalla.o -L../.. -lcrypto 5. Test your shared library using "openssl engine" as explained in the - previous section. Eg. from the top-level directory, you might try + previous section. Eg. from the top-level directory, you might try - apps/openssl engine -vvvv dynamic \ - -pre SO_PATH:./crypto/engine/dyn_atalla.so -pre LOAD + apps/openssl engine -vvvv dynamic \ + -pre SO_PATH:./crypto/engine/dyn_atalla.so -pre LOAD - If the shared-library loads successfully, you will see both "-pre" - commands marked as "SUCCESS" and the list of control commands - displayed (because of "-vvvv") will be the control commands for the - *atalla* ENGINE (ie. *not* the 'dynamic' ENGINE). You can also add - the "-t" switch to the utility if you want it to try and initialise - the atalla ENGINE for use to test any possible hardware/driver issues. +If the shared-library loads successfully, you will see both "-pre" +commands marked as "SUCCESS" and the list of control commands +displayed (because of "-vvvv") will be the control commands for the +*atalla* ENGINE (ie. *not* the 'dynamic' ENGINE). You can also add +the "-t" switch to the utility if you want it to try and initialise +the atalla ENGINE for use to test any possible hardware/driver issues. - PROBLEMS - ======== +PROBLEMS +======== - It seems like the ENGINE part doesn't work too well with CryptoSwift on Win32. - A quick test done right before the release showed that trying "openssl speed - -engine cswift" generated errors. If the DSO gets enabled, an attempt is made - to write at memory address 0x00000002. +It seems like the ENGINE part doesn't work too well with CryptoSwift on Win32. +A quick test done right before the release showed that trying "openssl speed +-engine cswift" generated errors. If the DSO gets enabled, an attempt is made +to write at memory address 0x00000002.