Merge pull request #3122 from mpg/drafts-became-rfcs
Update references from drafts to RFCs in config.h
diff --git a/.github/issue_template.md b/.github/issue_template.md
index 7c31353..18b87fc 100644
--- a/.github/issue_template.md
+++ b/.github/issue_template.md
@@ -1,7 +1,7 @@
Note: This is just a template, so feel free to use/remove the unnecessary things
### Description
-- Type: Bug | Enhancement\Feature Request | Question
+- Type: Bug | Enhancement\Feature Request
- Priority: Blocker | Major | Minor
---------------------------------------------------------------
@@ -38,4 +38,4 @@
## Question
-**Please first check for answers in the [Mbed TLS knowledge Base](https://tls.mbed.org/kb), and preferably file an issue in the [Mbed TLS support forum](https://forums.mbed.com/c/mbed-tls)**
+**Please first check for answers in the [Mbed TLS knowledge Base](https://tls.mbed.org/kb). If you can't find the answer you're looking for then please use the [Mbed TLS mailing list](https://lists.trustedfirmware.org/mailman/listinfo/mbed-tls)**
diff --git a/ChangeLog b/ChangeLog
index bcceebb..5d49004 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -6,9 +6,19 @@
* Deprecate MBEDTLS_SSL_HW_RECORD_ACCEL that enables function hooks in the
SSL module for hardware acceleration of individual records.
+Security
+ * Fix issue in DTLS handling of new associations with the same parameters
+ (RFC 6347 section 4.2.8): an attacker able to send forged UDP packets to
+ the server could cause it to drop established associations with
+ legitimate clients, resulting in a Denial of Service. This could only
+ happen when MBEDTLS_SSL_DTLS_CLIENT_PORT_REUSE was enabled in config.h
+ (which it is by default).
+
Bugfix
* Fix compilation failure when both MBEDTLS_SSL_PROTO_DTLS and
MBEDTLS_SSL_HW_RECORD_ACCEL are enabled.
+ * Remove a spurious check in ssl_parse_client_psk_identity that triggered
+ a warning with some compilers. Fix contributed by irwir in #2856.
Changes
* Mbed Crypto is no longer a Git submodule. The crypto part of the library
diff --git a/ChangeLog.d/00README.md b/ChangeLog.d/00README.md
new file mode 100644
index 0000000..b559e23
--- /dev/null
+++ b/ChangeLog.d/00README.md
@@ -0,0 +1,67 @@
+# Pending changelog entry directory
+
+This directory contains changelog entries that have not yet been merged
+to the changelog file ([`../ChangeLog`](../ChangeLog)).
+
+## Changelog entry file format
+
+A changelog entry file must have the extension `*.txt` and must have the
+following format:
+
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Security
+ * Change description.
+ * Another change description.
+
+Features
+ * Yet another change description. This is a long change description that
+ spans multiple lines.
+ * Yet again another change description.
+
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The permitted changelog entry categories are as follows:
+<!-- Keep this synchronized with STANDARD_CATEGORIES in assemble_changelog.py! -->
+
+ API changes
+ Default behavior changes
+ Requirement changes
+ New deprecations
+ Removals
+ Features
+ Security
+ Bugfix
+ Changes
+
+Use “Changes” for anything that doesn't fit in the other categories, such as
+performance, documentation and test improvements.
+
+## How to write a changelog entry
+
+Each entry starts with three spaces, an asterisk and a space. Continuation
+lines start with 5 spaces. Lines wrap at 79 characters.
+
+Write full English sentences with proper capitalization and punctuation. Use
+the present tense. Use the imperative where applicable. For example: “Fix a
+bug in mbedtls_xxx() ….”
+
+Include GitHub issue numbers where relevant. Use the format “#1234” for an
+Mbed TLS issue. Add other external references such as CVE numbers where
+applicable.
+
+Credit the author of the contribution if the contribution is not a member of
+the Mbed TLS development team. Also credit bug reporters where applicable.
+
+**Explain why, not how**. Remember that the audience is the users of the
+library, not its developers. In particular, for a bug fix, explain the
+consequences of the bug, not how the bug was fixed. For a new feature, explain
+why one might be interested in the feature. For an API change or a deprecation,
+explain how to update existing applications.
+
+See [existing entries](../ChangeLog) for examples.
+
+## How `ChangeLog` is updated
+
+Run [`../scripts/assemble_changelog.py`](../scripts/assemble_changelog.py)
+from a Git working copy
+to move the entries from files in `ChangeLog.d` to the main `ChangeLog` file.
diff --git a/doxygen/mbedtls.doxyfile b/doxygen/mbedtls.doxyfile
index 4732271..148fa27 100644
--- a/doxygen/mbedtls.doxyfile
+++ b/doxygen/mbedtls.doxyfile
@@ -1594,7 +1594,7 @@
# contain include files that are not input files but should be processed by
# the preprocessor.
-INCLUDE_PATH =
+INCLUDE_PATH = ../include
# You can use the INCLUDE_FILE_PATTERNS tag to specify one or more wildcard
# patterns (like *.h and *.hpp) to filter out the header-files in the
diff --git a/include/mbedtls/check_config.h b/include/mbedtls/check_config.h
index d904d5a..fa3caa7 100644
--- a/include/mbedtls/check_config.h
+++ b/include/mbedtls/check_config.h
@@ -619,6 +619,23 @@
#error "MBEDTLS_SSL_PROTO_TLS1_2 defined, but not all prerequisites"
#endif
+#if (defined(MBEDTLS_SSL_PROTO_SSL3) || defined(MBEDTLS_SSL_PROTO_TLS1) || \
+ defined(MBEDTLS_SSL_PROTO_TLS1_1) || defined(MBEDTLS_SSL_PROTO_TLS1_2)) && \
+ !(defined(MBEDTLS_KEY_EXCHANGE_RSA_ENABLED) || \
+ defined(MBEDTLS_KEY_EXCHANGE_DHE_RSA_ENABLED) || \
+ defined(MBEDTLS_KEY_EXCHANGE_ECDHE_RSA_ENABLED) || \
+ defined(MBEDTLS_KEY_EXCHANGE_ECDHE_ECDSA_ENABLED) || \
+ defined(MBEDTLS_KEY_EXCHANGE_ECDH_RSA_ENABLED) || \
+ defined(MBEDTLS_KEY_EXCHANGE_ECDH_ECDSA_ENABLED) || \
+ defined(MBEDTLS_KEY_EXCHANGE_PSK_ENABLED) || \
+ defined(MBEDTLS_KEY_EXCHANGE_DHE_PSK_ENABLED) || \
+ defined(MBEDTLS_KEY_EXCHANGE_RSA_PSK_ENABLED) || \
+ defined(MBEDTLS_KEY_EXCHANGE_ECDHE_PSK_ENABLED) || \
+ defined(MBEDTLS_KEY_EXCHANGE_ECJPAKE_ENABLED) )
+#error "One or more versions of the TLS protocol are enabled " \
+ "but no key exchange methods defined with MBEDTLS_KEY_EXCHANGE_xxxx"
+#endif
+
#if defined(MBEDTLS_SSL_PROTO_DTLS) && \
!defined(MBEDTLS_SSL_PROTO_TLS1_1) && \
!defined(MBEDTLS_SSL_PROTO_TLS1_2)
@@ -763,6 +780,10 @@
#error "MBEDTLS_X509_CREATE_C defined, but not all prerequisites"
#endif
+#if defined(MBEDTLS_CERTS_C) && !defined(MBEDTLS_X509_USE_C)
+#error "MBEDTLS_CERTS_C defined, but not all prerequisites"
+#endif
+
#if defined(MBEDTLS_X509_CRT_PARSE_C) && ( !defined(MBEDTLS_X509_USE_C) )
#error "MBEDTLS_X509_CRT_PARSE_C defined, but not all prerequisites"
#endif
diff --git a/library/Makefile b/library/Makefile
index 92799d9..f9cd468 100644
--- a/library/Makefile
+++ b/library/Makefile
@@ -201,8 +201,6 @@
echo " LD $@"
$(CC) -shared -Wl,-soname,$@ -Wl,--out-implib,$@.a -o $@ $(OBJS_CRYPTO) -lws2_32 -lwinmm -lgdi32 -static-libgcc $(LOCAL_LDFLAGS) $(LDFLAGS)
-libmbedcrypto.$(DLEXT): | libmbedcrypto.a
-
.c.o:
echo " CC $<"
$(CC) $(LOCAL_CFLAGS) $(CFLAGS) -o $@ -c $<
diff --git a/library/ssl_cli.c b/library/ssl_cli.c
index c0b440a..ff6b7b6 100644
--- a/library/ssl_cli.c
+++ b/library/ssl_cli.c
@@ -2344,7 +2344,7 @@
unsigned char *end )
{
int ret = MBEDTLS_ERR_SSL_FEATURE_UNAVAILABLE;
- size_t len;
+ uint16_t len;
((void) ssl);
/*
@@ -2361,7 +2361,7 @@
len = (*p)[0] << 8 | (*p)[1];
*p += 2;
- if( end - (*p) < (int) len )
+ if( end - (*p) < len )
{
MBEDTLS_SSL_DEBUG_MSG( 1, ( "bad server key exchange message "
"(psk_identity_hint length)" ) );
diff --git a/library/ssl_msg.c b/library/ssl_msg.c
index 18fa555..428ace5 100644
--- a/library/ssl_msg.c
+++ b/library/ssl_msg.c
@@ -3197,16 +3197,17 @@
* that looks like a ClientHello.
*
* - if the input looks like a ClientHello without cookies,
- * send back HelloVerifyRequest, then
- * return MBEDTLS_ERR_SSL_HELLO_VERIFY_REQUIRED
+ * send back HelloVerifyRequest, then return 0
* - if the input looks like a ClientHello with a valid cookie,
* reset the session of the current context, and
* return MBEDTLS_ERR_SSL_CLIENT_RECONNECT
* - if anything goes wrong, return a specific error code
*
- * mbedtls_ssl_read_record() will ignore the record if anything else than
- * MBEDTLS_ERR_SSL_CLIENT_RECONNECT or 0 is returned, although this function
- * cannot not return 0.
+ * This function is called (through ssl_check_client_reconnect()) when an
+ * unexpected record is found in ssl_get_next_record(), which will discard the
+ * record if we return 0, and bubble up the return value otherwise (this
+ * includes the case of MBEDTLS_ERR_SSL_CLIENT_RECONNECT and of unexpected
+ * errors, and is the right thing to do in both cases).
*/
static int ssl_handle_possible_reconnect( mbedtls_ssl_context *ssl )
{
@@ -3218,6 +3219,8 @@
{
/* If we can't use cookies to verify reachability of the peer,
* drop the record. */
+ MBEDTLS_SSL_DEBUG_MSG( 1, ( "no cookie callbacks, "
+ "can't check reconnect validity" ) );
return( 0 );
}
@@ -3233,16 +3236,23 @@
if( ret == MBEDTLS_ERR_SSL_HELLO_VERIFY_REQUIRED )
{
+ int send_ret;
+ MBEDTLS_SSL_DEBUG_MSG( 1, ( "sending HelloVerifyRequest" ) );
+ MBEDTLS_SSL_DEBUG_BUF( 4, "output record sent to network",
+ ssl->out_buf, len );
/* Don't check write errors as we can't do anything here.
* If the error is permanent we'll catch it later,
* if it's not, then hopefully it'll work next time. */
- (void) ssl->f_send( ssl->p_bio, ssl->out_buf, len );
- ret = 0;
+ send_ret = ssl->f_send( ssl->p_bio, ssl->out_buf, len );
+ MBEDTLS_SSL_DEBUG_RET( 2, "ssl->f_send", send_ret );
+ (void) send_ret;
+
+ return( 0 );
}
if( ret == 0 )
{
- /* Got a valid cookie, partially reset context */
+ MBEDTLS_SSL_DEBUG_MSG( 1, ( "cookie is valid, resetting context" ) );
if( ( ret = mbedtls_ssl_session_reset_int( ssl, 1 ) ) != 0 )
{
MBEDTLS_SSL_DEBUG_RET( 1, "reset", ret );
@@ -4415,6 +4425,7 @@
ssl->in_msglen = rec.data_len;
ret = ssl_check_client_reconnect( ssl );
+ MBEDTLS_SSL_DEBUG_RET( 2, "ssl_check_client_reconnect", ret );
if( ret != 0 )
return( ret );
#endif
diff --git a/library/ssl_srv.c b/library/ssl_srv.c
index 469c67e..006bc69 100644
--- a/library/ssl_srv.c
+++ b/library/ssl_srv.c
@@ -3812,7 +3812,7 @@
const unsigned char *end )
{
int ret = 0;
- size_t n;
+ uint16_t n;
if( ssl_conf_has_psk_or_cb( ssl->conf ) == 0 )
{
@@ -3832,7 +3832,7 @@
n = ( (*p)[0] << 8 ) | (*p)[1];
*p += 2;
- if( n < 1 || n > 65535 || n > (size_t) ( end - *p ) )
+ if( n == 0 || n > end - *p )
{
MBEDTLS_SSL_DEBUG_MSG( 1, ( "bad client key exchange message" ) );
return( MBEDTLS_ERR_SSL_BAD_HS_CLIENT_KEY_EXCHANGE );
diff --git a/library/x509.c b/library/x509.c
index 7f8181b..c451332 100644
--- a/library/x509.c
+++ b/library/x509.c
@@ -1064,7 +1064,7 @@
mbedtls_x509_crt_free( &clicert );
#else
((void) verbose);
-#endif /* MBEDTLS_CERTS_C && MBEDTLS_SHA1_C */
+#endif /* MBEDTLS_CERTS_C && MBEDTLS_SHA256_C */
return( ret );
}
diff --git a/programs/pkey/ecdsa.c b/programs/pkey/ecdsa.c
index b851c31..9feb160 100644
--- a/programs/pkey/ecdsa.c
+++ b/programs/pkey/ecdsa.c
@@ -189,7 +189,7 @@
sig, &sig_len,
mbedtls_ctr_drbg_random, &ctr_drbg ) ) != 0 )
{
- mbedtls_printf( " failed\n ! mbedtls_ecdsa_genkey returned %d\n", ret );
+ mbedtls_printf( " failed\n ! mbedtls_ecdsa_write_signature returned %d\n", ret );
goto exit;
}
mbedtls_printf( " ok (signature length = %u)\n", (unsigned int) sig_len );
diff --git a/programs/test/udp_proxy.c b/programs/test/udp_proxy.c
index 979910e..7447571 100644
--- a/programs/test/udp_proxy.c
+++ b/programs/test/udp_proxy.c
@@ -133,6 +133,7 @@
" modifying CID in first instance of the packet.\n" \
" protect_hvr=0/1 default: 0 (don't protect HelloVerifyRequest)\n" \
" protect_len=%%d default: (don't protect packets of this size)\n" \
+ " inject_clihlo=0/1 default: 0 (don't inject fake ClientHello)\n" \
"\n" \
" seed=%%d default: (use current time)\n" \
USAGE_PACK \
@@ -166,6 +167,7 @@
unsigned bad_cid; /* inject corrupted CID record */
int protect_hvr; /* never drop or delay HelloVerifyRequest */
int protect_len; /* never drop/delay packet of the given size*/
+ int inject_clihlo; /* inject fake ClientHello after handshake */
unsigned pack; /* merge packets into single datagram for
* at most \c merge milliseconds if > 0 */
unsigned int seed; /* seed for "random" events */
@@ -314,6 +316,12 @@
if( opt.protect_len < 0 )
exit_usage( p, q );
}
+ else if( strcmp( p, "inject_clihlo" ) == 0 )
+ {
+ opt.inject_clihlo = atoi( q );
+ if( opt.inject_clihlo < 0 || opt.inject_clihlo > 1 )
+ exit_usage( p, q );
+ }
else if( strcmp( p, "seed" ) == 0 )
{
opt.seed = atoi( q );
@@ -523,11 +531,41 @@
fflush( stdout );
}
+/*
+ * In order to test the server's behaviour when receiving a ClientHello after
+ * the connection is established (this could be a hard reset from the client,
+ * but the server must not drop the existing connection before establishing
+ * client reachability, see RFC 6347 Section 4.2.8), we memorize the first
+ * ClientHello we see (which can't have a cookie), then replay it after the
+ * first ApplicationData record - then we're done.
+ *
+ * This is controlled by the inject_clihlo option.
+ *
+ * We want an explicit state and a place to store the packet.
+ */
+typedef enum {
+ ICH_INIT, /* haven't seen the first ClientHello yet */
+ ICH_CACHED, /* cached the initial ClientHello */
+ ICH_INJECTED, /* ClientHello already injected, done */
+} inject_clihlo_state_t;
+
+static inject_clihlo_state_t inject_clihlo_state;
+static packet initial_clihlo;
+
int send_packet( const packet *p, const char *why )
{
int ret;
mbedtls_net_context *dst = p->dst;
+ /* save initial ClientHello? */
+ if( opt.inject_clihlo != 0 &&
+ inject_clihlo_state == ICH_INIT &&
+ strcmp( p->type, "ClientHello" ) == 0 )
+ {
+ memcpy( &initial_clihlo, p, sizeof( packet ) );
+ inject_clihlo_state = ICH_CACHED;
+ }
+
/* insert corrupted CID record? */
if( opt.bad_cid != 0 &&
strcmp( p->type, "CID" ) == 0 &&
@@ -592,6 +630,23 @@
}
}
+ /* Inject ClientHello after first ApplicationData */
+ if( opt.inject_clihlo != 0 &&
+ inject_clihlo_state == ICH_CACHED &&
+ strcmp( p->type, "ApplicationData" ) == 0 )
+ {
+ print_packet( &initial_clihlo, "injected" );
+
+ if( ( ret = dispatch_data( dst, initial_clihlo.buf,
+ initial_clihlo.len ) ) <= 0 )
+ {
+ mbedtls_printf( " ! dispatch returned %d\n", ret );
+ return( ret );
+ }
+
+ inject_clihlo_state = ICH_INJECTED;
+ }
+
return( 0 );
}
diff --git a/scripts/assemble_changelog.py b/scripts/assemble_changelog.py
new file mode 100755
index 0000000..ffa3f16
--- /dev/null
+++ b/scripts/assemble_changelog.py
@@ -0,0 +1,505 @@
+#!/usr/bin/env python3
+
+"""Assemble Mbed TLS change log entries into the change log file.
+
+Add changelog entries to the first level-2 section.
+Create a new level-2 section for unreleased changes if needed.
+Remove the input files unless --keep-entries is specified.
+
+In each level-3 section, entries are sorted in chronological order
+(oldest first). From oldest to newest:
+* Merged entry files are sorted according to their merge date (date of
+ the merge commit that brought the commit that created the file into
+ the target branch).
+* Committed but unmerged entry files are sorted according to the date
+ of the commit that adds them.
+* Uncommitted entry files are sorted according to their modification time.
+
+You must run this program from within a git working directory.
+"""
+
+# Copyright (C) 2019, Arm Limited, All Rights Reserved
+# SPDX-License-Identifier: Apache-2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# This file is part of Mbed TLS (https://tls.mbed.org)
+
+import argparse
+from collections import OrderedDict, namedtuple
+import datetime
+import functools
+import glob
+import os
+import re
+import subprocess
+import sys
+
+class InputFormatError(Exception):
+ def __init__(self, filename, line_number, message, *args, **kwargs):
+ message = '{}:{}: {}'.format(filename, line_number,
+ message.format(*args, **kwargs))
+ super().__init__(message)
+
+class CategoryParseError(Exception):
+ def __init__(self, line_offset, error_message):
+ self.line_offset = line_offset
+ self.error_message = error_message
+ super().__init__('{}: {}'.format(line_offset, error_message))
+
+class LostContent(Exception):
+ def __init__(self, filename, line):
+ message = ('Lost content from {}: "{}"'.format(filename, line))
+ super().__init__(message)
+
+# The category names we use in the changelog.
+# If you edit this, update ChangeLog.d/README.md.
+STANDARD_CATEGORIES = (
+ b'API changes',
+ b'Default behavior changes',
+ b'Requirement changes',
+ b'New deprecations',
+ b'Removals',
+ b'Features',
+ b'Security',
+ b'Bugfix',
+ b'Changes',
+)
+
+CategoryContent = namedtuple('CategoryContent', [
+ 'name', 'title_line', # Title text and line number of the title
+ 'body', 'body_line', # Body text and starting line number of the body
+])
+
+class ChangelogFormat:
+ """Virtual class documenting how to write a changelog format class."""
+
+ @classmethod
+ def extract_top_version(cls, changelog_file_content):
+ """Split out the top version section.
+
+ If the top version is already released, create a new top
+ version section for an unreleased version.
+
+ Return ``(header, top_version_title, top_version_body, trailer)``
+ where the "top version" is the existing top version section if it's
+ for unreleased changes, and a newly created section otherwise.
+ To assemble the changelog after modifying top_version_body,
+ concatenate the four pieces.
+ """
+ raise NotImplementedError
+
+ @classmethod
+ def version_title_text(cls, version_title):
+ """Return the text of a formatted version section title."""
+ raise NotImplementedError
+
+ @classmethod
+ def split_categories(cls, version_body):
+ """Split a changelog version section body into categories.
+
+ Return a list of `CategoryContent` the name is category title
+ without any formatting.
+ """
+ raise NotImplementedError
+
+ @classmethod
+ def format_category(cls, title, body):
+ """Construct the text of a category section from its title and body."""
+ raise NotImplementedError
+
+class TextChangelogFormat(ChangelogFormat):
+ """The traditional Mbed TLS changelog format."""
+
+ _unreleased_version_text = b'= mbed TLS x.x.x branch released xxxx-xx-xx'
+ @classmethod
+ def is_released_version(cls, title):
+ # Look for an incomplete release date
+ return not re.search(br'[0-9x]{4}-[0-9x]{2}-[0-9x]?x', title)
+
+ _top_version_re = re.compile(br'(?:\A|\n)(=[^\n]*\n+)(.*?\n)(?:=|$)',
+ re.DOTALL)
+ @classmethod
+ def extract_top_version(cls, changelog_file_content):
+ """A version section starts with a line starting with '='."""
+ m = re.search(cls._top_version_re, changelog_file_content)
+ top_version_start = m.start(1)
+ top_version_end = m.end(2)
+ top_version_title = m.group(1)
+ top_version_body = m.group(2)
+ if cls.is_released_version(top_version_title):
+ top_version_end = top_version_start
+ top_version_title = cls._unreleased_version_text + b'\n\n'
+ top_version_body = b''
+ return (changelog_file_content[:top_version_start],
+ top_version_title, top_version_body,
+ changelog_file_content[top_version_end:])
+
+ @classmethod
+ def version_title_text(cls, version_title):
+ return re.sub(br'\n.*', version_title, re.DOTALL)
+
+ _category_title_re = re.compile(br'(^\w.*)\n+', re.MULTILINE)
+ @classmethod
+ def split_categories(cls, version_body):
+ """A category title is a line with the title in column 0."""
+ if not version_body:
+ return []
+ title_matches = list(re.finditer(cls._category_title_re, version_body))
+ if not title_matches or title_matches[0].start() != 0:
+ # There is junk before the first category.
+ raise CategoryParseError(0, 'Junk found where category expected')
+ title_starts = [m.start(1) for m in title_matches]
+ body_starts = [m.end(0) for m in title_matches]
+ body_ends = title_starts[1:] + [len(version_body)]
+ bodies = [version_body[body_start:body_end].rstrip(b'\n') + b'\n'
+ for (body_start, body_end) in zip(body_starts, body_ends)]
+ title_lines = [version_body[:pos].count(b'\n') for pos in title_starts]
+ body_lines = [version_body[:pos].count(b'\n') for pos in body_starts]
+ return [CategoryContent(title_match.group(1), title_line,
+ body, body_line)
+ for title_match, title_line, body, body_line
+ in zip(title_matches, title_lines, bodies, body_lines)]
+
+ @classmethod
+ def format_category(cls, title, body):
+ # `split_categories` ensures that each body ends with a newline.
+ # Make sure that there is additionally a blank line between categories.
+ if not body.endswith(b'\n\n'):
+ body += b'\n'
+ return title + b'\n' + body
+
+class ChangeLog:
+ """An Mbed TLS changelog.
+
+ A changelog file consists of some header text followed by one or
+ more version sections. The version sections are in reverse
+ chronological order. Each version section consists of a title and a body.
+
+ The body of a version section consists of zero or more category
+ subsections. Each category subsection consists of a title and a body.
+
+ A changelog entry file has the same format as the body of a version section.
+
+ A `ChangelogFormat` object defines the concrete syntax of the changelog.
+ Entry files must have the same format as the changelog file.
+ """
+
+ # Only accept dotted version numbers (e.g. "3.1", not "3").
+ # Refuse ".x" in a version number where x is a letter: this indicates
+ # a version that is not yet released. Something like "3.1a" is accepted.
+ _version_number_re = re.compile(br'[0-9]+\.[0-9A-Za-z.]+')
+ _incomplete_version_number_re = re.compile(br'.*\.[A-Za-z]')
+
+ def add_categories_from_text(self, filename, line_offset,
+ text, allow_unknown_category):
+ """Parse a version section or entry file."""
+ try:
+ categories = self.format.split_categories(text)
+ except CategoryParseError as e:
+ raise InputFormatError(filename, line_offset + e.line_offset,
+ e.error_message)
+ for category in categories:
+ if not allow_unknown_category and \
+ category.name not in self.categories:
+ raise InputFormatError(filename,
+ line_offset + category.title_line,
+ 'Unknown category: "{}"',
+ category.name.decode('utf8'))
+ self.categories[category.name] += category.body
+
+ def __init__(self, input_stream, changelog_format):
+ """Create a changelog object.
+
+ Populate the changelog object from the content of the file
+ input_stream.
+ """
+ self.format = changelog_format
+ whole_file = input_stream.read()
+ (self.header,
+ self.top_version_title, top_version_body,
+ self.trailer) = self.format.extract_top_version(whole_file)
+ # Split the top version section into categories.
+ self.categories = OrderedDict()
+ for category in STANDARD_CATEGORIES:
+ self.categories[category] = b''
+ offset = (self.header + self.top_version_title).count(b'\n') + 1
+ self.add_categories_from_text(input_stream.name, offset,
+ top_version_body, True)
+
+ def add_file(self, input_stream):
+ """Add changelog entries from a file.
+ """
+ self.add_categories_from_text(input_stream.name, 1,
+ input_stream.read(), False)
+
+ def write(self, filename):
+ """Write the changelog to the specified file.
+ """
+ with open(filename, 'wb') as out:
+ out.write(self.header)
+ out.write(self.top_version_title)
+ for title, body in self.categories.items():
+ if not body:
+ continue
+ out.write(self.format.format_category(title, body))
+ out.write(self.trailer)
+
+
+@functools.total_ordering
+class EntryFileSortKey:
+ """This classes defines an ordering on changelog entry files: older < newer.
+
+ * Merged entry files are sorted according to their merge date (date of
+ the merge commit that brought the commit that created the file into
+ the target branch).
+ * Committed but unmerged entry files are sorted according to the date
+ of the commit that adds them.
+ * Uncommitted entry files are sorted according to their modification time.
+
+ This class assumes that the file is in a git working directory with
+ the target branch checked out.
+ """
+
+ # Categories of files. A lower number is considered older.
+ MERGED = 0
+ COMMITTED = 1
+ LOCAL = 2
+
+ @staticmethod
+ def creation_hash(filename):
+ """Return the git commit id at which the given file was created.
+
+ Return None if the file was never checked into git.
+ """
+ hashes = subprocess.check_output(['git', 'log', '--format=%H',
+ '--follow',
+ '--', filename])
+ m = re.search(b'(.+)$', hashes)
+ if not m:
+ # The git output is empty. This means that the file was
+ # never checked in.
+ return None
+ # The last commit in the log is the oldest one, which is when the
+ # file was created.
+ return m.group(0)
+
+ @staticmethod
+ def list_merges(some_hash, target, *options):
+ """List merge commits from some_hash to target.
+
+ Pass options to git to select which commits are included.
+ """
+ text = subprocess.check_output(['git', 'rev-list',
+ '--merges', *options,
+ b'..'.join([some_hash, target])])
+ return text.rstrip(b'\n').split(b'\n')
+
+ @classmethod
+ def merge_hash(cls, some_hash):
+ """Return the git commit id at which the given commit was merged.
+
+ Return None if the given commit was never merged.
+ """
+ target = b'HEAD'
+ # List the merges from some_hash to the target in two ways.
+ # The ancestry list is the ones that are both descendants of
+ # some_hash and ancestors of the target.
+ ancestry = frozenset(cls.list_merges(some_hash, target,
+ '--ancestry-path'))
+ # The first_parents list only contains merges that are directly
+ # on the target branch. We want it in reverse order (oldest first).
+ first_parents = cls.list_merges(some_hash, target,
+ '--first-parent', '--reverse')
+ # Look for the oldest merge commit that's both on the direct path
+ # and directly on the target branch. That's the place where some_hash
+ # was merged on the target branch. See
+ # https://stackoverflow.com/questions/8475448/find-merge-commit-which-include-a-specific-commit
+ for commit in first_parents:
+ if commit in ancestry:
+ return commit
+ return None
+
+ @staticmethod
+ def commit_timestamp(commit_id):
+ """Return the timestamp of the given commit."""
+ text = subprocess.check_output(['git', 'show', '-s',
+ '--format=%ct',
+ commit_id])
+ return datetime.datetime.utcfromtimestamp(int(text))
+
+ @staticmethod
+ def file_timestamp(filename):
+ """Return the modification timestamp of the given file."""
+ mtime = os.stat(filename).st_mtime
+ return datetime.datetime.fromtimestamp(mtime)
+
+ def __init__(self, filename):
+ """Determine position of the file in the changelog entry order.
+
+ This constructor returns an object that can be used with comparison
+ operators, with `sort` and `sorted`, etc. Older entries are sorted
+ before newer entries.
+ """
+ self.filename = filename
+ creation_hash = self.creation_hash(filename)
+ if not creation_hash:
+ self.category = self.LOCAL
+ self.datetime = self.file_timestamp(filename)
+ return
+ merge_hash = self.merge_hash(creation_hash)
+ if not merge_hash:
+ self.category = self.COMMITTED
+ self.datetime = self.commit_timestamp(creation_hash)
+ return
+ self.category = self.MERGED
+ self.datetime = self.commit_timestamp(merge_hash)
+
+ def sort_key(self):
+ """"Return a concrete sort key for this entry file sort key object.
+
+ ``ts1 < ts2`` is implemented as ``ts1.sort_key() < ts2.sort_key()``.
+ """
+ return (self.category, self.datetime, self.filename)
+
+ def __eq__(self, other):
+ return self.sort_key() == other.sort_key()
+
+ def __lt__(self, other):
+ return self.sort_key() < other.sort_key()
+
+
+def check_output(generated_output_file, main_input_file, merged_files):
+ """Make sanity checks on the generated output.
+
+ The intent of these sanity checks is to have reasonable confidence
+ that no content has been lost.
+
+ The sanity check is that every line that is present in an input file
+ is also present in an output file. This is not perfect but good enough
+ for now.
+ """
+ generated_output = set(open(generated_output_file, 'rb'))
+ for line in open(main_input_file, 'rb'):
+ if line not in generated_output:
+ raise LostContent('original file', line)
+ for merged_file in merged_files:
+ for line in open(merged_file, 'rb'):
+ if line not in generated_output:
+ raise LostContent(merged_file, line)
+
+def finish_output(changelog, output_file, input_file, merged_files):
+ """Write the changelog to the output file.
+
+ The input file and the list of merged files are used only for sanity
+ checks on the output.
+ """
+ if os.path.exists(output_file) and not os.path.isfile(output_file):
+ # The output is a non-regular file (e.g. pipe). Write to it directly.
+ output_temp = output_file
+ else:
+ # The output is a regular file. Write to a temporary file,
+ # then move it into place atomically.
+ output_temp = output_file + '.tmp'
+ changelog.write(output_temp)
+ check_output(output_temp, input_file, merged_files)
+ if output_temp != output_file:
+ os.rename(output_temp, output_file)
+
+def remove_merged_entries(files_to_remove):
+ for filename in files_to_remove:
+ os.remove(filename)
+
+def list_files_to_merge(options):
+ """List the entry files to merge, oldest first.
+
+ "Oldest" is defined by `EntryFileSortKey`.
+ """
+ files_to_merge = glob.glob(os.path.join(options.dir, '*.txt'))
+ files_to_merge.sort(key=EntryFileSortKey)
+ return files_to_merge
+
+def merge_entries(options):
+ """Merge changelog entries into the changelog file.
+
+ Read the changelog file from options.input.
+ Read entries to merge from the directory options.dir.
+ Write the new changelog to options.output.
+ Remove the merged entries if options.keep_entries is false.
+ """
+ with open(options.input, 'rb') as input_file:
+ changelog = ChangeLog(input_file, TextChangelogFormat)
+ files_to_merge = list_files_to_merge(options)
+ if not files_to_merge:
+ sys.stderr.write('There are no pending changelog entries.\n')
+ return
+ for filename in files_to_merge:
+ with open(filename, 'rb') as input_file:
+ changelog.add_file(input_file)
+ finish_output(changelog, options.output, options.input, files_to_merge)
+ if not options.keep_entries:
+ remove_merged_entries(files_to_merge)
+
+def show_file_timestamps(options):
+ """List the files to merge and their timestamp.
+
+ This is only intended for debugging purposes.
+ """
+ files = list_files_to_merge(options)
+ for filename in files:
+ ts = EntryFileSortKey(filename)
+ print(ts.category, ts.datetime, filename)
+
+def set_defaults(options):
+ """Add default values for missing options."""
+ output_file = getattr(options, 'output', None)
+ if output_file is None:
+ options.output = options.input
+ if getattr(options, 'keep_entries', None) is None:
+ options.keep_entries = (output_file is not None)
+
+def main():
+ """Command line entry point."""
+ parser = argparse.ArgumentParser(description=__doc__)
+ parser.add_argument('--dir', '-d', metavar='DIR',
+ default='ChangeLog.d',
+ help='Directory to read entries from'
+ ' (default: ChangeLog.d)')
+ parser.add_argument('--input', '-i', metavar='FILE',
+ default='ChangeLog',
+ help='Existing changelog file to read from and augment'
+ ' (default: ChangeLog)')
+ parser.add_argument('--keep-entries',
+ action='store_true', dest='keep_entries', default=None,
+ help='Keep the files containing entries'
+ ' (default: remove them if --output/-o is not specified)')
+ parser.add_argument('--no-keep-entries',
+ action='store_false', dest='keep_entries',
+ help='Remove the files containing entries after they are merged'
+ ' (default: remove them if --output/-o is not specified)')
+ parser.add_argument('--output', '-o', metavar='FILE',
+ help='Output changelog file'
+ ' (default: overwrite the input)')
+ parser.add_argument('--list-files-only',
+ action='store_true',
+ help=('Only list the files that would be processed '
+ '(with some debugging information)'))
+ options = parser.parse_args()
+ set_defaults(options)
+ if options.list_files_only:
+ show_file_timestamps(options)
+ return
+ merge_entries(options)
+
+if __name__ == '__main__':
+ main()
diff --git a/tests/ssl-opt.sh b/tests/ssl-opt.sh
index 35f742f..9b6eee1 100755
--- a/tests/ssl-opt.sh
+++ b/tests/ssl-opt.sh
@@ -7279,8 +7279,8 @@
not_with_valgrind # spurious resend
run_test "DTLS client reconnect from same port: reference" \
- "$P_SRV dtls=1 exchanges=2 read_timeout=1000" \
- "$P_CLI dtls=1 exchanges=2 debug_level=2 hs_timeout=500-1000" \
+ "$P_SRV dtls=1 exchanges=2 read_timeout=20000 hs_timeout=10000-20000" \
+ "$P_CLI dtls=1 exchanges=2 debug_level=2 hs_timeout=10000-20000" \
0 \
-C "resend" \
-S "The operation timed out" \
@@ -7288,8 +7288,8 @@
not_with_valgrind # spurious resend
run_test "DTLS client reconnect from same port: reconnect" \
- "$P_SRV dtls=1 exchanges=2 read_timeout=1000" \
- "$P_CLI dtls=1 exchanges=2 debug_level=2 hs_timeout=500-1000 reconnect_hard=1" \
+ "$P_SRV dtls=1 exchanges=2 read_timeout=20000 hs_timeout=10000-20000" \
+ "$P_CLI dtls=1 exchanges=2 debug_level=2 hs_timeout=10000-20000 reconnect_hard=1" \
0 \
-C "resend" \
-S "The operation timed out" \
@@ -7318,6 +7318,14 @@
-s "The operation timed out" \
-S "Client initiated reconnection from same port"
+run_test "DTLS client reconnect from same port: attacker-injected" \
+ -p "$P_PXY inject_clihlo=1" \
+ "$P_SRV dtls=1 exchanges=2 debug_level=1" \
+ "$P_CLI dtls=1 exchanges=2" \
+ 0 \
+ -s "possible client reconnect from the same port" \
+ -S "Client initiated reconnection from same port"
+
# Tests for various cases of client authentication with DTLS
# (focused on handshake flows and message parsing)
@@ -8387,8 +8395,8 @@
not_with_valgrind # spurious resend due to timeout
run_test "DTLS proxy: reference" \
-p "$P_PXY" \
- "$P_SRV dtls=1 debug_level=2" \
- "$P_CLI dtls=1 debug_level=2" \
+ "$P_SRV dtls=1 debug_level=2 hs_timeout=10000-20000" \
+ "$P_CLI dtls=1 debug_level=2 hs_timeout=10000-20000" \
0 \
-C "replayed record" \
-S "replayed record" \
@@ -8405,8 +8413,8 @@
not_with_valgrind # spurious resend due to timeout
run_test "DTLS proxy: duplicate every packet" \
-p "$P_PXY duplicate=1" \
- "$P_SRV dtls=1 dgram_packing=0 debug_level=2" \
- "$P_CLI dtls=1 dgram_packing=0 debug_level=2" \
+ "$P_SRV dtls=1 dgram_packing=0 debug_level=2 hs_timeout=10000-20000" \
+ "$P_CLI dtls=1 dgram_packing=0 debug_level=2 hs_timeout=10000-20000" \
0 \
-c "replayed record" \
-s "replayed record" \
diff --git a/tests/suites/host_test.function b/tests/suites/host_test.function
index 6467340..1069c24 100644
--- a/tests/suites/host_test.function
+++ b/tests/suites/host_test.function
@@ -425,7 +425,7 @@
*/
static void write_outcome_result( FILE *outcome_file,
size_t unmet_dep_count,
- char *unmet_dependencies[],
+ int unmet_dependencies[],
int ret,
const test_info_t *info )
{
@@ -443,7 +443,7 @@
mbedtls_fprintf( outcome_file, "SKIP" );
for( i = 0; i < unmet_dep_count; i++ )
{
- mbedtls_fprintf( outcome_file, "%c%s",
+ mbedtls_fprintf( outcome_file, "%c%d",
i == 0 ? ';' : ':',
unmet_dependencies[i] );
}
@@ -598,7 +598,7 @@
testfile_index++ )
{
size_t unmet_dep_count = 0;
- char *unmet_dependencies[20];
+ int unmet_dependencies[20];
test_filename = test_files[ testfile_index ];
@@ -647,19 +647,7 @@
int dep_id = strtol( params[i], NULL, 10 );
if( dep_check( dep_id ) != DEPENDENCY_SUPPORTED )
{
- if( 0 == option_verbose )
- {
- /* Only one count is needed if not verbose */
- unmet_dep_count++;
- break;
- }
-
- unmet_dependencies[ unmet_dep_count ] = strdup( params[i] );
- if( unmet_dependencies[ unmet_dep_count ] == NULL )
- {
- mbedtls_fprintf( stderr, "FATAL: Out of memory\n" );
- mbedtls_exit( MBEDTLS_EXIT_FAILURE );
- }
+ unmet_dependencies[unmet_dep_count] = dep_id;
unmet_dep_count++;
}
}
@@ -730,9 +718,8 @@
mbedtls_fprintf( stdout, "\n Unmet dependencies: " );
for( i = 0; i < unmet_dep_count; i++ )
{
- mbedtls_fprintf( stdout, "%s ",
+ mbedtls_fprintf( stdout, "%d ",
unmet_dependencies[i] );
- free( unmet_dependencies[i] );
}
}
mbedtls_fprintf( stdout, "\n" );
@@ -783,10 +770,6 @@
total_errors++;
}
fclose( file );
-
- /* In case we encounter early end of file */
- for( i = 0; i < unmet_dep_count; i++ )
- free( unmet_dependencies[i] );
}
if( outcome_file != NULL )
diff --git a/tests/suites/test_suite_cipher.aes.data b/tests/suites/test_suite_cipher.aes.data
index 6293408..4dbdd52 100644
--- a/tests/suites/test_suite_cipher.aes.data
+++ b/tests/suites/test_suite_cipher.aes.data
@@ -1062,6 +1062,10 @@
depends_on:MBEDTLS_AES_C:MBEDTLS_CIPHER_MODE_CBC:MBEDTLS_CIPHER_PADDING_PKCS7
enc_dec_buf_multipart:MBEDTLS_CIPHER_AES_192_CBC:192:16:16:-1:16:16:0:32
+AES-256 CBC - Decrypt empty buffer
+depends_on:MBEDTLS_AES_C:MBEDTLS_CIPHER_MODE_CBC:MBEDTLS_CIPHER_PADDING_PKCS7
+dec_empty_buf:MBEDTLS_CIPHER_AES_256_CBC:0:0
+
AES-256 CBC - Encrypt and decrypt 0 bytes with PKCS7 padding
depends_on:MBEDTLS_AES_C:MBEDTLS_CIPHER_MODE_CBC:MBEDTLS_CIPHER_PADDING_PKCS7
enc_dec_buf:MBEDTLS_CIPHER_AES_256_CBC:"AES-256-CBC":256:0:-1
diff --git a/tests/suites/test_suite_cipher.camellia.data b/tests/suites/test_suite_cipher.camellia.data
index 667f424..3e7bffa 100644
--- a/tests/suites/test_suite_cipher.camellia.data
+++ b/tests/suites/test_suite_cipher.camellia.data
@@ -1,7 +1,11 @@
-CAMELLIA CBC Decrypt empty buffer
+CAMELLIA-128 CBC - Decrypt empty buffer
depends_on:MBEDTLS_CAMELLIA_C:MBEDTLS_CIPHER_MODE_CBC:MBEDTLS_CIPHER_PADDING_PKCS7
dec_empty_buf:MBEDTLS_CIPHER_CAMELLIA_128_CBC:0:0
+CAMELLIA-256 CBC - Decrypt empty buffer
+depends_on:MBEDTLS_CAMELLIA_C:MBEDTLS_CIPHER_MODE_CBC:MBEDTLS_CIPHER_PADDING_PKCS7
+dec_empty_buf:MBEDTLS_CIPHER_CAMELLIA_256_CBC:0:0
+
CAMELLIA Encrypt and decrypt 0 bytes [#1]
depends_on:MBEDTLS_CAMELLIA_C:MBEDTLS_CIPHER_MODE_CBC:MBEDTLS_CIPHER_PADDING_PKCS7
enc_dec_buf:MBEDTLS_CIPHER_CAMELLIA_128_CBC:"CAMELLIA-128-CBC":128:0:-1
diff --git a/tests/suites/test_suite_cipher.gcm.data b/tests/suites/test_suite_cipher.gcm.data
index c526b2e..8d728bd 100644
--- a/tests/suites/test_suite_cipher.gcm.data
+++ b/tests/suites/test_suite_cipher.gcm.data
@@ -2,6 +2,10 @@
depends_on:MBEDTLS_CAMELLIA_C:MBEDTLS_GCM_C
dec_empty_buf:MBEDTLS_CIPHER_CAMELLIA_128_GCM:0:0
+AES GCM Decrypt empty buffer
+depends_on:MBEDTLS_CIPHER_AES_128_GCM:MBEDTLS_GCM_C
+dec_empty_buf:MBEDTLS_CIPHER_AES_128_GCM:0:0
+
Aria GCM Decrypt empty buffer
depends_on:MBEDTLS_ARIA_C:MBEDTLS_GCM_C
dec_empty_buf:MBEDTLS_CIPHER_ARIA_128_GCM:0:0
diff --git a/tests/suites/test_suite_psa_crypto.function b/tests/suites/test_suite_psa_crypto.function
index b6e6e5a..bc95f6f 100644
--- a/tests/suites/test_suite_psa_crypto.function
+++ b/tests/suites/test_suite_psa_crypto.function
@@ -5297,6 +5297,8 @@
size_t i;
unsigned run;
+ TEST_ASSERT( bytes_arg >= 0 );
+
ASSERT_ALLOC( output, bytes + sizeof( trail ) );
ASSERT_ALLOC( changed, bytes );
memcpy( output + bytes, trail, sizeof( trail ) );