tests: Limit each log to 10 GiB

Limit log output in compat.sh and ssl-opt.sh, in case of failures with
these scripts where they may output seemingly unlimited length error
logs.

Note that ulimit -f uses units of 512 bytes, so we use 10 * 1024 * 1024
* 2 to get 10 GiB.
diff --git a/tests/compat.sh b/tests/compat.sh
index 0eae1ea..80c2d31 100755
--- a/tests/compat.sh
+++ b/tests/compat.sh
@@ -15,6 +15,10 @@
 
 set -u
 
+# Limit the size of each log to 10 GiB, in case of failures with this script
+# where it may output seemingly unlimited length error logs.
+ulimit -f 20971520
+
 # initialise counters
 TESTS=0
 FAILED=0