Improve documentation for encode size calculation mode (#123)
This adds the constant SizeCalculateUsefulBuf which can be passed to QCBOR Encode and to UsefulOutBuf to indicate only size calculation should be performed, that no encoded CBOR is to be output.
The documentation for this mode is improved, an example is added and test cases are added.
Co-authored-by: Laurence Lundblade <lgl@securitytheory.com>
diff --git a/inc/qcbor/UsefulBuf.h b/inc/qcbor/UsefulBuf.h
index fb3c12b..1662fcc 100644
--- a/inc/qcbor/UsefulBuf.h
+++ b/inc/qcbor/UsefulBuf.h
@@ -42,6 +42,7 @@
when who what, where, why
-------- ---- --------------------------------------------------
+ 9/21/2021 llundbla Clarify UsefulOutBuf size calculation mode
8/8/2021 dthaler/llundbla Work with C++ without compiler extensions
5/11/2021 llundblade Improve comments and comment formatting.
3/6/2021 mcr/llundblade Fix warnings related to --Wcast-qual
@@ -793,17 +794,13 @@
* - Call UsefulOutBuf_OutUBuf() or UsefulOutBuf_CopyOut() to see
* there were no errors and to get the serialized output bytes.
*
- * @ref UsefulOutBuf can be used in a size calculation mode to
- * calculate the size of output that would be generated. This is
+ * @ref UsefulOutBuf can be used in a mode to calculate the size of
+ * what would be output without actually outputting anything. This is
* useful to calculate the size of a buffer that is to be allocated to
- * hold the output. To use @ref UsefulOutBuf in this mode, call
- * UsefulOutBuf_Init() with the @c Storage @ref UsefulBuf as
- * @c (UsefulBuf){NULL, MAX_UINT32}. Then call all the Insert and Add
- * functions. No attempt will be made to actually copy data, so only
- * the lengths have to be valid for inputs to these calls.
+ * hold the output. See @ref SizeCalculateUsefulBuf.
*
* Methods like UsefulOutBuf_InsertUint64() always output in network
- * bytes order (big endian).
+ * byte order (big endian).
*
* The possible errors are:
*
@@ -840,6 +837,23 @@
/**
+ * This is a @ref UsefulBuf value that can be passed to
+ * UsefulOutBuf_Init() to have it calculate the size of the output
+ * buffer needed. Pass this for @c Storage, call all the append and
+ * insert functions normally, then call UsefulOutBuf_OutUBuf(). The
+ * returned @ref UsefulBufC has the size.
+ *
+ * As one can see, this is just a NULL pointer and very large size.
+ * The NULL pointer tells UsefulOutputBuf to not copy any data.
+ */
+#ifdef __cplusplus
+#define SizeCalculateUsefulBuf {NULL, SIZE_MAX}
+#else
+#define SizeCalculateUsefulBuf ((UsefulBuf) {NULL, SIZE_MAX})
+#endif
+
+
+/**
* @brief Initialize and supply the actual output buffer.
*
* @param[out] pUOutBuf The @ref UsefulOutBuf to initialize.
@@ -849,6 +863,10 @@
* current position to the beginning of the buffer and clears the
* error state.
*
+ * See @ref SizeCalculateUsefulBuf for instructions on how to
+ * initialize a @ref UsefulOutBuf to calculate the size that would be
+ * output without actually outputting.
+ *
* This must be called before the @ref UsefulOutBuf is used.
*/
void UsefulOutBuf_Init(UsefulOutBuf *pUOutBuf, UsefulBuf Storage);
diff --git a/inc/qcbor/qcbor_common.h b/inc/qcbor/qcbor_common.h
index 51191e2..8b161de 100644
--- a/inc/qcbor/qcbor_common.h
+++ b/inc/qcbor/qcbor_common.h
@@ -288,8 +288,9 @@
and 31. */
QCBOR_ERR_ENCODE_UNSUPPORTED = 2,
- /** During encoding, the length of the encoded CBOR exceeded @c
- UINT32_MAX. */
+ /** During encoding, the length of the encoded CBOR exceeded
+ QCBOR_MAX_ARRAY_OFFSET, which is slightly less than
+ @c UINT32_MAX. */
QCBOR_ERR_BUFFER_TOO_LARGE = 3,
/** During encoding, the array or map nesting was deeper than this
diff --git a/inc/qcbor/qcbor_encode.h b/inc/qcbor/qcbor_encode.h
index 202a98f..0abc594 100644
--- a/inc/qcbor/qcbor_encode.h
+++ b/inc/qcbor/qcbor_encode.h
@@ -167,15 +167,14 @@
## Encoding
A common encoding usage mode is to invoke the encoding twice. First
- with no output buffer to compute the length of the needed output
- buffer. Then the correct sized output buffer is allocated. Last the
- encoder is invoked again, this time with the output buffer.
+ with the output buffer as @ref SizeCalculateUsefulBuf to compute the
+ length of the needed output buffer. The correct sized output buffer
+ is allocated. The encoder is invoked a second time with the allocated
+ output buffer.
The double invocation is not required if the maximum output buffer
size can be predicted. This is usually possible for simple CBOR
- structures. If the double invocation is implemented, it can be in a
- loop or function as in the example code so that the code doesn't have
- to actually be written twice, saving code size.
+ structures.
If a buffer too small to hold the encoded output is given, the error
@ref QCBOR_ERR_BUFFER_TOO_SMALL will be returned. Data will never be
@@ -444,22 +443,22 @@
that will ever be needed and hard code a buffer of that size.
Another way to do it is to have QCBOR calculate it for you. To do
- this set @c Storage.ptr to @c NULL and @c Storage.len to @c
- UINT32_MAX. Then call all the functions to add the CBOR exactly as if
- encoding for real. Then call QCBOREncode_Finish(). The pointer
- returned will be @c NULL, but the length returned is that of what would
- be encoded. Once the length is obtained, allocate a buffer of that
+ this, pass @ref SizeCalculateUsefulBuf for @c Storage.
+ Then call all the functions to add the CBOR exactly as if
+ encoding for real. Finally, call QCBOREncode_FinishGetSize().
+ Once the length is obtained, allocate a buffer of that
size, call QCBOREncode_Init() again with the real buffer. Call all
the add functions again and finally, QCBOREncode_Finish() to obtain
- the final result. This uses almost twice the CPU time, but that is
+ the final result. This uses twice the CPU time, but that is
usually not an issue.
See QCBOREncode_Finish() for how the pointer and length for the
encoded CBOR is returned.
- The maximum output buffer size allowed is @c UINT32_MAX (4GB). The
- error @ref QCBOR_ERR_BUFFER_TOO_LARGE will be returned by
- QCBOREncode_Finish() if a larger buffer length is passed in.
+ For practical purposes QCBOR can't output encoded CBOR larger than
+ @c UINT32_MAX (4GB) even on 64-bit CPUs because the internal offsets
+ used to track the start of an array/map are 32 bits to reduce the
+ size of the encoding context.
A @ref QCBOREncodeContext can be reused over and over as long as
QCBOREncode_Init() is called before each use.