Serialization tools¶

Serializing data on the fly is very common in embedded systems. For example, if your microcontroller needs to communicate with an external device through an UART line, chances are this device requires a specific protocol to be implemented. Unpadded provides a special kind of tuple which will help you encode and decode packets.

Formatting packets with the `upd::tuple` class¶

The upd::tuple works a bit like std::tuple. It even specializes the std::tuple_element and std::tuple_size structure template in order to be compatible with structured bindings. However, the internal layout is widly different. upd::tuple use an internal byte buffer to store its elements in an unaligned way. Therefore, it is not possible to get references to the elements of a upd::tuple instance, only copies.

However, upd::tuple is iterable like a container. When you iterate through it, it actually iterate the underlying the internal byte buffer. Therefore, do not use upd::tuple as a mean of storing data, but like a flexible packet encoder and decoder.

Example¶

Let’s assume we need to communicate with an external device that use big endian and ones’ complement and follows this protocol:

Each packet start with a 4-byte long header
Packet to the device contains 2 32-bit unsigned integer right after the header
Packet from the device contains a 16-bit signed integer and a 16-bit unsigned integer right after the header
Finally, each packet ends with a 16-bit CRC

Assuming write_byte_to_device and read_byte_from_device are defined and work as their names suggest, the following code will allow our microcontroller to send a packet and wait for a response from the device before unserializing it.

constexpr upd::byte_t header[4] = {0xff, 0xff, 0xff, 0xff};

// We fill our tuple with the header and the data and keep
// some room for the CRC.
upd::tuple output_tuple{
  upd::big_endian,
  upd::ones_complement,
  header,
  std::uint32_t{64},
  std::uint32_t{32},
  std::uint16_t{}
};

// We calculate the CRC from the data fields
upd::set<3>(output_tuple, calculate_crc(output_tuple.begin() + 4, output_tuple.end() - 2));

// Finally we send the packet to the device
for (auto byte: output_tuple) write_byte_to_device(byte);

// We prepare the tuple for the reception of the response
upd::tuple<
  upd::endianess::BIG,
  upd::signed_mode::ONES_COMPLEMENT,
  upd::byte_t[4],
  std::int16_t,
  std::uint16_t,
  std::uint16_t>
input_tuple{};

// We receive the packet from the device (assuming
// `read_byte_from_device` did not get rid of the header)
for (auto &byte: input_tuple) byte = read_byte_from_device();

// Finally we handle the values from the remote device
auto &&[rcv_header, arg1, arg2, crc] = input_tuple;
assert(std::equal(rcv_header.begin(), rcv_header.end(), header));
assert(crc == calculate_crc(input_tuple.begin() + 4, input_tuple.end() - 2));

return arg1 + arg2;

Make a view of a part of a tuple or out of an existing buffer using `upd::tuple_view`¶

In some case, having a view of a subtuple or interpreting a bare buffer as if it was a upd::tuple content is useful as it prevents any copy from occuring. A upd::tuple_view instance is either constructed from a buffer iterator or created from the upd::tuple::view() function.

Warning

upd::tuple_view will assume the end of the container it is associated with by incrementing the provided iterator according to the sizes of the elements. Therefore, this iterator might go past the real end iterator if the viewed container is not large enough.

Warning

upd::tuple_view do not extend the lifetime of the container it is associated with.

Customization points: defining serialization processes for foreign types¶

Unpadded only handles primitive types and array of those. However, you can make any type serializable with upd_extension.

// Here we define our object, and we want to make it
// serializable.
struct object_t {
  std::int16_t x, y;
  std::uint32_t z;
};

// Thanks to the wonders of template specialization, we can
// define how our object must be serialized and unserialized.
template<>
struct upd_extension<object_t> {
  
  // Here we define how `object_t` should be serialized by
  // packing it into a tuple view. Do not worry about the
  // exact type of the tuple view, the library will deduce
  // it from the signature of `unserialize`.
  template<typename View_T>
  static void serialize(const object_t &o, View_T &view) {
    upd::set<0>(view, o.x);
    upd::set<1>(view, o.y);
    upd::set<2>(view, o.z);
  }

  // Here we define how `object_t` should be unserialized
  // from a pack of integers.
  static object_t unserialize(std::int16_t x, std::int16_t y, std::uint32_t z) { return {x, y, z}; }
};

API References¶

`tuple`¶

template<endianess Endianess, signed_mode Signed_Mode, typename ...Ts> class tuple : public upd::detail::tuple_base<tuple<Endianess, Signed_Mode, Ts...>, Endianess, Signed_Mode, Ts...>¶

Unaligned storage tuple.

tuple instances hold values like a tuple but in an unaligned manner (i.e., there is no padding between two consecutive values).

Template Parameters:

Endianess, Signed_Mode – Serialization parameters
Ts... – Types of the serialized values

Public Types

using arg_t = detail::at<types_t, I>¶

Type of one of the serialized values.

Template Parameters:: I – Index of the requested type in Ts...

Public Functions

inline tuple()¶: Initialize the internal storage with default constructed values.

inline explicit tuple(const Ts&... args)¶

Serialize the provided values.

Parameters:: args... – Values to be serialized

inline explicit tuple(endianess_h<Endianess>, signed_mode_h<Signed_Mode>, const Ts&... values)¶

(C++17) Serialize the provided values

Template Parameters:: Endianess, Signed_Mode – Serialization parameters
Parameters:: values... – Values to be serialized

inline byte_t *begin()¶: Beginning of the internal storage.

inline const byte_t *begin() const¶: Beginning of the internal storage.

inline byte_t *end()¶: End of the internal storage.

inline const byte_t *end() const¶: End of the internal storage.

inline byte_t &operator[](std::size_t i)¶

Access the object content.

Warning

There is no bound check performed.

Parameters:: i – Index of the accessed byte

inline const byte_t &operator[](std::size_t i) const¶

Access the object content.

Warning

There is no bound check performed.

Parameters:: i – Index of the accessed byte

inline byte_t *src()¶: Beginning of the internal storage.

inline const byte_t *src() const¶: Beginning of the internal storage.

template<std::size_t I, std::size_t L> inline auto view()¶

Make a view out of a subset of the tuple.

Template Parameters:

I – First element in the view
L – Number of elements in the view

Returns:

a tuple_view instance including every elements in the given range

template<std::size_t I, std::size_t L> inline auto view() const¶

Make a view out of a subset of the tuple.

Template Parameters:

I – First element in the view
L – Number of elements in the view

Returns:

a tuple_view instance including every elements in the given range

auto get() const¶

Unserialize one of the value held by the object.

Template Parameters:: I – Index of the requested value
Returns:: A copy of the serialized value or a std::array instance if I designates an array type

inline void set(const arg_t<I> &value)¶

Set one of the value held by the object.

Template Parameters:: I – Index of the value which will be set
Parameters:: value – Value to be copied from

auto invoke(F &&ftor) const¶

Invoke a functor with the stored values.

Parameters:: ftor – Callback to be invoked
Returns:: ftor(upd::get<Is>()...) with Is = 0, 1, …, sizeof...(Ts)

Public Static Attributes

static constexpr auto size¶: Storage size in byte.

static constexpr auto storage_endianess¶: Equals the endianess given as template parameter.

static constexpr auto storage_signed_mode¶: Equals the signed mode given as template parameter.

template<typename ...Args, endianess Endianess, signed_mode Signed_Mode> tuple<Endianess, Signed_Mode, Args...> make_tuple(endianess_h<Endianess>, signed_mode_h<Signed_Mode>)¶

Construct a tuple instance holding default values.

Template Parameters:

Args... – Types of the values held by the tuple
Endianess, Signed_Mode – Serialization parameters

template<endianess Endianess, signed_mode Signed_Mode, typename ...Args> tuple<Endianess, Signed_Mode, Args...> make_tuple(endianess_h<Endianess>, signed_mode_h<Signed_Mode>, const Args&... args)¶

Construct a tuple instance from provided values.

Template Parameters:: Endianess, Signed_Mode – Serialization parameters
Parameters:: args... – Values to be serialized into the return value
Returns:: a tuple instance initialized from args...

`view_tuple`¶

template<typename It, endianess Endianess, signed_mode Signed_Mode, typename ...Ts> class tuple_view : public upd::detail::tuple_base<tuple_view<It, Endianess, Signed_Mode, Ts...>, Endianess, Signed_Mode, Ts...>¶

Binds a byte sequence to a tuple view.

Once bound, the byte sequence content can be read and modified as it were the content of a tuple instance. The byte sequence and the tuple view are bound through an iterator. It must at least be a forward iterator, but tuple views are faster with random access iterators. Best case would be a non-volatile plain pointer, as it can be called with memcpy.

Template Parameters:

It – Type of the iterator used for binding with the byte sequence
Endianess, Signed_Mode – Serialization parameters
Ts... – Types of the serialized values

Public Types

using types_t = upd::typelist_t<Ts...>¶: Typelist holding Ts...

using sizes_t = upd::typelist_t<serialization_size<Ts>...>¶: Typelist holding the sizes in byte of each type of Ts... when serialized.

using arg_t = detail::at<types_t, I>¶

Type of one of the serialized values.

Template Parameters:: I – Index of the requested type in Ts...

Public Functions

inline explicit tuple_view(const It &src)¶

Bind the view to a byte sequence through an iterator.

Parameters:: src – Iterator to the start of the byte sequence

inline const It &begin() const¶: Beginning of the byte sequence.

inline const It &end() const¶: End of the byte sequence.

inline const It &src() const¶: Beginning of the byte sequence.

auto get() const¶

Unserialize one of the value held by the object.

Template Parameters:: I – Index of the requested value
Returns:: A copy of the serialized value or a std::array instance if I designates an array type

inline void set(const arg_t<I> &value)¶

Set one of the value held by the object.

Template Parameters:: I – Index of the value which will be set
Parameters:: value – Value to be copied from

auto invoke(F &&ftor) const¶

Invoke a functor with the stored values.

Parameters:: ftor – Callback to be invoked
Returns:: ftor(upd::get<Is>()...) with Is = 0, 1, …, sizeof...(Ts)

Public Static Attributes

static constexpr auto size¶: Storage size in byte.

static constexpr auto storage_endianess¶: Equals the endianess given as template parameter.

static constexpr auto storage_signed_mode¶: Equals the signed mode given as template parameter.

template<typename ...Ts, typename It, endianess Endianess, signed_mode Signed_Mode> tuple_view<It, Endianess, Signed_Mode, Ts...> make_view(endianess_h<Endianess>, signed_mode_h<Signed_Mode>, const It &src)¶

Bind a byte sequence to a tuple view.

Template Parameters:

Ts... – Types held by the tuple
Endianess – Target endianess for serialization
Signed_Mode – Target signed representation for serialization

Parameters:

src – Start of the byte sequence

Returns:

a tuple_view instance bound to the byte sequence

template<std::size_t I, std::size_t L, endianess Endianess, signed_mode Signed_Mode, typename ...Ts> auto make_view(tuple<Endianess, Signed_Mode, Ts...> &tuple)¶

Bind a view to a slice of a tuple.

\upd_doc{MakeView_Tuple}

Template Parameters:

I – First element in the view
L – Number of elements in the view

Parameters:

tuple – Tuple to bind the view to

Returns:

a tuple_view including every elements in the given range

`get`¶

template<std::size_t I, typename D, endianess Endianess, signed_mode Signed_Mode, typename ...Ts> auto upd::get(const detail::tuple_base<D, Endianess, Signed_Mode, Ts...> &t)¶

Call the member function get() from an Unpadded tuple-like instance.

This function create a coherent interface with std::tuple for the sake of genericity.

Parameters:: t – Tuple to get a value from

`set`¶

template<std::size_t I, typename D, endianess Endianess, signed_mode Signed_Mode, typename ...Ts, typename U> void upd::set(detail::tuple_base<D, Endianess, Signed_Mode, Ts...> &t, U &&value)¶

Call the member function set() from an Unpadded tuple-like instance.

This function create a coherent interface std::tuple for the sake of genericity

Parameters:

t – Tuple to set a value in
value – Value to set

Serialization parameters¶

enum class upd::endianess¶

Used to specify endianess for serialization and unserialization.

Values:

enumerator LITTLE¶

enumerator BIG¶

enum class upd::signed_mode¶

Used to specify signed integer representation for serialization and unserialization.

Values:

enumerator SIGNED_MAGNITUDE¶

enumerator ONES_COMPLEMENT¶

enumerator TWOS_COMPLEMENT¶

enumerator OFFSET_BINARY¶

template<endianess Endianess> struct endianess_h : public upd::unevaluated<endianess, Endianess>¶: endianess enumerator holder for named parameters

template<signed_mode Signed_Mode> struct signed_mode_h : public upd::unevaluated<signed_mode, Signed_Mode>¶: signed_mode enumerator holder for named parameters

Other¶

using upd::byte_t = unsigned char¶

Portable type for inspecting object representation.

unsigned char being relevant to inspecting object representation regardless of the platform is enforced by the standard (Fundamental types - Character Types - cppreference).

Serialization tools¶

Formatting packets with the `upd::tuple` class¶

Example¶

Make a view of a part of a tuple or out of an existing buffer using `upd::tuple_view`¶

Customization points: defining serialization processes for foreign types¶

API References¶

`tuple`¶

`view_tuple`¶

`get`¶

`set`¶

Serialization parameters¶

Other¶

Unpadded

Navigation

Related Topics

Serialization tools¶

Formatting packets with the upd::tuple class¶

Example¶

Make a view of a part of a tuple or out of an existing buffer using upd::tuple_view¶

Customization points: defining serialization processes for foreign types¶

API References¶

tuple¶

view_tuple¶

get¶

set¶

Serialization parameters¶

Other¶

Formatting packets with the `upd::tuple` class¶

Make a view of a part of a tuple or out of an existing buffer using `upd::tuple_view`¶

`tuple`¶

`view_tuple`¶

`get`¶

`set`¶