Builder

Sometimes you want to generate JSON string outputs efficiently. The simdjson library provides high-performance low-level facilities. When using these low-level functionalities, you are responsible to define the structure of your JSON document. Our more advanced interface automates the process using C++26 static reflection: you get both high speed and high convenience.

• Builder
  • Overview: string_builder
  • Example: string_builder
  • C++26 static reflection
    • Renaming and skipping fields with annotations
    • Without `string_buffer` instance
    • Without `string_buffer` instance but with explicit error handling
    • Pretty formatted (fractured JSON)

Overview: string_builder

The string_builder class is a low-level utility for constructing JSON strings representing documents. It is optimized for performance, potentially leveraging kernel-specific features like SIMD instructions for tasks such as string escaping. This class supports atomic types (e.g., booleans, numbers, strings) but does not handle composed types directly (like arrays or objects). Note that JSON strings are always encoded as UTF-8.

An string_builder is created with an initial buffer capacity (e.g., 1kB). The memory is reallocated when needed. The efficiency of string_builder stems from its internal use of a resizable array or buffer. When you append data, it adds the characters to this buffer, resizing it only when necessary, typically in a way that minimizes reallocations. This approach contrasts with regular string concatenation, where each operation creates a new string, copying all previous content, leading to quadratic time complexity for repeated concatenations.

It has the following methods to add content to the string:

• append(number_type v): Appends a number (including booleans) to the JSON buffer. Booleans are converted to the strings "false" or "true". Numbers are formatted according to the JSON standard, with floating-point numbers using the shortest representation that accurately reflects the value.
• append(char c): Appends a single character to the JSON buffer.
• append_null(): Appends the string "null" to the JSON buffer.
• clear(): Clears the contents of the JSON buffer, resetting the position to 0 while retaining the allocated capacity.
• escape_and_append(std::string_view input): Appends a string view to the JSON buffer after escaping special characters (e.g., quotes, backslashes) as required by JSON.
• escape_and_append_with_quotes(std::string_view input) Appends a string view surrounded by double quotes (e.g., "input") to the JSON buffer after escaping special characters. For constant strings, you may also do escape_and_append_with_quotes<"mystring">().
• escape_and_append_with_quotes(char input): Appends a single character surrounded by double quotes (e.g., "c") to the JSON buffer after escaping it if necessary.
• append_raw(const char *c): Appends a null-terminated C string directly to the JSON buffer without escaping.
• append_raw(std::string_view input): Appends a string view directly to the JSON buffer without escaping.
• append_raw(const char *str, size_t len): Appends a specified number of characters from a C string directly to the JSON
• append_key_value(key,value): Appends a key and a value ("json":somevalue)
• append_key_value<"mykey">(value): Appends a key and a value ("json":somevalue), useful when the key is a compile-time constant (C++20).

After writing the content, if you have reasons to believe that the content might violate UTF-8 conventions, you can check it as follows:

• validate_unicode(): Checks if the content in the JSON buffer is valid UTF-8. Returns: true if the content is valid UTF-8, false otherwise.

You might need to do unicode validation if you have strings in your data structures containing malformed UTF-8. Note that we do not automatically call validate_unicode().

Once you are satisfied, you can recover the string as follows:

• operator std::string(): Converts the JSON buffer to an std::string. (Might throw if an error occurred.)
• operator std::string_view(): Converts the JSON buffer to an std::string_view. (Might throw if an error occurred.)
• view(): Returns a view of the written JSON buffer as a simdjson_result<std::string_view> (C++20).

The later method (view()) is recommended. For performance reasons, we expect you to explicitly call validate_unicode() as needed (e.g., prior to calling view()).

Example: string_builder

struct Car {
    std::string make;
    std::string model;
    int64_t year;
    std::vector<double> tire_pressure;
};
 
void serialize_car(const Car& car, simdjson::builder::string_builder& builder) {
    // start of JSON
    builder.start_object();
 
    // "make"
    builder.append_key_value("make", car.make);
    builder.append_comma();
 
    // "model"
    builder.append_key_value("model", car.model);
    builder.append_comma();
 
    // "year"
    builder.append_key_value("year", car.year);
    builder.append_comma();
 
    // "tire_pressure"
    builder.escape_and_append_with_quotes("tire_pressure");
    builder.append_colon();
    builder.start_array();
    // vector tire_pressure
    for (size_t i = 0; i < car.tire_pressure.size(); ++i) {
        builder.append(car.tire_pressure[i]);
        if (i < car.tire_pressure.size() - 1) {
            builder.append_comma();
        }
    }
    builder.end_array();
    builder.end_object();
}
 
bool car_test() {
    simdjson::builder::string_builder sb;
    Car c = {"Toyota", "Corolla", 2017, {30.0,30.2,30.513,30.79}};
    serialize_car(c, sb);
    std::string_view p{sb};
    // p holds the JSON:
    // "{\"make\":\"Toyota\",\"model\":\"Corolla\",\"year\":2017,\"tire_pressure\":[30.0,30.2,30.513,30.79]}"
    return true;
}

The string_builder constructor takes an optional parameter which specifies the initial memory allocation in byte. If you know approximately the size of your JSON output, you can pass this value as a parameter (e.g., simdjson::builder::string_builder sb{1233213}).

The string_builder might throw an exception in case of error when you cast it result to std::string_view. If you wish to avoid exceptions, you can use the following programming pattern:

std::string_view p;
if(sb.view().get(p)) {
    return false; // there was an error
}

In all cases, the std::string_view instance depends the corresponding string_builder instance.

C++20

If you have C++20, you can simplify the code, as the std::vector<double> is automatically supported. Further, we can pass the keys (which are compile-time constant) as template parameter (for improved performance).

Car c = {"Toyota", "Corolla", 2017, {30.0,30.2,30.513,30.79}};
simdjson::builder::string_builder sb;
sb.start_object();
sb.append_key_value<"make">(c.make);
sb.append_comma();
sb.append_key_value<"model">(c.model);
sb.append_comma();
sb.append_key_value<"year">(c.year);
sb.append_comma();
sb.append_key_value<"tire_pressure">(c.tire_pressure);
sb.end_object();
std::string_view p = sb.view();

With C++20, you can similarly handle standard containers transparently. For example, you can serialize std::map<std::string,T> types.

std::map<std::string,double> c = {{"key1", 1}, {"key2", 1}};
simdjson::builder::string_builder sb;
sb.append(c);
std::string_view p = sb.view();

You can also serialize std::vector<T> types.

std::vector<std::vector<double>> c = {{1.0, 2.0}, {3.0, 4.0}};
simdjson::builder::string_builder sb;
sb.append(c);
std::string_view p = sb.view();

You can also skip the creation for the string_builder instance in such simple cases.

std::vector<std::vector<double>> c = {{1.0, 2.0}, {3.0, 4.0}};
std::string json = simdjson::to_json(c);

We also have an overload for when you want to reuse the same std::string instance:

std::vector<std::vector<double>> c = {{1.0, 2.0}, {3.0, 4.0}};
std::string json;
auto error = simdjson::to_json(c, json);
if(error) { /* there was an error */ }

We do recommend that you create and reuse the string_builder instance for performance reasons.

You can also add custom serialization functions using a tag_invoke function. For example, the following function will allow you to serialize instances of the type Car.

#include <simdjson>
 
struct Car {
  std::string make;
  std::string model;
  int64_t year;
  std::vector<float> tire_pressure;
};
 
namespace simdjson {
 
template <typename builder_type>
void tag_invoke(serialize_tag, builder_type &builder, const Car& car) {
  builder.start_object();
  builder.append_key_value("make", car.make);
  builder.append_comma();
  builder.append_key_value("model", car.model);
  builder.append_comma();
  builder.append_key_value("year", car.year);
  builder.append_comma();
  builder.append_key_value("tire_pressure", car.tire_pressure);
  builder.end_object();
}
 
} // namespace simdjson

C++26 static reflection

Static reflection (or compile-time reflection) in C++26 introduces a powerful compile-time mechanism that allows a program to inspect and manipulate its own structure, such as types, variables, functions, and other program elements, during compilation. Unlike runtime reflection in languages like Java or Python, C++26’s static reflection operates entirely at compile time, aligning with C++’s emphasis on zero-overhead abstractions and high performance. It means that you can delegate much of the work to the library.

And then you can append your data structures to a string_builder instance automatically. In most cases, it should work automatically:

struct Car {
    std::string make;
    std::string model;
    int64_t year;
    std::vector<double> tire_pressure;
};
 
bool car_test() {
    simdjson::builder::string_builder sb;
    Car c = {"Toyota", "Corolla", 2017, {30.0,30.2,30.513,30.79}};
    sb << c;
    std::string_view p{sb};
    // p holds the JSON:
    // "{\"make\":\"Toyota\",\"model\":\"Corolla\",\"year\":2017,\"tire_pressure\":[30.0,30.2,30.513,30.79]}"
    return true;
}

Renaming and skipping fields with annotations

This is experimental: the syntax may change slightly in the future.

When using C++26 static reflection for automatic serialization (and deserialization), you can annotate your struct members to rename the corresponding JSON keys or to exclude fields from the JSON representation.

The syntax is:

// rename cppFieldName (in C++) to json_key_name (in JSON)
[[= simdjson::rename<"json_key_name">]] std::string cppFieldName;
// do not serialize or deserialize this field
[[= simdjson::skip]] int internalState;

For example:

struct Person {
    [[= simdjson::rename<"first_name">]] std::string firstName = "";
    [[= simdjson::rename<"last_name">]]  std::string lastName = "";
    [[= simdjson::skip]] int internalCache = 0;
    int age = 0;
};

Serialization then produces:

Person p{"Alice", "Smith", 999, 30};
std::string json = simdjson::to_json(p);
// json == R"({"first_name":"Alice","last_name":"Smith","age":30})"
// Note: internalCache is omitted entirely.

The skip annotation also affects deserialization: the field keeps its default value and any corresponding key in the input JSON is ignored.

Other annotations, modelled after the Rust serde library, affect serialization:

struct [[= simdjson::rename_all<simdjson::case_style::camel_case>]] Account {
  std::string user_name;                                // "userName"
  [[= simdjson::skip_serializing]] std::string password; // deserialized, never serialized
  [[= simdjson::skip_serializing_if<simdjson::is_none>]] std::optional<std::string> email; // omitted when empty
  [[= simdjson::skip_serializing_if<simdjson::is_empty>]] std::vector<std::string> roles;  // omitted when empty
  [[= simdjson::with<yes_no>]] bool active;             // custom serialization via yes_no::serialize
};

Enumerators can be renamed with rename (or rename_all on the enumeration), a structure annotated with transparent is serialized as its single data member, and the members of a member annotated with flatten are written in place of that member. See Customizing (de)serialization with annotations for the complete list, including the annotations that only affect deserialization (alias, default_value, default_from, deny_unknown_fields).

Without <tt>string_buffer</tt> instance

In some instances, you might want to create a string directly from your own data type. You can create a string directly, without an explicit string_builder instance with the simdjson::to_json template function. (Under the hood a string_builder instance may still be created.)

struct Car {
    std::string make;
    std::string model;
    int64_t year;
    std::vector<double> tire_pressure;
};
 
void f() {
    Car c = {"Toyota", "Corolla", 2017, {30.0,30.2,30.513,30.79}};
    std::string json = simdjson::to_json(c);
}

If you know the output size, in bytes, of your JSON string, you may pass it as a second parameter (e.g., simdjson::to_json(c, 31123)).

Sometimes you may want to reuse the same std::string instance. We have an overload for this purpose:

Car c = {"Toyota", "Corolla", 2017, {30.0,30.2,30.513,30.79}};
std::string s;
auto error = simdjson::to_json(c, s);
if(error) { /* there was an error */ }

You can then also add a third parameter for the expected output size in bytes.

Extracting just some fields

In some instances, your class might have many fields that you do not want to serialize. You can achieve this result with the simdjson::extract_from template. In the following example, we serialize only the year and price fields on the Car instance.

struct Car {
    std::string make;
    std::string model;
    int year;
    double price;
    bool electric;
};
Car car{"Ford", "F-150", 2024, 55000.0, false};
// Extract year and price
std::string json_result = simdjson::extract_from<"year", "price">(car);
// Alternatively:
// std::string json_result;
// auto error =  extract_from<"year", "price">(car).get(json_result);
// if(error) { /* error handling */ }

Without <tt>string_buffer</tt> instance but with explicit error handling

If prefer a version without exceptions and explicit error handling, you can use the following pattern:

std::string json;
if(simdjson::to_json(c).get(json)) {
  // there was an error
} else {
  // json contain the serialized JSON
}

Customization

If you want to serialize a value in a custom way, you can do it with a tag_invoke specialization like the following example which will map the year attribute to a string.

#include <simdjson>
 
struct Car {
  std::string make;
  std::string model;
  int64_t year;
  std::vector<float> tire_pressure;
};
 
namespace simdjson {
 
template <typename builder_type>
void tag_invoke(serialize_tag, builder_type &builder, const Car& car) {
  builder.start_object();
  builder.append_key_value("make", car.make);
  builder.append_comma();
  builder.append_key_value("model", car.model);
  builder.append_comma();
  builder.append_key_value("year", std::to_string(car.year));
  builder.append_comma();
  builder.append_key_value("tire_pressure", car.tire_pressure);
  builder.end_object();
}
 
} // namespace simdjson

Pretty formatted (fractured JSON)

In some instances, you may want your JSON to be more readable. For this pupose, we also support the Fractured JSON standard.

TableTestData data{
  {{1, "Alice", true}, {2, "Bob", false}, {3, "Carol", true}, {4, "Dave", false}}
};
 
fractured_json_options opts;
opts.enable_table_format = true;
 
std::string formatted = simdjson::to_fractured_json_string(data, opts);

The result might be as follows. Columns are laid out in the order in which keys first appear.

{
    "records": [
        {"id": 1, "name": "Alice", "active": true },
        {"id": 2, "name": "Bob",   "active": false},
        {"id": 3, "name": "Carol", "active": true },
        {"id": 4, "name": "Dave",  "active": false}
    ]
}

The fractured_json_options struct allows you to customize the formatting behavior. It includes the following options:

• max_total_line_length (default: 120): Maximum total characters per line. Content exceeding this will be expanded to multiple lines.
• max_inline_complexity (default: 2): Maximum nesting depth for inline rendering. Elements with complexity exceeding this will be expanded. Complexity 0 = scalar, 1 = flat array/object, 2 = one level of nesting.
• max_compact_array_complexity (default: 2): Maximum complexity for compact array formatting. Arrays with elements of this complexity or less may have multiple items per line.
• indent_spaces (default: 4): Number of spaces per indentation level.
• always_expand_depth (default: -1): Forces elements close to the root to always fully expand, regardless of other settings. -1 = none; 0 = root node only; 1 = root node and its children; etc.
• enable_table_format (default: true): Enable tabular formatting for arrays of similar objects or arrays. When enabled, the rows of such an array are written one per line with their columns aligned. Rows need not have identical keys: columns are ordered by the first occurrence of each key, and a row missing a key gets blank space in that column. Nested arrays/objects within rows are aligned recursively; if the aligned rows do not fit within max_total_line_length, the deepest levels are left unaligned, and if they still do not fit, the array is expanded normally.
• max_table_row_complexity (default: 2): Maximum complexity of each row of a table. 0 = rows may only be scalars (a single column); 1 = rows may be flat arrays/objects; higher values allow deeper nesting.
• enable_compact_multiline (default: true): Enable compact multiline arrays. When enabled, arrays of simple elements may have multiple items per line.
• min_compact_array_row_items (default: 3): Minimum number of items per line for an array to be formatted as a compact multiline array. If fewer items fit on a line, another layout is used.
• simple_bracket_padding (default: false): Add space inside brackets for containers that hold only scalar values. When true: { "key": "value" }, when false: {"key": "value"}.
• nested_bracket_padding (default: true): Add space inside brackets for containers that hold at least one nested array/object. When true: { "a": [1, 2] }, when false: {"a": [1, 2]}.
• colon_padding (default: true): Add space after colons. When true: "key": "value", when false: "key":"value".
• comma_padding (default: true): Add space after commas in inline content. When true: [1, 2, 3], when false: [1,2,3].
• comma_placement (default: before_padding_except_numbers): Placement of commas relative to column padding in table-formatted rows and compact multiline arrays. One of:
  • table_comma_placement::before_padding: commas come right after the value: "b": 1, "c": 2.
  • table_comma_placement::after_padding: commas come after the padding, so they line up in their own column: "b": 1 , "c": 2.
  • table_comma_placement::before_padding_except_numbers: like before_padding, except that number columns use after_padding.
• number_alignment (default: left): Controls alignment of numbers in table columns or compact multiline arrays. One of number_list_alignment::left or number_list_alignment::right. Numbers are always written exactly as in the input. (FracturedJson defaults to decimal-point alignment, which simdjson does not support.)

Compared with the reference FracturedJson implementation, simdjson does not support the options related to comments, blank-line preservation, trailing commas, line-ending style, tab indentation, line prefixes, property-name padding in expanded objects (MaxPropNamePadding, ColonBeforePropNamePadding), or the Decimal and Normalize number alignments.

The min_table_rows and table_similarity_threshold options found in earlier versions of simdjson have been removed.