simdjson::dom::document_stream Class Reference

A forward-only stream of documents. More...

#include <document_stream.h>

Classes
class	iterator
	An iterator through a forward-only stream of documents. More...

Public Member Functions
simdjson_inline	document_stream () noexcept
	Construct an uninitialized document_stream.
simdjson_inline	document_stream (document_stream &&other) noexcept=default
	Move one document_stream to another.
simdjson_inline document_stream &	operator= (document_stream &&other) noexcept=default
	Move one document_stream to another.
size_t	size_in_bytes () const noexcept
	Returns the input size in bytes.
size_t	truncated_bytes () const noexcept
	After iterating through the stream, this method returns the number of bytes that were not parsed at the end of the stream.
simdjson_inline iterator	begin () noexcept
	Start iterating the documents in the stream.
simdjson_inline iterator	end () noexcept
	The end of the stream, for iterator comparison purposes.

Detailed Description

A forward-only stream of documents.

Produced by parser::parse_many.

Definition at line 72 of file document_stream.h.

Constructor & Destructor Documentation

document_stream()

simdjson_inline simdjson::dom::document_stream::document_stream ( )

noexcept

Construct an uninitialized document_stream.

document_stream docs;
error = parser.parse_many(json).get(docs);

Definition at line 112 of file document_stream-inl.h.

~document_stream()

simdjson_inline simdjson::dom::document_stream::~document_stream ( )

noexcept

Definition at line 125 of file document_stream-inl.h.

Member Function Documentation

begin()

simdjson_inline document_stream::iterator simdjson::dom::document_stream::begin ( )

noexcept

Start iterating the documents in the stream.

Definition at line 135 of file document_stream-inl.h.

end()

simdjson_inline document_stream::iterator simdjson::dom::document_stream::end ( )

noexcept

The end of the stream, for iterator comparison purposes.

Definition at line 141 of file document_stream-inl.h.

size_in_bytes()

size_t simdjson::dom::document_stream::size_in_bytes ( ) const

inlinenoexcept

Returns the input size in bytes.

Definition at line 275 of file document_stream-inl.h.

truncated_bytes()

size_t simdjson::dom::document_stream::truncated_bytes ( ) const

inlinenoexcept

After iterating through the stream, this method returns the number of bytes that were not parsed at the end of the stream.

If truncated_bytes() differs from zero, then the input was truncated maybe because incomplete JSON documents were found at the end of the stream. You may need to process the bytes in the interval [size_in_bytes()-truncated_bytes(), size_in_bytes()).

You should only call truncated_bytes() after streaming through all documents, like so:

document_stream stream = parser.parse_many(json,window); for(auto doc : stream) { // do something with doc } size_t truncated = stream.truncated_bytes();

IMPORTANT: this value is only meaningful under the conditions below. It is computed from stage-1 bookkeeping that the other stream formats do not maintain, and outside these conditions it is not merely imprecise, it is arbitrary – it can exceed size_in_bytes() or wrap around to a huge value. Check it only when all of the following hold:

• the format is whitespace_delimited or newline_delimited. In json_sequence and comma_delimited mode the stage-1 filter rewrites the structural index in place and the bookkeeping is lost, so the value is meaningless even for a stream that parsed completely;
• you iterated all the way to the end of the stream;
• no document reported an error. Iteration stops at the first failed document, which can leave the bookkeeping from a mid-stream batch.

If you need to know about a truncated tail outside those conditions, track it yourself from the last successful document (see iterator::current_index() and iterator::source()).

Definition at line 279 of file document_stream-inl.h.

---

The documentation for this class was generated from the following files:

• include/simdjson/dom/document_stream.h
• include/simdjson/dom/document_stream-inl.h