Datatypes

SDL provides the following datatypes.

In addition, several higher-order types are provided:

• array(T) - an array of type T;
• map(K, V) - a map with keys of type K and values of type V;
• one_of(Ts...) - a sum type over types Ts;
• set(T) - a distinct set of type T;
• structs.

Booleans and integers

Example

export class MyMessage extends KafkaMessage {
	BoolField = field(bool);
	SmallNumber = field(i8);
	MediumNumber = field(i16);
	BigNumber = field(i32);
	VeryBigNumber = field(i64);
	// ...
}

Implementations

Serialisation

These types serialise natively to JSON. Numbers in JSON do not have any specific size or precision (although some implementations parse them as IEEE 754 floats, which is unfortunate but sometimes unavoidable).

Decimals

The decimal type provides a way to communicate fractional numbers without floating-point error. A decimal field may be either fixed (where the number of decimal places is known at design time) or dynamic.

Example

export class MyMessage extends KafkaMessage {
	Dynamic = field(decimal); // up to 23 decimal places
	Fixed = field(decimal(4)); // always 4 decimal places
	// ...
}

Implementations

Serialisation

JSON string formatted with the required number of decimal places. When parsing, if the number of decimals is not specified at design time, it is deduced from the serialised string.

json_object

The json_object type is provided for backwards compatibility with legacy producers that emit nested values of arbitrary shape.

No validation is performed on json_object fields.

🛑 New code should use structs or maps instead.

Example

export class MyMessage extends KafkaMessage {
	LegacyField = field(json_object);
	// ...
}

Implementations

Serialisation

Literal JSON.

Strings

Example

export class MyMessage extends KafkaMessage {
	Note = field(str); // unlimited length
	ShortNote = field(str(40)); // up to 40 bytes
	// ...
}

Note that the the size of a string is given in bytes, not any other metric by which a string might be measured.

Implementations

Serialisation

Native JSON strings.

String Literals

Example

export class MyMessage extends KafkaMessage {
	LiteralValue = field("foobar"); // always "foobar"
}

Implementations

Serialisation

Native JSON strings.

Dates

A date with a year, month, and day.

Dates do not have an associated time zone.

Example

export class MyMessage extends KafkaMessage {
	Date = field(date);
	// ...
}

Implementations

Serialisation

JSON string formatted in YYYY-MM-DD format.

Nils

An empty value.

Nils can be used within one_of types to provide an empty choice. They do not have much use elsewhere: a field whose sole type is nil is disallowed, and a field whose type is one_of(T, nil) is lifted to an optional field of type T.

Example

export class MyMessage extends KafkaMessage {
	OneOfAOrBOrNil = field(one_of(A, B, nil));
	// ...
}

Implementations

Serialisation

JSON null.

Timestamps

A point in time with abitrary, undefined precision, usually nanoseconds.

In memory, timestamps do not have an associated time zone. In serialised form, a time zone offset code (Z or +00:00, to indicate UTC) should be used to avoid any ambiguity about how to deserialise the value.

Example

export class MyMessage extends KafkaMessage {
	Time = field(timestamp);
	// ...
}

Implementations

Serialisation

JSON string formatted according ISO 8601, including the timezone (which should without exception be UTC).

UUIDs

A version 4 UUID (122 randomly-generated bits, 128 in total).

Example

export class MyMessage extends KafkaMessage {
	Id = field(uuid);
	// ...
}

Implementations

Serialisation

JSON string formatted according to the string representation set out in IETF RFC 4122 § 3.

Enums

Enumerations can be declared by extending the SdlEnum class. On the wire, enum values are serialised to strings (for maximum compatibility) but in memory, they are represented using the language's native enum type (usually backed by an integer, but it should be fairly opaque).

Example

// in namespace Enums in enums.ts
export class OrderStatus extends SdlEnum.define(
	"OmsPending",
	"OmsRejected",
	"OmsSubmitted",
	"ExchangeRejected",
	"ExchangePlaced",
	"ExchangeFilled",
) {}

// in commands.ts
export class MyMessage extends KafkaMessage {
	Status = field(Enums.OrderStatus);
	// ...
}

Implementations

Serialisation

JSON string containing the name of the enum value.

Structs

Structures can be created using the Struct type. A struct consists of named fields of specific types. Messages and models are also subtypes of Struct.

Example

// Struct definition
export class MyStruct extends Struct {
	Foo = field(i64);
	Bar = field(str);
}

// Usage
export class MyMessage extends KafkaMessage {
	FooBar = field(MyStruct);
	// ...
}

Implementations

Serialisation

JSON object.

Arrays

The array() function is used to define an array of any other type (including another array). Arrays have an optional maximum length.

Example

export class MyMessage extends KafkaMessage {
	Notes = field(array(str)); // any number of notes
	UpTo10Notes = field(array(str, 10)); // up to 10 notes
	// ...
}

Implementations

Serialisation

JSON array.

Sum types (unions)

one_of defines a sum type where each consitutent type is either a struct, a string, or a number. A one_of field may be inhabited by only one of those types. On the wire, the value is represented:

• in the case of a primitive type, normally;
• in the case of a struct, as an object with one property whose key is the name of the struct inhabiting the union.

one_of requires at least one constituent. If only one is given, the union is erased and replaced with the constituent type.

Example

export class OrderConfirmEvent extends KafkaMessage {
	Details = field(one_of(ExecutedOrderDetails, RejectedOrderDetails));
	// ...
}

This serialises to an object with a single entry whose key is the name of the active type inhabiting the union:

{
	"ExecutedOrderDetails": { ... }
}

It is possible to derive a union of string literals from either the tags of another union type, or the fields of a struct:

export class MyStruct extends Struct {
	Field1 = field(str);
	Field2 = field(i64);
}

const T = one_of(str, i64);

export class UpdateEvent extends KafkaMessage {
	Field = field(key_of(MyStruct)); // "Field1" | "Field2"
	Type = field(key_of(T)); // "str" | "i64"
}

Implementations

Serialisation

A JSON object containing a single property whose key is the tag of the active union member, and whose value is a valid serialised value for that type.

Limitations

In order to preserve backwards compatibility with existing non-SDL services, when strings, numbers, and decimal types are used in a union they are serialised without the enclosing object. Therefore, it is not possible to define a union that contains:

• multiple numeric types; or
• both a string and a decimal field (since decimals also serialise to strings).

Sets

The set() function is used to define a set of any primitive type. A set is a distinct, unordered collection of values. Like an array, a set may optionally have a maximum size defined.

Example

export class MyMessage extends KafkaMessage {
	Flags = field(set(Enums.MyFlag));
	// ...
}

Implementations

Serialisation

JSON array.

Limitations

A set may only contain values of a basic type. Permitted types are:

• integers
• enums
• strings
• foreign keys

It may be possible to support more complex types in the future. The chief limitation is that any type added to a set must be hashable and comparable, and while at some level all of our types are comparable, we currently do not generate hash functions for complex types such as maps, structs, or arrays.

Maps

map defines a map (dictionary) type whose keys are not statically known. The key type of the map may be a string or an enum; the value type may be any type.

Example

export class MyMessage extends KafkaMessage {
	MapField = field(map(str, i32)); // string keys, integer values
	// ...
}

Implementations

Serialisation

A JSON object representing the entries in the map. The keys and values are serialised according to the serialisation rules for the key type and value type, respectively.

Limitations

Because JSON objects only support string keys, only string or enum types are presently allowed as map keys, because they natively serialise to strings. It may be possible to relax this restriction in the future.