Description
UTF-8 scalar decoding with byte, line, and column tracking.
Diagnostics carry a full source span rather than a message, so every byte the parser consumes has to move a position forward exactly once. This package owns that arithmetic and nothing else, which keeps it small enough to reason about and free of any dependency on the term model.
Advance
procedure Advance
(State : in out Cursor_State;
Scalar : Scalar_Value;
Length : Positive)
Move a position past one decoded scalar.
CR, LF, and CRLF each advance the line exactly once, which is what makes a column number comparable across platforms.
Parameters
- State
Position to advance
- Scalar
The scalar just consumed
- Length
Its encoded length in bytes
Cursor_State
type Cursor_State is record
Byte_Offset : Natural := 0;
Line : Positive := 1;
Column : Positive := 1;
After_Carriage_Return : Boolean := False;
end record;
A position in the input.
Record fields
- Byte_Offset
Bytes consumed so far, zero before the first
- Line
One-based line number
- Column
One-based column, counted in scalars rather than bytes
- After_Carriage_Return
True when the previous scalar was CR, so that a following LF completes one line break instead of starting a second one
Decode
procedure Decode
(Text : String;
Index : Positive;
Scalar : out Scalar_Value;
Length : out Natural;
Status : out Decode_Status)
Decode the scalar beginning at Text (Index).
Incomplete is reported only when the remaining bytes are a valid prefix of some longer sequence, so a caller feeding input in chunks can distinguish "wait for more" from "this input is wrong".
Parameters
- Text
Buffer to read from
- Index
Position of the lead byte
- Scalar
The decoded scalar value, meaningful only when Decoded
- Length
Bytes consumed, meaningful only when Decoded
- Status
Outcome of the decode
Decode_Status
type Decode_Status is (Decoded, Incomplete, Invalid);
Outcome of decoding one scalar.
Enumeration literals
- Decoded
A complete, canonical scalar was decoded
- Incomplete
The input ends part-way through a sequence, which is a chunk boundary rather than an error
- Invalid
The bytes are not a canonical UTF-8 encoding
Initial_State
Initial_State : constant Cursor_State :=
(Byte_Offset => 0,
Line => 1,
Column => 1,
After_Carriage_Return => False);
The position before any input has been consumed.
Is_Line_Break
function Is_Line_Break (Scalar : Scalar_Value) return Boolean
Report whether a scalar ends a line.
Parameters
- Scalar
Scalar to classify
Return value
True for LF and CR
Maximum_Scalar
Maximum_Scalar : constant := 16#10FFFF#;
Largest scalar value Unicode defines.
Scalar_Value
subtype Scalar_Value is Natural range 0 .. Maximum_Scalar;
One Unicode scalar value, which is what a decode yields.