kids.kapish.data.morsels

unity

Entity management and chunk-based binary serialization for large-scale game data.

Installation

Add to your Unity project's package manifest:

{
  "kids.kapish.data.morsels": "file:../../src/unity/kids.kapish.data.morsels"
}

Requires Unity 6000.0+. No engine references -- this assembly is pure C#.

Architecture

Morsel Identity

Every game entity is a morsel -- a lightweight data object with identity. Two identity modes:

• Auto-incremented (uint): Morsel base class. The repo assigns incrementing IDs automatically via Create(). Good for entities whose identity is arbitrary (units, items, events).
• Custom-keyed (TKey): Morsel<TKey> base class. The repo requires an explicit key on Create(key). Good for entities with domain-specific identity (region codes, named resources).

Both expose their ID/key as a read-only property. The setter is internal -- only the repo assigns identity.

Repositories

MorselRepo<T> and MorselRepo<TKey, TMorsel> are typed collections that enforce unique keys. They are backed by RandomDictionary from Carrot.Collections, which enables weighted random selection via GetClause.

Repos handle:

• Create -- allocate a new morsel with identity assigned.
• Add / Set -- register an existing morsel (e.g. from deserialization). Add throws on duplicates; Set upserts.
• Delete -- remove a morsel and dispose it if it implements IDisposable.
• Enumeration -- repos implement IEnumerable<T>.
• Random selection -- GetClause builds a weighted random selector from a value function and optional filter.

Binary Format

The IO system implements a big-endian binary format with chunk-based structure. See Documentation/save-format.md for the full specification.

Writing pipeline:

MorselBinaryWriter          -- accumulates bytes in memory
├── MorselBinaryChunkWriter -- writes chunk header + body into parent writer
├── MorselBinaryArrayWriter -- writes array header + items into parent writer
└── MorselBinaryStreamWriter -- flushes accumulated bytes to a Stream

Reading pipeline:

MorselBinaryReader           -- sequential reads from byte[]
├── MorselBinaryChunkReader  -- reads/skips chunk headers
├── MorselBinaryArrayReader  -- reads/skips typed arrays
└── MorselBinaryStreamReader -- loads a Stream into the base reader

All multi-byte values are big-endian. Strings are UTF-16 (2 bytes per char), either length-prefixed (ushort count) or fixed-length (padded with null bytes). Addresses are 8-byte positions within the byte stream, used for skip-ahead navigation.

Chunk Format

Each chunk has a standard header:

Field	Type	Size
Type	ushort	2b
Version	version (major.minor.build)	8b
Next	address	8b

Unknown chunk types can be skipped by seeking to the Next address. This makes the format forward-compatible -- old readers skip new chunk types gracefully.

Array Format

Arrays have a small header followed by concatenated item bytes:

Field	Type	Size
Count	uint	4b
Skip	address	8b
Payload	binary	variable

File Structure

Runtime/
├── Core/     # IMorsel, Morsel base classes, IHasName
├── Repos/    # MorselRepo<T>, MorselRepo<TKey,TMorsel>, exception
└── IO/       # Binary readers and writers (stream, chunk, array)

Dependencies

Dependency	Kind
kids.kapish	runtime (Carrot core Unity package -- provides RandomDictionary, IRandomClause)

Build

Import via Unity Package Manager. Requires Unity 6000.0+.

---

Usage Guide

Entity management and chunk-based binary serialization for large-scale game data.

Setup

Add kids.kapish.data.morsels to your Unity project manifest. The package has no engine references and works in pure C# contexts.

Common Patterns

1. Define a morsel type

Extend Morsel for auto-incremented uint identity:

using Carrot.Data.Morsels;

public class Unit : Morsel
{
    public string Name { get; set; }
    public int Health { get; set; }
    public int Attack { get; set; }
}

Or extend Morsel<TKey> for a custom key type:

using Carrot.Data.Morsels;

public class Region : Morsel<ushort>
{
    public string Name { get; set; }
    public float Area { get; set; }
}

2. Create a repository

using Carrot.Data.Morsels;

// Auto-keyed repo -- IDs assigned automatically
var units = new MorselRepo<Unit>();

Unit soldier = units.Create(u =>
{
    u.Name = "Soldier";
    u.Health = 100;
    u.Attack = 15;
});

// Custom-keyed repo -- keys provided explicitly
var regions = new MorselRepo<ushort, Region>();

Region homeland = regions.Create(1, r =>
{
    r.Name = "Homeland";
    r.Area = 50000f;
});

3. Query and delete

Unit unit = units[soldier.MorselId];
Region region = regions[1];

units.Delete(soldier);   // returns true if found; disposes if IDisposable
regions.Delete(1);       // delete by key

4. Weighted random selection

// Select a random unit weighted by attack power, excluding dead units
var clause = units.GetClause(
    getValue: u => u.Attack,
    filter: u => u.Health > 0
);

5. Enumerate

foreach (Unit unit in units)
{
    // iterate all units
}

int count = units.Count;

6. Add pre-existing morsels

When loading from a save, morsels already have identity assigned. Use Add (throws on duplicate) or Set (upserts):

var loaded = new Unit { Health = 80, Attack = 12 };
// MorselId is set internally during deserialization

units.Add(loaded);   // throws MorselKeyAlreadyRegisteredException if duplicate
units.Set(loaded);   // overwrites if exists, adds if not

7. Write binary data

using Carrot.Data.Morsels.IO;

using var writer = new MorselBinaryStreamWriter(outputStream);

// Write a file header
writer.WriteBytes("DEADBEEF");              // magic bytes (hex string)
writer.WriteVersion(new Version(1, 0, 0));  // game version
writer.WriteVersion(new Version(1, 0, 0));  // save version
writer.WriteLong(timestamp);                // created timestamp
writer.WriteString("My Save");              // save name

// Write a chunk
using (var chunk = new MorselBinaryChunkWriter(writer, chunkId: 1, new Version(1, 0, 0)))
{
    chunk.WriteUInt(soldier.MorselId);
    chunk.WriteString(soldier.Name);
    chunk.WriteInt(soldier.Health);
    chunk.WriteInt(soldier.Attack);
}

// Write an array inside a chunk
using (var chunk = new MorselBinaryChunkWriter(writer, chunkId: 2, new Version(1, 0, 0)))
{
    using var array = new MorselBinaryArrayWriter(chunk);

    foreach (Unit unit in units)
    {
        var item = array.AddItem();
        item.WriteUInt(unit.MorselId);
        item.WriteString(unit.Name);
        item.WriteInt(unit.Health);
        item.WriteInt(unit.Attack);
    }
}

8. Read binary data

using Carrot.Data.Morsels.IO;

var reader = new MorselBinaryStreamReader(inputStream);

// Read file header
byte[] magic = reader.ReadBytes(8);
Version gameVersion = reader.ReadVersion();
Version saveVersion = reader.ReadVersion();
long created = reader.ReadLong();
string name = reader.ReadString();

// Read chunks
while (reader.HasRemaining)
{
    MorselChunkHeader header = MorselBinaryChunkReader.ReadChunkHeader(reader);

    switch (header.Type)
    {
        case 1:
            uint id = reader.ReadUInt();
            string unitName = reader.ReadString();
            int health = reader.ReadInt();
            int attack = reader.ReadInt();
            break;

        case 2:
            Unit[] loaded = MorselBinaryArrayReader.ReadArray(reader, r =>
            {
                var u = new Unit();
                // MorselId would be set internally
                u.Name = r.ReadString();
                u.Health = r.ReadInt();
                u.Attack = r.ReadInt();
                return u;
            });
            units.AddRange(loaded);
            break;

        default:
            // Unknown chunk -- skip it
            MorselBinaryChunkReader.SkipChunk(reader, header);
            break;
    }
}

Tips

• Big-endian everywhere -- all multi-byte values use network byte order (most significant byte first).
• Chunks are forward-compatible -- unknown chunk types are safely skipped via the Next address in the header.
• Strings are UTF-16 -- 2 bytes per character. Variable-length strings are prefixed with a ushort character count. Fixed-length strings are null-padded.
• Addresses are absolute -- they represent positions within the byte stream, not relative offsets.
• Dispose writers in order -- chunk and array writers write their headers on Dispose(). Always dispose inner writers before outer ones (use nested using blocks).
• 4 billion IDs per repo -- MorselRepo<T> uses uint auto-increment. Reassign IDs if you anticipate overflow.