Proposed 4.0 engine plan¶

This design supports the 4.0 product roadmap. It describes proposed contracts and implementation work, not new public signatures. The core contract and current compilation contract remain authoritative until a reviewed implementation changes them.

Starting point¶

Inklet 3.1 already separates live definitions from compiled snapshots, resolves explicit dependencies, measures physical text, fits nested layouts, and shares resolved output between SVG and PDF. It also has native vector 3D, external Blender rendering, bounded document caches and experimental figure planning.

Extend these foundations in stages. The 3.1 audit exposed stale nested-panel dependencies and incomplete array fingerprints, as well as numeric-input and reversed-span defects. A broader engine must make those classes of error easier to detect before adding more representations and caches.

1. Identity, data and revisions¶

Separate an author's persistent identity from a drawing allocation ID, a dependency revision and a content digest. A plot point, table row, region and scene object can refer to the same authored entity through explicit mappings; matching display text is not sufficient evidence of correspondence.

• Define IDs and their scope for documents, components, marks, datasets and imported objects. Specify behavior for copy, replacement, duplicate keys and removal. Preserve explicit identities across serialization and rebuilding.
• Keep literal snapshots distinct from live references. Dataset and file revisions must invalidate every dependent result, including nested content.
• Extend data adapters through one typed table boundary with explicit nulls, units, categories and source metadata. Preserve exact identifiers. Record transformations and joins, including their behavior when keys are missing.
• Hash immutable content once per recorded snapshot where possible. Use revision identities for live data; do not rescan an unchanged large array on every pointer interaction. Define cache keys for shape, dtype, content, renderer settings and any adapter metadata that changes meaning.
• Keep arbitrary Python factories available in Python builds. Serialization must describe their reproducibility requirements and distinguish supported browser operations from server-only evaluation.

Acceptance: generated edit sequences compare cached and uncached results, including large/strided arrays, nested dependencies, file changes, object replacement and removals. Old compiled snapshots must retain their output.

2. Coordinates, measurement and geometry¶

Retain millimetres for document geometry, shaped text before core rendering, and existing transform conventions. Represent data coordinates, image pixels, world coordinates, geographic coordinates and browser viewport pixels through explicit mappings at their boundaries.

Separate layout envelopes, painted bounds, clip regions and interaction hit regions. A clipped mark may retain data identity without retaining a visible hit target. A wide stroke or text halo must contribute to painted bounds even when the layout envelope is intentionally fixed.

Unify scale mapping, clipping and primitive traversal where behavior is shared. Extend robust predicates for reversed/degenerate spans, path intersections, mesh depth ordering and nested transforms. Approximation settings need physical or data-space error definitions and work limits. Picking reduced geometry must have a documented relationship to the original observations.

Acceptance: numerical transform and boundary checks, randomized geometry cases, and independent render comparisons cover curves, text, maps, images and scenes. Treat dates, time zones, map discontinuities and projection singularities as explicit cases rather than ordinary floating-point coordinates.

3. Incremental compilation and layout¶

Expose time and invalidation reasons for dependency evaluation, shaping, geometry construction, layout, routing, paint resolution and export. Distinguish an unchanged rebuild from a data edit, style edit, camera edit or page resize.

Cache immutable intermediate results with bounded memory and explicit ownership. Keep background work cancellable and prevent an older render from replacing a newer requested revision. Export only a complete, internally consistent snapshot.

Extend layout from grid placement toward shared data-region alignment, facets, annotation clearance, minimum readable size and author locks. Build on the existing measured layout and experimental planner. Hard requirements and soft preferences must remain distinguishable; finite candidate search must not claim that an infeasible result rules out every possible layout.

Acceptance: diagnostics identify the conflicting requirements; satisfied hard constraints remain satisfied after fitting and routing. Resize round trips and input revisions preserve feasible locks and report orphaned constraints. Font fallback, shaping and final backend text must agree on measurement.

4. Resolved output and rendering backends¶

The compiled-scene increment, after 4.0.0.dev2, implements shared native scene execution and bounded revision reuse. Native SVG/PDF/PNG export consumes resolved placement state and retains geometry resources and analysis. The next increment adds packed vector scatter buffers and a compiled-scene browser viewer with WebGL2 circles and Canvas fallback. Broader browser/GPU primitives, packed maps and meshes, GPU rendering and fine-grained layout invalidation remain open; the guide gives measured costs and the staged acceptance target.

The first post-dev1 compositing increment adds page-scoped paint-count/bounds reuse to PDF export and corrects PNG transparency and single-primitive opacity. The complete before/after figure and nested-group benchmark provide conformance and performance evidence; broader shared backend work below remains open. The subsequent hatch increment shares physical line geometry, reuses conservatively covered fill resources with exact per-shape clips, and tests inherited opacity, holes, transformations and export-order independence.

Extend the current resolved drawing representation with stable semantic references and explicit backend capabilities. Decide its exact form through prototypes; a second, inconsistent layout system in the browser would undermine static/interactive agreement.

SVG, PDF, raster output and HTML should consume a shared evaluated state. Allow backend-specific execution, batching and resource reuse while preserving geometry, clipping, text and paint semantics. Define support for opacity groups, blending, images, gradients, hatching and scene overlays in a capability matrix.

For dense geometry, investigate spatial indexes, shared buffers, batching and level of detail with measurable tolerances. Keep vector text and annotations over raster or GPU layers when possible. Record rasterization decisions and effective resolution at export size.

External Blender renders remain explicit jobs with scene/settings fingerprints, resource limits, pass metadata and the actual device recorded. Browser camera changes cannot silently promise a fresh photorealistic Blender render offline. Define which scenes can be manipulated locally and which use saved views or a Python-connected rendering job.

Acceptance: supported features have geometry/paint conformance cases across backends. Unsupported features fail or use a disclosed fallback. Deterministic SVG remains byte-stable in the fixed environment; GPU imagery uses documented visual/numerical tolerances rather than a cross-device byte-identity promise.

5. Interaction state and visual overrides¶

Define a versioned state format for selected IDs, filters, viewports, controls and camera parameters. Define a separate structured override layer for authored label positions, styles, panel dimensions and constraints. Transient hover state must not accidentally become a permanent source edit.

Export the data and supported operations needed by a standalone HTML document. Package its runtime and assets for offline use. Python callbacks require an explicit connected mode; saving a state must identify the data/specification revision against which it was evaluated.

The local editor uses commands with undo/redo and saves overrides instead of rewriting Python. Validate overrides on load and provide migration rules for versioned schemas. Data removed from a project must not linger in exports through unused embedded tables or stale asset references. Treat labels and imported metadata as data, not executable HTML.

Acceptance: save/reopen/rebuild reproduces the selected static output; code edits expose stale mappings and overrides. Keyboard selection, focus, text alternatives and accessible control labels work in the supported browsers.

6. Specialized capabilities through shared boundaries¶

Add geographic projections, field/volume processing and model formats through optional adapters that deliver explicit units, identity and geometry to the same document model. Start with the subsets required by the reference projects. Avoid coupling ordinary plots to all scientific or browser dependencies.

Diagram layout should expose reusable named symbols, ports, grouping and constraints, with measured connector labels and inspectable routing failures. Authored sections, exploded views and dimensions need a stated geometry scope; arbitrary CAD topology and robust solid Boolean operations require separate evaluation before being promised.

Prototype third-party extension boundaries with at least two adapters before stabilizing a plugin contract. Keep import attribution and asset manifests available to report and export tools.

The mesh-field workflow now exercises immutable triangle sources, supplied face fields, strict measurement joins, XY triangle picking, calibrated arrows and native 3D references through revisions and two-width exports. The separate nodal-grid workflow adds linear contours and RK4 streamlines, with masks, termination diagnostics and numerical oracles. Interactive 3D visibility and simulation format support remain open.

Measurement and acceptance corpus¶

During roadmap phase A, record the 3.1 baseline and choose numerical budgets for each workload on declared hardware. Reports must include input size, versions, backend, warm/cold state and at least build/edit/export times, peak memory and output size. Interactive tests additionally record latency distributions and dropped or superseded work.

Preserve original adversarial reproducers alongside permanent regression tests. Add property-based and metamorphic checks where they give independent evidence: cached versus clean build, permutation with preserved IDs, inverse transforms, resize round trips and equivalent array representations. Visual baselines must be reviewed rather than automatically replaced when a comparison fails.

Set performance budgets before optimizing. Approve any budget or fidelity tradeoff explicitly with recorded measurements; avoid a single speed multiplier that hides regressions in other workloads.

Migration and implementation order¶

Start with observability, regressions and identity/state prototypes. Implement the analyst workflow before expanding into the engineering and scientific projects, then integrate the editor against the same model. Harden each stage with complete exports and installation checks.

Preserve current Python authoring where practical through adapters. Document any unavoidable breaking behavior and provide conversion guidance before the 4.0 beta. Do not recover editable instructions by guessing from a fixed legacy drawing: preserve it as fixed artwork with a stated editing boundary.

The new plan does not weaken the current core contract or introduce a runtime dependency by itself. Each implementation change needs its own reviewed design, tests and compatibility decision. Time, keyframes and presentation sequencing remain deferred to the 5.0 design.

Curve-preserving geometric clipping and explicit painted windows now share resolved clip regions across SVG/PDF, with separate layout and painted bounds. See the clipping review for semantics, limits and measurements.