06. Scalable Implementation Plans for Kyna

Based on our in-depth research into the Go language repository (/Users/ahmedmansour/Documents/go), this document outlines four concrete, phased engineering plans to implement Go's scalable architecture inside Kyna.

Plan A: Package & Module Architecture Overhaul

Objective

Transform Kyna from a script-centric single-file runner into a package-based language supporting multi-file compilation units, compiler-enforced internal/ encapsulation, and fast incremental builds.

1graph TD
2    DIR["Package Directory (e.g. ./services/auth)"] -->|Collect all .kyna files| PKG["Single Package AST (Merged Namespace)"]
3    PKG -->|Check Access Control| INT{"Import contains '/internal/'?"}
4    INT -->|Yes| CHK["Verify Importer is inside parent directory"]
5    INT -->|No| TC["Type Check Package"]
6    CHK -->|Valid| TC
7    CHK -->|Invalid| ERR["Diagnostic: KMOD1004 (Internal boundary violation)"]
8    TC -->|Serialize Interface| EXP["Write Binary Export Data (.kyc)"]
9    TC -->|Lower to Bytecode| BC["Package Bytecode Module"]

Technical Deliverables

1. Multi-File Package Resolution:
   • Update tools/kyna_cli/src/command/run_command.cpp and check_command.cpp to accept directory paths in addition to file paths (ky run ./mypackage).
   • When a directory is targeted, collect all .kyna files (excluding *_test.kyna), parse them in parallel, and merge their declarations into a single package scope.
2. Compiler-Enforced internal/ Boundary Rule:
   • In compiler/kyna_typecheck/src/checkers/module_analyzer.cpp, inspect the resolved filesystem path of every import declaration.
   • If /internal/ appears in the path, extract the parent directory of internal. Verify that the importing file's canonical directory begins with this parent path.
   • If the check fails, report diagnostic:
     Instructions

     Plain textCopy

     1error [KMOD1004]: use of internal module 'services/auth/internal/crypto' not allowed from 'api/handler.kyna'

3. Module Export Data Caching:
   • Add a lightweight binary serializer for compiler/kyna_symbols to emit .kyc headers containing only exported function signatures and struct types.
   • Subsequent compilations load the .kyc header in $< 1\text{ms}$ instead of re-parsing source text.

Plan B: Type System Modernization (types2 Architecture)

Objective

Replace Kyna's lightweight string-based TypeRef model with a robust, polymorphic type object graph inspired by Go's cmd/compile/internal/types2, introducing tri-color cycle detection and formal method sets.

Technical Deliverables

1. Polymorphic Type Graph (compiler/kyna_types): Refactor compiler/kyna_types/include/kyna/semantics/type_model.hpp into a class hierarchy:
   cpp

   cppCopy

   1namespace kyna::semantics {
   2class Type {
   3public:
   4    virtual ~Type() = default;
   5    virtual const Type* underlying() const = 0;
   6    virtual std::string str() const = 0;
   7    virtual bool is_identical(const Type* other) const = 0;
   8    virtual bool assignable_to(const Type* target, std::string* reason) const = 0;
   9};
   10}

2. Tri-Color Dependency Checking (compiler/kyna_typecheck): Implement Go's declaration checking algorithm in compiler/kyna_typecheck/src/checkers/type_checker.cpp:
   • Maintain a checking stack: std::vector<Symbol*> obj_path; and state map: enum State { White, Grey, Black };.
   • Before checking symbol S, mark Grey and push onto obj_path.
   • If an uncompleted (Grey) dependency is encountered without an indirection (e.g., recursive direct struct embedding), fail immediately with a cycle diagnostic.
   • When checking succeeds, pop and mark Black.
3. Method Set Calculation & Structural Satisfaction:
   • Implement lookup.cpp to calculate the complete method set of any type (including embedded structs).
   • In interface_checker.cpp, verify interface implementation via structural method set subset matching: $\text{Methods}(\text{Interface}) \subseteq \text{Methods}(\text{Target})$.

Plan C: Universal Streaming & Concurrency Architecture

Objective

Introduce universal streaming interfaces (Reader, Writer) and structured lifecycle propagation (Context) across Kyna's standard library and host adapters.

Technical Deliverables

1. Universal Streaming Seam in library/core:
   • Define canonical Reader and Writer interfaces in Kyna's core standard library.
   • Implement streaming byte buffers, file streams, and HTTP socket streams complying with Reader and Writer.
   • Provide io.copy(writer, reader, buffer_size) in native C++ (runtime/kyna_host), enabling efficient file copy, HTTP upload/download, and pipe processing with fixed $O(1)$ memory consumption.
2. Context Propagation Seam:
   • Add kyna.context with Context, WithTimeout(ctx, duration), and WithCancel(ctx).
   • Thread context handles through db.query and http.fetch. When a timeout fires or context is canceled, abort the underlying libcurl transfer or PostgreSQL query immediately.

Plan D: Systematic Standard Library Expansion

Objective

Implement the highest-priority standard library packages identified in the Go gap analysis, strictly adhering to Kyna's AGENTS.md clean-code guidelines and 4-point registration invariant.

The 4-Point Registration Invariant (AGENTS.md)

Every newly added standard library native must be registered in all 4 required locations:

1. library/core/src/bytecode/bytecode_standard_library.cpp (VM invocation)
2. library/core/src/catalog/standard_library_catalog.cpp (Tree-walk native)
3. compiler/kyna_symbols/src/catalog/standard_library_symbols.cpp (Semantic analysis symbols)
4. tests/tooling/verify_language_examples.py (Builtin test coverage)

Phased Roadmap

1┌─────────────────────────────────────────────────────────────┐
2│ PHASE 1: FOUNDATIONAL UTILITIES                             │
3│ • kyna_time: Time, Duration, now(), sleep(Duration)         │
4│ • kyna_io: Reader, Writer, Closer, copy(dst, src)           │
5└──────────────────────────────┬──────────────────────────────┘
6                               │
7                               ▼
8┌─────────────────────────────────────────────────────────────┐
9│ PHASE 2: SECURITY & OBSERVABILITY                           │
10│ • kyna_crypto: sha256, sha512, hmac, randomBytes            │
11│ • kyna_log: slog.info, slog.warn, slog.error, JSON handler  │
12└──────────────────────────────┬──────────────────────────────┘
13                               │
14                               ▼
15┌─────────────────────────────────────────────────────────────┐
16│ PHASE 3: CONCURRENCY & NETWORKING                           │
17│ • kyna_sync: Mutex, RWMutex, WaitGroup                      │
18│ • kyna_net_http: HTTP Server (ListenAndServe), route mux    │
19└─────────────────────────────────────────────────────────────┘

Detailed Phase 1 Implementation Blueprint

Module 1: kyna_time

• Host Adapter: runtime/kyna_host/src/adapters/clock_adapter.cpp (query monotonic std::chrono::steady_clock and wall-time std::chrono::system_clock).
• Standard Library Natives:
  • timeNow() -> TimeObject
  • timeSleep(nanoseconds: int) -> void
  • timeFormat(time: TimeObject, layout: str) -> str
• Symbols Registered:
  • timeNow: Arity 0, returns Time
  • timeSleep: Arity 1, accepts int, returns void
  • timeFormat: Arity 2, accepts (Time, str), returns str

Module 2: kyna_crypto

• Host Adapter: Integrate SHA-256 and cryptographically secure random bytes in runtime/kyna_host.
• Standard Library Natives:
  • cryptoSha256(data: str) -> str
  • cryptoRandomBytes(length: int) -> str
• Symbols Registered:
  • cryptoSha256: Arity 1, accepts str, returns str
  • cryptoRandomBytes: Arity 1, accepts int, returns str

Summary of Implementation Order & Verification