01. Repository and Package Architecture

1. The Official Go Repository Layout

The Go repository (/Users/ahmedmansour/Documents/go) is organized with extreme intentionality. Unlike typical projects that fragment code across dozens of arbitrary top-level folders, Go maintains a lean, predictable top-level layout:

1go/
2├── api/            # Versioned public API dumps (go1.21.txt, go1.22.txt) for backward-compatibility verification
3├── doc/            # Canonical language specification, release notes, and contributor documentation
4├── lib/            # Time zone database files and runtime support tables
5├── misc/           # Editor plugins (vim, emacs), cgo examples, and auxiliary scripts
6├── src/            # THE SOURCE ROOT: standard library, runtime, compiler, and core CLI tools
7│   ├── cmd/        # Command-line tools (compiler, linker, assembler, go tool, vet, gofmt)
8│   ├── internal/   # Standard library internal helper packages (hidden from external imports)
9│   ├── runtime/    # Go runtime engine (goroutines, garbage collector, memory allocator, stack management)
10│   └── <pkg>/      # 50+ standard library packages (fmt, net, os, io, sync, time, etc.)
11└── test/           # Comprehensive compiler validation suites, golden regression tests, and errorcheck tests

Key Structural Invariants

1. Everything Lives in src/: The entire Go ecosystem (standard library, compiler, runtime, CLI tools) originates from a single unified import root: src/. When Go compiles import "fmt", it looks for src/fmt. When it compiles import "cmd/compile/internal/syntax", it looks for src/cmd/compile/internal/syntax. There is no configuration drift between third-party resolution and standard library resolution.
2. api/ Enforces the Compatibility Promise: Go guarantees that code written for Go 1.0 will compile and run without modification on any future Go 1.x version. The api/ directory contains sorted textual signatures of every exported function, type, method, and constant added in each release. The cmd/api tool scans the standard library before any release and fails if any existing signature was altered or removed.

2. Package Structure & Directory Rules

In Go, the directory is the package, and the package is the directory.

Rules Governing Go Packages

1. One Package per Directory: All .go source files in a single directory must declare the exact same package name via package <name> at the top of the file. (The sole exception is test files declaring package <name>_test for external blackbox testing).
2. Multi-File Packages Without Forward Declarations: A package can be split across multiple files (client.go, server.go, transport.go, request.go, response.go in net/http). All files in that directory share a single package namespace:
   • Types, functions, constants, and variables declared in one file are visible across all other files in that directory without any import or header inclusion.
   • No forward declarations are needed. Order of files does not matter. The compiler treats all files in the directory as a single compilation unit.
3. Package Naming Conventions:
   • Short, concise, lowercase, single-word names (e.g., bytes, json, syntax, walk).
   • No snake_case (snake_case) or camelCase (camelCase).
   • Package names should describe what the package provides, not what it contains (e.g. http, not http_utilities).
   • Do not use generic names like util, common, or helper.

3. The internal/ Boundary Rule: Compiler-Enforced Encapsulation

One of Go's greatest architectural inventions for codebases of any size is the internal/ package rule.

The Problem in Traditional Languages (and Kyna)

In C++, Java, or early language runtimes, declaring a class or function public so that a sibling module can use it inadvertently makes it public to all users of the library or compiler. Over time, third parties depend on unstable internals, creating architectural debt and preventing refactoring.

The Go Solution

Go enforces a hard rule baked directly into the compiler and go build tool:

The internal/ Rule: An import path containing the element internal is only importable by code rooted in the parent directory of that internal directory.

Visualizing the Boundary

1src/
2├── net/
3│   ├── http/
4│   │   ├── internal/
5│   │   │   └── ascii/          <-- Can ONLY be imported by net/http and net/http/...
6│   │   ├── client.go
7│   │   └── server.go
8│   └── url/                    <-- CANNOT import net/http/internal/ascii (Hard Compile Error!)
9└── myapp/                      <-- CANNOT import net/http/internal/ascii

If net/url or an external application attempts:

1import "net/http/internal/ascii"

The compiler halts immediately:

1use of internal package net/http/internal/ascii not allowed

How the Go Compiler Implements This

In src/cmd/go/internal/load/pkg.go, the package loader checks:

1func (p *Package) checkInternal(importer *Package, pos token.Pos) error {
2    // Determine the directory enclosing the "internal" directory
3    // If importer is not within that parent directory, emit an error
4}

This simple rule allows Go's standard library and compiler teams to refactor thousands of internal files without risking broken external dependencies.

4. The cmd/ Architecture: Separation of Driver from Business Logic

Looking inside /Users/ahmedmansour/Documents/go/src/cmd, we see how Go organizes its own toolchain:

1src/cmd/
2├── compile/                # The Go compiler binary
3│   ├── doc.go              # Architecture documentation
4│   ├── main.go             # Tiny driver entry point (1,300 bytes)
5│   └── internal/           # 52 private packages forming the compiler pipeline
6│       ├── syntax/         # Lexer and parser
7│       ├── types2/         # Type checker
8│       ├── ir/             # Intermediate representation
9│       ├── ssa/            # Static single assignment optimizer
10│       └── ssagen/         # SSA code generator
11├── go/                     # The 'go' build & tool binary
12│   ├── main.go             # CLI command router
13│   └── internal/           # 45 isolated command packages
14│       ├── base/           # Shared Command interface and flag helpers
15│       ├── run/            # 'go run' implementation
16│       ├── test/           # 'go test' implementation
17│       ├── modcmd/         # 'go mod' implementation
18│       └── work/           # Build graph execution and caching
19├── vet/                    # Static code analyzer
20└── gofmt/                  # Source code formatter

The Driver Pattern

Notice that cmd/compile/main.go does not contain the compiler logic. It is merely a 50-line entry point that parses global flags and delegates to cmd/compile/internal/gc.Main().

Similarly, cmd/go/main.go defines a central command table:

1var commands = []*base.Command{
2    bug.CmdBug,
3    work.CmdBuild,
4    clean.CmdClean,
5    doc.CmdDoc,
6    envcmd.CmdEnv,
7    fix.CmdFix,
8    fmtcmd.CmdFmt,
9    generate.CmdGenerate,
10    modcmd.CmdMod,
11    run.CmdRun,
12    test.CmdTest,
13    vet.CmdVet,
14    version.CmdVersion,
15}

Every subcommand is a self-contained package under cmd/go/internal/<command>. A subcommand cannot pollute the namespace or state of another subcommand.

5. Import Graphs and DAG Enforcement

Go strictly prohibits circular dependencies between packages.

1graph LR
2    subgraph Valid_DAG ["Valid DAG (Compilation Succeeds)"]
3        A["Package A"] --> B["Package B"]
4        A --> C["Package C"]
5        B --> D["Package D"]
6        C --> D
7    end
8 
9    subgraph Invalid_Cycle ["Invalid Cycle (Hard Compile Error)"]
10        X["Package X"] --> Y["Package Y"]
11        Y --> Z["Package Z"]
12        Z --> X
13    end

Why Go Enforces a Strict DAG

1. Linear Compilation Time: Packages at the leaves of the DAG can be compiled in parallel. Once dependencies are compiled, downstream packages compile without back-tracking.
2. No Header Re-parsing: Package A only needs the compiled interface (export data) of Package B. It does not parse B's code.
3. Clear Conceptual Layering: Circular dependencies indicate poor design where two components have not properly separated their shared abstraction into a lower-level leaf package.

6. Actionable Blueprint for Kyna

Kyna's current architecture is based on deep CMake modules (kyna_lexing, kyna_syntax, kyna_parsing, kyna_typecheck, kyna_hir, kyna_mir, kyna_bytecode, kyna_vm). To achieve Go-level scalability for programs written in Kyna, we should adopt the following enhancements:

Recommendation 1: Multi-File Packages in Kyna

Currently, Kyna scripts are primarily executed as single files or individual module imports. We should introduce package directories where:

• Running ky run ./mypackage compiles all .kyna files in that directory as a single namespace.
• Types and functions in user.kyna are visible to service.kyna without manual import statements.

Recommendation 2: Semantic Enforcement of internal/ in Kyna Modules

In Kyna's module analyzer (compiler/kyna_typecheck/src/checkers/module_analyzer.cpp):

• Check the import path of any imported module.
• If the import path contains /internal/, verify that the importing file's directory is a descendant of the directory containing internal/.
• If not, emit diagnostic KMOD1004: use of internal module '<path>' not allowed.

Recommendation 3: Align CLI Commands with cmd/go

Refactor tools/kyna_cli so that each subcommand (check, run, build, hir, mir, bytecode, fmt) is an independent module with its own argument parsing and execution handler, registered through a central table rather than nested branching logic.