Open-source Contribution

Sep 10, 2024

Open-source Projects

Hopter

Hopter is a Rust-based embedded operating system designed to enable memory-safe, robust, and responsive embedded applications. Applications built with Hopter require a customized Rust compiler that generates instrumented code to cooperate with the kernel. The compiler helps enhance memory safety and system reliability without altering standard Rust syntax. Hopter can interoperate with community-driven open-source HAL libraries.

  • Memory safety: Hopter prevents stack overflows alongside other memory safety guarantees provided by Rust. The customized compiler generates a function prologue (initial instructions executed at function entry) that checks available stack space before executing the function body.
  • Robustness: Hopter gracefully recovers applications from Rust panics by running a stack unwinder, which cleans up panicked tasks or IRQ handlers. Hopter supports restartable tasks that automatically restarts execution from the entry closure after a panic. Additionally, Hopter converts stack space exhaustion into a panic and allows recovery using the same mechanism.
  • Responsiveness: Hopter features zero-latency IRQ handling. The kernel never disables IRQs, even within traditionally critical sections, ensuring immediate handling of pending IRQs. Hopter achieves it through a novel internal synchronization primitive called soft-locks, which resolves race conditions between IRQs and tasks without disabling IRQs.

While Hopter strongly encourages using safe Rust, it also permits the cautious use of unsafe Rust code. Therefore, Hopter expects benign but not malicious applications, and its threat model is similar to that assumed by FreeRTOS.

Pie

Pie is a programmable LLM serving system designed for flexibility and efficiency. Pie decomposes the traditional generation loop into fine-grained service handlers exposed via APIs and delegates control of the generation process to user-provided programs, called inferlets. This enables applications to implement new KV caching strategies and generation logic, and to seamlessly integrate computation and I/O—entirely within the application, without modifying the serving system. Pie executes inferlets using WebAssembly, benefiting from its lightweight sandboxing. To realize this programmable model, Pie employs a layered architecture designed for flexibility and efficiency.

  • Application layer: Execute user inferlets and accept API calls.
  • Control layer: Orchestrates API calls, manage resources, virtualizing accesses, and intelligently batch API calls destined for the GPU.
  • Inference layer: Execute batched operations and translate them into low-level computations (e.g., GPU kernel invocations) using specialized handlers for different model operations.

The evaluation shows that Pie matches state-of-the-art performance on standard tasks (3-12% latency overhead) while significantly improving latency and throughput (1.3x-3.4x higher) on agentic workflows by enabling application-specific optimizations.

Other Contributions

stm32f4xx-hal

A Rust embedded-hal HAL for all MCUs in the STM32F4 family.

Features and bug fix patches merged:

LLVM

A collection of modular and reusable compiler and toolchain technologies.

Features and bug fix patches merged:

owning-ref

A library for creating references that carry their owner with them.

Feature submitted as pull request: