BTG Labs
We collaborate with researchers and students through BTG Labs in two major areas of AI: harness engineering and AI model adaptation for edge AI.
Harness engineering
A model alone does not make a reliable AI system; the harness around it does, i.e., how the system remembers, which tools it can use and in what order, what it is not allowed to do, and how its work is evaluated. Therefore, we study the design of these harnesses, so that an AI system works reliably in a real workflow and remains accountable to the people who use it.
Our work in harness engineering draws on the following methods:
Memory architecture
The management and compaction of context; long-term memory for episodic, semantic and procedural knowledge; retrieval-augmented generation; and hierarchical, virtual management of context.
Tool orchestration
Function and tool calling; loops of reasoning and acting; planning and the decomposition of tasks; the coordination of multiple agents; and open protocols for tools and for communication between agents.
Guardrails
The validation of inputs and outputs; structured outputs; least-privilege permissions; human approval for consequential actions; sandboxed execution; defenses against prompt injection; and audit logs.
Evaluation
Task-based test suites and benchmarks; regression tests; model-graded evaluation, calibrated against human judgment; red-teaming; the tracing of agent runs; and the measurement of cost and latency.
Model adaptation for edge AI
Many problems need a model that runs close to the data, i.e., on a local device or an edge server, within its memory, power and cost constraints. Therefore, we study how a large model can be adapted into a smaller one that keeps the quality needed for a defined task.
Our work in model adaptation draws on the following methods:
Parameter-efficient fine-tuning
Low-rank adaptation (LoRA) and its quantized variant, QLoRA, which adapt a model to a task by training a small number of added parameters.
Preference optimization
Direct preference optimization (DPO), which aligns a model with human preferences without a separate reward model.
Quantization
Post-training quantization, such as GPTQ and activation-aware weight quantization (AWQ), which reduce the precision of model weights, e.g., to 4 bits, and therefore the memory and the cost of inference.
Pruning
One-shot pruning methods, such as SparseGPT and Wanda, which remove the weights that contribute least to the output of a model.
Knowledge distillation
The training of a smaller student model to reproduce the outputs of a larger teacher model.
Efficient inference
Speculative decoding, in which a small draft model proposes tokens that the larger model verifies; and local runtimes for devices and edge servers.
Collaboration
We research these focus areas through iLyceum16 with our current partner institutions, while actively seeking new collaborations to explore the domain of AI. Contact us to schedule a discovery call and explore collaborative research opportunities.