The problem
Service robots must coordinate navigation, perception, manipulation, and human instructions in real time. Building those pieces separately produces a robot that can perform isolated demos but struggles with complete tasks.
Loading page
Preparing the content for you.
One robot, one planning loop.
An autonomous AI service robot combining ROS 2 navigation, object detection, manipulation, voice-command task planning, and a browser-based 3D dashboard. It connects robot-level execution with FIN as a high-level planner for multi-step objectives.
Overview
Kora brings perception, navigation, manipulation, and voice interaction into one robot system. ROS 2 provides the communication layer between components, Nav2 handles route planning, and MoveIt 2 plans arm motion for manipulation tasks.
Camera input feeds object detection through YOLOv8 and OpenCV, while voice commands are interpreted into ordered tasks. A browser-based 3D dashboard makes robot state visible to an operator, and FIN can act as a high-level planner for multi-step objectives.
Service robots must coordinate navigation, perception, manipulation, and human instructions in real time. Building those pieces separately produces a robot that can perform isolated demos but struggles with complete tasks.
Kora integrates the robot stack around ROS 2, uses FIN for multi-step planning, and exposes state through a browser-based 3D dashboard so an operator can follow and intervene.
Architecture
A high-level conceptual view of the main stages in this system.
Spoken requests are captured and interpreted into an intended task.
FIN decomposes multi-step objectives into ordered robot tasks.
Nav2 plans routes while YOLOv8 and OpenCV detect relevant objects.
MoveIt 2 plans arm motion, and the outcome is reported to the operator dashboard.
Capabilities that are documented as part of the current implementation.
Plans and follows routes through mapped environments with ROS 2 and Nav2.
Detects and locates objects from camera input using YOLOv8.
Plans and executes arm motion for pick-and-place style tasks.
Translates spoken requests into ordered robot tasks.
Shows robot state and activity in a browser-based 3D view.
Uses FIN as a high-level planner for multi-step objectives.
These items are planned and are not available yet. They are listed separately so completed functionality is never confused with future work.
Structured testing on the physical robot platform.
Navigation across larger and changing environments.
Formal review of operating limits and human-override behavior.
Technology
The tools and technologies associated with the implementation.
Perception, planning, and control code.
Robotics middleware connecting robot components.
Navigation stack for autonomous movement.
Motion planning for arm manipulation.
Real-time object detection from camera input.
Image processing and vision pipelines.
Local LLM runtime for voice-command understanding.
Task planning and natural-language interpretation.
Browser-based 3D visualization of robot state.
Simulation environment for testing behaviors.
Challenges
The engineering constraints that shaped the design.
Perception and localization drift in real environments, so navigation must tolerate imperfect input.
Planning, detection, and motion must complete within a window that still feels responsive.
A robot acting in a shared space needs clear operating limits and a reliable human override.
Outcomes
Qualitative outcomes from the design. Measured results are only reported where they have been formally evaluated.
Links
Explore the project through the destinations that exist. Missing destinations are labelled instead of being replaced with a placeholder URL.
No public link has been provided for this project yet.
No public link has been provided for this project yet.
No public link has been provided for this project yet.
No public link has been provided for this project yet.
0 of 4 external links are currently available. Unavailable links are shown deliberately rather than pointing to a placeholder destination.
Keep exploring
Browse the full portfolio or start a conversation about this project.