May 15, 2026
3we: AI-First Python API for Mobile Robot Navigation (Open Source, $300 BOM)

Hi everyone,

I’d like to share **3we** — an open-source platform I’ve been building that provides an AI-First Python API on top of ROS2/Nav2, targeting Embodied AI researchers who want to focus on algorithms rather than ROS2 infrastructure.

## The Problem

AI researchers (especially those working with VLMs/VLAs) often want to deploy models on real robots but face:

- Steep ROS2 learning curve (launch files, topics, services, actions)

- No clean path from simulation to hardware

- Existing platforms are either too expensive (TurtleBot 4: $1,200+) or simulation-only (Habitat, Isaac Lab)

## What 3we Does

```python

from threewe import Robot

async with Robot(backend=“gazebo”) as robot:

image = robot.get_camera_image()     # (H,W,3) uint8

scan = robot.get_lidar_scan()        # LaserScan

**await** robot.move_to(x=5.0, y=3.0)   # Nav2 under the hood

```

Change `backend=“gazebo”` to `backend=“real”` — same code runs on physical hardware. The ROS2/Nav2 stack is fully transparent to the user.

**Four backends with identical API:**

- `mock` — zero-dependency 2D kinematics (no ROS2 needed, runs anywhere)

- `gazebo` — Gazebo Harmonic with full physics

- `isaac_sim` — NVIDIA Isaac Sim for GPU-accelerated RL training

- `real` — Physical hardware via ROS2 topics

## Architecture

```

┌─────────────────────────────────────────┐

│ AI-First Python API (user layer) │ ← Researchers write code here

├─────────────────────────────────────────┤

│ 3we-core (middleware) │ ← Backend dispatch, sensor fusion

├─────────────────────────────────────────┤

│ ROS2 / micro-ROS (infrastructure) │ ← Transparent to users

│ Nav2, slam_toolbox, ESP32 drivers │

└─────────────────────────────────────────┘

```

This is NOT a replacement for ROS2 — it’s a layer on top that makes ROS2 accessible to ML researchers while preserving full ROS2 compatibility for roboticists who want low-level access.

## VLM-Controlled Navigation

The killer feature for AI researchers: GPT-4o (or any OpenAI-compatible VLM) can directly control the robot through natural language:

```python

async with Robot(backend=“gazebo”) as robot:

result = **await** robot.execute_instruction(

    "find the red bottle and stop near it"

)

print(f"Success: {result.success}")

```

Internally this runs a perception-action loop: capture image → send to VLM → parse JSON action → execute → repeat until done. Works with GPT-4o, Qwen-VL, or local LLaVA.

## Hardware ($300 BOM)

Fully open reference hardware under CERN-OHL-P v2:

| Component | Selection |

|-----------|-----------|

| Compute | Raspberry Pi 5 (8GB) |

| AI Accelerator | Hailo-8L (13 TOPS) |

| MCU | ESP32-S3 + micro-ROS |

| LiDAR | LD06 (360°, 2D) |

| IMU | BNO055 (9-axis) |

| Drive | 4× N20 motors + Mecanum wheels + DRV8833 |

| Safety | Dual-channel relay (ISO 13850 E-stop) |

KiCad 8 PCB files, DXF mechanical drawings, and assembly docs all included.

## ROS2 Integration Details

For ROS2 developers who want to know what’s under the hood:

- **Navigation**: Nav2 with DWB planner, parameters tuned for mecanum kinematics

- **SLAM**: slam_toolbox (online async) with LD06

- **MCU bridge**: micro-ROS on ESP32-S3 via USB-C serial transport

- **Sensor fusion**: robot_localization EKF (IMU + wheel odometry)

- **Launch**: Composable nodes, configurable via YAML profiles

You can always drop down to raw ROS2 topics/services if needed — the Python API doesn’t hide or lock you out.

## Benchmark Suite

7 standardized scenes with reproducible baselines:

```bash

threewe benchmark run --task pointnav --scene office_v2 --episodes 100

```

Gymnasium-compatible environments for RL:

```python

import gymnasium as gym

env = gym.make(“3we/Navigation-v1”, scene=“office_v2”, backend=“mock”)

```

## Demo

![Navigation Demo](https://img.xuexiao.eu.org/1778836342754-19b737g.gif)

Autonomous point-to-point navigation in office_v2 scene with 360° LiDAR visualization.

## Links

- **GitHub**: GitHub - telleroutlook/3we-robot-platform: 3we Universal Modular Mobile Robot Platform — Open-source firmware, ROS2, hardware, and SDK · GitHub

- **Documentation**: https://3we.org

- **Paper**: Paper - 3we

- **PyPI**: `pip install threewe`

Feedback welcome — especially from Nav2 users on whether the API abstraction makes sense, and from AI researchers on what’s missing for their workflows.

Software: Apache 2.0 | Hardware: CERN-OHL-P v2 | Docs: CC-BY-SA 4.0

3 posts - 3 participants

Read full topic

by telleroutlook on May 15, 2026 03:47 PM

What patterns of logs or warnings should an automated bridge promote to structured faults in ROS 2?

We have been working on ros2_medkit, an Apache 2.0 fault aggregation gateway for ROS 2 that follows the SOVD model (ISO 17978-3). All of it lives in GitHub - selfpatch/ros2_medkit: ros2_medkit - diagnostics gateway for ROS 2 robots. Faults, live data, operations, scripts, locking, triggers, and OTA updates via REST API. No SSH, no custom tooling. · GitHub

Two integration paths today:

  1. /diagnostics topic - drop-in, no code changes on the publisher side. Works for any package already using diagnostic_updater.

  2. Native FaultReporter instrumentation - each failure surface emits a structured fault code directly. We tried this on a manymove fork to see how invasive it is to add per-action-node fault reporting. PR is here for reference: Feat/medkit integration by mfaferek93 · Pull Request #1 · selfpatch/manymove · GitHub - the integration itself was small (one mixin + a fault-codes header), but that fork is a fairly clean codebase. Most production stacks have a much messier RCLCPP_ERROR / RCLCPP_WARN history that nobody is going to retroactively convert.

Native FaultReporter is the right answer when you control the codebase end-to-end - structured codes from day zero, lowest friction long-term. The painful case is the long tail of existing ROS 2 packages that already work fine and never emitted /diagnostics. For those, the drop-in bridge has nothing to subscribe to, and asking maintainers to instrument every node won’t happen. If the goal is to make structured diagnostics adoptable across the ecosystem, plug-and-play needs to mean more than “use /diagnostics”.

That gap is what we want to validate with you.

We are considering a third path: a logs-to-faults bridge that watches /rosout (or arbitrary log streams) and promotes selected patterns to structured fault events, with configurable rules (severity mapping, dedup, rate limiting). Goal: a team can adopt structured diagnostics without touching their existing code. If it works out, it ships in the same open repo as the rest of medkit.

Three questions where your experience would help more than ours:

  • What log patterns in your stack would you actually want auto-promoted to structured faults? (specific examples > taxonomies)

  • What blocks your team from using /diagnostics more widely today?

  • For a logs-to-faults bridge to be useful and not noisy, what would have to be true? (rules engine, allowlist-only, ML, something else?)

Curious what others have tried, especially on the failure-modes side.

How do you currently surface fault/error state from your ROS 2 nodes?
How do you currently surface fault/error state from your ROS 2 nodes?
  • /diagnostics (DiagnosticArray)
  • Custom error/event topics
  • Logs (RCLCPP_ERROR / RCLCPP_WARN)
  • Action results / service error codes
  • Behavior tree / lifecycle state changes
  • Tracing / OpenTelemetry
  • No consistent pattern yet

Click to view the poll.

4 posts - 3 participants

Read full topic

by Michal_Faferek on May 15, 2026 10:04 AM

May 14, 2026
The accountability gap in ROS2: where does "why did the robot do that?" get answered?

A question I keep running into and don’t have a clean answer for: when a
ROS2-based autonomous system makes a consequential decision — a mobile
robot reroutes around a person, an arm stops mid-motion, a drone aborts —
we can answer what it did. ros2 bag captures the topics. But why it
did that, in a form a safety officer, an insurance adjuster, or a regulator
can read, is almost always reconstructed after the fact, by hand.

Four specific observations, curious where I’m wrong:

1. Rule provenance is invisible. When a BehaviorTree node fires, we
log the node, not the human-authored policy that made the node legal. No
first-class link from “robot stopped” to “rule §3.2 of safety policy v4
triggered.”

2. Guardrails are one-way safety, not auditable downgrades. Most ROS2
safety layers I’ve seen are kill switches or velocity caps. They prevent
harm but produce no signed record of “planner wanted X, guardrail
downgraded to Y, here’s the chain.”

3. LLM-in-the-loop adds a new failure mode. With VLA stacks plugging
into task planning, the “why” gets harder. Did the model suggest the
action? Was it followed, overridden, sanitized? I don’t see standard hooks
for any of this in the stack.

4. EU AI Act Article 12 and 14 are now in force for high-risk autonomous
systems.
Most teams I talk to plan to handle “logging” and “human
oversight” with ros2 bag plus a spreadsheet. That will not survive a
regulator audit, and CE marking deadlines for some categories hit in 2027.

Three questions for people deeper in this than me:

  • Is there an active REP or working group on decision provenance that I
    missed? I found scattered threads, no spec.
  • For Nav2 + BehaviorTree.CPP teams: how do you currently answer “why
    did the robot decide that?” for non-engineer stakeholders?
  • Has anyone added cryptographic signing to the rosbag pipeline, or is
    everyone trusting the filesystem and timestamps?

I’ve been building an opinionated implementation of some of this — rule
provenance, signed audit chain, guardrail-downgrade-only pattern, LLM
sanitization — outside of ROS2, and I’m trying to figure out if the pieces
that generalize are worth porting and open-sourcing.

If this resonates, drop a reply or DM. Looking for both “you’re missing
existing work X” and “yes this is broken in our deployment, here’s how.”

3 posts - 2 participants

Read full topic

by altunbulakemre75 on May 14, 2026 07:35 PM

[Open Source] rviz_2d_plot_plugin: Live 2D Plotting Inside RViz 2

Hi everyone,

I’m happy to share an open-source RViz 2 plugin I have been working on:

rviz_2d_plot_plugin

The plugin provides live 2D plotting directly inside RViz 2 as a screen-space overlay. The goal is to make it easier to monitor ROS 2 topic data, controller signals, odometry-related values, diagnostics, and other runtime signals without leaving the RViz environment.

Some of the current features include:

  • Runtime discovery of plottable ROS 2 topic fields
  • Time-series plotting
  • XY plotting
  • Multi-series plotting from different topics
  • Reference lines, limits, setpoints, and tolerance bands
  • Axis control, auto-scaling, grid, legend, and styling options
  • QoS configuration
  • Pause, clear, and history preservation

The plugin is currently supported and tested on ROS 2 Humble. Support for additional ROS 2 distributions is planned and will be released soon.

The project is released under the MIT license.

I would be happy to receive feedback from the ROS community, especially regarding:

  • API/design improvements
  • Compatibility with other ROS 2 distributions
  • Useful plotting features for robotics debugging workflows
  • Packaging and release suggestions

Repository:

Thanks, and I hope this can be useful for others working with RViz 2 and ROS 2 system visualization.

3 posts - 2 participants

Read full topic

by Abdelrahman on May 14, 2026 03:58 PM

How long did your first cross-device ROS2 setup take?

Setting up ROS2 across multiple devices (different boards, distros, DDS config) for the first time — how many hours or days did it take before you had nodes talking reliably?

Specifically curious about:

  • Device combo (RPi + Jetson, x86 + ARM, etc.)

  • What broke (DDS discovery, distro mismatch, network config?)

  • Rough time lost before it worked

Building a scaffolding tool and want real data, not estimates.

1 post - 1 participant

Read full topic

by ambigram on May 14, 2026 03:58 PM

MBF - My quadruped robot dog

Building a quadruped robot dog has been a personal goal of mine ever since I started engineering. Over the past months (or even years), I’ve been working on MBF, an open-source quadruped robotics platform designed and built entirely from the ground up using affordable and accessible hardware.

This project has pushed me to learn across multiple engineering domains simultaneously — from mechanical design and fabrication to embedded communication, robotics middleware, and locomotion software architecture.

Everything on the robot was independently designed and integrated by myself, including:

  1. Mechanical design and CAD
  2. 3D-printed structural components
  3. Parts sourcing and assembly
  4. Electrical wiring and CAN bus communication
  5. ROS2-based software architecture and control stack
    **
    Current Hardware Highlights:**
  6. Affordable quadruped platform with predominantly 3D-printed components
  7. GIM6010-8 planetary drive actuators running FOC control over CAN bus
  8. Design-for-Assembly (DFA) considerations such as standardized screw sizing and modular assembly layout
  9. Entire robotics stack running on a single Raspberry Pi 4 without additional microcontrollers (selectable inference using either ONNX or Torch C++)

Current Software Stack:

  1. Custom ROS2 joint impedance controller
  2. Custom ros2_control hardware interface over CAN bus
  3. Integration with the CHAMP framework
  4. Extensible reinforcement learning inference node
  5. ROS2/Gazebo simulation workflow for sim-to-sim locomotion deployment withe easy switch to real hardware with a simple argument change

One thing this project taught me is that modern robotics is deeply built on open-source collaboration. A lot of the knowledge, frameworks, and tools used throughout MBF came from developers and researchers who chose to make their work publicly accessible.

While I’m proud of the progress made so far, this project is also a reminder that meaningful robotics development has become more accessible than ever. With enough dedication and willingness to learn, it’s genuinely possible for individuals to build complex systems today thanks to the open-source community. What I am doing here today is to merely return that favor back.

The robot is still actively being developed, but seeing the full software stack communicate reliably with custom hardware has been incredibly rewarding so far.

For more info, please check:
github.com/adwng/mbf_ros2

1 post - 1 participant

Read full topic

by adwng on May 14, 2026 03:57 PM

Polka - Your everything pointcloud node Release v0.2

Polka was intended to be a low latency pointcloud merger which also publishes laser scans, provides you with granular control over filtering, even deskewing, here are some features and bug fixes that have been implemented in the latest release.

New Features

  • Per-source IMU topic override: each LiDAR source can specify its own imu_topic for robots with multiple IMUs on different body segments. Falls back to global motion_compensation.imu_topic when unset.

  • Gravity subtraction in deskew: linear acceleration corrected by removing gravity using IMU orientation, improving motion compensation accuracy.

  • Configurable output QoS: full QoS control (reliability, durability, history depth, liveliness, deadline, lifespan) via outputs.cloud.qos / outputs.scan.qos parameters.

  • Multi-LiDAR deskew example config in config/example_params.yaml.

Bug Fixes

  • Fix IMU-to-sensor frame rotation in deskew: angular velocity and acceleration now rotated from IMU frame into each sensor’s frame via TF. Previously only sensors aligned with the IMU got correct deskewing. (fixes #3)

  • Fix degenerate quaternion fallthrough: zeroes acceleration instead of passing raw gravity through when IMU orientation is degenerate.

  • Fix thread safety in SourceAdapter: mutex protection for frame_id/timestamp during concurrent deskewing.

  • Fix stale IMU timestamps: removed dead average_imu(), simplified to atomic snapshot pattern.

  • Fix duplicate missing-intensity warning.

  • Add CUDA error checking in merge engine kernels.

Improvements

1 post - 1 participant

Read full topic

by Panav on May 14, 2026 12:03 AM

May 13, 2026
jros2 Cellphone Sensor Bridge: Native ROS 2 nodes on Android

Hey everyone,

I was recently reading a book on ROS 2 and went looking for ros2_java. After finding out that project essentially died, I discovered jros2 and decided to just build an app to test its limits.

The result is the jros2 Cellphone Sensor Bridge. It is an Android (Jetpack Compose) application that exports live phone sensor telemetry to ROS 2 using standard sensor_msgs, std_msgs message types, and custom mobile_sensor_msgs definitions over the IHMC jros2-android stack (Fast DDS + JavaCPP JNI).

How it works:

  • It initializes a fully compliant ROS 2 Node (phone_sensor_node) directly on your physical Android smartphone.

  • It streams real-time, high-frequency telemetry from 11+ hardware/software sensors and input devices (IMU, Magnetometer, GPS, Touch Screen, Dual Cameras, etc.).

  • It employs an Android multicast Wi-Fi lock to ensure robust, real-time discovery of DDS participants directly on the local network without intermediary servers.

Here is a quick demo showing the phone acting as a dual-joystick controller and streaming data: https://www.youtube.com/watch?v=skNQdbO8yrw

Repo, architecture details, and the v1.1.0 APK are available here: https://github.com/SinfonIAUniandes/jros2_cellphone_interface/tree/main

1 post - 1 participant

Read full topic

by thegrayguy on May 13, 2026 10:40 PM

Where do robot arm specs usually break down in real deployments?

I’m interested in examples from people who have deployed, integrated, or specified robot arms in real environments.

When a robot arm looks good on paper but struggles in the actual cell, what is usually the limiting factor?

Some examples I’m curious about:

  • payload-at-reach

  • wrist torque

  • EOAT / tooling weight

  • stiffness or compliance

  • thermal limits

  • continuous duty cycle

  • safe speed vs. required cycle time

  • dust, water, shock, or harsh environments

  • integration cost / cell complexity

Have you seen projects where the arm was technically close, but the system had to be oversized, slowed down, redesigned, or abandoned?

I’m especially interested in concrete examples:

  1. What was the task?

  2. What constraint showed up?

  3. What did the team do instead?

Looking for field lessons, not brand debates.

1 post - 1 participant

Read full topic

by rac_ludvig on May 13, 2026 08:15 PM

GitHub & Code Hosting

I just want to open a discussion here regarding github.

These days, github’s availability has been in the gutter. I have been finding it difficult to get any work done whenever I need to do things as simple as read the code to see how certain things are implemented, as github just won’t load pages sometimes

Lately I haven’t been doing much maintainer work, but I can imagine that the experience is even worse for anyone who is.

I am genuinely considering hosting my own code forge just so I can easily mirror repos to browse them.

I think it would be a good idea to start at least looking at migration options for alternatives to github.
I would personally recommend going with either codeberg or hosting a forgejo instance (the software codeberg is based on), however gitlab is also a viable option.

Obviously, I don’t expect anything to happen soon, nor do I even expect that my post will necessarily incite the change, as this would be a massive change and requires a lot of thought & work before it can happen, I just want to bring it up to maybe get some people thinking about it

4 posts - 2 participants

Read full topic

by solonovamax on May 13, 2026 01:49 AM

May 12, 2026
Preparing for State of Cloud Robotics Survey | Cloud Robotics WG Meeting 2026-05-18

Please come and join us for this coming meeting at Mon, May 18, 2026 4:00 PM UTCMon, May 18, 2026 5:00 PM UTC, where we plan to write the survey questions for a new State of Cloud Robotics survey. The last survey was in 2024 (see https://cloudroboticshub.github.io/survey), and we’d like to refresh the results as of this year. Therefore, the meeting will be going over previous results and updating the questions ready to release.

Last session, we continued our Transitive Robotics tryout by writing a custom capability. We were able to get a working setup by the end of the meeting and create custom code running using the Transitive Robotics framework. If you’re interested in watching, the meeting recording is available on YouTube.

The meeting link for next meeting is here, and you can sign up to our calendar or our Google Group for meeting notifications or keep an eye on the Cloud Robotics Hub.

Hopefully we will see you there!

1 post - 1 participant

Read full topic

by mikelikesrobots on May 12, 2026 03:55 PM

Official state and timeline of ROS2 Bazel build

Hi all

This is a follow up question to Will intrinsic supports ros2 on bazel with bzlmod? .

There has been recent work by Intrinsic to create a ROS2 Bazel build ( GitHub - intrinsic-opensource/ros-central-registry: ROS packages as Bazel modules · GitHub ).

Also, the “Open Robotics Technology Strategy for 2026” ( Open Robotics Technology Strategy 2026 — Open Robotics ) explicitly states

supporting new infrastructure tools such as Bazel

However, development seems to have stopped since March, and there are no updates on Will intrinsic supports ros2 on bazel with bzlmod? or the tracking issue ( Bazel integration in ROS · Issue #1726 · ros2/ros2 · GitHub ).

So, my question, is there a plan for Bazel to be officially supported, and if yes, is there a timeline for it?

Best

Jonathan

2 posts - 2 participants

Read full topic

by jm7s on May 12, 2026 03:55 PM

Handling high hardware/communication latency (~360ms)

Hello,

I am developing an industrial Autonomous Mobile Robot (AMR) using ROS 2 Humble and a B&R PLC (communicating via OPC UA PubSub/UDP). I have measured a significant round-trip latency between ROS 2 cmd_vel and the actual wheel odometry feedback.

Hardware Setup:

Processor: Intel i5-14500 (PC-side)

PLC: B&R (Automation Studio)

Communication: OPC UA (UDP)

Sensors: SICK Safety Lidars (connected directly to ROS 2)

The Problem

Using a custom latency logger, I’ve measured a ~364ms delay from the moment cmd_vel is published until the PLC-based odometry reflects the physical movement. This includes:

Network serialization (OPC UA/UDP)


Mechanical inertia and electromagnetic brake release.

When using CLOSED_LOOP feedback in the velocity_smoother, the robot exhibits significant jitter and oscillations because the smoother reacts to “old” odometry data.
Current Workaround

Questions

Is a ~360ms round-trip latency considered "within acceptable limits" for heavy industrial AMRs in the Nav2 ecosystem?

Is the "negative timestamping" approach (now - delay) considered a safe practice for temporal synchronization between high-speed Lidars and high-latency PLC Odometry?

1 post - 1 participant

Read full topic

by storyco on May 12, 2026 03:54 PM

[Demo] Bridging ROS 2 and VDA5050-style Fleet Telemetry: Command Integrity via NARH

Hi everyone,

Following the recent discussion around /cmd_vel, timeouts, stamped commands, and ros2_control, I realized there is a useful missing layer between vehicle-internal command execution and fleet-level observability.

I’ve updated ros2_kinematic_guard to explore that layer.

The project is no longer only a local /cmd_vel guard. It now also exposes command-execution-integrity telemetry.

The core idea is:

ROS 2 executes robot behavior.
Fleet systems coordinate robot behavior.
But something needs to report whether the recent command window is still trustworthy.

What changed

I added a reporter_node.py that translates the NARH Guard state into:

  1. ROS-native diagnostics
/diagnostics
diagnostic_msgs/DiagnosticArray
  1. VDA5050-style fleet telemetry
/command_integrity/vda5050_state
  1. Compact fleet-readable summary
/command_integrity/summary

Example output during a Wi-Fi collapse stress test

state=RESYNCING latency=CRITICAL R_NAR=5.478 vehicle_response=RESYNC_REQUIRED fleet_action=HOLD_NEW_ORDERS
state=RESYNCING latency=CRITICAL R_NAR=1412.078 vehicle_response=RESYNC_REQUIRED fleet_action=HOLD_NEW_ORDERS
state=RECOVERED latency=NORMAL R_NAR=0.000 vehicle_response=NONE fleet_action=NONE

This means the local command stream is not yet trusted again, and a fleet/orchestration layer could avoid assigning new orders, intersection-heavy tasks, or timing-critical maneuvers until the vehicle recovers.

Why this is different from a timeout

Timeouts answer:

Did a command arrive recently?

The NARH Guard asks:

Is the recent command/feedback window still trustworthy?

This is not meant to replace ros2_control controller-side mechanisms such as stamped references, speed limiting, timeouts, or smooth stop behavior.

Instead, the goal is to expose a higher-level signal that can be consumed by:

  • ROS diagnostics
  • bag / MCAP post-analysis
  • fleet managers
  • VDA5050-style orchestration layers
  • heterogeneous robots that do not all run the same controller stack
Failure Mode Heartbeat / Timeout NARH Kinematic Guard
Packet loss Detects silence only Detects silence + local predictive braking
Stale command Often ignored Detected via timing + kinematic drift
Burst command Hidden by buffer Detected via residual spikes
Replay/Out-of-order Hard to catch Caught via phase-continuity check
Command/Odom Conflict Blind to physics Directly measured consistency
Recovery Logic Binary (On/Off) State-aware resync gate
Fleet Visibility Internal failure only Structured telemetry (ROS/VDA5050)

Repository:

I’d be very interested in feedback from anyone working on VDA 5050 bridges, mixed-fleet integration, ROS diagnostics, or fleet-level degraded-mode reporting.

Does a quantitative command-integrity signal like this fit any real orchestration or post-incident analysis needs you have seen?

1 post - 1 participant

Read full topic

by zc_Liu on May 12, 2026 03:08 PM

How to Build a Robot Arm IK Solver in ROS2 | NERO Arm Parametric Inverse Kinematics

Complete Tutorial on Nero Arm Angle Parametric IK

Reference paper from Tsinghua University Paper —— Inverse kinematic optimization for 7-DoF serial manipulators with joint limits


Part 1. Overview

This document provides a complete mathematical tutorial on parameterized inverse kinematics (IK) for the NERO 7-DoF robotic arm.

The content mainly corresponds to:

  • Tsinghua University paper: Inverse Kinematics Solution for 7-DoF Robotic Arms with Joint Limit Optimization
  • Implementation: ik_solver.py
  • ROS2 real-time runtime node: ik_joint_state_publisher.py

Part 2. Algorithmic Background and Core Concepts

2.1 Fundamental Characteristics of 7-DoF Redundant Robot Arms

A 7-DoF robotic arm with an S-R-S configuration (Spherical Shoulder – Revolute Elbow – Spherical Wrist) introduces one additional redundant degree of freedom compared with a conventional 6-DoF manipulator.

This means that:

  • When the end-effector pose is fixed, the joint configuration may still have infinitely many solutions, and the arm can still move internally while keeping the end-effector stationary.

This type of motion, where the end-effector remains fixed while the robot reconfigures itself, is referred to as null-space motion.

Redundancy provides several important advantages:

  1. Joint limit avoidance
  2. Obstacle avoidance
  3. Elbow posture optimization
  4. Smoother trajectory generation

2.2 Elbow Angle Parameterization (Core Contribution of the Paper)

The core idea of the paper is:

Use a single parameter to represent the entire redundant degree of freedom —— this paramter are called elbow angle \psi (theta \theta in the code implementation).

Geometric Definition of the Elbow Angle

When the end-effector pose is fixed, both points S and point W are fixed in space.

The elbow point E then traces a circle in 3D space.
The rotational angle within the plane of this circle is defined as the elbow angle \psi.

  • S: Shoulder center (intersection point of the first 3 joint axes)
  • E: Elbow center (location of Joint 4)
  • W: Wrist center (intersection point of the last 3 joint axes)
  • Points S–E–W form a triangle with fixed side lengths
  • The elboww angle \psi determines the position of point E on the circle.

In one sentence:

  • \psi → elbow posture changes → joint angles change → end-effector remains unchanged

2.3 Differences Between This Method and Traditional Numerical IK Solvers

Comparison Aspect Numerical Iterative Methods (Jacobian / Damped Least Squares) Elbow-Angle Parameterized Analytical IK
Solution Strategy Iterative convergence, dependent on initialization Geometric derivation with closed-form solution
Computational Speed Slow (ms–10 ms) Extremely fast (<0.1 ms)
Convergence May fail to converge; susceptible to local minima Globally optimal and divergence-free
Joint Limit Handling Passive constraint handling; easy to violate limits Active feasible-region control; never exceeds limits
Null-Space Control Requires projection operators; prone to instability Direct control through \psi; naturally stable

Part 3.Complete Algorithm Workflow

The entire algorithm consists of four core stages:

  1. Extract S, W, and θ_4 from the target pose.
  2. Compute the elbow point E from the elbow angle ψ, and analytically solve q_1q_3 and q_5q_7
  3. Compute the feasible region of the elbow angle under all joint-limit constraints
  4. Optimize the elbow angle within the feasible region using a weighted quadratic objective function

The following sections correspond directly to the equations in the paper and the implementation in code.


3.1 Step 1: Solving for S, W, and \theta_4 from the Target Pose

Theory from the Paper

Given the end-effector pose T_{07}, we first solve for:

  • Shoulder point S
  • Wrist point W (obtained by offsetting the end-effector frame backward by d_6)
  • Elbow joint angle \theta_4 (uniquely determined from the S–E–W triangle using the law of cosines)
  • As illustrated in the figure, points S, W, E, and D

Law of Cosines

cos \theta_4 = \frac{||SW||^2-||SE||^2-||EW||^2}{2||SE|| ||EW||}

Code Implementation: _compute_swe_from_target

def _compute_swe_from_target(T07: np.ndarray, p: NeroParams) -> Tuple[np.ndarray, np.ndarray, Optional[float], np.ndarray]:
    R = T07[:3, :3]
    p_target = T07[:3, 3]
    z7 = R[:, 2]
    d6 = float(p.d_i[6])
    d1 = float(p.d_i[0])

    # End-effector flange center
    O7 = p_target - p.post_transform_d8 * z7
    # Wrist center W: offset backward from the flange by d6
    W = O7 - d6 * z7
    # Shoulder center S: fixed at height d1 above the base
    S = np.array([0.0, 0.0, d1], dtype=float)

    # Solve the absolute value of θ4 using the law of cosines
    q4_abs = _solve_theta4_from_triangle(S, W, p)

    # Unit vector from shoulder to wrist
    v_sw = W - S
    n_sw = np.linalg.norm(v_sw)
    u_sw = v_sw / n_sw if n_sw > 1e-12 else np.array([0.0, 0.0, 1.0])

    return S, W, q4_abs, u_sw

Helper Function: _solve_theta4_from_triangle

def _solve_theta4_from_triangle(S: np.ndarray, W: np.ndarray, p: NeroParams) -> Optional[float]:
    l_sw = np.linalg.norm(W - S)
    l_se = abs(p.d_i[2])
    l_ew = abs(p.d_i[4])

    c4 = (l_sw**2 - l_se**2 - l_ew**2) / (2.0 * l_se * l_ew)
    c4 = np.clip(c4, -1.0, 1.0)

    return math.acos(c4)

Key Insight

The elbow joint angle θ_4 depends only on the geometric link lengths and is completely independent of the arm angle ψ.

3.2 Step 2: Solving the Elbow Point E from the Arm Angle \psi (Core Geometry)

Theory from the Paper

The elbow point E lies on a circle whose chord is defined by the segment SW:

E= C + r (cos\psi*e_1 + sin\psi*e_2)

Where:

  • C: circle center
  • r: circle radius
  • e_1,e_2: orthonormal basis vectors spanning the circle plane

Code Implementation: _elbow_from_arm_angle

def _elbow_from_arm_angle(S: np.ndarray, W: np.ndarray, theta0: float, p: NeroParams) -> Optional[np.ndarray]:
    l_se = abs(p.d_i[2])
    l_ew = abs(p.d_i[4])

    sw = W - S
    l_sw = np.linalg.norm(sw)
    u_sw = sw / l_sw

    # Projection of circle center C onto line SW
    x = (l_se**2 - l_ew**2 + l_sw**2) / (2.0 * l_sw)

    r2 = l_se**2 - x**2
    r = math.sqrt(max(0.0, r2))

    C = S + x * u_sw

    # Construct circle-plane coordinate system e1, e2
    os_vec = S.copy()
    t = np.cross(os_vec, u_sw)

    e1 = t / np.linalg.norm(t)

    e2 = np.cross(u_sw, e1)
    e2 = e2 / np.linalg.norm(e2)

    # Compute elbow point E from arm angle theta0
    E = C + r * (math.cos(theta0) * e1 + math.sin(theta0) * e2)

    return E

This is the geometric core of the entire algorithm.

3.3 Step 3: Analytically Solving All Joint Angles from S–E–W

3.3.1 Shoulder Joints: q1,q2,q3

The paper derives a direct closed-form solution using geometric projection:

  • q1 is obtained from the projection of point E onto the base plane
  • q2 is determined by the height of E
  • q3 is solved from the direction of the wrist relative to the elbow

Code: _solve_q123_from_swe

def _solve_q123_from_swe(E: np.ndarray, W: np.ndarray, q4: float, p: NeroParams) -> List[np.ndarray]:
    d0 = p.d_i[0]
    d2 = p.d_i[2]
    d4 = p.d_i[4]

    Ex, Ey, Ez = E

    # q2
    c2 = (Ez - d0) / d2
    c2 = np.clip(c2, -1.0, 1.0)

    s2_abs = math.sqrt(max(0.0, 1.0 - c2**2))

    s4 = math.sin(q4)
    c4 = math.cos(q4)

    sols = []

    # Traverse both positive and negative s2 configurations
    for s2 in (s2_abs, -s2_abs):

        # q1
        c1 = -Ex / (d2 * s2)
        s1 = -Ey / (d2 * s2)

        n1 = math.hypot(c1, s1)

        c1 /= n1
        s1 /= n1

        q1 = math.atan2(s1, c1)
        q2 = math.atan2(s2, c2)

        # q3
        v = W - E
        col2 = -v / d4

        u1, u2, u3 = col2

        b1 = (s2 * c1 * c4 - u1) / s4
        b2 = (u2 - s1 * s2 * c4) / s4

        s3 = s1 * b1 + c1 * b2
        c2c3 = -c1 * b1 + s1 * b2

        c3 = c2c3 / c2 if abs(c2) > 1e-8 else (u3 + c2 * c4) / (s2 * s4)

        n3 = math.hypot(s3, c3)

        s3 /= n3
        c3 /= n3

        q3 = math.atan2(s3, c3)

        sols.append(np.array([q1, q2, q3]))

    return sols

3.3.2 Wrist Joints: q5,q6,q7

The paper analytically extracts the wrist joint angles directly from the transformation matrix T_{47}

  • cos \theta_6 = T_{47}[1,2]
  • \theta_5 and \theta_7 are computed from neighboring matrix element ratios

Code: _extract_567_from_T47_paper

def _extract_567_from_T47_paper(T47: np.ndarray) -> List[np.ndarray]:
    sols = []

    c6 = np.clip(T47[1, 2], -1.0, 1.0)

    for sgn in (1.0, -1.0):

        s6 = sgn * math.sqrt(max(0.0, 1.0 - c6**2))

        if abs(s6) < 1e-8:
            continue

        th6 = math.atan2(s6, c6)

        th5 = math.atan2(T47[2, 2] / s6, T47[0, 2] / s6)

        th7 = math.atan2(T47[1, 1] / s6, -T47[1, 0] / s6)

        sols.append(np.array([th5, th6, th7]))

    return sols

3.4 Step 4: Joint Limits → Feasible Region of the Arm Angle

Theory from the Paper

Each joint limit interval [q_{min},q_{max}] corresponds to a certain invalid region of the arm angle.

The intersection of all valid intervals yields the feasible arm-angle region \Psi_F.

Only arm angles within this feasible region guarantee that all joints remain inside their limits.

Code: _get_theta0_feasible_region

def _get_theta0_feasible_region(T07: np.ndarray, p: NeroParams, step: float = 0.01) -> List[float]:
    feasible = []

    for theta0 in np.arange(-math.pi, math.pi, step):

        if _ik_one_arm_angle(T07, theta0, p):
            feasible.append(float(theta0))

    return feasible

Internally, the function calls _ik_one_arm_angle, which performs the following steps:

  • Substitute the arm angle \psi
  • Solve the complete joint configuration
  • Check whether all joints satisfy their limits
  • If valid → add the arm angle to the feasible region

3.5 Step 5: Optimal Arm-Angle Selection (Weighted Quadratic Objective Function)

Theory from the Paper

The objective function is defined as:
f(\psi) = \sum w_i(q_i(\psi)-q_{i,prev})^2

  • w_i:Weight coefficient, which increases as the corresponding joint approaches its mechanical limit.
  • Objective: To minimize the overall joint motion while keeping all joints as far as possible from their limits.

Weighting Function (Equation 20 in the Paper)

  • w_i=\frac{bx}{e_{a(1-x)-1}},x ≥ 0
  • w_i=\frac{-bx}{e_{a(1-x)-1}},x \lt 0

Where

  • a=2.28
  • b=2.28

Code: _weight_limits

def _weight_limits(q: float, q_min: float, q_max: float) -> float:
    span = q_max - q_min

    x = 2.0 * (q - (q_min + q_max) * 0.5) / span

    a = 2.38
    b = 2.28

    if x >= 0:
        den = math.exp(a * (1 - x)) - 1
        return b * x / den
    else:
        den = math.exp(a * (1 + x)) - 1
        return -b * x / den

Optimal Arm-Angle Search

def _optimal_theta0(feasible_theta0, T07, p, q_prev):

    best_cost = inf
    best_t = feasible_theta0[0]

    for t in feasible_theta0:

        sols = _ik_one_arm_angle(T07, t, p)

        for q_full in sols:

            q = q_full[:7]

            cost = 0

            for i in range(7):

                lo, hi = p.joint_limits[i]

                w = _weight_limits(q[i], lo, hi)

                dq = abs(q[i] - q_prev[i])

                cost += w * dq * dq

            if cost < best_cost:
                best_cost = cost
                best_t = t

    return best_t

This is the optimal solution selection strategy proposed in the paper.

In essence, it transforms the problem into:

One-dimensional quadratic-function minimization → globally optimal solution → no iterative solving and no local minima.


Part 4.Null-Space Motion Principle (Naturally Embedded)

For a 7-DoF manipulator, the null space is directly controlled by the arm angle \psi.

The principle is straightforward:

  • The end-effector pose T_{07} remains unchanged
  • Only the arm angle ψ is varied
  • The robot joints automatically perform self-reconfiguration while keeping the end-effector fixed
    This is known as null-space motion.

In the implementation, null-space motion can be generated simply by sweeping the arm angle:

for psi in np.linspace(-pi, pi, 100):
    q = _q_from_theta0(psi, T07, p)

No Jacobian matrix is required,
no projection operator is needed,
and the motion remains smooth and stable without oscillation.


Part 5.Code Structure Overview (Clean Version)

Core Functions in ik_solver.py(链接)


Part 6.Quick Start Guide

import numpy as np
from ik_solver import ik_arm_angle, NeroParams

# Define target end-effector pose
T = np.eye(4)
T[:3, 3] = [0.5, 0.0, 0.5]

# Solve inverse kinematics
q_best, feasible_set = ik_arm_angle(T)

print("Optimal joint configuration:", q_best)
print("Number of feasible arm angles:", len(feasible_set))

Part 7.Summary
This method presents a closed-form inverse kinematics solver for a 7-DoF S–R–S robotic manipulator, combined with a 1D quadratic optimization over the arm-angle null space.

Key characteristics:

1.Pure geometric closed-form solution

  • No iterative optimization
  • No Jacobian-based numerical solving

2.Automatic joint limit compliance

  • Feasible region explicitly constrained

3.Optimality guaranteed via quadratic cost function

  • Efficient 1D optimization over arm angle

4.Natural support for null-space motion

  • Arm angle acts as redundancy parameter

5.Real-time performance

  • Extremely fast computation suitable for control loops and embodied systems

2 posts - 1 participant

Read full topic

by Agilex_Robotics on May 12, 2026 06:51 AM

May 11, 2026
Introducing an Rviz Alternative that runs SUPER FAST

A new ROS2-native, SUPERFAST visualizer written in Rust — `fastviz`

Hi everyone,

I’ve been working on a project called **fastviz**: a Rust-based 3D visualizer that runs as a native ROS2 node, built on `wgpu` and `egui`. RViz has been the workhorse of the community for many years and isn’t going anywhere — fastviz is just an experiment to see how much smoothness and headroom we can get out of a pure-Rust + GPU-native pipeline, and I wanted to share where it’s at in case it’s useful to others.

It’s at a preliminary stage — only a handful of message types are wired up so far — but the core architecture is in place and it already renders things like TurtleBot 4 in Gazebo end-to-end.

**Repo:** GitHub - ksatyaki/fastviz: Rust based HOPEFULLY superfast viz for ROS2 · GitHub

-–

## The bits I’m most excited about

### 1. It IS a ROS2 node

No bridge, no middleware, no separate process. fastviz subscribes directly to topics via `r2r`, so there’s nothing extra to wire up between your robot and the visualizer.

### 2. The render thread never touches ROS2

The `r2r` executor runs on a dedicated thread; the renderer talks to it through an `Arc<RwLock>` with brief, write-only handoffs. The UI never blocks on DDS — frames stay smooth even when a noisy topic is flooding the graph.

### 3. GPU-accelerated via `wgpu`

Vulkan on Linux, Metal on macOS, DX12 on Windows, and WebGPU is on the menu too. Same renderer everywhere.

### 4. Revision-cached render passes

A `revision()` counter on the scene graph drives pass-level caching, so an idle scene costs ~zero CPU. Walking away from the visualizer doesn’t pin a core.

### 5. GPU-side per-entity transforms for point clouds

The point-cloud pipeline is instanced, per-entity transforms happen on the GPU, and the prepare step is revision-cached with buffer reuse. PointCloud2 streams stay cheap.

### 6. TF tree reimplemented in Rust

No `tf2` C++ dependency — TF maintenance lives in pure Rust alongside the rest of the ingestion layer.

### 7. TOML config as the source of truth

Layouts are declared in a TOML file — diff-friendly, version-controllable, and easy to commit alongside your robot’s launch config.

### 8. Polled wildcard topic discovery

Drop `“*”` into a topic list and every matching message type in the ROS graph gets auto-subscribed within about a second. Handy when you’re exploring an unfamiliar bag or sim and don’t want to enumerate topics by hand.

### 9. Per-topic QoS overrides in config

`reliability`, `durability`, and `depth` are all settable per topic from the same TOML file.

### 10. URDF support with STL / OBJ / DAE meshes

URDF parsing via `urdf-rs`; mesh loading covers STL, OBJ, and Collada. `package://` URIs resolve through `AMENT_PREFIX_PATH`, and `JointState` drives the FK.

### 11. Dev container + release Docker image

The `.devcontainer/` ships an Ubuntu 24.04 + ROS2 Jazzy image with `r2r` build deps, the Vulkan loader, and NVIDIA passthrough already wired up. A root `Dockerfile` also builds a release image you can `docker run`.

-–

## What’s supported today (early days!)

This is very preliminary — only a few message types are supported right now:

| Topic kind | Message |

| -------------- | -------------------------------- |

| `[map]` | `nav_msgs/OccupancyGrid` |

| `[poses]` | `geometry_msgs/PoseStamped` |

| `[pose_arrays]`| `geometry_msgs/PoseArray` |

| `[paths]` | `nav_msgs/Path` |

| `[scans]` | `sensor_msgs/LaserScan` |

| `[points]` | `sensor_msgs/PointCloud2` |

| `[tf]` | `tf2_msgs/TFMessage` |

| `[urdf]` | `std_msgs/String` + `JointState` |

`MarkerArray`, `Image`, `Imu`, `Odometry`, and friends are on the near-term roadmap. ROS2 Jazzy is the only distro currently tested.

-–

## Try it

```sh

git clone GitHub - ksatyaki/fastviz: Rust based HOPEFULLY superfast viz for ROS2 · GitHub

cd fastviz

source /opt/ros/jazzy/setup.bash

cargo build --release

cargo run -p app – --config configs/turtlebot4.toml

```

Or via the dev container — open the folder in VS Code / Cursor and pick “Reopen in Container”.

-–

## Help wanted

If you give it a spin, I’d genuinely love to hear:

- which message types you’d want supported next,

- what kinds of bags would make good benchmarks,

- any architectural input on plugins, MCAP playback, or multi-window layouts.

Issues, PRs, and “this completely broke on my robot” reports are all very welcome.

Hopefully this can grow into something useful for the community. Thanks for taking a look!

**GitHub:** GitHub - ksatyaki/fastviz: Rust based HOPEFULLY superfast viz for ROS2 · GitHub

3 posts - 2 participants

Read full topic

by JackMcMurdo on May 11, 2026 07:47 PM

ROSCon Global 2026 Registration Now Open! Workshop and exhibitor info now available!

ROSCon Global 2026 Registration Now Open!

Workshop and exhibitor info now available


Hi Everyone,

I am happy to announce that registration for ROSCon Global in Toronto is now open! We highly encourage you to register as soon as possible as ROSCon often sells out and our most popular workshops fill up fast. Early bird ticket prices will be available until July 12th, 2026. Our early bird rates are quite generous and effectively make workshop registration free! Even if you don’t plan to attend a workshop, we recommend you join us for all three days of the event as there will be a number of other activities, like birds of a feather sessions, happening on the first day of ROSCon Global. Given what I’ve heard from other community members, there will likely be a number of other events happening immediately before and after the official ROSCon Global event (I might be cooking something up for the Friday after the conference :wink:).

If you are curious about what else there is to do in the Toronto area while visiting, our hosts have graciously put together a micro-site with dining, shopping, and tourist activities in Toronto.

ROSCon Workshops

Due to the incredible demand for ROSCon workshops last year we’ve expanded our workshop capacity for 2026! We’re excited to announce that this year we will be offering eight half-day workshops and two full-day workshops. ROSCon Global is now officially a three day event, and even if you choose not to attend workshops there will be Birds of a Feather sessions and other events during the first day of the event. We recommend that you plan to be in Toronto for the entire week as we have a couple other big announcements coming out in the next few weeks!

I’ve summarized our ROSCon Global workshops below, but a full list is available on the website.

  • [Half-day] From URDF to USD: A Complete Pipeline for High-Fidelity ROS 2 Simulation in NVIDIA Isaac Sim with Ji Yuan Feng and Ayush Ghosh. Build a complete robot simulation pipeline from raw URDF to ROS 2 integration using NVIDIA Isaac Sim.
  • [Half-day] Train and Deploy Contact-Rich Robot Manipulation Skills With Isaac Lab and Isaac ROS, with Raffaello Bonghi, Rishabh Chadha, Ashwin Varghese Kuruttukulam, and Ayusman Saha. Develop and deploy contact-rich manipulation policies for tasks like gear assembly using Isaac Lab and Isaac ROS.
  • [Half-day] Train, Simulate, Deploy: Agentic AI from Cloud to Robot with Ken O’Brien, Graham Schelle, Sarunas Kalade, Mehdi Saeedi, and Adam Dąbrowski, Explore practical pathways for training and deploying embodied AI models across cloud environments and on-device NPUs.
  • [Half-day] Scaling ros2_control: From Async Hardware Drivers to RL Inference Engines with Sai Kishor Kothakota, Bence Magyar, Christoph Fröhlich, and Denis Štogl, Architect asynchronous hardware interfaces to prevent I/O bottlenecks and deploy reinforcement learning models efficiently.
  • [Full-day] Advanced Aerial Robotics with PX4 and ROS 2: Custom Flight Modes and Beyond with Ramon Hernan Roche Quintana, Beniamino Pozzan, and Patrik Dominik Pordi, Build custom flight modes and sequence complex autonomous behaviors for aerial robots entirely in ROS 2.
  • [Half-day] Motion Planning Fundamentals with Moveit with Yara Shahin and Timotej Gaspar, Learn the core concepts of path planning and collision avoidance by configuring MoveIt 2 for a real robot from scratch.
  • [Half-day] Introduction to ROS and Building Robots with Open-Source Software with Geoff Biggs and Katherine Scott, Master the fundamentals of building robots using open-source software.
  • [Half-day] Declarative ROS workspaces with Pixi and RoboStack: A hands-on workshop for reproducible ROS development with Ruben Arts, Wolf Vollprecht, and Bas Zalmstra. Use Pixi and RoboStack to declare your entire ROS environment in a single file to guarantee completely reproducible development setups.
  • [Half-day] Mastering the Jazzy RMW: A Performance-Driven Framework for ROS 2 Middleware Selection and Tuning with Nathan Van Heyst, Tony Baltovski, Luis Camero, and Jose Mastrangelo, Optimize ROS 2 middleware performance through systematic tuning, benchmarking, and high-scale stress testing.
  • [Full-day] Navigation University with David Lu and Binit Shah, Configure a simulated mobile robot step-by-step to successfully navigate its environment using maps, localization, and planning.

ROSCon Global Sponsors

ROSCon Global wouldn’t be possible without our wonderful sponsors. Below you will find a list of the initial batch of ROSCon Global Sponsors. Make sure to check them out at our ROSCon Global Expo Hall. Many of our sponsors will be holding exclusive demonstrations during ROSCon Global that you’ll want to check out.

:1st_place_medal: Gold Sponsors

:2nd_place_medal: Silver Sponsors

:3rd_place_medal: Bronze Sponsors

:seedling: Startup Alley Sponsors

3 posts - 2 participants

Read full topic

by Katherine_Scott on May 11, 2026 11:11 AM

Built an Autonomous Mobile Robot (AMR) for warehouse automation - from CAD to code

Designed the chassis in Fusion 360, exported to URDF, and built the full stack using ROS 2.

Stack:

Nav2 for navigation & path planning

ArUco-based visual docking for precise alignment Custom waypoint sequencing for multi-shelf tasks • Gazebo + RViz for simulation & visualization

Challenge:

LiDAR point cloud rotated with the robot in RViz, breaking the mapping and navigation.

Root cause:

odom/TF mismatch during turns.

Developed a Ground TruthOdom node using Gazebo pose data to publish stable /odom and consistent TF, including handling ROS-Gazebo timestamp issues.

In the video: robot autonomously services requests for Shelf B and Shelf C and delivers them to the drop-off zone.

Happy to discuss the system or challenges!

ros2 robotics #AMR nav2 Gazebo urdf #WarehouseAutomation #OpenRobotics opencv #ComputerVision

588430240-ba0cc7cf-33bd-4987-a1d0-1731533b003f

1 post - 1 participant

Read full topic

by Sourav24 on May 11, 2026 04:58 AM

May 10, 2026
RoboInfra is the missing infrastructure layer for robotics development

Robot models defined in URDF act as the “source code” of robotic systems, but the tooling around them is fragmented, slow, and dependent on full ROS installations.

RoboInfra solves this by providing a unified API platform that enables developers to:

  1. Validate URDF files instantly with 9+ structural checks
  2. Analyze robot kinematics including degrees of freedom and chain structure
  3. Compare URDF versions semantically instead of raw XML diffs
  4. Convert URDF to simulation-ready formats like SDF (Gazebo) and MJCF (MuJoCo)
  5. Preview robots in 3D directly in the browser
  6. Integrate validation into CI/CD pipelines using a lightweight GitHub Action
  7. Automate workflows via a Python SDK

All of this works without installing ROS, reducing setup time from hours to seconds.

RoboInfraapi is built for:

  1. Robotics developers
  2. Simulation engineers
  3. Research labs (RL, control, autonomy)
  4. DevOps teams managing robot pipelines

Free URDF Validator RoboInfra

1 post - 1 participant

Read full topic

by Robotic on May 10, 2026 06:55 PM

May 09, 2026
ROS 2 Launch YAML/XML Schema + VS Code Integration (Validation, Completion, Substitutions)

Hi everyone,

I’ve been working on a project to improve the developer experience when writing ROS 2 Jazzy launch files.
It provides JSON Schema for ROS 2 launch YAML, XSD for launch XML, and VS Code integration including auto-completion, validation, hover docs, and substitution snippets.

:package: Repository
https://github.com/ok-tmhr/ros2_awesome


:rocket: Features

1. YAML Schema for ROS 2 Launch Files

Provides full validation and auto-completion for:

  • launch: structure

  • actions (node, group, arg, set_env, etc.)

  • parameters

  • substitutions

  • conditions (if, unless)

Schema URL:

https://ok-tmhr.github.io/ros2_awesome/schema/launch.yaml

You can enable it by adding this at the top of your .launch.yaml:

# yaml-language-server: $schema=https://ok-tmhr.github.io/ros2_awesome/schema/launch.yaml


2. XML Schema (XSD) for launch XML

https://ok-tmhr.github.io/ros2_awesome/schema/launch_ros.xsd

Usage:

<launch
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:noNamespaceSchemaLocation="https://ok-tmhr.github.io/ros2_awesome/schema/launch_ros.xsd">


3. Substitution Snippets for VS Code

The repo includes a snippet extension (launch_substitution.json) that can be installed via:

It provides auto-completion for:

$(var ...)
$(env ...)
$(not ...)
$(eval ...)

Typing $ triggers suggestions.


4. Substitutions also work in parameter files

When a parameter YAML is loaded via a launch file, substitutions are evaluated:

my_node:
  ros__parameters:
    use_sim_time: $(var use_sim_time)
    robot_name: $(env ROBOT_NAME)

This is supported by the schema and by the snippet extension.


5. Sample directory included

The sample/ directory contains working examples for both YAML and XML launch files.

2 posts - 2 participants

Read full topic

by ok-tmhr on May 09, 2026 12:41 PM

May 08, 2026
ros2_lingua: A safe, dependency-aware grounding engine for LLMs

​Hi everyone,

​Like many of us, I’ve been experimenting with giving LLMs control over robot hardware. However, I quickly ran into the classic problems: LLMs hallucinate actions, assume prerequisites that haven’t been met (e.g., trying to drive a humanoid before stabilizing it), and most existing integrations are just tightly coupled, hardcoded scripts.

​To solve this, I built ros2_lingua — an open-source bridge that introduces a structured capability contract between ROS 2 nodes and LLMs.

​Instead of letting the LLM guess what topics or actions to call, ros2_lingua forces the LLM to output a plan based only on explicitly registered capabilities, and uses a backward-chaining planner to automatically inject missing prerequisite steps.

​How it works:

  1. ​Capability Advertisement: Any ROS 2 node can inherit from LinguaMixin to self-advertise its capabilities at boot. It defines its name, ROS action/service, parameters, preconditions, and postconditions.
  2. ​Backward-Chaining Planner: When a user gives a natural language instruction (e.g., “go to the table and pick up the bottle”), the Grounding Engine checks the robot’s current state against the capability schema. If the robot isn’t balanced, the planner automatically injects a stabilize_robot capability before the navigation step.
  3. ​Safe Dispatch: The DispatcherNode safely executes the validated plan over standard ROS 2 actions and services.

Decoupled Architecture

​One of my main goals was to ensure the core logic was highly testable. The project is split into two layers:

ros2_lingua_core: A pure Python library containing the schema, registry, planner, and LLM backends (Ollama, OpenAI, Anthropic). It has zero ROS 2 dependencies, meaning the grounding engine can be unit-tested purely in Python.

ros2_lingua: The ROS 2 interface layer containing the GroundingNode, DispatcherNode, and mixins.

Links & Demo

​You can see a demo of the engine running with a local Ollama model and a mock humanoid setup, along with the full architecture documentation here:

​Documentation & Architecture: ros2_lingua — Documentation

​GitHub Repository: GitHub - purahan/ros2_lingua: Natural language to ROS2 actions — a structured LLM grounding engine for any robot. · GitHub

​What’s Next & Feedback Request

​The project is currently a working prototype in Python. My immediate roadmap includes taking this to a release-ready state and building a C++ bridge so native controller nodes can easily advertise their capabilities.

​Since this is early development, I would love to get feedback from the community on the architecture—specifically on the schema design for the capability registry and how best to handle complex, long-running action pre-emptions within the Dispatcher.

​Thanks for your time, and I’d love to hear your thoughts!

1 post - 1 participant

Read full topic

by purahan on May 08, 2026 09:51 PM

Control Algorithm Dominance Survey

Hey guys, I’m doing a survey to ascertain the dominance of different control engineering paradigms in the industry, to ascertain whether there has been a noticeable shift from classical controls to more modern algorithms, or whether modern algorithms, while looking good on paper, are stuck on research papers for the most part.
I would love everyone’s inputs, from student to seasoned researcher.
Your still welcome to contribute if you don’t work directly in controls, or if your work is controls-adjacent, like SWE or mechanical design.

2 posts - 2 participants

Read full topic

by demi on May 08, 2026 01:57 PM

May 06, 2026
Where does latency in WebRTC video streaming come from? An analysis

We analyzed the glass-to-glass latency of streaming video from robots to the web using WebRTC. Typical, total latency for remote streaming is 150-180 ms but how does this break down?

Tl;dr:

  • The vast majority of latency actually comes from the camera itself and the USB bus (~100 ms).
  • H264 encoding and decoding add around 10 ms each (or less).
  • WebRTC only adds around 10 ms of latency for remote streaming (jitter buffers).
  • The rest is due to static network delay (“ping timing”, speed of light).

Read the full analysis here:

1 post - 1 participant

Read full topic

by chfritz on May 06, 2026 05:40 PM

ROS2 + Gazebo Harmonic on macOS 26

Hello,

It seems ROS2 on macOS 26 installation guide is this: Installing ROS 2 on macOS — ROS 2 Documentation: Crystal documentation
Gazebo Harmonic on macOS 26 installation guide is this: Binary Installation on macOS — Gazebo harmonic documentation

I read online there might be some incompatibility issues. I would like to understand what’s the current picture of the ROS2+Gazebo Harmonic. Would those two guides work and are there any expected issues?

1 post - 1 participant

Read full topic

by MT1234 on May 06, 2026 05:36 PM

Camera streaming made easy [NEW Release]

Greetings fellow roboticists,

you might know me from other projects such as ros_babel_fish, the QML ROS 2 module, or RQml.
Maybe as the dude who ends all their posts with an AI-generated image.

TLDR: ros_camera_server out now (link below). Efficient streaming to ROS, robust low-bandwidth to operator station over H.264/H.265 using rtp/srt/webrtc. Simple config, automatically optimized pipelines.

Camera streaming in ROS is kind of cumbersome, even though it’s always the same issue.
On the robot, you want raw (or compressed between PCs) ROS images for processing, and on the remote operator station, you want a low-latency, ideally low-bandwidth live video feed.
In academia, many setups I have encountered use compressed images for remote operator setups because it’s simple and, in good network conditions, works well enough.
Using usb_cam or gscam with compressed images is also what Opus 4.7 would suggest when asked.
While this does work, it’s quite resource- and bandwidth-intensive and is one of the roadblocks that need to be addressed to bring research solutions to the field, especially in rescue robotics, where bandwidth is a limited resource.

To save bandwidth, you can use something like gst_bridge to receive the ROS image, encode it in H.264, and send it to your remote operator station.
This will be approximately 1/10 to 1/30 of the bandwidth for comparable quality.

If your camera is a high-resolution USB camera, it will most likely stream jpeg encoded data for the high resolution, high fps options.
So your pipeline becomes:
Camera (jpeg) → usb_cam (decodes jpeg) → image_transport (re-encodes as jpeg for compressed) → gst_bridge → handwritten gstreamer pipeline (requires some technical knowledge to get right) → stream

Doesn’t take an expert to see that this is not optimal.
Here’s where my new ros_camera_server comes in.
You specify one yaml file with your cameras, each with one input and as many outputs as you want (and your compute can handle).
The outputs can differ in resolution and framerate.
Currently supported are ROS 2, RTP, SRT, and WebRTC.
The camera server will automatically create and optimize GStreamer pipelines based on your available hardware accelerations, which, in parallel, produce your ROS output and streams applying scale and framerate limiters as necessary.
Cutting the decode and re-encode overheads and significantly reducing latency and CPU usage.
JPEG camera input can also be published directly as a ROS-compressed image or forced to be decoded if needed.
Check the plots from my benchmark in the comments to see that the much easier configuration is not paid for with higher latency or overhead, and it beats the alternatives in both.

Here’s the repo:

If you can’t comply with the AGPL, you can contact me to see if we can find a suitable license for your use.

PS: The ros_camera_server preserves the image capture/header timestamp as a custom RTP header extension and can restore it from ros_camera_server H.264/H.265 streams.
So you can stream from the robot over RTP/SRT/WebRTC and restore it to ROS on the operator station or another robot, and the timestamp will be preserved in the ROS image output.

Benchmarks



I hope this helps groups without video streaming experts to create more robust remote control setups.
If you read this far and this was not of interest to you, I’m sorry, here’s your AI picture:

1 post - 1 participant

Read full topic

by StefanFabian on May 06, 2026 01:02 PM


Powered by the awesome: Planet