Robotics

2014-2020

Tools and infrastructure for robots.

Concept

I’ve always been passionate about building tools that make robotics more accessible, and its practitioners more productive.

This was my sole focus at Google X and Formant (building tools for engineers and researchers), and a dimension of my work at Bot & Dolly (building tools for artists).

Technology

At Google X [2014-2018] I built engineering tools for a diverse set of next-generation robotics applications. My work was inspired by the technologies, patterns, and practices that Alphabet had developed over time to support its products at scale.

That experience informed my decision to join Formant [2018-2020] as a founding engineer, where I would have the chance to bring my ideas about robotics tools and infrastructure to a more public audience.

Formant’s software helps robotics companies operate fleets of robots: in stores, in hospitals, on city sidewalks, or elsewhere. Formant provides cloud connectivity, telemetry ingestion, monitoring, teleoperation, remote intervention, analytics, and other capabilities.

The technology stack at Formant encompassed on-robot software, backend services, and a frontend web application. Our on-robot software had to be portable, secure, performant, and robust to poor network conditions. In the backend, we needed services capable of ingesting large amounts of high-dimensional time-series data, indexing and persisting this data durably for the long-term, and serving it with low-latency at an affordable cost. And in the frontend, we had to support 3D visualization, video, and low-latency bidirectional communication with machines in the field.

Besides building and leading the engineering team, I made core contributions to the backend application layer, the telemetry ingestion pipeline, and the teleoperation stack.

Result

A typical view in the Formant web application, combining live video, text, numeric, and geospatial data on a unified timeline.

A typical view in the Formant web application, combining live video, text, numeric, and geospatial data on a unified timeline.

A user filters fleet-wide telemetry data by robot location to diagnose an issue.

An analytics view in Formant, displaying aggregations of sensor data to measure fleet-wide performance.

An analytics view in Formant, displaying aggregations of sensor data to measure fleet-wide performance.

A technician uses Formant to take notes and photos describing an issue encountered in the field.

A technician uses Formant to take notes and photos describing an issue encountered in the field.

A fleet overview in Formant, displaying health status and active alerts along different dimensions of the fleet.

A fleet overview in Formant, displaying health status and active alerts along different dimensions of the fleet.

A teleoperation view in Formant, allowing for remote control of a robot alongside live video and sensor data.

A teleoperation view in Formant, allowing for remote control of a robot alongside live video and sensor data.

One of the robotics initiatives I worked closely with at Google X.

Tools for a different type of robotics practictioner: the student architect. I installed this robotics lab on the IDEAS Campus in UCLA’s Department of Architecture and Urban Design.