For decades, robotics has captured the imagination of innovators and technologists alike. From Hal 9000 in 2001: A Space Odyssey to Baymax in Big Hero 6, robots have symbolized the future of technology—a cutting-edge vision of what’s possible. Despite their potential, robotics has largely remained within the confines of academia, where progress is often measured in decades rather than quarters. However, we’re starting to see robotics follow a similar path of transformation from pure research play to broadly valuable horizontal companies, thanks to a groundbreaking development: Robotic Foundation Models (RFMs).
The AI parallel: Lessons from the evolution of AI
To understand the significance of RFMs, it’s helpful to draw a parallel with the evolution of AI. In the early days, AI was the domain of research institutions. This first wave of AI was all about foundational research with little to no immediate commercial application. It was slow, methodical, and largely inaccessible to those outside of academia.
In the early 2000s, the second wave of AI brought more purpose-built technologies—vertically integrated, expensive, and specialized enough to show promise but not yet ready to revolutionize industries. These early AI solutions were useful, but they were also siloed, often requiring extensive expertise to implement and maintain.
Then came the introduction of foundation models, which democratized AI by enabling people without advanced degrees in machine learning to create impactful applications. These models, like GPT for natural language processing, provided a versatile, scalable foundation upon which various applications could be built. This shift lowered the barrier to entry for builders and opened the floodgates for innovation, allowing AI to move from niche applications into widespread use across industries.
Now, with Robotic Foundation Models, we’re witnessing a similar evolution in robotics.
Robotics 101: A quick journey through history
Before diving into the transformative potential of RFMs, it’s worth revisiting how robotics has evolved over the years. Before the 1980s, robotics was primarily confined to research institutions like MIT and NASA. These early efforts were exploratory, with progress measured slowly, often over decades. The late 20th century marked a turning point where robotics began to be commercialized at scale, particularly within the automotive industry. Industrial robots, especially robotic arms, became staples on assembly lines, leading to the rise of companies like FANUC, which specializes in industrial automation.
The 2000s saw another leap forward as robotics technology became more advanced and versatile. This period marked the rise of e-commerce, driving the need for automation in warehousing and logistics. Companies like Kiva Robotics (now part of Amazon) led the charge, deploying robots at scale in Amazon fulfillment centers, setting the stage for robotics to enter new industries and applications. Two major categories emerged:
- Highly controlled environments with specific constraints on robotic movement and functionality, such as autonomous mobile robots (AMRs) in warehouses (think Locus Robotics)
- Industries with deep pockets and long timelines, such as self-driving car companies, where early investments in robotics held the potential for massive impact
During this era, robotics startups were required to build full-stack solutions, mastering hardware, autonomy, and application software to create task-specific robots.
Foundational technology in both hardware and software reached a point where startups could enter the game (Locus Robotics in AMRs, Zoox in self-driving cars, Skydio in high-capability drones). These were significant first steps, akin to the second wave of AI, where early companies like DeepMind and UiPath were starting to make their mark.
The dawn of Robotic Foundation Models: But, wait, what are they?
Now, let’s get to the heart of the matter: what exactly are Robotic Foundation Models? In essence, an RFM is an advanced, generalized AI model designed to serve as a versatile base for a wide range of robotic applications. Much like LLMs in AI, RFMs are pre-trained on massive and diverse datasets, enabling them to understand and perform various tasks without the need for extensive task-specific programming.
At first glance, RFMs might seem like another academic project with limited commercial applications. But there’s something truly special here: RFMs imbue robots with the ability to generalize knowledge and skills across multiple domains, such as navigation, manipulation, and perception. This generalization capability is what makes RFMs so revolutionary. It’s the key to achieving a level of flexibility and intelligence in robots that has long been considered the holy grail of robotics.
The holy grail: Generalization in robotics
Generalization has been a core goal in robotics for decades, becoming the moonshot focus during the era of robotic adoption in automotive and industrial settings in the 1980s and 1990s. Industries began to realize the limitations of task-specific robots—machines that excelled at repetitive tasks in controlled environments but fell short when it came to adaptability. These early robots, while efficient, were rigid and struggled to adapt to new tasks or changes in their environment or the production process. The late 1900s and early 2000s marked a turning point as advancements in machine learning and neural networks opened new venues for robots to learn, adapt, and perform a broader range of tasks without extensive reprogramming. These technologies enabled robots to move beyond their initial constraints, learning from data and operating in more complex, dynamic environments. Today, with the rise of RFMs, the promise of truly adaptable, general-purpose robots is closer than ever, reigniting the conversation around generalization and its potential to transform the robotics industry.
But what does generalization actually mean in the context of robotics? Simply put, it means creating systems that can adapt to various conditions, tasks, and environments, essentially demonstrating a form of intelligence. This concept can be broken down into several dimensions:
- Generalization across embodiments (physical form factors): RFMs can deploy on the most appropriate form factor for each use case, whether it’s a Boston Dynamics Spot robot, a drone, or perhaps a humanoid in the distant future. This flexibility eliminates the need to rely on a single, optimized physical form, offering a more versatile and customer-centric solution.
- Generalization across object handling: RFMs can handle edge cases more robustly, significantly improving uptime and consistency in task execution. This improvement makes it feasible for robots to fully automate tasks that are dull, dirty, or dangerous—tasks that humans shouldn’t have to do.
- Generalization across operational design domains: RFMs enable robots to efficiently navigate complex, dynamic environments, including previously uncharted territories such as hazardous or inaccessible areas. This adaptability could dramatically reduce setup times and expand the range of applications for robotic systems, bringing us closer to a future where robots can be deployed seamlessly across diverse settings.
- Generalization across tasks: A single robotic platform could handle a wide array of functions, making it a versatile tool for multiple applications. This level of flexibility can deliver substantial ROI, transforming robotics from a specialized solution into a broadly applicable technology.
Building the future: The path to commercialization
So, how do we turn this immense potential into reality? One promising approach involves positioning Robotic Foundation Models as a middleware layer between hardware and application-specific software. In this model, there’s a provider for the robotic hardware platform (like Boston Dynamics with its Spot robot), an RFM provider in the middle solving for a core pillar of autonomous functionality (such as navigation or coarse manipulation), and an application-specific software layer on top that processes the data captured by the robot, generates insights, and integrates these results into existing workflows.
This separation allows for generalizability across form factors, facilitating the deployment of versatile robotic systems. Additionally, when considering the application-specific software, a construction manager doesn’t just need a robot to capture site images; they need a system that processes these images, generates reports, and integrates into existing workflows. This kind of application-specific software, sitting atop the RFM layer, provides a comprehensive solution tailored to industry needs.
By leveraging RFMs as the software foundation, a new wave of robotics software companies could emerge, focusing on vertical-specific applications and creating a robust ecosystem around robotics technology. If this model proves correct, an entirely new category of software companies can come into existence – companies that previously didn’t exist due to the challenges of collecting, sorting and analyzing data from robotics systems.
The road ahead
While the vision of a versatile, RFM-powered future is compelling, several challenges remain. One significant bottleneck is the limited availability of advanced robotic hardware platforms. For example, Boston Dynamics’ Spot, one of the leading robotic hardware platforms, has only just over a thousand units in the market after four years of selling. This limited hardware availability could constrain the scale at which RFMs can be implemented.
Additionally, developing application-specific software is no trivial task. Companies that specialize in these applications often have deep industry expertise, making it challenging for generalist RFM providers to compete. The market needs time to mature, and it’s a classic chicken-and-egg scenario: widespread adoption of robotic systems can drive the development of specialized software, but the latter is needed to maximize the utility of the former.
Seizing the opportunity
Despite these challenges, the landscape is ripe with opportunity. We’ve seen an explosion of companies racing to develop RFMs, creating a fertile ground for innovation. However, not all players in the RFM space are created equal. Some have captured early attention, buoyed by substantial capital infusions, while others are operating in stealth mode, waiting for the right moment to emerge. This landscape is still in its infancy, but there’s a clear trend: companies are honing in on specific pillars of robotics software functionality, with significant focus areas including navigation and manipulation.
Looking ahead, the future remains uncertain. We might see specialized RFMs dominate distinct categories, with clear leaders in each segment. Alternatively, these players might start expanding into each other’s territories, evolving into comprehensive RFMs that can seamlessly integrate all robotic functionalities. Judging by the size of the funding rounds we’ve observed, it’s evident that many in the field are betting on the latter.
As investors, the theoretical potential of Robotic Foundation Models is undeniably compelling, but the real test lies in market adoption. The true impact of this technology will be measured by how effectively it can be deployed and scaled in real-world scenarios. We’re eager to see these RFMs move from concept to execution, proving that the promised ROI from generalization isn’t just possible, but tangible. This next step in robotics development has the potential to redefine industries, and we’re excited to watch as it unfolds in the marketplace.
A new dawn for robotics
Robotics is on the cusp of a major transformation. With the advent of RFMs, we’re witnessing a paradigm shift similar to what AI experienced a few years ago. I believe there’s still ample room to build meaningful companies focused on developing RFMs, as this area is just beginning to take shape, with vast, untapped potential for innovation and growth—think getting in on the ground floor of OpenAI.
This shift also opens up the field to a broader range of innovators by separating the application layer, the autonomy and infrastructure layer, and the hardware layer. People can now build companies around their core expertise: robotics folks can focus on creating groundbreaking robots without the need for task-specific vertical integration; domain experts can build application software that solves real business problems without needing to hire a robotics team; and hardware builders can bring their most ambitious ideas into physical reality. If this is how the future unfolds, we could see not only more companies but better companies, as each solution no longer needs to be bespoke.
