earlier this week, the Toyota Research Institute opened the doors of its Bay Area offices to members of the media for the first time. It was a day full of demos, from driving simulators and drift instructors to conversations about machine learning and sustainability.
Robotics, a longtime focus of Toyota’s research division, was also on display. SVP Max Bajracharya presented a couple of projects. The first was something more like what you’d expect from Toyota: an industrial arm with a modified grapple designed for the surprisingly complex task of moving boxes from the back of a truck to nearby conveyor belts – something most factories hope to automate soon. . the future.
The other is a little more surprising – at least for those who haven’t closely followed the division’s work. A shopping robot retrieves different products from the shelf based on barcodes and general location. The system is able to extend to the top shelf to find items, before determining the best method for grabbing the wide range of different objects and placing them in your basket.
The system is a direct outgrowth of the 50-person robotics team’s focus on elderly care, with the goal of addressing Japan’s aging population. However, it represents a departure from his original work of building robots designed to perform menial tasks like washing dishes and preparing food.
You can read a longer description of this pivot in an article published on TechCrunch earlier this week. This is excerpted from a conversation with Bajracharya, which we are printing in a more complete state below. Please note that the text has been edited for clarity and length.
TechCrunch: I was hoping to get a demo of the home robot.
Max Bajracharya: We’re still doing some home robot stuff (…) What we’ve done has changed. The house was one of our original challenge assignments.
Seniors was the first pillar.
Absolutely. One of the things we learned in this process is that we don’t measure our progress very well. The house is so difficult. We choose challenging tasks because they are difficult. The problem with the house wasn’t that it was too difficult. It was that it was very difficult to measure the progress we were making. We’ve tried many things. We tried to make a mess of it procedurally. We put flour and rice on the tables and tried to clean them. We would put things around the house to keep the robot tidy. We were settling into Airbnbs to see how we were doing, but the problem was that we couldn’t get the same house every time. But if we did, we’d be over-adjusted to that house.
Isn’t it ideal that you don’t always have the same house?
Exactly, but the problem is that we couldn’t measure how well we were doing. Let’s say we were a little better at tidying this house, we don’t know if it’s because our skills have improved or if that house was a little easier. We were doing the pattern, “show a demo, show a cool video. We’re still not good enough, here’s a cool video.” We didn’t know if we were making good progress or not. The grocery store challenge task where we said, we need an environment where it’s as difficult as a house or has the same representative issues as a house, but where we can measure how much progress we’re making.
You’re not talking about specific home or supermarket goals, but solving problems that can span both places.
Or just measure whether we are advancing in the state of the art in robotics. We’re able to do the perception, the movement planning, the behaviors that are, in fact, general purpose. To be totally honest, the problem of the challenge doesn’t matter. The DARPA Robotics Challenges were just made up tasks that were difficult. This also applies to our challenge tasks. We like the house because it is representative of where we eventually want to help people at home. But it doesn’t have to be the house. The small market is a very good representation because it has that enormous diversity.
There is a frustration, however. We know how difficult these challenges are and how far off things are, but a random person sees your video and all of a sudden it’s something that’s over the horizon, even if you can’t deliver that.
Absolutely. That’s why Gill (Pratt) always says, ‘Reemphasize why this is a challenging task.’
How do you translate that to normal people? Normal people don’t stick to challenging tasks.
Exactly, but that’s why in the demo you saw today, we tried to show the tasks in the challenge, but also an example of how you take the features that come out of that challenge and apply them to a real application, like unloading a container. That’s a real problem. We went to the factories and they said, ‘yes, that’s a problem. Can you help us?’ And we said, yes, we have technologies that apply to this. So now we’re trying to show that these challenges are a few advances that we think are important, and then apply them to real applications. And I think that’s helped people understand that, because they see that second step.
How big is the robotics team?
The split is about 50 people evenly split between here and Cambridge, Massachusetts.
You have examples like Tesla and Figure who are trying to make all-purpose humanoid robots. You seem to be heading in a different direction.
A little. Something we observe is that the world was built for humans. If you just get a blank slate, you’re saying I want to build a robot to work in human spaces. You tend to end up with human proportions and human-level capabilities. You end up with human arms and legs, not because that’s necessarily the ideal solution. It’s because the world was designed around people.
How do you measure milestones? What does success look like for your team?
Moving from home to the supermarket is a great example of this. We were making progress at home, but not as fast and not as clearly as when we go to the supermarket. When we go to the grocery store, it’s very evident how you’re doing and what the real problems are in your system. And then you can really focus on solving those problems. When we toured Toyota’s logistics and manufacturing facilities, we saw all these opportunities where they are basically the grocery shopping challenge, except a little different. Now, the part instead of the parts being grocery items, the parts are all parts from a distribution center.
You hear from 1,000 people you know, home robots are really hard, but then you feel like you have to try it yourself and then you actually make the same mistakes as them.
I think I’m probably just as guilty as everyone else. It’s like, now our GPUs are better. Oh, we have machine learning and now you know we can do this. Oh, okay, maybe it was harder than we thought.
Something has to bring it down at some point.
Perhaps. I think it will take too long. As with automated driving, I don’t think there’s a silver bullet. There’s not just this magic thing, this is going to be ‘okay, now we’ve got this out of the way’. It will be thinning, thinning, incrementally. That’s why it’s important to have those kind of roadmaps with shorter timeframes, you know, shorter or shorter milestones that give you the small wins, so that you can keep working towards really achieving that long-term vision.
What is the process for actually producing any of these technologies?
This is a very good question that we are trying to answer ourselves. I think we kind of understand the landscape now. Maybe I was naive at first thinking that, okay, we just need to find this person that we’re going to pass the technology on to a third party or someone inside Toyota. But I think we’ve learned that whatever it is – whether it’s a business unit, a company, a startup or a unit within Toyota – they don’t seem to exist. So we’re trying to find a way to create and I think that’s the story of TRI-AD, a little bit too. It was created to take the automated driving research we were doing and translate it into something more real. We have the same problem in robotics and many of the advanced technologies we work on.
You’re thinking about potentially getting to a place where you can have spinoffs.
Potentially. But it’s not the main mechanism by which we would commercialize the technology.
What is the main mechanism?
We do not know. The answer is that the range of things we are doing is likely to be different for different groups.
How has TRI changed since its founding?
When I started, I feel like we were clearly just doing robotics research. Part of that is because we were a long way from the technology being applicable to almost any challenging real-world application in a human environment. Over the past five years, I feel like we’ve made enough progress on this very challenging problem that we’re now starting to see it morph into real-world applications. We consciously change. We’re still 80% pushing the state of the art with research, but now we’re allocating maybe 20% of our resources to finding out if that research is maybe as good as we think it is and if it can be applied to world-applications. We can fail. We can see that we think we’ve made some interesting discoveries, but it’s nowhere near reliable or fast enough. But we are putting 20% of our effort into trying.
How does elderly care fit into this?
I would say that, in a way, it’s still our north star. Projects are still looking at how we ultimately amplify people in their homes. But over time, as we pick through these challenging tasks, if things come up that are applicable to these other areas, that’s where we’ll use these short-term milestones to show progress in the research that we’re doing.
How realistic is the possibility of a fully erased factor?
I think if you could start from scratch maybe in the future that could be a possibility. If I look at manufacturing today, specifically at Toyota, it seems very unlikely that you’re going to come close to that. We (we talk to factory workers) are building robot technology, where do you think it could be applied? They showed us many, many processes where there were things like, you take this harness, you run it through here, then you pull it out here, then you attach it here, and you attach it here, and you take it here, and you take it here , and then run like this. And it takes five days for a person to learn the skill. We thought, ‘yeah, this is very difficult for robot technology.’
But the things that are most difficult for people are the ones you’d like to automate.
Yes, difficult or potentially injury-prone. Sure, we’d love to do stepping stones to get there eventually, but where I see robot technology today, we’re a long way from that.