About-Robots

Using Nao Image Recognition in your app

Nao and Pepper have this awesome feature to recognize objects from a database of images. You can create the database quite easily with Choregraphe and upload it to your robot. It's rather simple. For more details on the basics of creating and using a Vision Recognition Database, check the official documentation here.

 

But what happens if you want to have 2 apps with 2 completely different databases? Or if you want to distribute your app with an image recognition database through the App Store?

 

So in this little tutorial, I show you some simple trick to package your object recognition in your app. You can find the completed sample project on Github. 

 

/!\ Important note: the feature to export the database to Nao is broken in 2.1.3. The following tutorial only works with a Choregraphe 2.1.2 or older (your Nao can be in a more recent version).

 

So once you have created your vision database as explained here, you should:

1. export the database to your robot
2. on your PC, create a new folder to put the DB files in
3. using scp, from your new folder, you can do (in 1 line):
   scp -r nao@<ROBOTIP>:~/.local/share/naoqi/vision/visionrecognition/current/ .
   Or you can use an ftp client to download the folder from the robot.
4. You should see a bunch of files like .tiff images and database files among other things in that folder. Copy all the files in a folder of your project.
5. Create a new Python box in your project and give it 2 parameters: a "Attached file" parameter called "path" and a "string" parameter called "db name".
6. Copy paste the code from this Github Gist into your box.
7. the "db name" parameter can be anything you like. The "path" parameter should be the relative path from the behavior.xar file of your app to the DB folder. In my app, I set it to "../vision/".
   An easy way to do it is just click the browse button and select any file in your DB folder. Then erase the name of the file to keep only the folder path.
8. Call the onStart of your new box in your behavior, and then simply use the Vision Reco. box available in the Standard box library in the vision folder.

By default, the Vision Reco. box will keep sending a signal every time it recognizes something. So you often get several signals for the same object over 1 or 2 seconds.

To avoid that, I right-click on the onPictureLabel output, choose Edit, and change the "nature" to "onStopped". When the nature of the output is "onStopped", the box is unloaded and the robot stops the vision recognition. So you can process that one object, and then go back to Vision Reco. later in a loop.

 

Don't forget to check the completed sample project on Github.

Building Great Humanoid Robot Apps Part 3: Dealing With Uncertainty

This is the 3^rd and last part of the 3 part series on how to build great humanoid robot apps

So much uncertainty

An important lesson we learned early with Nao, is that you have to handle an extreme amount of uncertainty.

What could possibly go wrong?

When using Nao, I found people would not necessarily quit an app before leaving the robot. The robot would then get stuck waiting for the answer to the quizz forever.

Then another user, or even the same user 10 minutes later would come back, and not realize that he is still in the quizz game.

What do you do when the user leaves the robot? What do you do when the robot doesn't understand the user?

Well, first, knowing what can go wrong is a great step forward. You can then decide which cases you actually want to handle (know your limits!).

As an example, here is a (non-exhaustive) list of cases that every apps in SB shops must handle:

• the app plays from beginning to end without errors (no bugs!)
• the app speaks with the right voice and personality of Pepper
• for each question Pepper asks, you have to handle the case Pepper doesn't understand the answer, or the person answers something you didn't expect
• for each question Pepper asks, you should handle the case where the user asks Pepper to repeat the question
• when the user replies the expected answer, Pepper should understand it 90% of the time
• if the user leaves Pepper alone, Pepper must quit the activity by himself and stay ready for the next user
• if the user is too close from Pepper, animations will be slowed down or stopped. Animations that are important for the meaning (like when you take a pose, or mimic something) must be recognizable even when the user is too near.
• apps should be easy to quit by saying "quit" at any time.

These guidelines already give you an idea of right could go wrong.

Identification of these corner cases through proper UX Testing is critical to the success of your application.

Example: Taking a Picture

When Pepper takes pictures of people, there is often a strong backlighting. Pepper is smaller than most adults, and therefore is looking up to you. The ceiling lights usually make for pretty bad pictures.

=> once you know this, you can decide how you want to manage this: tell the user to bring his face down? tell him to step back a little to reduce the angle? use post-processing to compensate for backlighting in the picture?

For info, I tried them all, and decided to ask the user to lower a little (makes for cuter pictures too), and used post-processing to enhance the image.

Yoshimoto Robotics Lab asks the user to step back, and shows the picture only briefly before hiding a thumbnail of the face in an image (it's called Cameraman Wally!).

As you see, once you know what can go wrong, you have many ways to decide how to fix the problem.

 

This is the 3rd part of a 3 part series. Click below to read the first 2 parts:

The first part on defining clear goals for your app.

The second part on designing the interface of your app.

Building Great Humanoid Robot Apps Part 2: Imagine The Interface

This is the second part in the 3-part series on how to build great humanoid robot apps. The goal is to find out the answer to: what makes the difference between succesful applications and failed ones?

In the first part, you can see how to define clear goals: goals that are testable, actionable and give a reasonable scope to your future work.

In this second part, we will see how to think about the human-robot interface (HRI). What is different between a mobile app and robot app? What traps to avoid when designing the HRI?

Let's see 3 techniques on how to design a great interface for your robot app.

Building Great Humanoid Robot Apps Part 1: Defining Clear Goals

After creating dozens of apps on Nao and Pepper, I have been thinking: what made the difference between my success and failures?

I worked for 6 months full time on a big application that ended up boring and mostly thrown away. The idea was great, but the app itself just wasn't.

On the other hand, I created several apps in 48h or 72h to showcase at important demonstrations that were extremely successful (people laughing and remembering it, some of them are even on Pepper in Softbank shops, …).

How can I avoid spending 6 months on failures and make sure an app will be successful from the start?

Pepper Tech Festival: Demos Summary

Yesterday the Pepper Tech Festival took place in Tokyo. More than 1200 people attended and over 500 Pepper were pre-ordered.

That is a huge success, as nearly 50% of people present ordered the robot.

However, only 200 robots are available this year. More will be coming in February.

For more info on the announced price, the features included and reservation, check out the Pepper robot price page on my website.

Now, there were a lot of demos by lucky and highly creative teams who had the chance to work on Pepper since this summer.

Let's see what's possible to do on Pepper!

Aldebaran Ateliers Opens in Tokyo Omote Sando

After Paris, Aldebaran opened its first Atelier in Japan! You can now discover Pepper and Nao in their natural environment in Omote Sando, also known as the Japanese "Champs Elysée".

What's the difference between the Softbank shop and Aldebaran Atelier?

More than 5 robots, both Pepper and Nao, are available there at all time. Compared to SB shop robots, the Ateliers features more games and activities for you to play with. You can hold the robot's hand and walk around with them, take the pose with Pepper, see Nao dancing, play the exclusive Akinator apps and much more.

Google Buying Nest: A Step Forward for Robots at Home

These days, robotics news have talked very much about Google buying many robotics companies in 2013. A good summary is available on Everything-Robotics. There is also this post that puts hopes that Google will create robots for your home.

Last week Google announced they also bought Nest, an IoT (Internet of Things) company specialized in Smart Home monitoring products. Articles on this topic talk about Privacy Issues and new ways of collecting data for Google.

I think 1 point that was missed is that Nest is as much a robotics company as many other companies bought by Google. Let me explain that.

Nao at IREX 2013: Success of the Interactive Humanoid Robot

During November 6-9th 2013 was iREX, the International Robotics Exhibition. The exhibition is a major worldwide event that happens every 2 years. It is THE place to be for all robotics companies that want to get noticed on the Japanese market, and many experts are scouting here for the latest trends for their business or their concurrence.

 

Of course, Nao was there, and I spent most of the time representing Aldebaran Robotics at our stand with our Sales Team.

 

The Best Interactive Robot

This year, there were much less humanoids, and very few interactive robots. So many people could see how unique Nao is.

While one Nao would attract eye balls and cameras with a trendy dance, we were letting people play simple games, or take Nao by the hand for a walk.

 

A lot of press and TVs came by, and for once I was happy that they wanted to show the interaction with the robot, and not just the dance. So Tokyo TV showed their reporter taking the robot by the hand, and playing the Guess Sport game during their morning show, for example.

 

Nice Stories

Some funny things happened during the show, when people met Nao. Some people get frustrated when Nao doesn't understand more than its list of applications (we are working on that!).

I am glad we had some really nice moments: an 80+ year old man in a wheel chair took Nao by the hand for a walk, with his wife pushing the chair. Far from being afraid of robots, the elderly Japanese people see it as the future of nursing care, with robots working alongside nurses.

And the opposite, we had a young child (about 8 years old?), who stayed for hours at our stand to play with Nao. I showed him pretty much every applications we had, he played all the games (Animal Card Game, Guess Sport,…), saw many dances, took Nao by the hand… Until finally it was closing time and his mother told him it was time to go. Even when Nao was sitting and resting, the child would stare at it and wait.

 

Conference About Interaction With Nao

Atsushi Tanizaki (Sales Director for Japan) and I gave a bilingual En-JP conference on Human-Robot Interaction With Nao.

I believe Safety, Autonomy and Interaction are the 3 fundamental areas that Robotics need to work on to put robots in everybody's homes.

During the talk, we focused on how we work on making Nao more and more interactive.We talked about the progress we've made, and our goals for creating a natural interaction and conversation between Nao and the user.

[VIDEO] Trying on the Powered-Jacket MK3 (or at least the legs)

Last week, Nao was invited on Japanese TV along with a few other robots. Being in Aldebaran Studio team, we are a little like Nao's dad, telling what to do and how to behave. That also means I got invited to the set and could play with other people's robots!

 

I met two exoskeletons that day: the HAL suit from Cyberdine (read more about the HAL Exoskeleton here) and the Powered Jacket MK3 from Sagawa Electronics, with its creator Machi Kosuke.

Nao User Group Tokyo Part 2

I presented in the previous post the first applications presented by Nao Developers during the morning of the Nao User Group Tokyo on August 31st.

 

Next presentation was by Angelica Lim. Angelica presented her Naovatar game. Naovatar is a kind of "Choose Your Own Story" with Nao telling the story instead of you reading it. There are also some clever puzzles to solve that use Nao's sensors and perception. You can see the game in a video from Angelica: