Pushing the human body to the limit! A researcher dedicated to developing of a new AI gymnastics scoring system

If I were to describe myself, I would say that I am someone that always leans towards actively participating rather than just being an observer. Perhaps this sounds a bit abstract, but for example, I prefer to make the things that I use everyday life, or when it comes to sports, I would rather play rather than just watch the game. As a child, something I loved doing was to build furniture from scratch with just the manual to guide me. I got real satisfaction from the tangible results of putting together useful things such as bookshelves. Perhaps this is one of the reasons why I decided to study image processing in graduate school, because I was drawn to the fact that the results generated by programming are reflected in images that are easy to understand.

I have a good friend to thank for my initial interest in IT, during my third year of high school. He had the ability to use a computer to search for information and even create games through programming. I found that fascinating and wanted to try it myself. At that time, I didn't have my own computer, so I borrowed my father's computer and spent an hour or two at night trying to create programs. However, as I was self-taught, things always go too well.

Whenever that happened, I would ask my friend about the mechanics of programming during breaks in class, and he would teach me how to solve the problems I encountered, so that I could then repeat the testing process at home. It took me three months, but I still remember the excitement I felt when I completed my first falling block puzzle game. This experience sparked my interest in software, and I enrolled in the engineering department at university. Even now, I am filled with gratitude towards that friend and the help he gave me.

I joined Fujitsu Research after a three-month internship, finding out just how committed Fujitsu is to researching image processing, which I believed would allow me to leverage my expertise. Also importantly, I really enjoyed the atmosphere, where I could easily converse with other researchers and ask questions.

However, once I started to work on actual projects, I struggled to find my own way of working. For the first five years after joining the company, I was involved in the development of medical monitoring systems and automobile driver monitoring systems. For example, when I was researching and developing technology to monitor whether a driver was falling asleep at the wheel, I had the opportunity to present our technology at exhibitions. But I struggled, because there was little direct feedback from users and I had no idea what the real issues were or what improvements I needed to make. It was like steering a ship in the dark! I became very anxious, not really knowing where to aim my efforts or fearing that the ship might capsize if it hit something.

Of course, I knew that I wasn’t on my own - as a team, we repeatedly discussed and extracted high-priority issues from a vast array of challenges and determined where to direct our development efforts. I also made an effort every day to gather opinions from my seniors and experienced developers to help me conduct the optimal research and development. For each assigned task, I would spend time thinking about the issues on my own and improve them to achieve numerical goals. However, there were times when I sometimes went off in the wrong direction due to my lack of experience. As a result, I felt frustrated, because the opinions of the senior members of the team and my supervisor dominated much of the development content. I repeatedly asked myself whether this was really good for the research and development that I wanted to achieve and my sleepless nights continued.

I had been struggling and worrying about my own working pattern for some time, but then along came a project that gave me a real sense of hope. I was given the opportunity to be involved in the development of the world's first AI gymnastics scoring support system “Judging Support System” (*1), using 3D sensing technology. I joined the project around the time when the world's first highly accurate 3D skeletal estimation technology had been successfully developed, after a one-year demonstration experiment. One of the challenges I faced in this project was discovering about deep learning, which I hadn't touched on much before. I studied it on my own using reference books. While familiarizing myself with deep learning, I put a lot of thought into ideas for improving various challenges, experimented with them, and tried dozens of neural network for a month.

Even during meals or while taking a shower, I found myself engrossed in brainstorming ideas and enjoyed the process of thinking in a completely different way to before. One day, while on my way home, I came up with a hypothesis and tested an idea based on it. To my surprise, it resulted in a high level of accuracy that we had not achieved before, and I felt a great sense of achievement.
In addition, this project involved stakeholders from the International Gymnastics Federation and professional gymnastics judges within the company. As a result, I was able easily to gather feedback from users who would actually use the technology, and their expert opinions were very helpful for the next round of technology improvements.

Titles, numerical values, and proper nouns in this document are those reported when this interview was made.