Aoyama Gakuin University faculty members:
He is an uncompromising researcher.
Aiming for a prosperous society,
We are always conducting cutting-edge research.
We will explore the research results of our faculty members who are shaping the future.
TOPIC
About the JST Emergent Research Support Program (JST Emergent)
This is a support program for young researchers to "promote emergent research, which aims to create seeds that will lead to disruptive innovation through diversity and fusion" (JST official website). Under the guidance of program officers (emergent POs) who mentor researchers, the program aims to produce research results by taking advantage of opportunities to interact with other researchers.
The acceptance rate of JST Emergence
The rate of projects selected for JST Emergence is approximately 9.2% (243 projects selected out of 2,644 applications | FY2023 results: from the Japan Science and Technology Agency). The project also has Stage 1 (3 years) and Stage 2 (4 years), with an intermediate review (stage gate) taking place in the third year. Professor Raku's research project passed this review and advanced to Stage 2.
Evaluation points
There are still many challenges to overcome in reproducing the real world through CG, and currently artists are filling in the gaps by hand. Professor Raku's research has a long track record, with several papers published at SIGGRAPH, the top conference in the CG field, and he was recognized for his potential to lead to innovations that will allow for more precise reproduction of object movements on computers.
Explore the topic with your teacher
ProfessorYonghao Yue
Faculty of Science and Engineering, Department of Information Technology
Graduated from the Department of Information Science, Faculty of Science, The University of Tokyo. Completed the Master's and Doctoral Programs in Computer Science, Graduate School of Information Science and Technology, The University of Tokyo. PhD (Information Science and Technology). Although his parents are painters, he was interested in computer programming from an early age, and discovered the world of CG at university, where he devoted himself to research into the field of reproducing artistic techniques with computers. After a period as a postdoctoral researcher overseas, he will become an associate professor at our university in April 2019 and a professor in April 2023. His areas of expertise are computer graphics and computer science.
My specialty is the field of computer graphics, and I am particularly researching physical simulation techniques for producing realistic images. In order to calculate the appearance of objects, research is being conducted on optical simulation techniques that reproduce light and shadow, as well as dynamic simulation techniques that calculate the movement of fluids, cloth, and collision phenomena. In optical simulation, I am researching simulations of light transport, such as when light emitted from light sources such as the sun or fluorescent lights is reflected by objects or scattered by media such as smoke, and I am also researching the estimation of the materials of objects and media. In the field of dynamic simulation, I am researching simulation techniques for fluids with complex fluidity, such as cream or sauce, and estimating the fluidity of such complex fluids in the real world.
In addition to these studies, we are also conducting research into NPR (Non-Photorealistic Rendering). For example, this is a technology to reproduce the characteristic brushstrokes of Van Gogh's oil paintings. Based on a three-dimensional scene model and several images, we are working on developing technology to generate animations that maintain the brushstrokes while taking into account the lighting, shape, and appearance of the scene. In this way, our research can be broadly divided into two genres: research into the appearance and movement of objects through simulations of light and dynamics, and research into NPR, which automatically generates images while retaining the artist's characteristics.
Yes. That's roughly how it works. For example, if there is a brush stroke in an illustration, it naturally has the direction in which the brush flowed. Then, of course, there is the color, and there is also information such as length and thickness, but we are considering how to mathematically recognize this and correctly calculate and express it when the way the light hits it or the orientation of an object changes.
In the early stages of research, we had artists draw strokes to determine the direction, but what I found interesting was the part of the object that curves in both directions, like a potato chip. When lighting hits this area and highlights it, the direction of the brush seems to follow the curve, but when you actually analyze the shape of a potato chip, the direction is a little different. In fact, the line is drawn halfway between the outline and the direction of the curve. Artists naturally draw these subtle expressions, but in order to reproduce this on a computer, we are considering a simulation formula that incorporates elements of linear algebra, thinking that in this case, since it is halfway between the outline and the curve, it would be fine to combine these elements in the calculation.
That's right. For example, I'm interested in food, so I wanted to simulate food. I'm researching how thick fluids like sauce move. The sauce is not a smooth fluid, but one that has grains in it and is in a porridge-like state. Sesame dressing also has grains inside. In this way, when trying to simulate the movement of a fluid mixed with impurities, it is necessary to understand "what are the parameters (elements) that determine the movement of this object?" Because the grains are mixed in, it is not possible to measure the detailed movement with normal fluidity measuring devices. We are conducting research to make it possible to calculate on a video basis and reproduce accurate movement. In experiments, we actually devise a device, pour sauce from it, and film it on video. Since we specialize in physical simulation, we leave the parameters unknown and seek ways to estimate the appropriate parameters by optimizing and matching the movement of the video and the simulation. By repeating trial and error in this way, we ultimately aim to be able to accurately reproduce the movement of fluids mixed with various impurities in video.
I want to expand the expressiveness of computers, but I don't really want to use deep learning. One reason is that copyright issues inevitably arise when a huge amount of data is required. Furthermore, even if machine learning is used, there is the issue of not being able to clearly explain why a certain result was output. In aiming for non-realistic expressions, I am conscious of wanting to be able to use an open mathematical model to determine which elements to focus on in order to reproduce the characteristics of an artist, how to collect information, and what calculations are required to finally create the expression.
Being selected for the JST Emergent Research Support Program has been a great help. Of course, this is in terms of funding, but more than that, this program has given us many opportunities to interact with researchers from different fields, allowing us to look at our research from a variety of perspectives. Although we do not directly conduct joint research, through our conversations we often find that everyone has similar concerns, get hints that can be used in future research, and are greatly inspired.
I also believe that having friends is very important in conducting research, so the fact that many of my friends encourage me is a major factor in my dedication to research on physical simulation techniques. Thanks to my classmate from my student days, Professor Hideki Todo, who is now at Takushoku University, I was able to push forward in NPR research, feeling that "I can do it somehow," even in fields that were difficult to get into. No matter how interesting a research topic you are, if you don't have anyone to work with you or you don't have enough budget, you will inevitably feel like you're hitting a wall. That's why being able to interact with teachers in other fields through the JST Emergent Research Support Program is a great support. I also think that listening to what students want to do in the lab and developing the content by digging into interesting themes is also what makes me happy to continue my research.
One of the major goals of my research is to find interest not just in video production, but in a broader scientific area, and to achieve this, I want to express complex physical phenomena with simple mathematical models. In the case of the simulation of source movement that I introduced earlier, by being able to simulate how such complex mixtures move, it is expected that in the future it will be applied to fields such as disaster prediction and earth science, for example, landslides and glacier collapses.
My goal going forward is to keep a constant eye on the latest trends in cutting-edge research, update my lectures to incorporate newly developed fundamental theories and cutting-edge applied research topics, and build new mathematical models in my research, so that phenomena that were previously difficult to handle can be handled more easily.
I think that one aspect of learning at university is that you become able to do things that you couldn't do before. Facing things that you can't do can be frustrating, and you may feel that it's difficult. However, if you carefully understand what you have learned up until that point and proceed, you will definitely find a way to the next step. The mathematics we are studying now is something that humanity has developed over the past 3,000 or 4,000 years, and we are learning it in just 10 or 20 years. The speed is incredible, but that is why we can be confident that we are standing firmly on the history and intellect built by our predecessors. If you are aware of this, you will be able to do many things and your learning will become even richer.
The NPR and physical simulation techniques I am researching are being carried out in a way that mobilizes almost everything that has been used in physical simulations up until now. In particular, the process of attaching directions to curved surfaces is based on the mathematical framework used in Einstein's general theory of relativity. For this reason, I sometimes tell my students, "We are considering a framework for describing the world of art using a mechanism for describing the world and the universe." University is a place where you can take on new challenges using the "knowledge" you have cultivated up until now, so I hope you will make the most of this environment and enjoy your studies to the fullest.
As for the Department of Information Technology in the School of Science and Engineering, I think the relaxed atmosphere is one of its charms. Of course, there are many classes and assignments, but I think you can study at your own pace without being pressured. Whether it's a lecture or research, it's very important to be able to say "I don't understand, can you explain this?" when you don't understand something. In this department, you can face learning with a long-term perspective, so I think it's easy to develop a learning style where you stop for a moment, reflect on what you've learned so far, and then move on to the next step. I want students to take their time and learn slowly, and gain confidence that they are standing on the history of academic studies.