Columns that reveal the world
- Getting up close and personal with the researchers -

Reproducing and visualizing the movements and appearance of familiar objects on a computer. This is the main theme of computer graphics (CG). For example, as light travels from the sun to the earth, it hits particles in the atmosphere and clouds, scattering, and propagating in all directions before reaching our eyes. I am researching methods to calculate these physical phenomena and simulate them on a computer. In one project, I created a CG image of sunlight shining through gaps in the clouds, painting the mountainside with a sunset every moment. The result of this research is technology that can calculate changes in light, such as which areas are in shadow, which areas are bright, and which areas are hazy, with high precision and high speed. This has made it possible to reproduce optical phenomena, including the entire sky, in great detail.

Light simulation that reproduces scattering in the atmosphere and clouds, and reflection from the terrain

We are also working on simulating the movement of materials. Each of the materials that are familiar to us has its own characteristics. For example, cream-like materials will maintain their shape with some parts protruding unless a certain amount of force is applied. Taking this characteristic into account, we are able to recreate the way the cream stops with sharp edges the moment the pie plate hits the opponent in a pie-throwing scene. Furthermore, in the cooking scenes, there are scenes where multiple seasonings are mixed. To recreate this, we also model the fluidity of materials with different characteristics when they are mixed together, such as mayonnaise and honey.

When dealing with the movement of powder such as sand, the standard method is to use the distinct element method, which simulates the effects of each grain of sand rubbing against each other, colliding and bouncing off, but as the scale of the scene to be reproduced becomes larger, the number of sand grains to be handled becomes enormous. For this reason, we combine the two, using a modeling technique that treats the collection of sand grains as a continuum for the central part of the sand's movement and the distinct element method for the outer parts, to efficiently simulate while maintaining a high level of reproducibility.

Simulation of non-Newtonian fluids whose viscosity changes depending on the magnitude of applied force

The surface is made of individual sand grains (blue) that simulate collisions and friction.
Simulation using a continuum of sand grains (red) inside to simulate pressure and plastic flow instead of collisions and friction.

This type of CG technology is incorporated into video production software and is actually used to create the movies and animations that you see. In addition, each topic is linked to issues in other fields. For example, by applying a model of the movement of powder, it is possible to simulate the movement of glaciers. If we can predict how much of a glacier will disappear as temperatures rise, we may be able to make some contribution to solving the problem of global warming.

Furthermore, because CG is a technology that processes and reproduces shapes, it can also be used in manufacturing by outputting it to a 3D printer. Light can be controlled by processing lenses to make patterns stand out, and it can also be used in structural design to make tables out of toy blocks that can withstand a certain amount of weight. It can also be applied in the field of architecture, where blocks can be used repeatedly to create new things without producing waste, which could contribute to achieving the SDGs.

An example of using CG technology in structural design to create a table that can withstand a certain amount of load by assembling toy blocks into any shape.

CG is not only useful for simulations that accurately capture real-world physical phenomena and physical laws, but it is also a technology with great potential for application in manufacturing and solving social issues. It would be my hope if we could clear the problems that need to be solved in the world of CG one by one and make it useful in a wider range of fields.

In recent years, I have been working on a project to recreate the painting style of an artist using CG technology. I am researching a system to automatically generate new videos on a computer in the style of Dutch painter Van Gogh, and if this is completed, it can be applied to full-scale animation production. With just one work by a painter, I can extract the distinctive style from it and combine it with a separate simple 3DCG to easily create a unique animation.

I grew up with parents who were painters, but as a child I didn't like the world of art, where it was difficult to judge what was good and what was bad. So I started learning programming in my upper elementary school years, aiming for the opposite world. I guess I was attracted to a logical and unambiguous world. Later, at university, I learned about the photon mapping method, a technique for simulating the appearance of light, and was deeply impressed by the fact that programming could do something like this, and I entered the world of CG. It was also interesting to see that the mathematics I had learned up until then, such as trigonometric functions and linear algebra, were fully utilized. Since then, I have been involved in CG research for a long time, and now I am trying to incorporate a painter's touch into animation production. In a sense, it may be the influence of my parents that has brought me closer to the world of art.

Previously, I had avoided the world of art because I didn't know how to formulate it, but as I delved into mathematical models, I realized that the mathematics used there could be used in the art world. It's not easy to create animation with Van Gogh's touch, but by gathering all the knowledge I have gained so far, I'm beginning to see a path forward. Of course, there are methods to generate similar images without using interpretable mathematical models, but I want to incorporate the painter's touch into animation production by solving the equations that need to be solved and creating a system that can be explained mathematically. In the long term, I think this will be a technology that can be applied to a wider range of fields.

Generate animations that mimic an artist's brushstrokes

While CG technology is expected to be applied in many areas, it is still in its infancy, and we are working on research with the aim of reproducing more complex movements. For example, when trying to reproduce meat sauce, it is not easy to model various elements such as small chunks of meat, viscous and fluid sauce, and powdered cheese individually. We would like to be able to reproduce materials with such complex structures as a continuum. If we can reproduce more complex movements with a high degree of accuracy, it may become possible to reproduce a wide range of scenes using CG technology alone, which is currently impossible to reproduce without the efforts of an artist.

Simulation that takes into account that fluid properties change with the mixing ratio due to the mixing of fluids

Looking back on my time at university, I think I took classes in my first to third years without really understanding what the knowledge and theories were, wondering, "What use is this knowledge or theory?" However, later in the research process, I often realized that I needed to solve a problem and re-learned it, or learned connections to other things and understood. There is no need to push yourself to completely understand or master the content of lower-year classes. However, I hope you will expand your repertoire so that you can later recall things like, "I heard this story, there was that method."

In addition, the content learned from elementary school to high school and in the early years of university is the "knowledge" that humanity has built up over thousands of years of history. It is only natural that it is difficult to learn it in a dozen years, reproducing it 100 times. University specialized research is about challenging new things on top of thousands of years of accumulated knowledge, so it is no wonder that it is difficult. That is why you proceed slowly, one step at a time. If studying up until then was like sightseeing on the Shinkansen, then university research can be considered a leisurely stroll. The way of approaching things is completely different, so if you are used to Shinkansen trains, you may overlook important points in your haste to get results. Take your time and think carefully as you work on it.

Also, from the perspective of research, I recommend going on to graduate school. If learning from elementary school to the third year of university is about tracing the history of human knowledge, then you only have one year in your fourth year to work on your own research. Why not extend that period for a few more years and really focus on the topic you want to pursue? I'm sure it will be beneficial for your long life as a working member of society.

When I first became involved in CG research in the early 2000s, technology was advancing at a remarkable rate, and research was also required to be fast. In that environment, I switched my focus to solving problems carefully and reliably without compromising accuracy, and tackled the aforementioned "optical simulation technique to reproduce changes in sunlight and sky." It was a difficult challenge, but I think I was able to grasp the direction of my research when this research was recognized at a prestigious international conference. The most satisfying moment is when I put a formula into a computer program, calculate the movement of something that could not be calculated, and achieve high-speed calculations. I feel that the real joy of CG research is seeing the results of the calculations rendered as beautiful images.