Copyright © 2021 Jiangnan University. All rights reserved.
SIGGRAPH caught up with SIGGRAPH 2021 Art Papers contributor Ruimin Lyu to learn more about his team’s research presentation, “Imaginary Stroke Movement Measurement and Visualization.” This research examines brush stroke movement in a quantitative way, while providing a new outlook on artistic creation. Read on to learn more about what inspired the project, how it advances current art research, and Lyu’s take on the intersection of art, science, and technology.
SIGGRAPH: Share some background about “Imaginary Stroke Movement Measurement and Visualization.” What inspired your team to pursue this project?
Ruimin Lyu (RL): This project originated from a special painting method I began to explore in 2018. The following is the earliest work:
[And, this image] is a work [that was created] after improving the painting method, which was completed in 2019. It was used as an experimental sample in this paper:
What are the characteristics of this kind of work? Perhaps you will realize that they all use the eastern ink style to express modern scientific and technological products; however, this is not its unique feature. It can be compared with the following work, which was completed in 2015:
Both [works] use ink painting to represent scientific and technological products, but they still have some key differences. In fact, the fundamental difference is that the former is completed by continuous brush movement, while the latter is not. Although the stroke of the former is not physically continuous to the end, it does imply a continuous brush moving trace. The following is its brush movement trace:
For the latter, although it is still my own work, I can’t reconstruct the stroke order. I think there are two reasons to conceive this continuous stroke painting. First, I trained in calligraphy since I was a child, and one of the basic requirements of calligraphy practice is that it should be completed at one time. This requires that each stroke should be completed with clear and simple actions, and multiple strokes should be completed with the specified stroke order. I draw with the same tools as when I write calligraphy, which naturally brings the possibility of one stroke completion.
Second, my working conditions at that time did not allow me to spend a lot of time elaborately painting. Therefore, painting is mainly a leisure activity for me. I have the impulse to complete it at once with few strokes. This painting method is a natural integration of my past practical experience in my subconscious under the constraints of objective conditions, rather than my intentional conception. This painting method is very different from writing.
What is the meaning of continuous strokes other than to show off the mastery of skills? It actually expands an expression dimension of static works — time. Usually, a static visual art work is a plane. But the artist can express a higher dimension in a two-dimensional space. For example, perspective can be used to express three-dimensional space. In addition, time is a dimension outside the plane, and time dimension is almost implicit in various expressions of motion. For example, as shown in the following pictures, Leonardo da Vinci expressed the birds flying by using scribbles. Duchamp used the ghost shadow technique to show the moving human body, and in the cartoon, continuous lenses and some moving lines are used to show the intense movement process over a period of time.
In addition, time can be shown through a brush stroke. For example, the above Leonardo da Vinci manuscript is his famous mirror script, so it is obvious that we can trace the movement process of his writing. Therefore, calligraphy experts often compare appreciating a masterpiece of calligraphy to appreciating a piece of music, such as the following figure:
When we continue to use the brush movement process to examine past manual creation works in different cultural backgrounds, we find that it is a broad technique. It is likely that the creators themselves are not clearly aware of it, but they use it subconsciously with their own talents:
In order to explore how to further use the brushstroke movement technique, I created a calligraphy work in 2020. It uses stroke movement for encoding information, and loads two groups of information within my customized coding rules:
The first group uses the relationship between adjacent strokes in time sequence, as shown in the figure below. When we explore the position of connection between each stroke and the preceding stroke along the specified stroke order of the character, we find that there are three types: connection, separation, and intersection. They can be translated into a Morse code, corresponding to the English word “bit.”
In addition, the communication concept of channel reuse is integrated here. If you continue exploring the process of starting shape, bending direction, and thickness change of strokes, you can decode a five-bit binary code from each stroke. For example, the following figure is the decoding of two strokes:
Why did I encode the words bit, order, and tide? They were closely related to the theme of a lecture I prepared. In short, they all involve the principle of information and communication (as shown in the figure below). It directly inspired our research, that is, a visual art work is no longer an isolated physical entity but a dynamic process of information transmission. The part outside its physical components — creation and appreciation — should be studied.
Further, I considered applying the ideas of mathematics and experiment to study this phenomenon. I considered how to quantify the feeling of stroke movement with experimental methods, then used mathematics to answer the previous question — why is it difficult for us to feel the pen movement process from continuous strokes? We need to consider how to quantify it and design experiments.
SIGGRAPH: Define “velocity sense” and “trace sense.” Why did you choose to measure these in your research?
RL: The discussion in our paper shows that “velocity sense” and “trace sense” are two closely related feelings, or two representations of the same feeling. My first thought was to quantify the trace sense, which stems from the original motivation of answering why one can’t feel the brush movement process in my paintings. This involved measuring the feeling of brush movement process. After several different pre-experiments, it was not necessary to directly measure the single index of the feeling of the brush movement process but to find a way to record the viewer’s feeling of it and then conduct quantitative analysis on the data. Therefore, the concept of experiment No. 2 in the paper is letting the user draw the trace to record the process of the brush movement they feel.
The conception and measurement of velocity sense came later. After reading a lot of calligraphy theory literature and watching some contemporary digital art works, I paid special attention to the visual psychological phenomenon of movement implied by strokes, and I pondered it when watching calligraphy art works. I thought that strokes are like conveyor belts with a fine structure. I realized that this conveyor belt with different sizes and directions of thrust at different positions is the force field or velocity field in mechanics, and it also is the vector field often described by differential equations. I thought about how to quantify them, especially the velocity field reconstructed by the viewer in their own mind. After that, I did a lot of pre-experiments and made minor adjustments and detailed design before adding experiment No. 1 to the paper.
I came up with the titles “velocity sense” and “trace sense” when writing the paper. They are named to reflect the relationship between the two — the former is the instantaneous and local motion feeling, and the latter is the accumulated and global motion feeling. As far as I am concerned, the connection between the two is obvious and natural, but it was not so at first. Linking the two is largely due to our later visualization work.
SIGGRAPH: What visualization methods did you use for the two kinds of experimental data?
RL: In this study, visualization plays an important role in two aspects. One is conducting a statistical analysis of data. We first designed experiment No. 1 and obtained a large amount of trace sense data. I didn’t know how to analyze the data at that time. I thought of the visualization method of expanding the motion trajectory of the object on the time axis in order to explain the theory of relativity, so I considered that the trajectory obtained in our experiment can be expanded and displayed in the three-dimensional space of X-Y plane plus T- axis. When I first saw the shape of the trace sense data of different works expanded on the T-axis, I thought of the stochastic process.
With this experience, after obtaining the data of experiment No. 2, we tried various visualization methods, one of which is to let a virtual ball move in the vector field and record its trajectory. Finally, based on my previous physical knowledge, I realized that the measurements of experiments No. 1 and 2 are the differential and integral of the subjective stroke motion phenomenon. When we get a large amount of data, we don’t have to worry about quantitative analysis, but we can try to visualize it in a variety of ways. At this time, we should visualize it in rich and different ways as much as possible, because each visualization is likely to correspond to a way of thinking that we haven’t tried before. When it is presented to us visually, it can stimulate our visual thinking ability and urge us to associate that with abstract thinking.
The second is for artistic creation. When we visualize data in a variety of ways, we find that there are always two motives at the same time — intuitively show statistical characteristics and be more beautiful. These two motives are usually difficult to be absolutely distinguished. In particular, when we visualized the data of experiment 1, we were amazed by the effect. We realized that this in itself is the creation of a visual art work.
To summarize, after obtaining the experimental data, we should not be limited to a preset visualization method but should try different visualization schemes as much as possible. Maybe this will bring new cognition. This is my deepest experience of visualization method in this study.
SIGGRAPH: How does your work on “Imaginary Stroke Movement Measurement and Visualization” advance the implementation of brushstrokes in digital media/art?
RL: It has two functions. One is to study a phenomenon commonly ignored in the past — the sense of stroke movement in a quantitative way. The second is to provide an idea of artistic creation, using experimental methods to obtain the viewer’s perception of traditional works of art and then visually re-create it based on the data. This can be regarded as the digital inheritance of traditional art. We expect it to play these roles. Of course, we also hope it can inspire exploration in other directions.
In addition, this study has greatly expanded my understanding of stroke. In the past, I used to see strokes as a technical means to achieve a certain visual effect. Now I see strokes in a more diverse way. They are not simply a tool to achieve some artistic effect. Our feeling of stroke is a cognitive ability. This feeling helps us understand the abstract mathematical concepts.
SIGGRAPH: Your Art Paper was presented during SIGGRAPH 2021. What was it like to present virtually? What was your favorite part of the SIGGRAPH 2021 conference?
RL: In terms of the characteristics of our works, the online presentation was enough to express our core concepts. We still wish we could have had face-to-face communication with the audience, though.
SIGGRAPH: What advice do you have for someone planning to submit their own work to a future SIGGRAPH conference?
RL: In fact, I’m a novice myself. It’s hard to give guidance on how to write a paper or how to prepare an art work. I want to share my own changes in my understanding of art and technology, which may help you find new directions worthy of research. In my educational environment, art, science, and engineering often imply different purposes rather than ways of thinking, which is a major obstacle to creativity. For example, in art colleges, students are usually told that all the skills they learn are for the purpose of creating great artworks, and the way for works to be recognized is nothing more than to be exhibited in a well-known art museum or expo, or to win an award in an art competition, or to be realized in a commercial form. However, they rarely emphasize the various training they receive, which is helpful to understand the laws of nature or produce a certain functionality. There are similar purposeful implications for science and engineering. For example, the purpose of science is to explore the truth, and the purpose of engineering is to make useful things. Their own goal is not aesthetics. In my current understanding, art, science, and engineering should correspond to several different ways of thinking. They come from the exploration of three different purposes but are not limited to their original purposes. They can serve other purposes and complement each other. Based on this understanding, if I’m asked to give some suggestions to younger friends who want to study calligraphy, painting, and other art forms, I recommend liberating painting from the purpose of aesthetics and explore the truth and usefulness behind it. This will not damage its artistic value; it will bring new aesthetic experience.
Feeling inspired? SIGGRAPH 2022 is still accepting submissions for a variety of programs. Learn more.
When he was very young, Ruimin Lyu became interested in painting and calligraphy and taught himself for a long time, acquiring rich experience of traditional calligraphy and painting. Since university, he has not entered art colleges to receive “formal” art training but has studied electronic science and technology, optical engineering, and digital art and design. After obtaining the doctorate degree in computer application, he participated in developing graphic art applications and games, studied in the fields of computer graphics and human-computer interaction, and created a large number of calligraphy and painting works. In recent years, his research interest has shifted to looking for the relationship between calligraphy, painting art, and science and technology, trying to use art to enhance intuition for science and scientific thinking to enhance the rational understanding of art.
