Ancient Japanese documents dating back more than 1,000 years lost their readers as writing styles evolved and fell out of use. Now, AI developed in Japan is reconnecting people with the country’s literary and cultural heritage.

In Japan, a wealth of historical documents dating more than 1,000 years has survived due to the durability of traditional Japanese washi paper. However, they are written in kuzushiji, a cursive style of Japanese writing in which characters are rendered in a flowing manner, and which is now thought to be readable by only a few people. This is because kuzushiji rapidly fell out of use as kaisho, or standard script, became widespread from the late 19th century onward.

What if AI could be used to read historical documents? The researcher who turned this once-distant dream into reality is Thailand-born Tarin Clanuwat. She first encountered classical Japanese literature through a manga based on The Tale of Genji, considered Japan’s oldest full-length novel. Intrigued by the original work, she came to Japan on a scholarship from the Ministry of Education, Culture, Sports, Science and Technology and studied early commentaries on The Tale of Genji at the graduate school at Waseda University. “I was fascinated by the atmosphere of society at the time—how people lived, the ceremonies they observed, and the food they ate—as well as by the delicate, brush-written kuzushiji,” says Clanuwat.

Unlocking Historical Documents with AI

After completing her doctorate, Clanuwat decided to develop an AI model capable of recognizing kuzushiji, hoping to make classical texts more accessible. She went on to study machine learning at a University of Tokyo laboratory before joining the Center for Open Data in the Humanities (CODH) at the Research Organization of Information and Systems, where she began developing the model.

Clanuwat trained the AI model on one million characters from the Kuzushiji Dataset created by the National Institute of Japanese Literature, and in 2019 launched a web service using the kuzushiji recognition model KuroNet. Seeking to make the technology even easier to use, she released an app called “miwo” in 2021. Users simply photograph a historical document with a smartphone or tablet and press a button, and the characters are transcribed within seconds. “I made simplicity of use a priority,” says Clanuwat. Miwo has been downloaded 250,000 times to date and is reportedly used more than 3,000 times a day on average, primarily by students studying classical Japanese literature.

The model currently used by the app, RURI, offers improved accuracy, including the ability to detect characters against complex backgrounds. The miwo app supporting 20 languages is also under development, with the aim of making the technology available to people overseas who are interested in Japanese culture. Clanuwat says that using AI to decipher historical documents could unlock knowledge from the past, such as information about medicines used at the time and records of natural disasters. Such initiatives could extend beyond humanities research, opening the way for more people to make use of Japan’s rich documentary heritage.

The Challenges of Connecting Research to Society

Clanuwat sees the relationship between AI and classical literature as a two-way street. Even the latest AI models may struggle to read centuries-old letters written in kana, the Japanese phonetic script. This is because there is still insufficient training data for handwriting, including the individual characteristics of different writers. Examining such cases can reveal AI’s weaknesses and ultimately help drive advances in the technology.

Clanuwat now works at Sakana AI K.K., a Japan-based AI startup, where she is working on generative AI models that create ukiyo-e-style images. AI models developed overseas often generate inaccurate depictions of features such as hairstyles and buildings. Concerned that such inaccuracies could perpetuate misleading images of Japan, Clanuwat believes that “it is important to create AI that has a deep understanding of Japanese culture and society.”

Clanuwat’s research came to fruition in the form of an AI-powered kuzushiji recognition app, greatly lowering the barrier to accessing classical texts that were once accessible only to specialists. Behind this endeavor are the efforts of public institutions that support research and companies that bring its results to practical use. By connecting research, technology, and society, such initiatives are driving innovation in Japan. The knowledge accumulated and passed down through generations in Japan will be carried into a new era by AI.