Based on the sound hitting the smartphone AI could guess input characters - Is it necessary to take measures?

Recently NHK News On-line reported the following:

ＡＩ スマホに指当たる音で文字推測可能に 対策必要か

2020年1月29日 5時45分

スマートフォンで文字を入力する際に、画面と指が当たるごくわずかな音をＡＩで解析すると入力された文字を高い精度で推測できることが静岡大学のグループの研究で分かりました。グループでは新たなセキュリティー対策が必要になる可能性があるとしています。

これは静岡大学の西垣正勝教授らのグループが28日、高知市で開かれた学会で発表しました。

グループでは、スマートフォンに文字を入力する際に指や爪が画面に当たってわずかに音が鳴ることに注目し、実際のスマートフォンで「０」から「９」までの数字をそれぞれ100回ずつ入力して、その際に出る音をＡＩに学習させました。

そして、実際に画面をタップしてその音から、どの数字を入力したのかＡＩに解析させたところ、95％以上の精度で正しい数字を推測することができたということです。

グループによりますと、スマートフォンの画面は内部の部品の取り付け方などで場所により音質や反響にわずかな違いがあり、ＡＩはその違いからタップした位置を推測しているとみられるということです。

グループでは、こうした技術がさらに進むと、入力の際の音からスマートフォンの暗証番号などを読み取られるおそれがあると指摘しています。

西垣教授は「画面をたたく音だけでここまで高い精度で推測できるとは思っておらず、結果には驚いた。どの程度リスクになるのか、しっかりと評価して、有効なセキュリティー対策を検討していく必要がある」と話しています。

Translation

A study done by a group at Shizuoka University had found out that when inputting characters with a smartphone, by analyzing  the very slight sound of the screen and  the finger hitting, the input characters could be guessed with a high accuracy using AI. The group stated that new security measures might be needed.

This was announced at a conference held in Kochi City on the 28th by Professor Masakatsu Nishigaki’s group from Shizuoka University.

In the group, attention was paid to the fact that when inputting characters on the smartphone, fingers and nails hitting the screen would make a slight sound. In entering the numbers from "0" to "9" individually on the actual smartphone for 100 times, the sound that came out was learned by AI.

After that, from the tapping sound, what number had been actually input on the screen was analyzed using AI. It was able to guess the correct number with an accuracy of 95% or more.

According to the group, there was a slight difference in sound quality and reverberation on the screen of the smartphone depending on the location of how the internal components were connected, and it was said that AI could guess the tapped position based on these differences.

The group had pointed out that if the technology advanced further, it might be possible to read the password of the smartphone from the sound of the inputting.

Professor Nishigaki commented that "I never think that it could guess with such accuracy just based on the sound of tapping the screen and was surprised by the results. On the extent of the risk, after doing a seriously evaluation, a review on effective security measures would be needed.”

            So, AI could be used to perform spying duties based on sounds by building up a huge data base for the purpose of  “learning”. The working logic behind is similar to using AI to do face recognition, or to drive a motor car by recognizing all the road situations.