◆研究概要
本研究では、ギターリスト日々の練習をサポートするため、センサーやマーカーなどの物理的な制限なしにギターの自動化教師システムを実現することを最終目標としています。本研究では、深層学習をベースとして研究を展開していきます。具体的には、（１）全畳み込み層ベースの手領域画像分割、言い換えれば画像入力と画像出力、全画素の分割方法（２）CNNベースのフレーム毎でギターリストの手関節を推定するアルゴリズム（３）RNNベースのギター運指評価法を研究しています。また、過去の研究において（１）時系列特徴量のクラスター分析と拡張Particle Filterを用いる多指先追跡法（２）RANSACと透視変換を用いて、特徴点ベースのギターネック追跡研究　（３）Support Vector Regressionのギター運指評価法も研究していました。

◆発表文献
〇ジャーナル
[1] Zhao, WANG, Jun OHYA. A Fingertips Tracking Algorithm for Guitarist Based on CNN-Segmentation and Extended Particle Filter IIEEJ Journal Paper (Coming Soon)

〇国際会議
[1] Zhao, W., & Ohya, J. Detecting and Tracking the Guitar Neck Towards the Actualization of a Guitar Teaching-aid System. In The… international conference on advanced mechatronics: toward evolutionary fusion of IT and mechatronics: ICAM: abstracts Vol. 2015, No. 6, pp. 187-188. 2015
[2] WANG, Zhao, and Jun OHYA. Tracking the Guitarist’s Fingers as Well as Recognizing Pressed Chords from a Video Sequence. Electronic Imaging 2016.15 : 1-6. 2016
[3] Wang, Z. and Ohya, J., 2016, November. Fingertips Tracking Algorithm for Guitarist Based on Temporal Grouping and Pattern Analysis. In Asian Conference on Computer Vision (pp. 212-226). 2016
[4]Gao, S., Tatematsu, N., Ohya, J., & Wang, Z. (2015, December). Estimating Clean-up Robots’ Mechanical Operations of Objects Using a SLAM Based Method. In The… international conference on advanced mechatronics: toward evolutionary fusion of IT and mechatronics: ICAM: abstracts Vol. 2015, No. 6, pp. 249-250. 2015
[5] Keishi Nishikawa, Zhao Wang, Jun Ohya, Takashi Matsuzawa, Kenji Hashimoto, and Atsuo Takanishi ,“Automatic, Accurate Estimation of the position and pose of a Ladder in 3D Point Cloud”, The 5th IIEEJ International Workshop on Image Electronics and Visual Computing

〇国内会議
[1] Zhao WANG, Ye LI, Jing YAN, Jun OHYA, Study of Detecting the Frets and Strings on the Neck of the Guitar from RGBD Images towards the Actualization of an Autonomous Guitar Teaching System, Section D7,Media Computing Conference 2014
[2] Zhao WANG, Jun OHYA, Study of a Vision Based Method for Checking the Position of Each Finger of Guitar Players – Towards the Actualization of an Autonomous Guitar Chord Teaching System -,Section D-11-7,IEICE Society Conference 2015.　
[3] Zhao WANG, Jun OHYA, A Method for Tracking Guitar Neck and Fingertips: Necking Tracking Robust against Occlusions Based on Geometry Analysis and Fingertips Tracking Based on Temporal Probability Map ,FIT Information and Science Technology Forum 2015.