表題番号：2017S-127
日付：2018/04/08
研究課題A study on neural activation in the learning process of deep neural networks
| 研究者所属（当時） | 資格 | 氏名 | |
|---|---|---|---|
| （代表者） | 理工学術院 情報生産システム研究センター | 助手 | 李 ウェイテ |
| （連携研究者） | 理工学術院 大学院情報生産システム研究科 | 教授 | 古月 敬之 |
- 研究成果概要
- Deep neural networks are currently receiving a great attention from both industry and academia. However, its internal mechanism is still unclear.In our research, we start from analyzing the activation functions of neural networks and continue to study its internal mechanism from a space mapping perspective. To be specific,1) We discover the parameters inside a neural network play two roles at a time if a piecewise linear activation function is implemented. The two roles are a task-driven hidden signal producer and a gate signal generator respectively.2) We experimentally find the learning speeds of the two different roles are different.3) We propose a method to decouple the two roles for a more stable learning process. Besides, a two-stage learning process is formulated.4) We prove the second stage can be optimized via a kernel trick, which accelerates the learning process for medium size datasets.