HTML5技术

【深度学习系列】卷积神经网络详解(二)——自己手写一个卷积神经网络 - Charlotte77(2)

字号+ 作者:H5之家 来源:H5之家 2017-11-23 08:01 我要评论( )

\begin{equation} \begin{aligned} net_{o_{11}} = i_{11} \times h_{11} + i_{12} \times h_{12} +i_{21} \times h_{21} + i_{22} \times h_{22} \\ net_{o_{12}} = i_{12} \times h_{11} + i_{13} \times h_{12}

\begin{equation}
\begin{aligned}
net_{o_{11}} = i_{11} \times h_{11} + i_{12} \times h_{12} +i_{21} \times h_{21} + i_{22} \times h_{22} \\
net_{o_{12}} = i_{12} \times h_{11} + i_{13} \times h_{12} +i_{22} \times h_{21} + i_{23} \times h_{22} \\
net_{o_{12}} = i_{13} \times h_{11} + i_{14} \times h_{12} +i_{23} \times h_{21} + i_{24} \times h_{22} \\
net_{o_{21}} = i_{21} \times h_{11} + i_{22} \times h_{12} +i_{31} \times h_{21} + i_{32} \times h_{22} \\
net_{o_{22}} = i_{22} \times h_{11} + i_{23} \times h_{12} +i_{32} \times h_{21} + i_{33} \times h_{22} \\
net_{o_{23}} = i_{23} \times h_{11} + i_{24} \times h_{12} +i_{33} \times h_{21} + i_{34} \times h_{22} \\
net_{o_{31}} = i_{31} \times h_{11} + i_{32} \times h_{12} +i_{41} \times h_{21} + i_{42} \times h_{22} \\
net_{o_{32}} = i_{32} \times h_{11} + i_{33} \times h_{12} +i_{42} \times h_{21} + i_{43} \times h_{22} \\
net_{o_{33}} = i_{33} \times h_{11} + i_{34} \times h_{12} +i_{43} \times h_{21} + i_{44} \times h_{22} \\
\end{aligned}
\end{equation}

  然后依次对输入元素$i_{i,j}$求偏导

  $i_{11}$的偏导: 

\begin{equation}
\begin{aligned}
\frac{\partial E}{\partial i_{11}}&=\frac{\partial E}{\partial net_{o_{11}}} \cdot \frac{\partial net_{o_{11}}}{\partial i_{11}}\\
&=\delta_{11} \cdot h_{11}
\end{aligned}
\end{equation}

  $i_{12}$的偏导:

\begin{equation}
\begin{aligned}
\frac{\partial E}{\partial i_{12}}&=\frac{\partial E}{\partial net_{o_{11}}} \cdot \frac{\partial net_{o_{11}}}{\partial i_{12}} +\frac{\partial E}{\partial net_{o_{12}}} \cdot \frac{\partial net_{o_{12}}}{\partial i_{12}}\\
&=\delta_{11} \cdot h_{12}+\delta_{12} \cdot h_{11}
\end{aligned}
\end{equation}

  $i_{13}$的偏导:

\begin{equation}
\begin{aligned}
\frac{\partial E}{\partial i_{13}}&=\frac{\partial E}{\partial net_{o_{12}}} \cdot \frac{\partial net_{o_{12}}}{\partial i_{13}} +\frac{\partial E}{\partial net_{o_{13}}} \cdot \frac{\partial net_{o_{13}}}{\partial i_{13}}\\
&=\delta_{12} \cdot h_{12}+\delta_{13} \cdot h_{11}
\end{aligned}
\end{equation}

  $i_{21}$的偏导:

\begin{equation}
\begin{aligned}
\frac{\partial E}{\partial i_{21}}&=\frac{\partial E}{\partial net_{o_{11}}} \cdot \frac{\partial net_{o_{11}}}{\partial i_{21}} +\frac{\partial E}{\partial net_{o_{21}}} \cdot \frac{\partial net_{o_{21}}}{\partial i_{21}}\\
&=\delta_{11} \cdot h_{21}+\delta_{21} \cdot h_{11}
\end{aligned}
\end{equation}

  $i_{22}$的偏导:

\begin{equation}
\begin{aligned}
\frac{\partial E}{\partial i_{22}}&=\frac{\partial E}{\partial net_{o_{11}}} \cdot \frac{\partial net_{o_{11}}}{\partial i_{22}} +\frac{\partial E}{\partial net_{o_{12}}} \cdot \frac{\partial net_{o_{12}}}{\partial i_{22}}\\
&+\frac{\partial E}{\partial net_{o_{21}}} \cdot \frac{\partial net_{o_{21}}}{\partial i_{22}}+\frac{\partial E}{\partial net_{o_{22}}} \cdot \frac{\partial net_{o_{22}}}{\partial i_{22}}\\
&=\delta_{11} \cdot h_{22}+\delta_{12} \cdot h_{21}+\delta_{21} \cdot h_{12}+\delta_{22} \cdot h_{11}
\end{aligned}
\end{equation}

  

  观察一下上面几个式子的规律,归纳一下,可以得到如下表达式:

\begin{equation}
{
\left[ \begin{array}{ccc}
0& 0& 0& 0& 0& \\
0& \delta_{11} & \delta_{12} & \delta_{13}&0\\
0&\delta_{21} & \delta_{22} & \delta_{23} &0\\
0&\delta_{31} & \delta_{32} & \delta_{33} &0\\
0& 0& 0& 0& 0& \\
\end{array}
\right ]
\cdot
\left[ \begin{array}{ccc}
h_{22}& h_{21} \\
h_{12}& h_{11} \\
\end{array}
\right]}=
\left[ \begin{array}{ccc}
\frac{\partial E}{\partial i_{11}}& \frac{\partial E}{\partial i_{12}}& \frac{\partial E}{\partial i_{13}}& \frac{\partial E}{\partial i_{14}} \\
\frac{\partial E}{\partial i_{21}}& \frac{\partial E}{\partial i_{22}}& \frac{\partial E}{\partial i_{23}}& \frac{\partial E}{\partial i_{24}} \\
\frac{\partial E}{\partial i_{31}}& \frac{\partial E}{\partial i_{32}}& \frac{\partial E}{\partial i_{33}}& \frac{\partial E}{\partial i_{34}} \\
\frac{\partial E}{\partial i_{41}}& \frac{\partial E}{\partial i_{42}}& \frac{\partial E}{\partial i_{43}}& \frac{\partial E}{\partial i_{44}} \\
\end{array}
\right]
\end{equation}

 

  图中的卷积核进行了180°翻转,与这一层的误差敏感项矩阵${delta_{i,j})}$周围补零后的矩阵做卷积运算后,就可以得到${\frac{\partial E}{\partial i_{11}}}$,即

$\frac{\partial E}{\partial i_{i,j}} = \sum_m \cdot \sum_n h_{m,n}\delta_{i+m,j+n}$

  第一项求完后,我们来求第二项$\frac{\partial i_{11}}{\partial net_{i_{11}}}$

\begin{equation}
\begin{aligned}
\because i_{11} &= out_{i_{11}} \\
&= activators(net_{i_{11}})\\
\therefore \frac{\partial i_{11}}{\partial net_{i_{11}}}
&=f'(net_{i_{11}})\\
\therefore \delta_{11} &=\frac{\partial E}{\partial net_{o_{11}}} \\
&=\frac{\partial E}{\partial i_{11}} \cdot \frac{\partial i_{11}}{\partial net_{i_{11}}}\\
&=\sum_m \cdot \sum_n h_{m,n}\delta_{i+m,j+n} \cdot f'(net_{i_{11}})
\end{aligned}
\end{equation}

  此时我们的误差敏感矩阵就求完了,得到误差敏感矩阵后,即可求权重的梯度。

  由于上面已经写出了卷积层的输入$net_{o_{11}}$与权重$h_{i,j}$之间的表达式,所以可以直接求出:

 

 

1.本站遵循行业规范,任何转载的稿件都会明确标注作者和来源;2.本站的原创文章,请转载时务必注明文章作者和来源,不尊重原创的行为我们将追究责任;3.作者投稿可能会经我们编辑修改或补充。

相关文章
  • Linux系列教程(二十二)——Linux的bash变量 - YSOcean

    Linux系列教程(二十二)——Linux的bash变量 - YSOcean

    2017-11-19 08:00

  • 极简版ASP.NET Core学习路径及教程 - 腾飞(Jesse)

    极简版ASP.NET Core学习路径及教程 - 腾飞(Jesse)

    2017-11-17 14:02

  • [深度学习]实现一个博弈型的AI,从五子棋开始(1) - xerwin

    [深度学习]实现一个博弈型的AI,从五子棋开始(1) - xerwin

    2017-11-15 12:01

  • JavaScprit30-5 学习笔记 - 喵20

    JavaScprit30-5 学习笔记 - 喵20

    2017-10-23 11:07

网友点评