Which pre- or Disadvantages have deep learning?

The advantages orStrengths of deep learning methods (e.g. neural networks) lie in the implicit learning of representations of the input data, which lead to an optimal result (based on the amount of available examples).

Before deep learning gained popularity again, mainly flat linear and nonlinear methods (e.g. Decision Trees, SVM, Logistic Regression) were used.In this context, Flat means that these methods are not normally directly applicable to the raw data (e.g. images, texts), but require a pre-processing step, the so-called feature extraction. The result of this extraction is a representation of the raw data, which can now be used in combination with the above methods to produce a result (e.g. classification into several classes).
In many cases, the development of this preprocessing step requires detailed knowledge of the domain and needs to be refined, adapted and tested over multiple interactions in order to obtain the best possible result.

In contrast, deep learning methods are able to learn an implicit representation directly from the raw data.For example, a neural network over several levels (layers) forms an increasingly abstract/compressed representation of the input data (for example, hierarchical structures in text or image based on the words or pixels) to produce the output signal (e.g. probability over a set of classes) in the last stage. This means that feature extraction is part of the process and is therefore also optimized during the training process to achieve an optimal result. Deep learning procedures have the advantage that they require little to no manual effort to optimize the feature extraction step.

Unfortunately, this advantage also has some disadvantages, which are more or less noticeable depending on the application.

  • Due to implicit feature extraction, deep learning methods are generally considered a black box and therefore have very low explainability.

Explainability in this context means that it is possible to understand how a method comes to a conclusion based on the input data. Due to the complex and highly nonlinear structure of most deep learning procedures, this is currently not possible.

  • The low explanatory ability may also lead to a systematic error (bias) induced by the available examples not being detected, since it is not possible to understand what the method is making a decision on (e.g. age, gender or nationality in a lending assessment).
  • Therefore, deep learning is of little use in areas where auditing is required.

  • Compared to flat procedures, deep learning procedures generally have a very high number of parameters (millions to billions) that need to be optimized during the training process.
  • Therefore, deep learning methods usually require a much larger set of examples from which to learn.

    Leave a Reply